YT YouTube2Text

YouTube2Text → Transcripts → Anthropic's New Study: You're Not Controlling AI as Much as You Think You Are

Anthropic's New Study: You're Not Controlling AI as Much as You Think You Are — Transcript

by Cole Medin · 3,576 words · 515 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00I have a question for you that I want
0:01you to answer to yourself honestly right
0:03now. When you were planning work with
0:05your coding agent and building
0:06something, how much are you deferring
0:08judgment to the agent and going with its
0:10recommendations versus actually
0:12critically thinking through the problem
0:14yourself and using your own judgment?
0:16Because Anthropic just put out this
0:18study that shows there's at least a good
0:20chance that you're deferring a lot more
0:22judgment to the agent than you realize.
0:25It's the idea of AI disempowerment and
0:28it's a big problem because coding agents
0:30they give us recommendations for things
0:32all the time. And a lot of times we go
0:34with that because it's the easiest route
0:35and we even think that we're making our
0:37own decisions, but it's really just the
0:39coding agent stringing us along. And so
0:42this Anthropic study gives us a good
0:44premise to talk about this problem. So I
0:46want to go through this with you here.
0:48There's some very interesting stuff and
0:50then I want to talk about some solutions
0:52to this problem as well. So this study
0:54covers a very specific use case in one
0:56industry, but everything here is
0:58universal because all of us are using
1:01agents in a very similar way these days.
1:03Whenever we go to implement anything, we
1:05always do a planning session first. Like
1:07Matt Pocock has his Grill Me skill. We
1:09all have some kind of version of that
1:11skill where it goes through a process of
1:13asking us questions, figuring out the
1:15scope, clarifying anything, and this is
1:17where we're supposed to be communicating
1:19our intent clearly. And so this is a
1:21very simplified version of what my
1:23planning looks like. I always describe
1:25what I want to build at a high level and
1:27then I have it use the ask user question
1:29tool to present a lot of these things to
1:31me. Like here are the questions that I
1:32answer multiple choice so it's fast for
1:34me to go through all these different
1:35things. And the coding agent generally
1:38always gives its recommendation for the
1:40top answer in the multiple choice list.
1:43And so this is where I think I'm
1:44applying my judgment cuz I can even
1:46clarify something if I really think it's
1:47off base, whatever. But most of the time
1:50we're just going with its
1:51recommendation, not applying judgment as
1:54much as we think we are. That's the
1:55problem with a lot of these planning
1:57processes that we create as skills if we
1:59are not careful. Now, I mentioned Matt's
2:02Grill Me skill as an example, but
2:04nothing against him. In fact, I know
2:06he's thinking about a lot of these
2:07problems. He even says right here in his
2:09docs, "The failure mode is passivity,
2:11answering agreed, agreed, agreed for 40
2:12questions, coming out with a plan the
2:15agent wrote and you nodded at." So, he
2:17knows about these things. In fact, he
2:18has a lot of the same ideas of solutions
2:20that I'll cover at the end of this
2:22video, but this is definitely the
2:23premise of the problem that applies to
2:25anyone building anything. AI
2:27disempowerment is a universal trap.
2:31Okay, so before we get into how to
2:32prevent losing our agency, let's cover
2:34what Anthropic has to say about this,
2:36because their study here is fascinating.
2:38And so, the premise is when experts hand
2:40their work to an AI agent, the usual
2:42assumption is that they still make
2:43decisions that matter. And so, the agent
2:46is writing the code, running the
2:47numbers, offering advice, but it's still
2:49us providing the final judgment for even
2:52the smaller details. Right? Like, that
2:54is what we assume, and now their
2:55studying here, is that really the case?
2:58So, they say, "We wanted to look at
2:59whether experts really do make those
3:01important decisions or whether in
3:02practice they defer to their AI agents.
3:05To see whether they were making these
3:06calls themselves, we set up each
3:08analyst's agent to favor one option on
3:10each decision, which the analysts
3:12weren't told about. And it was
3:13randomized which way the agent would
3:16lean. And then, the agent was also set
3:17up through its system prompt to argue
3:19its case, but also it would be able to
3:21change its mind if good enough evidence
3:24was presented to it." Now, Anthropic
3:26also covers a lot about how they set up
3:28these tests and what the questions are
3:29exactly. You can read the article if you
3:31want, a link to it in the description,
3:33but right now, I just want to get right
3:34to the kicker. Across the four
3:36analytical choices, analysts went with
3:39their agent's lean 86% of the time. That
3:42is a crazy number. If the agent's input
3:44made no difference, we'd expect about
3:4650%. Because it really is random which
3:49way it leans, so half the time it's
3:50going to go towards the theoretically
3:52correct answer, half of the time it
3:54wouldn't. When analysts had already
3:56expressed their own preference before
3:58the agent weighed in, they switched away
4:00from it 60% of the time if the agent
4:02disagreed, compared to 6% if it agreed.
4:05So, even if they had an idea in their
4:07mind, they still didn't really stick
4:08with it when the agent persuaded them.
4:10And I think this is really in part
4:11because LLMs are so confident in
4:13everything they say, even when they're
4:15dead wrong. And we haven't even gotten
4:17to the most interesting part yet,
4:18because this is it. Afterwards, when
4:21asked to explain their choices, 77% of
4:23analysts used their own agent's
4:26arguments, even sometimes completely
4:28word for word, but only 15% credited the
4:31agent as the source of those arguments.
4:33This is really the core of the problem
4:35with AI and disempowerment. When we're
4:37planning with our agents and going along
4:39with their recommendations, a lot of the
4:41time we actually think it's us making
4:43the choice. And so, that in part builds
4:44our confidence and making us think that
4:47we're applying our own critical thinking
4:48a lot more than we actually are.
4:50The sponsor of today's video is Revolut
4:52Business. My second brain already has
4:55its own virtual card with spending
4:56limits. So, the next thing I set up was
4:58a bigger job, allowing it to prepare
5:00payments to people in other countries,
5:03like my vendors and contractors. And
5:05that's real money going to real people.
5:08So, I want to check every payment before
5:09anything gets sent. So, take a look at
5:12this. This is exactly what my second
5:14brain looks like when it's preparing
5:15payments on my behalf with Revolut
5:17Business. Now, it's fake data, but it is
5:19my exact process. And so, my second
5:22brain, it reads the payment list, it
5:24matches each person to a saved receipt
5:26in Revolut Business, it checks the
5:28balance, and then it prepares a single
5:30payment draft. And then it stops,
5:32because it has to. My second brain's API
5:34access does not include permission to
5:36actually send the payment. And so, it
5:38sits as a draft until I go into the
5:40Revolut Business app and send it myself.
5:43And for paying people abroad, at Revolut
5:45Business, lets you exchange 25 plus
5:47currencies at interbank rates within a
5:50market hours and your plan's allowance.
5:52Now, this is just a little under the
5:53hood what my second brain looks like.
5:55But going into Revolut Business, here is
5:57the actual payment draft that it
5:58created. Everything is already filled
6:00in, and the only way this actually goes
6:02out is me reviewing it and clicking
6:04send. And so, this isn't just a line in
6:06the prompt where we're asking the agent
6:08to wait once it's created the payment
6:09draft. It's a guarantee that is built
6:11into how I've set up the access. My
6:13second brain does all the prep work and
6:15that's very useful for me, but nothing
6:17is sent out until I've checked it.
6:19Revolut Business is giving you $200 when
6:22you open an account with my link or with
6:24the QR code on screen and you add at
6:26least a cent. I'll have a link to them
6:27in the description.
6:29It's also really important to
6:30acknowledge the limitations of this
6:32study. I'm glad Anthropic does so. So,
6:34first of all, the most obvious thing,
6:36they deliberately strengthen the agent's
6:37lean. So, as a part of the system prompt
6:39telling it to argue something that it
6:41might otherwise not. And they studied
6:43one task in one field with one model. I
6:45really do wish they studied a couple of
6:47different industries and had some larger
6:48data sets here, but overall, I'm still
6:51confident that this applies to everybody
6:53because I've seen this time and time
6:55again in my own planning, seeing other
6:57people plan with agents. There's always
6:59this false confidence that we're tempted
7:01to be drawn into as we just go with what
7:03the agent gives us. So, it's good to be
7:05wary this is just one test, but the
7:06takeaways here are very eye-opening.
7:09Analysts mostly went with their agent's
7:11lean like we talked about. Analysts who
7:13already picked an option often switched
7:15when their agent disagreed. Experience
7:17and expertise did not determine who
7:19differed. This is another really
7:21interesting thing. You'd think that
7:22people who are more confident in their
7:23own opinions would be less likely to
7:26follow the guidance of the agent, but
7:27that's also not the case. Analysts who
7:29follow their agent's lean still call the
7:31recommendation their own. Like when we
7:33look at this, whose judgment does the
7:34final recommendation mostly reflect? 52%
7:37of people call the recommendation more
7:39their own than the agent's, even though
7:41the agent's lean had shifted their
7:42choices a majority of the time like we
7:45saw earlier. And this was proven because
7:47a lot of the time analysts explained
7:49their choices in their agent's words.
7:51And I got to be honest, I found myself
7:53doing this once in a while as well. Like
7:55when I'm working on an open source
7:57project whether people are working on my
7:58team in Dynamis, like I justify
8:00something to them by sharing something
8:02that I kind of, you know, think back on
8:04like, "Oh, yeah, that's actually exactly
8:05what my agent told me." And then the
8:07last takeaway that Anthropic has here is
8:08that analysts who work without AI made
8:11more serious errors and took longer,
8:13which maybe Anthropic just has to say
8:15this because they're a frontier model
8:16lab. Of course, they want to say that no
8:18matter these downsides, it's still
8:19better to use AI, but I mean, I buy it
8:22myself. I use AI to automate a very
8:23large part of my business and it does
8:25make less errors than I do doing those
8:27things manually like I did before. So,
8:30those are the takeaways. The last thing
8:32that's really worth covering here in the
8:33article is Anthropic's take on what this
8:35means for the more AI-centric future.
8:37This is also where Anthropic gets into
8:39some of their solutions, which I
8:41definitely want to expand on this for
8:42you here and share some of my own ideas.
8:44The biggest gripe I have with this
8:46article is they don't talk about
8:47solutions enough. Now, I get it to an
8:50extent because this really isn't a
8:51solved problem, but I've got some good
8:53ideas to share with you. And in the end,
8:56this isn't really a new problem because
8:58humans, we always have the temptation to
9:00be lazy and defer our judgment when
9:02possible. It's just with large language
9:03models, we have this new level of being
9:05able to do so. And that's what leads to
9:07the idea of disempowerment that I kept
9:09mentioning earlier. The work still goes
9:11under the expert's name, which there's a
9:13lot of problems with that as well, but
9:15the decisions that most depend on their
9:16judgment are in reality settled by the
9:19agent's prior beliefs. And one reason
9:21analysts defer so readily may be that
9:23experts don't always arrive with their
9:25views fully formed, which this is
9:27interesting because a lot of times we're
9:29told that we can use AI to help us flesh
9:31out our ideas, but this is also telling
9:33us that that can be dangerous. kind of
9:35this catch-22. Agents can be most useful
9:38when we have a vague idea we want to
9:40turn into harder requirements, but it
9:42also means that the temptation is the
9:43greatest to just go with its
9:45recommendations and not truly think
9:46through the problem. And I really like
9:48the way they put the larger issue here.
9:50The stakes extend beyond the individuals
9:52though, because so much of science and
9:53policy rests on judgment calls like
9:55these. Take a look at this. When
9:57separate research teams reach the same
9:59result, we trust it more because we
10:01assume each team reached it
10:02independently. But if those teams all
10:04defer to a model with the same
10:06preferences, even if they don't realize
10:08it, their agreement may no longer be
10:11independent. If everyone is just using
10:13Claude Opus 5.5 to draw the same
10:16conclusions, then we don't actually have
10:18independent studies. And I see this at
10:20the engineering level all of the time
10:22where someone uses a coding agent to
10:23create a pull request, and then another
10:25person uses a coding agent to review the
10:27pull request. And sure, maybe it's not
10:29the worst in the world because we use
10:30different skills, and we're looking at
10:32the problem from a different angle at
10:33each step, but it still really is the
10:35underlying model that's doing both the
10:36building and the review, no matter how
10:38much you try to separate it. And so,
10:40they end by saying this isn't an
10:41argument for using AI less because used
10:43well, AI can actually expand agency
10:46rather than eroding it. And I do truly
10:48believe this. Now, without really
10:50getting into solutions, they end by
10:52suggesting three areas for future study.
10:55So, first of all, how can we better
10:56measure deferral to see really how much
10:58we are giving up agency on average? And
11:00then, how can we design our agents
11:02differently? There definitely are
11:03opportunities to make it so that LLMs
11:05are less confident in things that are
11:06dead wrong, making sure that it asks
11:08better questions so we really have to
11:10think through the problem. It could be
11:12the larger harness around the LLM or
11:14even the LLM itself that gets better at
11:15this. And then, the last thing is
11:17changing institutions. Institutions
11:19often treat an expert sign-off as
11:21evidence of independent judgment, but as
11:23AI does more and more of the work, that
11:25assumption becomes shakier. So, how do
11:27we design institutions and processes
11:29better so that we acknowledge the fact
11:31that things might not be independently
11:33reviewed when we think they are. So,
11:35that's the end to their article. You can
11:37see what I'm saying, they're not really
11:38presenting any solutions, at least
11:40nothing concrete. They talk about things
11:42we're going to have to look into over
11:43the next year and beyond, but I wasn't
11:45left very satisfied after reading
11:47through this. And I know you won't be
11:48either. And so, that's why for the last
11:50part of this video, I want to talk about
11:52some solutions that I've come up with
11:53for this problem.
11:55Now, I acknowledge that this is not a
11:57solved problem. AI disempowerment,
11:59whatever you want to call it, it's
12:00always going to be a trap as long as we
12:02have access to LLMs that are this good.
12:04And so, that's also why I'm not going to
12:05spend 20 minutes showing you exactly how
12:07to implement these things. I more just
12:08want to give you some high-level ideas
12:10of strategies that I've been working
12:12with over the past few months cuz this
12:14isn't a new problem. I just really
12:16appreciate how Anthropic has framed the
12:18issue in this article I put out a couple
12:20of days ago. So, the first solution here
12:22that I'll cover very quickly is having
12:24the agent ask us about priorities and
12:27focusing more on the high-level before
12:29we get down to the details of the
12:30individual solutions with the agent's
12:33recommendations. Cuz like we talked
12:34about earlier, the core of the problem
12:36can be summed up in this simple Claude
12:38code session. We want to solve our
12:40problem. We have the agent ask us how to
12:42solve it so we feel like we're applying
12:44our judgment, but really we just go with
12:46its recommendation. And so, what if we
12:48change the behavior of the agent through
12:50our rules, our skills, whatever, so that
12:53instead of just getting to this point
12:54right away, we focus more on our
12:56priorities. What's the pain we're trying
12:58to solve? What are we seeing? Having it
13:00really lean on us instead of just
13:02dumping its recommendation right away.
13:04And so, for example right here, this is
13:05the recurring annoyance that I'm seeing.
13:08I mean, I I don't need to get super
13:09specific in the example here, but the
13:11point is I'm focusing more on like the
13:12overarching problem before we even get
13:14into the solutions. And so, that way I'm
13:16able to kind of feed my judgment into
13:19the agent before we get to that point
13:21where it's asking the questions and
13:23giving recommendations. We still get
13:24here eventually because this is the way
13:26to work with coding agents. The most
13:28efficient way is to have us blitz us
13:29with questions and we get through it
13:31quickly because it gives
13:32recommendations. So, I'm not saying we
13:34scrap that entirely, but we have that
13:35sort of prerequisite that we build into
13:38its behavior. Now, I can make an entire
13:40video how I do this with my rules and
13:42skills. Let me know in the comments if
13:43you want me to cover that cuz I want
13:45this video to stay higher-level. And so,
13:47let's go on to the second strategy here
13:48with prototyping. A lot of times we
13:50defer to the agent's judgment even when
13:52we don't realize because we're just
13:54uncertain ourselves what the best option
13:56is. Especially when we're going through
13:58a lot of different questions with the
14:00agent, there's the fatigue that sets in
14:01where we're like, I don't even know how
14:03to envision this or how to really decide
14:05what would mean this option is better
14:06than the other. And so, one of the best
14:08ways that we can knock away that
14:10uncertainty is just to build prototypes
14:12for the different things that we're
14:13considering. So, if there's a few core
14:15questions in our mind, where I don't
14:16know how the UI should look, or I don't
14:18know what API endpoints we need, just
14:21build it. Build both versions and see
14:23what works better. The beauty of coding
14:25agents these days is when we use these
14:27smaller models that are still quite
14:29capable, we can build a ton of versions
14:31of our application in parallel very
14:33cheaply. We're not using the best model,
14:36but we're also not building the
14:37production version. We're building
14:38something solely for the purpose of
14:41making a decision. So, we build these
14:43prototypes, we make a decision based on
14:45that, and then we scrap it. So, instead
14:47of just forever living in these
14:48questions with the agents where there's
14:49a lot of uncertainty for us, we provide
14:52certainty by building something that's
14:54easy for us to visualize, see what the
14:56answer is, then we give that to the
14:58agent. So, in this case, we definitely
14:59are applying our judgment more. So, the
15:01last strategy here is to play devil's
15:03advocate with your coding agent.
15:04Literally asking it the question, "What
15:06would change your advice?" Like, "Okay,
15:08Claude. Okay, Codex. You went with
15:10option A. What would have to change with
15:12the problem or the codebase itself so
15:14option B is optimal?" Forcing the agent
15:18to answer this question is going to give
15:19you a better perspective for what led to
15:21it deciding option A in the first place.
15:23Otherwise, it's a lot more of a black
15:25box, and we defer our judgment to it cuz
15:26we don't really understand what got
15:28there anyway, and we're tempted to just
15:30not look into it at all. So, this is a
15:32very quick way to see what would lead to
15:34other options. And maybe we'll realize
15:36that what would lead to option B
15:38actually does apply here, and the agent
15:40was wrong. And so, that way we can
15:41change its direction applying our
15:43judgment that way. So, there you go.
15:45Those are the three strategies at a high
15:46level, and each one of these are very
15:48easy to incorporate, no matter what your
15:50existing AI coding process or skills
15:53look like, because they're just small
15:54ideas to just build in. And if you have
15:57any other ideas, please share in the
15:58comments. I'd love to know how you make
16:00sure you maintain your judgment. You
16:02don't defer things too much to your
16:03coding agents. And overall, I think this
16:06is just really important for us to
16:07consider as we're building anything. I
16:09mean, what Anthropic is presenting here
16:11is a universal problem. And so, with
16:13that, if you appreciated this video,
16:15you're looking forward to to things on
16:17problems in the industry like this and
16:18agenetic engineering as a whole. I would
16:20appreciate a like and a subscribe. And
16:22with that, I will see you in the next
16:24video.

About this transcript

This page contains the full transcript of Anthropic's New Study: You're Not Controlling AI as Much as You Think You Are by Cole Medin, generated from the public captions YouTube serves with the video. The transcript has 3,576 words across 515 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.