YT YouTube2Text

YouTube2Text → Transcripts → Anthropic Just Released A New System For Using Claude Code

Anthropic Just Released A New System For Using Claude Code — Transcript

by AI LABS · 3,516 words · 485 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00There's a huge problem with using AI
0:01agents. Whenever you give them a task,
0:03they write a lot of code and then call
0:05the task done. But when you actually use
0:07what they built, you find they've missed
0:08a lot of the things you asked for.
0:10That's why the most important thing you
0:12need to do is check whether the agents
0:14actually did what they say they did.
0:16There are a lot of ways to do that and
0:17Anthropic Claude code team has shared
0:19their own. With their setup, they put
0:21out more than twice as many changes as
0:23before and the share of changes they had
0:25to undo dropped by over 40%. They shared
0:28the ideas their system is built on and
0:30we turned that system into a workflow
0:32you can follow to get the same quality
0:34of output. If this is your first time,
0:36we're a software company and this is our
0:38channel AI Labs. And in this video,
0:40we're going to go through each step of
0:42their process and why it matters and
0:44show you the prompts we used so you can
0:45build it for yourself. As the models get
0:48better, reviewing what your agent builds
0:50matters even more than it ever did
0:52before. They've gotten so good that you
0:53can just give them a whole list of
0:55features and let them work on their own
0:57for hours. When you come back, every
0:59feature is built without you watching
1:00any of it. The Claude Code team sees
1:02that as a problem because more features
1:04being built faster also means there's
1:07way more to review. Across Anthropic,
1:09the number of changes to review every
1:11day went from about 2500 to over 30,000
1:15in just 4 months. But even though the
1:17agents are good enough, you still need
1:19to check that things are working right.
1:21And with the amount of code they're
1:22writing each day, the chance of a
1:24mistake slipping through gets higher,
1:26too. So, with this much to review in so
1:28little time, you need a way to make
1:29reviewing easier. Review is really
1:31important because if a mistake gets to
1:33your app's real users without a check,
1:35it becomes a much bigger problem. The
1:37team says that instead of saving time,
1:39you end up spending twice as much time
1:41as you would on the review itself, first
1:43checking the changes and then fixing the
1:45bugs that slip through. If you rush the
1:46checks, you just push that time to later
1:48when the fix costs more. So review is a
1:51step you can't skip and the real
1:53question is who does it? To answer that
1:55the team first changed the question they
1:57ask when they review. Instead of asking
1:59whether someone read the code they
2:01started asking what could go wrong with
2:02a change and how to catch it. That puts
2:04the focus on how the thing you're
2:06building should work. The Claude Code
2:08team also changed when the checks run.
2:10Now they happen while Claude is building
2:12instead of waiting until the end. You
2:14can already see this when you build with
2:15cloud code because it goes through some
2:17steps to check that your app is working
2:19before it tells you the task is done. So
2:21by the time you look at a change, it's
2:23already been checked and fixed at least
2:25once. With that in place, you can spend
2:27your time on the parts that are most
2:29likely to break, and the rest of the
2:30team's process is about how to do
2:32exactly that. But before we move
2:34forward, it would be great if you
2:35subscribe to the channel and hit the
2:37hype button. This small gesture of
2:39support goes a long way for us. Now, the
2:41checks Claude runs while it's building
2:43are much smaller. You need deeper checks
2:45to see that the app doesn't only work
2:46the way you'd expect, but it also holds
2:48up when people use it in different ways.
2:50That's why deeper checks are the first
2:52step in the Claude code team's own
2:54process. Before a change goes anywhere,
2:56they ask Claude to run something that
2:58proves it works and to show them the
3:00result instead of just saying it's done.
3:02That can be the app's tests or a
3:04screenshot compared with the design.
3:05Boris Churnney, who is the creator of
3:07Claude Code, says that the best way to
3:10get Claude to work well is to give it
3:12away to prove it did the work right. And
3:14the team's advice is to give your agent
3:16a list of checks it can run. Each of
3:18those checks is called an eval, and it
3:20is a fixed test that tells you whether
3:22the work is done. It has to be strict
3:24enough that bad work fails it. We've
3:26shown many ways to give Claude evals in
3:28our earlier videos. Claude Codes team
3:30mentions their own way to set up these
3:32checks, and we turned that into a skill.
3:34In the demo app we were testing on, we
3:36had a specs folder with all the features
3:38in it. Each feature had its own folder
3:40named after the feature itself. And
3:42inside each folder, we had an RD.md file
3:45that listed the requirements for that
3:47feature. You can ask Claude to create a
3:49skill that turns each feature into an
3:51eval file. This is the file that has the
3:54actual method the agent will use to
3:56check its work. If the feature has a
3:58design, it should first make it as a
4:00simple web page called a mockup instead
4:02of building straight on the app. Then it
4:04saves that page as the design the app
4:06gets checked against. Claude can keep
4:08working on the real app until it matches
4:10that page and we can quickly see how the
4:12feature will look without wasting too
4:14many tokens. You should also tell it
4:16that once it has written the evals, it
4:18needs to get them reviewed by you and
4:20without that review, it shouldn't build
4:22anything. Then ask it to create a hook
4:24that makes sure agent doesn't mark the
4:26task as done until the evals have
4:28passed. Claude then creates the skill
4:30for you. It writes the skill in
4:31your.claude folder and creates a
4:33template next to it that gives all the
4:35eval files the same structure. So for
4:37each new feature, it starts by running
4:39the skill. It opens the folder for that
4:42feature and writes the requirements
4:43document along with the eval file. Then
4:46it uses the hook to check whether the
4:48requirements are set up properly. It
4:49goes back and forth several times
4:51checking whether the evals are written
4:53properly and it creates a mockup that
4:55you can look at. After it has finished
4:57everything and checked the evals, it
4:58changes the features status from draft
5:00to in review. Then it tells you every
5:02decision it made on its own and opens
5:04the files for you to check. You can
5:06check how the design looks on there. It
5:08also shows every other screen the app
5:10can be in along with the same design on
5:12mobile. So you can check the smaller
5:14screen layout as well. You need to
5:15review both the eval and the mockups. If
5:19you have any changes at this point, you
5:21can make them by asking Claude. If you
5:23don't, you can just approve them and it
5:24will mark the feature as reviewed and
5:26ready for building. Now, this skill
5:28along with all others that we'll be
5:29showing here are all available in AIBS
5:32Pro, which is our community. So, if you
5:34found value in what we do and want to
5:37support the channel, this is the best
5:38way to do it. The links in the
5:40description. Even after Claude builds
5:42the feature based on the evals, you
5:44still need to review it. And the agent
5:46that reviews it shouldn't be the same
5:47one that built it because that agent
5:49made every decision in the change and
5:51ends up agreeing with itself. So the
5:53Claude code team's second check is a
5:55separate reviewer agent. The reviewer
5:57checks every change before a person
5:59looks at it. Since the team started
6:01doing this, far more of their changes
6:03get a review that catches something
6:04real. That review agent is only
6:06available on the team and enterprise
6:08plans for now, but a lighter version
6:10already comes with clawed code as the
6:12code review command. When you run it, a
6:14separate agent reads everything that
6:16changed and reports back the mistakes
6:18that would break your app. You can also
6:20choose how deep it checks with an effort
6:22level. At low effort, it only reports
6:24what it's most sure about. And the
6:25higher levels check more of your app. to
6:27tell that command what matters in your
6:29app. Anthropic's article on this says to
6:31write your rules in a file called
6:33review.md. By default, this file isn't
6:36read unless you ask claude to do so. So,
6:38if you want the code review command on
6:40your computer to follow your rules, they
6:42need to be part of your claude.md. But
6:44instead of adding them directly to that
6:46file, it's better to put them in another
6:48file. Then you need to link that file
6:50inside claw.md. This way, whenever you
6:53open a new session, the review.md is
6:55also loaded along with the claw.md file.
6:58This file should have what counts as a
7:00real failure in the app and how serious
7:02each problem is. It should also say what
7:04to ignore and how to report the review
7:06along with all the ways you want the app
7:07tested. You can ask Claude to create
7:09this file for you, and you'll end up
7:11with a review.md file in your project's
7:13main folder. Every review you run on
7:15your app starts from it, and since it's
7:17linked from your claude.md, the code
7:19review command reads it, too. With the
7:21review rules written down, the next step
7:23is a loop that builds the feature and
7:25reviews it. You can ask Claude to create
7:27a review loop skill that builds a
7:29feature and then checks it with its
7:31eval. It uses sub aents with fresh
7:34context windows. This way, they don't
7:36have the same context as the main agent.
7:38So, they don't just agree with what the
7:39main agent did. It should keep looping
7:41until all the evals have passed. From
7:43that prompt, Claude makes the skill
7:45along with three agents and the main
7:47agent runs them. The main agent acts as
7:49the orchestrator which runs the loop and
7:51starts each agent in its own context
7:53window. The first agent is the builder
7:56which builds the feature from the
7:57requirements document that the right
7:59eval skill wrote. After the builder is
8:01done, the verifier starts and checks the
8:04app against the evals from that same
8:06skill. It checks whether everything the
8:08eval cover has been built and marks each
8:10eval as passed or failed. Then the
8:12reviewer goes through the app to find
8:14anything the eval missed and it uses the
8:16rules in your review.md. To use the
8:19skill, you run the review loop command
8:21and then name the feature you want to
8:22build. It starts by writing a state.md
8:25file that keeps track of the loop. This
8:27file shows which round the loop is on
8:29and what's still failing. After that,
8:31the first round starts and the builder
8:33builds the feature. While the builder
8:34works, the hook from the right eval
8:36skill stops the main agent from calling
8:38the feature done too early. Then the
8:40verifier and the reviewer start and the
8:42loop keeps fixing issues in rounds. When
8:45it hits a decision it can't make on its
8:47own, it shows you a progress report and
8:49asks you. The loop only stops when all
8:51the eval pass and the review doesn't
8:53find anything new. But before we move
8:54on, let's have a word by our sponsor,
8:56Unstoppable Code. You probably switch
8:59between different coding agents,
9:01terminals, and desktop apps all the
9:03time, feeling frustrated by all the
9:05juggling and context switching.
9:06Unstoppable is an agentic IDE that lets
9:09you orchestrate a team of coding agents
9:11by running cloud code, codecs, and
9:13cursor all in the same desktop app. One
9:16thing that sets Unstoppable apart is
9:18their multi- aent workflows, which chain
9:20together prompts and scripts to automate
9:22your entire dev life cycle. We built a
9:24code reviewer that iterates with both
9:26Claude and Codeex until the PR gets a
9:29passing grade and another scheduled task
9:31for fixing bug reports automatically.
9:33The terminal, the browser, and the diffs
9:35all sit in the same window, so you can
9:37see everything at a glance and switch
9:39between agents seamlessly. The best part
9:42is that with their free plan, you can
9:43run codecs and open models without
9:45paying a thing or plug in the
9:47subscriptions you already have. Download
9:48the desktop app and start building at
9:51unstoppable.ai.
9:52Full link in the description. But fixing
9:55each issue doesn't stop it from coming
9:57back because Claude doesn't remember
9:58mistakes from earlier sessions. So, it
10:00makes the same mistake on the next
10:02feature and you have to fix it all over
10:03again. That's why when the Claude code
10:05team's reviewer keeps finding the same
10:07mistake, they change the instructions
10:09Claude reads and Claude stops making it.
10:11We do the same with the review.md file
10:14from earlier. At the bottom of it,
10:15there's a section for rules learned over
10:18time. When a review misses a bug that
10:20shows up later, a new rule goes into
10:22that section. It also adds a rule when
10:24the review keeps flagging something that
10:26turns out not to be a real problem. That
10:28way, the next review doesn't make the
10:30same mistake again. The review loop
10:32keeps track of this for you. When the
10:33same problem comes up in a later round,
10:35the loop matches it to the earlier one.
10:37At the end, it suggests a oneline rule
10:39for that part and only adds it if you
10:41approve. But a careful review on every
10:44change costs time and usage. So, the
10:46Clawed Code team's rule is that a change
10:48needs to get checked harder when a
10:50mistake in that feature could do more
10:52damage. And they don't choose on their
10:53own if a feature needs a full review or
10:55a quick one. They have an agent that
10:57sorts every change for them, which they
10:59call a classifier. You can sort your own
11:01changes as well, and all it takes is one
11:03question, which is, "What happens to a
11:05customer if a feature breaks?" A mistake
11:06that only makes something look wrong is
11:08low risk. If it stops a feature from
11:10working, but nobody loses anything.
11:12That's medium because you can still fix
11:14it later. But some mistakes can make a
11:16customer lose money or get locked out,
11:18and some can show them another person's
11:20data. You can't take those back once
11:22they happen. So, those changes are high
11:24risk. You can also turn that question
11:26into a skill that rates each feature on
11:28this scale for you. Before you build a
11:30feature, you need to run the skill to
11:32check whether it's low, medium, or high.
11:34Then when you use the right eval skill,
11:36you pass it that rating too. So a
11:38high-risk feature gets stricter evals
11:40than a low-risk one. But even on the
11:42changes that do need the full review,
11:44there's one more problem. The builder
11:45and the reviewer agents don't know when
11:47to stop, so they can keep going back and
11:49forth trying to find and fix issues. The
11:51Claude Code team once had a change that
11:53went back and forth more than 50 times
11:55like this. The agents kept going until
11:57someone manually stopped the loop. When
11:59the team looked at what the reviewer
12:01kept reporting, about 59% of it had been
12:04there since the very first round. So,
12:06the reviewer was mostly finding the old
12:08problems again and again. And on your
12:10plan, every one of those rounds uses up
12:12your limits. People's main complaint
12:14about Anthropic's article was that it
12:16only works if you have unlimited tokens.
12:18So, the team added three rules. And the
12:20first one is that after three rounds,
12:21the reviewer stops listing more problems
12:24and tells the builder to rethink how
12:26it's building the feature. If the
12:27feature is still failing after three
12:29rounds of fixes, more work in the same
12:31way wouldn't get it there. The second is
12:33that after the first round, the reviewer
12:35only checks what the last fixes changed
12:37and doesn't report anything it has
12:39already reported. That stops it from
12:41checking the whole feature again and
12:43finding the old problems one at a time.
12:45And the third is that when the reviewer
12:46finds a problem, it says what kind of
12:48problem it is. That way, the builder
12:50fixes that kind of problem everywhere it
12:52shows up and not only in the one spot
12:54the reviewer found. With those three
12:56rules in place, more of their changes
12:58past review on the very first try. Our
13:00review loop works the same way, but each
13:02of our rounds does more work. It builds
13:04the feature and checks every eval in the
13:07browser and then it reviews the code.
13:08That's why we set it to stop after five
13:10rounds instead of three. The loop can
13:12also stop before it gets to five when
13:14the same problem is still there after
13:16two rounds of fixes. But even when the
13:19loop finishes and every eval passes,
13:21some mistakes will still get through. So
13:23you need two more things which are a way
13:25to find those mistakes and a way to undo
13:27them fast. The Claude Code team can undo
13:30almost anything they release in minutes
13:32and that's what lets them read less. To
13:34find the mistakes, they use the product
13:35themselves every day, so they often spot
13:38a problem before their customers do. You
13:40can do the same by using every new
13:42feature yourself the way a customer
13:44would. But you already know how the
13:45feature works, so you'll miss some
13:47things. To catch those, you can send a
13:49fresh agent that never saw the app being
13:51built to use the live app like a
13:53firsttime customer. To set up that fresh
13:55agent, you can ask Claude to create a
13:57skill that shows you the app from a new
13:59user's point of view. It should use sub
14:01agents that are blocked from reading the
14:03app's code, so they don't know what the
14:04app is about. They should go through the
14:06app the way a firsttime user would and
14:08find every place where someone might get
14:10stuck. From this prompt, Claude creates
14:12a firsttime user skill and the agent
14:14that the skill sends to use the app. It
14:16also gives the test a temporary
14:18database, so it doesn't touch your real
14:20data, and it creates a hook that stops
14:22the agent from reading the app's code.
14:24The agent would normally get around that
14:26block with terminal commands. So, the
14:28hook stops it from reading the code that
14:30way, too. To run the test, you just use
14:32the skills command and tell it which
14:34feature you want it to check. Based on
14:36your prompt, it opens the app and starts
14:38an agent that knows nothing about how
14:40the app was built and then it waits for
14:41that agent to finish. When the agent is
14:43done, the skill reports everything the
14:45agent found and suggests fixes you can
14:47apply to make your app better. Finding
14:49the mistakes is one half and the other
14:51half is undoing the changes that caused
14:53them. For that, you can use git, which
14:55basically saves every version of your
14:57app. With git, each feature gets its own
14:59copy of the project, which is called a
15:01branch. If any feature in your app
15:03causes issues later on, you can just
15:05undo it from git. To find and undo the
15:07change that broke something, you can ask
15:09Claude to create a skill called fix a
15:11bug. Basically, when a feature isn't
15:13working the way you need it to, you give
15:15this skill the problem and it finds the
15:17cause and how to fix it. It tries to
15:19make the bug happen again by going
15:21through the same steps and then checks
15:23whether its guess about the cause was
15:24right. After that, it reports the
15:26problem and fixes it. For this skill to
15:28work, you also need to tell Claude in
15:30your claude.md to use git and make its
15:33own branch for each feature. It should
15:35also save every version that works. That
15:37way, when a feature isn't working as
15:39expected, Claude can go back through the
15:41saved versions to find the change that
15:43caused it and roll back to an earlier
15:45version. That brings us to the end of
15:46this video. If you'd like to support the
15:48channel and help us keep making videos
15:50like this, you can do so by using the
15:52super thanks button below. As always,
15:54thank you for watching and I'll see you
15:56in the next one.

About this transcript

This page contains the full transcript of Anthropic Just Released A New System For Using Claude Code by AI LABS, generated from the public captions YouTube serves with the video. The transcript has 3,516 words across 485 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.