He Finally 10x Claude Code With This Method — Transcript
Full transcript
- 0:00People have been using loops to get AI
- 0:01agents to build a lot of things on their
- 0:03own. And the reason that works is the
- 0:05structured workflow behind each loop.
- 0:07That's why people let their agents work
- 0:09on big tasks for hours at a time, and
- 0:11they're getting good results. There are
- 0:12a lot of ways to set one of these
- 0:14workflows up, and one of them is
- 0:16Karpathy loop. So, we set up the
- 0:17Karpathy loop on our own apps to see how
- 0:20well it actually holds up. It held up
- 0:22well, but we found one issue that the
- 0:24loop can't see on its own. To fix that
- 0:26issue, we made one change to the loop.
- 0:28If this is your first time, we're
- 0:30software company, and this is our
- 0:31channel AI Labs. And in this video,
- 0:33we're going to show you what the
- 0:34Karpathy loop is, how to set it up on
- 0:36your own apps, and the change that fixes
- 0:38that issue. But before [snorts] we go
- 0:40into the workflow, let's go over
- 0:42Karpathy's method itself, because this
- 0:44loop is built on it. If you don't know
- 0:45loops, we already have a full video on
- 0:47them on our channel. You can watch it so
- 0:49that you understand loops better. Now,
- 0:51Andre Karpathy made a project called
- 0:53Auto Researcher a while back, where an
- 0:54AI agent kept working on training a
- 0:56model and ran the code again and again
- 0:58until the model actually improved. The
- 1:00agent did this in a loop, so there was
- 1:02no need to direct its steps. Now, in
- 1:04Auto Researcher, the agent was only
- 1:06allowed to change one file, and that was
- 1:08the file that trained the model. Every
- 1:09round was one experiment where the agent
- 1:11made one change to that training file,
- 1:13and then trained the model for a few
- 1:15minutes. After that, a second file gave
- 1:17the model a score. If the score got
- 1:19better, the agent kept the change. But
- 1:21if the score stayed the same or got
- 1:22worse, the agent undid the change. The
- 1:24agent wasn't allowed to touch the
- 1:26scoring file. This was because if the
- 1:28agent could change the scoring file, it
- 1:30could have just made the score easier to
- 1:32beat instead of making the model better.
- 1:34And a third file called program.md told
- 1:36the agent how to run each round. And
- 1:38program.md was basically an instructions
- 1:41file that Karpathy wrote in plain
- 1:43English. Karpathy wrote that
- 1:44instructions file and let the loop run
- 1:46for 2 days. In that time, the agent ran
- 1:49700 experiments and found 20 changes
- 1:51that made the model train faster. Now,
- 1:53this was an experiment that showed how
- 1:55agents can keep going and check every
- 1:57single possibility, which wouldn't be
- 1:59possible if you controlled every step
- 2:01yourself. This is because controlling
- 2:02700 experiments yourself takes a long
- 2:05time, but the agent ran them on its own
- 2:07and found different methods that worked.
- 2:09The CEO of Shopify also ran the same
- 2:11kind of loop and got good results. He
- 2:13ran it on one of his own models while he
- 2:15slept, and the agent ran 37 experiments.
- 2:17In the morning, his model was performing
- 2:1919% better. So, Karpathy's method showed
- 2:22that when an agent is just given a goal,
- 2:24it can keep trying things and get better
- 2:26on its own without you directing every
- 2:28step. This is when a lot of people
- 2:30started getting into loop engineering.
- 2:32Now, before [snorts] you set up a loop,
- 2:33you need to know when you should even
- 2:35use one in the first place. If you use
- 2:37loops for the wrong kind of task, you
- 2:39waste tokens and might not even get what
- 2:41you want. Loops are really helpful when
- 2:43they're placed in the right scenarios,
- 2:44but the right scenario depends on what
- 2:46you're actually trying to get done with
- 2:48the task. For a loop to be worth setting
- 2:50up, the task needs four things. The
- 2:52first is that you repeat the task often.
- 2:54A loop takes time to build, and you only
- 2:56get that time back when you keep
- 2:58repeating the task. So, for a job you do
- 3:00once, one good prompt is enough. If you
- 3:02set up a loop for that kind of job,
- 3:04you'll spend more time on the setup than
- 3:06on the task itself. The second is your
- 3:08usage limit, because loops use a lot of
- 3:10tokens. In a loop, the agent reads your
- 3:12project again and tries a new fix on
- 3:14every round. That uses tokens even on
- 3:16the rounds that fail. So, a long loop on
- 3:18a $20 plan hits the usage limit before
- 3:21the work is even done. The third is that
- 3:23the work can be checked without by
- 3:25something that gives a clear score. In
- 3:27an app, that's done by small pieces of
- 3:29code that try a feature and confirm it
- 3:31works. The last is that the agent can
- 3:33actually run what it built and see what
- 3:35breaks, so it knows what to fix in the
- 3:37next round. We use different types of
- 3:39loops in our workflows, but only for
- 3:41tasks that have a clear score we can
- 3:43measure, and we never let a full app get
- 3:45built in one loop and call it done. We
- 3:48only use loops to build one feature at a
- 3:50time or a simple version of the app that
- 3:52the loops checks can verify. But before
- 3:54we set things up, it would be great if
- 3:56you subscribe to the channel and hit the
- 3:58hype button. This small gesture of
- 3:59support goes a long way for us. Karpathy
- 4:02[snorts] has shared a lot about how he
- 4:03works with AI agents. So we took what he
- 4:06shared and turned it into a setup we
- 4:08could use in our own workflow. In order
- 4:10to make his loop work for our own app,
- 4:12we set up a few skills and agents so
- 4:14that we could automate this job. We'll
- 4:15explain what each part does in depth so
- 4:18that you can ask Claude to build each
- 4:20one for you as well. The first thing we
- 4:21built was a skill that's really
- 4:22important called project context. This
- 4:25skill is specific to whatever you're
- 4:26building and contains everything there
- 4:28is to know about it. It acts as a memory
- 4:30bank for what the app does, which pages
- 4:32it has, the conventions it follows, and
- 4:34the things it should avoid. So when the
- 4:36model is working, it can pull context
- 4:38from there. The project context skill
- 4:40grows as the app grows. So it's always
- 4:42up to date with the latest version. We
- 4:44use a skill instead of a files where
- 4:46this information would normally live
- 4:48because only the skills short
- 4:50description stays in window all the
- 4:52time. That means the agent always knows
- 4:54it's available, but only loads the full
- 4:56details when needed instead of carrying
- 4:58all of them into every task. So whatever
- 5:01you're building, you can ask the agent
- 5:02you're working with to set up a skill
- 5:04that stores all of this context for you.
- 5:06After that, there's a main skill that
- 5:08drives the whole thing so that the loop
- 5:10can run without you watching it. It's
- 5:11the build skill and it has all the
- 5:13instructions for running the loop on
- 5:15each feature. The first job of the build
- 5:17skill is to write the checks and it
- 5:19doesn't do that on its own. It uses
- 5:21another skill called write checks. Write
- 5:23checks is similar to the test author
- 5:24agent we used in our previous videos,
- 5:26but it's tailored to this project and
- 5:28workflow. This makes sure that the
- 5:30checks are written before any feature is
- 5:32built so that the agent has something to
- 5:34verify against instead of judging its
- 5:36own code. Once the checks are written,
- 5:37the build skill lists what every check
- 5:39tests in simple words. This is where you
- 5:42need to go back and forth with your
- 5:43agent so that you can make sure the
- 5:45checks cover all the problems that
- 5:46commonly happen during the work. Once
- 5:48you give it a go-ahead, a small program
- 5:50called approve checks locks them in. The
- 5:52approved checks program moves the checks
- 5:54into a locked folder, and a rule in
- 5:56Claude code settings blocks the agent
- 5:58from editing that folder. So, the loop
- 6:00can't make its own checks easier. Once
- 6:02you approve the checks, the build skill
- 6:04commits them. This adds another layer of
- 6:06protection in case the agent tries to
- 6:08change them. Then, it runs the loop, but
- 6:09the build skill doesn't build the
- 6:11features in its own context window. To
- 6:13keep each feature's context separate, it
- 6:15gives every feature to the feature
- 6:17builder agent. The feature builder agent
- 6:19starts fresh each time and builds the
- 6:21feature as instructed. The result of
- 6:23every round is stored in a file so that
- 6:25we can use it later on. When every
- 6:27feature is done, the build skill gives
- 6:29out a report, and all the loop's
- 6:31instructions like the fixed rules and
- 6:33how to work are written in program.md.
- 6:35This follows the same format Karpathy
- 6:37used, too. That makes our setup complete
- 6:39for running the loop this way. But,
- 6:41before we run this setup, let's have a
- 6:42word by our sponsor, Upstash Box. Agents
- 6:45can build and fix real software now, but
- 6:47your laptop can only run one at a time.
- 6:49Because running more than one means
- 6:51they'll either collide with each other,
- 6:53slow everything down, or worse, they'll
- 6:55quietly read or change files on your
- 6:57machine. That's why you'd want a
- 6:58sandbox, and it's exactly the problem
- 7:00Upstash Box fixes by giving each agent
- 7:03its own sandbox. Each one is a real
- 7:05computer in the cloud with its own shell
- 7:07and file system. So, an agent can work
- 7:09on your repo without ever touching your
- 7:11machine. We spun up 10 of them, each
- 7:13with its own job. One built a new
- 7:15feature, another chased down a bug, and
- 7:17none of it touched our laptop. Because
- 7:18each Upstash sandbox saves its own
- 7:20state, they kept working for hours, even
- 7:23overnight. It's a whole software
- 7:24factory, and none of it lives on your
- 7:26machine. There's a free tier, you only
- 7:28pay for what you use, and it scales to
- 7:30zero when nothing's running. Have
- 7:31multiple agents powering your workflow
- 7:33at upstash.com, links in the
- 7:35description. Now, since we were working
- 7:37on a project that was already
- 7:39half-built, we prompted Claude to use
- 7:41the build skill and asked it to build a
- 7:43feature that lets guests order food for
- 7:45pick up from the restaurant's website.
- 7:47But that feature wasn't in our feature
- 7:49list yet. So Claude added it and wrote
- 7:51the rules for online ordering into
- 7:53features.md.
- 7:54This was the file that we were using for
- 7:56feature tracking. With the rules written
- 7:58down, Claude gave us all the checks it
- 8:00was going to make. The first list had 10
- 8:02checks. We wanted the guest's email as
- 8:04well. So we asked Claude to add it. That
- 8:07made 11 checks and the new check made
- 8:09sure the email was a real address. When
- 8:11we approved the checks, Claude committed
- 8:13them. Then the feature builder agent
- 8:15built the feature and it took about 6
- 8:16minutes. At the end, the report showed
- 8:18that all 11 checks passed in the first
- 8:21round, so the loop didn't need another
- 8:22round. But the loop also caught
- 8:24something the score didn't show. The
- 8:26checks only tested the ordering rules.
- 8:28So the loop had written those rules, but
- 8:30not the order form that guests use on
- 8:32the website. So the loop built the order
- 8:34form and connected it to those rules
- 8:36before it marked the feature as done.
- 8:38But this kind of loop has one problem,
- 8:40which is that the agent has habits. In a
- 8:42loop, if it sees that a particular way
- 8:44wasn't working, it's going to improve it
- 8:46in that loop, but it won't remember that
- 8:48for the next time it runs that loop. Our
- 8:50loop has the same problem because every
- 8:52feature starts with a fresh feature
- 8:54builder agent and the same files for
- 8:55instructions. So a mistake that a model
- 8:57made in one round is going to carry over
- 8:59to the next. The fix for that is a
- 9:01second loop that reads how the first
- 9:03loop ran and changes how the first loop
- 9:05works. We built our own version of that
- 9:07second loop as a skill called auto loop.
- 9:09In Carpen these method, a person writes
- 9:11program.md, but in our setup, the auto
- 9:13loop rewrites the how to work part of
- 9:15program.md for us. To do that, the auto
- 9:18loop reads results file, which stores
- 9:20every round the loop ran. For every
- 9:22round, results file also shows whether
- 9:24the round was kept or undone, what the
- 9:26agent tried, and which checks still
- 9:28failed. The auto loop basically checks
- 9:30for problems that keep coming back like
- 9:32the same mistake agent was making or the
- 9:34same gap showing up in two features.
- 9:36Then it writes down each habit with the
- 9:38rounds that show it. Then the auto loop
- 9:40rewrites how to work part in the
- 9:42program.md, so the next features are
- 9:44built with the new instructions. But the
- 9:46auto loop can't edit the checks. If the
- 9:48auto loop could edit the checks, it
- 9:50could make them easier to pass. Then the
- 9:52rounds would stop failing without the
- 9:54habits ever getting fixed. Now, to test
- 9:56the loop within the loop, we ran one on
- 9:58another project. It was a project
- 10:00management app that didn't have any way
- 10:02of people combining and sharing their
- 10:04workflows. To run it, we asked Claude to
- 10:06create another skill called auto loop.
- 10:08The auto loop skill basically runs the
- 10:10same loop with the same feature builder
- 10:12agent and write check skill that the
- 10:13previous setup was using, but it runs
- 10:16that loop inside another loop. This is
- 10:18different from the build skill that we
- 10:19used previously because the build skill
- 10:21only runs one pass. Every feature gets
- 10:24the same instructions and the agents
- 10:26build it, but nothing the loop learns
- 10:27carries over to the next feature. The
- 10:29auto loop skill also runs the loop one
- 10:32feature at a time, but after each
- 10:33feature, it reviews and updates the
- 10:36instructions, so the next feature starts
- 10:38from the method that actually worked.
- 10:40This helps make your loop much better.
- 10:42The whole purpose of this skill is
- 10:44actually to improve the loop itself and
- 10:45make changes in the instructions. So,
- 10:47whenever the loop runs the next time, it
- 10:49works better than the previous run. So,
- 10:51to add features to our app, we ran the
- 10:54auto loop skill and told it what we
- 10:55wanted. We asked for a way for many
- 10:57people to work together, so they could
- 10:59share focus sessions and run projects
- 11:01together. The auto loop skill then
- 11:03started by adding those features to the
- 11:05tracking file. Just like before, the
- 11:07auto loop skill first lists the checks.
- 11:10You can change them if needed or ask
- 11:11your agent to start building directly.
- 11:13The first feature was a shared database,
- 11:15so everyone's work could live in one
- 11:17place. The database passed all 10 of its
- 11:19checks, but the app never saved anything
- 11:22to it. So, the auto loop added a habit
- 11:24to the instructions, which was to
- 11:25connect each feature to the app in the
- 11:27same round that makes its checks pass.
- 11:30Then the auto loop spotted a second
- 11:31problem with mentions, which let you
- 11:33assign a task by typing someone's name
- 11:35after an at sign. Mentions passed its
- 11:38checks, but an older part of the app
- 11:39still read the at sign the old way.
- 11:41That's why the auto loop added a second
- 11:43habit, which was to find every place the
- 11:46app already does a job and make each one
- 11:48follow the new rules. Both habits worked
- 11:50because the next features came with
- 11:52their screens already connected. To see
- 11:54the checks in action, the builder got
- 11:56two rounds of shared projects wrong. In
- 11:58the first round, it broke mentions, so
- 12:00the mentions failed and the round was
- 12:02undone. In the second round, it
- 12:03estimated each person's share the simple
- 12:05way, so three people got 33 each, which
- 12:08only adds up to 99. That check failed,
- 12:11and in the third round, the builder
- 12:12fixed it and passed all 10 checks. When
- 12:14it is done, it gives a report telling
- 12:16you where the agent went wrong and how
- 12:18it fixed that, along with what the
- 12:20workflow looked like. So, whenever
- 12:21you're working on the next feature, you
- 12:23don't have to face the same issue again.
- 12:25Now, all the skills that we created in
- 12:27this video, along with other resources,
- 12:29are all available in our community, AI
- 12:31Labs Pro. So, if you found value in what
- 12:33we do and want to support the channel,
- 12:35this is the best way to do it. The
- 12:36link's in the description. That brings
- 12:38us to the end of this video. If you'd
- 12:40like to support the channel and help us
- 12:42keep making videos like this, you can do
- 12:44so by using the Super Thanks button
- 12:46below. As always, thank you for watching
- 12:48and I'll see you in the next one.
About this transcript
This page contains the full transcript of He Finally 10x Claude Code With This Method by AI LABS, generated from the public captions YouTube serves with the video. The transcript has 2,856 words across 394 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.