YouTube2Text

He Finally 10x Claude Code With This Method — Transcript

by AI LABS · 2,856 words · 394 segments · language en · Watch on YouTube

Full transcript

  1. 0:00People have been using loops to get AI
  2. 0:01agents to build a lot of things on their
  3. 0:03own. And the reason that works is the
  4. 0:05structured workflow behind each loop.
  5. 0:07That's why people let their agents work
  6. 0:09on big tasks for hours at a time, and
  7. 0:11they're getting good results. There are
  8. 0:12a lot of ways to set one of these
  9. 0:14workflows up, and one of them is
  10. 0:16Karpathy loop. So, we set up the
  11. 0:17Karpathy loop on our own apps to see how
  12. 0:20well it actually holds up. It held up
  13. 0:22well, but we found one issue that the
  14. 0:24loop can't see on its own. To fix that
  15. 0:26issue, we made one change to the loop.
  16. 0:28If this is your first time, we're
  17. 0:30software company, and this is our
  18. 0:31channel AI Labs. And in this video,
  19. 0:33we're going to show you what the
  20. 0:34Karpathy loop is, how to set it up on
  21. 0:36your own apps, and the change that fixes
  22. 0:38that issue. But before [snorts] we go
  23. 0:40into the workflow, let's go over
  24. 0:42Karpathy's method itself, because this
  25. 0:44loop is built on it. If you don't know
  26. 0:45loops, we already have a full video on
  27. 0:47them on our channel. You can watch it so
  28. 0:49that you understand loops better. Now,
  29. 0:51Andre Karpathy made a project called
  30. 0:53Auto Researcher a while back, where an
  31. 0:54AI agent kept working on training a
  32. 0:56model and ran the code again and again
  33. 0:58until the model actually improved. The
  34. 1:00agent did this in a loop, so there was
  35. 1:02no need to direct its steps. Now, in
  36. 1:04Auto Researcher, the agent was only
  37. 1:06allowed to change one file, and that was
  38. 1:08the file that trained the model. Every
  39. 1:09round was one experiment where the agent
  40. 1:11made one change to that training file,
  41. 1:13and then trained the model for a few
  42. 1:15minutes. After that, a second file gave
  43. 1:17the model a score. If the score got
  44. 1:19better, the agent kept the change. But
  45. 1:21if the score stayed the same or got
  46. 1:22worse, the agent undid the change. The
  47. 1:24agent wasn't allowed to touch the
  48. 1:26scoring file. This was because if the
  49. 1:28agent could change the scoring file, it
  50. 1:30could have just made the score easier to
  51. 1:32beat instead of making the model better.
  52. 1:34And a third file called program.md told
  53. 1:36the agent how to run each round. And
  54. 1:38program.md was basically an instructions
  55. 1:41file that Karpathy wrote in plain
  56. 1:43English. Karpathy wrote that
  57. 1:44instructions file and let the loop run
  58. 1:46for 2 days. In that time, the agent ran
  59. 1:49700 experiments and found 20 changes
  60. 1:51that made the model train faster. Now,
  61. 1:53this was an experiment that showed how
  62. 1:55agents can keep going and check every
  63. 1:57single possibility, which wouldn't be
  64. 1:59possible if you controlled every step
  65. 2:01yourself. This is because controlling
  66. 2:02700 experiments yourself takes a long
  67. 2:05time, but the agent ran them on its own
  68. 2:07and found different methods that worked.
  69. 2:09The CEO of Shopify also ran the same
  70. 2:11kind of loop and got good results. He
  71. 2:13ran it on one of his own models while he
  72. 2:15slept, and the agent ran 37 experiments.
  73. 2:17In the morning, his model was performing
  74. 2:1919% better. So, Karpathy's method showed
  75. 2:22that when an agent is just given a goal,
  76. 2:24it can keep trying things and get better
  77. 2:26on its own without you directing every
  78. 2:28step. This is when a lot of people
  79. 2:30started getting into loop engineering.
  80. 2:32Now, before [snorts] you set up a loop,
  81. 2:33you need to know when you should even
  82. 2:35use one in the first place. If you use
  83. 2:37loops for the wrong kind of task, you
  84. 2:39waste tokens and might not even get what
  85. 2:41you want. Loops are really helpful when
  86. 2:43they're placed in the right scenarios,
  87. 2:44but the right scenario depends on what
  88. 2:46you're actually trying to get done with
  89. 2:48the task. For a loop to be worth setting
  90. 2:50up, the task needs four things. The
  91. 2:52first is that you repeat the task often.
  92. 2:54A loop takes time to build, and you only
  93. 2:56get that time back when you keep
  94. 2:58repeating the task. So, for a job you do
  95. 3:00once, one good prompt is enough. If you
  96. 3:02set up a loop for that kind of job,
  97. 3:04you'll spend more time on the setup than
  98. 3:06on the task itself. The second is your
  99. 3:08usage limit, because loops use a lot of
  100. 3:10tokens. In a loop, the agent reads your
  101. 3:12project again and tries a new fix on
  102. 3:14every round. That uses tokens even on
  103. 3:16the rounds that fail. So, a long loop on
  104. 3:18a $20 plan hits the usage limit before
  105. 3:21the work is even done. The third is that
  106. 3:23the work can be checked without by
  107. 3:25something that gives a clear score. In
  108. 3:27an app, that's done by small pieces of
  109. 3:29code that try a feature and confirm it
  110. 3:31works. The last is that the agent can
  111. 3:33actually run what it built and see what
  112. 3:35breaks, so it knows what to fix in the
  113. 3:37next round. We use different types of
  114. 3:39loops in our workflows, but only for
  115. 3:41tasks that have a clear score we can
  116. 3:43measure, and we never let a full app get
  117. 3:45built in one loop and call it done. We
  118. 3:48only use loops to build one feature at a
  119. 3:50time or a simple version of the app that
  120. 3:52the loops checks can verify. But before
  121. 3:54we set things up, it would be great if
  122. 3:56you subscribe to the channel and hit the
  123. 3:58hype button. This small gesture of
  124. 3:59support goes a long way for us. Karpathy
  125. 4:02[snorts] has shared a lot about how he
  126. 4:03works with AI agents. So we took what he
  127. 4:06shared and turned it into a setup we
  128. 4:08could use in our own workflow. In order
  129. 4:10to make his loop work for our own app,
  130. 4:12we set up a few skills and agents so
  131. 4:14that we could automate this job. We'll
  132. 4:15explain what each part does in depth so
  133. 4:18that you can ask Claude to build each
  134. 4:20one for you as well. The first thing we
  135. 4:21built was a skill that's really
  136. 4:22important called project context. This
  137. 4:25skill is specific to whatever you're
  138. 4:26building and contains everything there
  139. 4:28is to know about it. It acts as a memory
  140. 4:30bank for what the app does, which pages
  141. 4:32it has, the conventions it follows, and
  142. 4:34the things it should avoid. So when the
  143. 4:36model is working, it can pull context
  144. 4:38from there. The project context skill
  145. 4:40grows as the app grows. So it's always
  146. 4:42up to date with the latest version. We
  147. 4:44use a skill instead of a files where
  148. 4:46this information would normally live
  149. 4:48because only the skills short
  150. 4:50description stays in window all the
  151. 4:52time. That means the agent always knows
  152. 4:54it's available, but only loads the full
  153. 4:56details when needed instead of carrying
  154. 4:58all of them into every task. So whatever
  155. 5:01you're building, you can ask the agent
  156. 5:02you're working with to set up a skill
  157. 5:04that stores all of this context for you.
  158. 5:06After that, there's a main skill that
  159. 5:08drives the whole thing so that the loop
  160. 5:10can run without you watching it. It's
  161. 5:11the build skill and it has all the
  162. 5:13instructions for running the loop on
  163. 5:15each feature. The first job of the build
  164. 5:17skill is to write the checks and it
  165. 5:19doesn't do that on its own. It uses
  166. 5:21another skill called write checks. Write
  167. 5:23checks is similar to the test author
  168. 5:24agent we used in our previous videos,
  169. 5:26but it's tailored to this project and
  170. 5:28workflow. This makes sure that the
  171. 5:30checks are written before any feature is
  172. 5:32built so that the agent has something to
  173. 5:34verify against instead of judging its
  174. 5:36own code. Once the checks are written,
  175. 5:37the build skill lists what every check
  176. 5:39tests in simple words. This is where you
  177. 5:42need to go back and forth with your
  178. 5:43agent so that you can make sure the
  179. 5:45checks cover all the problems that
  180. 5:46commonly happen during the work. Once
  181. 5:48you give it a go-ahead, a small program
  182. 5:50called approve checks locks them in. The
  183. 5:52approved checks program moves the checks
  184. 5:54into a locked folder, and a rule in
  185. 5:56Claude code settings blocks the agent
  186. 5:58from editing that folder. So, the loop
  187. 6:00can't make its own checks easier. Once
  188. 6:02you approve the checks, the build skill
  189. 6:04commits them. This adds another layer of
  190. 6:06protection in case the agent tries to
  191. 6:08change them. Then, it runs the loop, but
  192. 6:09the build skill doesn't build the
  193. 6:11features in its own context window. To
  194. 6:13keep each feature's context separate, it
  195. 6:15gives every feature to the feature
  196. 6:17builder agent. The feature builder agent
  197. 6:19starts fresh each time and builds the
  198. 6:21feature as instructed. The result of
  199. 6:23every round is stored in a file so that
  200. 6:25we can use it later on. When every
  201. 6:27feature is done, the build skill gives
  202. 6:29out a report, and all the loop's
  203. 6:31instructions like the fixed rules and
  204. 6:33how to work are written in program.md.
  205. 6:35This follows the same format Karpathy
  206. 6:37used, too. That makes our setup complete
  207. 6:39for running the loop this way. But,
  208. 6:41before we run this setup, let's have a
  209. 6:42word by our sponsor, Upstash Box. Agents
  210. 6:45can build and fix real software now, but
  211. 6:47your laptop can only run one at a time.
  212. 6:49Because running more than one means
  213. 6:51they'll either collide with each other,
  214. 6:53slow everything down, or worse, they'll
  215. 6:55quietly read or change files on your
  216. 6:57machine. That's why you'd want a
  217. 6:58sandbox, and it's exactly the problem
  218. 7:00Upstash Box fixes by giving each agent
  219. 7:03its own sandbox. Each one is a real
  220. 7:05computer in the cloud with its own shell
  221. 7:07and file system. So, an agent can work
  222. 7:09on your repo without ever touching your
  223. 7:11machine. We spun up 10 of them, each
  224. 7:13with its own job. One built a new
  225. 7:15feature, another chased down a bug, and
  226. 7:17none of it touched our laptop. Because
  227. 7:18each Upstash sandbox saves its own
  228. 7:20state, they kept working for hours, even
  229. 7:23overnight. It's a whole software
  230. 7:24factory, and none of it lives on your
  231. 7:26machine. There's a free tier, you only
  232. 7:28pay for what you use, and it scales to
  233. 7:30zero when nothing's running. Have
  234. 7:31multiple agents powering your workflow
  235. 7:33at upstash.com, links in the
  236. 7:35description. Now, since we were working
  237. 7:37on a project that was already
  238. 7:39half-built, we prompted Claude to use
  239. 7:41the build skill and asked it to build a
  240. 7:43feature that lets guests order food for
  241. 7:45pick up from the restaurant's website.
  242. 7:47But that feature wasn't in our feature
  243. 7:49list yet. So Claude added it and wrote
  244. 7:51the rules for online ordering into
  245. 7:53features.md.
  246. 7:54This was the file that we were using for
  247. 7:56feature tracking. With the rules written
  248. 7:58down, Claude gave us all the checks it
  249. 8:00was going to make. The first list had 10
  250. 8:02checks. We wanted the guest's email as
  251. 8:04well. So we asked Claude to add it. That
  252. 8:07made 11 checks and the new check made
  253. 8:09sure the email was a real address. When
  254. 8:11we approved the checks, Claude committed
  255. 8:13them. Then the feature builder agent
  256. 8:15built the feature and it took about 6
  257. 8:16minutes. At the end, the report showed
  258. 8:18that all 11 checks passed in the first
  259. 8:21round, so the loop didn't need another
  260. 8:22round. But the loop also caught
  261. 8:24something the score didn't show. The
  262. 8:26checks only tested the ordering rules.
  263. 8:28So the loop had written those rules, but
  264. 8:30not the order form that guests use on
  265. 8:32the website. So the loop built the order
  266. 8:34form and connected it to those rules
  267. 8:36before it marked the feature as done.
  268. 8:38But this kind of loop has one problem,
  269. 8:40which is that the agent has habits. In a
  270. 8:42loop, if it sees that a particular way
  271. 8:44wasn't working, it's going to improve it
  272. 8:46in that loop, but it won't remember that
  273. 8:48for the next time it runs that loop. Our
  274. 8:50loop has the same problem because every
  275. 8:52feature starts with a fresh feature
  276. 8:54builder agent and the same files for
  277. 8:55instructions. So a mistake that a model
  278. 8:57made in one round is going to carry over
  279. 8:59to the next. The fix for that is a
  280. 9:01second loop that reads how the first
  281. 9:03loop ran and changes how the first loop
  282. 9:05works. We built our own version of that
  283. 9:07second loop as a skill called auto loop.
  284. 9:09In Carpen these method, a person writes
  285. 9:11program.md, but in our setup, the auto
  286. 9:13loop rewrites the how to work part of
  287. 9:15program.md for us. To do that, the auto
  288. 9:18loop reads results file, which stores
  289. 9:20every round the loop ran. For every
  290. 9:22round, results file also shows whether
  291. 9:24the round was kept or undone, what the
  292. 9:26agent tried, and which checks still
  293. 9:28failed. The auto loop basically checks
  294. 9:30for problems that keep coming back like
  295. 9:32the same mistake agent was making or the
  296. 9:34same gap showing up in two features.
  297. 9:36Then it writes down each habit with the
  298. 9:38rounds that show it. Then the auto loop
  299. 9:40rewrites how to work part in the
  300. 9:42program.md, so the next features are
  301. 9:44built with the new instructions. But the
  302. 9:46auto loop can't edit the checks. If the
  303. 9:48auto loop could edit the checks, it
  304. 9:50could make them easier to pass. Then the
  305. 9:52rounds would stop failing without the
  306. 9:54habits ever getting fixed. Now, to test
  307. 9:56the loop within the loop, we ran one on
  308. 9:58another project. It was a project
  309. 10:00management app that didn't have any way
  310. 10:02of people combining and sharing their
  311. 10:04workflows. To run it, we asked Claude to
  312. 10:06create another skill called auto loop.
  313. 10:08The auto loop skill basically runs the
  314. 10:10same loop with the same feature builder
  315. 10:12agent and write check skill that the
  316. 10:13previous setup was using, but it runs
  317. 10:16that loop inside another loop. This is
  318. 10:18different from the build skill that we
  319. 10:19used previously because the build skill
  320. 10:21only runs one pass. Every feature gets
  321. 10:24the same instructions and the agents
  322. 10:26build it, but nothing the loop learns
  323. 10:27carries over to the next feature. The
  324. 10:29auto loop skill also runs the loop one
  325. 10:32feature at a time, but after each
  326. 10:33feature, it reviews and updates the
  327. 10:36instructions, so the next feature starts
  328. 10:38from the method that actually worked.
  329. 10:40This helps make your loop much better.
  330. 10:42The whole purpose of this skill is
  331. 10:44actually to improve the loop itself and
  332. 10:45make changes in the instructions. So,
  333. 10:47whenever the loop runs the next time, it
  334. 10:49works better than the previous run. So,
  335. 10:51to add features to our app, we ran the
  336. 10:54auto loop skill and told it what we
  337. 10:55wanted. We asked for a way for many
  338. 10:57people to work together, so they could
  339. 10:59share focus sessions and run projects
  340. 11:01together. The auto loop skill then
  341. 11:03started by adding those features to the
  342. 11:05tracking file. Just like before, the
  343. 11:07auto loop skill first lists the checks.
  344. 11:10You can change them if needed or ask
  345. 11:11your agent to start building directly.
  346. 11:13The first feature was a shared database,
  347. 11:15so everyone's work could live in one
  348. 11:17place. The database passed all 10 of its
  349. 11:19checks, but the app never saved anything
  350. 11:22to it. So, the auto loop added a habit
  351. 11:24to the instructions, which was to
  352. 11:25connect each feature to the app in the
  353. 11:27same round that makes its checks pass.
  354. 11:30Then the auto loop spotted a second
  355. 11:31problem with mentions, which let you
  356. 11:33assign a task by typing someone's name
  357. 11:35after an at sign. Mentions passed its
  358. 11:38checks, but an older part of the app
  359. 11:39still read the at sign the old way.
  360. 11:41That's why the auto loop added a second
  361. 11:43habit, which was to find every place the
  362. 11:46app already does a job and make each one
  363. 11:48follow the new rules. Both habits worked
  364. 11:50because the next features came with
  365. 11:52their screens already connected. To see
  366. 11:54the checks in action, the builder got
  367. 11:56two rounds of shared projects wrong. In
  368. 11:58the first round, it broke mentions, so
  369. 12:00the mentions failed and the round was
  370. 12:02undone. In the second round, it
  371. 12:03estimated each person's share the simple
  372. 12:05way, so three people got 33 each, which
  373. 12:08only adds up to 99. That check failed,
  374. 12:11and in the third round, the builder
  375. 12:12fixed it and passed all 10 checks. When
  376. 12:14it is done, it gives a report telling
  377. 12:16you where the agent went wrong and how
  378. 12:18it fixed that, along with what the
  379. 12:20workflow looked like. So, whenever
  380. 12:21you're working on the next feature, you
  381. 12:23don't have to face the same issue again.
  382. 12:25Now, all the skills that we created in
  383. 12:27this video, along with other resources,
  384. 12:29are all available in our community, AI
  385. 12:31Labs Pro. So, if you found value in what
  386. 12:33we do and want to support the channel,
  387. 12:35this is the best way to do it. The
  388. 12:36link's in the description. That brings
  389. 12:38us to the end of this video. If you'd
  390. 12:40like to support the channel and help us
  391. 12:42keep making videos like this, you can do
  392. 12:44so by using the Super Thanks button
  393. 12:46below. As always, thank you for watching
  394. 12:48and I'll see you in the next one.

About this transcript

This page contains the full transcript of He Finally 10x Claude Code With This Method by AI LABS, generated from the public captions YouTube serves with the video. The transcript has 2,856 words across 394 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.