YouTube2Text

LIVE: Poteto (creator of pstack) on shipping 1,000's of PR's a month at SpaceX — Transcript

by Matt Pocock · 12,134 words · 1,707 segments · language en · Watch on YouTube

Full transcript

  1. 0:00So, hello folks. I've got another treat
  2. 0:02for you today. Last time on this kind of
  3. 0:05podcasty thing, I suppose, we had Uncle
  4. 0:07Bob and we talked about software
  5. 0:09quality. We talked about agents. We
  6. 0:10talked about lots of cool stuff. Now, we
  7. 0:13have uh an incredible guest, someone who
  8. 0:15I'm delighted to welcome on, who's been
  9. 0:18exploding on Twitter recently about
  10. 0:21software factories, um about increasing
  11. 0:24the quality of your work, about
  12. 0:26increasing your velocity and climbing
  13. 0:28the trust ladder with agents so that you
  14. 0:30can ship more and more and more. And it
  15. 0:32is potato. Welcome. Thank you so much
  16. 0:35for joining.
  17. 0:36>> Thanks for having me. Yeah, very excited
  18. 0:38to be here. Yeah, big fan of yours
  19. 0:41>> and a huge fan of yours. I [laughter]
  20. 0:42think people have been talking about
  21. 0:44this like it's like the meeting of the
  22. 0:45skill minds, the skill Mount Olympus or
  23. 0:49something because both of us have very
  24. 0:50popular skill libraries. Um I've not, as
  25. 0:52I was saying before we started, I've not
  26. 0:54used a ton of yours and like I want to
  27. 0:57get all of the juice out of your brain
  28. 0:59so that I can go and use it properly and
  29. 1:01use it better. And I think where I want
  30. 1:04to start with this is you gave a talk um
  31. 1:06pretty recently like um about 10 days
  32. 1:08ago and posted on X which went
  33. 1:10absolutely nuts as about how I shipped
  34. 1:132,500 PRs last month to production got
  35. 1:16about 3 million views or something on X
  36. 1:19and I watched it and I loved it and I
  37. 1:21recommended it and I kind of want to run
  38. 1:24this as almost like a Q&A of that talk
  39. 1:26basically of giving you because it just
  40. 1:29I just had tons of questions about it
  41. 1:31and I wanted to dive into it. And I
  42. 1:33think where I want to start is you talk
  43. 1:36about a trust ladder with agents where
  44. 1:39you as you trust agents more, you can
  45. 1:42get them to do better and better things
  46. 1:44and or scale them to up to use more and
  47. 1:47more agents. So what is your story of
  48. 1:50how you climbed the trust ladder and how
  49. 1:53did that work when like you got SpaceX
  50. 1:55and
  51. 1:56>> started climbing more and more?
  52. 1:58So I think this the the journey sort of
  53. 2:00began even before I joined cursor uh
  54. 2:03which is now SpaceX AI. Uh [snorts] so
  55. 2:06the story is um
  56. 2:09after Meta so I I used to work at Meta
  57. 2:12on the React team. [snorts] Uh I took a
  58. 2:14month off uh because I was feeling kind
  59. 2:16of burnt out and of course when what
  60. 2:19what do you do when you're burnt out?
  61. 2:20You go and start a new side project. Um
  62. 2:22and so I started a side project. you
  63. 2:24know, I was uh of course using AI to to
  64. 2:27write code. Uh [snorts] but then I
  65. 2:29started to realize uh you know, I was
  66. 2:31spending like so many hours just
  67. 2:33micromanaging one agent, right? And you
  68. 2:36know, at the time, this was back in
  69. 2:38February, maybe February, early February
  70. 2:41or January, you know, people were really
  71. 2:43obsessed with this idea of like
  72. 2:44orchestration. This was like, you know,
  73. 2:46before, you know, things like cursor,
  74. 2:48you know, like the agents window was had
  75. 2:50become popular. So people were still in
  76. 2:53like like 2 land you know in their
  77. 2:55terminal and they were all talking about
  78. 2:56okay here you know I built a custom
  79. 2:58orchestrator right and so of course I
  80. 3:01had I was a bit nerd sniped by that
  81. 3:03[snorts] and you know as I was building
  82. 3:04my toy project uh I got nerd sniped by
  83. 3:08oh how do I make my AI coding setup more
  84. 3:10efficient and so you know I I kind of
  85. 3:13started the journey there where I just
  86. 3:16you know took a step back and realized
  87. 3:18you know I was spending all this time
  88. 3:20micromanaging a single agent you know I
  89. 3:23was [snorts] creating skills and I was
  90. 3:24like finding it quite difficult to
  91. 3:26measure the output or the the result the
  92. 3:29impact of the skill as well so I was
  93. 3:32kind of flying blind but I was you know
  94. 3:34iterating really fast um and um so that
  95. 3:39project eventually sort of became the
  96. 3:42basis of PAC even though I didn't know
  97. 3:44it at the time um and a lot of some
  98. 3:48tricks I had learned like building that
  99. 3:50early set of skills. Actually, it's
  100. 3:52still open source if you want to if
  101. 3:54anybody wants to take a look. It's on my
  102. 3:56GitHub like potato
  103. 3:58noodle n o d l e. Um, and in there you
  104. 4:02will see some skills and a brain
  105. 4:04directory. [snorts] And so I was really
  106. 4:06interested in this idea of how do I, you
  107. 4:09know, extract my own ability, if that
  108. 4:12makes sense, and give it to the agent,
  109. 4:14right? cuz I was I I realized that you
  110. 4:16know all I was trying to do was trying
  111. 4:18to teach the agent to write code more
  112. 4:19like me you know do do you do do
  113. 4:22workflows more like me. So you know the
  114. 4:24skills were like an entry point to doing
  115. 4:27that.
  116. 4:28Um and then you know after I joined
  117. 4:31cursor uh I was starting to work on the
  118. 4:33agents window and uh it had a lot of
  119. 4:36performance issues. Uh it was it was it
  120. 4:39was pretty laggy. Uh and so since I had
  121. 4:41experience working in React, I was asked
  122. 4:43like, "Hey, do you want to come and help
  123. 4:45out uh with the agents window?" Um and
  124. 4:48so the the this beginning of the cursor
  125. 4:52journey was very manual. [snorts] Uh I
  126. 4:55was deep in like looking at like flame
  127. 4:58graphs and heap snapshots and trying to
  128. 5:00see like why exactly is the app so slow.
  129. 5:03Uh but then coming back to the same
  130. 5:05realization like you know I was sort of
  131. 5:07the bottleneck. I was doing everything
  132. 5:09manually. I was sort of the meat proxy
  133. 5:11in a way, right? I was the meat proxy
  134. 5:13between my agent and Chrome DevTools. Uh
  135. 5:16and I was like really annoyed by that.
  136. 5:18>> And what month of the year is that?
  137. 5:20Let's say where are we in the timeline?
  138. 5:22>> Uh so I joined Cursor in March. So this
  139. 5:25was like early early April probably
  140. 5:28early April is when [snorts] you know uh
  141. 5:30I joined and I didn't have any skills,
  142. 5:32right? I had I I sort of abandoned my
  143. 5:35personal skills because I didn't think
  144. 5:36they'd be relevant anymore. Uh but then
  145. 5:39working on the agents window uh and
  146. 5:41[snorts] now working on grockbot uh I
  147. 5:44sort of realized that a lot of the
  148. 5:46lessons I had learned from those skill
  149. 5:48time building the the initial set of
  150. 5:50skills were very relevant especially
  151. 5:52around things like verification
  152. 5:55uh you know being very rigorous in your
  153. 5:57work um [clears throat] because I think
  154. 6:01from my experience even the the frontier
  155. 6:04ones tend to
  156. 6:07tend to take shortcuts. Uh they tend to
  157. 6:10do the easy thing. Uh so uh a lot of the
  158. 6:14skills that I've built have been around
  159. 6:16how do I make the easy thing the right
  160. 6:19thing? You know, how do I make that the
  161. 6:20best thing?
  162. 6:22>> The idea of sort of distilling your
  163. 6:25expertise and turning what you do every
  164. 6:28day into processes, that's something
  165. 6:30that feels super familiar to me. That's
  166. 6:32exactly what I've been doing with the
  167. 6:34skills. And I suppose there's something
  168. 6:37in that which is a lot of people think
  169. 6:39domain expertise is getting less useful
  170. 6:41now as people uh start to rely more on
  171. 6:44AI where what do you think about that
  172. 6:47just as a sort of vibe check before we
  173. 6:49start talking [laughter]
  174. 6:51>> I actually feel like domain expertise is
  175. 6:52is more important than ever you know uh
  176. 6:56I think I wrote this on my ex at some
  177. 6:58point but you know at times I sometimes
  178. 7:01think of you know AI as is like
  179. 7:05especially as the models get smarter and
  180. 7:06smarter and more capable and the
  181. 7:08frontier models are just getting so good
  182. 7:10like I [snorts] love Opus 5.5 by the way
  183. 7:13um uh you know as the models get really
  184. 7:17really really good it almost becomes
  185. 7:19like the bottleneck is no longer the
  186. 7:21agent right it becomes your ability to
  187. 7:24express your intent and your goals in a
  188. 7:27clear way that [snorts] the agent can
  189. 7:29understand and actually carry out and
  190. 7:33That's why I think you know like people
  191. 7:34with a lot of domain expertise are
  192. 7:37extremely have a have a huge advantage
  193. 7:39in my opinion especially if you're a
  194. 7:42little bit like you know tech technoc
  195. 7:44curious you know so I I think of people
  196. 7:46like you know like uh like a doctor or a
  197. 7:49lawyer or you know someone who who has a
  198. 7:52deep expertise in a particular
  199. 7:54non-engineering domain and [snorts] if
  200. 7:56they're actually just a little bit
  201. 7:57techsavvy and they can figure out how to
  202. 7:59use agents they can actually build
  203. 8:01really really great products, right? If
  204. 8:04they if they have a clear enough vision
  205. 8:06in their head and they can articulate it
  206. 8:09in a way that the agent can build it,
  207. 8:11you know, I think that that is really
  208. 8:14the the bottleneck these days is is like
  209. 8:17the transfer of your intent, right, and
  210. 8:20your vision to the agent.
  211. 8:23>> Yeah. I've been obsessed with language
  212. 8:25basically since agents um dropped. are
  213. 8:27just obsessed 100% and thinking
  214. 8:30constantly about the the composition of
  215. 8:32words, how I can make things sharper,
  216. 8:35what um what might be hidden in the
  217. 8:37phrases that I'm using. And it's and
  218. 8:40finding what I love is when you find a
  219. 8:42word that the agent then hooks on to and
  220. 8:45then goes, "Okay, I'm going to reinforce
  221. 8:47that word. I'm going to reuse that in my
  222. 8:49thinking traces." You know, I found that
  223. 8:51with um TDD was an early example of
  224. 8:53that. a lot of chat about TDD recently
  225. 8:55of like, you know, people say, should
  226. 8:57you use TDD with agents? Doesn't matter.
  227. 8:59What you're doing is you're getting the
  228. 9:00agent to think about TDD, getting it to
  229. 9:03write tests, getting it to prioritize
  230. 9:04things in a different way than it did
  231. 9:06before. And that's why sort of grilling,
  232. 9:08I think, works effectively. Grilling is
  233. 9:10a
  234. 9:11>> Yeah. Yeah. It it draws those words out
  235. 9:13of you, right? or or at least it helps
  236. 9:16the agent understand your thinking so
  237. 9:19that they can propose those words to you
  238. 9:20and you can pick up and say yes exactly
  239. 9:22that.
  240. 9:23>> Uh I've actually copied some of the the
  241. 9:25tips that you've shared as well where
  242. 9:27you know one of my favorite ones that
  243. 9:28you've shared recently or or not or like
  244. 9:30maybe in the past couple weeks is
  245. 9:32[snorts] about uh reducing or
  246. 9:34eliminating tautological tests. Like one
  247. 9:37of my pet peeves of agents is like all
  248. 9:39of the useless tests that they write.
  249. 9:41And so, you know, that was one thing
  250. 9:44where, you know, the word tutology,
  251. 9:45right, is is is I guess, you know, not
  252. 9:47many people necessarily know that if if
  253. 9:49especially if English isn't your first
  254. 9:51language, but there's a lot of meaning
  255. 9:53to that word. And it's like it's almost
  256. 9:56like compressed, right? Like you
  257. 9:58compress a lot of intent and meaning
  258. 10:00into words. And so I I I totally agree
  259. 10:04with you. I think language I've always
  260. 10:06been interested in language actually uh
  261. 10:08like programming languages natural human
  262. 10:11languages and how they came to be and
  263. 10:13it's so interesting that now with agents
  264. 10:15it's sort of like this meeting of
  265. 10:18natural language with programming
  266. 10:20language but it's all it's all language
  267. 10:21out of the hood it's all communication
  268. 10:23>> totally I did a drama degree right so
  269. 10:25you know I've been thinking about
  270. 10:26language and Shakespeare and stuff for a
  271. 10:28long time [laughter] and so this all
  272. 10:29feels very familiar
  273. 10:31>> um so okay there's sort
  274. 10:34before we get into like because I think
  275. 10:36the thing I want from you is like
  276. 10:38software factory stuff, right? Software
  277. 10:40factory is the big buzzword. Software
  278. 10:42factory is the thing that I'm thinking
  279. 10:43about too. I'm sort of releasing a
  280. 10:45course in that direction too.
  281. 10:47>> And it's this sort of scaling yourself
  282. 10:50up to un unrealistic numbers of PRs
  283. 10:53basically or PR numbers that sound
  284. 10:56ridiculous to people who don't
  285. 10:57understand how this works. So where I
  286. 10:59want to get to is sort of from people
  287. 11:01who are doing kind of like one to five
  288. 11:03agents today up to, you know, hundreds
  289. 11:06of agents running at once and how that
  290. 11:08sort of functions. And so I'd love to
  291. 11:10hear about your metaphor of the Michelin
  292. 11:12Kitchen instead of the software factory
  293. 11:15because I think that says a bit about
  294. 11:16the way you think about this stuff.
  295. 11:18>> Yes.
  296. 11:20Yeah. I I I've I've never really liked
  297. 11:22the term software factory. Not because
  298. 11:25you know it's not accurate but I think I
  299. 11:28think the a lot of people when they
  300. 11:29think factory right they don't
  301. 11:31necessarily equate that with quality or
  302. 11:34craft right things which are very
  303. 11:36important to me and a lot of people and
  304. 11:39technologists who work you know building
  305. 11:41products we care about the user
  306. 11:44experience we care about the things
  307. 11:45we're building. So while so while I do
  308. 11:48think software factory is an apt term,
  309. 11:51it also I guess maybe conjures up
  310. 11:53negative, you know, maybe sometimes
  311. 11:55negative connotations. So Michelin
  312. 11:58Kitchen is the thing that I've sort of
  313. 12:00landed on where it's much more I feel
  314. 12:02like it's much more aspirational and uh
  315. 12:05I like the metaphor a lot cuz you know I
  316. 12:06like food. I'm called potato of course
  317. 12:09[snorts] and I like cooking [laughter]
  318. 12:10and I see a lot of parallels right like
  319. 12:13with food right when you're cooking a
  320. 12:15meal for yourself for example
  321. 12:17[clears throat] it's both utilitarian
  322. 12:19like you're trying to just feed yourself
  323. 12:20right and and survive uh but it can
  324. 12:23actually be transformed into art right
  325. 12:25and that's what what a Michelin starred
  326. 12:28chef or even just a chef or a cook can
  327. 12:31do with food is take something very
  328. 12:33ordinary and turn it into a delicious
  329. 12:36meal that you know takes you back to
  330. 12:38your childhood days or something like
  331. 12:39that. Um and so it almost like mirrors
  332. 12:43that trust letter that I talk about
  333. 12:45where uh you can sort of imagine your
  334. 12:48own journey as a home cook, right? Uh as
  335. 12:50a home cook, you are doing all of the
  336. 12:52food, the cooking yourself. You cut all
  337. 12:55the vegetables, you do all the prep
  338. 12:57work, you do all the cleanup, you know,
  339. 12:59you are the one man or one woman show
  340. 13:03really. Um, and it's an interesting
  341. 13:07thought experiment like, okay, if you
  342. 13:09were to cook a meal and then you add
  343. 13:11people, right, your your your partner
  344. 13:14trying to your brother, your sister, and
  345. 13:16now suddenly you have your whole family
  346. 13:18in the kitchen. I think most people
  347. 13:19would get very stressed by that, right?
  348. 13:21The thought of, oh, so many people are
  349. 13:23just mocking around in my kitchen. They
  350. 13:24have no no idea where all the utensils
  351. 13:26are.
  352. 13:27>> I have a max capacity of one person in
  353. 13:28the kitchen. Yeah, absolutely. So, I
  354. 13:31feel like that that's really apt because
  355. 13:33when you ask yourself that question of
  356. 13:35how do I go from being a solo cook,
  357. 13:37right, to having an army or even not not
  358. 13:41even an army but a few sue chefs, right,
  359. 13:44that that are helping me in the kitchen.
  360. 13:45How do I think about dividing the work
  361. 13:48in a way that makes sense? You know, I'm
  362. 13:50not dividing work just for the sake of
  363. 13:52it, but in a way that actually makes the
  364. 13:55sum the to the better than, you know,
  365. 13:57the total of its parts. And so the
  366. 14:00Michelin kitchen metaphor to me like
  367. 14:02works really well in that regard because
  368. 14:05you know as a chef you're you know if
  369. 14:08you become a chef you're in a position
  370. 14:10where you're not necessarily cooking all
  371. 14:12the food yourself anymore but you are
  372. 14:14thinking you're almost like the tech
  373. 14:17lead right for the kitchen where uh you
  374. 14:20know chefs have to think about you know
  375. 14:21not just cooking but they have to
  376. 14:23basically organize the whole kitchen and
  377. 14:25they're like the CEO of the kitchen they
  378. 14:27have to think about when do you order
  379. 14:29ingredients, how do you store them, how
  380. 14:30do you prepare them, when do they have
  381. 14:32to be prepared, you know, it's a whole
  382. 14:34it's a whole job, right? That's not just
  383. 14:36cooking. Um, and I think that again it
  384. 14:39mirrors so much of how engineers write
  385. 14:42code today where you are not writing the
  386. 14:45code yourself anymore. You have agents,
  387. 14:47right? But you as the human are still
  388. 14:49responsible for the final outcome,
  389. 14:51right? your name still is associated
  390. 14:53with the work that you do, your
  391. 14:55reputation and you know so how you set
  392. 14:58up your kitchen right and how you set up
  393. 15:01your skills your environment your
  394. 15:03codebase I think are ultimately the new
  395. 15:07ingredients that go into um building
  396. 15:10product
  397. 15:12>> yeah I think what I love about your
  398. 15:14approach is the amount of focus that you
  399. 15:16put into the environment that the agent
  400. 15:18operates in right because I think a lot
  401. 15:20of people they think, right, the agent
  402. 15:22is good. I'm probably not going to be
  403. 15:25able to make it better. Let's just trust
  404. 15:27what these magic model people have put
  405. 15:29into the harness and the model
  406. 15:31combination. Uh, there's nothing I can
  407. 15:33really do, like I can't mess about with
  408. 15:35claw codes internals or something or
  409. 15:36whatever you're using. Um, but what I
  410. 15:40love about your approach, and it's
  411. 15:42something I advocate for too, is that
  412. 15:43you can change the environment the agent
  413. 15:45operates in, right? you can make changes
  414. 15:48in the codebase and also give it tools
  415. 15:51for verification as well and allow it to
  416. 15:54verify its own work. So the thing I I
  417. 15:56loved about watching that talk is the
  418. 15:58amount of focus you put in verification
  419. 16:01and like that is the lever that you can
  420. 16:03start to generate trust. Can you talk
  421. 16:05about that and what that concretely
  422. 16:07looks like? Let's start like looking at
  423. 16:09practical ways that people can improve
  424. 16:10their own processes, their own kitchens.
  425. 16:14Yeah, I've I've I've said this a lot
  426. 16:16actually that you know even if you don't
  427. 16:18use PAC or you know your skills I think
  428. 16:21that the single most important skill
  429. 16:24that should be in your toolkit is
  430. 16:27verification because without
  431. 16:29verification and for for by the way for
  432. 16:32those watching who don't know what that
  433. 16:33means it's this idea that you can give
  434. 16:35you can sort of give your agent uh hands
  435. 16:38and eyes in a way that's the the analogy
  436. 16:42I
  437. 16:43where the agent is able to run the code,
  438. 16:46right? And actually
  439. 16:48>> [snorts]
  440. 16:48>> uh interact with it like a normal human
  441. 16:50user would and also do things like you
  442. 16:53know debug it, you know, take traces and
  443. 16:56snapshots. Um and uh funnily enough like
  444. 17:00that was actually the first skill I
  445. 17:01built when I joined Cursor. uh that gave
  446. 17:03me a lot of that was that was the thing
  447. 17:05that actually started to let me ascend
  448. 17:07the trust ladder a little bit in a way
  449. 17:09that some of the other skills I had
  450. 17:11looked at or built had not really let me
  451. 17:14do because no matter how good you know
  452. 17:17some of the other skills were like the
  453. 17:19how skill, the why skill, the unsop
  454. 17:21skill were, I was still relying on me
  455. 17:24right as the proxy between my agent and
  456. 17:27the output. So that you know if the
  457. 17:30agent can't actually see the result of
  458. 17:31its work there's no way it can actually
  459. 17:34iterate right and so this is where
  460. 17:36people start to talk about loops this
  461. 17:38idea of a loop and really I think the
  462. 17:40term loop you know seems kind of uh
  463. 17:43almost abstract like people like what
  464. 17:46what what is a loop what is an agent
  465. 17:47loop but really to me like the most
  466. 17:50important part of a loop that allows it
  467. 17:52to be a loop is the verification part
  468. 17:55because the agent is able to to verify
  469. 17:58by its own work and uh you know that
  470. 18:01takes you out of the equation where now
  471. 18:04I can actually do something like so the
  472. 18:06very one of the very first use cases I
  473. 18:07had for verification was you know like
  474. 18:09the performance work that I was doing on
  475. 18:11cursors agent window and I want I wanted
  476. 18:14to get to a point where I could do
  477. 18:15something called hill climbing uh which
  478. 18:17is a term that I I think the labs uh
  479. 18:20talk about a lot which is this idea that
  480. 18:22you know you have some kind of rubric or
  481. 18:25a way to judge or score something And
  482. 18:28now because you have a loop, you can
  483. 18:30have an agent continually try to make
  484. 18:33improvements to that. Uh I think
  485. 18:35Carpathy, Andre Carpathy also famously
  486. 18:38released uh something called auto
  487. 18:40research that has a lot of these ideas.
  488. 18:43Um but yeah, verification I would say is
  489. 18:46probably the most important skill in PAC
  490. 18:49uh and many [clears throat] other you
  491. 18:51know tool sets. Uh and I think it's the
  492. 18:55most important thing to focus on. So a
  493. 18:57lot of the a lot of I spent a lot of
  494. 19:00time actually you know tuning the
  495. 19:02verification the creative verification
  496. 19:05skill um and also internally the the we
  497. 19:09have so many verification skills now
  498. 19:10like every app that cursor has or
  499. 19:12spaceexai has has a uh verification
  500. 19:16skill that is automaintained as well
  501. 19:19[clears throat]
  502. 19:19>> uh and it's become critical
  503. 19:21infrastructure for our team because
  504. 19:22everybody uses it
  505. 19:24>> and you went pretty far with that too
  506. 19:26right like you had a um in your talk I
  507. 19:28saw that you actually built a custom CLI
  508. 19:30for that too. So what does that CLI do?
  509. 19:32Like how does it [clears throat] execute
  510. 19:33things and why did you I mean that's
  511. 19:35proof of how deep you're going right of
  512. 19:37how much you're pushing that.
  513. 19:39>> Yeah. So this is actually a tip I
  514. 19:42learned early on where
  515. 19:45um I guess you know back in January or
  516. 19:49or late last year the thing that people
  517. 19:51were concerned about was context window,
  518. 19:54right? That was the big the big topic at
  519. 19:56the time was how do I you know manage
  520. 19:58the context window because you know
  521. 20:00compaction summarization wasn't really
  522. 20:02that good yet [snorts] and people were
  523. 20:05always people had this there was almost
  524. 20:07this meme in the community that you know
  525. 20:09once your agent summarized or compacted
  526. 20:12once it would become sort of stupid
  527. 20:15right for the rest of your session. So
  528. 20:17there was a lot of thinking around like
  529. 20:20you know being very efficient with your
  530. 20:22context usage and so that was actually
  531. 20:25the inspiration for some of the uh the
  532. 20:28CLI work inside of the verification
  533. 20:31skills. I guess now it's less so about
  534. 20:34context because uh you know agents are
  535. 20:38much better or harnesses have gotten a
  536. 20:40lot better with summarization.
  537. 20:42Um, I still think there's some benefits
  538. 20:45to, you know, uh, having a clean context
  539. 20:49window. Uh, so the CLI is really just
  540. 20:51more of a way for me to take the
  541. 20:53deterministic parts of what the skill
  542. 20:56does and encode that into a script or
  543. 20:58CLI to reduce to kind of take away the
  544. 21:03judgment that would otherwise
  545. 21:05unnecessarily be used because with
  546. 21:08judgment so I also think of you know
  547. 21:10agents and skills in sort of like it's
  548. 21:13like a gradient
  549. 21:15you have some parts of the work that are
  550. 21:18entirely judge measurement based right
  551. 21:20you know something that requires thought
  552. 21:22you know putting together multiple
  553. 21:24pieces of context thinking um and then
  554. 21:27you have the more deterministic parts
  555. 21:28like I don't know if you wanted to uh
  556. 21:31refactor some code right from one
  557. 21:34pattern to another that's very
  558. 21:35mechanical right you don't you don't
  559. 21:37need an agent to think about it and come
  560. 21:40up with it in a novel way each time
  561. 21:43right and so that that was really the
  562. 21:45inspiration for the CLI and you'll see
  563. 21:47this in a lot of the other skills that I
  564. 21:48built is like I try to extract out the
  565. 21:52deterministic parts and turn that into
  566. 21:54code and just leave only the parts that
  567. 21:56actually require judgment to the agent.
  568. 22:00So in a way I think of the seal as kind
  569. 22:01of like your wrapper, right? It's a
  570. 22:03wrapper with some light instructions
  571. 22:04around how to use these custom tools
  572. 22:06that are inside of the skill. Um but
  573. 22:10yeah, I don't think the CL is really
  574. 22:11that interesting in its own really. It's
  575. 22:13not like a novel piece of software. It's
  576. 22:16just something that interacts with like
  577. 22:17Playright and the Chrome DevTools
  578. 22:20protocol and calls a bunch of APIs. It's
  579. 22:22like it's just a bunch of glue.
  580. 22:24>> No, it's fascinating because it's a way
  581. 22:25of hiding information from the skill,
  582. 22:27right? It's a way of conserving the
  583. 22:28skill, keeping the skill quite small, I
  584. 22:31imagine, and then you're able to
  585. 22:32delegate more of the complicated
  586. 22:33deterministic stuff into a script within
  587. 22:36the skill. So it's almost you're
  588. 22:38compressing information and making the
  589. 22:41agent do more consistent things more
  590. 22:44consistently.
  591. 22:45>> Yeah,
  592. 22:45>> that's fascinating.
  593. 22:47And it it [clears throat] also helps I
  594. 22:49guess if you care about context window
  595. 22:50it it does help because now the agent
  596. 22:53doesn't need to uh you know re reinvent
  597. 22:56uh things cuz uh one thing I had noticed
  598. 22:59early on when we didn't have a CLI
  599. 23:02was that uh well the the agent would try
  600. 23:05to verify it work but it would basically
  601. 23:07rebuild the world each time and then
  602. 23:09every agent did it differently and I was
  603. 23:11starting to notice like that's very
  604. 23:12inefficient right I was wasting it it
  605. 23:14was actually not just about context
  606. 23:16usage but also speed, right? Like
  607. 23:18because now an agent had to actually go
  608. 23:19off and write the scripts or the CLI and
  609. 23:22test it and you know and it doesn't work
  610. 23:24and the last agent did it and it worked
  611. 23:26but it discarded it. So it was just very
  612. 23:28obvious at that point like I should just
  613. 23:30turn this into a CLI and put that inside
  614. 23:32of the skill uh so that every agent that
  615. 23:35uses it now benefits from that same
  616. 23:38piece. Um but I I also think like you
  617. 23:42know it's a good push for people to
  618. 23:43think about is how much of your skills
  619. 23:46and rules could actually be
  620. 23:48deterministic.
  621. 23:50Um that's like another core thing or or
  622. 23:53one of my core principles that I like to
  623. 23:55think about is yeah how do I uh make
  624. 23:59very efficient use of determinism and
  625. 24:02non-determinism
  626. 24:04and you know let Asians shine at the
  627. 24:07non-deterministic parts right because
  628. 24:09that's what they're trained to do. Um
  629. 24:11and the other parts which are much more
  630. 24:14mechanical or you know straightforward
  631. 24:16can be just pure determinism. Um, and
  632. 24:19you'll see this as well for things like
  633. 24:21doing migrations. Um, which is another
  634. 24:24big thing that I've I've talked about
  635. 24:26is, you know, going from one technology
  636. 24:29to another, especially one that is
  637. 24:32better for agents, right? And a lot of
  638. 24:34how you can do that migration is, I
  639. 24:36think, through things like scripts and
  640. 24:38CLIs, like the deterministic parts like
  641. 24:40code mods, you know, like crawling the
  642. 24:44abstract syntax tree and transforming
  643. 24:46code literally mechanically, right? like
  644. 24:48a script does it for you instead of the
  645. 24:50agent.
  646. 24:52>> Totally makes sense. I I mean I think
  647. 24:55what there's another thing there which
  648. 24:57is you're taking stuff away from the
  649. 25:00agent and you're kind of putting it in
  650. 25:03the environment too a little bit which
  651. 25:05is let's say you have a a thing that you
  652. 25:08notice the agent always gets wrong. You
  653. 25:10want to make that um just impossible
  654. 25:13within the environment. And that sort of
  655. 25:15comes down to code quality as well. I
  656. 25:18mean, I talk about a lot like having a
  657. 25:22what a good codebase means, right? What
  658. 25:24is a good codebase? And there's a
  659. 25:26definition I like which is a a good
  660. 25:28codebase is a codebase that's easy to
  661. 25:30make changes in, right? Easy to um
  662. 25:34change stuff without things screwing up.
  663. 25:36And that means that you have a lot of
  664. 25:37guard rails that you have a lot of um
  665. 25:40the agent or the human is constrained to
  666. 25:42very narrow paths. And that's again
  667. 25:44something you talk about in your talk.
  668. 25:46>> And you talk about this not only on the
  669. 25:48kind of sort of automated checks side of
  670. 25:52things. So linting and type checking
  671. 25:53blah blah blah but also in the way you
  672. 25:55design abstractions. And you guys even I
  673. 25:58think built a framework uh for your
  674. 25:59agent to work in too.
  675. 26:01>> I think what I'd love to hear is you
  676. 26:04obviously think of that as very
  677. 26:06important, right? And that's how
  678. 26:08important is that compared to other
  679. 26:10things you could be doing like building
  680. 26:11features or shipping work.
  681. 26:14Yeah, I think that's a um I almost feel
  682. 26:17like the new job of the engineer is
  683. 26:19really to to spend time on the
  684. 26:21environment. Um I almost actually wrote
  685. 26:23a tweet about this yesterday, but I but
  686. 26:25I didn't. But I think that
  687. 26:29I think that you you know if you if you
  688. 26:31haven't really spent time, you know,
  689. 26:33building trust in your agents and
  690. 26:34building skills and tools, you can get
  691. 26:37stuck in this mode where you're very low
  692. 26:39on that trust ladder, right? you don't
  693. 26:41have a lot of trust in your agents work.
  694. 26:43And so the only way to cope in that when
  695. 26:47you're in that situation is just to kind
  696. 26:48of lock in and micromanage your agents.
  697. 26:51And that's very time consuming. And when
  698. 26:53you're stuck in that mode, you don't
  699. 26:55really have the luxury to think about,
  700. 26:58you know,
  701. 27:00uh higher level things like like making
  702. 27:03yourself more productive. In the same
  703. 27:05way that uh I guess analogy would be
  704. 27:09like if you've never taken the time to
  705. 27:11learn like your tools right as a
  706. 27:13developer when you were writing code
  707. 27:14yourself and you know you've never heard
  708. 27:17of VS Code, you've never heard of Vim,
  709. 27:19you only knew about Notepad [laughter]
  710. 27:22uh and you had hadn't even heard about
  711. 27:23Git. That's sort of the analogy. It's
  712. 27:25like you you haven't spent the time
  713. 27:27sharpening your own knives, right? And
  714. 27:29so, of course, if you have a dull knife,
  715. 27:32then everything's going to take a long
  716. 27:33time. Um, and you're going to be you're
  717. 27:36just going to be and and especially if
  718. 27:38you know deadlines are looming, then you
  719. 27:41don't have the now you're stuck in this
  720. 27:42rut, right? Where where you you you
  721. 27:45don't have sharp knives, you don't have
  722. 27:47good tools, but you're under all this
  723. 27:49pressure to ship, right? And so, all you
  724. 27:51can do is just focus on that. But I do
  725. 27:54think that, you know, if you can find
  726. 27:55yourself the time to actually spend time
  727. 27:58thinking about your setup, it's again
  728. 28:00going back to the cooking, you know,
  729. 28:02analogy,
  730. 28:04it's like uh, you know, if you, for
  731. 28:06example, if if cutting cutting cutting
  732. 28:09the garlic is like super slow, right?
  733. 28:11There are garlic mashers, right? You can
  734. 28:13you can buy and you put it in the thing
  735. 28:14and you like squeeze it out, right? It's
  736. 28:17super fast. Uh, machines and tools were
  737. 28:20invented for a reason, right? And so if
  738. 28:22you're operating a Michelin kitchen and
  739. 28:25your your your cooks had no tools, then
  740. 28:27of course everything's going to be
  741. 28:29extremely inefficient, very very, you
  742. 28:31know, every every every cook is going to
  743. 28:33make something up of their own. So I
  744. 28:35think the tools and the determinism to
  745. 28:37me are you know taking that part away
  746. 28:41and and just like you said about
  747. 28:42constraints as well. It's the
  748. 28:44constraints are are to me as well like
  749. 28:48uh actually a slight tangent on that is
  750. 28:50uh I think we should talk about
  751. 28:53TypeScript cuz like we we actually both
  752. 28:55share like a background in Typescript
  753. 28:57where you know you obviously have done a
  754. 28:58lot of work with TypeScript and total
  755. 29:00TypeScript and you know you're a leader
  756. 29:02in that space and I uh had adopted
  757. 29:05TypeScript pretty early and I had given
  758. 29:07like a talk or two at Typescript conf uh
  759. 29:10many years ago and so one of the the
  760. 29:13talk that I did actually was about type
  761. 29:15systems and constraining the
  762. 29:18constraining types. Like one of my most
  763. 29:20favorite things about Typescript is
  764. 29:22actually type narrowing, right? This
  765. 29:23idea that you go from a very broad type,
  766. 29:26right? That could be anything and then
  767. 29:28you through type guards and you know
  768. 29:31type narrowing and you know runtime
  769. 29:33checks you can actually narrow the space
  770. 29:35and say like oh this isn't just a string
  771. 29:38this is a very special type of string.
  772. 29:40It's a constant, right? like I but I I
  773. 29:42determine that through the type system
  774. 29:45and in a way it's like uh there's a lot
  775. 29:47of parallels I think to that with
  776. 29:49constraints in your codebase where is it
  777. 29:52you're you're constraining the space
  778. 29:55right if you if you think about category
  779. 29:57theory as well you know you're
  780. 29:58constraining the the number of possible
  781. 30:01types right that can can exist
  782. 30:05and you're saying there's only one type
  783. 30:07right and for for us like that framework
  784. 30:10that I'm called Dune. Uh it's not an
  785. 30:13open source framework. It's the the way
  786. 30:15I describe it to people. It's it's kind
  787. 30:17of like a internal Nex.js for our
  788. 30:20Electron apps. Uh but it comes with a
  789. 30:23lot of really really restrictive lit
  790. 30:25rules and the codebase is designed in a
  791. 30:28way that there's really only one way to
  792. 30:30do something. So we make use a we make
  793. 30:32use of a lot of conventional
  794. 30:35patterns. So like features all go into a
  795. 30:38specific directory. Well, every feature
  796. 30:41has its own directory. As an example,
  797. 30:43you know, there's like a a thing that
  798. 30:44discovers features like through a
  799. 30:46registry and like crawling the codebase
  800. 30:48and stuff like that. But this
  801. 30:51conventional pattern and the lint rules
  802. 30:55make for an environment where it's
  803. 30:57actually very hard to write bad code.
  804. 31:01And that sort of frees up the it both
  805. 31:04frees up your own mental uh you know
  806. 31:09capacity as well as the agent sort of
  807. 31:11doesn't have to think about that anymore
  808. 31:14where it's just like oh there's only
  809. 31:15there's I should just if I want to add a
  810. 31:17new feature it just goes in the feature
  811. 31:18the new feature directory and all the
  812. 31:20code goes in there and I'm not going to
  813. 31:22append to a god file right that was
  814. 31:25really actually the inspiration for
  815. 31:26those feature directories is the very
  816. 31:29first couple of versions of Grockbot
  817. 31:32were composed of like eight god files
  818. 31:36which were like at least 10,000 lines
  819. 31:38long if not longer and so I kind of had
  820. 31:40to break it up into smaller pieces.
  821. 31:43Uh but it was just observing you know
  822. 31:46actually that's another important part
  823. 31:47is observing how agents fail and then
  824. 31:50every time you see a mistake every time
  825. 31:52you see something that could be done
  826. 31:54better you think you step back and think
  827. 31:57how do I turn this into a lint rule? How
  828. 31:58do I make it so that the code base makes
  829. 32:01this impossible? Right? And it comes
  830. 32:03back to me for my you know my background
  831. 32:05learning Typescript and types uh type
  832. 32:08systems is how do I constrain the space
  833. 32:12so that you know I know precisely what
  834. 32:15I'm working with and I think yeah
  835. 32:16there's a lot of parallels there.
  836. 32:18>> Totally makes sense. And don't I mean
  837. 32:21it's funny that you mentioned TypeScript
  838. 32:22and Goth files in the same sentence
  839. 32:23because Typescript famously has a
  840. 32:2525,000line type uh file.
  841. 32:29>> [laughter]
  842. 32:30>> Although I don't know if they've
  843. 32:30rewritten that and go as they probably
  844. 32:32have, haven't they? Um,
  845. 32:34okay. So, environment is important. You
  846. 32:38should watch your agent like a hawk to
  847. 32:39make sure that any mistakes it makes.
  848. 32:42You turn them into things in the
  849. 32:44environment. And the benefit of the
  850. 32:45environment is you're not overloading
  851. 32:47your agent, right, in terms of rules, in
  852. 32:49terms of things it has to remember. It's
  853. 32:51just in the environment. And so it
  854. 32:53stumbles into the rules and exactly um
  855. 32:56you know bounces off them and hits them
  856. 32:57at the right moment.
  857. 32:59>> So okay, we still haven't talked about
  858. 33:01the 2,500 PRs. Where do those come from?
  859. 33:03Like how do you you've built your trust
  860. 33:06ladder, you've worked on your
  861. 33:07environment, and you understand, okay,
  862. 33:09um I now want to scale up. So what are
  863. 33:13the mechanics of that scaling? Are you
  864. 33:16um initiating 2500
  865. 33:19like chats per month? That can't be
  866. 33:21right. So there must be are there any
  867. 33:23kind of automated triggers that trigger
  868. 33:26stuff in your repo? Like how do you get
  869. 33:28the software factory kind of triggering
  870. 33:30work by itself?
  871. 33:32>> Right.
  872. 33:34Um I'll definitely say that the
  873. 33:36prerequisite to you know something like
  874. 33:40a very high volume of of pull requests
  875. 33:43um is the environment. you know, the the
  876. 33:46stuff we just talked about where I
  877. 33:48definitely would not have been able to
  878. 33:49do this if I had not spent the time, you
  879. 33:52know, thinking about the kitchen, right,
  880. 33:53and the knives and the tools for my
  881. 33:55Asians. And so, in a way, I I think of
  882. 33:58this as
  883. 34:00I've spent the time building one kitchen
  884. 34:02and one restaurant. And now I I'm in a
  885. 34:05position where I don't actually have to
  886. 34:07be there anymore because the
  887. 34:10environment, you know, that the same
  888. 34:12analogy, right? It works really well.
  889. 34:14[laughter] Yeah. You open chain of
  890. 34:15restaurants, right? That's
  891. 34:16>> Yeah. Exactly. Yeah. Exactly. It's like
  892. 34:19you're Gordon Ramsay and you know, you
  893. 34:21you've taught your executive chef like
  894. 34:23all the tricks of the of coming up with
  895. 34:25great menu. Uh and like the kitchen is
  896. 34:28set up really well. Everything's just
  897. 34:30perfect and you're now in a position
  898. 34:32where you can open your second your
  899. 34:33third restaurant. And [snorts]
  900. 34:36I guess I I sort of see each project
  901. 34:38that I work on, like each big chat is
  902. 34:41sort of like a restaurant, right? and
  903. 34:43and I'm I'm I have multiple of them
  904. 34:45operating at the same time and I'm sort
  905. 34:48of like helicoptering between them
  906. 34:50sometimes some more than others
  907. 34:52depending on how in the loop I am but
  908. 34:55yeah definitely I think there's there
  909. 34:56there are external triggers and context
  910. 35:00that those projects don't have that
  911. 35:05for a long time I was the proxy for that
  912. 35:08so uh the best example I have is like
  913. 35:10you know you have a project that's
  914. 35:11working on a feature
  915. 35:13uh or you're trying to fix a bug and
  916. 35:15you're getting bug reports, but the bug
  917. 35:17reports are going to things like Slack
  918. 35:19or linear or X, right? And these are
  919. 35:24external systems that aren't connected
  920. 35:26to your inner loop. So, I like to talk
  921. 35:28about this outer loop and the inner
  922. 35:30loop.
  923. 35:32Uh I don't know if I'm using the
  924. 35:33definition correctly but to me my inner
  925. 35:35loop is like basically my engineers my
  926. 35:38agent engineers working on the code to
  927. 35:41an building towards an intent or
  928. 35:45snapshot of my intent right and the
  929. 35:48thing about that is that the snapshot
  930. 35:50can go stale right new information comes
  931. 35:52to light that I then have to be the
  932. 35:54proxy of and you know transfer that
  933. 35:57context to my agent so you know if you
  934. 35:59if you don't have these triggers pulling
  935. 36:02information back into the interloop,
  936. 36:04then you sort of have to play that role
  937. 36:07where you're you're off, you know, in
  938. 36:09Slack or X or or whatever and you're
  939. 36:12gathering context, right? You're getting
  940. 36:14context about bug reports, about feature
  941. 36:16requests, about, you know, something
  942. 36:19someone said about, you know, our
  943. 36:20backend infrastructure has some
  944. 36:22limitation, you know, all that
  945. 36:24information, you have to f that across
  946. 36:27to your agent. So that's where I think
  947. 36:29like tools like Grogbot are really good
  948. 36:32because they help you automate the outer
  949. 36:35loop as well. And when you connect those
  950. 36:37two loops, it's very very powerful
  951. 36:39because now all of a sudden your agents
  952. 36:41have the ability to get context for this
  953. 36:44for themselves, right? If for example
  954. 36:48uh you know either through just as a
  955. 36:51simple example like maybe you have the
  956. 36:52Slack MCP, right? Or you have uh your
  957. 36:56own harness, right, that you've built a
  958. 36:58Slack subscription into for a particular
  959. 37:00Slack channel. Now all of a sudden you
  960. 37:03can tell your agents, okay, subscribe to
  961. 37:05the Slack channel. Every time there's a
  962. 37:08uh, you know, bug report about
  963. 37:11something, go off and triage that thing,
  964. 37:14right? Go reproduce the issue, right?
  965. 37:16Using the verification skills that we've
  966. 37:18already spent time building and all of
  967. 37:20those other skills that we've set up so
  968. 37:22that I have a lot of trust, right? I
  969. 37:24have a lot of trust that these agents
  970. 37:25can actually go off and understand the
  971. 37:28bug, you know, uh verify that the bug
  972. 37:31actually still exists on main and it
  973. 37:34wasn't
  974. 37:35something about you maybe the users
  975. 37:38setup or their data or maybe I don't
  976. 37:40know they didn't install a dependency or
  977. 37:42something like that like uh basically I
  978. 37:45think uh creating that yeah creating
  979. 37:49those two loops and connecting them is
  980. 37:52really a very important part of the job
  981. 37:53these days. Um, especially if you are
  982. 37:56thinking about how to scale yourself.
  983. 37:58So, a big theme here is really just like
  984. 38:01always thinking about like what where am
  985. 38:04I the bottleneck in this process? Why do
  986. 38:07my agents need me, you know, to answer
  987. 38:09this question? I I always like to think
  988. 38:11about that. And so I try to think about
  989. 38:13how do I actually get the agent to
  990. 38:15answer its own question, right? But not
  991. 38:17by hallucinating, not by guessing, but
  992. 38:19actually real data. And you know, a lot
  993. 38:22of people talk about this idea of a
  994. 38:23company brain, right, or a context
  995. 38:25graph. I feel like those terms are
  996. 38:27unnecessarily
  997. 38:29complex uh or even abstract. To me, it's
  998. 38:33just about um how do I take information
  999. 38:37that my agent needs that I would
  1000. 38:39otherwise have to go and pass it myself
  1001. 38:42and just teach it how to do it, right?
  1002. 38:44And that removes me from the equation.
  1003. 38:47And so how I arrive at 2,000 or however
  1004. 38:50many PRs is the fact that I have all
  1005. 38:53these loops set up, right? And so uh it
  1006. 38:57allows me to open chain restaurants,
  1007. 39:00right? I can I can really parallels
  1008. 39:02myself. So yeah, I'm not sitting there
  1009. 39:04creating 2,500 chats, right? Of course,
  1010. 39:06it's really like these projects are um
  1011. 39:10actually cursor has a new feature called
  1012. 39:12projects which are these like
  1013. 39:13coordinator agents. Um and so the
  1014. 39:17coordination co coordinator agents are
  1015. 39:19really good at sort of delegating and
  1016. 39:21not doing work of their own but they
  1017. 39:23manage and supervise like almost a list
  1018. 39:25of tasks and they spawn sub agents to go
  1019. 39:28and do them. And so I'm just constantly
  1020. 39:30feeding context or teaching the agents
  1021. 39:32how to get their own context and then
  1022. 39:33they're going off and doing the work for
  1023. 39:35me. Uh and really the big the last thing
  1024. 39:38I'll say to this is like the big unlock
  1025. 39:40for me for getting to 2,000 PRs is
  1026. 39:43starting from the question and working
  1027. 39:45backwards of how do I get to the point
  1028. 39:48where my agent can merge its own code?
  1029. 39:51Because
  1030. 39:53the obvious thing people ask me when
  1031. 39:55when they when I tell them, "Oh, I
  1032. 39:56shipped 2,000 and 2,500 pull requests
  1033. 39:59last month." They'll be like, "How did
  1034. 40:00you review that?" Right? That that's a
  1035. 40:02lot of PRs to review. Like your team
  1036. 40:04must hate you.
  1037. 40:05>> Do do you mind if we go there in a
  1038. 40:07second? Because a good question about
  1039. 40:09that.
  1040. 40:09>> Yeah. Yeah. Yeah.
  1041. 40:10>> I want to like this analogy is great. I
  1042. 40:12want to like deepen it a bit which is
  1043. 40:15before if you're like manually
  1044. 40:17initiating all those chats it's like
  1045. 40:18you're bringing the orders to your chefs
  1046. 40:20manually right whereas if you've got an
  1047. 40:22agent sort of like doing the expo then
  1048. 40:25you're able to sort of run it yourself
  1049. 40:26itself what is what does that concretely
  1050. 40:30look like then you've got these sort of
  1051. 40:31grock bots that are um subscribing to
  1052. 40:33channels pulling in Slack messages and
  1053. 40:36you it sounds like have a couple of
  1054. 40:37coordinator agents or like chief of
  1055. 40:39staff agents that like monitor that or
  1056. 40:43something like when you look at your
  1057. 40:45computer to manage your agents, what
  1058. 40:47does it look like?
  1059. 40:49>> Yeah. So, [clears throat] so uh this is
  1060. 40:52I guess somewhat confusing but we're
  1061. 40:53working on you know simplifying and
  1062. 40:55unifying but so uh there's graphbot uh
  1063. 40:59which or you know you can use other
  1064. 41:01tools of course as well but I I largely
  1065. 41:03think of these tools as like your outer
  1066. 41:05loop. These are tools like you know
  1067. 41:07Grabbot that have connectors right these
  1068. 41:09are connectors I guess they a lot of
  1069. 41:11people call them personal agents um but
  1070. 41:14they're connectors to things like your
  1071. 41:16email your calendar slack uh plaid I
  1072. 41:20don't know like all these different
  1073. 41:21services and they are a great source of
  1074. 41:24pulling context in to your work so the
  1075. 41:29same way that a human like you know if I
  1076. 41:31were if I was a manager and I was
  1077. 41:34leading a team of engineers years. Um,
  1078. 41:37you know, like when I used to work in
  1079. 41:38Netflix, one of the biggest things that
  1080. 41:40managers would talk about was this idea
  1081. 41:42of context not control, which funnily
  1082. 41:45enough, you know, has so much uh has so
  1083. 41:48much uh carry over to the agents world.
  1084. 41:51Uh, of you know, you you know, you you
  1085. 41:54of course can drive to an outcome you
  1086. 41:56want by control, right? Like by
  1087. 41:58micromanaging, but what you want is to
  1088. 42:00provide context instead, right? like
  1089. 42:02teach the agent, teach your engineers
  1090. 42:04how to be self-sufficient and then you
  1091. 42:06don't have to micromanage them.
  1092. 42:09>> Um, and so I see a lot of parallels
  1093. 42:11there. Uh, but yeah, graphbot. So,
  1094. 42:13concretely, I have some graph bots that
  1095. 42:16look at my Slack channels, look at my X,
  1096. 42:19uh, or my emails, uh, or linear, and
  1097. 42:23they're just constantly they have
  1098. 42:24routines that subscribe. So they're
  1099. 42:26constantly watching and I have I I'll
  1100. 42:29tell them things like you know uh I'll
  1101. 42:31watch for issues with uh bugs in the
  1102. 42:34graphbot desktop app as an example. Uh
  1103. 42:37and whenever you find that send it to my
  1104. 42:40cursor project. So one of the really
  1105. 42:42cool things about grabbot is it connects
  1106. 42:43to cursor. So cursor has uh like I I
  1107. 42:47just mentioned this new feature called
  1108. 42:49projects. And a project is really a uh
  1109. 42:52again like a you get a coordinator agent
  1110. 42:55that's in the cloud. It has its own
  1111. 42:57computer and all it really does is like
  1112. 42:59it's a manager of agents. It's like your
  1113. 43:01executive chef, right? Your your chief
  1114. 43:03of staff. It doesn't do the work itself.
  1115. 43:06It delegates and orchestrates and
  1116. 43:08manages the work of other sub agents to
  1117. 43:12you know that report to your chief your
  1118. 43:16chief uh of staff. And it basically is
  1119. 43:20responsible for driving the work forward
  1120. 43:22and managing things and uh passing
  1121. 43:25context to them.
  1122. 43:27>> So if you get a sudden burst of issues,
  1123. 43:28let's say you get 30 issues at once in
  1124. 43:30one payload or something or very quickly
  1125. 43:32the coordinator agent can figure it out
  1126. 43:34and delegate.
  1127. 43:34>> Yeah, exactly. It gets like uh you know
  1128. 43:3630 the 30 or so payloads and spawns a
  1129. 43:39sub agent or a single coordinator agent.
  1130. 43:41It can actually do a bunch of different
  1131. 43:42topologies of agents and it will sort of
  1132. 43:46figure out the best way to uh you know
  1133. 43:48efficiently distribute the tasks to your
  1134. 43:52team of agents. Um so I use uh cursor
  1135. 43:57projects a lot um and I also use grapot
  1136. 44:01a lot and cursor projects are my inner
  1137. 44:03loop and grabbot is my outer loop.
  1138. 44:05Grabbot takes all the context, external
  1139. 44:08context, gives it to the projects
  1140. 44:11because it can actually just send
  1141. 44:12messages to those projects, right? You
  1142. 44:14don't even have to open cursor. You can
  1143. 44:16just tell your grabbot, okay, create a
  1144. 44:18project, right, for these series of
  1145. 44:20tasks. They're all related, right? Maybe
  1146. 44:23as an example, you know, you've had a uh
  1147. 44:26[clears throat] a big burst of issues
  1148. 44:28that are all about performance, right?
  1149. 44:29Your app is slow uh and they're all
  1150. 44:33connected, right? Maybe some of them
  1151. 44:34even have a similar fix, right? But and
  1152. 44:38you can certainly go off and just spawn
  1153. 44:40one agent per task, but then you've lost
  1154. 44:42that sort of thread between them, right?
  1155. 44:45And and you may duplicate work or you
  1156. 44:47may not really think about the higher
  1157. 44:48level problem. You know, sometimes when
  1158. 44:50you you you solve bugs, you know, it
  1159. 44:52helps to have multiple bug reports that
  1160. 44:54are are slightly different because it
  1161. 44:56helps you really, you know, zoom out and
  1162. 44:58see actually, you know, the problem when
  1163. 44:59I looked at this one report, I thought
  1164. 45:01the bug was here, but actually when when
  1165. 45:03I see the other multitude of bugs is
  1166. 45:05actually up here,
  1167. 45:07>> right?
  1168. 45:07>> Yeah. Got you. So that that's why you
  1169. 45:09have so many agents in that loop then,
  1170. 45:10right? Because it's not just you have um
  1171. 45:13like you have a bug report comes in, you
  1172. 45:15spawn a single agent to look at that bug
  1173. 45:17report. that a that single agent will be
  1174. 45:19duplicating work with other um other
  1175. 45:21agents, right? Because if there are
  1176. 45:23multiple bug reports coming in through
  1177. 45:24the same thing, that can be duplicated
  1178. 45:25work.
  1179. 45:26>> Yeah.
  1180. 45:26>> Fascinating. [snorts]
  1181. 45:28>> That's really fascinating. Okay. And so
  1182. 45:30this just this endless series of
  1183. 45:32triggers um coming from real users
  1184. 45:34reporting real reports um builds up this
  1185. 45:38sort of and accelerates the factory sort
  1186. 45:40of adds more orders in. Other than bug
  1187. 45:43reports, are there any other sources
  1188. 45:44that you use for like um accelerating
  1189. 45:48for pushing these PRs?
  1190. 45:51>> Uh well, funnily enough, it's some of it
  1191. 45:54comes from uh reading the code, too. So,
  1192. 45:57I guess I have sort of uh well, so to
  1193. 46:01clarify that, you know, the 2,500 PRs,
  1194. 46:03they're not obviously like 2,500
  1195. 46:06features, right? they are a lot of the
  1196. 46:08work actually is spent on gardening like
  1197. 46:12another term that I really love. Uh
  1198. 46:15where
  1199. 46:17so I guess this is more important when
  1200. 46:19you have a big team of engineers human
  1201. 46:21engineers that you work with where and
  1202. 46:25also this goes back a little bit to what
  1203. 46:26I was talking about with the
  1204. 46:27environment. You know, setting up a
  1205. 46:28really good environment that doesn't
  1206. 46:29just help you and your agents, but
  1207. 46:31everybody on your team, right? Think of
  1208. 46:33a new hire who doesn't have a lot of
  1209. 46:34context on all of your engineering
  1210. 46:36practices joining your your team. And if
  1211. 46:39you have a really good environment, they
  1212. 46:41can be productive from day one, right?
  1213. 46:43They can they don't have to like, you
  1214. 46:44know, make open a bunch of lowquality
  1215. 46:46PRs. They can start, you know, they can
  1216. 46:49start just turning out really good code.
  1217. 46:52Um,
  1218. 46:54and uh, yeah, I think I I sort of lost
  1219. 46:57my train of thought.
  1220. 46:58>> I've got a I've got a followup, which is
  1221. 47:00what's what are the mechanics of like
  1222. 47:02triggering
  1223. 47:03>> like how when when do you trigger a a
  1224. 47:06agent
  1225. 47:08to go and look at the code, right?
  1226. 47:10Because some people might say, "Oh,
  1227. 47:11let's just do that every hour or
  1228. 47:13something or like on a chron job or
  1229. 47:15what's
  1230. 47:16>> Oh, yeah. Yeah. Yeah. Yeah. Uh I saw
  1231. 47:19some of your recent tweets as well about
  1232. 47:20like you know the some of the tweets
  1233. 47:22you've been doing which are great for
  1234. 47:24setting up your routines. Uh I have some
  1235. 47:27routines like that as well. Um so uh one
  1236. 47:32of them is uh
  1237. 47:35like looking through just another simple
  1238. 47:38example is you know [clears throat]
  1239. 47:40React has a lot of foot guns. Um, so,
  1240. 47:43uh, as as I'm sure you're aware. And so
  1241. 47:45I have an agent that's just constantly
  1242. 47:47looking for band patterns. And the
  1243. 47:50interesting thing about that one is that
  1244. 47:52I don't actually tell it to fix the
  1245. 47:53issue first. I tell it to append it to a
  1246. 47:55document. And then every couple of days
  1247. 47:58I look at it and I see actually these
  1248. 48:00are all the same thing, you know, and so
  1249. 48:02that gives me, you know, you almost want
  1250. 48:04like a buffer, a queue. Sometimes that's
  1251. 48:07actually more effective than just
  1252. 48:09spawning off a couple of like a lot of
  1253. 48:10sub agents to fix every single thing
  1254. 48:13because when you are [clears throat] in
  1255. 48:15kind of pure execution mode and just
  1256. 48:17trying to like you know f uh you know
  1257. 48:20execute on the orders that are coming in
  1258. 48:21very fast you sometimes miss the big
  1259. 48:24picture. So sometimes having a buffer
  1260. 48:26forces you to think about the big
  1261. 48:28picture because you you you have these
  1262. 48:31artifacts and things that you can look
  1263. 48:33at as a human um and sort of use your
  1264. 48:36own human judgment to or I guess you can
  1265. 48:39use an agent to do that as well. But you
  1266. 48:41give the [clears throat] agent and
  1267. 48:42yourself a way to identify patterns,
  1268. 48:46right, that you might otherwise miss if
  1269. 48:48you're just only solving each bug at a
  1270. 48:51time. And that's also really the benefit
  1271. 48:53of having something like a chief of
  1272. 48:54staff agent is uh it can see the forest
  1273. 48:59right uh in addition to actually doing
  1274. 49:02the execution.
  1275. 49:04>> Fascinating. That's I mean my brain is
  1276. 49:07exploding a bit there with the sort of
  1277. 49:08chief of staff at the software factory.
  1278. 49:10I might have to change some of the
  1279. 49:12course that I'm filming next week.
  1280. 49:13That's [laughter]
  1281. 49:15>> uh all right. Let's talk about let's
  1282. 49:16talk about review, right? because this
  1283. 49:18is the reply that you get, you know, is
  1284. 49:20>> did you read did you taste all 2500 of
  1285. 49:24those dishes as they swept past you?
  1286. 49:27>> And I assume the answer is a variety is
  1287. 49:30a version of no.
  1288. 49:34>> Yeah, I think you you you don't want to
  1289. 49:36be in a position where you're not
  1290. 49:37tasting your food ever again. Uh but you
  1291. 49:40also, you know, for scale, you cannot be
  1292. 49:42tasting every single dish that comes out
  1293. 49:44of your kitchen, especially if you have
  1294. 49:46multiple restaurants. So it becomes more
  1295. 49:48about sampling right and thinking about
  1296. 49:50the processes in the same way that you
  1297. 49:53know if I guess maybe this is where the
  1298. 49:55the the factory analogy is a bit more
  1299. 49:58apt is you know as a quality supervisor
  1300. 50:01on a factory you you can't look at every
  1301. 50:04single item you sample right you take
  1302. 50:06you you you go in there every day and
  1303. 50:09you look at the quality of the pull
  1304. 50:11requests you look at the code that the
  1305. 50:12agents are writing and you scrutinize it
  1306. 50:16very rig rigorously and you think about
  1307. 50:19all the inefficiencies, the bad patterns
  1308. 50:21that the agents are doing and then you
  1309. 50:24think about how to course correct the
  1310. 50:26environment, right? Not not that single
  1311. 50:28agent. Uh because if maybe if it if it
  1312. 50:33was a one-off incident, it's fine. You
  1313. 50:35know that maybe there's nothing to fix
  1314. 50:36there. But if you actually notice that
  1315. 50:39multiple agents are are having the same
  1316. 50:41issue, right? They're taking the same
  1317. 50:43shortcut. they're they're propagating
  1318. 50:45the same workaround everywhere. Uh
  1319. 50:48that's a sign that you should go off and
  1320. 50:50think about how to uh amend your kitchen
  1321. 50:53or your factory, right? Like thinking
  1322. 50:55about your skills, your constraints,
  1323. 50:57your lints, your type systems um and
  1324. 51:02setting or adjusting it so that that
  1325. 51:04problem doesn't happen again. And when
  1326. 51:06you do that enough times, then you get
  1327. 51:08to a place where the codebase is again
  1328. 51:10like the environment is so constrained
  1329. 51:13and so it guides you so well that you
  1330. 51:16can just you can just step away, right?
  1331. 51:19That's the dream. And I'll I'll
  1332. 51:20definitely say it um it's very hard to
  1333. 51:23get to this point. I don't want to sell
  1334. 51:26this as like, you know, something that
  1335. 51:27you can just do easily by using PAC.
  1336. 51:30Like it takes a lot of time and effort
  1337. 51:32to think about your code and where you
  1338. 51:35see your agents failing and thinking
  1339. 51:38very thoughtfully, intentionally and
  1340. 51:40setting up guard rails and constraints
  1341. 51:43so that they do the right thing by
  1342. 51:45default.
  1343. 51:47>> And you're not [clears throat] like if
  1344. 51:49to go back to the software factory
  1345. 51:51analogy, this isn't a dark factory,
  1346. 51:53right? This the lights are on, right?
  1347. 51:55>> It kind of is. Yeah, actually.
  1348. 51:56>> Is it?
  1349. 51:57>> Yeah. Well, it's dark in the sense that
  1350. 51:59so um it's dark in the sense that well I
  1351. 52:02think if my agents are merging their own
  1352. 52:04pull requests it's sort of become dark
  1353. 52:07where I go to sleep my agents now work
  1354. 52:0924/7
  1355. 52:11uh I have I have the equivalent of like
  1356. 52:14more than 10 chiefs of staff right each
  1357. 52:16working on a different area like for
  1358. 52:18example I have one that's working on
  1359. 52:20performance of the Grockbot desktop app
  1360. 52:23I have one that's working on uh fixing
  1361. 52:25bugs that users report I have one that's
  1362. 52:28exploring rewriting it in a different
  1363. 52:31language just for fun, you know, like
  1364. 52:32what if what if, you know, just
  1365. 52:34reimagining what what it would be if it
  1366. 52:35was like a native app. It's just a toy.
  1367. 52:38Um, but the idea is like yeah, I uh I
  1368. 52:41when you spend the time setting up your
  1369. 52:43environment, I've gotten to a point
  1370. 52:45where I review the pull request after
  1371. 52:47it's landed, right? I I tell my agents
  1372. 52:50full autopilot is is something that you
  1373. 52:52can do in in PAC and that will trigger
  1374. 52:56off this very intense rigorous
  1375. 52:58verification loop where it will spawn a
  1376. 53:00bunch of verifier agents for every pull
  1377. 53:03request and it will fuzz right fuzzing
  1378. 53:06meaning that it will actually run the
  1379. 53:07application. It's going to click around
  1380. 53:09and try to use it like a real human.
  1381. 53:11look for regressions, look for bugs in
  1382. 53:13your implementation and um it will try
  1383. 53:17to find issues with the thing and then
  1384. 53:19it will fix it itself. It'll do that
  1385. 53:21again and eventually get the PR to a
  1386. 53:24state where it can land.
  1387. 53:26Uh so it does it does it is quite token
  1388. 53:29intensive. You can tune this of course.
  1389. 53:32Uh so you know instead of like 10
  1390. 53:34verifier agents you might do like one,
  1391. 53:37right? Or you just tell the agent to
  1392. 53:39verify it's done work. But yeah, the key
  1393. 53:41thing is the verification part is really
  1394. 53:44the key piece that gives me a lot of
  1395. 53:46confidence
  1396. 53:48that I guess verification plus the
  1397. 53:49environment, right? It's these the
  1398. 53:51combination of these two things that
  1399. 53:52allow me to step away and say agents go
  1400. 53:54off and merge your thing. I'll review it
  1401. 53:56in the morning by looking at my commit
  1402. 53:58history
  1403. 53:59>> and if I see problems, I go and course
  1404. 54:02correct,
  1405. 54:02>> right? And I'll go and revert or modify,
  1406. 54:06add new link rules and whatever. Um, so
  1407. 54:10it does it does take time to get to that
  1408. 54:12point, but once you get it, oh, it's so
  1409. 54:14it feels so magical. Uh, I I I tell
  1410. 54:17people like I'm sleeping so much better
  1411. 54:19now because, you know, it took it the
  1412. 54:22very first day I turned on the sort of
  1413. 54:24dark factory was very scary because I
  1414. 54:26was like, "Ooh, what if I call the SE,
  1415. 54:28right? What if I break something
  1416. 54:29overnight?"
  1417. 54:30>> Uh, and it took a lot of it took a lot
  1418. 54:34of uh bravery, I think, to do that, but
  1419. 54:37>> somehow I did it. And yeah, now I'm in a
  1420. 54:39place where my Asians are are merging
  1421. 54:42their own code while I sleep.
  1422. 54:43>> It sounds like
  1423. 54:44>> I think it's dark in that sense.
  1424. 54:46>> Yes, it's dark sometimes, right? You do
  1425. 54:48sometimes.
  1426. 54:48>> That's true. That's true.
  1427. 54:49>> Because I think of a dark factory is
  1428. 54:51like almost like if you take the
  1429. 54:53original definition of Kapathy's vibe
  1430. 54:55coding, right, which is the code almost
  1431. 54:57doesn't exist. You forget that code
  1432. 54:59might be a thing. I think your approach
  1433. 55:01is totally different from that, which is
  1434. 55:03that code and the environment is
  1435. 55:06essential. And if the code in the
  1436. 55:07environment are bad, then you will get
  1437. 55:09bad outputs. Garbage in, garbage out. So
  1438. 55:12I I I think this is a this is a
  1439. 55:15different thing. It's like, you know,
  1440. 55:18the I don't know, maybe there's a dimmer
  1441. 55:20switch or something, right? Like, you
  1442. 55:21know, some parts of dark, some parts
  1443. 55:23were light. This is why the maybe the
  1444. 55:24the kitchen is a better analogy
  1445. 55:27>> the restaurant because you know even as
  1446. 55:29a as a as a
  1447. 55:32as a [clears throat] restaurant
  1448. 55:33restaurant you still might go to your
  1449. 55:36restaurants every now and then to take
  1450. 55:37take a peek in taste the food right
  1451. 55:40>> uh I think that's
  1452. 55:41>> the idea of sampling instead of blocking
  1453. 55:42I think is really important
  1454. 55:44>> I think what would you say to people who
  1455. 55:46are in I guess you're obviously in a
  1456. 55:49pretty security conscious environment
  1457. 55:50where you're working very security
  1458. 55:52conscious
  1459. 55:53>> mh [clears throat] Um maybe there are
  1460. 55:56folks working in like um medical
  1461. 55:58applications or law or finance or
  1462. 56:01something. I think of the like some PRs
  1463. 56:04are kind of like two-way doors which is
  1464. 56:06you can merge it and then revert it,
  1465. 56:08right? It's cheap back through. But
  1466. 56:10there are some PRs that are one-way
  1467. 56:11doors, right? That will cause data loss
  1468. 56:13of some kind that will
  1469. 56:16>> do something that can't be easily walked
  1470. 56:18back. How do you deal with situations
  1471. 56:21where most of your PRs, let's say, are
  1472. 56:23one-way doors? Like, is this something
  1473. 56:25you just wouldn't recommend or like what
  1474. 56:27do you think?
  1475. 56:28>> Yeah, I think that's a really good
  1476. 56:30question. I think that
  1477. 56:32it all comes back to me to the quality
  1478. 56:34of the verification that you're able to
  1479. 56:37um get out of your agent. And I think
  1480. 56:41for domains where the work is
  1481. 56:44verifiable,
  1482. 56:46this is easier, right? and the the
  1483. 56:48oneway doors become two-way doors in a
  1484. 56:51sense.
  1485. 56:53But I guess I don't know if you're
  1486. 56:54working on something that is like
  1487. 56:56extremely
  1488. 56:57is very hard to verify programmatically
  1489. 57:00then I think yeah you're definitely in a
  1490. 57:02position where it's very hard to get to
  1491. 57:04that point. Um so I do think like yeah
  1492. 57:07verifiability of the domain is an
  1493. 57:10important aspect to be able to do this.
  1494. 57:13Um and software engineering is just one
  1495. 57:15of those things where it's quite
  1496. 57:16verifiable in in a lot of cases maybe
  1497. 57:19not totally um you know like other
  1498. 57:22domains like mathematics I think are
  1499. 57:24another example of not all of it of
  1500. 57:26course but some aspects of mathematics
  1501. 57:30can be verifiable
  1502. 57:32if you write a proof for example um and
  1503. 57:35so yeah I think it's a great question
  1504. 57:38that I don't really have the answer to
  1505. 57:40and I think that this is something the
  1506. 57:41industry and us as engineers will have
  1507. 57:43to figure out is you know my sort of uh
  1508. 57:48hope and prediction for the future is
  1509. 57:49that we'll see more and more interesting
  1510. 57:52new agentoriented programming languages
  1511. 57:56and one of the most fascinating ones
  1512. 57:57that I've seen so far is this one called
  1513. 57:59bend bend d bend um and that language is
  1514. 58:04one where it kind of marries programming
  1515. 58:08with proofs right there used to be a
  1516. 58:12time, you know, where you actually had
  1517. 58:14to write your proofs in a different
  1518. 58:16language. And proofs, by the way, for
  1519. 58:18those uh who who aren't familiar is this
  1520. 58:20idea of uh that you can sort of formally
  1521. 58:24verify that some code is correct
  1522. 58:26mathematically, right? Especially if
  1523. 58:29you've written your code in a very
  1524. 58:31functional programming way.
  1525. 58:34uh but for the longest time you had to
  1526. 58:36do that in a separate language like lean
  1527. 58:38or tla+ or uh I'm blanking on some of
  1528. 58:42the other other examples but uh like
  1529. 58:45languages like that where you would
  1530. 58:46construct the mathematical proof and
  1531. 58:48then use a solver essentially to det
  1532. 58:52that that you've covered all the cases
  1533. 58:54you don't have like a race condition or
  1534. 58:56or whatever.
  1535. 58:58So yeah, I think
  1536. 59:02trying to sum up the question, I think
  1537. 59:03yeah, if you are in a position where you
  1538. 59:05can figure out how your agents can truly
  1539. 59:07verify the work in a way that gives you
  1540. 59:10confidence, you can actually, you know,
  1541. 59:13uh have the PRs merge cuz if it
  1542. 59:16compiles, right, if it if the proofs
  1543. 59:18show you that it's correct, then why
  1544. 59:21wouldn't you just merge it? Um but of
  1545. 59:24course, yeah, not all the means are
  1546. 59:26verifiable. Yeah, it's a tough one. Um,
  1547. 59:31okay. I think we've got to think about
  1548. 59:34wrapping up because we are nearly on the
  1549. 59:35hour. Have you Have you got something
  1550. 59:36after this? I mean, I've got something
  1551. 59:38before I give my son dinner, but
  1552. 59:40>> I I can go a bit longer after you.
  1553. 59:42>> Okay, let's let's go five minutes longer
  1554. 59:43then. Um I think I just want to have one
  1555. 59:46more question which is
  1556. 59:49I think I want to ask how you see Pstack
  1557. 59:55and how you see skills in general like
  1558. 59:57in terms of we talked about this before
  1559. 1:00:00we went on air which is like people
  1560. 1:00:02think of as like my skills versus your
  1561. 1:00:05skills and how do you combine frameworks
  1562. 1:00:08together? How do you use Pstack with my
  1563. 1:00:11stuff? like what should you take from
  1564. 1:00:13each one and because I think I see
  1565. 1:00:16skills as sort of just derived from
  1566. 1:00:19process basically like they're just
  1567. 1:00:21processes turned into words and I would
  1568. 1:00:25love to know
  1569. 1:00:27how you recommend people take Pstack and
  1570. 1:00:29take my stuff as well and turn it into
  1571. 1:00:31their own processes.
  1572. 1:00:35I think you shared a tip actually today
  1573. 1:00:36that I thought was actually very
  1574. 1:00:38relevant, which is this idea that you go
  1575. 1:00:40off and look at your previous
  1576. 1:00:42transcripts, right? And you sort of mine
  1577. 1:00:44for information of, you know, your own
  1578. 1:00:48look through your own your own prompts,
  1579. 1:00:50right, to the agents where you correct
  1580. 1:00:51them where you have to constantly
  1581. 1:00:53intervene and uh you know take that
  1582. 1:00:57higher level learning and turn that into
  1583. 1:00:59a a reusable skill, right? So that
  1584. 1:01:00agents stop repeating that mistake. I
  1585. 1:01:04think that uh PAC and your skills are
  1586. 1:01:08very complimementaryary. I I totally
  1587. 1:01:10agree with you that they're like a skill
  1588. 1:01:12is really much just process. I mean it's
  1589. 1:01:14just at the end of the day a skill is
  1590. 1:01:15just English or or language. It's just
  1591. 1:01:18markdown.
  1592. 1:01:19>> Um and I think you can you can
  1593. 1:01:21definitely weave them, combine them in a
  1594. 1:01:24way that makes sense to you. But I do
  1595. 1:01:27think that uh everyone should have their
  1596. 1:01:30own set of knives, right?
  1597. 1:01:33I I keep going back to the the the
  1598. 1:01:35cooking analogy, but it's so apt because
  1599. 1:01:37like, you know, every chef when they go
  1600. 1:01:39to a different job, right, when they go
  1601. 1:01:41to a different restaurant, they carry
  1602. 1:01:42they bring their knives with them. The
  1603. 1:01:44tools go with them, right? And so trust
  1604. 1:01:46to me is really about trust in your own
  1605. 1:01:48tools. And when you spend the time
  1606. 1:01:51sharpening them and understanding them
  1607. 1:01:52really, really well, you can do great
  1608. 1:01:54things. And everybody's skills and tool
  1609. 1:01:56set is going to look different. you
  1610. 1:01:58know, someone might find a lot of
  1611. 1:02:00success combining, you know, like your
  1612. 1:02:03grill me with docs, uh, or wayfinder
  1613. 1:02:05skill with some of the execution skills
  1614. 1:02:07in Pstack as an example. Some people
  1615. 1:02:10might use more of your skills, some
  1616. 1:02:11people might use more of my skills. I
  1617. 1:02:13think at the end end of the day, it
  1618. 1:02:15really just comes back to how much do
  1619. 1:02:17you trust, you know, me and Matt, right?
  1620. 1:02:19Like if you if you trust us both, of
  1621. 1:02:21course, use our skills, but I also
  1622. 1:02:23encourage you to, you know, look at your
  1623. 1:02:25own transcripts. Um um tell the agent to
  1624. 1:02:29look through, you know, some of the all
  1625. 1:02:31of the patterns that you've used, the
  1626. 1:02:33the times you've had to intervene, you
  1627. 1:02:36know, suggest turning them into lint
  1628. 1:02:38rules or new skills, right? The the past
  1629. 1:02:41chats I I often say is like a a treasure
  1630. 1:02:44trove of context because that, you know,
  1631. 1:02:46it's it's like the process materialized,
  1632. 1:02:51right? Like it's the real process. It's
  1633. 1:02:53not an abstract idea in your head. it's
  1634. 1:02:55the actual thing right and you can
  1635. 1:02:57actually see how it happened in practice
  1636. 1:02:59and extract so much information from
  1637. 1:03:01that and there's so much so that I
  1638. 1:03:02actually turn I have a skill in pac
  1639. 1:03:04called recall which is exactly that um
  1640. 1:03:07where this was a pattern where
  1641. 1:03:10you know I was working in I was working
  1642. 1:03:12on a similar problem so specifically I
  1643. 1:03:14was working on virtualization for the
  1644. 1:03:17cursor application and there were a lot
  1645. 1:03:18of bugs and so you know every time I
  1646. 1:03:21started a new chat I was like ah this
  1647. 1:03:22there's so much good context from the
  1648. 1:03:24last one So, you know, I want to bring
  1649. 1:03:25it over to the new chat. How do I do
  1650. 1:03:27that? And that's where the transcript
  1651. 1:03:29came, uh, you know, looking at the past
  1652. 1:03:31transcript came about. And then recall
  1653. 1:03:33was just a way for me to collapse
  1654. 1:03:35collapse and compress that workflow into
  1655. 1:03:37a skill. So that I didn't have to just
  1656. 1:03:39say I didn't have to write a long essay
  1657. 1:03:42every time. Go look at all these chats,
  1658. 1:03:43right? And, you know, blah blah blah.
  1659. 1:03:45So, I I largely think of skills,
  1660. 1:03:47especially as agents get more capable as
  1661. 1:03:50really encoding workflows. you know
  1662. 1:03:52skills from last year were really more
  1663. 1:03:54about like almost like implementation
  1664. 1:03:56details like here are the exact script
  1665. 1:03:59commands you know you should use right I
  1666. 1:04:02think with the latest models you can
  1667. 1:04:04just delete those parts and just really
  1668. 1:04:06focus on the workflow right until it
  1669. 1:04:09it's more the skill becomes more like a
  1670. 1:04:11series of steps
  1671. 1:04:13a series of your process uh and I think
  1672. 1:04:16over time we'll see that skills get
  1673. 1:04:18smaller and smaller you know more
  1674. 1:04:20compact Um, and yeah, they're very
  1675. 1:04:24compatible. Or you can, you know, if you
  1676. 1:04:26want, why not read our skills, right,
  1677. 1:04:29and com and combine them in your of your
  1678. 1:04:31own, right? Combine Wfinder with potato
  1679. 1:04:34mode and make your own custom mode,
  1680. 1:04:36right? Like like skills are the the
  1681. 1:04:38thing I love about skills that is are
  1682. 1:04:40that they're so malleable. You can do
  1683. 1:04:42anything you want. It's just language.
  1684. 1:04:44>> Absolutely. There's nothing magical in
  1685. 1:04:46them, right? They're just words. And
  1686. 1:04:48>> Exactly. If if there is any magic in
  1687. 1:04:49them, it's just the words chosen and the
  1688. 1:04:52phrases used and the thinking that's
  1689. 1:04:55been done to turn those like take
  1690. 1:04:59abstract process and turn them into
  1691. 1:05:01language. And once that thinking has
  1692. 1:05:04been done, then it's just there. It's
  1693. 1:05:05available. It's on the surface and you
  1694. 1:05:06just nick it. Um Lauren, thank you so
  1695. 1:05:10much. This has been glorious.
  1696. 1:05:11>> Yeah, this has been super fun. I really
  1697. 1:05:14enjoyed talking to you. Hope we can do
  1698. 1:05:16it again. I'd love to do it again. I'd
  1699. 1:05:19love to do it again. Absolutely. Um
  1700. 1:05:20yeah, we'll check in in uh
  1701. 1:05:22>> I don't know. Yeah, Monday. Let's do it.
  1702. 1:05:25>> Yeah, [laughter] let's do it. Part two.
  1703. 1:05:28>> Well, thank you so much. I'm going to
  1704. 1:05:29close the stream here. Laura and I will
  1705. 1:05:31uh chat a little bit and stay here. But
  1706. 1:05:33thank you guys so much for watching. The
  1707. 1:05:34glorious.

About this transcript

This page contains the full transcript of LIVE: Poteto (creator of pstack) on shipping 1,000's of PR's a month at SpaceX by Matt Pocock, generated from the public captions YouTube serves with the video. The transcript has 12,134 words across 1,707 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.