YouTube2Text

SpaceX AI engineer Lauren Tan part 2 — Transcript

by Ai insider · 10,284 words · 541 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Um, personally I think that uh the best use of you  of AI is really as like a like a pair like someone
  2. 0:09you're pair programming with and not someone as  not not a not a tool to just replace the act of
  3. 0:14writing code or worse outsourcing your thinking.  And I think there's a tendency like, you know,
  4. 0:21because AI is so exciting, you want to put AI  everywhere and be seen as someone who's very,
  5. 0:26you know, um, what do you call it? Like,  uh, on the ball, I guess, with with AI. Uh,
  6. 0:33that you feel this pressure of like, you know, I'm  just going to I got to increase my productivity. I
  7. 0:38got to ship like 100 PRs this week and barely  understand what I'm doing and, you know,
  8. 0:42vive code my way to a million ARR or whatever.  Um but I think you know um one of the things that
  9. 0:52I've personally seen is that although when I use  AI I save some time you know writing the code I
  10. 0:59actually find myself spending more time reviewing  what the AI did and like correcting it. So,
  11. 1:05you know, like I've I've tried the the thing  that a lot of people say like, you know, you
  12. 1:09uh instead of prompting an AI to just build the  feature outright, you sort of get it to uh build
  13. 1:15a spec for you first, then you review that spec,  and then you do this whole thing, then it's like,
  14. 1:20uh, okay. So, I assume you can see my screen. Yes.  Yeah, we're good. [clears throat] So yeah, today
  15. 1:25yeah I think I think the big theme for me as I've  been using agents to write code and I'm sure a lot
  16. 1:31of you have had the same experience as well is how  do you trust it? You know especially if you are an
  17. 1:36engineer that's been writing code for a very long  time. You have a lot of opinions and lessons that
  18. 1:43you've learned about doing good engineering. And  when you see agents just, you know, winging it
  19. 1:49and, you know, guessing, hallucinating, uh, you  know, confidently stating that they found the
  20. 1:55smoking gun, uh, for the hundth time, uh, but it's  actually not the real problem. You lose a lot of
  21. 2:01trust. And when you lose when you don't have much  trust [clears throat] in your agents, I feel like
  22. 2:06you you really can't get the most out of them. And  for me, the parallel is like with management. Uh
  23. 2:12so if I'm an a manager an engineering manager of a  team and I have a bunch of you know I have a team
  24. 2:18of engineers uh on my team and I don't trust them  then the mode of operation I'm going to be in is
  25. 2:25going to be like micromanagement right I'll have  to spend a lot of time looking over my reports
  26. 2:30shoulders and checking that they're doing their  work well you know that they're not shipping bugs
  27. 2:35to production and so I drew this chart because uh  it's It's not it's not a very scientific chart but
  28. 2:43like this is how I imagine myself and my journey  through using agents. So you know like fast
  29. 2:51forward or back forward uh or fast back uh fast  backwards like a year or so when you know nobody
  30. 2:59was or not many people were using agents to code.  Uh I think you uh you know get into this mode
  31. 3:06where you are in very heavily in the loop with one  or several like a handful of agents and you find
  32. 3:15yourself just constantly fig uh you know trying  to understand what your agents are doing uh and
  33. 3:20you're very very in loop. You're watching every  single output. you are sitting there prompting
  34. 3:26um and you really can't parallelize beyond that  because you don't again you don't have that trust
  35. 3:32right you can't go to a 100 agents uh like spawn  100 agents when you don't even trust the output
  36. 3:37of one agent so over the past 5 months I feel  like I've really been able to uh like ascend this
  37. 3:47trust curve and now I'm at the point where uh I  actually have this sounds kind of scary to say And
  38. 3:53it it makes me sound like a slop artist, but I I  promise I'm not. But I actually have my agents now
  39. 3:59um autoemerging PRs for me. Uh which is like a  wild thing to say, but um like I woke up today and
  40. 4:06there were like 20 PRs landed and I just reviewed  them on main like they were already landed and
  41. 4:12they were good. Uh so how did I get to that point  is basically what I wanted to talk about today.
  42. 4:19Uh and again like yeah feel free to jump in if  you have questions. colon. Um but uh oh yeah, of
  43. 4:27course I got to show this this chart. [laughter]  Uh where uh no do not trust to someone requested
  44. 4:36to control my computer. Uh I probably won't do  that. Uh but yeah, so this chart I think I I'm I'm
  45. 4:43sharing this chart not to kind of like flex but  to kind of show like the journey like so you can
  46. 4:49see like the curve like it sort of like inversely  matches the contributions I've been able to land
  47. 4:54at cursor. So I joined five months ago and five  months ago like I you know my first month I was
  48. 5:00like not very productive because I was obvious you  know I was learning the codebase didn't know what
  49. 5:04the heck was going on and as I got more confident  in in my agents uh I've really been able to kind
  50. 5:12of ramp up my productivity uh and again like yeah  like last month I shipped a thousand PRs which is
  51. 5:19ridiculous. Uh, and then this month we're only  on the 12th. I'm already at like almost 800 PRs
  52. 5:26landed. Uh, so the velocity is definitely high  and you you I'm sure a lot of you will definitely
  53. 5:33be questioning like how how much of this code is  actually good. Um, and I think yeah, like that's
  54. 5:38definitely fair to question. Um but uh yeah, I  think I think if you set up your agents well,
  55. 5:48you can definitely get to a very similar level.  Um and so I'm going to talk about how we do that.
  56. 5:56Uh so for me I think I'm curious like I guess  call in your experience as well but uh for me
  57. 6:03I think the most important skill that you should  have in your toolbox when you work with agents is
  58. 6:10verification. Uh and by verification I mean the  ability for an agent to actually run the code
  59. 6:18uh or take CPU traces or heap snapshots or uh  you know open an iOS simulator whatever you know
  60. 6:27however your application is exposed to your users.  It can do the same thing and uh run it for real
  61. 6:35and actually test and verify it don't work because  that's the thing that really closes the loop.
  62. 6:41uh it doesn't guarantee your agent writes  good code uh but it allows them to at least
  63. 6:46write correct code uh which is a big a really big  step forward for being able to trust your agents.
  64. 6:53Um I will I can share one example that we have  uh within cursor. Uh oops where let me open this
  65. 7:14let's make this uh make full screen. There you  go. Uh so for the for cursor's agent window
  66. 7:23uh so this is actually an interesting story  but uh when I joined cursor five months ago
  67. 7:29uh they're actually uh well I was supposed to join  a different team I was supposed to join like the
  68. 7:34cloud agents team uh but then since I have a lot  of experience working on react and agents window
  69. 7:40is a react application uh I was uh I was asked to  basically help out with the agent window work. Uh
  70. 7:50but um there wasn't really a lot of like skills  to help me. So I just found myself that okay
  71. 7:58uh agents window is going to launch in like week  right we have a really tight deadline and um there
  72. 8:05was uh you know I was just sitting there like okay  I'm going to open up the performant the the chrome
  73. 8:09dev tools and just like take a trace look at it  myself and try to make sense of this flame graph.
  74. 8:14And keep in mind I was just like in my first week  so I had no idea what I was looking at. No idea
  75. 8:19where you know I mean I had some idea but you know  the code base was completely fresh to me. Uh and
  76. 8:25I realized like my agent had no idea either you  know like I would take a screenshot of the tree I
  77. 8:30would download trades I would send it to it and it  be like yeah it kind of looks like this you know
  78. 8:34uh and it would like confidently state like it's  this thing and then I try to fix that and turns
  79. 8:39out that's not the actual thing. So this was very  very slow process and if you've ever done any like
  80. 8:46performance work yourself or you know just even  development with an agent where you don't have a
  81. 8:50verification skill you are the verifier right you  you're the bottleneck you you you tell your agent
  82. 8:57to do something and then it goes off and write  some code then you open up your you know local
  83. 9:01dev build and then you start to say oh you know  doesn't work then you got to copy paste screen uh
  84. 9:06you know screenshots or console errors or whatever  uh and then your agent like slowly kind of like
  85. 9:12uh you know works with that and then tries  to understand it and um fix the thing but
  86. 9:19then you're constantly just in the loop and and  being a bottleneck. So there's really no way to
  87. 9:22parallelize. So the control glass skill is like  one of the first skills I built uh for cursor. Uh
  88. 9:29and glass by the way is the code name for agents  window that we use internally but it's just cursor
  89. 9:36I guess. Um, and so this skill uh is I guess the  the the code itself is not super interesting. Your
  90. 9:43agent can very easily make one for you. Uh where  if if you're building an electron app or a web app
  91. 9:49or even iOS uh applications, uh you can teach  your agent how to use like the Chrome DevTools
  92. 9:56protocol or through uh Apple has some utilities as  well for running the simulator and taking traces
  93. 10:03and controlling programmatic control as well.  Uh so that's really useful. Uh but one thing I
  94. 10:11want to talk about is uh the this thing uh where  is the read me? Uh so this skill comes with this
  95. 10:21very unique feature called or not feature uh  unique file called a feature map. And so the
  96. 10:29story then is like I built this skill and so now  the agent was able to uh actually run the agent
  97. 10:35window and take traces and whatnot. Uh but it  had no idea what what the agent's window was. So
  98. 10:43um you know like someone would say like oh the  the left sidebar is like laggy or something like
  99. 10:49that or you know the right side the the PR tab is  not working and the agent would just be like kind
  100. 10:55of flailing around. it would spend a lot of time  trying to like look up the code and you know where
  101. 10:59is this feature? How do I actually get to it on  the UI which made it basically completely useless.
  102. 11:04Uh you know like we would I would run this skill  locally and you know it would spawn a dev build
  103. 11:12uh but then it just be turnurning like I just  try to click here. It wouldn't know how to get
  104. 11:17to things um and it was just an awful experience.  Uh so who's putting arrows on my screen? Um, so,
  105. 11:26uh, yeah, this this feature map has been really  useful, uh, because it teaches the agent how to
  106. 11:32get to all of the features that you have. Um,  and in PAC, the plugin that I I've made, uh,
  107. 11:39if you search for PAC cursor on Google, you you'll  find it. Uh, but there is a create verification
  108. 11:46skill in that plugin where it actually helps  you set up something like this for yourself. Um,
  109. 11:51including the feature map. So it will actually  explore the code and build up this initial feature
  110. 11:56map that tells your agent how to get to all of the  different features that you have. Uh and this is
  111. 12:02extremely powerful because now like you have these  user reports that come in uh you you can actually
  112. 12:08map even like a vague report or even a screenshot.  So we have this uh internally at cursor where uh
  113. 12:16we have a slack channel with you know lots of  people giving us feedback on the agents window
  114. 12:21and rockbot and whatnot. Uh, and often times  the report is very bad, like very low quality,
  115. 12:28like someone just put very often we get like  a screenshot like and then someone just says
  116. 12:32question mark question mark question mark  like what is this? [laughter] And you know
  117. 12:35like the without this your agent is like I have  no clue, right? But with a feature map like this,
  118. 12:41it has a lot more context and understanding of  how to actually navigate how to get to all of
  119. 12:47the different features. Uh so like you know  example like I guess like the sidebar like
  120. 12:52what is the sidebar uh you know like all the  different sub features that are present in it
  121. 12:58um like from the user point of view here's where  to how do I get to it all the different keyboard
  122. 13:03shortcuts uh even like the the what do you call  it the DOM elements or yeah like the attributes
  123. 13:10that you use for selecting things through the  CDP uh are all there. So uh again yeah this is
  124. 13:17like really really powerful uh for for agents uh  and piece stack ships uh that create verification
  125. 13:25skill but also a maintain verification skill uh  so you can keep this up to date. Cool. Yeah, I was
  126. 13:32just going to ask how you created that. So do you  mind sharing a little bit more about um that that
  127. 13:36process in the context of Pstack and maybe just  what Pstack is for the folks who aren't familiar?
  128. 13:41Yeah. So, Pstack is pretty interesting because  uh well, first of all, the name is kind of goofy.
  129. 13:46Like the P the P in P stack is like potato potato  stack because I um so uh uh there's a pretty
  130. 13:57uh famous person Gary Tan who is the CEO of Y  Combinator and he's come up with this plugin
  131. 14:03called GStack uh Gary Stack and uh funnily enough  we share the last name. We have no relations. Uh,
  132. 14:11but I thought it would be funny to kind of, you  know, poke fun at Gary and make Pstack my version
  133. 14:16of of of his plug-in. Uh, but kind of tailor it  to my own set of p engineering practices. Uh,
  134. 14:25but I honestly actually never set out to build  PAC. Uh, it just started with a bunch of skills,
  135. 14:30right? Like I started with that control glass  skill and then I started with another skill
  136. 14:35like called how which I also noticed through  like observing agents. Um so like you know in
  137. 14:42the early days of me you know trying to climb  this ladder I was like super in the loop and I
  138. 14:46was basically nitpicking my agents to an extreme  degree. I was like I would tell it um you know
  139. 14:53this feature has stopped working. Here's a bug  report like why isn't it working? And very often
  140. 15:00the agent would just like confidently state like,  "Oh, it has to be this, right? It has to be this
  141. 15:05thing." And I noticed like when I looked at the  actual tool calls, I noticed it wasn't actually
  142. 15:10reading the code that I thought should be  affected. And that made me just extremely
  143. 15:16suspicious. And at that point, I was like, I'm not  going to I can't trust any this agent anymore cuz
  144. 15:20it's just it's just completely hallucinating. And  I think I think it's very easy to just, you know,
  145. 15:27like build up that distrust and not and kind of  feel helpless like, you know, you don't know how
  146. 15:33to help your agents succeed. But like again, I  think the the the management analogy is super
  147. 15:38helpful because like imagine if you were a manager  of an engineering team and you had an engineer on
  148. 15:44your team who was a really good coder, no business  context whatsoever. you know they they just you
  149. 15:49just hired them and they they onboarded you know  like 5 seconds ago. Uh and so how do you actually
  150. 15:55teach that person to be effective? So how you  do that is through a skill. uh skill being just
  151. 16:01you know it's just markdown right but you know it  encodes a lot of information instructions a lot of
  152. 16:08uh you can really draw out a lot of intelligence  from an agent by well some people on Twitter call
  153. 16:15it like you know pull the agent to a different  latent space which is kind of like a fancy way
  154. 16:19of just saying like since uh you LLMs are sort of  like they predict the next token uh when you give
  155. 16:26it some high quality tokens uh to begin with then  you know it it can kind of pattern match on like
  156. 16:32a higher space that's you know smarter. Um so  that's like a very interesting model there. But
  157. 16:40yeah I built PAC very very incrementally. Uh  so uh started with just really observing how
  158. 16:46agents you know all the fail different failure  modes of of that agents were having and every
  159. 16:51time I saw that I just okay I'm just going to  make that a skill right like stop hallucinating
  160. 16:56actually go and search up look up the code use  a lot of sub agents uh and yeah stop guessing
  161. 17:06yeah that makes sense one one kind of followup  question here both from myself and from a bunch
  162. 17:10of people in the chat so I guess it's two two  parts So one is like how do you maintain these
  163. 17:14skills? So like the product changes over time.  Obviously there's a lot of people who are shipping
  164. 17:17against the codebase. So how do these skills get  maintained? Uh and then second to that is like
  165. 17:23how do you know when your verification is is good  enough? Uh like and you know you can trust that
  166. 17:28the ver verification loops that you've built are  going to I guess you trust that the outputs uh
  167. 17:32when they're done. Uh yeah maybe I'll talk about  um I think somewhat related. Maybe I'll start with
  168. 17:40this one first. So like how do I maintain these  skills? So um if you're not familiar with this
  169. 17:49concept, an eval is essentially like a way to uh  well I think the mental model I have is like it's
  170. 17:55like a unit test for an agent. Um and uh you can  actually make your own eval. You don't need like a
  171. 18:02special framework for them. You can build you can  you can build one depending on like you know how
  172. 18:08scientific and how rigorous you want to be. Uh, my  screen is red. Yeah, there's a little button. Um,
  173. 18:15sorry. Do you mind like disabling the drawing or  something? I I can't see my screen. Yeah, sorry.
  174. 18:21If you guys could not draw on the screen, that'd  be great. But, um, there's a little button in the
  175. 18:24Is that a troll? Yeah, the little drop down. Um,  how do I clear? Yeah, you got it. Perfect. Yeah.
  176. 18:36Yeah, evals are a way to unit test your skills  basically. And actually in Pstack, we ship
  177. 18:43uh under potato mode there's a playbook if you  search for it called eval playbook. Um and it's
  178. 18:49uh uh it's like not it's actually pretty pretty  rigorous the way it's done. Uh but um essentially
  179. 18:57what I do is I spawn a lot of different sub  agents. I have like my main coordinator agent
  180. 19:03uh come up with a rubric for uh what I want the  skill to do. Um and then it spawns all these sub
  181. 19:11agents and it it creates individual directories  for them uh which are cleverly named to not
  182. 19:18let the sub agent know that it's being evaluated  because uh agents can actually tell and when they
  183. 19:24do they change their behavior. Uh, but it does a  bunch of stuff like that to um essentially yeah
  184. 19:32like test whether or not the skill I'm making or  changing is actually doing what I think it does.
  185. 19:38Um, and one of the really nice things about  cursor is that we are we we support so many
  186. 19:43different models. So you can actually eval your  skill across all sorts of different models. Um,
  187. 19:49and you know get a sense of how well it performs  across that different matrix. um especially for
  188. 19:55the models that you use. Uh so I do this a lot.  Every time I I modify a skill, I will run one of
  189. 20:02these uh like the Ebal playbook uh and make sure  that you know it's actually leading to a result
  190. 20:08I want. Uh but I will say like maintaining skills  is actually pretty hard. Uh it requires I think a
  191. 20:16lot of taste and observation. So you kind of need  to be very good at being a backseat driver. You
  192. 20:23know what I mean? Like if you do pair if you ever  done pair programming for example uh and you watch
  193. 20:28a coworker code and you just like you you could  probably do this better you know you could do you
  194. 20:32know like why did you not do this right you you  ask a lot of questions to your coworker and it's
  195. 20:37kind of a similar thing here you like you don't  want to just be a passive observer of your agent
  196. 20:41you want to be very in the driver seat in the  initial stages when you're building up your own
  197. 20:46set of skills uh you know obviously you can use  something like PAC but if you're building your own
  198. 20:50set of skills it's very I think you opening up the  all the tool calls and like reading the code and
  199. 20:57reading all the to uh the agent behavior and their  thinking blocks is a really great way to see where
  200. 21:04they they fail, right? like what what you know  where are they being done and then you can go and
  201. 21:10build a skill for that and then with verification  how you trust it is it's I think it's also a very
  202. 21:16similar iteration loop uh where you know like I  actually did the same process for verifying the
  203. 21:21verification skill where I actually get um so one  thing that's interesting about eval is that you
  204. 21:28can sort of hill climb them meaning that uh your  eval can produce a score right uh a score that you
  205. 21:36can get your coordinator to produce uh but also  comp uh you can have a judge agent of a different
  206. 21:41model to uh kind of cross reference and make  sure that the first model is not being biased,
  207. 21:48right? The model that's judging all of the sub  aents that are running the thing. Uh but you can
  208. 21:53also like hill climb. So meaning that you can you  can use like /loop in cursor and you can say okay
  209. 21:59keep looping on this eval right until everything  is 10 out of 10 as an example. Uh and I did the
  210. 22:06same the basically the same approach with the  control skill. And so I kind of it was very it
  211. 22:10was very hands-off actually. Uh so you know I uh I  kind of built I built that skill that way like the
  212. 22:16CLI and that skill. Um and over time it's gotten  really good. Uh but yeah, it was definitely not
  213. 22:23super smooth at the beginning. It required a lot  of iteration and I think there's an analogy here
  214. 22:30for me which is um well I make this analogy later  in a different slide on my drawing here. Uh but I
  215. 22:38think of it like uh you know as a as a engineer  now you're sort of more like you uh like maybe a
  216. 22:46manager or the analogy I like is like you're  like a a chef in a restaurant. uh you know,
  217. 22:51you're the head chef. Uh you're not cooking  all the food yourself anymore. You have a team
  218. 22:56of cooks, right? You have line cooks, you have a  sue chef, you have, you know, all these different
  219. 23:00stations. Um and it's your job to really design  the environment. You know, you you're in charge of
  220. 23:06setting up the kitchen. You're in charge of, you  know, like giving tasks to different people. So,
  221. 23:14um yeah, it's a very interesting way of working.  Uh but yeah, that's that's how I've basically
  222. 23:20built uh these verification skills. Yeah, just  just one fault there on like to go try to go
  223. 23:26one layer deeper. So are you let's say we wanted  to build um an eval or a skill for for something
  224. 23:33and we wanted to kind of get better on its own  which is is what I think you're suggesting. Uh
  225. 23:38are you doing that in like a work tree kind of  isolated with like the sub aents and and then the
  226. 23:43reviewer agent and and all that? Is it happening  like in some type of cloud hosted environment?
  227. 23:48like what's the the more the practical steps  if I wanted to go do this uh and like set up
  228. 23:52a verification system for something? What would  I what would I do or where would I start? Um
  229. 23:58I think that uh the best place to start is local  because you can observe you can definitely observe
  230. 24:04what your agents are doing. So, uh, if you're  building a verification skill for yourself,
  231. 24:09uh, I would definitely start local and just  have your agent bring up the application,
  232. 24:14whether it's like a CLI or, uh, desktop app or  whatever. And so, you can actually observe, right?
  233. 24:19You can see how the agent is interacting with the  the application. You can see it, you know, how it
  234. 24:26calls like the different APIs that that allow it  to interact with the uh the application. Um but
  235. 24:35uh for me personally uh I have basically been kind  of all in mostly all in on cloud agents because
  236. 24:41they're extremely powerful. Uh and the really  powerful thing about cursor is the the cloud
  237. 24:47agents actually where if you spend a little bit  of time setting up your environment these control
  238. 24:52skills these verification skills pay a huge amount  of dividends because it's not just something that
  239. 24:58makes you as a single engineer better. It actually  levels up your whole team uh and even your whole
  240. 25:05company because uh you can actually start thinking  about cloud agents. You can start thinking about
  241. 25:09automations that automatically do things like  uh I I get I kind of talk about this a bit later
  242. 25:17but I'll just kind of get into it. Uh where where  you know for example like I talk a lot about this
  243. 25:23agent we have called Benny, right? who who uh you  know takes all of the bug reports that we get and
  244. 25:30it automatically goes off in the cloud, opens up a  cloud uh it's you know its desktop. It runs cursor
  245. 25:37in its own computer and it uses the same control  skills to interact with the application and try
  246. 25:43to reproduce the bug uh or the user report, right?  And this is so so powerful because at once I can
  247. 25:49immediately I I get so much information from this  automatically like here in this example you can
  248. 25:54see that uh the Benny actually reproduced the bug  uh but it's already fixed on main. So it actually
  249. 26:02confirms that we fixed this problem already and  all I need to do is just release another build
  250. 26:06of of cursor. Uh so that's like huge information  there that I didn't have to go off and sit with
  251. 26:12an agent you know and spend an hour trying to  figure like is this fixed? this is not fixed. So
  252. 26:17you you you gain back so much time. Uh but you  know everybody on my team benefits from this.
  253. 26:22Everybody in the company benefits from this. Uh so  definitely think that uh you know keeping these uh
  254. 26:29using cloud agents is super powerful. Uh but yeah  it's like a journey. You have to trust it first
  255. 26:35right before you you get to this point. And that's  it goes back to what I was saying here where you
  256. 26:40know it's very hard. It's it's almost impossible.  And I would definitely encourage you not to try
  257. 26:45to jump from, you know, like if you're still in  this zone, you don't want to jump to like I'm
  258. 26:51going to spawn hundred of thous or thousands of  cloud agents right now because you're just going
  259. 26:56to waste a lot of tokens. Um and it's going to  be extremely expensive. Yeah. So just to kind of
  260. 27:02recap so far, basically the if we wanted to go on  the journey that you've kind of gone on, it would
  261. 27:06be to start with verification, building some some  skills and some some ways of determining that the
  262. 27:12agents are producing at least like correct code.  Whether like you said, whether it's good code or
  263. 27:16not is maybe a separate question, but like it's  it's technically solving the problem by looking
  264. 27:20at, you know, stack traces, looking at, you know,  the the actual behavior in the app and so on.
  265. 27:25Um, and then once we trust it locally, then we can  start to think about scaling into the cloud and
  266. 27:29running more agents that are picking up signals I  guess on their own, right? So whether that's like
  267. 27:33a bug report that comes in or something, they  can go and pick it up and solve the problem and
  268. 27:38give us back a PR. And then maybe the last step is  like automerging the PRs, which uh is where you're
  269. 27:43at, [laughter] maybe not where everyone is at.  Um, and then reviewing the one main, but um, is
  270. 27:48that is that about right? Yeah, exactly. I think  yeah, that's why I drew this this uh this curve,
  271. 27:53right? because that this this basically describes  my journey of you know when I started barely could
  272. 27:59use a couple agents and I was just observing every  single thing. I think there's really no shortcut
  273. 28:03for going from here to there because this is  really about your personal level of trust in
  274. 28:08agents, right? Um obviously, you know, as a as  an engineer, you don't want to just slop code
  275. 28:13into production. So, how do you actually build  up that trust? Takes um a lot of uh I guess taste
  276. 28:20and judgment. Um but uh you know, like I think  plugins like Pstack definitely kind of help you
  277. 28:27uh get up to speed much quicker. Uh and so I  guess it's like if you trust me and you trust
  278. 28:33PAC then in by extension you can maybe trust  your agents but if you don't trust me and I I
  279. 28:39definitely would not encourage people to blindly  trust me uh uh you know if you build up your own
  280. 28:46set of skills that you can obviously you know  take a look at PAC and kind of fork it make it
  281. 28:51your own improve the skills definitely encourage  that uh but for me it's really all about it just
  282. 28:57keeps coming back to trust you know every one of  us here in this chat have a different standard for
  283. 29:02engineering. Uh and there are different things  that are important for us in our codebase and
  284. 29:08uh when you are able to encode all of that into  skills and you can verify that your agent is
  285. 29:13actually doing them that allows you to really  kind of ascend this curve and um uh you know
  286. 29:20start automating things. Uh there's another piece  I wanted to talk about um if there's more. Yeah,
  287. 29:26go for it. I I'll pick up more questions as I  go. Yeah, I think there's a third part to this
  288. 29:30which I haven't talked about yet which is kind of  an interesting one which is like refactoring and
  289. 29:35rewriting. Like one of the uh I guess most  controversial one of the most controversial
  290. 29:41topics in the industry I think is like should  you rewrite your app or not? Um because I think
  291. 29:48engineers are very prone to this where especially  when you join a company you come in and you see
  292. 29:53like the code base and you're like oh man this  is [ __ ] like who wrote this code you know it's
  293. 29:58terrible I want to rewrite the whole thing is a  very common inclination and I think a lot of you
  294. 30:04know before agents um and I guess arguably even  now people will definitely discourage you from
  295. 30:09re rewriting stuff but I'm actually here to make  a case for why you might want to consider it. Um
  296. 30:16because I think it really depends. Uh you know uh  brownfield applications I think are actually in a
  297. 30:24pretty good spot especially if they're set up well  already. Uh and like recently I've been talking to
  298. 30:29some people but uh you know I I was just observing  I I just noticed this parallel which is that a lot
  299. 30:37of big tech company problems are now everybody's  problems. Um, and the big tech company problem,
  300. 30:43you know, like when I was working at Meta,  like we had this giant monor repo, we had like,
  301. 30:48I don't know, tens of thousands of engineers just,  you know, like banging on their keyboards and and
  302. 30:53shipping code. And a lot of really great engineers  at Meta. Uh, but, uh, I'll say like, you know,
  303. 31:01you'll be surprised that the code quality is  actually not that good. Um, and so I often joke
  304. 31:06that like, you know, before AI slop, we had human  slop. Um and so uh you know I think a lot of big
  305. 31:12tech infra like uh like what Meta has or Google  you know you know really big tech companies are
  306. 31:19actually designed for that where you you're sort  of like you're catering to the the you know like
  307. 31:24uh this sounds so bad to say but like the  the least capable engineer on your team
  308. 31:28right you build you build frameworks you build  conventions you build guard rails you know you
  309. 31:34restrict credentials so that you know your intern  doesn't wipe your production database Um there's
  310. 31:41uh you know if you have that level of infra  already I think your agents can actually already
  311. 31:46do a very solid job right because they have the  the guard rails are already in place for agents
  312. 31:53to not cause havoc or not cause too much havoc  uh in your codebase. Um and you can always add
  313. 31:59more you know guard rails. Uh but I think like  green field applications especially are you know
  314. 32:04like the brand new applications are like the  biggest risk in my opinion. Uh and also the
  315. 32:10greatest opportunity because you know if you vibe  code a project uh a prototype um like we did for
  316. 32:16Grockbot you know Grockbot was spun up very very  very quickly. Um and if you if you haven't heard
  317. 32:22of of Grockbot it's like our a new application we  just launched yesterday. Uh it's it's really cool.
  318. 32:28uh lets you orchestrate your create like  individual agents that have their own identity
  319. 32:33and you can kind of orchestrate them. It's super  cool. Definitely check it out. U but yeah, that
  320. 32:37was it's like a very it was a very green field  application like most prototypes are so like vibe
  321. 32:43coded very quickly. Humans were not reading the  code at all. And uh I had this tweet recently uh
  322. 32:51where I said something about organic architecture.  Um maybe I'll find it. Uh but the idea is that uh
  323. 33:01when you have a completely vibe coded application,  you essentially have no guard rails whatsoever. So
  324. 33:07uh your agents when you give them a task, they  will just solve it in whatever method is the
  325. 33:14most convenient. And over time you get into this  uh situation where you have a codebase that is
  326. 33:20spiraling out of control because you don't  understand it. uh your agents understand it
  327. 33:26I guess in a way but like they've built something  that is you know optimized for short for shortcuts
  328. 33:32uh and uh you know it will you will suffer you'll  have a lot of of issues with that application
  329. 33:40uh so I think starting your codebase with uh like  very strong constraints is very much needed uh
  330. 33:48because like when you have a codebase that you can  trust right when you have guardrails that actually
  331. 33:53help you uh uh help your agents write good code.  You can get into the you know like into this part
  332. 34:00of the curve where I I where like I I said you  know I woke up today and I had like 20 PRs merged
  333. 34:07u by my agents and that's because I invested a  lot a lot of time uh over 600 PRs I I I calculated
  334. 34:14yesterday uh when I refactored all of Grockbot to  this new architecture that I've been building. Um,
  335. 34:23and yeah, I've gotten to a point where I  I don't really look I really don't look at
  336. 34:28the code anymore. And um, I say that not just, you  know, to sell you tokens, but because I, you know,
  337. 34:35it it it took a lot of work to get to that point.  I spent a lot of tokens to get the codebase to
  338. 34:40this point where I no longer have to look at  it. Uh, but I'm very excited because, you know,
  339. 34:45of the potential where, you know, it's not just  this doesn't just benefit me. It benefits everyone
  340. 34:50contributing to Grockbot and it also empowers you  know designers and product managers and you know
  341. 34:57pe uh even GTM people to add features to Grockbot  and I don't have to worry you know I don't have
  342. 35:03to to wake up at night in in the middle of the  night and worry like oh [ __ ] someone's just
  343. 35:07merged a perf regression right I have a ton  of constraints and CI like it's actually very
  344. 35:13annoying to write code in in graph web but like  agents absorb all of that annoyance Um but yeah,
  345. 35:19I'm happy to talk about what exactly that is. Um  yeah, I think one question um yeah, before we get
  346. 35:27into the this part here is just around that  element of like what your your your CI looks
  347. 35:32like or maybe some of the constraints and then  also like the average PR size. I saw a question
  348. 35:35about that earlier just to give people you know  kind of a a glance. It doesn't have to be like
  349. 35:40mathematically average but just uh you know like  what generally the size of the a PR is. um if it's
  350. 35:46only a couple lines of code or you know um yeah  um I think it depends uh let me I'm trying to do
  351. 35:55this in a way where I'm not going to like you yeah  you don't have to share the actual number like an
  352. 36:00actual average I think this is fine benchmark but  like we have so okay this is not that interesting
  353. 36:07but uh well fun fact is that virtualization in  grockbot and in uh cursor is actually powered by
  354. 36:15uh Pretext uh which is a sort of new library that  someone's built. Um that's really interesting. You
  355. 36:22should you should check it out, but that's not  really that important. Uh I think the average
  356. 36:26PR size I actually don't know. I I don't know  if I want to click on these. Uh I probably can,
  357. 36:33but I would say like they can range anywhere from  a few hundred lines or 50 lines to like a thousand
  358. 36:39depending on what the thing is doing. Uh, so like  here I'm actually like deleting a bunch of files.
  359. 36:45So I expect that it's just like mostly deletion.  Uh, but yeah, it kind of varies. There's no like
  360. 36:52Yeah, there's no like hard cap or hard limit. Are  they're all like 50 line PR? There's no hard cap.
  361. 36:55Yeah, there's definitely no hard cap, but I I do  encourage my agents to split up their work into
  362. 36:59multiple PRs. Uh I do that mostly because uh I  like I like the idea of the I guess maybe this is
  363. 37:08much harder to do now as in the world of agents  and you have like so many commits but I like
  364. 37:13the idea that you know the git history is a very  rich source of context. Uh, and I like the I like
  365. 37:19each PR to sort of atomically describe what that  small piece of thing is doing, which also makes
  366. 37:26it easier for me to revert changes and like figure  out, you know, oh, I shipped a bug and it's just
  367. 37:30it's here, right? It's not in this 40,000 line PR  where you who knows what landed in there. Uh, but
  368. 37:38I don't have a hard cap on PR size. Cool. And then  um yeah, also quick question on like CI. So again,
  369. 37:46you don't have to go into like uh the screen share  of like your CI does, but just generally would you
  370. 37:50describe what the CI kind of looks like uh or how  strict it is? Uh yeah. So uh well specifically
  371. 37:58for Grockbot. So Dune is the is the sort of  cheeky code code name for the architecture that
  372. 38:04we've built for Grockbot. Um the CI looks pretty  annoying because there's checks for everything.
  373. 38:13So like literally I have um uh well if you've  written any react for example you know you know
  374. 38:18that one of the biggest foot guns in react is use  effect. Uh so in uh Dune and in graphbot we've
  375. 38:26banned use effect. So Dune is just you can the the  mental model of what Dune is uh you can kind of
  376. 38:32think of it as like Nex.js JS for uh electron  apps and it's designed for agents to write
  377. 38:38uh and it's like custom for you know our agent  powered applications. Um so the CI checks are
  378. 38:45very like specific to that like you know don't use  use effect it's it's it's banned like CI will fail
  379. 38:51uh and yell at you. We have like some of the more  interesting ones that people might raise eyebrows
  380. 38:56is like I actually ban code comments as well uh  which is very interesting. Uh, but I've noticed
  381. 39:03that 99% of the time agents just write code  comments that kind of describe some historical
  382. 39:10thing that is actually totally irrelevant to the  code. Um, like it will often say like, you know,
  383. 39:15oh, Lauren said you should never do this and it's  now in in a code comment. I'm like, what? Like
  384. 39:19why? What? That was I didn't say that as like a  durable, you know, global rule. I just meant like
  385. 39:24your this PR sucks and you should change that  part. agents don't really understand us that
  386. 39:31well surprisingly uh and or they kind of assume  too much and they kind of do things in like very
  387. 39:37stupid ways. So like yeah we just ban everything  everything you can imagine like the agents are bad
  388. 39:43at we ban. Uh so one example that we actually  suffer a lot in the agents window is we have
  389. 39:51uh you know you if you've used the agents window  you've definitely seen performance issues and you
  390. 39:55know we're constantly trying to fix them. Uh  but it's like a it's a never- ending struggle
  391. 40:01because there's so many pull requests that get  merged. Every any one of them could just regress
  392. 40:06performance or stability or reliability. Uh  you know the agents window doesn't have this
  393. 40:10architecture yet. I plan to do bring this learning  back there and kind of refactor everything there.
  394. 40:17uh but uh it just regresses super often uh because  uh there's just one example is like we have very
  395. 40:24poor um isolation between processes. So like  on you know on on Electron you have a renderer
  396. 40:30thread that renders your UI but you also have like  a main thread that you can run other code that you
  397. 40:35know doesn't need to block the renderer. Um, but  we do a poor job of separating those things and so
  398. 40:42often times you just accidentally have code that  gets pulled into running on the renderer thread
  399. 40:47and then all of a sudden you're competing with the  the renderer that you know that has a very if you
  400. 40:53want like 60 fps you have to every frame that  gets drawn has to be done in 16 milliseconds.
  401. 40:59So very very small you know deadline per frame uh  if you want you know a very smooth product. Uh and
  402. 41:05when you start building bringing in accidentally  bringing in you know things that are like very
  403. 41:10computationally heavy or they have a lot of IO  uh then you just get into like a lot of jank your
  404. 41:17FPS really drops. You start uh you know losing  frames. You get long tasks that take more than 16
  405. 41:23milliseconds and you just get this really choppy  experience. So all of those patterns that we've
  406. 41:28learned basically building electron apps we've  encoded into this framework and it becomes like
  407. 41:34a hard failure. So I literally in in grabbot  we literally have a directory called electron
  408. 41:39main electron renderer and we have a import uh CI  guess where we actually check the dependency graph
  409. 41:48to make sure you're not accidentally importing  code from one directory to another. Uh so that's
  410. 41:54enforced by CI um as well as bug bots uh which  is our which cursors um like code review tool
  411. 42:02that runs on CI uh you know in our agents MD it's  everywhere like so I I I I um I have this thing
  412. 42:10here where I I talk about like um you know like  there are multiple layers I think for building a
  413. 42:16good codebase. Uh obviously the codebase is one  where uh if you have an architecture like this
  414. 42:22where it's extremely strict uh you know the the  the way to build features is very conventional
  415. 42:28that's like the strongest strongest level of  enforcement because agents just love to copy
  416. 42:33existing patterns. So uh one example of this  in Rockbot is like we have this these concepts
  417. 42:39called like a feature and we have entry points  and transcript cards like oh you know the cards
  418. 42:44that you see in the chat these are all like like  nouns I guess in in the framework and so there's
  419. 42:50a very conventional way of creating them and so  like a feature is all in in a single directory as
  420. 42:57an example and so all of the code that contributes  to that feature lives in one directory so it's all
  421. 43:02collocated in one place makes It's super easy. You  know, agents don't have to like uh grap around and
  422. 43:08try to figure out like where all the things  are. It just looks at the feature and like,
  423. 43:12oh, okay, I'm working on the onboarding feature  in Grockbot. Uh, I'm just going to work in this
  424. 43:17directory. And for 80% of the work, it's mostly  just very encapsulated there. [clears throat] But,
  425. 43:23uh, like it's like very it's like designed again  for you know like the dumbest agent like you don't
  426. 43:29have to think, right? the the the one of the key  principles I have for this framework is like the
  427. 43:36shortest the shortest path is the best path. So uh  because that plays exactly to how agents love to
  428. 43:43write code is like they like to take shortcuts  really you know they they'll find the quickest
  429. 43:47way to solve the problem. So why not make that  the best way to solve the problem? Uh so I I
  430. 43:55probably won't get into all the specific details.  Um and uh the the this framework is really more
  431. 44:01of a collection of ideas and principles rather  than something that will open source. Uh you can
  432. 44:06you can you know screenshot this I guess if you  want and uh tell your agent to uh do some build
  433. 44:11build something like this for you too. Um yeah,  but it's really all about the layers uh you know
  434. 44:18like the the codebase is one part with features  uh and directories and you know import or blocking
  435. 44:25import dependencies uh that shouldn't be imported  uh but and and it all enforces that and static
  436. 44:31analysis. So like uh there's CI checks, we have  a lot of lints for bad patterns that we observe.
  437. 44:39uh compiler diagnostics uh there's also rules and  bugbot which are um I think like three four five
  438. 44:46are more soft right these two actually make  make CI red right so that you know there's a
  439. 44:53hard constraint where the agent can't just write  crappy code for rules and skills and bogbot your
  440. 45:02agents can still forget right you can still or  it may not always consistently apply them So,
  441. 45:08I like to layer them, but I don't I don't  like to rely on them as the only source of
  442. 45:15enforcement because it's very very soft, right?  And if you if you only have rules and bug bar and
  443. 45:20skills and a style guide for your code, you  will it's only a matter of time before your
  444. 45:24codebase looks like complete trash. I'm sorry to  say that, but uh I definitely recommend yeah like
  445. 45:30you know investing in you know things that can  be hard and forced, right? And this is why you
  446. 45:37know maybe uh the choice of text stack that you  use is also very important. Um like I think for
  447. 45:42example Rust is sort of making you know it's  like getting super popular again. Uh because
  448. 45:48the compiler is so strict right the compiler  enforces so many different things. you know,
  449. 45:53there's a borrow checker that you have to appease  and if as long as you make sure your agents don't
  450. 45:58write unsafe code blocks, uh you can more or less  feel somewhat confident that if the code compiles,
  451. 46:04it's probably works and it's good. Uh but you see,  it gives you that level of trust and confidence
  452. 46:10that you as a human engineer no longer need to go  and check it yourself. you know, you you rely on
  453. 46:18code and static analysis to actually make that uh  a lot smoother. Um, and I I guess the worst part,
  454. 46:27the worst place to be in is if you are stuck in  code review land where you actually enforce all of
  455. 46:33the constraints, the invariance in your codebase  by literally the human person saying, you know,
  456. 46:39reading the code and like, okay, you should not do  this, right? Every time you have to do that, you
  457. 46:44should consider that as a code smell, like a anti-  pattern and you should say, "Okay, instead of me
  458. 46:49commenting on the PR, how do I turn this into a  hard rule, right? How do I turn this into a lint
  459. 46:56rule? How do I turn this into a CI failure? Or how  do I even categorically eliminate this problem uh
  460. 47:02entirely?" Uh I I can talk about another migration  I've done, but I'll probably pause here. Sure.
  461. 47:09Yeah. I feel like that's that's where I am to be  honest. is is what you're describing right now,
  462. 47:13which is that like I don't have all of these  rules. So, I have some things to go do after this
  463. 47:17session in terms of being able to scale my agents.  I'm I'm definitely on like the uh you know, maybe
  464. 47:23a couple of parallel ones locally stage. So, like  two to three locally, and I'm sure most people
  465. 47:27here are on the same. So, uh yeah, I know we only  couple minutes left. Lauren, was there anything
  466. 47:32else that you wanted to to highlight? Obviously,  there's lots of questions, so I can grab more,
  467. 47:36but I want to give you a few minutes if there's  anything else you want to talk Well, I think I've
  468. 47:39been yapping for quite a lot, so I'm maybe let's  just do questions. Okay, cool. Uh, one question
  469. 47:44that had uh a couple of uh came up a couple times  was just around like token usage. So, the the
  470. 47:50question is like is is what you're describing a  realistic thing for people who are on, you know,
  471. 47:56uh a normal set of token usage. They don't have,  you know, basically unlimited tokens uh to work
  472. 48:01with. I think that's a really good point. I mean  like obviously you know I work at a AI lab where
  473. 48:07we have unlimited tokens. So uh I definitely  cannot say that you know this is something
  474. 48:14everyone should do in the exact same way that I  did it. I think it's possible to get to this point
  475. 48:19without you know breaking the bank. But you know  if you're like an engineering leader or you know
  476. 48:25you're you have a startup that you lead um I think  to me it's a question of ROI. Um, and it's like,
  477. 48:32uh, yes, you spend a lot of money on tokens in  the upfront stage, you know, like refactoring
  478. 48:39a codebase is going to take a lot of tokens.  Uh, adding all these things uh, is going to
  479. 48:44take a bunch of tokens. But if we're heading to a  world where agents are writing all the code and,
  480. 48:50you know, you want to be very lean, right? You  don't want to have to hire, you don't want to be,
  481. 48:55you don't want to become like meta, right?  Like I mean like in terms of you don't want
  482. 48:58to become a 10,000 person engineering org because  I mean that's a cool problem to have but also you
  483. 49:05have so much overhead. There's like planning you  know like you it's it's a personally I I wouldn't
  484. 49:12uh it it's not super fun but um I think you want  to stay very nimble right and you want to you want
  485. 49:18to be like agents are all about allowing you to do  things that you couldn't do before. That's really
  486. 49:23to me like the value of agents, you know? It's not  just storing tokens on every single little thing,
  487. 49:29but um to me like the thing I couldn't do before  is like enforce this level of constraints in
  488. 49:36a codebase by myself, right? Like I'm just a  single person, you know? Uh it would have taken
  489. 49:41me years to build this framework uh and do all the  refactoring and test everything myself and verify,
  490. 49:50you know, like run imagine if there it was just  me, right? you know in in pre- agent era just
  491. 49:55like running you know by it would take me so long  right and my salary is pretty high right like so
  492. 50:01you know the the question I think an engineering  leader might have is just then you know like what
  493. 50:07is there's a trade-off of do you hire someone  to do this or do you spend the tokens to set up
  494. 50:13a code base so that even the the most naive right  the dumbest agents can do a good job and when you
  495. 50:20actually get to this point like even agents that  are not you know fable size do an excellent job
  496. 50:26of writing code and this pays a lot of dividends  as well for me personally where I've empowered not
  497. 50:33just myself but again like PMS designers engineers  who are not familiar with Grockbot to just
  498. 50:39contribute in a way that is sustainable so I think  yeah it's definitely like a trade-off for sure you
  499. 50:47know like nothing is like free for sure uh and  tokens are pretty expensive Uh but oh actually
  500. 50:53uh I I I don't know how many of you have seen this  but we actually announced Grock 4.6 today. So very
  501. 50:59exciting finally out. Um so yeah, GRO 4.6 would  be like a great it was very very smart. Uh it's
  502. 51:06really good on the on the benchmarks. Uh and it's  the same the tokens uh well uh I hopefully I'm not
  503. 51:12saying this incorrectly, but uh I believe the cost  per token is the same as 4.5. So you're actually
  504. 51:18getting more intelligence for the same cost. Uh I  think this is an area that cursor tries to cursor
  505. 51:26and SpaceX AI try to really optimize for like  that heredto frontier of you know cost versus
  506. 51:32intelligence. Uh you know we don't necessarily  want to build the biggest model ever because that
  507. 51:37is extremely expensive to run. It's really about  like how do you find that sweet spot right? you
  508. 51:42don't you don't need a giant model, but it's just  super smart, right? And it's not very expensive
  509. 51:46for inference. Uh but um yeah, I think to kind  of round it up, um I think it's like a it's it's
  510. 51:55there's a if you do your own analysis, I feel like  it's pretty positive. It it'll be pretty positive
  511. 52:01that the ROI you get from investing in stuff like  this uh just empowers not just yourself, but your
  512. 52:08whole team to be so much more productive, right?  Like imagine if you have an army of engineers like
  513. 52:13me who are shipping so much improvements and and  bug fixes uh you know every day, right? Like that
  514. 52:21is pretty exciting. Cool. Uh one last question  before we wrap up. This one is for the people
  515. 52:26in product on the on the call. So let's say we do  have an army of engineers who are shipping like
  516. 52:32Lauren. I'm just curious like how is the product  team or other functions of your company keeping up
  517. 52:36given that like if you're shipping so quickly,  have are they using AI more to do their jobs?
  518. 52:42Like as much as you can speak to that obviously  you don't have like you're not in that role but
  519. 52:46just curious about how that works. Um I think this  is where grathbot has been actually exceedingly
  520. 52:52powerful. uh where so before graphbot like you  know uh obviously cursor only had cursor like we
  521. 52:59only had agents window we had a CLI we had an IDE  and these are really like power user tools right
  522. 53:05like de they're designed for developers so it's  very very developer centric you can do knowledge
  523. 53:10work in them but it like the UI is not really  optimized for that so we actually didn't really
  524. 53:17have uh well I think like a lot of people like you  know GTM product like they might have use cursor
  525. 53:23uh to do their work but it definitely wasn't like  a delightful experience for them. Um I think now
  526. 53:29with grabbot uh it's become Grockbot is basically  like the kusher moment for people who are not in
  527. 53:37tech in my opinion like it's like it's like  a very very accessible way to use agents in
  528. 53:43a very comfortable very familiar interface. It  looks like iMessage. Um, and it's very fun to,
  529. 53:49you know, you can give your agent a fun name. Uh,  you can have you can kind of do orchestration with
  530. 53:54in a very like natural way where you can sort of,  you know, each agent's like a person, right? Now,
  531. 53:59you got a team of agents like working on you  have one one agent per account that you manage
  532. 54:02as an example. Or if you're a PM, you have, you  know, you can have an agent that summarizes all
  533. 54:07the work that Lauren did last night and then now  you know what I did, right? So I think our PMs
  534. 54:12are leveraging that a lot and they're shipping  code too. Uh so you know like often times they
  535. 54:18will just say oh here's a bug I fixed it can you  look at it and then I'll go review it and actually
  536. 54:22it's just perfect. I'm like okay stamp. Uh so uh  that I think that shows that you know the the Dune
  537. 54:28architecture is holding up right the all the the  really strict constraints allow people who are not
  538. 54:34experts in engineering to contribute at a high  level. Uh so I'm I feel like I'm already seeing
  539. 54:40that pay off a lot where uh you know designers and  PMs are just able to to to ship features directly.
  540. 54:48Um and that just makes the Grockbot team super  fast, right? Where we can ship so quickly. Um
  541. 54:56and we have a lot planned. So, I'm very excited  uh to, you know, uh to to ship more ship more

About this transcript

This page contains the full transcript of SpaceX AI engineer Lauren Tan part 2 by Ai insider, generated from the public captions YouTube serves with the video. The transcript has 10,284 words across 541 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.