YT YouTube2Text

YouTube2Text → Transcripts → Lauren Tan - SpaceXAI engineer

Lauren Tan - SpaceXAI engineer — Transcript

by Vision Sabbat · 10,321 words · 1,400 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Um, personally I think that uh the best
0:05use of you of AI is really as like a
0:08like a pair like someone you're pair
0:09programming with and not someone as not
0:12not a not a tool to just replace the act
0:14of writing code or worse outsourcing
0:17your thinking.
0:19And I think there's a tendency like, you
0:21know, because AI is so exciting, you
0:22want to put AI everywhere and be seen as
0:25someone who's very, you know, um, what
0:28do you call it? Like, uh, on the ball, I
0:30guess, with with AI. Uh, that you feel
0:33this pressure of like, you know, I'm
0:35just going to I got to increase my
0:37productivity. I got to ship like 100 PRs
0:39this week and barely understand what I'm
0:41doing and, you know, vive code my way to
0:44a million ARR or whatever.
0:48Um but I think you know um one of the
0:52things that I've personally seen is that
0:54although when I use AI I save some time
0:57you know writing the code I actually
0:59find myself spending more time reviewing
1:02what the AI did and like correcting it.
1:04So, you know, like I've I've tried the
1:06the thing that a lot of people say like,
1:08you know, you uh instead of prompting an
1:10AI to just build the feature outright,
1:12you sort of get it to uh build a spec
1:15for you first, then you review that
1:17spec, and then you do this whole thing,
1:19then it's like,
1:20>> uh, okay. So, I assume you can see my
1:22screen.
1:22>> Yes. Yeah, we're good. [clears throat]
1:24So yeah, today yeah I think I think the
1:26big theme for me as I've been using
1:29agents to write code and I'm sure a lot
1:31of you have had the same experience as
1:33well is how do you trust it? You know
1:35especially if you are an engineer that's
1:37been writing code for a very long time.
1:40You have a lot of opinions and lessons
1:43that you've learned about doing good
1:45engineering. And when you see agents
1:47just, you know, winging it and, you
1:49know, guessing, hallucinating,
1:52uh, you know, confidently stating that
1:54they found the smoking gun, uh, for the
1:56hundth time, uh, but it's actually not
1:59the real problem. You lose a lot of
2:01trust. And when you lose when you don't
2:03have much trust [clears throat] in your
2:04agents, I feel like you you really can't
2:07get the most out of them. And for me,
2:10the parallel is like with management. Uh
2:12so if I'm an a manager an engineering
2:14manager of a team and I have a bunch of
2:17you know I have a team of engineers uh
2:20on my team and I don't trust them then
2:23the mode of operation I'm going to be in
2:25is going to be like micromanagement
2:26right I'll have to spend a lot of time
2:29looking over my reports shoulders and
2:32checking that they're doing their work
2:33well you know that they're not shipping
2:35bugs to production
2:38and so I drew this chart because uh it's
2:41It's not it's not a very scientific
2:42chart but like this is how I imagine
2:46myself and my journey through using
2:48agents. So you know like fast forward or
2:52back forward uh or fast back uh fast
2:56backwards like a year or so when you
2:59know nobody was or not many people were
3:01using agents to code. Uh I think you
3:05uh you know get into this mode where you
3:07are
3:09in very heavily in the loop with one or
3:12several like a handful of agents and you
3:15find yourself just constantly fig uh you
3:17know trying to understand what your
3:18agents are doing uh and you're very very
3:21in loop. You're watching every single
3:23output. you are sitting there prompting
3:26um and you really can't parallelize
3:28beyond that because you don't again you
3:31don't have that trust right you can't go
3:33to a 100 agents uh like spawn 100 agents
3:36when you don't even trust the output of
3:37one agent
3:40so over the past 5 months I feel like
3:42I've really been able to uh like ascend
3:45this trust curve and now I'm at the
3:49point where uh I actually have this
3:52sounds kind of scary to say And it it
3:54makes me sound like a slop artist, but I
3:55I promise I'm not. But I actually have
3:58my agents now um autoemerging PRs for
4:01me. Uh which is like a wild thing to
4:03say, but um like I woke up today and
4:06there were like 20 PRs landed and I just
4:08reviewed them on main like they were
4:10already landed and they were good. Uh so
4:14how did I get to that point is basically
4:16what I wanted to talk about today.
4:19Uh and again like yeah feel free to jump
4:21in if you have questions. colon. Um but
4:25uh oh yeah, of course I got to show this
4:28this chart. [laughter]
4:29Uh where uh
4:33no do not trust to someone requested to
4:36control my computer. Uh I probably won't
4:39do that. Uh but yeah, so this chart I
4:42think I I'm I'm sharing this chart not
4:44to kind of like flex but to kind of show
4:47like the journey like so you can see
4:49like the curve like it sort of like
4:50inversely matches the contributions I've
4:53been able to land at cursor. So I joined
4:56five months ago and five months ago like
4:58I you know my first month I was like not
5:01very productive because I was obvious
5:02you know I was learning the codebase
5:04didn't know what the heck was going on
5:06and as I got more confident in in my
5:09agents uh I've really been able to kind
5:12of ramp up my productivity uh and again
5:15like yeah like last month I shipped a
5:17thousand PRs which is ridiculous. Uh,
5:20and then this month we're only on the
5:2212th. I'm already at like almost 800 PRs
5:26landed. Uh, so the velocity is
5:29definitely high and you you I'm sure a
5:32lot of you will definitely be
5:33questioning like how how much of this
5:35code is actually good. Um, and I think
5:37yeah, like that's definitely fair to
5:39question.
5:41Um but uh yeah, I think I think if you
5:45set up your agents well, you can
5:48definitely get to a very similar level.
5:51Um and so I'm going to talk about how we
5:53do that.
5:56Uh so for me I think I'm curious like I
5:59guess call in your experience as well
6:01but uh for me I think the most important
6:05skill that you should have in your
6:08toolbox when you work with agents is
6:10verification. Uh and by verification I
6:13mean the ability for an agent to
6:15actually run the code uh or take CPU
6:20traces or heap snapshots or uh you know
6:24open an iOS simulator whatever you know
6:27however your application is exposed to
6:29your users. It can do the same thing and
6:34uh run it for real and actually test and
6:36verify it don't work because that's the
6:39thing that really closes the loop. uh it
6:42doesn't guarantee your agent writes good
6:43code uh but it allows them to at least
6:46write correct code uh which is a big a
6:49really big step forward for being able
6:51to trust your agents. Um
6:55I will I can share one example that we
6:58have uh within cursor. Uh
7:03oops
7:05where let me open this
7:14let's make this uh make full screen.
7:17There you go.
7:18Uh so for the for cursor's agent window
7:23uh so this is actually an interesting
7:25story but uh when I joined cursor five
7:28months ago uh they're actually uh well I
7:32was supposed to join a different team I
7:33was supposed to join like the cloud
7:34agents team uh but then since I have a
7:37lot of experience working on react and
7:39agents window is a react application uh
7:42I was
7:44uh I was asked to basically help out
7:47with the agent window work. Uh but um
7:53there wasn't really a lot of like skills
7:55to help me. So I just found myself that
7:57okay uh agents window is going to launch
7:59in like week right we have a really
8:01tight deadline and um there was uh you
8:05know I was just sitting there like okay
8:07I'm going to open up the performant the
8:08the chrome dev tools and just like take
8:10a trace look at it myself and try to
8:13make sense of this flame graph. And keep
8:15in mind I was just like in my first week
8:17so I had no idea what I was looking at.
8:19No idea where you know I mean I had some
8:21idea but you know the code base was
8:23completely fresh to me. Uh and I
8:25realized like my agent had no idea
8:28either you know like I would take a
8:29screenshot of the tree I would download
8:30trades I would send it to it and it be
8:32like yeah it kind of looks like this you
8:34know uh and it would like confidently
8:36state like it's this thing and then I
8:38try to fix that and turns out that's not
8:40the actual thing.
8:42So this was very very slow process and
8:44if you've ever done any like performance
8:46work yourself or you know just even
8:48development with an agent where you
8:50don't have a verification skill you are
8:53the verifier right you you're the
8:55bottleneck you you you tell your agent
8:57to do something and then it goes off and
8:59write some code then you open up your
9:01you know local dev build and then you
9:02start to say oh you know doesn't work
9:04then you got to copy paste screen uh you
9:06know screenshots or console errors or
9:08whatever uh and then your agent like
9:10slowly kind of like uh you know works
9:13with that and then tries to understand
9:15it and um fix the thing but then you're
9:19constantly just in the loop and and
9:21being a bottleneck. So there's really no
9:22way to parallelize. So the control glass
9:25skill is like one of the first skills I
9:26built uh for cursor. Uh and glass by the
9:30way is the code name for agents window
9:32that we use internally but it's just
9:35cursor I guess. Um, and so this skill uh
9:39is I guess the the the code itself is
9:41not super interesting. Your agent can
9:43very easily make one for you. Uh where
9:46if if you're building an electron app or
9:48a web app or even iOS uh applications,
9:52uh you can teach your agent how to use
9:54like the Chrome DevTools protocol or
9:57through uh Apple has some utilities as
10:01well for running the simulator and
10:02taking traces and controlling
10:04programmatic control as well. Uh so
10:07that's really useful. Uh but one thing I
10:11want to talk about is uh the
10:15this thing
10:17uh where is the read me? Uh so this
10:20skill comes with this very unique
10:22feature called or not feature uh unique
10:25file called a feature map. And so the
10:29story then is like I built this skill
10:31and so now the agent was able to uh
10:34actually run the agent window and take
10:36traces and whatnot. Uh but it had no
10:39idea what what the agent's window was.
10:42So um you know like someone would say
10:45like oh the the left sidebar is like
10:48laggy or something like that or you know
10:50the right side the the PR tab is not
10:53working and the agent would just be like
10:55kind of flailing around. it would spend
10:56a lot of time trying to like look up the
10:58code and you know where is this feature?
10:59How do I actually get to it on the UI
11:02which made it basically completely
11:04useless. Uh you know like we would I
11:06would run this skill locally and you
11:09know it would spawn a dev build uh but
11:12then it just be turnurning like I just
11:14try to click here. It wouldn't know how
11:16to get to things um and it was just an
11:19awful experience. Uh so who's putting
11:22arrows on my screen? Um, so, uh, yeah,
11:27this this feature map has been really
11:29useful, uh, because it teaches the agent
11:31how to get to all of the features that
11:33you have. Um, and in PAC, the plugin
11:37that I I've made, uh, if you search for
11:41PAC cursor on Google, you you'll find
11:43it. Uh, but there is a create
11:45verification skill in that plugin where
11:48it actually helps you set up something
11:49like this for yourself. Um, including
11:52the feature map. So it will actually
11:53explore the code and build up this
11:55initial feature map that tells your
11:57agent how to get to all of the different
11:59features that you have. Uh and this is
12:02extremely powerful because now like you
12:04have these user reports that come in uh
12:07you you can actually map even like a
12:09vague report or even a screenshot. So we
12:12have this uh internally at cursor where
12:16uh we have a slack channel with you know
12:17lots of people giving us feedback on the
12:20agents window and rockbot and whatnot.
12:23Uh, and often times the report is very
12:26bad, like very low quality, like someone
12:28just put very often we get like a
12:30screenshot like and then someone just
12:31says question mark question mark
12:33question mark like what is this?
12:34[laughter] And you know like the without
12:36this your agent is like I have no clue,
12:39right? But with a feature map like this,
12:41it has a lot more context and
12:43understanding of how to actually
12:46navigate how to get to all of the
12:48different features. Uh so like you know
12:50example like I guess like the sidebar
12:52like what is the sidebar uh you know
12:54like all the different sub features that
12:56are present in it um like from the user
12:59point of view here's where to how do I
13:01get to it all the different keyboard
13:03shortcuts
13:04uh even like the the what do you call it
13:07the DOM elements or yeah like the
13:10attributes that you use for selecting
13:12things through the CDP uh are all there.
13:16So uh again yeah this is like really
13:18really powerful uh for for agents
13:22>> uh and piece stack ships uh that create
13:24verification skill but also a maintain
13:27verification skill uh so you can keep
13:29this up to date.
13:31>> Cool. Yeah, I was just going to ask how
13:32you created that. So do you mind sharing
13:33a little bit more about um that that
13:36process in the context of Pstack and
13:37maybe just what Pstack is for the folks
13:39who aren't familiar?
13:41Yeah. So, Pstack is pretty interesting
13:43because uh well, first of all, the name
13:45is kind of goofy. Like the P the P in P
13:48stack is like potato potato stack
13:52because I um so uh uh there's a pretty
13:57uh famous person Gary Tan who is the CEO
14:00of Y Combinator and he's come up with
14:02this plugin called GStack uh Gary Stack
14:06and uh funnily enough we share the last
14:09name. We have no relations. Uh, but I
14:11thought it would be funny to kind of,
14:13you know, poke fun at Gary and make
14:15Pstack my version of of of his plug-in.
14:19Uh, but kind of tailor it to my own set
14:22of p engineering practices.
14:25Uh, but I honestly actually never set
14:27out to build PAC. Uh, it just started
14:29with a bunch of skills, right? Like I
14:31started with that control glass skill
14:33and then I started with another skill
14:35like called how which I also noticed
14:38through like observing agents. Um so
14:41like you know in the early days of me
14:43you know trying to climb this ladder I
14:44was like super in the loop and I was
14:47basically nitpicking my agents to an
14:49extreme degree. I was like I would tell
14:51it um you know this feature has stopped
14:54working. Here's a bug report like why
14:56isn't it working?
14:59And very often the agent would just like
15:01confidently state like, "Oh, it has to
15:03be this, right? It has to be this
15:05thing." And I noticed like when I looked
15:07at the actual tool calls, I noticed it
15:10wasn't actually reading the code that I
15:12thought should be affected. And that
15:15made me just extremely suspicious. And
15:16at that point, I was like, I'm not going
15:18to I can't trust any this agent anymore
15:20cuz it's just it's just completely
15:22hallucinating. And I think
15:25I think it's very easy to just, you
15:27know, like build up that distrust and
15:29not and kind of feel helpless like, you
15:32know, you don't know how to help your
15:33agents succeed. But like again, I think
15:36the the the management analogy is super
15:38helpful because like imagine if you were
15:40a manager of an engineering team and you
15:43had an engineer on your team who was a
15:45really good coder, no business context
15:47whatsoever. you know they they just you
15:49just hired them and they they onboarded
15:51you know like 5 seconds ago. Uh and so
15:55how do you actually teach that person to
15:56be effective? So how you do that is
15:59through a skill. uh skill being just you
16:01know it's just markdown right but you
16:03know it encodes a lot of information
16:05instructions a lot of uh you can really
16:09draw out a lot of intelligence from an
16:12agent by well some people on Twitter
16:15call it like you know pull the agent to
16:17a different latent space which is kind
16:19of like a fancy way of just saying like
16:20since uh you LLMs are sort of like they
16:23predict the next token uh when you give
16:26it some high quality tokens uh to begin
16:29with then you know it it can kind of
16:30pattern match on like a higher space
16:33that's you know smarter.
16:36Um so that's like a very interesting
16:39model there. But yeah I built PAC very
16:41very incrementally. Uh so uh started
16:44with just really observing how agents
16:47you know all the fail different failure
16:48modes of of that agents were having and
16:51every time I saw that I just okay I'm
16:53just going to make that a skill right
16:54like stop hallucinating actually go and
16:57search up look up the code use a lot of
16:59sub agents uh and yeah stop guessing
17:06>> yeah that makes sense one one kind of
17:08followup question here both from myself
17:09and from a bunch of people in the chat
17:10so
17:12>> I guess it's two two parts So one is
17:13like how do you maintain these skills?
17:15So like the product changes over time.
17:16Obviously there's a lot of people who
17:17are shipping against the codebase. So
17:19how do these skills get maintained? Uh
17:21and then second to that is like how do
17:23you know when your verification is is
17:25good enough? Uh like and you know you
17:27can trust that the ver verification
17:29loops that you've built are going to I
17:31guess you trust that the outputs uh when
17:32they're done.
17:35>> Uh yeah maybe I'll talk about um I think
17:39somewhat related. Maybe I'll start with
17:40this one first. So like how do I
17:44maintain these skills?
17:47So um if you're not familiar with this
17:49concept, an eval is essentially like a
17:52way to uh well I think the mental model
17:55I have is like it's like a unit test for
17:56an agent. Um and uh you can actually
18:00make your own eval. You don't need like
18:02a special framework for them. You can
18:04build you can you can build one
18:06depending on like you know how
18:08scientific and how rigorous you want to
18:10be. Uh, my screen is red.
18:13>> Yeah, there's a little button. Um,
18:15sorry. Do
18:16>> you mind like disabling the drawing or
18:18something? I I can't see my screen.
18:20>> Yeah, sorry. If you guys could not draw
18:22on the screen, that'd be great. But, um,
18:23there's a little button in the
18:24>> Is that a troll?
18:26>> Yeah, the little drop down.
18:28>> Um, how do I clear?
18:31>> Yeah, you got it. Perfect.
18:36>> Yeah. Yeah, evals are a way to unit test
18:39your skills basically. And actually in
18:42Pstack, we ship uh under potato mode
18:44there's a playbook if you search for it
18:46called eval playbook. Um and it's uh
18:50uh it's like not it's actually pretty
18:53pretty rigorous the way it's done. Uh
18:55but um essentially what I do is I spawn
18:58a lot of different sub agents. I have
19:00like my main coordinator agent uh come
19:03up with a rubric for uh what I want the
19:07skill to do. Um and then it spawns all
19:10these sub agents and it it creates
19:13individual directories for them uh which
19:16are cleverly named to not let the sub
19:19agent know that it's being evaluated
19:21because uh agents can actually tell and
19:24when they do they change their behavior.
19:26Uh, but it does a bunch of stuff like
19:28that to um essentially yeah like test
19:33whether or not the skill I'm making or
19:35changing is actually doing what I think
19:37it does. Um, and one of the really nice
19:39things about cursor is that we are we we
19:42support so many different models. So you
19:44can actually eval your skill across all
19:46sorts of different models. Um, and you
19:49know get a sense of how well it performs
19:52across that different matrix. um
19:55especially for the models that you use.
19:58Uh so I do this a lot. Every time I I
20:00modify a skill, I will run one of these
20:02uh like the Ebal playbook uh and make
20:05sure that you know it's actually leading
20:08to a result I want. Uh but I will say
20:10like maintaining skills is actually
20:14pretty hard. Uh it requires I think a
20:16lot of taste and observation. So you
20:19kind of need to be very good at being a
20:22backseat driver. You know what I mean?
20:24Like if you do pair if you ever done
20:25pair programming for example uh and you
20:28watch a coworker code and you just like
20:30you you could probably do this better
20:31you know you could do you know like why
20:33did you not do this right you you ask a
20:35lot of questions to your coworker and
20:36it's kind of a similar thing here you
20:38like you don't want to just be a passive
20:40observer of your agent you want to be
20:41very in the driver seat in the initial
20:44stages when you're building up your own
20:46set of skills uh you know obviously you
20:48can use something like PAC but if you're
20:50building your own set of skills it's
20:51very I think you opening up the all the
20:54tool calls and like reading the code and
20:57reading all the to uh the agent behavior
21:00and their thinking blocks is a really
21:02great way to see where they they fail,
21:05right? like what what you know where are
21:08they being done and then you can go and
21:10build a skill for that and then with
21:12verification how you trust it is it's I
21:15think it's also a very similar iteration
21:16loop uh where you know like I actually
21:19did the same process for verifying the
21:21verification skill where I actually get
21:25um so one thing that's interesting about
21:27eval is that you can sort of hill climb
21:30them meaning that uh your eval can
21:33produce a score right uh a score that
21:35you can get your coordinator to produce
21:38uh but also comp uh you can have a judge
21:40agent of a different model to uh kind of
21:44cross reference and make sure that the
21:46first model is not being biased, right?
21:48The model that's judging all of the sub
21:50aents that are running the thing. Uh but
21:52you can also like hill climb. So meaning
21:54that you can you can use like /loop in
21:57cursor and you can say okay keep looping
22:00on this eval right until everything is
22:0310 out of 10 as an example. Uh and I did
22:06the same the basically the same approach
22:07with the control skill. And so I kind of
22:09it was very it was very hands-off
22:11actually. Uh so you know I uh I kind of
22:14built I built that skill that way like
22:16the CLI and that skill. Um and over time
22:20it's gotten really good. Uh but yeah, it
22:23was definitely not super smooth at the
22:25beginning. It required a lot of
22:27iteration and I think there's an analogy
22:30here for me which is um well I make this
22:33analogy later in a different slide on my
22:36drawing here. Uh but I think of it like
22:40uh you know as a as a engineer now
22:42you're sort of more like you uh like
22:45maybe a manager or the analogy I like is
22:48like you're like a a chef in a
22:50restaurant. uh you know, you're the head
22:52chef. Uh you're not cooking all the food
22:54yourself anymore. You have a team of
22:56cooks, right? You have line cooks, you
22:58have a sue chef, you have, you know, all
22:59these different stations.
23:01Um and it's your job to really design
23:04the environment. You know, you you're in
23:06charge of setting up the kitchen. You're
23:07in charge of, you know, like giving
23:10tasks to different people. So,
23:14um yeah, it's a very interesting way of
23:17working. Uh but yeah, that's that's how
23:19I've basically built uh these
23:21verification skills.
23:23>> Yeah, just just one fault there on like
23:25to go try to go one layer deeper. So are
23:28you let's say we wanted to build um an
23:31eval or a skill for for something and we
23:34wanted to kind of get better on its own
23:36which is is what I think you're
23:37suggesting. Uh are you doing that in
23:39like a work tree kind of isolated with
23:41like the sub aents and and then the
23:43reviewer agent and and all that? Is it
23:45happening like in some type of cloud
23:46hosted environment? like what's the the
23:49more the practical steps if I wanted to
23:50go do this uh and like set up a
23:52verification system for something? What
23:53would I what would I do or where would I
23:55start?
23:57>> Um I think that uh the best place to
24:00start is local because you can observe
24:04you can definitely observe what your
24:05agents are doing. So, uh, if you're
24:07building a verification skill for
24:08yourself, uh, I would definitely start
24:10local and just have your agent bring up
24:13the application, whether it's like a CLI
24:15or, uh, desktop app or whatever. And so,
24:18you can actually observe, right? You can
24:20see how the agent is interacting with
24:22the the application. You can see it, you
24:25know, how it calls like the different
24:28APIs that that allow it to interact with
24:30the uh the application.
24:34Um but uh for me personally uh I have
24:37basically been kind of all in mostly all
24:40in on cloud agents because they're
24:41extremely powerful. Uh and the really
24:45powerful thing about cursor is the the
24:47cloud agents actually where if you spend
24:49a little bit of time setting up your
24:50environment
24:52these control skills these verification
24:54skills pay a huge amount of dividends
24:56because it's not just something that
24:58makes you as a single engineer better.
25:02It actually levels up your whole team uh
25:04and even your whole company because uh
25:06you can actually start thinking about
25:08cloud agents. You can start thinking
25:09about automations that automatically do
25:12things like uh I I get I kind of talk
25:16about this a bit later but I'll just
25:17kind of get into it. Uh where where you
25:20know for example like I talk a lot about
25:22this agent we have called Benny, right?
25:25who who uh you know takes all of the bug
25:29reports that we get and it automatically
25:31goes off in the cloud, opens up a cloud
25:33uh it's you know its desktop. It runs
25:36cursor in its own computer and it uses
25:39the same control skills to interact with
25:42the application and try to reproduce the
25:43bug uh or the user report, right? And
25:46this is so so powerful because at once I
25:49can immediately I I get so much
25:51information from this automatically like
25:53here in this example you can see that uh
25:55the Benny actually reproduced the bug uh
25:58but it's already fixed on main. So it
26:01actually confirms that we fixed this
26:03problem already and all I need to do is
26:05just release another build of of cursor.
26:08Uh so that's like huge information there
26:10that I didn't have to go off and sit
26:12with an agent you know and spend an hour
26:14trying to figure like is this fixed?
26:15this is not fixed. So you you you gain
26:18back so much time. Uh but you know
26:20everybody on my team benefits from this.
26:22Everybody in the company benefits from
26:24this. Uh so definitely think that uh you
26:28know keeping these uh using cloud agents
26:30is super powerful. Uh but yeah it's like
26:33a journey. You have to trust it first
26:35right before you you get to this point.
26:38And that's it goes back to what I was
26:39saying here where you know it's very
26:41hard. It's it's almost impossible. And I
26:43would definitely encourage you not to
26:45try to jump from, you know, like if
26:47you're still in this zone, you don't
26:50want to jump to like I'm going to spawn
26:52hundred of thous or thousands of cloud
26:54agents right now because you're just
26:56going to waste a lot of tokens. Um and
26:58it's going to be extremely expensive.
27:00>> Yeah. So just to kind of recap so far,
27:03basically the if we wanted to go on the
27:04journey that you've kind of gone on, it
27:06would be to start with verification,
27:09building some some skills and some some
27:11ways of determining that the agents are
27:12producing at least like correct code.
27:15Whether like you said, whether it's good
27:16code or not is maybe a separate
27:17question, but like it's it's technically
27:18solving the problem by looking at, you
27:20know, stack traces, looking at, you
27:22know, the the actual behavior in the app
27:24and so on. Um, and then once we trust it
27:26locally, then we can start to think
27:27about scaling into the cloud and running
27:29more agents that are picking up signals
27:32I guess on their own, right? So whether
27:33that's like a bug report that comes in
27:34or something, they can go and pick it up
27:36and solve the problem and give us back a
27:38PR. And then maybe the last step is like
27:41automerging the PRs, which uh is where
27:43you're at, [laughter] maybe not where
27:44everyone is at.
27:45>> Um, and then reviewing the one main, but
27:47um, is that is that about right?
27:49>> Yeah, exactly. I think yeah, that's why
27:51I drew this this uh this curve, right?
27:53because that this this basically
27:55describes my journey of you know when I
27:57started barely could use a couple agents
28:00and I was just observing every single
28:01thing. I think there's really no
28:03shortcut for going from here to there
28:06because this is really about your
28:07personal level of trust in agents,
28:09right? Um obviously, you know, as a as
28:12an engineer, you don't want to just slop
28:13code into production. So, how do you
28:15actually build up that trust? Takes um a
28:18lot of uh I guess taste and judgment. Um
28:22but uh you know, like I think plugins
28:24like Pstack definitely kind of help you
28:27uh get up to speed much quicker. Uh and
28:30so I guess it's like if you trust me and
28:33you trust PAC then in by extension you
28:36can maybe trust your agents but if you
28:38don't trust me and I I definitely would
28:40not encourage people to blindly trust me
28:43uh uh you know if you build up your own
28:46set of skills that you can obviously you
28:48know take a look at PAC and kind of fork
28:50it make it your own improve the skills
28:53definitely encourage that uh but for me
28:55it's really all about it just keeps
28:57coming back to trust you know every one
28:59of us here in this chat have a different
29:01standard for engineering. Uh and there
29:04are different things that are important
29:05for us in our codebase and uh when you
29:09are able to encode all of that into
29:11skills and you can verify that your
29:13agent is actually doing them that allows
29:15you to really kind of ascend this curve
29:17and um uh you know start automating
29:21things. Uh there's another piece I
29:23wanted to talk about um if there's more.
29:26>> Yeah, go for it. I I'll pick up more
29:27questions as I go. Yeah, I think there's
29:29a third part to this which I haven't
29:31talked about yet which is kind of an
29:33interesting one which is like
29:34refactoring and rewriting. Like one of
29:37the uh I guess most controversial one of
29:40the most controversial topics in the
29:42industry I think is like should you
29:44rewrite your app or not? Um because I
29:47think engineers are very prone to this
29:50where especially when you join a company
29:52you come in and you see like the code
29:53base and you're like oh man this is
29:55like who wrote this code you know it's
29:58terrible I want to rewrite the whole
29:59thing is a very common inclination and I
30:02think a lot of you know before agents um
30:05and I guess arguably even now people
30:07will definitely discourage you from re
30:09rewriting stuff but I'm actually here to
30:12make a case for why you might want to
30:13consider it.
30:15Um because
30:18I think it really depends. Uh you know
30:20uh brownfield applications I think are
30:23actually in a pretty good spot
30:24especially if they're set up well
30:26already. Uh and like recently I've been
30:29talking to some people but uh you know I
30:31I was just observing I I just noticed
30:34this parallel which is that a lot of big
30:38tech company problems are now
30:40everybody's problems. Um, and the big
30:42tech company problem, you know, like
30:44when I was working at Meta, like we had
30:45this giant monor repo, we had like, I
30:48don't know, tens of thousands of
30:49engineers just, you know, like banging
30:52on their keyboards and and shipping
30:53code. And
30:56a lot of really great engineers at Meta.
30:58Uh, but, uh, I'll say like, you know,
31:01you'll be surprised that the code
31:02quality is actually not that good. Um,
31:04and so I often joke that like, you know,
31:07before AI slop, we had human slop. Um
31:10and so uh you know I think a lot of big
31:12tech infra like uh like what Meta has or
31:15Google you know you know really big tech
31:17companies are actually designed for that
31:20where you you're sort of like you're
31:21catering to the the you know like uh
31:24this sounds so bad to say but like the
31:26the least capable engineer on your team
31:28right you build you build frameworks you
31:31build conventions you build guard rails
31:33you know you restrict credentials so
31:35that you know your intern doesn't wipe
31:37your production database Um
31:40there's uh you know if you have that
31:43level of infra already I think your
31:45agents can actually already do a very
31:47solid job right because they have the
31:49the guard rails are already in place for
31:52agents to not cause havoc or not cause
31:55too much havoc uh in your codebase. Um
31:58and you can always add more you know
32:00guard rails. Uh but I think like green
32:02field applications especially are you
32:04know like the brand new applications are
32:06like the biggest risk in my opinion. Uh
32:09and also the greatest opportunity
32:11because you know if you vibe code a
32:13project uh a prototype um like we did
32:16for Grockbot you know Grockbot was spun
32:18up very very very quickly. Um and if you
32:21if you haven't heard of of Grockbot it's
32:23like our a new application we just
32:24launched yesterday. Uh it's it's really
32:27cool. uh lets you orchestrate your
32:30create like individual agents that have
32:32their own identity and you can kind of
32:33orchestrate them. It's super cool.
32:35Definitely check it out. U but yeah,
32:37that was it's like a very it was a very
32:38green field application like most
32:41prototypes are so like vibe coded very
32:43quickly. Humans were not reading the
32:45code at all. And uh I had this tweet
32:49recently uh where I said something about
32:52organic architecture. Um
32:56maybe I'll find it. Uh but the idea is
33:00that uh when you have a completely vibe
33:03coded application, you essentially have
33:05no guard rails whatsoever. So uh your
33:08agents
33:10when you give them a task, they will
33:11just solve it in whatever method is the
33:14most convenient. And over time you get
33:16into this uh situation where you have a
33:20codebase that is spiraling out of
33:21control because you don't understand it.
33:24uh your agents understand it I guess in
33:26a way but like they've built something
33:28that is you know optimized for short for
33:31shortcuts uh and uh you know it will you
33:34will suffer you'll have a lot of of
33:36issues with that application
33:40uh so I think starting your codebase
33:42with uh like very strong constraints is
33:46very much needed uh because like when
33:50you have a codebase that you can trust
33:52right when you have guardrails that
33:53actually help you uh uh help your agents
33:57write good code. You can get into the
33:59you know like into this part of the
34:01curve where I I where like I I said you
34:03know I woke up today and I had like 20
34:05PRs merged u by my agents and that's
34:09because I invested a lot a lot of time
34:12uh over 600 PRs I I I calculated
34:14yesterday uh when I refactored all of
34:18Grockbot to this new architecture that
34:20I've been building. Um,
34:23and yeah, I've gotten to a point where I
34:26I don't really look I really don't look
34:28at the code anymore. And um, I say that
34:31not just, you know, to sell you tokens,
34:33but because I, you know, it it it took a
34:36lot of work to get to that point. I
34:37spent a lot of tokens to get the
34:39codebase to this point where I no longer
34:41have to look at it. Uh, but I'm very
34:44excited because, you know, of the
34:46potential where, you know, it's not just
34:48this doesn't just benefit me. It
34:49benefits everyone contributing to
34:51Grockbot and it also empowers you know
34:54designers and product managers and you
34:57know pe uh even GTM people to add
34:59features to Grockbot and I don't have to
35:02worry you know I don't have to to wake
35:03up at night in in the middle of the
35:05night and worry like oh someone's
35:06just merged a perf regression right I
35:09have a ton of constraints and CI like
35:12it's actually very annoying to write
35:14code in in graph web but like agents
35:16absorb all of that annoyance
35:18Um but yeah, I'm happy to talk about
35:21what exactly that is. Um
35:23>> yeah, I think one question um
35:25>> yeah,
35:26>> before we get into the this part here is
35:28just around that element of like what
35:30your your your CI looks like or maybe
35:32some of the constraints and then also
35:34like the average PR size. I saw a
35:35question about that earlier just to give
35:37people you know kind of a a glance. It
35:39doesn't have to be like mathematically
35:40average but just uh you know like what
35:43generally the size of the a PR is. um if
35:46it's only a couple lines of code or you
35:47know um yeah
35:49>> um
35:51I think it depends uh let me
35:55>> I'm trying to do this in a way where I'm
35:57not going to like
35:57>> you yeah you don't have to share the
35:59actual number like an actual average
36:01>> I think this is fine
36:02>> benchmark
36:02>> but like we have so okay this is not
36:06that interesting but uh well fun fact is
36:08that virtualization in grockbot and in
36:12uh cursor is actually powered by uh
36:15Pretext uh which is a sort of new
36:18library that someone's built. Um that's
36:21really interesting. You should you
36:22should check it out, but that's not
36:24really that important. Uh I think the
36:26average PR size I actually don't know. I
36:30I don't know if I want to click on
36:31these. Uh I probably can, but I would
36:34say like they can range anywhere from a
36:36few hundred lines or 50 lines to like a
36:39thousand depending on what the thing is
36:41doing. Uh, so like here I'm actually
36:43like deleting a bunch of files. So I
36:45expect that it's just like mostly
36:47deletion. Uh, but yeah, it kind of
36:49varies.
36:51>> There's no like Yeah,
36:52>> there's no like hard cap or hard limit.
36:54Are they're all like 50 line PR?
36:55>> There's no hard cap. Yeah, there's
36:56definitely no hard cap, but I I do
36:57encourage my agents to split up their
36:58work into multiple PRs. Uh I do that
37:02mostly because uh I like I like the idea
37:06of the I guess maybe this is much harder
37:08to do now as in the world of agents and
37:10you have like so many commits but I like
37:13the idea that you know the git history
37:14is a very rich source of context. Uh,
37:17and I like the I like each PR to sort of
37:20atomically describe what that small
37:23piece of thing is doing, which also
37:26makes it easier for me to revert changes
37:27and like figure out, you know, oh, I
37:29shipped a bug and it's just it's here,
37:31right? It's not in this 40,000 line PR
37:34where you who knows what landed in
37:36there.
37:37Uh, but I don't have a hard cap on PR
37:41size.
37:42>> Cool. And then um yeah, also quick
37:44question on like CI. So again, you don't
37:46have to go into like uh the screen share
37:47of like your CI does, but just generally
37:50would you describe what the CI kind of
37:52looks like uh or how strict it is?
37:55>> Uh yeah. So uh well specifically for
37:58Grockbot. So Dune is the is the sort of
38:01cheeky code code name for the
38:03architecture that we've built for
38:06Grockbot. Um the CI looks pretty
38:09annoying because there's checks for
38:12everything. So like literally I have um
38:16uh well if you've written any react for
38:17example you know you know that one of
38:19the biggest foot guns in react is use
38:21effect. Uh so in
38:24uh Dune and in graphbot we've banned use
38:27effect. So Dune is just you can the the
38:29mental model of what Dune is uh you can
38:32kind of think of it as like Nex.js JS
38:34for uh electron apps and it's designed
38:37for agents to write uh and it's like
38:39custom for you know our agent powered
38:42applications. Um so the CI checks are
38:45very like specific to that like you know
38:47don't use use effect it's it's it's
38:49banned like CI will fail uh and yell at
38:52you. We have like some of the more
38:54interesting ones that people might raise
38:56eyebrows is like I actually ban code
38:58comments as well uh which is very
39:00interesting. Uh, but I've noticed that
39:0499% of the time agents just write code
39:07comments that kind of describe some
39:09historical thing that is actually
39:10totally irrelevant to the code. Um, like
39:14it will often say like, you know, oh,
39:15Lauren said you should never do this and
39:17it's now in in a code comment. I'm like,
39:18what? Like why? What? That was I didn't
39:21say that as like a durable, you know,
39:23global rule. I just meant like your this
39:25PR sucks and you should change that
39:27part.
39:29agents don't really understand us that
39:31well surprisingly uh and or they kind of
39:34assume too much and they kind of do
39:36things in like very stupid ways. So like
39:39yeah we just ban everything everything
39:41you can imagine like the agents are bad
39:43at we ban. Uh so one example that we
39:47actually suffer a lot in the agents
39:49window is we have uh you know you if
39:52you've used the agents window you've
39:53definitely seen performance issues and
39:55you know we're constantly trying to fix
39:56them. Uh but it's like a it's a never-
40:00ending struggle because there's so many
40:02pull requests that get merged. Every any
40:04one of them could just regress
40:06performance or stability or reliability.
40:08Uh you know the agents window doesn't
40:10have this architecture yet. I plan to do
40:12bring this learning back there and kind
40:14of refactor everything there. uh but uh
40:18it just regresses super often uh because
40:21uh there's just one example is like we
40:24have very poor um isolation between
40:27processes. So like on you know on on
40:29Electron you have a renderer thread that
40:31renders your UI but you also have like a
40:33main thread that you can run other code
40:35that you know doesn't need to block the
40:37renderer.
40:39Um, but we do a poor job of separating
40:41those things and so often times you just
40:43accidentally have code that gets pulled
40:45into running on the renderer thread and
40:48then all of a sudden you're competing
40:49with the the renderer that you know that
40:52has a very if you want like 60 fps you
40:54have to every frame that gets drawn has
40:57to be done in 16 milliseconds. So very
41:00very small you know deadline per frame
41:02uh if you want you know a very smooth
41:04product. Uh and when you start building
41:06bringing in accidentally bringing in you
41:08know things that are like very
41:10computationally heavy or they have a lot
41:12of IO uh then you just get into like a
41:15lot of jank your FPS really drops. You
41:18start uh you know losing frames. You get
41:21long tasks that take more than 16
41:23milliseconds and you just get this
41:24really choppy experience.
41:27So all of those patterns that we've
41:28learned basically building electron apps
41:30we've encoded into this framework and it
41:33becomes like a hard failure. So I
41:35literally in in grabbot we literally
41:37have a directory called electron main
41:39electron renderer and we have a import
41:44uh CI guess where we actually check the
41:47dependency graph to make sure you're not
41:49accidentally importing code from one
41:51directory to another. Uh so that's
41:54enforced by CI um as well as bug bots uh
41:58which is our which cursors um like code
42:01review tool that runs on CI uh you know
42:04in our agents MD it's everywhere like so
42:06I I I I um I have this thing here where
42:10I I talk about like um you know like
42:13there are multiple layers I think for
42:15building a good codebase. Uh obviously
42:18the codebase is one where uh if you have
42:20an architecture like this where it's
42:23extremely strict uh you know the the the
42:26way to build features is very
42:27conventional that's like the strongest
42:29strongest level of enforcement because
42:31agents just love to copy existing
42:34patterns. So uh one example of this in
42:37Rockbot is like we have this these
42:39concepts called like a feature and we
42:41have entry points and transcript cards
42:43like oh you know the cards that you see
42:44in the chat these are all like like
42:48nouns I guess in in the framework and so
42:50there's a very conventional way of
42:52creating them and so like a feature is
42:55all in in a single directory as an
42:57example and so all of the code that
42:59contributes to that feature lives in one
43:01directory so it's all collocated in one
43:03place makes It's super easy. You know,
43:05agents don't have to like uh grap around
43:07and try to figure out like where all the
43:09things are. It just looks at the feature
43:11and like, oh, okay, I'm working on the
43:13onboarding feature in Grockbot. Uh, I'm
43:16just going to work in this directory.
43:18And for 80% of the work, it's mostly
43:20just very encapsulated there.
43:23[clears throat] But, uh, like it's like
43:25very it's like designed again for you
43:27know like the dumbest agent like you
43:29don't have to think, right? the the the
43:32one of the key principles I have for
43:34this framework is like the shortest the
43:36shortest path is the best path.
43:39So uh because that plays exactly to how
43:42agents love to write code is like they
43:44like to take shortcuts really you know
43:46they they'll find the quickest way to
43:48solve the problem. So why not make that
43:51the best way to solve the problem? Uh so
43:54I I probably won't get into all the
43:56specific details. Um and uh the the this
44:00framework is really more of a collection
44:01of ideas and principles rather than
44:03something that will open source. Uh you
44:05can you can you know screenshot this I
44:07guess if you want and uh tell your agent
44:10to uh do some build build something like
44:13this for you too.
44:15Um yeah, but it's really all about the
44:17layers uh you know like the the codebase
44:19is one part with features uh and
44:22directories and you know import or
44:24blocking import dependencies uh that
44:27shouldn't be imported uh but and and it
44:30all enforces that and static analysis.
44:32So like uh there's CI checks, we have a
44:35lot of lints for bad patterns that we
44:38observe. uh compiler diagnostics
44:41uh there's also rules and bugbot which
44:43are um I think like three four five are
44:46more soft right these two actually make
44:50make CI red right so that you know
44:53there's a hard constraint where the
44:54agent can't just write crappy code
44:58for rules and skills and bogbot your
45:02agents can still forget right you can
45:04still or it may not always consistently
45:07apply them So, I like to layer them, but
45:11I don't I don't like to rely on them as
45:13the only source of enforcement because
45:16it's very very soft, right? And if you
45:18if you only have rules and bug bar and
45:20skills and a style guide for your code,
45:22you will it's only a matter of time
45:24before your codebase looks like complete
45:26trash. I'm sorry to say that, but uh I
45:29definitely recommend yeah like you know
45:31investing in you know things that can be
45:33hard and forced, right? And this is why
45:36you know maybe uh the choice of text
45:38stack that you use is also very
45:40important. Um like I think for example
45:43Rust is sort of making you know it's
45:45like getting super popular again. Uh
45:48because the compiler is so strict right
45:50the compiler enforces so many different
45:52things. you know, there's a borrow
45:54checker that you have to appease and if
45:56as long as you make sure your agents
45:57don't write unsafe code blocks, uh you
46:00can more or less feel somewhat confident
46:02that if the code compiles, it's probably
46:04works and it's good. Uh but you see, it
46:07gives you that level of trust and
46:09confidence that you as a human engineer
46:13no longer need to go and check it
46:15yourself. you know, you you rely on code
46:18and static analysis to actually make
46:21that uh a lot smoother. Um, and I I
46:26guess the worst part, the worst place to
46:28be in is if you are stuck in code review
46:31land where you actually enforce all of
46:33the constraints, the invariance in your
46:35codebase by literally the human person
46:39saying, you know, reading the code and
46:40like, okay, you should not do this,
46:42right? Every time you have to do that,
46:44you should consider that as a code
46:45smell, like a anti- pattern and you
46:48should say, "Okay, instead of me
46:49commenting on the PR, how do I turn this
46:52into a hard rule, right? How do I turn
46:55this into a lint rule? How do I turn
46:57this into a CI failure? Or how do I even
46:59categorically eliminate this problem uh
47:02entirely?" Uh I I can talk about another
47:06migration I've done, but I'll probably
47:07pause here.
47:08>> Sure.
47:09>> Yeah. I feel like that's that's where I
47:11am to be honest. is is what you're
47:12describing right now, which is that like
47:14I don't have all of these rules. So, I
47:16have some things to go do after this
47:17session in terms of being able to scale
47:19my agents. I'm I'm definitely on like
47:21the uh you know, maybe a couple of
47:23parallel ones locally stage. So, like
47:25two to three locally, and I'm sure most
47:27people here are on the same. So, uh
47:29yeah, I know we only couple minutes
47:31left. Lauren, was there anything else
47:32that you wanted to to highlight?
47:34Obviously, there's lots of questions, so
47:35I can grab more, but I want to give you
47:37a few minutes if there's anything else
47:38you want to talk Well, I think I've been
47:39yapping for quite a lot, so I'm maybe
47:41let's just do questions.
47:43>> Okay, cool. Uh, one question that had uh
47:45a couple of uh came up a couple times
47:46was just around like token usage.
47:49>> So, the the question is like is is what
47:52you're describing a realistic thing for
47:53people who are on, you know, uh a normal
47:57set of token usage. They don't have, you
47:59know, basically unlimited tokens uh to
48:01work with.
48:03>> I think that's a really good point. I
48:04mean like obviously you know I work at a
48:06AI lab where we have unlimited tokens.
48:09So uh I definitely cannot
48:13say that you know this is something
48:14everyone should do in the exact same way
48:17that I did it. I think it's possible to
48:19get to this point without you know
48:20breaking the bank.
48:22But you know if you're like an
48:24engineering leader or you know you're
48:25you have a startup that you lead um I
48:28think to me it's a question of ROI. Um,
48:31and it's like, uh, yes, you spend a lot
48:35of money on tokens in the upfront stage,
48:38you know, like refactoring a codebase is
48:40going to take a lot of tokens. Uh,
48:42adding all these things uh, is going to
48:44take a bunch of tokens. But if we're
48:46heading to a world where agents are
48:48writing all the code and, you know, you
48:51want to be very lean, right? You don't
48:53want to have to hire, you don't want to
48:54be, you don't want to become like meta,
48:56right? Like I mean like in terms of you
48:58don't want to become a 10,000 person
49:00engineering org because I mean that's a
49:03cool problem to have but also you have
49:05so much overhead. There's like planning
49:08you know like you it's it's a personally
49:10I I wouldn't uh it it's not super fun
49:14but um I think you want to stay very
49:16nimble right and you want to you want to
49:18be like agents are all about allowing
49:20you to do things that you couldn't do
49:22before. That's really to me like the
49:24value of agents, you know? It's not just
49:27storing tokens on every single little
49:28thing, but um to me like the thing I
49:31couldn't do before is like enforce this
49:34level of constraints in a codebase by
49:37myself, right? Like I'm just a single
49:39person, you know? Uh it would have taken
49:41me years to build this framework uh and
49:45do all the refactoring and test
49:48everything myself and verify, you know,
49:50like run imagine if there it was just
49:52me, right? you know in in pre- agent era
49:54just like running you know by it would
49:56take me so long right and my salary is
49:59pretty high right like so you know the
50:02the question I think an engineering
50:04leader might have is just then you know
50:06like what is there's a trade-off of do
50:08you hire someone to do this or do you
50:11spend the tokens to set up a code base
50:13so that even the the most naive right
50:17the dumbest agents can do a good job and
50:20when you actually get to this point like
50:22even agents that are not you know fable
50:24size do an excellent job of writing code
50:27and this pays a lot of dividends as well
50:29for me personally where I've empowered
50:32not just myself but again like PMS
50:36designers engineers who are not familiar
50:38with Grockbot to just contribute in a
50:41way that is sustainable
50:44so I think yeah it's definitely like a
50:45trade-off for sure you know like nothing
50:47is like free for sure uh and tokens are
50:50pretty expensive Uh but oh actually uh I
50:53I I don't know how many of you have seen
50:55this but we actually announced Grock 4.6
50:58today. So very exciting finally out. Um
51:01so yeah, GRO 4.6 would be like a great
51:03it was very very smart. Uh it's really
51:06good on the on the benchmarks. Uh and
51:08it's the same the tokens uh well uh I
51:11hopefully I'm not saying this
51:12incorrectly, but uh I believe the cost
51:15per token is the same as 4.5. So you're
51:18actually getting more intelligence for
51:20the same cost. Uh I think this is an
51:23area that cursor tries to cursor and
51:26SpaceX AI try to really optimize for
51:29like that heredto frontier of you know
51:32cost versus intelligence. Uh you know we
51:34don't necessarily want to build the
51:35biggest model ever because that is
51:38extremely expensive to run. It's really
51:40about like how do you find that sweet
51:42spot right? you don't you don't need a
51:43giant model, but it's just super smart,
51:45right? And it's not very expensive for
51:47inference.
51:49Uh but um yeah, I think to kind of round
51:52it up, um I think it's like a it's it's
51:55there's a if you do your own analysis, I
51:57feel like it's pretty positive. It it'll
52:00be pretty positive that the ROI you get
52:03from investing in stuff like this uh
52:05just empowers not just yourself, but
52:08your whole team to be so much more
52:10productive, right? Like imagine if you
52:11have an army of engineers like me who
52:14are shipping so much improvements and
52:16and bug fixes uh you know every day,
52:19right? Like that is pretty exciting.
52:23>> Cool. Uh one last question before we
52:25wrap up. This one is for the people in
52:27product on the on the call.
52:28>> So let's say we do have an army of
52:30engineers who are shipping like Lauren.
52:32I'm just curious like how is the product
52:34team or other functions of your company
52:36keeping up given that like if you're
52:38shipping so quickly, have are they using
52:41AI more to do their jobs? Like as much
52:43as you can speak to that obviously you
52:44don't have like you're not in that role
52:45but just curious about how that works.
52:48>> Um I think this is where grathbot has
52:50been actually exceedingly powerful. uh
52:53where so before graphbot like you know
52:56uh obviously cursor only had cursor like
52:58we only had agents window we had a CLI
53:01we had an IDE and these are really like
53:04power user tools right like de they're
53:06designed for developers so it's very
53:08very developer centric you can do
53:10knowledge work in them but it like the
53:12UI is not really optimized for that so
53:16we actually didn't really have uh well I
53:18think like a lot of people like you know
53:19GTM product like they might have use
53:22cursor uh to do their work but it
53:25definitely wasn't like a delightful
53:26experience for them. Um I think now with
53:29grabbot
53:31uh it's become Grockbot is basically
53:34like the kusher moment for people who
53:36are not in tech in my opinion like it's
53:38like it's like a very very accessible
53:41way to use agents in a very comfortable
53:44very familiar interface. It looks like
53:45iMessage. Um, and it's very fun to, you
53:49know, you can give your agent a fun
53:50name. Uh, you can have you can kind of
53:53do orchestration with in a very like
53:55natural way where you can sort of, you
53:57know, each agent's like a person, right?
53:58Now, you got a team of agents like
54:00working on you have one one agent per
54:01account that you manage as an example.
54:03Or if you're a PM, you have, you know,
54:05you can have an agent that summarizes
54:07all the work that Lauren did last night
54:09and then now you know what I did, right?
54:11So I think our PMs are leveraging that a
54:13lot and they're shipping code too. Uh so
54:16you know like often times they will just
54:18say oh here's a bug I fixed it can you
54:20look at it and then I'll go review it
54:21and actually it's just perfect. I'm like
54:23okay stamp. Uh so uh that I think that
54:26shows that you know the the Dune
54:28architecture is holding up right the all
54:31the the really strict constraints allow
54:34people who are not experts in
54:35engineering to contribute at a high
54:37level. Uh so I'm I feel like I'm already
54:39seeing that pay off a lot where uh you
54:42know designers and PMs are just able to
54:44to to ship features directly. Um and
54:48that just makes the Grockbot team super
54:52fast, right? Where we can ship so
54:54quickly. Um and we have a lot planned.
54:57So, I'm very excited uh to, you know, uh
55:01to to ship more ship more

About this transcript

This page contains the full transcript of Lauren Tan - SpaceXAI engineer by Vision Sabbat, generated from the public captions YouTube serves with the video. The transcript has 10,321 words across 1,400 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.