AIE CODE: Google DeepMind, Anthropic, SpaceXSI, TypeSafe, Lovable, Ramp, etc (System One theme) — Transcript
Full transcript
- 0:521
- 2:56>> Good morning and welcome to AI Engineer
- 2:59Code Summit 2026.
- 3:02We're excited to bring together some of
- 3:04the brightest minds in AI engineering
- 3:07for a day of fresh ideas, new
- 3:09perspectives, and conversations shaping
- 3:12the future of how we build software. To
- 3:15officially kick things off and lead us
- 3:17through today's keynotes, please put
- 3:19your hands together for our MC, Corey
- 3:22Quinn, Chief Cloud Economist at the
- 3:24Duckbill Group.
- 3:32>> [music]
- 3:43>> Thank you all for coming. This is the
- 3:45last flagship event of the year for AI
- 3:50engineering.
- 3:51How are folks feeling so far?
- 3:55Yay, that's exciting.
- 3:57What have we learned? Anything good?
- 3:59Anything exciting? Anything terrible?
- 4:03>> Ontology.
- 4:04>> There we go. That's right, ontology.
- 4:06What does it mean? We're going to put a
- 4:07pin in that, and we're going to come
- 4:08back to it. I Corey Quinn. If you've
- 4:11never heard of me, you're A, what we
- 4:12tend to think of as being happy, and B,
- 4:15uh if you are curious, you can look at
- 4:17all the various things I do at my site,
- 4:19shitposting.ai,
- 4:21which in itself tells you everything you
- 4:23need to know about what I do.
- 4:25So today we are going to be kicking
- 4:28things off and having some fun talks
- 4:31with people who have interesting, useful
- 4:33things to say rather than me flopping my
- 4:35gums up here. Uh we're going to start by
- 4:38bringing up our friends from Swix.
- 4:41Swix, are you available yet, or are you
- 4:43still hiding back there? Terrific. Come
- 4:44on out, please. And Leah as well.
- 4:48Oh, did I jump the gun?
- 4:52Oh, no, please, by all means.
- 4:54Swix, thank you. [applause]
- 4:57Thank you.
- 5:00Oh.
- 5:02I'm in. Uh it's so nice to have him MC
- 5:05as well. And give a warm welcome to
- 5:07Leah, General Manager of AIE.
- 5:08>> Hi, morning.
- 5:10>> Um
- 5:11[applause]
- 5:11uh yeah, you know, we we're so happy to
- 5:13to open the show for you. And hi to
- 5:15everyone watching along online. AIE code
- 5:18is
- 5:19kind of our vertical, very focused,
- 5:21smaller format
- 5:23you know, format on on
- 5:26coding agents. And I think it's been
- 5:27such a huge year in coding agents.
- 5:29There's also been a lot of different
- 5:32evolutions in terms of like how we think
- 5:33about what the mission of coding is. We
- 5:36used to have
- 5:37you know, in in specific track titles
- 5:39like coding agents and software
- 5:40factories and all these things, but now
- 5:41I'm talking about like system one
- 5:43thinking and and
- 5:45raising your ambition and all these like
- 5:47very fancy higher level abstractions of
- 5:49of work. So I'm excited to see for you
- 5:51to see and excited to me personally to
- 5:54attend all these talks that are coming
- 5:56along.
- 5:57>> Um and I'm just going to talk a little
- 5:59about 2026 um with AIE. So we've had a
- 6:02really big year this year. We had our
- 6:04first event in London. Uh, we also
- 6:06graduated up into Moscone for our
- 6:08World's Fair.
- 6:09Um, we also had a bunch Oh, here we go.
- 6:11I I didn't know if it was minus. Um, we
- 6:14uh we had a
- 6:15um multiple partner events, which if any
- 6:17of you went to, uh we'd love to hear
- 6:19feedback there cuz it's a great program
- 6:20program that we're running. Um, we've
- 6:22also grown our team um substantially. So
- 6:25I think there was a slide somewhere.
- 6:28Oh, I don't know. I think they've
- 6:30>> Oh, there we go.
- 6:31>> Um,
- 6:31>> people.
- 6:32>> I think
- 6:33>> They're so much more excited about
- 6:34clicking through everything.
- 6:35>> Um, yep, we uh here we are. So uh
- 6:39there we go. So yeah, we've grown our
- 6:41team. We went from um four in the team
- 6:43to 20 this year, uh which was very
- 6:45exciting. So this is everyone. Big shout
- 6:48out to the team. Um, as as you can see,
- 6:50the program is growing and um everyone
- 6:52here is responsible for that. Um, so
- 6:55yep.
- 6:55>> And I I'll slowly interject like why are
- 6:57we growing? It's because we want to
- 6:58provide a consistent product for the
- 7:00rest of you uh that like wherever in
- 7:02inside the major cities that you're
- 7:04working in, uh you have an AIE to go to.
- 7:06Um, and we try to spread out. You know,
- 7:08we're doing something like four flagship
- 7:10conferences a year and a lot of partner
- 7:12conferences around the world.
- 7:13>> Yep.
- 7:14Um, we also I think I just saw that. Um,
- 7:16our most popular talk, which just came
- 7:19up there, uh has almost 2 million views
- 7:21on YouTube. But point's not coming up.
- 7:24[laughter]
- 7:25And then we are also just about to
- 7:27announce 2027 um with a wider program.
- 7:30We're going to hopefully announce that
- 7:31in about 3 weeks time. And we are doing
- 7:33more partner events. We're entering into
- 7:35some different regions.
- 7:37It's very exciting for 2027. Um, a
- 7:39couple other things we've done this year
- 7:40is we've partnered with Major League
- 7:42Hacking um and they're going to be
- 7:44running a hackathon with us at every
- 7:46event next year. Um, we actually have
- 7:48one this Friday. So if you guys haven't
- 7:50signed up and you want to come, uh
- 7:51please look on the website.
- 7:53Uh, we are also working with Neo 4J on A
- 7:55Kids event series also. Um, and that is
- 7:58happening this Saturday. So if you have
- 8:00got any kids between the age of kind of
- 8:026 to 14, uh feel free to come down.
- 8:05Again, you can find that on the website.
- 8:06>> Or
- 8:07>> aie.kids.
- 8:08>> Uh,
- 8:08>> I'm in my domain buying era.
- 8:10So
- 8:11we got aie.kids for kids and aie.sc for
- 8:13the schedule.
- 8:14>> Yeah.
- 8:15Yes. So and then we have one final event
- 8:18for the year, which is the Sydney event.
- 8:20Swix will be there. I will not. And that
- 8:22closes out 2026. So yeah, thank you so
- 8:24much for coming everyone. And we hope
- 8:26you enjoy the the next couple of days.
- 8:28>> All right.
- 8:29>> Thanks.
- 8:30>> Enjoy the show.
- 8:31>> [applause]
- 8:49>> What a great way to get us started.
- 8:52What we have found here is that events
- 8:55like this don't happen without help from
- 8:58a lot of our partners. It is incredibly
- 9:01important that they are there and
- 9:03support us. We want to thank them for
- 9:04it. Beginning with our presenting
- 9:05sponsor, Google DeepMind. Can we get a
- 9:08round of applause for them, please?
- 9:10[applause]
- 9:16Oh, there they are.
- 9:19AIV challenge is always fun and exciting
- 9:21on that one.
- 9:22Stall, it's great.
- 9:25I also want to thank our supporting
- 9:26partners as well, without whom we would
- 9:28again not be able to be here.
- 9:32Normally it shows up there. Oh, there,
- 9:33sorry. Wrong slot. I looked left. It's
- 9:35right. Can we get a round of applause
- 9:36for them as well? Thank you all for your
- 9:38support. It's appreciated.
- 9:40[applause]
- 9:49[laughter]
- 9:50Yes. And of course the gold sponsors as
- 9:51well, which without whom again we would
- 9:54not be able to be here. At least not in
- 9:55this venue. We'd basically be yelling
- 9:57out on a street corner at people, which
- 9:59while not unheard of is generally not
- 10:01the premiere experience you try to have
- 10:03for something like this.
- 10:05Our silver sponsors also. Thank you for
- 10:07them as well. One more round of applause
- 10:08for them, please.
- 10:10[applause]
- 10:14One of the hard things about dealing
- 10:17with the modern era in AI is it feels
- 10:21like this year has been about 10 years
- 10:23in technology. We look back at what we
- 10:25were doing in January, and it feels like
- 10:28it was almost medieval times. It The
- 10:31things that were best practices and made
- 10:32sense then now just seem absolutely
- 10:37bizarre. It's Things get moving so
- 10:39quickly, and it's easy to basically take
- 10:41a nap and feel like you've woken up and
- 10:44gotten like surprised by what happened
- 10:47while you were sleeping. Everything's
- 10:48moving faster.
- 10:50And that is the theme of our first talk
- 10:53that we're going to have cuz you're not
- 10:54here to listen to me ramble. Um Natalie
- 10:57Serino is the co-founder of Gimlet Labs
- 11:01and is going to talk to us now about
- 11:04what happens when AI becomes 10 times
- 11:07faster.
- 11:08Natalie, thank you very much.
- 11:10Please come on out here.
- 11:26>> Hello, everyone.
- 11:32It's so good to be here. Thank you for
- 11:33being with me this morning. First
- 11:35session of the day. I see everyone here
- 11:37is a morning person.
- 11:38So I'm here to talk to you about what
- 11:40will you build when AI is 10 times
- 11:43faster?
- 11:44We're all builders here, so we're all
- 11:45thinking about what's next.
- 11:48Just a bit about me. Um my name's
- 11:50Natalie. I'm a co-founder of Gimlet
- 11:51Labs.
- 11:52At Gimlet, we're building an inference
- 11:54cloud that's designed to produce much,
- 11:56much faster tokens at high efficiency,
- 11:59leveraging different categories and
- 12:00types of chips.
- 12:02At Gimlet, I work on our kernel
- 12:04generation side of things across
- 12:06different hardware types.
- 12:07Um another thing about me is I love
- 12:10agents. I use my coding agents for
- 12:12everything, but I really, really hate
- 12:14waiting.
- 12:17And one thing that given that kind of
- 12:19surprises me is when you're building an
- 12:20ultra fast inference cloud, people ask
- 12:22me this all the time actually.
- 12:24like isn't inference already fast
- 12:26enough?
- 12:28And I'm like what?
- 12:30But they have a point, right? Because I
- 12:32can do things and we can all do things
- 12:34with our agents so much faster than we
- 12:36could have done before. For example,
- 12:38this slide here, I actually made pretty
- 12:41using my agent and it took me with my
- 12:44agent much less time than it would take
- 12:46me to do that by myself. I'm not very
- 12:48good at design.
- 12:50And so I think that from an end-to-end
- 12:53perspective, obviously we're getting
- 12:54more done, but in terms of is it fast
- 12:57enough? Like that's still a question. So
- 12:59let's take a look at what happened when
- 13:01I used my agent to make that slide look
- 13:03pretty. You can see the ugly before
- 13:05slide on the right.
- 13:08So we're going to press play here.
- 13:12And I said to my agent, can you improve
- 13:14the visual look of this slide?
- 13:16I love Figma. I use Figma for
- 13:17everything, so of course I'm going to
- 13:19use it with my agents. So what we can
- 13:21see on the bottom here is the timeline
- 13:24of what happened. And the text is a
- 13:26little small to see, but red is
- 13:28inference. Red is the time spent that
- 13:29the model's in inference. Green is the
- 13:31time it's spent making tool calls and
- 13:33blue is the time that it's spent taking
- 13:35my input.
- 13:37And we can see that, you know, this this
- 13:39agent's doing good work. It's doing a
- 13:41lot of thinking, but I'm waiting here.
- 13:44I'm actually waiting so long I sped up
- 13:45this video 9x cuz I didn't want all of
- 13:48you to have to wait for the full turn to
- 13:50end.
- 13:50It ended up being about 2 minutes long.
- 13:53But we can see at the end of this after
- 13:54my one input, we have this nice pretty
- 13:56slide at the end. So, you know, on one
- 13:58hand this is actually much faster than a
- 14:00human can do. But on the other hand, in
- 14:02those about 2 minutes I have probably
- 14:04moved on to another task. I'm not really
- 14:06focused on this anymore. I've contact
- 14:08switched.
- 14:11And when we look to, you know, the
- 14:13inevitable progression of inference
- 14:16getting faster faster, I think it helps
- 14:18to look back in time and look back to
- 14:20the internet era.
- 14:21Um, I mean, we're still in the internet,
- 14:22but like, you know, what people
- 14:23canonically call the web era. And what
- 14:26we always see with a new shift is that
- 14:28with each new major wave, new
- 14:30capabilities are unlocked. And so we do
- 14:32things that weren't possible before that
- 14:34technology, like using an agent to edit
- 14:36my slide.
- 14:37So I think we should take a step back to
- 14:392000 in the early days of the internet.
- 14:41I think many people here weren't even
- 14:42alive, so it'll be a history lesson for
- 14:44you.
- 14:45And we're going to look at a little
- 14:47application called Squirrel Mail.
- 14:49Squirrel Mail is actually open source,
- 14:50so you can run it today. It was a web
- 14:52application for you know, doing your
- 14:54email back then.
- 14:56So on the left, what we have here is I'm
- 14:58running Squirrel Mail and I'm running it
- 15:01with simulated 2000 dial-up speed.
- 15:05And then on the right, it'll run with
- 15:062026 speed, so the speed that we're used
- 15:08to today.
- 15:10So the both sides are going to take the
- 15:11same actions, but
- 15:13the left side I'm using it with that
- 15:16simulated lower
- 15:17speed internet.
- 15:21And we can see that, you know, it's
- 15:23doing its job.
- 15:25Back in 2000, loading an image like this
- 15:27would have blown people's minds, but our
- 15:29modern eyes are like, wow, that took
- 15:31forever to show up.
- 15:34And so we can see these lags. We can see
- 15:35a noticeable thing, but on the same
- 15:37time, someone might have looked at this
- 15:38back in 2000 and said, this is fast
- 15:40enough. I don't actually need this to be
- 15:42faster.
- 15:46The thing is though, all of the major
- 15:48giants back in that era did find out it
- 15:51mattered.
- 15:52Pretty much every major company from
- 15:54Amazon to Yahoo to Google to Bing, all
- 15:57of or Bing at part of Microsoft, like
- 15:59all of them found that even very small
- 16:02delays cost them users. It cost them
- 16:05people coming back. It cost them people
- 16:07doing as many clicks. It cost them
- 16:09literal money. Even 100 milliseconds of
- 16:11delay added on to the typical experience
- 16:15had a demonstrable impact on the bottom
- 16:16line.
- 16:18And even when you got rid of that delay,
- 16:20those same users didn't come back to the
- 16:22platform as often as they did without
- 16:24that delay.
- 16:29And this makes sense because fast
- 16:31products keep us in flow state. They
- 16:33provide a stickier UX.
- 16:35About 100 milliseconds feels instant to
- 16:37us.
- 16:381 seconds 1 second of wait, I still keep
- 16:41my train of thought. It's not a big
- 16:42deal. I don't really notice it.
- 16:4410 seconds, we're going to start turning
- 16:46to something else from a psychology
- 16:48perspective. Maybe we pull out our
- 16:49phone, maybe we switch to another agent.
- 16:51But today, from data from June, the
- 16:53median coding agent turn is 63 seconds.
- 16:59Each workload has different speed
- 17:01requirements. We don't need the same
- 17:03speed for everything, but over time our
- 17:05expectations do rise. And the fastest
- 17:07days of fastest internet in the squirrel
- 17:10mail days would feel incredibly slow to
- 17:12us, and we can keep we can expect that
- 17:13trend to continue for inference speed as
- 17:15well.
- 17:18There's a problem though, and the
- 17:19problem is everyone knows that we have
- 17:21massive compute scarcity.
- 17:23Today's infrastructure is under
- 17:25incredible pressure to deliver two
- 17:27things at the same time. More tokens and
- 17:29faster tokens.
- 17:31Unfortunately, these goals are
- 17:33fundamentally at odds because faster
- 17:35tokens require more energy to produce
- 17:38than slower tokens. So with the same
- 17:40energy footprint, you can't make as many
- 17:41of them.
- 17:44Back before this was a session and not a
- 17:47keynote, thanks to upgrading this to a
- 17:49keynote, everyone, um we uh we're going
- 17:52to play a game that I made uh called
- 17:55Token Mint Factory. And you can actually
- 17:57play this. I will not be offended if you
- 17:58play it right now even, but just so you
- 17:59know, it does not work on phones. It
- 18:00only works on your laptop.
- 18:02And basically in this game, you're
- 18:04running a Token Mint Factory. You're
- 18:05printing literal tokens. And there's
- 18:07different modes. The modes are you can
- 18:09play to maximize how quickly you can put
- 18:12out one single token or you can maximize
- 18:14how many tokens you can put out in 60
- 18:16seconds. Or you can play the SLA mode
- 18:19where you have to maintain a 20 second
- 18:21SLA on each token for it to count, but
- 18:23you're still trying to put out as many
- 18:24as you can.
- 18:25And as you play, you'll quickly see that
- 18:27the strategy is extremely different for
- 18:29doing fast tokens versus doing slow
- 18:31tokens. And neither one is actually
- 18:33better or worse from a decision
- 18:35perspective. It's just a matter of what
- 18:37the application needs and what makes
- 18:38sense for your factory.
- 18:41So I took real runs from people at my
- 18:43company that played this game and I made
- 18:46a Pareto frontier of these tokens in the
- 18:48game.
- 18:49And we can see there's a real life
- 18:51Pareto frontier of this game as well.
- 18:53If we have faster tokens, we don't have
- 18:55as many. And if we have more tokens,
- 18:57they're not as fast.
- 18:59But there is a way to get more up into
- 19:01the right. And up into the right is
- 19:02where we want to go. We want more faster
- 19:04tokens. I don't want to compromise. I
- 19:06want both.
- 19:08So the game has a concept of adding
- 19:09upgrades to your equipment.
- 19:11And what we can see is that compared to
- 19:13the no upgrade baseline, these
- 19:15technological innovations help drive us
- 19:18up into the right and get more tokens
- 19:19and faster tokens.
- 19:23So how do we upgrade our real life
- 19:25assembly lines, so to speak, to make
- 19:28making our actual tokens faster?
- 19:31Uh one thing that we do at Gimlet for
- 19:33this is we really believe that you have
- 19:35to use the right accelerator for the
- 19:36right job. And so we actually take
- 19:39models and we split them up into
- 19:41different pieces and we run each piece
- 19:43on a different category of accelerator.
- 19:45So we might run one piece of the
- 19:47workload on a GPU and another piece of
- 19:49the workload on something like an SRAM
- 19:51centric accelerator. And we find that
- 19:53even with the network overhead, this
- 19:55delivers dramatically faster tokens.
- 19:57It's essentially like the same thing as
- 19:59the upgrade to the equipment in the
- 20:00token factory when you better match the
- 20:03accelerator to the job.
- 20:04Unfortunately, we have ended the time of
- 20:07free lunch transistor scaling, right? So
- 20:09now we have to look to other approaches
- 20:10to make things faster.
- 20:14And so with this, it's a little hard to
- 20:16read, but we can see the same real life
- 20:18Pareto frontier for a frontier model.
- 20:21If you look at, you know, simple uh
- 20:23homogeneous GPU inference, it's very
- 20:25high throughput, and then the latency
- 20:26drops off a little bit. You know, you
- 20:28can't get it as fast cuz the y-axis is
- 20:30how many tokens that I can make, and the
- 20:32x-axis how how fast these tokens are for
- 20:34each user.
- 20:36And we can see that depending on how you
- 20:37split the model up, because there's
- 20:39different ways of doing it, you can get
- 20:42much faster tokens for the same power
- 20:44footprint.
- 20:45You can get many more tokens for the
- 20:47same speed,
- 20:49or you can get more and faster tokens.
- 20:52So as an industry, we need to continue
- 20:54pushing, pushing, pushing on this
- 20:56because otherwise our poor data centers
- 20:57are just going to keel over.
- 21:01So I just wanted to show a quick demo of
- 21:03this. So um using a coding agent, build
- 21:06the same app using heterogeneous
- 21:08inference that leverages different chip
- 21:09types versus the traditional one.
- 21:14So this is going to be the game Snake,
- 21:16and we can see on the left the
- 21:17heterogeneous fast one's done. I can
- 21:20stay in my flow state. I can keep adding
- 21:21things. I can keep playing. But on the
- 21:23right side, it's still going. And by the
- 21:25way, I still think the left side's too
- 21:26slow. We need to make that even faster.
- 21:31And then we can add a competitor to the
- 21:33game, and we can see even with the tool
- 21:35calls, even with all the things that
- 21:36it's doing, it's going to complete, and
- 21:39then the other one will still be hanging
- 21:40because it's fundamentally about how can
- 21:42you drive more efficiency and better
- 21:44performance with the same resources.
- 21:49It's going to go on. I'm not going to
- 21:50make you guys wait for this.
- 21:53One thing that people often point out
- 21:54too is
- 21:56is inference the bottleneck? Our tool
- 21:58calls the bottleneck. And I think this
- 21:59is a great point. If we look back to the
- 22:02turn timeline from the time that I was
- 22:04spending updating my slide, we can see
- 22:07that 80% of it was inference and 18% of
- 22:09it was tool calls.
- 22:11If we simulate that same exact thing and
- 22:14have 10x faster inference, we can see
- 22:17that 67% of that will be tool calls at
- 22:19that point if we don't optimize the tool
- 22:21calls.
- 22:22That's why it's important that we have
- 22:24to constantly attack the bottlenecks of
- 22:26the application and get the low-hanging
- 22:27fruit where we can. Today, people are
- 22:29focused on inference speed, and maybe we
- 22:31haven' optimized the tool call side and
- 22:33how tools are called as much because
- 22:35it's not the bottleneck. If I cut away
- 22:36the tool calls on the upper one, it
- 22:38wouldn't even really matter to the end
- 22:39application. But once we get much
- 22:41faster, we have to think about how to
- 22:43call these tools much more efficiently
- 22:45and much more performantly because
- 22:46otherwise they're going to be the reason
- 22:48that we're all waiting around.
- 22:53I think the good news is that optimizing
- 22:55these speeds and what we're all
- 22:56collectively doing as an industry is a
- 22:58verifiable task. If I know what my
- 23:00inputs and outputs should be, I can have
- 23:03agents do this very very full stack. And
- 23:06that's where we're increasingly moving
- 23:08to. We're having self-improvement loops
- 23:10with these agents to drive better
- 23:12efficiency on the fly as our inference
- 23:14is running. And at Gimlet, we're
- 23:16building recursive self-optimization for
- 23:18this that pulls from low-level telemetry
- 23:20from our data centers all the way up to
- 23:22the model layer and trying to use all of
- 23:24it to to improve the overall efficiency
- 23:27of the workload because if you just
- 23:28optimize one small piece, it might not
- 23:30make that much of a dent.
- 23:35I want to show another demo because I
- 23:36think that cyber security is on
- 23:38everyone's mind right now. Um we're all
- 23:40seeing like the huge incredible advances
- 23:43in AI driving awareness that there's all
- 23:46these vulnerabilities in our software.
- 23:48And I think that this is an area where
- 23:50faster inference can really really help.
- 23:53So this is a dumb fake scenario, but
- 23:55it's a real application and a real demo
- 23:57where imagine that you're running the
- 23:59dog club, and that's a website for dog
- 24:01lovers to upload and vote on pictures of
- 24:03dogs.
- 24:05However, unfortunately, like we often
- 24:07see, one of your node dependencies has a
- 24:10vulnerability.
- 24:11And this is a real vulnerability. This
- 24:13is really a node tar.
- 24:15And you're running a security agent on
- 24:17your application, and it's trying to
- 24:19monitor live traffic and see is there
- 24:21anything going wrong that I don't know
- 24:23about. This agent doesn't know about
- 24:24this vulnerability, but it does know its
- 24:26job is to protect your application.
- 24:31So, let's do a quick turn with normal
- 24:33user traffic.
- 24:34We're going to upload a dog
- 24:37biscuit.
- 24:39And on the upper side, we have the
- 24:41application logs, and on the bottom, we
- 24:43have the defender. And the defender is
- 24:44monitoring, but it's not in the critical
- 24:46path because it's not fast enough to
- 24:48actually respond to every user request
- 24:50live. It's monitoring things async.
- 24:54And we can see biscuit gets uploaded,
- 24:56and we see that this is an okay, you
- 24:58know, this is an okay upload. The
- 24:59defender doesn't see any problem.
- 25:02But now what we're going to do is have
- 25:04this defender operating when there's a
- 25:05malicious upload. And this upload is
- 25:07exploiting the real vulnerability where
- 25:09in node tar, there was a release where
- 25:11you could basically overwrite files that
- 25:13you weren't supposed to overwrite based
- 25:15on the destinations of what was being
- 25:17unzipped.
- 25:19So, in this one, we have
- 25:22Ranger, a fake, bad, uh, malicious dog,
- 25:26not really a dog.
- 25:27And what we're going to see is that the
- 25:29defender is going to figure out that
- 25:32this is a malicious thing. The agent is
- 25:34smart enough to detect that it's doing
- 25:35something that it shouldn't be doing
- 25:37even though it hasn't seen this exact
- 25:38vulnerability before.
- 25:40But unfortunately, cats malicious upload
- 25:43has taken over. So, your website's gone.
- 25:46It's too late. You didn't react in time.
- 25:52But if we retry this same exact
- 25:54malicious upload, but this time with an
- 25:56ultra fast defender, and again, this is
- 25:57a real a real demo fake use case,
- 26:00um, we can see that with ultra fast
- 26:03inference, we can react to things and do
- 26:05things online much more quickly. And
- 26:07this should change the way that we think
- 26:08of how to use agents in the first place.
- 26:11We're all thinking about how to use them
- 26:12in an async way, but we should think
- 26:14about how to use them in online ways as
- 26:16well. So, we can see on the bottom that
- 26:18the worker blocked their request, and it
- 26:19was like, what are you doing? This is
- 26:21obviously wrong.
- 26:26So, faster internet took us from
- 26:28squirrel mail to
- 26:30on demand high resolution streaming,
- 26:32right? Like, if you just look from this
- 26:34to this, it's crazy. If you showed this
- 26:36to my like five-year-old self or
- 26:38something, I would have been like My
- 26:39mind would have been blown.
- 26:41And this is just one of many things that
- 26:42I can select. So like what does a 10,000
- 26:45token per second world look like? We're
- 26:47at in the like low hundreds today. What
- 26:50does it look like when we get to a
- 26:51million tokens per second? Like what
- 26:52does that unlock for us?
- 26:56And I think that we should all ask
- 26:58ourselves this too. It's important in AI
- 27:00to skate to where the puck is moving,
- 27:01right? It's changing so fast. We need to
- 27:04be ready. We need to think boldly. It's
- 27:06not enough to just do what we've always
- 27:08been doing. I think we've all had the
- 27:09experience where we make an audacious
- 27:11request to our agent and after the
- 27:13latest model release, it just works. And
- 27:16it's going to be the same thing with
- 27:17latency. And so I think that we can
- 27:19think about ideas that are both coming
- 27:21soon when we hit a certain latency
- 27:23threshold, what we can do. But also the
- 27:25stuff that is real life sci-fi. Like if
- 27:27you look at something like Jarvis or
- 27:29something, there's no way today's
- 27:31inference could be fast enough to power
- 27:32that experience for Iron Man, but we
- 27:34will get there.
- 27:36So I'm really excited about online use
- 27:38cases, online security, online things of
- 27:41like I want to just tell my agent what I
- 27:44want to build and it's building it right
- 27:45in front of me and I don't even have to
- 27:47type anything. I think we're going to
- 27:48have new patterns of interaction. You
- 27:51know, think tab complete for everything.
- 27:53How can we have the user side of things
- 27:55be as quick and immediate as possible
- 27:57because the inference side is also just
- 27:59as fast to accommodate it.
- 28:01And we're going to have so many more
- 28:03agents. We're going to have even maybe
- 28:05Transformers driving robots, right?
- 28:07There's so much that we can build when
- 28:09this gets unlocked. And I think as
- 28:11builders on the frontier here, we should
- 28:12all be thinking about this too.
- 28:16Thank you so much for being with me um
- 28:19this morning. Uh you can find me on X at
- 28:21N Serino. Um we're hiring at Gimlet.
- 28:24Would love to talk to you all. And then
- 28:25if you want to play the Token Mint game,
- 28:27uh you can also find that QR here. Thank
- 28:30you so much everyone.
- 28:32>> [applause]
- 28:42>> As AI models get better, are our old
- 28:45best practices holding us back? To
- 28:47rethink how we work with Claude code,
- 28:49please welcome Lydia Halle, member of
- 28:52technical staff at Anthropic.
- 29:14>> Hi everyone. Oh, my microphone already
- 29:15works. Sorry. We have some technical
- 29:18issues this morning, so hopefully
- 29:19everything works.
- 29:56Yeah. Have you all tried Fable 5.2 yet?
- 29:59Yeah? Anyone? How is it? Like it?
- 30:02Yeah? Oh, it's 5.5 better? Worse?
- 30:05Different use cases, right?
- 30:08Cheaper Yeah, cheaper for sure.
- 30:11>> I decided to hide the Figma.
- 30:14It's all It should be good.
- 30:17>> So this should be on that screen.
- 30:19>> Yeah, we're looking down there.
- 30:21Right? Are you seeing anything, Chase?
- 30:33Let me see.
- 30:33>> So this was to be on my
- 30:36We need like Claude for IT.
- 30:38>> Yeah, we got it.
- 30:51>> Let's make sure to hide those
- 30:56>> I want this to go small so we can drag
- 30:58over again.
- 31:04>> You can drag it.
- 31:07>> Start your slide
- 31:08Start your slideshow.
- 31:12Good. And then drag
- 31:16>> Where?
- 31:17>> It's not It's going to show up down
- 31:18there.
- 31:19Can you see anything yet?
- 31:27I I did what you said.
- 32:01>> Okay.
- 32:13>> [snorts]
- 33:07>> Did it just die?
- 33:10>> Can I unplug it real quick?
- 33:11>> Yeah, please.
- 33:15I think it just died.
- 33:17>> Oh my gosh. Okay.
- 33:19I mean, I don't know what to do.
- 33:20It shouldn't die. It had battery.
- 33:25>> Yeah, now it now it it it's just died, I
- 33:28think.
- 33:29>> Should I just do my talk later today and
- 33:31we fix it?
- 33:31>> We don't know.
- 33:36>> [snorts]
- 33:39>> That's fine. I fine but I have one make
- 33:41sure that was over there scheduled.
- 33:53We do a little break and then the next
- 33:55guy can go.
- 33:57Okay, y'all. My laptop just died. So I
- 34:00nothing is happening. I don't know what
- 34:02to do.
- 34:03We uh Oh, it just restarted. Anyway.
- 34:06I I feel like I've just spent 20 minutes
- 34:07just waiting here and I don't want to
- 34:08take the next guy. But
- 34:10TL;DR, we went from being super in the
- 34:13loop with our cloud code sessions to,
- 34:15you know, running these long running
- 34:18sessions that run for hours in the cloud
- 34:20with these models like
- 34:22that are so good at verifying themselves
- 34:26that the role of the human has really
- 34:27shifted. Let's see.
- 34:38I don't think this is going to work.
- 34:51>> That's scary.
- 35:02Also, today make sure you come by our
- 35:03Anthropic booth. We're there all day. We
- 35:05have good merch. We also have good demos
- 35:07and just fun stuff. We have like a photo
- 35:08booth there as well, which is fun.
- 35:11And you'll see me there as well if you
- 35:12have any questions. First you got to
- 35:13make this work properly.
- 35:15Ignore.
- 35:16My Why did it restart?
- 35:18>> I don't know.
- 35:19>> Windows Server Experience Problem What?
- 35:24Oh, Apple.
- 35:29Okay, now I just got to
- 35:31Just want to make sure I'm not
- 35:32>> People often wonder what the story is
- 35:34behind what's the future of engineering
- 35:36in this space. Uh and it turns out that
- 35:38fixing the computer and making it go
- 35:41It's It's always a fun story. Like the
- 35:43more advanced technology gets, the
- 35:45harder it is to basically make it do
- 35:47your bidding.
- 35:48And
- 35:50conferences are conferences, AV is AV,
- 35:52and laptops are always going to laptop.
- 35:55Uh I'm told that this might be the first
- 35:57time that an Anthropic employee has not
- 35:59been prodding an agent and to wind
- 36:01[laughter] up paying the computer to do
- 36:02that bidding.
- 36:06>> That's what I'm saying when you clap for
- 36:07IT.
- 36:09And I see the little cloud robot helping
- 36:10me here. Oh, let's see.
- 36:14Exactly. I don't want to step on your
- 36:15toes, but I can also keep running my
- 36:18mouth.
- 36:19Some people run agents, I run my mouth.
- 36:21It There's a production joke in there
- 36:23somewhere, but I'm too tired to find it.
- 36:25Things work in theory, but not in
- 36:26production. That's uh why I call the
- 36:28staging environment theory.
- 36:30>> It's all black.
- 36:31Okay.
- 36:33Can you show us in the DSMs, please?
- 36:37>> [snorts]
- 36:38>> Okay.
- 36:38>> Oh, it looks like you're up.
- 36:40Maybe.
- 36:42>> No, I just have to see
- 36:44without my glasses for me.
- 36:46>> Remember, every code review is
- 36:47adversarial if you're a big enough jerk.
- 36:53>> Yes.
- 36:54Wait. No, no, no, no.
- 36:55>> Oh, that's not
- 36:56>> One second.
- 36:58Sorry. It was Yeah.
- 37:00>> And of course, the number one reason to
- 37:01found a company is that its co-founder
- 37:02main becomes your feature branch.
- 37:05Good luck everyone else.
- 37:20>> All right. Tonight, I want to see that
- 37:22on my screen real quick
- 37:23cuz I don't know.
- 37:25Yeah. No, no, no. It's not showing.
- 37:32What should I go to?
- 37:35We're going to
- 37:41I just don't want to
- 37:42Um
- 37:44>> Say want to
- 37:49>> Can you close your mic so everyone can
- 37:51hear us talking?
- 38:10>> Okay.
- 38:11>> Oh, okay.
- 38:13>> I'll see you later. Hopefully, I'll come
- 38:15back. You'll hear from me.
- 38:18>> [applause]
- 38:19>> Slight change in programming here.
- 38:22I believe our next speaker is Michelle
- 38:24Chen.
- 38:25Correct me if I'm wrong on that.
- 38:28No one is yanking the
- 38:31the cord there.
- 38:32If you can give us about 3 minutes, we
- 38:33will take a small pause here. Besides,
- 38:36what are you all doing listening to
- 38:37this? You shouldn't be poking and
- 38:38prodding agents and seeing how many more
- 38:40you can wind up going. The tokens aren't
- 38:41going to burn themselves.
- 38:44We need more bandwidth.
- 38:46>> Oh.
- 38:47Suddenly, the use case for local
- 38:49inference makes itself extraordinarily
- 38:51clear. Conference Wi-Fi. There we go. I
- 38:54I've done it on planes before, and what
- 38:56made that fun was suddenly my laptop
- 38:58became what sounded like the third
- 39:00engine on the jet, but it worked, and it
- 39:02wrote better code than I did. Uh
- 39:05that's not a high bar, but it does mean
- 39:07I'm not going back in time to be the
- 39:09old-timey typing things out by hand like
- 39:12some sort of agrarian farmer. So there's
- 39:14that.
- 39:17All right. Next up, we have Michelle
- 39:19Chen
- 39:20talking about Cloudflare is a Neo Lab.
- 39:22Uh personally, I believe that it's a
- 39:24database instead because I think
- 39:25everything's a database when you hold it
- 39:27wrong, but for some reason I wasn't
- 39:29consulted. Anyway, Michelle, thank you
- 39:31very much. Please come on up.
- 39:36>> What does it mean for Cloudflare to be a
- 39:38new kind of AI lab?
- 39:40To explore new models, edge AI, and the
- 39:43ambition to become the agent cloud.
- 39:46Please welcome Michelle Chen, Group
- 39:48Product Manager at Cloudflare.
- 40:09>> Hi everyone. My name is Michelle Chen.
- 40:11I'm so excited to be here today at AI
- 40:12Engineer Code Summit, chatting a little
- 40:14bit more about why Cloudflare became a
- 40:17Neo Lab. So at Cloudflare, I am the
- 40:19group PM here. I run on the product team
- 40:21working on products like Workers AI, AI
- 40:23search, AI gateway as well. And so today
- 40:26we're going to go through this little
- 40:27journey of how Cloudflare came to be.
- 40:29You might have heard that we launched
- 40:30and trained some models recently. I'm
- 40:32going to tell you a little bit more
- 40:33about why and what you can be doing with
- 40:36it. And so we're going to go through a
- 40:37little bit of a philosophical journey.
- 40:39Uh when I was preparing for this talk, I
- 40:41wanted to use this quote, and then
- 40:43Tropic like launched a blog post last
- 40:45weekend and used my quote, but I'm still
- 40:47going to use it anyways.
- 40:48Um, so the quote goes
- 40:54in 1 second. Um, the quote is from
- 40:56Kierkegaard and it says, life can only
- 40:57be understood backwards, but it must be
- 40:59lived forwards. And I think this is
- 41:01really really apt quote here for us
- 41:03because at Cloudflare we're a
- 41:0516-year-old company and we've done so
- 41:07many things over the years, but somehow
- 41:09we got all some of the pieces right and
- 41:11we ended up having to become a new lab.
- 41:13So we're going to go through a little
- 41:14bit journey. The first thing we're going
- 41:15to do is look backwards and talk through
- 41:20why Cloudflare had to uh became a new
- 41:23lab.
- 41:26Give me 1 second here.
- 41:29Perfect. We're going to go backwards
- 41:31[laughter] and talk through why
- 41:32Cloudflare became a new lab. So
- 41:35Cloudflare is a 16-year-old CDN company.
- 41:38We just celebrated our 16th birthday
- 41:41uh at birthday week this year uh in
- 41:43September.
- 41:46And we first started as a CDN. So as a
- 41:48CDN company, what you have to do is you
- 41:50have to serve content as quick to your
- 41:52users as possible. You sit between the
- 41:55users request and the origin and you
- 41:57help uh sit in the middle and you can do
- 41:59things like cache and protect, do
- 42:00security and things like that. So that's
- 42:02kind of how Cloudflare started. But
- 42:05Cloudflare's infrastructure had to be
- 42:07adaptable in order to become the CDN.
- 42:10And so what we did was 16 years ago in
- 42:112010 we launched with our own
- 42:13infrastructure. Our fast CDN had to be
- 42:15everywhere. And this looks really
- 42:17different than than a traditional hyper
- 42:19scaler where compute is centralized in
- 42:20US East 1 or US West 1. Cloudflare's
- 42:23infrastructure is a completely different
- 42:25paradigm. We have data centers all
- 42:27around the world as close to the user as
- 42:29possible. And this is really important
- 42:31in becoming a CDN. But now we have these
- 42:33computers basically running as close to
- 42:35the user as possible. So we thought to
- 42:37ourselves, what else can we be doing
- 42:39with these computers that people would
- 42:40want to build with?
- 42:42And so came compute. We started
- 42:45launching these compute primitives as
- 42:46well. And so in 2017 we launched a
- 42:48product called Workers, which is
- 42:49basically like serverless functions that
- 42:52you can run. It contrasted VMs at that
- 42:54time where you had to rent a whole
- 42:56machine and pay for the whole machine,
- 42:58either you're using it or not using it.
- 43:00And so with workers, you just execute
- 43:02some code and then the worker goes away.
- 43:04You don't have to pay for it. It takes
- 43:05milliseconds to start up, not seconds as
- 43:07well. And then we followed with other
- 43:09compute primitives like durable objects,
- 43:11which are more stateful. And then in
- 43:122025 we launched containers as well
- 43:14because there's some workloads that do
- 43:16need something like a VM and we want all
- 43:18of those workloads to be able to run on
- 43:19Cloudflare. So now, you know, 2017 start
- 43:22of our developer platform, we have these
- 43:24compute primitives. People started using
- 43:26it. People started building more like
- 43:28static websites and things like that. So
- 43:31we thought to ourselves, how do we
- 43:32elevate this and take it to the next
- 43:34level?
- 43:35So we introduced storage as well. And so
- 43:38we have these storage primitives on
- 43:39Cloudflare's developer platform. We have
- 43:41KV, we have R2 dot object storage, we
- 43:44have databases like D1 as well. And the
- 43:46idea here is that people should be able
- 43:48to build full stack applications all on
- 43:51Cloudflare. Stateful applications where
- 43:53people can go on and start building um
- 43:56full applications and web apps and
- 43:58platforms on Cloudflare's developer
- 44:00platform.
- 44:01So we have this infrastructure. We added
- 44:03compute primitives to it. We added
- 44:05storage primitives to it. And then came
- 44:072023. And if you guys remember what
- 44:09happened in 2023, ChatGPT kind of blew
- 44:12up a little bit and it changed how we
- 44:14think about AI, right?
- 44:16And
- 44:17I remember it being birthday week and uh
- 44:20three weeks before birthday week, a lot
- 44:22of us sat together in a room and we were
- 44:23like, what should we do now that ChatGPT
- 44:25is on the rise? What can Cloudflare play
- 44:28in here with our infrastructure and our
- 44:30primitives in order to be in this AI
- 44:32game?
- 44:34So in 2023 we launched a product called
- 44:36Workers AI where we added GPUs to our
- 44:38network. And so again, this is really
- 44:40distributed where GPUs are running as
- 44:42close to the user as possible. It's not
- 44:44centralized like a US East 1 hyper
- 44:46scaler for example, but it's running as
- 44:48close to the edge as possible.
- 44:50And we had a hypothesis when we started
- 44:52Workers AI.
- 44:53Our hypothesis was that were going to
- 44:56get smaller so that network latency was
- 44:58going to become more and more important.
- 45:01We launched with small models that could
- 45:02fit on our small hardware around the
- 45:04world that were super distributed and we
- 45:06thought that people would have models
- 45:08get better and faster and smarter and
- 45:10that network latency was going to play a
- 45:12much more bigger impact here.
- 45:15Well, actually the opposite happened and
- 45:18models got larger and larger. So we went
- 45:20from something like
- 45:217 billion parameters, which was frontier
- 45:23scale at the time, to being the 3
- 45:25trillion parameter models that were
- 45:27getting launched. And I think this is
- 45:28not so
- 45:30shocking when you look back in time
- 45:32because, you know, as we're on the way
- 45:34to AGI, um, uh, if you're following
- 45:36scaling law, as you train on more data,
- 45:39as the models get larger, they're more
- 45:40generalizable. And so on the path to
- 45:42AGI, it makes sense that we're pushing
- 45:44the frontier of research here in order
- 45:46to get larger and larger and smarter
- 45:48models.
- 45:49So this goes against our hypothesis,
- 45:51right? That the models were going to get
- 45:53smaller. The models actually ended up
- 45:54being larger and the network latency
- 45:56mattered less and less.
- 45:58So what did we do?
- 46:01We ended up
- 46:03pivoting. So Cloudflare got really good
- 46:05at serving these giant models as well.
- 46:07So in March 2026, we launched Kimi and
- 46:10then we followed up with GLM and
- 46:11DeepSeek and these large trillion
- 46:12parameter models um hosted on our
- 46:15infrastructure. And so this is
- 46:16serverless. People hit an API call and
- 46:18get a token back. It actually took us a
- 46:20while in order to decide to do this,
- 46:23right? We March 2026 is actually pretty
- 46:26late in the game to to serving this
- 46:27model, but we were trying to make a
- 46:29conscious decision here on does this
- 46:31actually make sense for us to be serving
- 46:32it? It didn't really fit with our
- 46:34infrastructure story, but it fit really
- 46:36well with the developer platform.
- 46:39So in 2026, we had all these primitives
- 46:42in place. We had the infrastructure,
- 46:44which is unique to Cloudflare, running
- 46:46close to users. We had compute
- 46:48primitives where people were building
- 46:49agents on top of. So workers and durable
- 46:51objects, right? As your agent needs to
- 46:53complete a task, it goes was and it
- 46:56spins and does a task then sleeps when
- 46:57it doesn't. We had storage primitives,
- 46:59so your agents can actually interact
- 47:01with data and storage and then we had
- 47:03models, these large models that were
- 47:05powering agents. And so all of this fit
- 47:08into the Cloudflare agent cloud story.
- 47:10And so we started telling this narrative
- 47:11that we want to be the agent cloud. We
- 47:13want agents to be deployed on us,
- 47:15powered by models on us, using our
- 47:17compute and data primitives, all built
- 47:19on our infrastructure and our network.
- 47:21But it never felt quite right. We felt
- 47:24like we were always missing a piece here
- 47:26because the infrastructure pattern
- 47:27didn't necessarily fit in with what
- 47:29Workers AI was designed for.
- 47:33And so I'm just going to sit here and
- 47:34give you my second philosophical quote,
- 47:36which is a risk considering that I'm a
- 47:38girl in Chinese and AI, but I'm going to
- 47:40give you this Chinese quote anyways.
- 47:43[laughter]
- 47:44And uh it basically says
- 47:48which means everything is ready here.
- 47:50All that's missing is the east wind.
- 47:53And one of the things uh about this is
- 47:55like you can't really predict where the
- 47:57wind is going to come from or how it's
- 47:58going to come, but you need to be very
- 48:00reactive when it happens.
- 48:02And so in September 2026, the east wind
- 48:04came. TypeSafe launched a model called
- 48:07Jev and introduced this idea of system
- 48:09one models. And it contrasted what we
- 48:11typically knew with LLMs, which were
- 48:13autoregressive and taking a long time to
- 48:15generate tokens, it scales
- 48:17quadratically, etc. A system one model
- 48:19is a completely different paradigm where
- 48:21it actually just takes one forward pass
- 48:24through the inputs and generates an
- 48:26output that you can use to make
- 48:27decisions with it.
- 48:30And so I really think that this came
- 48:31down to perfect timing. I think people
- 48:33were starting to get fed up a little bit
- 48:35with LLMs because they were so costly,
- 48:37it costs so many tokens, it's super time
- 48:40consuming. And I don't know about you,
- 48:41but when I read a reasoning trace,
- 48:43sometimes my blood pressure rises when
- 48:44the LLM doesn't get the right the things
- 48:46right. And so I think we were sending
- 48:48everything to LLMs and started getting
- 48:49frustrated with it. And so when this
- 48:51system one and Jev paradigm started
- 48:53coming on board, we were like this makes
- 48:55so much sense for us to adopt, right? We
- 48:57don't have to send every request to an
- 48:59LLM, We can send some of it to a system
- 49:01one model.
- 49:03And so the system one models actually
- 49:04end up taking work off of the giant
- 49:06LLMs, you give it to the small fast
- 49:08models. And the two are supposed to work
- 49:09in tandem. supposed to complement each
- 49:11other and get to the end goal faster.
- 49:14so if you look, we are back at these
- 49:17small and fast models, right? This was
- 49:19the original Workers AI hypothesis.
- 49:23And so Cloudflare was not wrong here. We
- 49:26actually were just early. With these
- 49:28small and fast models, the network
- 49:30latency became important again. We want
- 49:32the model to be responding fast, but
- 49:34also the network latency to be even
- 49:35faster. And so we decided to do
- 49:38something here.
- 49:39But
- 49:40Jev was closed source.
- 49:42Cloudflare couldn't actually host the
- 49:44Jev models and take advantage of our
- 49:45infrastructure.
- 49:47So what did we decide to do?
- 49:49We built our own version of Jev. So Jev
- 49:51launched on September 15th.
- 49:53On September 18th, just three days
- 49:55later, we started experimenting and we
- 49:57launched our version of Jev, our first
- 49:59experiment of it. And then it took us a
- 50:01week of deliberation to decide, do we
- 50:03actually want to be going into this
- 50:05neolab territory? It's completely new to
- 50:08us. We don't know what we need to know.
- 50:10It, you know, are we going to set the
- 50:12expectation that we want to be a neolab
- 50:14and are we going to live up to those
- 50:15expectations? But we decided to do it. I
- 50:18remember on a Friday evening we were
- 50:19like, let's just go for it and we did
- 50:21it. We ended up training the model over
- 50:2348 hours over the weekend and then we
- 50:25launched it the following week. And so
- 50:27we ended up with a model called Clef.
- 50:30Clef gave power back to the agents to be
- 50:32able to make decisions on their own. So
- 50:34agents could ask act more autonomously.
- 50:37You didn't need to have an LLM reason
- 50:39through for minutes on if it should make
- 50:41this decision or not. You also didn't
- 50:43need to put a human in the loop and be
- 50:44interrupted by like a human's sleep
- 50:46schedule when you want your agents to be
- 50:47running 24/7. So you can now
- 50:49programmatically encode, decide, make
- 50:52decisions for your agent to take action
- 50:54on.
- 50:57So these decision models countered LLM
- 50:59models because they were a lot faster
- 51:01and they were also a cheaper as well.
- 51:03And so it actually as an inference
- 51:04provider takes a lot of pressure off the
- 51:06LLMs as well, because the LLMs were
- 51:08reserved to work for meaningful work,
- 51:10whereas the clef and uh decision models
- 51:12could be used for fast um intuitive
- 51:15thinking and work, right?
- 51:18And so we also went back and thought to
- 51:20ourselves, you know, what else do agents
- 51:23and people use for LLMs for that are
- 51:26overkill? So we came up with decisions
- 51:28and um there must be other workloads out
- 51:31there that are unsuitable for an LLM,
- 51:33both because they are either too
- 51:34expensive or take too long, or maybe
- 51:37that it just would get the answer wrong
- 51:39because the autoregressive nature of an
- 51:40LLM means that it would hallucinate.
- 51:43So we also launched a model three weeks
- 51:46later called Cleric. We actually ended
- 51:47up launching four models in the span of
- 51:49three weeks, so I think we did a pretty
- 51:51good job as a Neo Lab. Um but Cleric is
- 51:53a new model paradigm. So it's completely
- 51:55new. We're calling it an extraction
- 51:57model, but it's a system one model in
- 51:59that in the sense that it is just as
- 52:01fast and just as cheap, but it is also
- 52:03doing like one forward pass through the
- 52:05user. And the purpose of Cleric is when
- 52:07you're interacting with the world,
- 52:08you're not interacting in perfectly
- 52:10typed JSON objects, right? Sometimes
- 52:12you're getting an email, a tweet, a
- 52:14Slack message, but those people are not
- 52:16filling out perfect forms for you to
- 52:17extract inputs in out of. So what Cleric
- 52:20does is it can take unstructured input
- 52:22and it can extract, compute, estimate,
- 52:25and forecast values from those inputs,
- 52:27and it gives you back a perfectly typed
- 52:29JSON objects, which means the agents are
- 52:31able to do things with it. So we're kind
- 52:33of giving agents the ability to read
- 52:35here and take action with clef as well.
- 52:40And so in code again, you're giving
- 52:42Cleric, you know, typed values from
- 52:44unstructured data. You can pass it in
- 52:46whatever kind of unstructured data. It
- 52:47can also take image input as well. And
- 52:49so you can just tell it to read
- 52:51basically what's in text here, pass it
- 52:53to clef, decide what to do with this
- 52:55text, and then pass it to an LLM to
- 52:57decide what to do with the agent.
- 53:01And so Clef were had to become a Neo Lab
- 53:03here. This was our missing bit. We had
- 53:05the infrastructure, we had the compute
- 53:07primitives, we had the data primitives,
- 53:08we had the large models powering the
- 53:10agents, but in order to take advantage
- 53:12of all of that and our infrastructure
- 53:14that we were built on, we had to be a
- 53:16neolab and ship these system one models
- 53:17that fit perfectly with our
- 53:19infrastructure. And I think one of the
- 53:20more important things is that we're not
- 53:22limited to this. Being a neolab means
- 53:24that we can create what we are missing,
- 53:27right? It means that we're not uh if
- 53:29we're missing a system one model because
- 53:31we're um at risk of things being closed
- 53:33source or regulatory risks or things
- 53:35like that, we can actually create and
- 53:37invent things. So I think being a neolab
- 53:39has actually opened a lot of doors for
- 53:40Cloudflare to do more in the AI world.
- 53:45So we went through backwards, we went
- 53:46through a little bit of a history
- 53:47lesson. I want to go forwards as well. I
- 53:49want to teach you guys what you guys can
- 53:51be doing with Cloudflare Cleric. And I'm
- 53:53cognizant that I am at AI Engineer Code
- 53:55Summit. So in a in a room full of
- 53:57developers and engineers, how can you
- 53:59guys leverage Cleric and Cleric and
- 54:01system one models in order to better use
- 54:03your workflows?
- 54:05So we'll do a quick demo here. Um it is
- 54:08in let's think you have an agent, you're
- 54:10triaging like a CI build that went
- 54:12wrong, right? Um let's say you have 40
- 54:14tool called tools that are available to
- 54:16this agent. You have your PR that went
- 54:18wrong um and then you have your agent.
- 54:20Maybe it's running Py Durable on a
- 54:21durable object. So it's all of this is
- 54:23built on Cloudflare. So this is the
- 54:25setup here.
- 54:26Tech typically what you would do is you
- 54:28would take an LLM and in the LLM would
- 54:30take a tool make a tool call, the tool
- 54:32call would go back to the LLM, the LLM
- 54:33would make another tool call, etc etc
- 54:35until the issue is resolved. So this is
- 54:37like five back and forths with GLM 5.3.
- 54:41You're sending 40 tools every single
- 54:43time and maybe you're blowing up the
- 54:45context, right? Because you're sending
- 54:46the same kind of inputs back and forth
- 54:48to the LLM.
- 54:49And so what if we augment this with
- 54:51system one models?
- 54:54With system one models uh this could
- 54:56look a lot different. And so let's say
- 54:58we where we want speed, so we pick Clef
- 55:00Flash. Clef Flash can actually help you
- 55:02prune your tool list. Maybe you only
- 55:03need 10 out of the 40 tools that are
- 55:05available, so you only need to pick 10
- 55:07of those. Then let say you take clerk
- 55:09and you use it to extract the values
- 55:12from the input. So it can get the PR
- 55:13number without reasoning and thinking
- 55:15about it too long.
- 55:16Um and then let's say you just, you
- 55:18know, orchestrate some code here to
- 55:19execute these API calls and you can
- 55:22actually just, you know, get those
- 55:23answers back. Then you ask clerk again
- 55:26um to pull the data from the logs. So
- 55:28normally when you're using with working
- 55:30with an LLM, you probably just paste in
- 55:31all of the logs, right? Like all 300 600
- 55:34lines of logs. But clerk can actually
- 55:36just go through and read those and say
- 55:38here's the exact error, here's the error
- 55:40message, here the lines of code it's
- 55:42affecting. And you're basically
- 55:43shrinking your context lot. And then
- 55:45finally you use an LLM to actually do
- 55:47take action and um make decisions and
- 55:51take action and extract values all
- 55:53together. This same agent can run the
- 55:55system one.
- 55:56Let's do a little bit of a live demo.
- 55:58Let's see how it goes. We're going to
- 55:59have to click run both here. And this is
- 56:01a live demo and I will say that
- 56:05Cloudflare our workers AI is uh
- 56:09has a very diurnal usage pattern. So
- 56:11we're in the middle of the workday. So
- 56:12we'll see how this how this goes. But
- 56:14it'll give you some illustrative example
- 56:16here of how it goes.
- 56:17Uh and we're also dealing with
- 56:18conference Wi-Fi. So so bear with me
- 56:20here. All right. The system one model
- 56:22side finished already. And all LLM one
- 56:25is is going. So I think even
- 56:27illustratively you can see that the
- 56:30uh system one set of models is a lot
- 56:32faster. Even by just this simple
- 56:34waterfall chart, you can see that the
- 56:35system one part takes so much less time
- 56:38than the uh actual generation which is
- 56:40the purple bars here. And I think they
- 56:43both ended up getting to the answer at
- 56:44the end. So the quality was not
- 56:46affected. But it took
- 56:48a lot less time. It was a lot cheaper.
- 56:50Six LLM calls versus one. 40,000 LLM
- 56:53tokens versus 1,800. And so you can see
- 56:56how you can use these kind of system one
- 56:58and system two models even in like
- 57:00developer workflows in order to get you
- 57:02to your answer faster.
- 57:06So the same answer, but it took a
- 57:08fraction of the LLM time and cost,
- 57:10right? I ran this when we were not doing
- 57:12conference Wi-Fi, but we had about 9.2
- 57:15seconds and 17.2 seconds on the LLM. It
- 57:18was almost two times faster. It was 72%
- 57:22less expensive. We shrunk our LLM calls
- 57:25from five down to one and about 94% less
- 57:29tokens.
- 57:30And so this is an illustrative example
- 57:32of how you could be using the system one
- 57:35and system two models in tandem. It's
- 57:37not intended to completely replace your
- 57:39LLM usage, but it can augment it and
- 57:41make it better by making allowing you to
- 57:43free up time and compute for your other
- 57:47more important tasks here and you can
- 57:49now do, you know, higher concurrency
- 57:51tasks because these models are able able
- 57:53and capable to do so much more.
- 57:56And so that was one illustrative
- 57:57example. I just wanted to flash on the
- 57:59screen really quickly as well. What else
- 58:00are people building with Clef and
- 58:01Claric, right? There's a lot of
- 58:03routers that you can build and and
- 58:05making decisions there. There's fraud
- 58:08that we've been seeing a lot like spam
- 58:10or phishing. It can make decisions on
- 58:12there. With Claric, you can use it to
- 58:13extract data from like web pages or
- 58:16support tickets and things like that.
- 58:18I'm excited to see we actually have a
- 58:20new launch coming out for Claric. We
- 58:21released an early preview, but it's
- 58:23coming out. So I think these use cases
- 58:24will grow as people are trying it out
- 58:27more and more. But the idea is that
- 58:29there's so much in the world we're not
- 58:31limited to just LLMs. Now these new
- 58:33paradigms have been introduced and I
- 58:35think we should all take advantage of
- 58:36that and try building with it.
- 58:39And so we went backwards, we went
- 58:41forwards, but I think the most important
- 58:42thing here is that we go onwards, right?
- 58:45We have to be very reactive in this
- 58:47industry. Love it or hate it, things
- 58:50just change on a whim.
- 58:52You know, if we as Cloudflare
- 58:55back in the day said our hypothesis was
- 58:56wrong, we have to go and actually pivot
- 59:00and start serving these giant LLMs in
- 59:02order to take advantage of the fact that
- 59:04we want to be an agent cloud and that it
- 59:05makes sense for our story. When an east
- 59:07wing wind comes and we need to
- 59:11have decision models because it fit
- 59:13perfectly with our infrastructure story.
- 59:15We had to pivot and be a neolab as well.
- 59:17it's not it's one thing to sit here and
- 59:20look back, but the more important thing
- 59:21is that we
- 59:22took what we had in stride and moved
- 59:24forwards.
- 59:26So I'll leave you with one last quote
- 59:27here,
- 59:30which is a story from Tolstoy.
- 59:32Remember then there's only one time that
- 59:34is important now. It is the most
- 59:36important time because it's the only
- 59:37time when we have any power. And so I'll
- 59:39leave you with this quote and a question
- 59:42of what are you going to build today?
- 59:46>> [applause]
- 1:00:02[laughter]
- 1:00:11>> Hundreds of pull requests a day and
- 1:00:13nobody reads them all. So how does code
- 1:00:16quality keep improving? To share how the
- 1:00:19Grok bot team turned slop into gold,
- 1:00:22please welcome Lauren Tan, principal
- 1:00:24engineer at SpaceX Super Intelligence.
- 1:01:02>> Hello everyone.
- 1:01:04Don't know if you can see my slides, but
- 1:01:06my name is Potato,
- 1:01:08also known as Lauren.
- 1:01:09And I'm very excited to chat with all of
- 1:01:11you today.
- 1:01:13Maybe we can get a big round of applause
- 1:01:14our tech team helping me with my laptop.
- 1:01:17[applause]
- 1:01:36All right. Um so again I'm my name is
- 1:01:40Potato.
- 1:01:41Uh I work on Graph Bot at SpaceX SI.
- 1:01:45And can I just get a quick show of
- 1:01:46hands?
- 1:01:47How many of you have ever looked at code
- 1:01:50that somebody else wrote and you thought
- 1:01:52that's all slop.
- 1:01:54[laughter]
- 1:01:55Well
- 1:01:56I have too.
- 1:01:59But today I'm going to convince you that
- 1:02:00you can turn that slop into gold.
- 1:02:06Uh and so I did something crazy last
- 1:02:08month and I shipped 3,000 pull requests
- 1:02:11to production.
- 1:02:12Now before agents we never really cared
- 1:02:14about volume uh except for the outliers.
- 1:02:18And at my previous company we called
- 1:02:20these people coding machines. People
- 1:02:22like Brett Taylor who famously rewrote
- 1:02:24all of Google Maps uh in a weekend.
- 1:02:27Just an insanely productive engineer.
- 1:02:30Uh and well with agents anyone can now
- 1:02:32be a coding machine and my thesis is
- 1:02:34that the job of the software engineer,
- 1:02:37the meta of being a software engineer is
- 1:02:39figuring out how to maintain or even
- 1:02:41improve the quality of your code while
- 1:02:44massively increasing your productivity.
- 1:02:47To get there some people like to call
- 1:02:49this a software factory uh but I like to
- 1:02:51keep it simple and I think of it as all
- 1:02:54about trust.
- 1:02:56Not blind trust but investing in your
- 1:02:59agents environment so you can trust that
- 1:03:01your agents are going to do the right
- 1:03:03thing.
- 1:03:04Now it takes quite a lot of effort to
- 1:03:07get there um and it took me six months
- 1:03:09to get to a point where I could ship
- 1:03:12thousands of pull requests a lot of them
- 1:03:14in my sleep.
- 1:03:15But it's certainly possible and it will
- 1:03:17take you much less time than I did.
- 1:03:20So let's start by talking about how we
- 1:03:22move from a low trust environment to a
- 1:03:24high trust one.
- 1:03:27So this inverse pyramid is how I think
- 1:03:29about where to spend time on your
- 1:03:31agent's environment.
- 1:03:34The first two layers are deterministic.
- 1:03:36So using your code itself. So things
- 1:03:38like good API design, architecture,
- 1:03:41refactoring or using lint rules, type
- 1:03:44systems, compilers or you know different
- 1:03:46kinds of testing.
- 1:03:48And you can use both of these layers
- 1:03:50together to constrain what agents can do
- 1:03:52and force them to do the right thing by
- 1:03:54default.
- 1:03:56Verification, both runtime and formal
- 1:03:59are great tools to layer on to find bugs
- 1:04:02and check properties and the invariance
- 1:04:04in your system that you want to hold
- 1:04:05true
- 1:04:06and also help you check whether or not
- 1:04:08the agent understood your goals and your
- 1:04:10intent.
- 1:04:13And then finally you want to layer with
- 1:04:14a little bit of pros. So things like
- 1:04:16rules, skills, agents.md, things that
- 1:04:20require judgment.
- 1:04:23But let's start with your codebase.
- 1:04:25Agents generate their outputs based on
- 1:04:27their inputs and so the the quality of
- 1:04:30the input really matters and it just so
- 1:04:32happens that your codebase is one of the
- 1:04:34largest inputs.
- 1:04:36So whenever you're correcting your
- 1:04:37agent, think about how you can make that
- 1:04:39a deterministic check instead of adding
- 1:04:42another line to your rules or your
- 1:04:44agents.md because it's far more
- 1:04:46reproducible when it's in code and it
- 1:04:48lets you trust it more.
- 1:04:51And the analogy that I like to use is
- 1:04:53that your codebase is kind of like a
- 1:04:55garden.
- 1:04:56You know code used to arrive a few pull
- 1:04:58requests at a time
- 1:05:00and uh I remember going into work in
- 1:05:022024 and reviewing the three pull
- 1:05:05requests that my colleagues wrote
- 1:05:06colleagues wrote.
- 1:05:08And today that number is more like 300.
- 1:05:11So uh you know every engineer is now a
- 1:05:13team of hundreds or even thousands of
- 1:05:16agents and code is arriving so rapidly
- 1:05:19that it almost feels like a continuous
- 1:05:22stream, and it almost feels organic like
- 1:05:25a living thing.
- 1:05:27And in this analogy, your codebase, the
- 1:05:30constraints that you impose, the
- 1:05:32guardrails that you put up are the
- 1:05:34concrete foundations that the new code
- 1:05:36that your agents write grow around, just
- 1:05:39like uh the building in this picture.
- 1:05:42And so in many ways, the job of being a
- 1:05:43software engineer is kind of becoming
- 1:05:45like gardening.
- 1:05:48Because code is the biggest input for
- 1:05:50agents, it means that any workarounds,
- 1:05:53um anti-patterns, deprecated code, tech
- 1:05:56debt, it's now likely to be just copied
- 1:05:59by the next agent since it looks like
- 1:06:01the established pattern in your
- 1:06:02codebase.
- 1:06:03And uh it's like you have some weeds or
- 1:06:07pests growing in your garden, and then
- 1:06:09they spread everywhere before you know
- 1:06:11it.
- 1:06:15So you want to make your codebase as
- 1:06:17pristine as possible so that agents are
- 1:06:19always copying the good stuff.
- 1:06:22All right, let's look at some real
- 1:06:23examples.
- 1:06:24Uh so Dune is a client framework that we
- 1:06:26created for Graphot to implement this
- 1:06:29idea for organic architecture.
- 1:06:32It's not really that interesting
- 1:06:33technically.
- 1:06:34Uh you can kind of think of Dune as um
- 1:06:36kind of like Next.js for Electron apps.
- 1:06:39But the interesting part of Dune is that
- 1:06:41it has a lot of constraints and lint
- 1:06:43rules that we've encoded uh a lot of the
- 1:06:45lessons that we've learned building
- 1:06:48desktop applications.
- 1:06:50And one such lesson is that anything
- 1:06:53that requires nuance to use well often
- 1:06:55leads to slop.
- 1:06:57Uh you know your classic example is uh
- 1:06:59you know use effect in React, right? It
- 1:07:01has its use cases, uh but it's often
- 1:07:04misused. You know some people joke that
- 1:07:06you need a PhD to use use effect.
- 1:07:09Um and in in other words, it really
- 1:07:11requires a lot of nuance to use well.
- 1:07:14And agents are not good at using it.
- 1:07:16Um so our solution has been to just ban
- 1:07:19it in product code, and it's really only
- 1:07:22allowed in some places inside of Dune
- 1:07:24APIs.
- 1:07:26And effectively, this problem has now
- 1:07:28been categorically eliminated, and we
- 1:07:30never need to catch that during code
- 1:07:32review again.
- 1:07:34Another interesting pattern that we use
- 1:07:36is context through directory structure.
- 1:07:39Uh so one anti-pattern that we saw with
- 1:07:41Cursor, which is an Electron app, is
- 1:07:43that agents would often
- 1:07:45import main thread code into the
- 1:07:47renderer thread, which would cause long
- 1:07:49tasks and jank.
- 1:07:52So in uh Grotbot and in Dune apps, we've
- 1:07:54made this very, very conventional.
- 1:07:56Uh we have these two top-level
- 1:07:58directories literally called Electron
- 1:08:00Renderer and Electron Main.
- 1:08:02And imports between them are forbidden
- 1:08:05or banned by our dependency graph
- 1:08:07analyzer and some lint rules.
- 1:08:10And again, you know, these are simple
- 1:08:11lint rules to set up, but it means one
- 1:08:14less thing that we need to review and
- 1:08:16think about uh you know, during code
- 1:08:18review.
- 1:08:20And then one more surprising thing that
- 1:08:23I've seen is agents using Markdown. So
- 1:08:26things like documentation, uh code
- 1:08:28comments, rules or skills, and they use
- 1:08:31this as a place to write down their
- 1:08:33justifications for not doing the right
- 1:08:35thing. Or I don't know if you've ever
- 1:08:37seen this, but they'll over-rotate on
- 1:08:39some small correction that you made, and
- 1:08:41they treat it like a hard rule that can
- 1:08:43never be broken ever again.
- 1:08:46And here's another example from Cursor.
- 1:08:48So agents really loved using imperative
- 1:08:51timers, so things like setTimeout,
- 1:08:53requestAnimationFrame,
- 1:08:55requestIdleCallback.
- 1:08:56And they were using these as quick fixes
- 1:08:58because
- 1:08:59they were really patching over an
- 1:09:01architecture clash between
- 1:09:03uh VS Code's imperative style and
- 1:09:05React's declarative style.
- 1:09:08So our choice of architecture actually
- 1:09:10made both the client and the agent
- 1:09:13performance much worse.
- 1:09:15And if you've ever done any performance
- 1:09:16work in JavaScript, you'll know that
- 1:09:18when you have tons of timers being
- 1:09:20rapidly created and destroyed, causes a
- 1:09:22lot of uh major garbage collection,
- 1:09:24which causes a lot of lag in gen.
- 1:09:27But the worst part is that agents wrote
- 1:09:29a lot of code comments like you can see
- 1:09:31in red here,
- 1:09:32explaining why this quick fix was
- 1:09:34justified.
- 1:09:36And before you know it,
- 1:09:37we had timers spreading everywhere and
- 1:09:40the you know the performance of the app
- 1:09:41really suffered.
- 1:09:44Now in Graphbot we learned our lesson
- 1:09:46there and we decided to just ban 99% of
- 1:09:49comments
- 1:09:50because they go stale easily. You know
- 1:09:53humans just really don't bother reading
- 1:09:55the comment slop anyway.
- 1:09:57And again it gives an agent the an
- 1:09:59escape hatch to justify not doing the
- 1:10:02right thing because it's hard.
- 1:10:05And the worst part about comments though
- 1:10:07is that they're not enforceable.
- 1:10:09So what seems well intentioned you know
- 1:10:11like I'm going to tell the the next
- 1:10:13agent not to do something
- 1:10:15never ends up actually being anything
- 1:10:16that you can trust.
- 1:10:18So
- 1:10:19in Graphbot and Dune and in Dune
- 1:10:22we now have lint rules that give
- 1:10:25good and very helpful error messages
- 1:10:26telling the agent to fix the root cause,
- 1:10:29the real problem
- 1:10:31rather than just patch around it.
- 1:10:34And so our general principle is that
- 1:10:36every time we correct our agents it
- 1:10:37turns into one of these deterministic
- 1:10:39checks or we think about how to
- 1:10:41categorically eliminate that problem in
- 1:10:43the code through better API design,
- 1:10:45through better architecture.
- 1:10:47And if you do this enough times you'll
- 1:10:50find that you have just less things to
- 1:10:52look out for in code review.
- 1:10:54And at some point you'll realize there's
- 1:10:56there's nothing left to review.
- 1:10:58And you can just let agents merge their
- 1:11:00most of their own pull requests.
- 1:11:04All right
- 1:11:05moving on verification is also a really
- 1:11:08big part of how we make our agent more
- 1:11:10autonomous and also increase our
- 1:11:12confidence and trust in their work.
- 1:11:14And our code review process has now
- 1:11:16really shrunk into looking at
- 1:11:18verification artifacts instead. So
- 1:11:20things like videos, screenshots,
- 1:11:22diagrams, formal verification.
- 1:11:26And the most basic example of this is
- 1:11:28runtime verification,
- 1:11:30which is just a fancy way of saying
- 1:11:31giving your agent a way to run the code
- 1:11:33for itself and use debugging tools that,
- 1:11:36you know, human developers use as well,
- 1:11:38like the Chrome DevTools protocol.
- 1:11:40Um in our runtime verification skills,
- 1:11:42we have this thing that we call a
- 1:11:44feature map.
- 1:11:45And that's basically kind of like a
- 1:11:47sitemap of all of the features that are
- 1:11:49available in your application from the
- 1:11:51user's point of view. And so with this,
- 1:11:54agents don't need to guess when you say,
- 1:11:56you know, the thingy in the left
- 1:11:57sidebar, it gives you some shared
- 1:11:59vocabulary.
- 1:12:01And we also make use of custom CLIs
- 1:12:03inside these skills so that uh agents
- 1:12:06can reuse the the CLI and the driver
- 1:12:09every time, which cuts down on token
- 1:12:10usage and it's also testable.
- 1:12:14Um and then recently we've also started
- 1:12:16looking at formal verification.
- 1:12:18Uh so we now use TLA+ to do some model
- 1:12:21checking of major user flows such as
- 1:12:23notifications across multiple devices
- 1:12:25and platforms.
- 1:12:27Uh we use a lean as well for proving
- 1:12:28certain algorithms.
- 1:12:30Uh for example, we have our own We wrote
- 1:12:32our own custom virtualization library
- 1:12:34and we use lean there to check that we
- 1:12:36don't have, you know, flickering or
- 1:12:38jittering problems.
- 1:12:40Um and we're still experimenting with
- 1:12:41this approach overall. It seems very
- 1:12:43promising. Uh we've already found quite
- 1:12:45a few bugs uh with formal verification.
- 1:12:49Uh but it's certainly not a silver
- 1:12:50bullet and we're treating it as another
- 1:12:54tool, another layer in our agents'
- 1:12:57environment.
- 1:12:59Uh and then this is a very fast video,
- 1:13:01but another thing that we just started
- 1:13:03looking at is property-based testing as
- 1:13:05well for UIs.
- 1:13:07Uh so we use this library called
- 1:13:08Bombadil.
- 1:13:09Um and this has also caught a few UI
- 1:13:11bugs for us already.
- 1:13:15Okay, so zooming out, I saw this tweet
- 1:13:17from Cindy and 6 months ago she she said
- 1:13:20that she was
- 1:13:21Oh, she was talking about not being able
- 1:13:23to trust her agents because of subtle
- 1:13:25bugs that were getting merged.
- 1:13:29And And what was missing for her in
- 1:13:30being able to trust her agents was the
- 1:13:32verification process.
- 1:13:34And now, six months later, she says that
- 1:13:3670% of her daily tokens go into
- 1:13:38verification. And I'd say that that
- 1:13:40tracks with what I see as well, that
- 1:13:43most of my token usage goes to
- 1:13:44verification.
- 1:13:46And if you really look closely at her
- 1:13:47process, none of these things here are
- 1:13:50new or novel techniques that we've never
- 1:13:52had before.
- 1:13:54These are just good software engineering
- 1:13:56tools, but ones that might have been
- 1:13:58prohibitively expensive to use before as
- 1:14:01they were hard to understand or write or
- 1:14:03maintain.
- 1:14:04But agents drastically cut this cost
- 1:14:07down. And really we are entering a
- 1:14:09golden age of software testing and
- 1:14:10verification because agents let us test
- 1:14:13everything to a much greater degree than
- 1:14:15used to be economical.
- 1:14:18And then formal verification can also
- 1:14:20help you better reason about how your
- 1:14:22product should work and whether your
- 1:14:23agents really understand your intent and
- 1:14:25goals.
- 1:14:26Uh I made this new skill called state
- 1:14:28chart
- 1:14:29in my plugin p stack, potato stack,
- 1:14:32which helps you visualize state machines
- 1:14:33from
- 1:14:35from the user's point of view.
- 1:14:37And I find this much easier to review
- 1:14:39alongside the videos and screenshots
- 1:14:41that we provide.
- 1:14:42Um and so that's been great.
- 1:14:45Sorry for the flashback.
- 1:14:47Uh okay, now let's look at what the
- 1:14:49promised land looks like. What do you
- 1:14:51get in return for all of this you know
- 1:14:53investment?
- 1:14:56So on the Graph Bot team, our velocity
- 1:14:58is taking off and rapidly increasing.
- 1:15:01And last month our team shipped
- 1:15:04a ridiculous 11,000 pull requests to
- 1:15:06production,
- 1:15:08which is a wild number.
- 1:15:09And our pace of releasing the client has
- 1:15:12also increased. And we cut a new stable
- 1:15:14version about once a day.
- 1:15:18And then the core team is also shipping
- 1:15:19much more. And we also have a lot more
- 1:15:23casual or drive by contributors who
- 1:15:25don't always have a lot of context,
- 1:15:28but they're also contributing.
- 1:15:30And in October we had
- 1:15:32about 227 people who contributed at
- 1:15:35least one pull request.
- 1:15:38So despite this velocity accelerating,
- 1:15:40you might be thinking to yourself like
- 1:15:41isn't this going to be chaos? You know,
- 1:15:43isn't this going to be a bunch of slop?
- 1:15:45But actually, the number of PRs that get
- 1:15:47reverted has actually gotten lower on a
- 1:15:49relative basis and is steady at about
- 1:15:51half a percent.
- 1:15:54But I think the really interesting thing
- 1:15:56about this shipping pace and this volume
- 1:15:58of pull requests
- 1:16:00is that the performance of Graph Bot has
- 1:16:03stayed not just stable, but has improved
- 1:16:05on multiple dimensions compared to when
- 1:16:07we launched.
- 1:16:08So startup, typing, opening new chats,
- 1:16:11switching chats, all of this has gotten
- 1:16:13faster.
- 1:16:15And keep in mind Graph Bot is still an
- 1:16:17Electron app
- 1:16:19for now.
- 1:16:20So there is a ceiling, right? But it's
- 1:16:22already much faster than many other
- 1:16:24agent UIs out there.
- 1:16:26And for an Electron app with this high
- 1:16:29velocity and this many people
- 1:16:31contributing,
- 1:16:32I'd say that's pretty good.
- 1:16:35So before you jump into building out a
- 1:16:37software factory and you know,
- 1:16:38automating everything that you can,
- 1:16:41whenever you correct your agent, try and
- 1:16:42do it by improving your codebase first.
- 1:16:45You know, start with a lint rule to
- 1:16:46prevent new occurrences of anti-patterns
- 1:16:49and then clean up those old patterns as
- 1:16:51well. And then make sure that your
- 1:16:53verification story is really strong.
- 1:16:56A lot of this work is a prerequisite to
- 1:16:58automation anyway. And if your inner
- 1:17:01coding loop produces slop, then
- 1:17:04automating that loop just produces more
- 1:17:06slop and you need to throw more and more
- 1:17:08and more tokens at the problem, which we
- 1:17:10all know is very expensive.
- 1:17:13And once you make these investments
- 1:17:14into, you know, code into your codebase,
- 1:17:17static analysis, runtime informal
- 1:17:19verification,
- 1:17:20you'll start to find that many things
- 1:17:22become possible with these foundations
- 1:17:25and that you can really easily build
- 1:17:26your own factory on top of these
- 1:17:28foundations.
- 1:17:31And when you're ready, Graph Bot is a
- 1:17:32great way to build that hour loop. We
- 1:17:35have tons of connectors, routines that
- 1:17:36can kick off Cursor, cloud agents, and
- 1:17:39projects
- 1:17:40uh which let you orchestrate thousands
- 1:17:41of uh hundreds or even thousands of
- 1:17:43agents.
- 1:17:44And as a quick example, you know,
- 1:17:46whenever we cut a new release now,
- 1:17:48Grogbot will automatically spin up a new
- 1:17:50Cursor project for us that will fuzz the
- 1:17:53entire application, the entire build,
- 1:17:55and it will find bugs uh by running the
- 1:17:57app and using it like a real user.
- 1:17:59And then it's going to open fixed PRs
- 1:18:01for each of those bugs that it finds,
- 1:18:03and it's going to auto merge it as uh
- 1:18:06crazy as that sounds. And that's because
- 1:18:08our codebase is good enough that pull
- 1:18:10requests tend to just be good by
- 1:18:12default, and most product code that we
- 1:18:14land no longer really requires deep
- 1:18:16human review.
- 1:18:20So
- 1:18:21when you have solid foundations and good
- 1:18:23organic architecture, you can turn slop
- 1:18:26into gold, and you can ship with high
- 1:18:28quality uh
- 1:18:30high velocity without compromising on
- 1:18:32quality.
- 1:18:33My name is Potato, also known as Lauren.
- 1:18:35Thanks to you again for listening to my
- 1:18:36talk.
- 1:18:38>> [applause]
- 1:19:02>> That concludes this morning's uh keynote
- 1:19:05sessions. Thank you for your patience
- 1:19:06with the various technical challenges.
- 1:19:09The root cause was of course a GitHub
- 1:19:11outage.
- 1:19:12Uh
- 1:19:13not really, but the fact that no one
- 1:19:15would think to question that does say
- 1:19:16something. I can say that because they
- 1:19:18are not sponsors, but do you know who
- 1:19:20is? That's right, presenting sponsor
- 1:19:22Google DeepMind. Another round of
- 1:19:24applause for them, please. Without them,
- 1:19:26this would not be doable.
- 1:19:29[applause]
- 1:19:30Of course, our Lounge and Platinum
- 1:19:32sponsors as well make the Lounges and
- 1:19:34Platinum things there. I am not allowed
- 1:19:37to say snarky things about AWS for once.
- 1:19:40So we're going to smile, nod, and keep
- 1:19:41it rolling. Our gold sponsors as well
- 1:19:44have been incredibly supportive of
- 1:19:46various things, not my nonsense, but you
- 1:19:49know, most of the rest of the good
- 1:19:50stuff, absolutely. And let's give it up
- 1:19:52for our silver sponsors, too.
- 1:19:56[applause]
- 1:19:59And finally, all of our supporting
- 1:20:01partners as well. There are uh and of
- 1:20:05course viewers like you to quote PBS
- 1:20:07from my childhood. I'm old back when we
- 1:20:09used to watch it on the Telegraph. There
- 1:20:11are now some breakout sessions. My
- 1:20:13favorite part of those if you erase part
- 1:20:14of the letter B, it reads freak out
- 1:20:16sessions. You can read into that what
- 1:20:17you'd like. There are another series of
- 1:20:20closing keynotes later today starting at
- 1:20:224:30 and I'll see you all back here then
- 1:20:25if they don't wind up having a
- 1:20:26conversation for me out back next to the
- 1:20:28dumpsters with some broken beer bottles.
- 1:20:30Thank you all for coming.
- 1:20:33>> [applause]
- 1:30:40>> I'm Ari, the founder of Intrinsic AI.
- 1:30:44This talk is for people who build agents
- 1:30:47and run their own inference stack or
- 1:30:50plan to. And if that's you, you
- 1:30:52associate typically the KV cache, the
- 1:30:54burden butter of
- 1:30:56modern inference engines usually as a
- 1:30:57throughput optimization, the keys and
- 1:30:59values computed per token kept so the
- 1:31:01model doesn't compute them again. Today,
- 1:31:03I'll like to show KV cache in a
- 1:31:06different light.
- 1:31:07The way agents, agent, AI agent
- 1:31:09application could go wrong are keep
- 1:31:11growing, right? Prompt injection,
- 1:31:12jailbreaks, broken tool calls, sensitive
- 1:31:14tool calls, sensitive data exposures,
- 1:31:16hidden backdoors, and drifts, and so
- 1:31:19forth, right? So the way right now we
- 1:31:21typically guard against against these
- 1:31:23numerous ways that an AI agent
- 1:31:24application could go wrong is by
- 1:31:26guarding the edge, by perhaps passing a
- 1:31:29judge model that checks the
- 1:31:32prompts, the inputs, and the response
- 1:31:34outputs against your policy,
- 1:31:36right? Or you build a classifier for
- 1:31:38each case.
- 1:31:40But here is a different light. The model
- 1:31:42turns every
- 1:31:44input into a computation before it acts,
- 1:31:46before it decodes, before it generates
- 1:31:48the response, before that response
- 1:31:50becomes another tool call. The KV cache
- 1:31:53is your engines record of that
- 1:31:55computation that modern inference engine
- 1:31:57for free are using it and reading it.
- 1:32:00And therefore
- 1:32:02in this talk I would like to suggest as
- 1:32:04as a complement to the way you're
- 1:32:06already using your own guardrails to use
- 1:32:08this data and integrate it with your KV
- 1:32:11cache infrastructure.
- 1:32:13I refer to this as a KV for security
- 1:32:15signals and basically through reasons
- 1:32:18that matters, right? So the data is
- 1:32:21already there.
- 1:32:22You pick the gems from the pile your
- 1:32:24inference stack already were throwing
- 1:32:26away.
- 1:32:27It covers the guardrails blind spots
- 1:32:30because whatever it slips past those
- 1:32:32guardrails it still becomes computation
- 1:32:35and it stays in place so it's perfect
- 1:32:37for data privacy application for cases
- 1:32:39where you do not have the luxury to read
- 1:32:41the inputs and outputs.
- 1:32:43And the recipe is pretty
- 1:32:44straightforward, right? It has three
- 1:32:46steps. You learn a baseline for your AI
- 1:32:49workload, for your AI inference traffic,
- 1:32:51how your KV page typically look like and
- 1:32:54then find the KV pages that are rare
- 1:32:58against your baseline, typically your
- 1:33:00anomalies. And then you can even
- 1:33:02classify and explain those. So first
- 1:33:05let's start with where in this stack
- 1:33:07this integration could happen.
- 1:33:10The interesting thing about the KV cache
- 1:33:12is that in a way existence of those had
- 1:33:15made the inference in a way stateful,
- 1:33:18right? So for example things like KV
- 1:33:20cache management layers such as LM cache
- 1:33:23stores them
- 1:33:24from GPU down to a remote storage or LMD
- 1:33:28and NVIDIA Dynamo route each new request
- 1:33:30towards the worker that already holds
- 1:33:32perhaps that
- 1:33:34cached KV page. So there are two
- 1:33:36integration points you you you can
- 1:33:39integrate those anomaly detection or
- 1:33:40those classifiers into.
- 1:33:43Either you can implement it directly
- 1:33:45inside of your inference engine via LM
- 1:33:47SG line so forth. The beauty of this
- 1:33:49approach is that it's real time
- 1:33:51and and the the the alert that it would
- 1:33:54perhaps generate could ride along your
- 1:33:58future tool calls or your future
- 1:34:00policies,
- 1:34:01so you can harden basically your policy
- 1:34:04or guard rails, but just generating the
- 1:34:06computational anomalies that you observe
- 1:34:09within your inference engines.
- 1:34:11And you perhaps you can integrate it
- 1:34:13with as a as a way of threat discovery
- 1:34:15in a sense. So you can integrate with
- 1:34:18your LLM cache. So reads the stored KB
- 1:34:22pages and find things that don't belong
- 1:34:24with the rest, right? So now I want to
- 1:34:26show you some numbers and
- 1:34:28just a very simple hello world use case
- 1:34:31if if this is actually working and this
- 1:34:33is a valid approach. And for that, let
- 1:34:36me use the use case that has no text to
- 1:34:39filter at all, a voice agent.
- 1:34:42So here's a case study of a voice agent.
- 1:34:44In this case, it was a gem of 4E4B. Our
- 1:34:46baseline was 72 ordinary clips and
- 1:34:50spoken in four voices. We're looking at
- 1:34:5216 token pages. Obviously this granular
- 1:34:54granularity could change,
- 1:34:56but here it was just use a
- 1:34:59shorter pages so we could have more data
- 1:35:01in the baseline. And then on the test,
- 1:35:03we have different voices and one clean
- 1:35:07voice and the other one has a hidden
- 1:35:08instruction within it. And as you can as
- 1:35:11as is shown here, not only you can
- 1:35:13detect these cases
- 1:35:15just using simple anomaly detection, but
- 1:35:18also you would know which part of
- 1:35:22which page, which part of your request
- 1:35:25had this anomalous event. We even went
- 1:35:27further and we trained a model that uses
- 1:35:30the KB pages as features.
- 1:35:33And we didn't be trying to explain it or
- 1:35:34tag it with things that are useful. In
- 1:35:37this case, although it wasn't even
- 1:35:38trained on audio, is able to generate
- 1:35:41correct information about personal
- 1:35:42information, home address, location
- 1:35:44data, travel information, and
- 1:35:45destination.
- 1:35:49So now back to text agents. In this
- 1:35:51case, it was a 3 14 billion live
- 1:35:55cache.
- 1:35:56The one on the left shows you an idea of
- 1:35:59how attacks and normal traffic would
- 1:36:01typically separate from each other. And
- 1:36:03then the one on the right, which is
- 1:36:04perhaps more meaningful, it was using
- 1:36:07the AT bench and hugging face.
- 1:36:10It has its own baseline of benign agent
- 1:36:13traces and eight kind of agent failure
- 1:36:16from injected tools description to
- 1:36:18misinformation and so forth.
- 1:36:20We look at the top 10 alerts that
- 1:36:23anomalous anomaly detection would
- 1:36:24generate and about seven to eight times
- 1:36:27of out of 10 of these alerts are
- 1:36:30actually anomalous meaningful and useful
- 1:36:32anomalous activities. We even went
- 1:36:35further and we looked at whether there's
- 1:36:38any kind of anomalous event that is
- 1:36:40stand out more than the rest.
- 1:36:44Um to make things hard in production,
- 1:36:47first, models change.
- 1:36:50Fine tunes and new releases, but also
- 1:36:53agents that each run on a different
- 1:36:55model
- 1:36:56and routers that pick a specialist model
- 1:36:59per request. That means each model
- 1:37:01writes its pages its own coordinates. So
- 1:37:04either you learn every baseline,
- 1:37:06classifier and explainer, which is
- 1:37:09computationally prohibitive,
- 1:37:10or you have to find a way to map them
- 1:37:14all into one shared representation
- 1:37:16space.
- 1:37:18Second, real time means moving big
- 1:37:21objects. For example, a 1 256 token
- 1:37:25chunk of coin 314 billion is about 40
- 1:37:28megabytes of KB.
- 1:37:30Compression and streaming like cache gen
- 1:37:33could help, but moving that for security
- 1:37:36analysis and acting in real times is
- 1:37:37still slow. And that's what we solve at
- 1:37:40Intrinsic AI. We are looking for teams
- 1:37:42building their own inference stacks who
- 1:37:44treat security as a foundational design
- 1:37:46decision.
- 1:37:47I hope this was helpful for you. Thank
- 1:37:49you so much for listening.
- 1:37:50>> Let me start with one number from last
- 1:37:52month.
- 1:37:53On average, more than one agent has
- 1:37:55started on our platform every single
- 1:37:57minute,
- 1:37:59day and night.
- 1:38:01And honestly,
- 1:38:02the agents were the easy part.
- 1:38:05Hi, I'm Joey.
- 1:38:07I built Spectre, the cloud agent
- 1:38:09platform we're building inside Harvey.
- 1:38:11After 10,000 PRs,
- 1:38:14the biggest lesson I can share is this.
- 1:38:16The thing that broke was an agent.
- 1:38:19It was everything around them.
- 1:38:23So what does one submit it add up to?
- 1:38:2610,000 PRs that an agent wrote.
- 1:38:29And it isn't just the team playing with
- 1:38:32a toy.
- 1:38:34Last month, about 460 different people
- 1:38:38at Harvey used it.
- 1:38:40And over the summer, it stopped being
- 1:38:42only about code.
- 1:38:44People started asking a data question in
- 1:38:46Slack and getting answers back.
- 1:38:49My favorite
- 1:38:51Recently, someone on our marketing team
- 1:38:53dropped a prototype in the talk track
- 1:38:55into Slack
- 1:38:56and asked for a storyboard for a product
- 1:38:58video.
- 1:39:00Once the harness exists, people bring
- 1:39:02the work that you never planned for.
- 1:39:06So a promise upfront, this is not a talk
- 1:39:09about models, evals, or leaderboards.
- 1:39:12This is talk about harness,
- 1:39:15odd things around the model.
- 1:39:18Checking out the repo, holding the key,
- 1:39:21opening the PR, getting the merge.
- 1:39:24When we first wrote about Spectre back
- 1:39:26in April,
- 1:39:27we put it like this.
- 1:39:29At some point, the harness become the
- 1:39:31product.
- 1:39:33Now, whenever I tell people this, I get
- 1:39:35the same question.
- 1:39:37Why on earth would you build this when
- 1:39:39Cloud Code or Cursor already exists?
- 1:39:43My answer
- 1:39:44We don't compete with it. We run them.
- 1:39:47And here's how I see it as my own view.
- 1:39:51Agents on your laptop are powerful, but
- 1:39:53they don't scale.
- 1:39:54You can realistically run five to 10 of
- 1:39:57them.
- 1:39:58Plus that, you need the cloud.
- 1:40:00And the cloud needs our secrets,
- 1:40:02our real stack,
- 1:40:04our alerts and tickets, one set of
- 1:40:06rules, and enough capacity that nobody
- 1:40:10gets starved.
- 1:40:11No single tool owns that for us, so we
- 1:40:13built layer, and that layer is exactly
- 1:40:16what broke. So I want to tell you three
- 1:40:19stories today.
- 1:40:21How we stopped competing on writing
- 1:40:22code,
- 1:40:24how we let any agent drive
- 1:40:26and what broke once they did.
- 1:40:30The first story starts with the question
- 1:40:31we keep hearing.
- 1:40:34More of our engineers were using Cursor
- 1:40:37and they ask,
- 1:40:38how do I get our secret there?
- 1:40:41How do I test against our real stack?
- 1:40:43We could have rebuilt all that inside
- 1:40:46Cursor and then again for next tool
- 1:40:49and the one after that.
- 1:40:51Instead, the answer became one sentence.
- 1:40:54Just ask Cursor to use Specter.
- 1:40:58And that's really the whole idea.
- 1:41:00People will always have a favorite agent
- 1:41:03and the favorite keeps changing.
- 1:41:05So pick yours, Claude code, Codex,
- 1:41:07Cursor, ChatGPT, dot, Slack, or your own
- 1:41:09hands.
- 1:41:11Start on your laptop, push the session
- 1:41:13to the cloud, pull it back.
- 1:41:16That's how Specter sits underneath all
- 1:41:18of that.
- 1:41:20Because once the code was written, the
- 1:41:22real work starts.
- 1:41:24Picture a PR stay.
- 1:41:26Alert files in Slack at 2:00 in the
- 1:41:28morning.
- 1:41:30An agent picks up. Nobody type a thing.
- 1:41:33It writes a fix, spins up real copy of
- 1:41:36our stack,
- 1:41:37and posts the proof on the PR.
- 1:41:40When CI fails, the follow-up loop reads
- 1:41:43the failure and posts another fix while
- 1:41:45everyone's still asleep.
- 1:41:48By morning, the PR sits in one review
- 1:41:50queue under the same rule as other PR,
- 1:41:54which our agent wrote it.
- 1:41:57The owner gets a nudge
- 1:41:59and only once they are working they
- 1:42:00starts.
- 1:42:02And once approved, a agent can auto
- 1:42:04merge it.
- 1:42:07If you only built one of those stages
- 1:42:10that I described, build the follow-up
- 1:42:12loop.
- 1:42:13We switched it on for everyone in early
- 1:42:15October, and that month four out of five
- 1:42:18agent PRs merged into our backend PR
- 1:42:22had been through it.
- 1:42:24That's the difference between the agent
- 1:42:26that write code
- 1:42:27and an agent that finished the job.
- 1:42:31And then there's the knowledge. Every
- 1:42:33team has that one person who knows
- 1:42:35exactly how to test the tricky service
- 1:42:38which flag you never touch.
- 1:42:40And that knowledge lives in their head
- 1:42:43or their laptop.
- 1:42:45So we gave it a home, projects.
- 1:42:49People write down what matters.
- 1:42:51Adrian write down what they learn.
- 1:42:53Team pings the skills it trusts, and
- 1:42:55there's a record of who ran what.
- 1:42:59Any agent working in the project with
- 1:43:00the same page,
- 1:43:02my personal view,
- 1:43:04whichever agent you pick, the audit
- 1:43:05trail and the tracks should be the same.
- 1:43:09The second story is about rule.
- 1:43:13Back in May, if you want an inspector to
- 1:43:15do something, you open a browser tab
- 1:43:18or tagging Slack.
- 1:43:20The whole month exactly one task came
- 1:43:23out from terminal.
- 1:43:25So we made a rule, if a person can click
- 1:43:27it,
- 1:43:28an agent can call it.
- 1:43:30Start a task,
- 1:43:32turn on the follow-up loop on auto
- 1:43:34merge, publish a page off from CLI.
- 1:43:39Which means your laptop agent can call
- 1:43:41it.
- 1:43:42Agent inside the sandbox can also call
- 1:43:44it.
- 1:43:45Even CI can call it too.
- 1:43:47And then people ran with it.
- 1:43:49By October, the command line was
- 1:43:51starting about 6500 tasks a month.
- 1:43:56The web app didn't shrink.
- 1:43:58Everyone else just grew around it.
- 1:44:01Task started from the terminal were
- 1:44:03merging more PRs than task started in
- 1:44:05browser.
- 1:44:06And a new kind of user showed up.
- 1:44:09People who start hundreds of tasks a
- 1:44:11month and never open the app at all.
- 1:44:14And this is where it gets fun.
- 1:44:17In October, I had a big task.
- 1:44:20So I call code session and ask to be the
- 1:44:22coordinator.
- 1:44:24It split the tickets into 10 different
- 1:44:26milestones.
- 1:44:28Get one for me and the rest to Specter
- 1:44:31with hundreds of tasks.
- 1:44:34Each has its own sandbox to validate end
- 1:44:36to end.
- 1:44:38Then at the end of the October, our IT
- 1:44:40give everyone ChatGPT tasks.
- 1:44:43My now watch Specter for me.
- 1:44:45It lists the questions Specter is
- 1:44:46waiting on, draft my answers, and once I
- 1:44:50say yes, send them back.
- 1:44:53I actually DM to my friends like that.
- 1:44:56I just use cursor, clock code, or
- 1:44:58whatever to spam Specter now.
- 1:45:01Well, that's the dream, but it's also
- 1:45:03the problem.
- 1:45:05A quick detour.
- 1:45:06When we look at where a task time
- 1:45:08actually went, the slowest step wasn't
- 1:45:11model syncing. It was Git.
- 1:45:14Just checking out every repo we owned
- 1:45:16took over a minute on average.
- 1:45:20The fix, keep a copy of every repo on
- 1:45:22every machine
- 1:45:24and let each task borrow it.
- 1:45:27The third story is the one where it
- 1:45:30broke.
- 1:45:31In mid August,
- 1:45:33a release goes out with a bug.
- 1:45:35That bug triggers alert that fires once
- 1:45:38per workspace.
- 1:45:39So one bug become hundreds of alerts
- 1:45:42all landing in one single Slack channel.
- 1:45:45Someone has set up workflow that tag our
- 1:45:47agent on every alert
- 1:45:49and every tag started task.
- 1:45:52On normal day, those tasks started about
- 1:45:5550 tasks.
- 1:45:56That day, in about 4 hours, they started
- 1:45:59around 900.
- 1:46:01900 agents are chasing the same bug
- 1:46:04until that wasn't room left for a single
- 1:46:06node.
- 1:46:08The fix was almost boring.
- 1:46:10A A alert now joins the investigation
- 1:46:12that's already open.
- 1:46:15Two months later, it happened again.
- 1:46:18And this time, it wasn't about
- 1:46:21On October,
- 1:46:22the engineer did exactly what we've been
- 1:46:25asking everyone to do.
- 1:46:27Hand work to Specter.
- 1:46:30About 950 tasks from the command line.
- 1:46:33In about 3 hours, nothing in our
- 1:46:36platform can work.
- 1:46:38The cluster filled up and everyone else
- 1:46:41work sat in line behind one laptop.
- 1:46:44That's not a user problem. That's our
- 1:46:46problem.
- 1:46:494 days later, we had a real fix, a fair
- 1:46:52launch queue.
- 1:46:54Now, starting a task gives you a place
- 1:46:57in line.
- 1:46:58And the dispatcher started when they're
- 1:47:00through.
- 1:47:01People sitting at keyboard goes first.
- 1:47:04Everyone gets a fair share and can
- 1:47:06borrow more than the cluster is quite.
- 1:47:09And whenever you start a task, you can
- 1:47:12see a place in line.
- 1:47:13When we hit it with a 300 task bursting
- 1:47:16low test,
- 1:47:18the task list went from almost 2 seconds
- 1:47:20to 70 milliseconds.
- 1:47:22So did it work?
- 1:47:25When it's the busiest day of every week
- 1:47:28back in August, 2 sudden task and about
- 1:47:311 in 10 didn't finish.
- 1:47:35Then in November 4th, more than twice
- 1:47:37that
- 1:47:38and
- 1:47:39much better in the finish line, but we
- 1:47:42still have some room to grow.
- 1:47:45I will be modest. We changed several
- 1:47:46things at once, so I cannot tell you
- 1:47:49which fix mattered the most.
- 1:47:51But the burst kept getting bigger and
- 1:47:53the platform kept up.
- 1:47:56And just when that was under control,
- 1:47:58the bottleneck moves.
- 1:48:00At end of the October,
- 1:48:02our dev platform team raised alarm.
- 1:48:06PRs across the company had jumped around
- 1:48:0960% in a single month.
- 1:48:12A merge that used to take 15 to 30
- 1:48:14minutes was taking over an hour now.
- 1:48:17And merge queue kept having to be
- 1:48:18switched off.
- 1:48:20So now a PR rebased onto main, we run CI
- 1:48:24and merge if it's green.
- 1:48:27If main breaks, everyone CI breaks with
- 1:48:29it.
- 1:48:30Honestly,
- 1:48:31tell you, we haven't cracked this one
- 1:48:33yet.
- 1:48:35But before I go, there's one more thing.
- 1:48:38While we were busy fixing all of it,
- 1:48:41something else was quietly growing.
- 1:48:44Most of what I've been showing you start
- 1:48:47with a person asking
- 1:48:49the fastest growing cells of the work on
- 1:48:52Specter doesn't.
- 1:48:54These are automations, agent that wake
- 1:48:57up on schedule on event and just do the
- 1:49:00work.
- 1:49:01Treasure the alert, read the PR, write
- 1:49:04the post.
- 1:49:06Last month, about 110 people had
- 1:49:09automation running and we learned to
- 1:49:11give them a guardrail on day one.
- 1:49:14If one fails five times in a row,
- 1:49:16it switches itself off.
- 1:49:18And it can carry a budget or expiry date
- 1:49:22because when no one's watching, an agent
- 1:49:25can start thousands of tasks too.
- 1:49:28So here's what I'm chasing next.
- 1:49:31And I don't have an answer yet,
- 1:49:32honestly.
- 1:49:34What if every Slack channel had its own
- 1:49:36agent that never locks off?
- 1:49:39One that follows the condition and start
- 1:49:41work on its own.
- 1:49:43What if every PR proof it works before a
- 1:49:45human even look at it?
- 1:49:48Today, only one merge in 20 carries that
- 1:49:51kind of proof.
- 1:49:53And how do you merge when agents never
- 1:49:56stop opening PRs?
- 1:49:58Ask me again six months.
- 1:50:00I will tell you what broke.
- 1:50:03If you building something like this when
- 1:50:04you work, here's three things I'd do
- 1:50:06again.
- 1:50:08Own what happens after the code is
- 1:50:10written.
- 1:50:11Make Eric action something an agent can
- 1:50:13call.
- 1:50:15And cap what an agent, human, or can
- 1:50:18start before they find out what they can
- 1:50:21do with it.
- 1:50:23And one last thought.
- 1:50:25That's my own view.
- 1:50:27Engineers can write code with whichever
- 1:50:29agent they like.
- 1:50:31The work that's lost is the harness that
- 1:50:33runs it,
- 1:50:34checks it and get it merged.
- 1:50:38Thank you so much for watching.
- 1:50:41Come find me on LinkedIn if you are
- 1:50:43building similar things.
- 1:50:46>> Hi everyone. My name is Rajat Kotial. I
- 1:50:48am part of the automated reasoning group
- 1:50:50at AWS.
- 1:50:52And today I'm going to talk about a
- 1:50:54project I worked on recently called hero
- 1:50:57specification analyzer.
- 1:50:59And we use certain techniques which
- 1:51:01involves formal methods and automated
- 1:51:03reasoning to prove
- 1:51:05specification that goes to your coding
- 1:51:07agent before it writes code.
- 1:51:16So let's take a certain bug, right?
- 1:51:20This is a specification
- 1:51:24snippet and there are two rules for
- 1:51:26example and rule two says when an order
- 1:51:29is submitted and this is for a a general
- 1:51:32e-commerce website let's say.
- 1:51:35When an order is submitted and the
- 1:51:36inventory is available, we place the
- 1:51:39order as a back order.
- 1:51:41Which makes sense when you read it
- 1:51:42individually. And there was also another
- 1:51:44rule in the same specification file
- 1:51:46which you hand to the coding agent which
- 1:51:48says
- 1:51:49no order shall be placed as a back order
- 1:51:52and this is a rule we've set.
- 1:51:54So individually read these two lines
- 1:51:57make sense, right?
- 1:51:58But in when an LLM reads this
- 1:52:02reads both the rules together, both
- 1:52:03cannot hold true at the same time.
- 1:52:06So we need to make sure
- 1:52:08the we resolve this ambiguity that the
- 1:52:10LLM is
- 1:52:12observing.
- 1:52:14And
- 1:52:17the scope and the reason
- 1:52:20we
- 1:52:22put an effort into this project is
- 1:52:24based on public research and recent
- 1:52:26research, we've observed there's a 20 to
- 1:52:2840% drop with unclear prompts on the
- 1:52:32accuracy the LLM
- 1:52:34generates and the code accuracy, right?
- 1:52:36And 60 to 90% of valid code is still
- 1:52:39wrong after it writes it.
- 1:52:41And also in that case, we observed two
- 1:52:44times the more more regressions when we
- 1:52:47deploy that code.
- 1:52:51So where do bugs like these hide, right?
- 1:52:54We want to detect these bugs before they
- 1:52:56reach the codebase and the only way to
- 1:52:59do that it's similar to how we write
- 1:53:01design docs, similar to how we draw
- 1:53:03diagrams and flow charts to make sure
- 1:53:06the flow is set before we write any
- 1:53:07code. We want to verify that without
- 1:53:10a human or anyone reading these files as
- 1:53:14the first reviewer.
- 1:53:17In one sentence,
- 1:53:19let's say the ambiguity and wording
- 1:53:21wrong level of detail, but across
- 1:53:24sentences, we can observe conflicts and
- 1:53:26gaps in the specification or rules,
- 1:53:28right?
- 1:53:30Sentences need review, such need search
- 1:53:32and between them and ambiguity searches.
- 1:53:36So we have a three-step process to do
- 1:53:39this where the LLM will first refine it,
- 1:53:42it will try to clean up the wording on
- 1:53:43its own
- 1:53:44and if it succeeds, then well and good,
- 1:53:48but if it cannot be then formalized
- 1:53:50to
- 1:53:52English language into SMT formulas.
- 1:53:55So SMT lib is a certain set of type of
- 1:53:59function which we pass to a SAT solver,
- 1:54:02a satisfiability checker where it's
- 1:54:04going to give out a Boolean saying this
- 1:54:06expression that you've given me is
- 1:54:08valid, true or false, invalid.
- 1:54:13And then once the solver gets back to us
- 1:54:15if that expression is valid, then we
- 1:54:17don't do anything, but if it detects any
- 1:54:19ambiguity between two sentences or
- 1:54:21within a sentence where there's
- 1:54:23two meanings to a sentence then we
- 1:54:25detect that and
- 1:54:26tell the LLM
- 1:54:28to recheck that sentence and it's going
- 1:54:30to reiterate on that.
- 1:54:33For example the rules we saw
- 1:54:37we've seen when order submitted and
- 1:54:39inventory unavailable we shall place the
- 1:54:41order as a back order and in SMT
- 1:54:45format it kind of looks like this this
- 1:54:47like a math formula right?
- 1:54:54Once we go deeper we
- 1:54:57once we detect this
- 1:54:58we also do
- 1:55:00a three-step process again where we
- 1:55:02sample the translation again and again
- 1:55:05we ask the LLM to translate seven times
- 1:55:07or 10 times
- 1:55:08and to understand whether this LLM is
- 1:55:11behaving this way
- 1:55:13based on
- 1:55:15its its inherent capabilities or is
- 1:55:17something affecting it to behave this
- 1:55:18way
- 1:55:19and then based on logical equivalence of
- 1:55:21these seven or 10
- 1:55:24different set of interpretations by the
- 1:55:26LLM we cluster them on logical
- 1:55:28equivalence which that which
- 1:55:32entries came under which assumptions
- 1:55:36that their LLM made for itself
- 1:55:38and then we accept or ask the user to
- 1:55:41clear out the ambiguity or abstain from
- 1:55:44implementing that in the code base.
- 1:55:51This is an example of asking the human
- 1:55:55what you want to do with that
- 1:55:59rule that was
- 1:56:00perhaps contradicting.
- 1:56:02Remove the record is very a very vague
- 1:56:04statement the LLM cannot can interpret
- 1:56:07anything it wants but we shouldn't allow
- 1:56:09that to happen and we either give two
- 1:56:12options we keep it as is the record is
- 1:56:14deleted or we retain it and but hide it
- 1:56:17from the users which one does the user
- 1:56:19prefer?
- 1:56:25A conflict is when the examples we
- 1:56:27example we saw rule one is conflicting
- 1:56:29with rule three.
- 1:56:31And a gap is where no rule covers an
- 1:56:33idle order
- 1:56:35in this example
- 1:56:37where the LLM detected an actual sorry
- 1:56:40our
- 1:56:41process or pipeline detected there's no
- 1:56:44entry for this rule where you are
- 1:56:47missing something.
- 1:56:52All in all
- 1:56:53it cannot do a lot of things end of the
- 1:56:55day.
- 1:56:56We want to
- 1:56:57make sure there is no oracle
- 1:57:00for for specification correctness other
- 1:57:02than the user.
- 1:57:03User needs to make sure to check every
- 1:57:07part of the snippet part of the rule
- 1:57:09book before sending it to the LLM.
- 1:57:11And
- 1:57:13we as a tool only can help them.
- 1:57:17And the user is more important here
- 1:57:19because
- 1:57:20of implicit domain knowledge the LLM
- 1:57:22doesn't know a lot of domain knowledge
- 1:57:24and there's no guarantee from an LLM
- 1:57:26only
- 1:57:27user can
- 1:57:29give that guarantee right and there's
- 1:57:32been evidence right now for
- 1:57:33formalization and how it can be used in
- 1:57:35agents we are emerging in this
- 1:57:37technology.
- 1:57:43What do we take back from this
- 1:57:45talk?
- 1:57:46Check the spec always before you pass it
- 1:57:48down to the agent.
- 1:57:50Cluster the translation we what we do is
- 1:57:52we cluster the translation by
- 1:57:55logical equivalent to determine
- 1:57:58the different LLM interpretations and
- 1:58:00where it's genuinely going wrong.
- 1:58:02The LLM translates the solver checks and
- 1:58:05the human decides.
- 1:58:07We give to give the human options and if
- 1:58:10there's no answer to that we use LLM
- 1:58:12judges or abstain from implementing that
- 1:58:15piece.
- 1:58:18Here are the references and this work
- 1:58:21has been majorly contributed by my team
- 1:58:28you can check out their and the
- 1:58:30website link provided.
- 1:58:32And delve deeper into that. Thank you.
- 1:58:35>> Hi, I'm Chinmay and with me are Swapnika
- 1:58:38and Vlad.
- 1:58:39We're excited to share our journey as
- 1:58:41well as our learnings around building an
- 1:58:43AI on-call teammate at Meta
- 1:58:46that now runs real on-call rotations
- 1:58:48every day and around the clock.
- 1:58:51We are part of the engineering team that
- 1:58:53builds the tooling platform and
- 1:58:55infrastructure for incident response,
- 1:58:58on-call experience, and mitigations at
- 1:59:00Meta.
- 1:59:01That space is evolving rapidly with the
- 1:59:03increase in AI capabilities.
- 1:59:07AI changed how fast we write code.
- 1:59:10About three quarters of new code is now
- 1:59:11written with AI
- 1:59:13and more AI-generated changes reach
- 1:59:16production every single day.
- 1:59:19And some of it
- 1:59:20can cause production failures just like
- 1:59:22any other code change can.
- 1:59:25But on-call did not speed up with it.
- 1:59:28High-quality code can pass review and
- 1:59:31still break in production. And when it
- 1:59:33does, someone still gets paged at 2:00
- 1:59:35a.m.
- 1:59:36The bottleneck moved from writing code
- 1:59:39to operating it in production.
- 1:59:41And on-call is where that operational
- 1:59:44load lands. So we asked
- 1:59:46if AI is writing the code,
- 1:59:49can AI also help operate it?
- 1:59:53And we think it can.
- 1:59:56Now let's quickly run through one of
- 1:59:57those on-call pages with a simplified
- 2:00:00example.
- 2:00:02It's 2:00 a.m. and 12 alerts are firing.
- 2:00:05Same night, it can go two different
- 2:00:07ways. On the left is how most of us do
- 2:00:09this today. And notice we already have
- 2:00:12good AI.
- 2:00:13First you sort the alerts by yourself,
- 2:00:16check out payments disk, which one to
- 2:00:18pick first. Let's say you pick checkout.
- 2:00:20Then you ask your assistant why P99
- 2:00:23latency is high.
- 2:00:25It's genuinely useful. It pulls the
- 2:00:27metrics and finds the deployment at
- 2:00:291:49.
- 2:00:31And when you ask, it reads what changed
- 2:00:34and checks the runbook.
- 2:00:36It figured out that the time config was
- 2:00:38changed to a lower value
- 2:00:41and based on the runbook, it suggests to
- 2:00:43rollback, which is all very helpful.
- 2:00:46However, if you look at the shape of it,
- 2:00:48it waited to be asked.
- 2:00:51You picked every next step.
- 2:00:53You were the glue between the questions.
- 2:00:57And once it answered, it went dormant.
- 2:01:00At 2:39, you know the cause,
- 2:01:04but the rollback, the changes, and the
- 2:01:06follow-ups are still yours.
- 2:01:10Now, on the right,
- 2:01:12the same 12 alerts,
- 2:01:14but an AI teammate
- 2:01:16started before you woke up.
- 2:01:19It had already ranked them.
- 2:01:21Checkouts first, payments merged in as
- 2:01:24the same cause, and the other 10 handled
- 2:01:26separately.
- 2:01:28It read the dashboards, diffed the same
- 2:01:30149 AM config change, and matched two
- 2:01:33past incidents.
- 2:01:35You get a hypothesis,
- 2:01:37a 30-second way to verify it,
- 2:01:40and a fix
- 2:01:41waiting for your review.
- 2:01:43At 2:01, you verified it, and you're
- 2:01:46back to sleep.
- 2:01:47That gap is what this talk is about.
- 2:01:53So, why were you the glue?
- 2:01:56Every team was bolting automation onto
- 2:01:58on-call,
- 2:02:00one silo at a time.
- 2:02:02An alert script for alerts, a triage bot
- 2:02:05for tickets, a deploy bot for deploys, a
- 2:02:08fix-it agent for incidents.
- 2:02:10Each one solves its slice,
- 2:02:12keeps its own context, and forgets it.
- 2:02:16Now, replaying your 2 AM page,
- 2:02:18the alert script saw P99 latency about 2
- 2:02:21seconds.
- 2:02:22The triage bot saw three tickets saying
- 2:02:24checkout is slow.
- 2:02:26The deploy bot knew about a config
- 2:02:28change at 149, and the incident history
- 2:02:30had two similar past incidents.
- 2:02:33Every clue existed,
- 2:02:35but nothing connected them.
- 2:02:38So the on-call became the glue exactly
- 2:02:42as we saw on the left side of the
- 2:02:44previous slide.
- 2:02:46Our bet was one AI teammate per rotation
- 2:02:50instead.
- 2:02:52One loop across every signal,
- 2:02:55one team memory that learns every shift,
- 2:02:57its own identity, so it acts as itself
- 2:03:00and a human gate. So you approve instead
- 2:03:03of operate.
- 2:03:05But giving one agent all that context
- 2:03:07and reach raises the stakes,
- 2:03:09which is why autonomy has to be earned.
- 2:03:15Under the hood,
- 2:03:16the teammate or the AI teammate is the
- 2:03:18simplest agent there is.
- 2:03:21Model using tools in a loop,
- 2:03:23acting on an environment and reading the
- 2:03:26feedback.
- 2:03:27Everything else in this talk is what we
- 2:03:29[music] built around that loop
- 2:03:31to make it safe to leave it running.
- 2:03:34And for that, I will hand over to
- 2:03:36Swapnika and Vlad next to get into the
- 2:03:39actual details of how it works.
- 2:03:43>> To achieve this, we started with an
- 2:03:45autonomy ladder.
- 2:03:46Think about how you onboard a new
- 2:03:48engineer to on-call.
- 2:03:49First, the shadow.
- 2:03:51They investigate and suggest, but they
- 2:03:53don't touch production.
- 2:03:55Then the pair.
- 2:03:57They write the fix and another engineer
- 2:03:59approves it. With time, they gain
- 2:04:01knowledge.
- 2:04:03We give our agent exactly the same
- 2:04:05ladder.
- 2:04:06At level one, it investigates and tells
- 2:04:08you what it found. It changes nothing.
- 2:04:11At level two, it acts with approval.
- 2:04:13It drafts the fix, but nothing reaches
- 2:04:15production until a human approves.
- 2:04:18At level three,
- 2:04:19it clears lower criticality work on its
- 2:04:22own and only interrupts you for what
- 2:04:24matters.
- 2:04:25High criticality work stays with humans.
- 2:04:29So what earns the next run?
- 2:04:32We use a simple test. How costly is a
- 2:04:34mistake and how easy is it to catch?
- 2:04:37So an action claims only when mistakes
- 2:04:40are cheap, they are caught by tests, CI
- 2:04:43or review
- 2:04:45and the agent already has a track record
- 2:04:47on that exact action.
- 2:04:49That's why the agent can close a noisy
- 2:04:52alert on its own while a production
- 2:04:54mitigation still waits for a human.
- 2:04:58At level one, the investigation is
- 2:05:00layered and stops at the first conflict
- 2:05:02answer.
- 2:05:03The agent investigated the issue using
- 2:05:05the team's memory and runbook. It also
- 2:05:08shared the root cause analysis,
- 2:05:10but human had to apply the fix.
- 2:05:13At level two,
- 2:05:15it writes the fix with the root cause
- 2:05:17and a test plan.
- 2:05:19It answers reviewer's comments and fixes
- 2:05:21CI,
- 2:05:23but nothing reaches production until a
- 2:05:24human approves.
- 2:05:26Most drafts don't land and that's fine.
- 2:05:30Review is the filter.
- 2:05:32At both levels, the agent does the
- 2:05:34homework,
- 2:05:35but you still had to grade every single
- 2:05:38paper.
- 2:05:39They have got much better,
- 2:05:41but the tasks requiring human input
- 2:05:43didn't get smaller.
- 2:05:45Level three shrinks the pile.
- 2:05:50Look at this example.
- 2:05:5123 items came in, but only three need
- 2:05:55you.
- 2:05:56On the right,
- 2:05:57the agent handled the long tail, noisy
- 2:05:59alerts, duplicates, routine fixes that
- 2:06:02went through normal review and CI.
- 2:06:05Two more are still being worked.
- 2:06:08You don't have to drive each one to
- 2:06:09completion.
- 2:06:11In the middle is your much smaller
- 2:06:13decision queue.
- 2:06:14Each item makes clear what it needs from
- 2:06:17you with a run this button that has a
- 2:06:20call to action.
- 2:06:21And remember our 2:00 a.m. checkout
- 2:06:23page,
- 2:06:24that still stays human own.
- 2:06:26The agent brings the investigation, the
- 2:06:28human owns the critical decision.
- 2:06:31It's the new high ladder again.
- 2:06:33Trusted to handle routine work, ready
- 2:06:36with the homework when it needs help,
- 2:06:38not just less work per item,
- 2:06:40but fewer items waiting on you.
- 2:06:45That board is where the work lives, but
- 2:06:47nobody stares at a board all the time,
- 2:06:49right?
- 2:06:50So the agent is also available across
- 2:06:53many surfaces like chat and native
- 2:06:55incident surfaces.
- 2:06:57At 9:00 every morning,
- 2:06:59it posts a digest.
- 2:07:0123 came in,
- 2:07:02three need you, 18 handled.
- 2:07:05Each thing that needs you come with a
- 2:07:08one click next step.
- 2:07:10So approving the quota bump
- 2:07:12takes one tap from your phone.
- 2:07:14The 18 at handled are summarized, not
- 2:07:17listed.
- 2:07:18And the two that are still in progress
- 2:07:20tell you what it's doing about them.
- 2:07:22And when you want to dig in, you just
- 2:07:25reply in the thread.
- 2:07:27As you see on the right,
- 2:07:28here you open the checkout incident and
- 2:07:30ask for the evidence.
- 2:07:32It already has it.
- 2:07:34Errors start at 8:41.
- 2:07:36Two minutes after a config push, cut the
- 2:07:38payment client timeout.
- 2:07:40The same pattern as two past incidents.
- 2:07:43You ask a follow-up, is it only EU? And
- 2:07:45it answers with a breakdown.
- 2:07:47It drafts the role back. You approve it
- 2:07:50right there and errors start recovering.
- 2:07:53You never opened a dashboard and you
- 2:07:55never had to explain the context
- 2:07:57because it was already there.
- 2:07:58That's what a teammate feels like. A
- 2:08:00morning handoff a colleague you can ask.
- 2:08:06Now
- 2:08:07another hood of this AI teammate it's AI
- 2:08:09automations.
- 2:08:11The first investigates.
- 2:08:13On every new signal, auto triage
- 2:08:16classifies it,
- 2:08:17checks its criticality and kicks off the
- 2:08:19right agent.
- 2:08:21An alert goes to the incident
- 2:08:23investigator,
- 2:08:24a task to the team's own agent,
- 2:08:26a test failure to a coding agent,
- 2:08:29a lookup to memory and code search.
- 2:08:32The second
- 2:08:34keeps the system healthy.
- 2:08:35Unattended automation fills quietly.
- 2:08:39So So a schedule, it looks stuck or
- 2:08:41failed runs,
- 2:08:42retrace and repairs broken schedules,
- 2:08:45and it only pages a human if it can't
- 2:08:48fix it itself.
- 2:08:50Nothing stalls silently.
- 2:08:53The third, hands off.
- 2:08:55At every shift change and every morning
- 2:08:57at 9:00, it summarizes the shift.
- 2:08:59What needs you with the fix ready and
- 2:09:02what it handled on its own posted to the
- 2:09:04team chat.
- 2:09:05Not only in the morning, but every
- 2:09:07urgent issue that needs user attention
- 2:09:10gets notified in chat with findings and
- 2:09:12draft fixes.
- 2:09:14It also tags the next on call in the
- 2:09:15chat during handoff. The entire cycle
- 2:09:18still continues.
- 2:09:23And because it belongs to the team, it
- 2:09:25runs the agents and the skills the team
- 2:09:28already trusts.
- 2:09:29That's the slot in the middle.
- 2:09:31Bring your own agent, add your own
- 2:09:33skills.
- 2:09:35If a team already has an agent that has
- 2:09:37the team's runbooks and the knowledge
- 2:09:38base, the automations leverage it.
- 2:09:41Alongside that, it has our own incident
- 2:09:43investigator and a deep dive coding
- 2:09:46session.
- 2:09:48Work comes in from alerts, tickets,
- 2:09:51incidents,
- 2:09:52team posts and chat.
- 2:09:54The agent itself is generic. What makes
- 2:09:57it good at an on call rotation is the
- 2:09:59team's agents, skills, memory and
- 2:10:02playbook.
- 2:10:04And every run is logged, so anyone on
- 2:10:06the team can inspect the trajectory or
- 2:10:08resume the conversation.
- 2:10:11Now I'm handing off to my teammate Vlad
- 2:10:13to talk more on the shared agent.
- 2:10:19>> An on call agent can belong to just a
- 2:10:21single person because the on call itself
- 2:10:23doesn't.
- 2:10:24So it belongs to the team.
- 2:10:26There's one teammate per rotation on one
- 2:10:29board
- 2:10:30where every on call can ask it to
- 2:10:31correct it or pick up its work mid
- 2:10:33investigation.
- 2:10:34It carries the team's on call expertise,
- 2:10:37the playbook, alert profiles and past
- 2:10:39decisions from one shift to the next
- 2:10:41one, the the a good handoff should.
- 2:10:44Here's what that looks like.
- 2:10:46Monday at 2:00 a.m. same as on call and
- 2:10:48tells it. Remember, if checkout times
- 2:10:51out after a config push, roll back the
- 2:10:53config first.
- 2:10:55It saves it to the memory.
- 2:10:573 days later, Jordan is on call hits the
- 2:11:00exact same alert and just asks for a
- 2:11:03fix.
- 2:11:04The agent answers with same fix,
- 2:11:06says where it came from and has the
- 2:11:08rollback drafted.
- 2:11:10Teach it once, the next on call gets it.
- 2:11:14And it acts as the team.
- 2:11:16It's on a data day owned by the rotation
- 2:11:18with only the access the team is
- 2:11:20comfortable granting it.
- 2:11:23And it learns from every on call shift.
- 2:11:26Handoffs, decisions and corrections all
- 2:11:29flow into memory. So shift number 30 is
- 2:11:32better than the shift number one.
- 2:11:34The same way a new on call gets better
- 2:11:35every week.
- 2:11:37But memory erodes over time, so ours has
- 2:11:40to earn its place.
- 2:11:42It retrieves semantically, uses what it
- 2:11:45finds, scores what it actually helped
- 2:11:47and promotes
- 2:11:49or retires entries automatically.
- 2:11:52When someone says that was wrong,
- 2:11:54the item reopens and the correction
- 2:11:56becomes memory.
- 2:11:59Three things made this work be on our
- 2:12:01team.
- 2:12:02First,
- 2:12:03a loop that never stops listening. Every
- 2:12:05piece of feedback or course correction
- 2:12:08reopens the item and ultimately gets
- 2:12:10absorbed into memory.
- 2:12:12Feedback is an input to the system, not
- 2:12:14a sorry read once a quarter.
- 2:12:17Second,
- 2:12:18customize instead of forking.
- 2:12:21Teams bring their own playbooks and
- 2:12:23scale workflows
- 2:12:25and it's still the same teammate
- 2:12:26underneath.
- 2:12:28Third, what we call one rotation to end.
- 2:12:32Identity, config and memory are per
- 2:12:34team, so at a rotation becomes a setup
- 2:12:37not a new project.
- 2:12:39The agent is generic while the expertise
- 2:12:41is owned by the team.
- 2:12:45So what has changed for the people on
- 2:12:47call?
- 2:12:48Well, first, their job shifted from
- 2:12:50driving deep investigations to making
- 2:12:52key decisions.
- 2:12:53They start from a hypothesis with linked
- 2:12:55evidence, not a blank task or a
- 2:12:57dashboard.
- 2:12:59Second, the load is getting lighter.
- 2:13:02Early signs of on-call load reduction
- 2:13:04are promising for teams that run it
- 2:13:06automatic autonomously, and we want to
- 2:13:08measure it rigorously before we put a
- 2:13:10number on it.
- 2:13:12And the third, every change that agent
- 2:13:14writes
- 2:13:15still meets a human,
- 2:13:17fixes land only through normal review
- 2:13:20and continuous integration.
- 2:13:21And most drafts don't land.
- 2:13:24That's the point. Review is the filter.
- 2:13:27On-call stops being attacks and becomes
- 2:13:29a review queue.
- 2:13:32Here's what we're focusing on next.
- 2:13:35First, mitigations that get approved,
- 2:13:37not just drafted.
- 2:13:38We grade every mitigation that agent
- 2:13:40proposes and feed the outcome back
- 2:13:42so it earns its next level instead of
- 2:13:44being handed it.
- 2:13:46Second,
- 2:13:47skills are written from the chat so a
- 2:13:49team can turn its runbooks into skills
- 2:13:51without writing any code.
- 2:13:53And third, measuring the time we give
- 2:13:55back properly with a rigorous study of
- 2:13:58on-call time saved rather than usage
- 2:14:00counts.
- 2:14:01And the next rung on the ladder is
- 2:14:04earning medium criticality autonomy with
- 2:14:07evidence.
- 2:14:08On that, thank you so much for listening
- 2:14:10to us and hopefully that was helpful.
- 2:14:13Bye.
- 2:15:42>> This
- 2:19:47>> Yeah.
- 2:21:16>> harness in the less code we need. And if
- 2:21:19we look at what we have been doing at
- 2:21:21Google DeepMind for the last six months.
- 2:21:23So at May we had Google I/O
- 2:21:25I/O and today it's November 11 and we
- 2:21:28have been shipping relentlessly
- 2:21:30basically. Just earlier today we
- 2:21:32announced our first decision model with
- 2:21:35Gemini decision one. Yesterday we made
- 2:21:38Gemini four carbon available and then if
- 2:21:40we just look at the last six months we
- 2:21:41basically ship now models every few
- 2:21:43weeks.
- 2:21:44As already said Gemini carbon is now
- 2:21:46available for you. So if you are a
- 2:21:48Google AI Ultra subscriber you can use
- 2:21:50it in Anti-Gravity, you can use it in
- 2:21:52the Gemini app, you can use it in AI
- 2:21:53Studio.
- 2:21:54Carbon is number one on the cursor bench
- 2:21:57eval and also number one on the
- 2:21:59perplexity benchmark and internally at
- 2:22:01Google we have been using it already a
- 2:22:03lot.
- 2:22:04And much more exciting is that I'm happy
- 2:22:07to share that we launched the Gemini
- 2:22:08Decisions API roughly one hour ago with
- 2:22:11our new Gemini decision one model. So
- 2:22:14it's a similar model to chef. You can
- 2:22:16provide all of the existing Gemini
- 2:22:18inputs meaning it's multimodal so you
- 2:22:21can provide text, audio, images, video
- 2:22:24and also PDFs natively and then provide
- 2:22:26up to 1,000 questions in a single
- 2:22:28request. The average end-to-end latency
- 2:22:31including all of the network overhead is
- 2:22:33between 150 to 250 milliseconds and it
- 2:22:36costs 10 cents per 1 million tokens. So
- 2:22:40how do models get good at agentic work?
- 2:22:42And that's a rough
- 2:22:44overview on how post training works. So
- 2:22:46we have a task, we have our model, the
- 2:22:49model has a certain set of tools and
- 2:22:51then we have some kind of compute
- 2:22:52environment and at the end we have some
- 2:22:54verifier and the model tries to
- 2:22:55basically solve that task to use
- 2:22:57different tools and then based on the
- 2:22:59score from the verifier we update the
- 2:23:01model but no one is telling the model
- 2:23:03which tools to use and basically it only
- 2:23:06receives that reward feedback.
- 2:23:09So if we look at two examples on how a
- 2:23:11model can do a certain tasks with tools
- 2:23:14we have the path A where we have
- 2:23:16dedicated tools for file system control
- 2:23:18so you all have probably used coding
- 2:23:21agents and you have seen a read file
- 2:23:23call or an edit file call so that's one
- 2:23:26way of doing it and the other way of
- 2:23:28doing it would be just using a single
- 2:23:30inline bash script for all of the
- 2:23:31changes so on the left side we roughly
- 2:23:34need around 10 turns to make the same
- 2:23:36changes to a coding
- 2:23:38repository compared to the two change
- 2:23:40turns on the right side just using bash
- 2:23:43and it also reflects in the amount of
- 2:23:45tokens we need so the left path takes
- 2:23:48around 10x more tokens than the life the
- 2:23:51right path and it's doing the exact same
- 2:23:53thing
- 2:23:54at the end for the model to achieve that
- 2:23:56task. So in my experiment every roughly
- 2:24:00six months I try to build a new coding
- 2:24:02agent harness from scratch and I always
- 2:24:04ask myself okay how much of the harness
- 2:24:06do I really need to achieve the same
- 2:24:08task and what is overhead to the model
- 2:24:11and we no longer need. So if we look at
- 2:24:13the coding agent harness which
- 2:24:17from 2025 we had a roughly around 10 to
- 2:24:2015 to 20 tools we had dedicated tools
- 2:24:23for
- 2:24:24file editing we had dedicated tools for
- 2:24:26planning for to-do lists and also we had
- 2:24:29a lot of dedicated rules in our system
- 2:24:32prompt which told the model what to do
- 2:24:34and what not to do and that worked of
- 2:24:37course but since then we made a lot of
- 2:24:38changes so in my experiments
- 2:24:41first one was done roughly now two years
- 2:24:43ago in February
- 2:24:46where we did not really had agents right
- 2:24:47we had workflows we basically had
- 2:24:49predefined steps rock applications back
- 2:24:51then were very popular we had the
- 2:24:53planner we had search we reflected on
- 2:24:54whatever was found and then we generated
- 2:24:56an answer. Then we started to see our
- 2:24:58first agents we had many many different
- 2:25:01tools for all sorts of fine kernel
- 2:25:02actions.
- 2:25:04And then came the coding agents where we
- 2:25:06had those roughly ten to 20 tools.
- 2:25:09And now in my last experiment I tried to
- 2:25:11see, okay, what's going to happen if I
- 2:25:13provide only one tool to my agent?
- 2:25:16And the outcome is was pretty exciting
- 2:25:18to me. So this one tool agent works the
- 2:25:21same way as every other agent except
- 2:25:23that it only has one tool, which is
- 2:25:25Bash.
- 2:25:26And the model can use that tool for any
- 2:25:28sorts of task. And the Bash tool is very
- 2:25:31simple, so it runs in a separate clean
- 2:25:33subshell. And it supports the command
- 2:25:36argument. I also added a description so
- 2:25:38that it's nice and easier to display to
- 2:25:40the user what the model is actually
- 2:25:42doing. And then like some async
- 2:25:43parameters. So basically when the model
- 2:25:44runs a Bash command,
- 2:25:47the parameter tells the model, okay, if
- 2:25:49it is the command longer running than
- 2:25:51this milliseconds, it's being
- 2:25:54that the tool call resolves. And the
- 2:25:56model receives a local file handle where
- 2:25:59it can look up later, okay, if the
- 2:26:00process is still running. That's very
- 2:26:02interesting if you do like all sorts of
- 2:26:04like web development, for example. And
- 2:26:07the single Bash tool actually works for
- 2:26:09all of the things we are using agent. So
- 2:26:10we can have long running work where we
- 2:26:12can start on the foreground and then
- 2:26:14automatically move it to the background.
- 2:26:16The model receives some file handle and
- 2:26:19since it has Bash available, if we store
- 2:26:21it to a file system, it can read the
- 2:26:23logs which are being streamed. If the
- 2:26:26agent itself is a CLI, it can call
- 2:26:28itself using Bash. So there is
- 2:26:30practically no need to create sub agent
- 2:26:33tools for it because it can invoke
- 2:26:35itself. And then of course we don't need
- 2:26:37dedicated planning or memory tools
- 2:26:39because we have our file system
- 2:26:40available. We can provide some context
- 2:26:43to the model so the model can keep track
- 2:26:44of its own work with the file system and
- 2:26:47using the Bash tools.
- 2:26:48And in addition, of course, to our
- 2:26:50tools, we have system prompts. And
- 2:26:52similar to tools, system prompts evolved
- 2:26:55over time.
- 2:26:57Every dot here is roughly around 100
- 2:26:59tokens and you can see that coding
- 2:27:01agents mid 2025 had
- 2:27:04a lot of
- 2:27:05context needed to send in every request.
- 2:27:08Of course this also improved and similar
- 2:27:10story here is like okay, what do I
- 2:27:12really need to provide to my agent to
- 2:27:14make it successful and work and what of
- 2:27:17all of the system information and
- 2:27:19details we had in the past are now
- 2:27:20basically
- 2:27:22part of the model.
- 2:27:23So
- 2:27:24how much of the harness helps and how
- 2:27:26much is in their way?
- 2:27:28And to find that out I was running a lot
- 2:27:30of different evaluations of a lot of
- 2:27:32different agent harnesses with a lot of
- 2:27:34different tools. And one example I like
- 2:27:36to show here which was done with Gemini
- 2:27:384 Carbon from an eval task from DeepSVE
- 2:27:41is like okay, what happens if I provide
- 2:27:44my existing coding agent tools with
- 2:27:47read, write, search, grep
- 2:27:49and my bash only tool? And I think you
- 2:27:51can pretty clearly see that the top line
- 2:27:54is taking a lot of more tokens, a lot of
- 2:27:57more turns and a lot of more times to
- 2:27:59achieve the same task. And what stood
- 2:28:02out really well is that okay, reading is
- 2:28:05done in parallel in the beginning, but
- 2:28:07as soon as we start going into making
- 2:28:09code changes we see that same behavior
- 2:28:11that the model is making turn by turn a
- 2:28:15single edit which of course consumes a
- 2:28:17lot of time and a lot of tokens.
- 2:28:20So
- 2:28:21how that does this look at Prodoscope?
- 2:28:24So I have my own eval set of around 350
- 2:28:27tasks for coding, knowledge work and
- 2:28:30terminal commands
- 2:28:32all run with Gemini 4 Argon here and
- 2:28:35four different agent harnesses. One is
- 2:28:37the bash only harness and then okay,
- 2:28:40with bash we still have that issue on
- 2:28:43how can I read images, right? If I'm I
- 2:28:45cannot cut screenshot.png it would
- 2:28:48return a binary and it will fill my
- 2:28:50context and the model cannot see the
- 2:28:52image. So I added a view media tool
- 2:28:54which basically returns the image in a
- 2:28:56way the model can see.
- 2:28:58Then we have
- 2:28:59two different agent harnesses with
- 2:29:01structure tools for file system
- 2:29:03operations. And if we just look at the
- 2:29:06score
- 2:29:07all of them look roughly identical,
- 2:29:10right? So if we would just evaluate our
- 2:29:12model in an agent harness looking at the
- 2:29:15score, it would
- 2:29:18basically not really matter which one of
- 2:29:19those we would pick. Of course, there's
- 2:29:21a slight variance in the scores, but I
- 2:29:23only ran
- 2:29:25each task one time, so might be just
- 2:29:27some noise or maybe something else. So
- 2:29:29you would probably need to run a few
- 2:29:31more samples per harness, but the score
- 2:29:34looks roughly the same. So how would we
- 2:29:36now know which harness should I choose?
- 2:29:40If we just look all at all of the evals
- 2:29:43being shown, it's always mostly about
- 2:29:45score. But if we look at the token
- 2:29:47column or even more at the dollar per
- 2:29:49task,
- 2:29:50there's a picture which is changing. So
- 2:29:52the token per task is getting lower.
- 2:29:55Interestingly, if we just use bash, and
- 2:29:58then also the cache hit rate increases.
- 2:30:00So if we just compare like the bash
- 2:30:02media as the the baseline, the tasks are
- 2:30:05almost 2x cheaper compared to
- 2:30:08more complex, more tools agent
- 2:30:11harnesses.
- 2:30:12And we would not see it if we just look
- 2:30:15at the score.
- 2:30:16And to show you a few examples, I
- 2:30:19prepared some demo. So here I'm using
- 2:30:22the pie harness, which makes it super
- 2:30:24simple to basically define what tools
- 2:30:26you want to use.
- 2:30:28I have the the same prompt for 3.5
- 2:30:31flash, which we released roughly 6
- 2:30:33months ago
- 2:30:34for all tools, so like whatever pie
- 2:30:36ships out of the box. Then we have it
- 2:30:38for bash only, and then we have it for
- 2:30:41carbon
- 2:30:42and sorry, for carbon with the default
- 2:30:45tools and then with with bash only.
- 2:30:48This runs now since we only have roughly
- 2:30:51around 20 minutes. I recorded all of
- 2:30:54that and clicked that very nice
- 2:30:56accelerate button here. And what we can
- 2:30:57see on the right side is that
- 2:31:00the same paradigm basically applies
- 2:31:03when models get better, they roughly
- 2:31:06need less tools. So with 3.5 flash, we
- 2:31:09see with the default tools compared to
- 2:31:11bash only, we had the same amount of
- 2:31:13model calls, we had the same amount of
- 2:31:15tool calls, and we had roughly the same
- 2:31:17amount of tokens. And if we look at the
- 2:31:19or the traces, we can see for bash only,
- 2:31:22we have some cut commands to read
- 2:31:24different files. We have some inline
- 2:31:26scripts to make changes and then like
- 2:31:28some git changes.
- 2:31:29If we look at the default tools, we can
- 2:31:32see roughly the same. We had
- 2:31:34read file commands, we used the edit
- 2:31:36tool, and then had some git commands to
- 2:31:38make those changes. If we now look at
- 2:31:40Carbon with the default tools, it
- 2:31:42already used way less
- 2:31:45tool calls and model calls. So it it's
- 2:31:48basically doing work in parallel, but
- 2:31:51also trying to reduce it. So we had for
- 2:31:52example the still same for write read
- 2:31:56calls and also the edit calls. But what
- 2:31:58starts to get interesting if we are
- 2:32:00going to look into bash. And in bash, we
- 2:32:03had even less tool calls and model
- 2:32:05calls. And the interesting part here is
- 2:32:08that the model writes a single Python
- 2:32:10script to make all of that changes in
- 2:32:12one go and no longer needs to call
- 2:32:14multiple tools.
- 2:32:16So if we go
- 2:32:18further, what does it means in terms for
- 2:32:20for Carbon here and how bash pays off is
- 2:32:22that
- 2:32:23the better the model gets or the more
- 2:32:26reasoning and like capability it gets,
- 2:32:28bash helps the model to be able to chain
- 2:32:31more commands in the single call. For
- 2:32:33Carbon, it was roughly 2.2 to 3.7
- 2:32:38commands per bash call. And for flash,
- 2:32:40it was only 1.3 to 1.8.
- 2:32:43So we are seeing up to 3x less tokens
- 2:32:48by Carbon to achieve the same tasks
- 2:32:51compared to having the same tools we
- 2:32:53need to use.
- 2:32:55So
- 2:32:56two more examples to look at it as well
- 2:32:58is like we have on the left side Carbon
- 2:33:01with the default tools. So we see our
- 2:33:03coding agent tools with write, with
- 2:33:05edit, and we see okay, it does some
- 2:33:08parallel work in turn two, some parallel
- 2:33:10work in turn three and four. And then we
- 2:33:13have all of the changes. And on the
- 2:33:14right side, where just has the bash
- 2:33:16tool, it uses Python to write an inline
- 2:33:18script to make all of that changes at
- 2:33:21the same time with much less token. And
- 2:33:24the same counts for if we compare Gemini
- 2:33:273.5 Flash to Gemini 4 Carbon Flash with
- 2:33:31bash only is not chaining those
- 2:33:33commands. So here it is not really it
- 2:33:35doesn't matter if I'm having a a retool,
- 2:33:38a write tool, an edit tool and a bash
- 2:33:40tool, or if I have just a bash tool
- 2:33:41because 3.5 Flash used to use bash for
- 2:33:45single turn commands. So it had a single
- 2:33:48turn for reading, it had a single turn
- 2:33:50for writing. But if we look at Carbon,
- 2:33:52which in here in this example is the
- 2:33:54exact same task, it needs less way
- 2:33:56turns. So the reading is not needed to
- 2:33:59read all of the different files and
- 2:34:01completely. It just uses rip grep and
- 2:34:03like some intelligence searching to find
- 2:34:05where we need to make changes. And then
- 2:34:06it writes a single Python inline bash
- 2:34:09command to make all of those edits in
- 2:34:11one go, saving us a lot of tokens. So
- 2:34:15what does it bring us? Models will keep
- 2:34:17eating the harness. And I'm
- 2:34:20pretty sure that 2027 will also be a lot
- 2:34:22more about how many tokens do we need to
- 2:34:24solve a task. Even for the few
- 2:34:27improvements we made between Argon and
- 2:34:29Carbon, is the the better model needs
- 2:34:31less tokens per task, which of course is
- 2:34:33very exciting for all of us because then
- 2:34:35we can use it more. And also it matters
- 2:34:38a lot what the harness provides. And we
- 2:34:40will might not see it if we just look at
- 2:34:43the accuracy. It's also very important
- 2:34:45that we start measuring the token
- 2:34:47efficiency because
- 2:34:49harnesses can look the same on accuracy
- 2:34:50because our models are now very good. So
- 2:34:53if your harness
- 2:34:54is more complex to use, the model will
- 2:34:56work around it, will find solutions
- 2:34:58around it, but needs more turns and more
- 2:35:00tokens to achieve the same results.
- 2:35:03And then also what comes afterwards. So
- 2:35:05I will try to do the same experiment six
- 2:35:08months from now. And currently the trend
- 2:35:10we see slowly
- 2:35:13rising at least on like social media and
- 2:35:16like the industry is that we might go
- 2:35:18back to some code mode tool that we
- 2:35:21instead of writing a bash commands that
- 2:35:23we write TypeScript or Python directly
- 2:35:26that MCP will come back. I'm a big fan
- 2:35:29of MCP and like they are making great
- 2:35:31changes to make it more
- 2:35:33easily easily usable, especially
- 2:35:37interesting for all of that code mode
- 2:35:39work is MCP servers now allow you to
- 2:35:41return structured outputs so the model
- 2:35:43can write better scripts to take more
- 2:35:45actions in the single turn and it also
- 2:35:47allows us to do some better proxy around
- 2:35:50the security because bash calls are very
- 2:35:52hard to understand, but if I have some
- 2:35:55obstructive layer in TypeScript I can
- 2:35:58watch all of the commands being
- 2:36:00executed and make sure those are secure
- 2:36:03and for you
- 2:36:05build
- 2:36:06to delete your code before the model
- 2:36:08does.
- 2:36:08Give your model to a computer, not a
- 2:36:11toolbox
- 2:36:12and when a new model ships I definitely
- 2:36:14recommend you trying to think a little
- 2:36:17bit outside of the box, be more
- 2:36:19ambitious, try to really see, okay, what
- 2:36:21if I remove my complete harness and just
- 2:36:23give it one tool? And then also please
- 2:36:25scan the QR code. You will get free
- 2:36:28credits. We will send them out after the
- 2:36:31AI Engineering event to try out Carbon
- 2:36:33and also to try out the new Gemini
- 2:36:35Decisions API. So if you have some time
- 2:36:37today I really can recommend checking
- 2:36:39out the new Decisions model, Carbon and
- 2:36:42thank you for your time.
- 2:36:45>> [applause]
- 2:37:15>> <music>
- 2:37:15>> </music>
- 2:40:13>> This
- 2:42:18>> Yeah.
- 2:44:18>> This
- 2:46:35>> All right.
- 2:46:36Well,
- 2:46:37super excited to be here. My name's
- 2:46:39Ankit. I'm from Databricks. I work on AI
- 2:46:41infrastructure there, and today I'm
- 2:46:43going to talk about [music] the rise of
- 2:46:45the agent workforce and how we're
- 2:46:47building our agent infrastructure to
- 2:46:49support what we think is the next
- 2:46:50generation of agents.
- 2:46:52I wanted to start by taking a step back
- 2:46:54and and thinking about how we helped
- 2:46:57computers become good in other things,
- 2:47:00better even than humans. And one of the
- 2:47:02games I grew up playing a lot was chess.
- 2:47:04It's one of the great examples of
- 2:47:06something where computers became way
- 2:47:07better than humans a long time ago, but
- 2:47:10I think it's important to think about
- 2:47:12why that happened and how we can apply
- 2:47:14those lessons today.
- 2:47:15One of the biggest, you know, to
- 2:47:17summarize at a high level, a lot of work
- 2:47:18went into it.
- 2:47:20The biggest thing that agents were able
- 2:47:21to do, that computers were able to do,
- 2:47:23was explore the space of decisions much
- 2:47:26more effectively than humans.
- 2:47:28For most complex problems, there's
- 2:47:30actually a bunch of different ways you
- 2:47:31can solve the problem. And at each turn,
- 2:47:34you can kind of
- 2:47:35think of three more decisions that you
- 2:47:37can make. And really it's the
- 2:47:38exploration of that space that all of us
- 2:47:40are engaged in. All of us who solve hard
- 2:47:42problems are engaged in the search space
- 2:47:44for what's an amazing product, what's
- 2:47:46the right solution to this math problem,
- 2:47:48and etc., right?
- 2:47:49And I think we've seen some really
- 2:47:51exciting progress from the labs like
- 2:47:53Anthropic, OpenAI, and Google
- 2:47:56on solving problems that are well
- 2:47:57formed. So, you know, if you think about
- 2:47:59chess, it can be modeled effectively.
- 2:48:01You can evaluate your position.
- 2:48:04You can actually play simulate it. You
- 2:48:05can try different moves and safely think
- 2:48:08about all the different alternatives.
- 2:48:10Similarly with math problems, often the
- 2:48:12ones that agents are really good at
- 2:48:13solving,
- 2:48:14they you can actually verify if you got
- 2:48:16the result right.
- 2:48:18And how do we then take that mindset and
- 2:48:21apply it to the problems that we
- 2:48:22actually face day to day?
- 2:48:25I think like let's a lot everyone here
- 2:48:27is an engineer, so let's take it to
- 2:48:29engineering for a bit.
- 2:48:31I think, you know, if you take it to the
- 2:48:32pre-AI days,
- 2:48:34most people think about solving a single
- 2:48:36problem and they're kind of single
- 2:48:38threaded on that problem for the time
- 2:48:39that they're writing code on it.
- 2:48:41And you know, as a company if or a
- 2:48:43product if you wanted to solve more
- 2:48:44problems, you brought in more people.
- 2:48:46And the very basic version of agents
- 2:48:49just helps create leverage for that
- 2:48:51engineer. Now you can be single threaded
- 2:48:53on three things at the same time and
- 2:48:55ultimately that helps you explore new
- 2:48:56product ideas, ship faster, iterate
- 2:48:59faster, and that's obviously very
- 2:49:00exciting.
- 2:49:02But it's it's not actually close to what
- 2:49:03we needed to do to win a chess or any of
- 2:49:06these other problems. What we're seeing
- 2:49:08at the absolute frontier are these teams
- 2:49:11of agents that are able to make
- 2:49:12significant progress by exploring the
- 2:49:14decision space more effectively.
- 2:49:16So how do we move towards that world
- 2:49:18where we actually have these teams of
- 2:49:19agents that are coherently attempting to
- 2:49:21solve problems? It's actually quite
- 2:49:23complex and we think it requires a
- 2:49:25little bit different type of
- 2:49:26infrastructure.
- 2:49:28Specifically, I think an important
- 2:49:30question to ask is how do we give agents
- 2:49:32a realistic world to explore.
- 2:49:35Because in other problems we're able to
- 2:49:37help define that world very effectively,
- 2:49:39but even in software engineering we
- 2:49:41actually really struggle to do that.
- 2:49:43Like just consider this movement towards
- 2:49:45software factories, right? Where you're
- 2:49:47away from the from the agents and you're
- 2:49:49actually operating a bunch of agents
- 2:49:50that are working on a problem.
- 2:49:52How do you validate that those agents
- 2:49:54are effective?
- 2:49:56Like before you had a branch, uh you had
- 2:49:58main, you branched your PR off of that.
- 2:50:00Maybe you have an agent working on that
- 2:50:02PR and you have a some reasonable
- 2:50:05assertion that this is kind of a you
- 2:50:07know you have some like some tests or
- 2:50:09something like that that help you guide
- 2:50:11your development.
- 2:50:12But if you're actually merging tens of
- 2:50:15PRs a minute or even more, right? Like
- 2:50:17if you play out the future of factories
- 2:50:20of agents building code and software,
- 2:50:23your PR is just out of date the moment
- 2:50:24you branched. The the moment you started
- 2:50:27you're already out of date and so it
- 2:50:29demands a different mindset of thinking
- 2:50:31of how do we verify these things. You
- 2:50:32actually might want to move most of your
- 2:50:34verification to post merge.
- 2:50:36And so that kind of brings me to this a
- 2:50:38little bit radical idea. I I don't think
- 2:50:41we should try to write hundreds or
- 2:50:43thousands of unit tests.
- 2:50:45Instead I want to explore this idea of
- 2:50:47simulating reality.
- 2:50:49Can I give my agent a faithful
- 2:50:51reproduction of what's actually there in
- 2:50:53the real world in my production code, in
- 2:50:55my production agent and let it work on
- 2:50:58that.
- 2:50:59And with you when you can do that you
- 2:51:01can actually do that at scale. If you
- 2:51:03find a way to simulate reality then you
- 2:51:05can actually do that across lots of
- 2:51:07different explorations.
- 2:51:09At DataBricks one of the big things that
- 2:51:10people do on our platform is build data
- 2:51:12pipelines. We've got tens of thousands
- 2:51:14of data pipelines on DataBricks.
- 2:51:17Uh and the biggest problem that happens
- 2:51:19with pipelines is that they break. Um
- 2:51:21and in this era of agents what people
- 2:51:24want is they want to fix those broken
- 2:51:25pipelines with agents
- 2:51:28because these are serious business
- 2:51:29problems that they that are you know
- 2:51:31caused by a broken pipeline.
- 2:51:33If a pipeline breaks that business can
- 2:51:35operate that day. And so, this becomes a
- 2:51:37major emergency.
- 2:51:40But like, how do you actually validate
- 2:51:41this with an agent? Do you trust it on
- 2:51:43your production data pipeline?
- 2:51:45It's pretty risky because writes might
- 2:51:47not be idempotent. Even reads might be
- 2:51:50extremely expensive and affect other use
- 2:51:52cases in production, right? Sometimes
- 2:51:53it's not fully broken, it's just
- 2:51:54partially broken.
- 2:51:56One of the things we started doing was
- 2:51:58building on top of branchable
- 2:51:59infrastructure.
- 2:52:01And so what we did was we branched the
- 2:52:02entire data platform.
- 2:52:04We branched the tables, the compute, the
- 2:52:07governance, and we gave the agents a
- 2:52:10place where they can actually play with
- 2:52:12the data platform and explore different
- 2:52:14hypotheses.
- 2:52:15And these hypotheses can be ambitious.
- 2:52:17Like, can I investigate the upstream
- 2:52:19pipeline is healthy? Maybe I can do that
- 2:52:21by changing the pipeline code and
- 2:52:23running it again.
- 2:52:24In any reasonable data platform, you
- 2:52:27wouldn't do that with an upstream
- 2:52:28pipeline unless you had a bunch of
- 2:52:29engineers take a look at it, make sure
- 2:52:30it was super safe.
- 2:52:32With our branched data platform, you can
- 2:52:33actually do that now. So it can
- 2:52:35investigate a bunch of different things.
- 2:52:37So we can investigate 10 ideas in
- 2:52:38parallel with our agents,
- 2:52:41actually get to a faithful conclusion
- 2:52:43about whether it fixed the real data
- 2:52:44pipeline
- 2:52:45and then make that assumption, make that
- 2:52:47assertion back either to a main agent or
- 2:52:49to a human.
- 2:52:50And what we saw before was happening was
- 2:52:52that people would just like
- 2:52:53LLMs would give us texts back. Some
- 2:52:55human would take a look at it and be
- 2:52:57like, okay, this might be an interesting
- 2:52:58theory. Then they would go apply that
- 2:53:00theory in some way. And ultimately, that
- 2:53:02became more of this like AI-assisted
- 2:53:05version of fixing the data pipeline.
- 2:53:07But really where we want to go is is is
- 2:53:09hands off, full full auto
- 2:53:12for this kind of problem.
- 2:53:14And we were we were struggling to do it
- 2:53:15without this branchable infrastructure.
- 2:53:18So how do we do it? I'm going to talk a
- 2:53:19little bit about that in this talk.
- 2:53:21Uh we built this on top of our Postgres
- 2:53:23native agent platform, which started
- 2:53:26with the Neon database, which is a
- 2:53:27branchable database that gives us a
- 2:53:29single place that we can actually use
- 2:53:31for our services and for our agents.
- 2:53:33And then we added in a bunch of really
- 2:53:36core agent primitives that allow us to
- 2:53:37build sophisticated different types of
- 2:53:39agents.
- 2:53:40Uh and those are the ones that are
- 2:53:41probably familiar to all of you. Things
- 2:53:42like memory, things like sandboxes, file
- 2:53:45systems, and ways to host those agents.
- 2:53:47And these low-level primitives we can we
- 2:53:49can build with them, but the key is that
- 2:53:51we made just all of them branchable. So
- 2:53:53you can branch the entire reality of
- 2:53:55stuff that's built on this platform for
- 2:53:57us for ourselves.
- 2:53:58Uh so how do we actually do that? Well,
- 2:54:01let me walk it through like kind of
- 2:54:03case by case.
- 2:54:04The first thing is just the database,
- 2:54:06right? And you know, we can take an
- 2:54:07example of application services, but
- 2:54:09here I'm going to talk about just even
- 2:54:10storing the conversation state for for
- 2:54:12an agent.
- 2:54:13When you fork or branch an agent, you
- 2:54:15actually might want to inherit the
- 2:54:17history and you might even if you have a
- 2:54:19team of agents give access to a given
- 2:54:21member of that team to what the other
- 2:54:23members of the team are doing.
- 2:54:24Uh and we found the database to be a
- 2:54:25convenient abstraction for this.
- 2:54:28Uh the key was that hey, like when a new
- 2:54:30reply comes in on the on the branched
- 2:54:32agent, it shouldn't necessarily change
- 2:54:34anything about the other one.
- 2:54:35Uh and this is possible because Neon has
- 2:54:38a branched database.
- 2:54:40But then we actually found that agents
- 2:54:41need more than just structured
- 2:54:43databases. Of course, they need
- 2:54:44structured databases, but they also like
- 2:54:46to do this, you know, write these things
- 2:54:48called memories.
- 2:54:49And memories are, you know, at a very
- 2:54:51high level they compile down to an
- 2:54:53unstructured database for agents.
- 2:54:55There are two ways that memories get
- 2:54:56generated. The first is just by
- 2:54:58synthesizing the context and the
- 2:55:00conversations that someone's already had
- 2:55:01with them. And the second by agents
- 2:55:04actually explicitly saving content to an
- 2:55:07unstructured place.
- 2:55:09The key thing here is that you actually
- 2:55:11want to consolidate and distill that
- 2:55:12information. There's a ton of
- 2:55:13information coming in through there.
- 2:55:15And if you just give that, you know, if
- 2:55:17you just let it kind of go on its own,
- 2:55:19you have problems of staleness, you have
- 2:55:22conflicts in the information, and all of
- 2:55:24those can be extremely damaging to
- 2:55:25teams.
- 2:55:26So we have this thing that basically
- 2:55:28dreams overnight. We use offline compute
- 2:55:32uh and it takes the conversations, a
- 2:55:34different agent essentially highlights
- 2:55:35things that are conflicts
- 2:55:37and consolidates the ad hoc notes that
- 2:55:40were written by various members of the
- 2:55:41team.
- 2:55:42And then all members of the team can
- 2:55:43actually recall from that memory
- 2:55:45database.
- 2:55:46And you can use this in a bunch of
- 2:55:48different ways. We use this within our
- 2:55:49Slack agents, but we also use this when
- 2:55:52we're launching a parallel set of agents
- 2:55:54to investigate a deep issue, for
- 2:55:56example, in the data platform.
- 2:55:58Now, how does branching come into this?
- 2:56:00Well, some of the agents, like in many
- 2:56:02of the use cases, they propose memories.
- 2:56:05And it's not clear whether those
- 2:56:06memories generalize to the actual
- 2:56:09broader set of agents or if that's a
- 2:56:10specific thing for just that agent.
- 2:56:12And so with the branchable database for
- 2:56:14memory, which is built on top of this
- 2:56:15neon database, we can actually just
- 2:56:17propose that memory somewhere in there,
- 2:56:19and that branch can continue to iterate,
- 2:56:22and at some point we can find a way to
- 2:56:23merge it back in if it's necessary.
- 2:56:26We found this to be an important layer
- 2:56:28of flexibility because without the
- 2:56:30ability to completely branch the memory,
- 2:56:32we found it was very hard to maintain
- 2:56:35the memory for a team of agents.
- 2:56:37Especially as we increase parallelism,
- 2:56:38you can actually experience like
- 2:56:40significantly reduced quality if all the
- 2:56:42agents are pursuing different goals.
- 2:56:45I I like to think a little bit of it as
- 2:56:47as like kind of similar to dynamic
- 2:56:48programming, right? If you've explored a
- 2:56:50branch of exploration
- 2:56:52and then you have a different agent that
- 2:56:53happens upon that same pathway, it's
- 2:56:55very damaging to like always revisit
- 2:56:58that, but sometimes you want to. And so
- 2:57:00the ability to propose a memory is
- 2:57:01extremely important to us.
- 2:57:04How is all of this kind of branchable
- 2:57:06database possible? Well, the way that
- 2:57:08neon does this is it separates compute
- 2:57:10from storage.
- 2:57:11Compute
- 2:57:12for Postgres is running in some separate
- 2:57:14place, and then we actually have the
- 2:57:16shared history and the storage in a
- 2:57:18different layer. And what that lets us
- 2:57:20do is copy on write for branching. So
- 2:57:23when we have a branched version of the
- 2:57:24database, you can actually have A's, you
- 2:57:26know, compute A's writes separated from
- 2:57:28compute B. And that's the kind of the
- 2:57:30core layer of infrastructure that allows
- 2:57:32us to build these applications on top of
- 2:57:34that database.
- 2:57:37But then we found, you know, obviously
- 2:57:39agents are far more than storage, and
- 2:57:41and branching storage wasn't enough.
- 2:57:43We also found that as they were running
- 2:57:45in sandboxes, agents like to take notes
- 2:57:47on stuff and write down scripts and use
- 2:57:50those to actually iterate on various
- 2:57:52kinds of problems. It It's one of the
- 2:57:54most exciting things about agents that
- 2:57:56I've found that they are able to
- 2:57:58generalize this capability that's needed
- 2:58:00maybe for testing code and and things
- 2:58:02like that and solve other more broader
- 2:58:04general problems using that same
- 2:58:06construct.
- 2:58:08But then when you branch it, you
- 2:58:10actually need to give the future agent a
- 2:58:11similar context. If you We found that
- 2:58:13when we started the branched agents
- 2:58:15without the same files that that agent
- 2:58:17had to begin with, it just wasn't as
- 2:58:19effective. So when you're investigating
- 2:58:21an outage, for example with an agent,
- 2:58:23you might have notes and plans and a
- 2:58:25future branch of that agent might update
- 2:58:27those notes. It might find new things.
- 2:58:30But we don't want that to go back cuz
- 2:58:31we're actually exploring 10 different
- 2:58:33pathways at the same time.
- 2:58:35So building a branchable file system was
- 2:58:37extremely important and allowed us to
- 2:58:39build what we call a branchable sandbox.
- 2:58:41Because then what we can do is we can
- 2:58:44launch that sandbox compute and and
- 2:58:45that's a hard problem in and of itself.
- 2:58:47It's not one we're going to talk about
- 2:58:48today.
- 2:58:49But that compute can be attached to
- 2:58:51storage.
- 2:58:52And critically,
- 2:58:54the agents actually really aggressively
- 2:58:55leverage files very often. And so it can
- 2:58:58be very painful to make all your storage
- 2:59:00fully remote.
- 2:59:01And so the really core piece of what
- 2:59:03we've been building here is a sinking
- 2:59:05process between what we call a local
- 2:59:07cache and the remote cache.
- 2:59:10And then what that allows us to do is as
- 2:59:12we're able to make progress on that,
- 2:59:14when we want a branch, we can actually
- 2:59:16branch a second sandbox, use that
- 2:59:18similar file system, and it reduces our
- 2:59:20latency for for branching while also
- 2:59:22keeping performance of the regular agent
- 2:59:25reasonable.
- 2:59:27So
- 2:59:28what this allows us to do at a high
- 2:59:29level is branch the entire working
- 2:59:31state.
- 2:59:33And this gives us the platform layer to
- 2:59:35actually go do that. Now this isn't
- 2:59:38obviously sufficient. You're going to
- 2:59:39want to branch even more. If you're
- 2:59:41building on top of software, you want to
- 2:59:42find a way to branch that too.
- 2:59:44And so this is along the lines of how
- 2:59:46we're thinking, right? Like we the data
- 2:59:47platform was built on top of some of
- 2:59:49these things, but there's a lot more
- 2:59:51that we've been able to branch there as
- 2:59:52well.
- 2:59:53Uh but we're really excited about this
- 2:59:55layer for just even building agents.
- 2:59:57And just to like kind of separate a
- 2:59:59little bit even from software
- 3:00:00development, like another thing a lot of
- 3:00:02us are spending a lot of time thinking
- 3:00:03about
- 3:00:04is security.
- 3:00:06Uh
- 3:00:07I mean just taking a step back, I think
- 3:00:09this is probably the single most
- 3:00:11important thing for any product that
- 3:00:12anyone sells at this point
- 3:00:14because trust is kind of the most
- 3:00:16important thing that we're selling when
- 3:00:17we work with other people.
- 3:00:19And uh with agents being the ones who
- 3:00:21generate the vulnerabilities,
- 3:00:24uh it's kind of
- 3:00:26impossible to find a different solution
- 3:00:27than having agents be the ones who are
- 3:00:29monitoring for those exact
- 3:00:31vulnerabilities in your code. So you
- 3:00:33should be thinking about how you can
- 3:00:34build an agent factory that's detecting
- 3:00:36security issues in your codebase.
- 3:00:38Um and so this is something we're
- 3:00:39spending a lot of time spending a lot of
- 3:00:41time on.
- 3:00:43So typically, what happens in a security
- 3:00:44incident? Well, let's say you suspect
- 3:00:47something. An agent might be always
- 3:00:48looking for it.
- 3:00:49Why why is there this like network
- 3:00:51traffic that's egressing all of a sudden
- 3:00:52from my control plane?
- 3:00:54Well, I can assemble an incident
- 3:00:56timeline and that's obviously the first
- 3:00:57step in an investigation. But anyone
- 3:00:59who's been a part of a serious security
- 3:01:01investigation knows that time is of the
- 3:01:03essence. It's a perfect problem for
- 3:01:05parallelism and that's exactly what you
- 3:01:07do when a security incident happens. You
- 3:01:08bring everyone online so you can explore
- 3:01:1110, 15, 20 different theories at once
- 3:01:14because an hour earlier that you can you
- 3:01:16can detect the real issue is an hour
- 3:01:18that you're actually more safe.
- 3:01:20So okay, what if we did that with a
- 3:01:22branchable agent?
- 3:01:24Let's say there's two paths to
- 3:01:25investigate.
- 3:01:26Uh so you know, the agents assemble the
- 3:01:28incident timeline and now you can ask
- 3:01:31two questions. Hey, did a recent
- 3:01:32deployment introduce some kind of new
- 3:01:34integration? Like maybe I can go look at
- 3:01:36the code.
- 3:01:37Or I can explore a totally different
- 3:01:39theory. Is a compromised identity
- 3:01:41accessing this data?
- 3:01:42And the core thing that we are able to
- 3:01:44do here is we branch that whole original
- 3:01:46agent that was starting the
- 3:01:47investigation.
- 3:01:50And then the new one is able to work in
- 3:01:51a completely isolated copy of the world.
- 3:01:54And in some cases, when we're able to,
- 3:01:55we can even branch the actual service
- 3:01:57that we're investigating.
- 3:01:59Uh and so when you have this isolated
- 3:02:00copy of conversation, memory, and
- 3:02:02storage, you can fully explore that
- 3:02:05security risk and and let the agent loop
- 3:02:07on its own. And that's kind of the key
- 3:02:08thing here.
- 3:02:09The way that these agents are good is
- 3:02:11that they're able to loop, verify, try
- 3:02:14new theories. I think Andrej Karpathy
- 3:02:16said it really well. They're far more
- 3:02:18tenacious than any human probably in the
- 3:02:22history of software.
- 3:02:24And so they're able to just But like we
- 3:02:26keep getting in the way by inserting
- 3:02:27ourselves as an approval layer.
- 3:02:30Uh and so this is a solution that we
- 3:02:32think helps with that. Just like take
- 3:02:33away the consequences of making a
- 3:02:35mistake a little bit. Branch the entire
- 3:02:37state so that you're not actually,
- 3:02:40you know, worrying about that same risk.
- 3:02:43And then you we you know, we've been
- 3:02:44able to find this to actually uh come to
- 3:02:47a reasonable conclusion as well.
- 3:02:49So what is this all add up to?
- 3:02:52For us, we think that your
- 3:02:53infrastructure needs to be branchable.
- 3:02:55And that's what we're building with
- 3:02:57Neon, with Databricks. Uh and we're
- 3:02:59super excited about that branchable
- 3:03:00future because we think that teams are
- 3:03:02going to be the next generation of
- 3:03:04agents, both for software and the way
- 3:03:06that we actually build products.
- 3:03:08Thank you.
- 3:03:11>> [applause]
- 3:04:46>> Oh.
- 3:05:51>> [music]
- 3:08:42>> I
- 3:10:06>> Okay.
- 3:10:08All right. Can you all hear me?
- 3:10:12All right. My name is Andrew and
- 3:10:15I think we're starting now. I don't know
- 3:10:17if there's a timer anywhere.
- 3:10:19Uh my name is Andrew and I'm excited to
- 3:10:21talk to you today about the safe
- 3:10:24software factory.
- 3:10:27Uh but first a little bit about me. Uh
- 3:10:30I am a staff software engineer at Sonar.
- 3:10:33I work on Guitar, which is Sonar's AI
- 3:10:36code review product.
- 3:10:38Uh and previously I was at uh Uber
- 3:10:41working on the driver trip experience
- 3:10:44and navigation. Uh and then after that I
- 3:10:47was at a health tech startup called
- 3:10:48Forward for a little while
- 3:10:50uh building a sort of an AI-enabled
- 3:10:53primary healthcare system.
- 3:10:59Okay. So the software factory.
- 3:11:02This is like
- 3:11:04the hottest buzzword in our industry
- 3:11:06right now.
- 3:11:08Everyone's talking about software
- 3:11:09factories. Everyone's trying to build a
- 3:11:11software factory.
- 3:11:15And the question is Well, oh so first of
- 3:11:19all
- 3:11:20um
- 3:11:22most of you probably know what this is
- 3:11:25already.
- 3:11:26Um but if you don't, I'll just give a
- 3:11:29really quick overview. Uh this is sort
- 3:11:32of the the main software development
- 3:11:34lifecycle loop uh as it existed before
- 3:11:37the AI era.
- 3:11:38So we have uh inputs, basically product
- 3:11:42specs, feature requests, tickets,
- 3:11:45things like that. Um we take those into
- 3:11:48the planning step if they're especially
- 3:11:50if they're complicated.
- 3:11:52Uh and then we then we build, then we
- 3:11:54write the code.
- 3:11:56Uh and then we um uh put up a PR and
- 3:12:00that gets uh reviewed and it goes
- 3:12:03through CI and then it eventually
- 3:12:06merges.
- 3:12:08Uh and then we deploy that to
- 3:12:09production.
- 3:12:11And then users interact with our
- 3:12:13software and they give us feedback or
- 3:12:15they find bugs.
- 3:12:17Um
- 3:12:18or we might also get some monitoring or
- 3:12:20and alerts as well. And then those go
- 3:12:22back in as more inputs. And the the
- 3:12:26cycle continues.
- 3:12:28And and so um in the in the era of
- 3:12:32agentic coding and AI automation has
- 3:12:36followed every bottleneck. So the
- 3:12:38bottleneck used to be the the coding
- 3:12:40part, the building.
- 3:12:41Um now with coding agents that's been
- 3:12:44automated.
- 3:12:45Um
- 3:12:47and now the bottleneck is sort of
- 3:12:48shifted over to the that review step.
- 3:12:51And we're starting to introduce uh AI
- 3:12:54code review systems for that.
- 3:12:57Um
- 3:12:58and then and we're seeing uh AI systems
- 3:13:01and automation happen really at every
- 3:13:03step of this now. Uh so we're starting
- 3:13:06to see AI systems for monitoring,
- 3:13:09deployments, controlling rollouts,
- 3:13:12triaging uh triaging you know tickets
- 3:13:16and incidents, planning. And so this
- 3:13:19this here is the sort of software
- 3:13:22factory that we're that we're creating.
- 3:13:28And and so the question that we all have
- 3:13:30now is how do we make this factory safe,
- 3:13:35right? Uh in other words,
- 3:13:37how can we introduce enough layers of
- 3:13:39verification
- 3:13:41into this so that we can trust that the
- 3:13:43software coming out of it is it works as
- 3:13:46expected, it's free of bugs, free of
- 3:13:49security vulnerabilities.
- 3:13:54Um our CEO
- 3:13:56uh Tarek said recently code is abundant,
- 3:13:58confidence is not.
- 3:14:01So what I'm I'm to talk about today is
- 3:14:03just how at Sonar have been thinking
- 3:14:05about uh how to introduce uh
- 3:14:08confidence uh at at all sort of all
- 3:14:12different points within this software
- 3:14:13factory.
- 3:14:17And uh
- 3:14:19we're approaching this this challenge
- 3:14:20with a defense in depth approach
- 3:14:23that comes from adding more loops, you
- 3:14:26might not be surprised to hear.
- 3:14:29And I'm going to talk about four of them
- 3:14:31today in particular. So there's the loop
- 3:14:34one, which is the PR checks loop.
- 3:14:37Uh
- 3:14:38loop two, uh which is the post deploy
- 3:14:42verifications loop.
- 3:14:45Loop three, uh which is the maintenance
- 3:14:48loop.
- 3:14:51And loop four, which is the knowledge
- 3:14:52loop. And this is kind of a kind of a
- 3:14:54special loop.
- 3:14:55Uh
- 3:14:56so first I'm going to talk about loop
- 3:14:58one,
- 3:15:00the PR checks loop. So this is the loop
- 3:15:01that we probably all have in in some
- 3:15:03form.
- 3:15:04Uh you put up a PR, it goes through some
- 3:15:06CI checks, it gets a code review.
- 3:15:09Nowadays it's probably getting an AI
- 3:15:10code review as well. Um but we're
- 3:15:13starting to see a lot more automation
- 3:15:15being introduced here as well. So we're
- 3:15:17starting to see agents pushing fixes to
- 3:15:20respond to the code review comments.
- 3:15:23Um and this loop kind of goes around and
- 3:15:26around until the PR turns green.
- 3:15:34And there's actually a lot more that we
- 3:15:37can be doing here
- 3:15:38um because catching a problem at this
- 3:15:41point is still cheaper than after it
- 3:15:43merges into the codebase.
- 3:15:45Uh so we have a a real opportunity here
- 3:15:47to widen this review past the sort of
- 3:15:50normal tests and code review that most
- 3:15:53of us do.
- 3:15:55So of course we have AI code review.
- 3:15:58We can also do static analysis here.
- 3:16:00Many of us do that.
- 3:16:03Uh we could do what's called functional
- 3:16:05validation. So this is where we can look
- 3:16:07uh what the ticket asks for and see if
- 3:16:09the PR actually addresses those things.
- 3:16:15Um we can look at rules that the team
- 3:16:19may set for coding conventions,
- 3:16:23uh see if uh best practices, things like
- 3:16:26that, see if the PR is adhering to
- 3:16:28those.
- 3:16:30We can look at knowledge. So this can be
- 3:16:32knowledge accumulated from company
- 3:16:34documentation,
- 3:16:36from things that have been learned on
- 3:16:37past code reviews or past outages.
- 3:16:43Uh we could do cross repo checks. So we
- 3:16:45could actually look at other
- 3:16:46repositories and see if anything in in
- 3:16:48this PR uh
- 3:16:50will break or cause something to break
- 3:16:52in another repository.
- 3:16:55Uh we could do cross PR analysis. So we
- 3:16:59can look at all the other open PRs as
- 3:17:01well and see if there's anything uh in
- 3:17:05this PR that conflicts with one of
- 3:17:06those.
- 3:17:10We could do security review, check for
- 3:17:12vulnerabilities, secrets, dependencies.
- 3:17:17We could do end-to-end testing. So for
- 3:17:19example, if this is a web app, we could
- 3:17:20open up in a browser and and click
- 3:17:22around and see if it works as expected.
- 3:17:27Um and finally we can do risk
- 3:17:30assessment. So we could sort of uh look
- 3:17:32at this and compute uh
- 3:17:35the blast radius and sort of categorize
- 3:17:37this into like low, medium, or high
- 3:17:39risk. And that can actually help us
- 3:17:41determine if this is a PR that a human
- 3:17:44even needs to look at at all.
- 3:17:47And so all of these things can flow into
- 3:17:49this this sort of decision, this merge
- 3:17:51gate, this decision of whether this PR
- 3:17:53is mergeable.
- 3:17:55And of course uh every time a a fix is
- 3:17:57pushed, all of these run again,
- 3:18:00at least in some degree.
- 3:18:04And and with all of this, it's important
- 3:18:06to gate the that merge decision not just
- 3:18:09on the fact that a test ran or that it
- 3:18:11passed, but on the evidence.
- 3:18:13So that would be things like the actual
- 3:18:15code review findings, the results of the
- 3:18:17quality gates,
- 3:18:19uh any vulnerabilities,
- 3:18:22um the risk level. So all of that can be
- 3:18:26uh basically fed into this merge gate
- 3:18:29and checked against criteria that the
- 3:18:31the team can set for what is a mergeable
- 3:18:33PR.
- 3:18:35And when you record that evidence uh to
- 3:18:37the PR, it just makes this
- 3:18:40uh the merge gate much stronger.
- 3:18:45And so what we're seeing now uh with
- 3:18:47teams who do this well
- 3:18:49uh is that they can actually delegate
- 3:18:50more and more trusts uh to this type of
- 3:18:53automation on the PR review.
- 3:18:56Uh so we're seeing teams begin to turn
- 3:18:59on not just automatic merge blocking,
- 3:19:02but uh even auto approve in some cases
- 3:19:05for low risk changes and and even auto
- 3:19:08merge for certain types of PRs that are
- 3:19:10that are low risk and approved.
- 3:19:14And this can just free up a lot of
- 3:19:15review bandwidth for human reviewers to
- 3:19:17review the the more high risk changes.
- 3:19:22Okay, so now um I'm going to talk about
- 3:19:25loop two, which is our post deploy
- 3:19:27verification loop.
- 3:19:29Uh so this is I would say very much in
- 3:19:32its early stages right now. People are
- 3:19:34experimenting with all kinds of things
- 3:19:36here.
- 3:19:37Um but the idea is that once your code
- 3:19:40is deployed,
- 3:19:41uh you can have you you can build AI
- 3:19:44systems to check uh to read like
- 3:19:46production logs, traces, metrics, and
- 3:19:49see if your code is actually behaving as
- 3:19:51you expected to in production.
- 3:19:55And then from there you can actually um
- 3:19:58uh we're starting to see things like
- 3:20:00agentic rollout where agents can decide
- 3:20:03whether to widen a rollout, a rollback,
- 3:20:06turn a feature flag off.
- 3:20:09And you can also do uh AI outage
- 3:20:11monitoring so you can uh
- 3:20:14monitor alerts and uh use AI systems to
- 3:20:17help you uh root cause and fix outages.
- 3:20:24Okay. Uh
- 3:20:25so moving on to loop three. Uh so this
- 3:20:29is what I call the maintenance loop.
- 3:20:31Uh this is kind of a a parallel loop
- 3:20:33that runs alongside your main software
- 3:20:36development uh life cycle loop.
- 3:20:39And this is where we continuously scan
- 3:20:41the codebase on some schedule.
- 3:20:43And we check for a lot of the same
- 3:20:45things that we check for in the PR. We
- 3:20:46we could do code review, static
- 3:20:48analysis, security checks, etc.
- 3:20:51And then we can If we find an issue, we
- 3:20:53can open up a remediation PR
- 3:20:56to address that.
- 3:20:58And this lets us catch things that we
- 3:21:00couldn't catch in the PR. So for
- 3:21:02example, you could imagine
- 3:21:04um
- 3:21:06maybe a new rule was introduced after
- 3:21:07code merged
- 3:21:09or a new vulnerability was published
- 3:21:11after code merged.
- 3:21:13Um
- 3:21:14or even something like slow drift where
- 3:21:17you you slowly get duplication or
- 3:21:19complexity building up that you can't
- 3:21:21really see on a single PR.
- 3:21:23Uh you can catch these kinds of things
- 3:21:25here
- 3:21:27with these uh with the continuous
- 3:21:28codebase scans.
- 3:21:33Okay. And so finally we move over to the
- 3:21:36knowledge loop. And And this is becoming
- 3:21:38We're seeing this become a very
- 3:21:39important uh loop
- 3:21:41for a lot of companies.
- 3:21:43Um every company has a lot of knowledge,
- 3:21:46documentation, product specs,
- 3:21:50um
- 3:21:51business decisions,
- 3:21:54um and
- 3:21:56the agents do a better job if they have
- 3:21:58that knowledge, right?
- 3:22:00So
- 3:22:01[clears throat] this loop basically
- 3:22:03collects all that knowledge from
- 3:22:04different sources
- 3:22:06as well as things that were learned on
- 3:22:07prior code reviews and from prior
- 3:22:10outages.
- 3:22:11It's curated and then it can be uh fed
- 3:22:15into all of the other loops, all of
- 3:22:17these other agents,
- 3:22:19um so they can uh do a better job.
- 3:22:25And for example, this is really powerful
- 3:22:28with your coding agents because if they
- 3:22:30have some of this knowledge, they can
- 3:22:32probably do a better job the first time,
- 3:22:35fewer issues,
- 3:22:36um and then you'll you'll wind up with
- 3:22:39basically fewer review laps at the uh
- 3:22:41review stage.
- 3:22:47Okay, so what does this mean for our
- 3:22:51role as human software engineers in this
- 3:22:53factory?
- 3:22:55Uh you might be asking, okay, so if if
- 3:22:58we nail all of this, can we just uh turn
- 3:23:00the lights out and run a dark factory?
- 3:23:04Uh and I would say probably not, not
- 3:23:06right now.
- 3:23:08Uh I like to say the factory runs dim,
- 3:23:10not dark. It's still very important for
- 3:23:13us to
- 3:23:14[cough]
- 3:23:15intervene in places where our expertise
- 3:23:18is is really important.
- 3:23:20Uh
- 3:23:21so that would be things like auditing
- 3:23:23risky code,
- 3:23:25um
- 3:23:26verifying that evidence, verifying
- 3:23:29things are actually working as expected,
- 3:23:32handling exceptions like rollbacks and
- 3:23:35incidents,
- 3:23:37uh
- 3:23:38reviewing architecture.
- 3:23:41Agents are not very good at this yet, I
- 3:23:43would argue.
- 3:23:45Um
- 3:23:46and supplying the why or or helping to
- 3:23:49build that knowledge that that powers
- 3:23:51all these agents and all these loops,
- 3:23:54supplying the product intent, the
- 3:23:56trade-offs, the business decisions,
- 3:23:58things like that.
- 3:24:01And of course, we engineer the factory
- 3:24:04ourselves, right? We're we're we've sort
- 3:24:06of moved into this era where we're we're
- 3:24:07building the software factory as a
- 3:24:09whole.
- 3:24:13So with that, I hope that gave you sort
- 3:24:17of a framework for how to think about um
- 3:24:19not just how to build your software
- 3:24:21factory, uh
- 3:24:23but how to build a safe software
- 3:24:25factory.
- 3:24:26So, thank you. That's all for me.
- 3:24:29>> [applause]
- 3:28:20>> <music>
- 3:34:06>> This
- 3:34:28>> <music>
- 3:37:18>> Yeah.
- 3:42:45>> All set.
- 3:42:47Hi everyone. I'm Luke. I lead our core
- 3:42:50agent team here at Factory. Today I want
- 3:42:52to talk about autonomy and why the teams
- 3:42:55that are getting the most out of coding
- 3:42:57agents are not actually the ones that
- 3:42:58are handing over like the most work on
- 3:42:59day one. It's instead the folks who are
- 3:43:02ramping up autonomy gradually, one
- 3:43:04checkpoint at a time until what they
- 3:43:05have starts to resemble a software
- 3:43:07factory.
- 3:43:09So for those who don't know us, Factory
- 3:43:11is a platform that builds Droid and then
- 3:43:14a whole like surface for software
- 3:43:16engineering and building software on top
- 3:43:18of that coding agent. Hundreds of
- 3:43:20thousands of developers use us around
- 3:43:22the world at companies like Nvidia,
- 3:43:23T-Mobile, and Adobe.
- 3:43:26What these companies' engineers are
- 3:43:28building with us is is really like a
- 3:43:30software factory, a system that runs
- 3:43:3124/7 and builds software under the
- 3:43:34company's control.
- 3:43:36And that's a loop that takes in signals
- 3:43:38from wherever information is already
- 3:43:39available, turns them into changes,
- 3:43:42validates them, and then learns from
- 3:43:44what production shows. A signal is
- 3:43:46really like anything that tells you how
- 3:43:48your software should evolve. So that
- 3:43:49could be you know a sentry alert or
- 3:43:51someone's idea on Slack. Every signal
- 3:43:54goes through the same stages, so you
- 3:43:55standardize you standardize the the
- 3:43:57process, not the output.
- 3:44:00So we built that platform on five
- 3:44:02foundations. It has to be transparent,
- 3:44:05modular, governable, efficient, and
- 3:44:07frontier.
- 3:44:10Most people picture a software factory
- 3:44:12as this kind of like lights out building
- 3:44:14where there's no people inside of it.
- 3:44:16Requirements come in one end and merge
- 3:44:18PRs come out the other. And and really
- 3:44:21what we saw firsthand in the field is
- 3:44:23that that's not the case. The best ones
- 3:44:24aren't set up like that. They're built
- 3:44:26around people instead. Humans decide
- 3:44:29what matters and how much to trust
- 3:44:31Droids over time, and those Droids earn
- 3:44:33their keep as they deliver more work
- 3:44:34that's higher quality.
- 3:44:36Every company's software factory looks
- 3:44:38very different to each other. So what we
- 3:44:40what we sell is not really this factory,
- 3:44:41it's the tools and the building blocks
- 3:44:43to compose them.
- 3:44:46Now it's it's worth kind of looking at
- 3:44:47how we got here in the first place. Back
- 3:44:50in April, one of our flagship products
- 3:44:51was Missions, which was one of like the
- 3:44:53first multi-agent uh products to really
- 3:44:56take like a bite out of the human
- 3:44:58attention bottleneck that existed in in
- 3:45:00software engineering.
- 3:45:02You describe a goal, you like argue with
- 3:45:04the orchestrator until you were aligned,
- 3:45:06you'd approve the plan, and then it
- 3:45:08would go handle implementation for like
- 3:45:10days or even weeks. When the goal was
- 3:45:12clear and verifiable, you really could
- 3:45:14come back to like completed work pretty
- 3:45:16consistently.
- 3:45:18But as more teams adopted it, we heard
- 3:45:20some consistent like pieces of feedback.
- 3:45:22The first was how Missions asked you to
- 3:45:24work. The product assumed that you could
- 3:45:25plan everything up front and then it
- 3:45:27would just work. Uh and the UX sort of
- 3:45:29suggested that you shouldn't interrupt
- 3:45:31them. Missions didn't really like ask
- 3:45:32you any questions or provide like an
- 3:45:34interactive surface for that uh
- 3:45:36communication between you and the agent
- 3:45:38once it was starting implementation. So
- 3:45:40this suits like migrations and big uh
- 3:45:43big coding like feature projects, but it
- 3:45:45didn't really fill fit the bill for like
- 3:45:47most kinds of other work.
- 3:45:49The second was cost and time. And that
- 3:45:51came from honestly like the model's
- 3:45:53getting really really good. I'm sure
- 3:45:55you've all seen now adversarial review
- 3:45:58is really really good at finding these
- 3:45:59issues in your code. And so uh at the
- 3:46:02time we were adding adversarial review
- 3:46:04into every milestone's boundary. That
- 3:46:07just kind of like grew the scope of that
- 3:46:09mission from something like
- 3:46:1160 features to over 200.
- 3:46:14And so you ended up having a token bill
- 3:46:16and a timeline that matched that.
- 3:46:18And then the last one was that Missions
- 3:46:20were keeping their context to one
- 3:46:21person. So that was fine for like a
- 3:46:23single project, but it meant that the
- 3:46:25rest of the team couldn't really build
- 3:46:27on what that mission had learned.
- 3:46:30So we went back to the drawing board.
- 3:46:32And one mental model kept on coming up.
- 3:46:34When you have like an org that is
- 3:46:36running dozens or hundreds of agents at
- 3:46:39a given point in time, you actually have
- 3:46:41like a lot of the the the problems that
- 3:46:43operating systems back in the day
- 3:46:45solved.
- 3:46:46How to schedule work, how to run what
- 3:46:48many things at once safely, and how to
- 3:46:50share state.
- 3:46:53So, every operating system starts with
- 3:46:55the thing that actually does the work,
- 3:46:57and we wanted to revisit the agent
- 3:46:58itself.
- 3:46:59Long Horizon agents run for thousands of
- 3:47:01turns, and every inefficiency across of
- 3:47:04those gets multiplied. So, you could
- 3:47:06have, you know, an extra inference
- 3:47:07request that's unnecessary or a a tool
- 3:47:10result that's pretty bloated or a prompt
- 3:47:12cache miss. All of those things compound
- 3:47:15over time and lead to waste.
- 3:47:18So, we decided to make some pretty
- 3:47:19significant changes to the harness
- 3:47:20itself.
- 3:47:22The biggest one was programmatic tool
- 3:47:23calling, also known as code mode.
- 3:47:25Instead of calling one tool, waiting on
- 3:47:27the model, and then calling the next, we
- 3:47:29have the model write a small program
- 3:47:31that executes all those tools in one go
- 3:47:33and then prints out only what's
- 3:47:35relevant.
- 3:47:36So, there's this example here where you
- 3:47:38have 150 linear tickets that are getting
- 3:47:40moved from one project to another.
- 3:47:42Done the normal way, that's like 30
- 3:47:44inference requests, a million tokens,
- 3:47:46but as one model generated program, you
- 3:47:49actually only have like 60k tokens. It
- 3:47:51runs much faster, and the model only
- 3:47:52sees what is actually relevant to the
- 3:47:54task, the the the fact whether it was
- 3:47:56like successful or not.
- 3:47:59So, this layer of the OS is where being
- 3:48:00frontier and efficient matters the most.
- 3:48:03A new model lands every few weeks, and
- 3:48:05no two fail fail the same way. So, we
- 3:48:07tune our harness in such a way that
- 3:48:09evens out this jaggedness across all
- 3:48:11these different models. The result is
- 3:48:12that Droid is now, you know, higher
- 3:48:14quality at a lower cost than every other
- 3:48:16coding agent on the market.
- 3:48:20Now, you have a faster agent, but that
- 3:48:21that doesn't really give you like a step
- 3:48:23function increase in in how much work
- 3:48:25gets done inside of your system.
- 3:48:27For that, you need many agents running
- 3:48:28at once, and they need somewhere to run.
- 3:48:31Remote missions were kind of held back
- 3:48:33by this like fundamental limitation in
- 3:48:35the system
- 3:48:36where there was one computer that
- 3:48:37handled all the work.
- 3:48:39But to run a software factory, you need
- 3:48:41a compute that scales with the volume of
- 3:48:43work in the system. Otherwise, every
- 3:48:45agent is is fighting for the same files
- 3:48:47or system resources.
- 3:48:49And so that's why we built environments,
- 3:48:51which are reusable definitions of the
- 3:48:52sandboxes that your droids need,
- 3:48:54complete with the repos, tools, and
- 3:48:56dependencies that they that they need to
- 3:48:58get started with.
- 3:49:00This system is governable, so there's
- 3:49:01one control plane that orchestrates all
- 3:49:04of these sandboxes together wherever you
- 3:49:06need them.
- 3:49:08They can run inside of your own VPC, on
- 3:49:10on EC2, Kubernetes, whatever compute you
- 3:49:12already operate. So your code, your
- 3:49:14data, and your network are in your
- 3:49:16control.
- 3:49:17Compute becomes pluggable, just like we
- 3:49:19made the BYOK options and auto model
- 3:49:22options that we provide make that the
- 3:49:24provider sort of like this like model
- 3:49:26pluggable layer.
- 3:49:28So now we had agents that were cheap
- 3:49:30enough to run all the time
- 3:49:31and the compute to run many at once.
- 3:49:34And so then the next question was how do
- 3:49:35we orchestrate all of this work?
- 3:49:38At first we tried making like this DAG,
- 3:49:40very complex system of agents composed
- 3:49:43together by like these very like strong
- 3:49:45rules, and it didn't really work. You
- 3:49:47end up having a lot of the same problems
- 3:49:49that plagued missions in the first place
- 3:49:51because it was too rigid, and as a
- 3:49:53result it was too expensive to run, and
- 3:49:55it didn't generalize well.
- 3:49:58So what we landed on instead is
- 3:49:59something that we call a work stream.
- 3:50:01It's a standing goal with a team of
- 3:50:03agents to support it. It has this goal,
- 3:50:05it has an identity, an environment that
- 3:50:08it runs on, and memory.
- 3:50:10A new work stream runs as you, and it
- 3:50:12has your access. When it outgrows one
- 3:50:15person, it can be moved to a service
- 3:50:17account with its own credentials. So you
- 3:50:19stay on as a member, but the work keeps
- 3:50:21happening under an identity that the the
- 3:50:23org actually controls.
- 3:50:26At the center of every work stream is an
- 3:50:28orchestrator, and its job looks a lot
- 3:50:30like a scheduler in an OS.
- 3:50:32Unlike the graph that we started with,
- 3:50:34it assembles its agents in whatever way
- 3:50:36is most natural for the model.
- 3:50:38So the instructions are pretty short,
- 3:50:40right? It's like delegate first, give
- 3:50:42each agent an area of ownership, and
- 3:50:44then for tasks that can fall into an
- 3:50:46existing area of ownership, delegate to
- 3:50:48those.
- 3:50:50You start by talking to it,
- 3:50:51Uh, just kind of hand things over as you
- 3:50:53think of them, and then it fans out the
- 3:50:55work so that then uh you can get the the
- 3:50:58control back to you.
- 3:51:00Uh, agents report back when they finish,
- 3:51:02when they get blocked, when there's
- 3:51:03something that changes the plan overall.
- 3:51:07But the work stream doesn't just stop
- 3:51:08there because it takes on more on its
- 3:51:10own and it learns from your habits. At
- 3:51:13some point the orchestrator may ask you
- 3:51:14how you decide what to work on, and it
- 3:51:16learns that maybe bugs are coming from
- 3:51:18two Slack channels.
- 3:51:21So here it can set up an intake at a
- 3:51:23triage listener.
- 3:51:24Every 30 minutes it can go in and pull
- 3:51:27in those reports from those channels,
- 3:51:28dedupe them against, you know, existing
- 3:51:30linear issues or uh sentry sentry alerts
- 3:51:33that already exist, and then weigh each
- 3:51:35of those against the work stream's goal.
- 3:51:38So a Slack complaint and a customer
- 3:51:40error report become one change.
- 3:51:43The output of triage is a spec that you
- 3:51:45review and uh you're capable of like
- 3:51:49weighing in on before any code gets
- 3:51:50written.
- 3:51:51Once you approve a spec, the same worker
- 3:51:53opens a PR,
- 3:51:55stewards it until it's merge ready, uh
- 3:51:57and then make sure that, you know,
- 3:51:58review uh comments and CI are passing.
- 3:52:03So each one of those stages actually
- 3:52:05maps to an autonomy level in this
- 3:52:06system.
- 3:52:08At L1 the work stream pauses for human
- 3:52:10review at the spec creation, at the PR,
- 3:52:14and before hitting merge.
- 3:52:16But, you know, as you level up that the
- 3:52:18those levels of autonomy you can get to
- 3:52:21L4 where you're auto merging everything
- 3:52:23by default.
- 3:52:24The owner gets to set the level,
- 3:52:26but the confidence to raise it comes
- 3:52:28from watching the work.
- 3:52:29So when proposed changes are accurate or
- 3:52:32PRs consistently merge at the quality
- 3:52:34that you'd expect, that's actually your
- 3:52:36cue to move to the next checkpoint.
- 3:52:38And so that becomes the dial for you.
- 3:52:40You don't need to decide on day one
- 3:52:41whether you trust a coding agent to own
- 3:52:43you know you like CI health work stream
- 3:52:45off the bat.
- 3:52:47You just need to trust it to to like
- 3:52:49unlock the next checkpoint.
- 3:52:52So in a factory an agent that starts
- 3:52:54from scratch repeats the same mistakes
- 3:52:56the last one already did. And so we
- 3:52:58built work streams to have a shared file
- 3:53:00system.
- 3:53:02Agents are are encouraged to keep memory
- 3:53:05and version skills and so their PRs get
- 3:53:08better over time. They're also proactive
- 3:53:10and they go looking for context that
- 3:53:11they're missing. So for example, say
- 3:53:14that your work stream notices that other
- 3:53:15people are shipping related changes.
- 3:53:18It sets up an automation that pulls in
- 3:53:20their PRs,
- 3:53:21analyzes what they've been working on
- 3:53:23and what comments they've been receiving
- 3:53:25in those PRs and then incorporates them
- 3:53:27into their memory and skills.
- 3:53:29So what what agent learns others can use
- 3:53:32and nothing starts cold.
- 3:53:36That folder is also how agents work
- 3:53:38together on a project. When the
- 3:53:40orchestrator spawns an agent, it's
- 3:53:42capable of telling, you know, each agent
- 3:53:44where does it where where do where does
- 3:53:46its results sort of belong inside of
- 3:53:47this shared folder.
- 3:53:49And so every agent can read and write to
- 3:53:52the same place.
- 3:53:54If you ask it a a work stream why
- 3:53:56checkout got slower, for example, it can
- 3:53:58send dozens of agents out to track out
- 3:54:00all of those different leads that you
- 3:54:01have and then ultimately each one can
- 3:54:04write its findings to the folder and the
- 3:54:06orchestrator can merge them.
- 3:54:09Now you still need a barrier between
- 3:54:10implementation and merging and for us
- 3:54:13that barrier is platform level
- 3:54:14validation.
- 3:54:15Every PR gets code review, security
- 3:54:18review, and dedicated QA and a live
- 3:54:20preview.
- 3:54:21And so once every review thread is
- 3:54:23resolved, a risk judge then goes in and
- 3:54:25decides whether it's safe to approve or
- 3:54:27not.
- 3:54:28The judge gets the exact diff and a
- 3:54:30checkout of the branch so it can like
- 3:54:31build and test it. Uh but ultimately it
- 3:54:34reads its policy from our mainline
- 3:54:35branch so that the change can't rewrite
- 3:54:37the rules that it's being judged by.
- 3:54:40Every week Android mines the review
- 3:54:42comments and goes through and figures
- 3:54:44out which one authors have actually
- 3:54:46acted on and then pulls them pull pulls
- 3:54:49those into like review guidelines so
- 3:54:51that the the loop sort of improves over
- 3:54:53time.
- 3:54:54We also have droids that every day run
- 3:54:56and analyze, revert, hot fixes, new
- 3:54:58century errors, for example, and traces
- 3:55:01them back to the PRs that that caused
- 3:55:02them. And so that also is a signal into
- 3:55:04this same loop.
- 3:55:07What we're really hill climbing on here
- 3:55:09is the regret-free merge rate, the share
- 3:55:12of changes that merge and never need a
- 3:55:14revert or a follow-up fix.
- 3:55:16We measure it for Droid-written and
- 3:55:18human-written PRs side by side. And it's
- 3:55:20the same way that we let like
- 3:55:21self-driving cars onto the road. We
- 3:55:23didn't just give them the green light.
- 3:55:25We only did so once they crashed less
- 3:55:28often than human-driven ones did.
- 3:55:30And so once AI-written changes break
- 3:55:32things less often than human ones and
- 3:55:34you can actually measure that, then you
- 3:55:36can let them merge. And that's the bar
- 3:55:38for turning a work stream to L4.
- 3:55:41Work stream is now like the platform
- 3:55:43that we build new features on top of. So
- 3:55:45migrate is the first one. It sort of
- 3:55:47fulfills a niche in our product
- 3:55:50like framework for for what missions
- 3:55:52used to be. There's like dedicated
- 3:55:54one-shot migration experience. And
- 3:55:56continuous remediation is the second
- 3:55:58where it follows that circular shape
- 3:56:00that work streams are best suited for
- 3:56:01because they are the work never ends.
- 3:56:06We run factory on top of work streams
- 3:56:08ourselves. And so this is what it
- 3:56:10actually looks like in production.
- 3:56:12In a week in this past October, we
- 3:56:13shipped about 800 pull requests in our
- 3:56:16dev branch. And Droid wrote 99% of them
- 3:56:18and reviewed all of them.
- 3:56:20Now work landed in one of two ways. It
- 3:56:22was either human picking up a signal
- 3:56:24manually
- 3:56:25or increasingly nowadays it's a work
- 3:56:28stream picking it up
- 3:56:29automatically, triaging it against the
- 3:56:31goal of the work stream, and then
- 3:56:33ultimately writing a spec and merging
- 3:56:35it.
- 3:56:3752% of PRs that we ship right now are
- 3:56:39coming from work streams themselves. And
- 3:56:41some of these are operating on full
- 3:56:43autonomy. So for example, the one that
- 3:56:46fixes bug reports from a dedicated Slack
- 3:56:48channel merges 53 PRs per week. Another
- 3:56:51one is our flaky test work stream, which
- 3:56:53merges about 40.
- 3:56:56Fully autonomous PRs aren't really the
- 3:56:58only thing worth tracking though.
- 3:57:00It What matters a lot more is the
- 3:57:02progression.
- 3:57:03How much of your work flows through work
- 3:57:04streams and how many checkpoints does
- 3:57:06each one still need?
- 3:57:08As more work flows unrestricted, your
- 3:57:10engineers become the regulators of this
- 3:57:11system. They decide, you know, what
- 3:57:13signals to watch, uh what are the goals
- 3:57:16and the owners of those, what are the
- 3:57:17skills and review guidelines that you
- 3:57:19need in order to keep this code high
- 3:57:20quality, and how much autonomy each work
- 3:57:23stream gets.
- 3:57:25And so there's just one question that
- 3:57:26tells you, you know, whether whether you
- 3:57:28have a software factory or faster
- 3:57:30engineers, and that is can you name and
- 3:57:32track your signals?
- 3:57:34Every team already has them. You have
- 3:57:35bug reports in Slack, you know, failing
- 3:57:37CI jobs, error spikes, tickets that
- 3:57:40nobody ever picks up, but most of them
- 3:57:42wait on a person to notice.
- 3:57:44And so my ask to all of you is to write
- 3:57:46them down and then pick the one that you
- 3:57:48waste the most time on.
- 3:57:50Give it to a work stream on L1 and then
- 3:57:52set up our validation experience, and
- 3:57:54then you're well on your way to building
- 3:57:56software that improves itself with all
- 3:57:58the signals that you choose to track.
- 3:58:01Thanks everyone, and I'll be around to
- 3:58:02answer any questions today.
- 3:58:04>> [applause]
- 3:59:46>> Hey guys, this is Era. I'm a founding
- 3:59:49engineer at Klein and today I'm going to
- 3:59:51be talking about recursively
- 3:59:52self-improving coding agents. So
- 3:59:55this is a very
- 3:59:57interesting conversation in the sense
- 3:59:59that everyone wants to understand what
- 4:00:01RSI is, how we accomplish it, how we go
- 4:00:03through it and how it affects our
- 4:00:04career. So I'm going to start with
- 4:00:06giving you just three goals of this
- 4:00:08entire conversation, just three simple
- 4:00:09goals and that would help you understand
- 4:00:13like what what we get out of it and what
- 4:00:14the future for this looks like. So the
- 4:00:16goal number first is that understanding
- 4:00:18the intent of Frontier Labs, right?
- 4:00:20Everyone who's outside of the Frontier
- 4:00:21Labs is always trying to speculate like,
- 4:00:23hey, what what is the topic in OpenAI or
- 4:00:25what is Deep Mind going to come out with
- 4:00:26or what what they're trying to make?
- 4:00:28And what's the future going to look
- 4:00:30like? Who has cracked the RSI? So like
- 4:00:32this kind of helps you get get a rough
- 4:00:34outline of what their roadmap looks
- 4:00:36like. And the second thing is that like
- 4:00:38if the models are improving so fast,
- 4:00:40like how do you think about your career
- 4:00:42or the next few years, right? That is
- 4:00:44really what we want to understand. We
- 4:00:45want to understand like, okay,
- 4:00:47things are changing so fast, like the
- 4:00:49ground underneath us is shifting so
- 4:00:51quickly. Right now, like it's like
- 4:00:55there used to be a time when you could
- 4:00:56like
- 4:00:57demonstrate your side projects on your
- 4:01:00personal website or on GitHub. And now
- 4:01:02it's just like you could literally one
- 4:01:03shot them using any frontier coding
- 4:01:05agent.
- 4:01:06And with that, it's like it's like oh,
- 4:01:08how do we demonstrate proof of work? How
- 4:01:10do we demonstrate, you know, that that
- 4:01:12we know what we're doing and how do we
- 4:01:14think of our career for the next coming
- 4:01:16years, right? So, it's very interesting
- 4:01:18times that we're living in. And then the
- 4:01:20third and the most important goal of the
- 4:01:22conversation is escaping the permanent
- 4:01:23underclass, right? So, I know this seems
- 4:01:27like a joke, but this really is a very
- 4:01:28valid possibility that people are
- 4:01:31contemplating. Everyone's like, oh,
- 4:01:32well, if everything that we want to be
- 4:01:34doing can potentially easily done by a
- 4:01:37frontier model, like what are we what
- 4:01:38are we good for? And like where does
- 4:01:40software go? Where where where does the
- 4:01:41future of technology going to go? And
- 4:01:43what are we going to do as as things
- 4:01:44evolve so fast, right? So, I think that
- 4:01:47there's like a lot of like
- 4:01:49fearmongering, but also there's like
- 4:01:50some very valid merits where people are
- 4:01:53concerned that like if all the revenue
- 4:01:55of the planet is going to go to um
- 4:01:58a few hardware companies that make
- 4:02:00frontier GPUs and then a few few uh AI
- 4:02:03companies that that make the best
- 4:02:05models, like
- 4:02:06how are we how are the rest of us going
- 4:02:08to survive and what what are we going to
- 4:02:09do here? So, I think I'll I'll walk you
- 4:02:12through a few examples and a few ideas
- 4:02:13that will like really really help you
- 4:02:15reconsider the career choices and and
- 4:02:18and the life choices you're making and
- 4:02:20what your problems you're picking up.
- 4:02:22And then and then that would help you
- 4:02:23like realign
- 4:02:25what it is that we should be working on
- 4:02:27and what it is that you, you know, would
- 4:02:29would
- 4:02:30that you could work on that will stand
- 4:02:31the test of time and help you both just
- 4:02:33not just like on a psychological but
- 4:02:35like financial and other levels as well.
- 4:02:36So,
- 4:02:37uh to give you a very precise example,
- 4:02:39let's start with something very precise
- 4:02:40that I'd done. So, uh a couple months
- 4:02:42ago, I uh came up with recursive super
- 4:02:47improvements on AI coding harness. So,
- 4:02:50this was the first a public example of
- 4:02:53uh an automatic evaluation improvement.
- 4:02:55So, to give an example, uh we [snorts]
- 4:02:57used to do this thing where we would
- 4:02:58measure
- 4:03:00the quality of a coding agent using
- 4:03:02evaluation scores. So,
- 4:03:04on terminal 2.1, Kimi K 3 with Klein
- 4:03:07would score um usually like you know
- 4:03:1060 70% and then depending on like how
- 4:03:13good the model is like you get better
- 4:03:15scores on thermal bench 2.1 so we
- 4:03:18noticed that when K3 came out our score
- 4:03:20was 77.5% which is a good score but we
- 4:03:22were like oh is this is this truly is
- 4:03:25this truly the best score that this
- 4:03:26model can get on this benchmark and also
- 4:03:29like what is the model release paper
- 4:03:31claim about what their best score is so
- 4:03:33the model's release paper said that they
- 4:03:35could score up to 85% or even more so we
- 4:03:37were just like okay okay so we are not
- 4:03:40getting the sort of score and often what
- 4:03:41we do with model releases incline is
- 4:03:43that we try to like optimize your score
- 4:03:45and try to get the best out of the
- 4:03:46model.
- 4:03:47So
- 4:03:48when we started hill climbing on the on
- 4:03:51the model like we we realized this we
- 4:03:53realized that like you know there's a
- 4:03:55lot of steps that we do such that we do
- 4:03:57it gracefully such that we pursue the
- 4:03:59right kind of problems and do it the
- 4:04:00right way and we started thinking like
- 4:04:03okay if we there's a lot of groundwork
- 4:04:05here how do we automate it and this is a
- 4:04:07demonstration of like how we automated
- 4:04:09it so I'll I'll just leave that as a
- 4:04:11placeholder for now and we'll get get
- 4:04:12back to it later where we talk about in
- 4:04:14detail of like how does like this
- 4:04:16automatic improvement loop of a client
- 4:04:18harness work so this basically means
- 4:04:20that like you have a coding agent
- 4:04:21harness that could just like basically
- 4:04:22improve itself and become better on on
- 4:04:25the same set of problems.
- 4:04:26So let's start with the ground terms
- 4:04:28what is your recursive self from
- 4:04:29recursive self-improvement is the idea
- 4:04:31that an AI system that helps build
- 4:04:33[snorts] a more
- 4:04:34capable successor which will piece the
- 4:04:36process you know until you heat
- 4:04:38artificial general intelligence right
- 4:04:40you have better systems you have better
- 4:04:41improvements you have a better system
- 4:04:42and then you know you have singularity.
- 4:04:44So a classic example is like a
- 4:04:46self-improving credit coding agent right
- 4:04:48self-improving coding agent is just like
- 4:04:49a coding agent that can improve itself
- 4:04:51so
- 4:04:52you know this this is all like a pie in
- 4:04:54the sky idea like it's like it's like
- 4:04:55you know yeah sure this looks cool but
- 4:04:57how did how did this even come out to be
- 4:04:58like where how did we even get to here.
- 4:05:01So
- 4:05:02if you notice in topic short map in 2021
- 4:05:04in 2021's roadmap they very specifically
- 4:05:07said that
- 4:05:09they want to a general AI assistant that
- 4:05:11can interact conversationally with a
- 4:05:13user to create, add and explain
- 4:05:15software. It would display all the
- 4:05:17capabilities of a software engineer
- 4:05:18colleague. You would write functions,
- 4:05:20you would identify bugs, you would
- 4:05:21design an API, discuss style and best
- 4:05:23practices, right? So we don't expect to
- 4:05:25reach this goal right away, but it's in
- 4:05:26the direction which we are striving.
- 4:05:28It's interesting how this was more than
- 4:05:30five years ago. More than five years
- 4:05:31ago, people had predicted that there
- 4:05:34does exist a day where you will have
- 4:05:35like a full-blown general AI assistant
- 4:05:37that can basically act like a colleague,
- 4:05:39which we do have right now.
- 4:05:41Then in 2022, ChatGPT got released. We
- 4:05:43all know how that happened. It was it
- 4:05:45was a fun moment. It was like the first
- 4:05:47time things got sparked for us.
- 4:05:50Then in 2023, 2024, that was the first
- 4:05:52time chats
- 4:05:54models could call functions. So these
- 4:05:55functions would be simple functions
- 4:05:57like, hey, what's the weather? Or you
- 4:05:59know, what's the what's you know, what's
- 4:06:01this computation or what this
- 4:06:03computations result in Python?
- 4:06:05And then finally in 2024 when
- 4:06:08was released, that was when the agents
- 4:06:10could like truly call like the actual
- 4:06:12tools, the way we call like think of
- 4:06:14tools. So it's like reading a file,
- 4:06:15editing a file, you know, reading,
- 4:06:18searching something on web, listing
- 4:06:20listing a folder, searching and grabbing
- 4:06:21through, you know, a file system, things
- 4:06:24like that. So
- 4:06:26and in 2024, we had the intuition that
- 4:06:28like chatbot, what if chatbots could
- 4:06:30actually talk to each other? You know,
- 4:06:32so then that was the first time when we
- 4:06:33had like, you know,
- 4:06:35multi-agent conversations. Now at the
- 4:06:37time, these things were very janky,
- 4:06:38right? Like they were just like taped
- 4:06:40together in this like weird
- 4:06:42weird
- 4:06:43bizarre ways where you have just like
- 4:06:45these classes and these links and then
- 4:06:47like these hierarchies of classes and it
- 4:06:49was like it was all like kind of like a
- 4:06:50very janky attempt to like make us think
- 4:06:53that like these these models could like
- 4:06:55have this pipeline of flows of responses
- 4:06:56that can all communicate with each
- 4:06:57other.
- 4:06:58[snorts] But a lot of it was like mostly
- 4:07:00deterministic and it was like even you
- 4:07:02know, it was not as free-flowing as it
- 4:07:03is right now. So
- 4:07:05and you know, we started getting the
- 4:07:07intuition. So this was the autogen
- 4:07:09project, but like in only 2025 we we
- 4:07:11started [snorts] getting the intuition
- 4:07:12like what if the agents just kept
- 4:07:13getting better and faster and cheaper
- 4:07:15what what if that just kept happening
- 4:07:17and then in 2026 uh we we had the
- 4:07:20intuition that like what if what if the
- 4:07:22agent could improve itself which is like
- 4:07:24the final you know the final part of the
- 4:07:27whole process so let me give an example
- 4:07:29like what does it mean for an agent to
- 4:07:30improve itself right so like it's like a
- 4:07:32lot of times when we think of like
- 4:07:34evaluating an agent we initially used to
- 4:07:37think of evaluating an agent in the form
- 4:07:38of like just a single turn conversation
- 4:07:40so we would tell we would tell like um
- 4:07:44um agents that like ask a simple
- 4:07:45question like how many toes does a dash
- 4:07:48animal has like it could be let's say
- 4:07:50how many toes does a cat have and then
- 4:07:52LLM would go through and the LLM would
- 4:07:53like prepare a response and then we
- 4:07:55would grade the response so like a cat
- 4:07:56has maybe 10 toes 18 toes whatever many
- 4:07:59I don't I actually don't know how many
- 4:08:00toes a cat has but anyway
- 4:08:02um
- 4:08:04yeah so then we create the responses
- 4:08:06like a yes or no but like when you work
- 4:08:07with an agent there's a ton of things
- 4:08:09that you're doing so let me give an
- 4:08:11example let me give an example so for
- 4:08:12example you can ask an agent like hey
- 4:08:14set up this MCP for me so now it's not
- 4:08:17just like it's not just like you know a
- 4:08:19trivial one shottable like thing like it
- 4:08:21will like search online to go what's the
- 4:08:23MCP what is the config so I'll like edit
- 4:08:26and download some soft files like maybe
- 4:08:28I need to add database update the MCP
- 4:08:30tool maybe I need to read and edit some
- 4:08:32files maybe I need to run some bash
- 4:08:34commands right so you do all of that and
- 4:08:36then the agent responds like hey I did
- 4:08:38it and then you grade that with a bunch
- 4:08:40of tests to gauge like hey did the agent
- 4:08:42actually accomplish the thing that it
- 4:08:44said it would do
- 4:08:45so to do that thing the second part
- 4:08:48which is like more intricate where
- 4:08:49you're actually like truly you know
- 4:08:52testing agents like to their best extent
- 4:08:54like it's like you know um
- 4:08:57we use this thing called terminal bench
- 4:08:58terminal bench is a benchmark for you
- 4:09:01know AI agents in terminal environments
- 4:09:02and that is something that we are we
- 4:09:04have a lot of like experience with a
- 4:09:06client and it's like a it's like one of
- 4:09:08the best ways to like really gauge like
- 4:09:10is your coding agent accomplishing in
- 4:09:13general the tasks that you're doing
- 4:09:15that you want to be doing with the HR
- 4:09:16model. So, there's a couple there's many
- 4:09:19different parts of a like evaluation
- 4:09:20suite. So, ideally you want that to run
- 4:09:24in
- 4:09:25you know [snorts] you you want that to
- 4:09:26run in individual sandboxes. Then you
- 4:09:29have like independent graders that like
- 4:09:31check everything like how many turns it
- 4:09:33took, how many tokens it took, how much
- 4:09:35latency it took, how many how many how
- 4:09:37many tool calls did it make, what was
- 4:09:39the trajectory looking like, and then
- 4:09:40grading all the results for things like
- 4:09:42hey did you not reward a hack, did you
- 4:09:43not cheat through the solution, did you
- 4:09:44not just like look up the solution
- 4:09:46online, things like that.
- 4:09:48So, we we we're basically like a lot of
- 4:09:51these data sets of evaluation problems
- 4:09:54have a set of tasks. So, when is a
- 4:09:56terminal bench 2.1 they have 89 tasks.
- 4:09:58So, what we do is like we spin up 89
- 4:10:00parallel containers where they all run
- 4:10:03online and they run in parallel in their
- 4:10:05sandbox and then they they each try to
- 4:10:07solve a problem. So, then they're all
- 4:10:10in parallel and independent of each
- 4:10:11other and then at the end we get get get
- 4:10:14a score. So, our score could like give
- 4:10:16us measures like is is a
- 4:10:18is this is the model performing well and
- 4:10:21what kind of failures it has, right? So,
- 4:10:22like you run the evaluation, you get a
- 4:10:24score. So, you get an aggregate score
- 4:10:25and this
- 4:10:26aggregate score could reveal a set of
- 4:10:28failures. Failures could be of any kind.
- 4:10:30Failures could be hey the
- 4:10:32read file tool is not working. Sometimes
- 4:10:34the inference is so slow that like you
- 4:10:36know the agent's working, it's doing,
- 4:10:37it's making progress, maybe it's running
- 4:10:39in circle, but it's so slow that it
- 4:10:40literally like runs out of time, it runs
- 4:10:42out of like 45 minutes or whatever many
- 4:10:44that were given for the task and just
- 4:10:45like stops working. So,
- 4:10:47once you do that, once you understand
- 4:10:49hey this is what we ran, this is what
- 4:10:50the failures we got, we bucket the
- 4:10:52failures in different categories. So,
- 4:10:54like 10 failures of you know this kind
- 4:10:56of tool got 20 failures of like say
- 4:10:58inference not working or something else,
- 4:10:59right? And then we once we bucket the
- 4:11:01failures into different categories where
- 4:11:02like we try to make an improvement to
- 4:11:04the harness because like
- 4:11:05what if there's like something wrong
- 4:11:07with the harness that we could change
- 4:11:08such that the model score would improve,
- 4:11:10right? And on and on and on and you do
- 4:11:12this iteratively, it takes a while. It's
- 4:11:13just like you know hill climbing is like
- 4:11:15a very
- 4:11:16kind of process and you know usually
- 4:11:19usually we took we used to take a lot of
- 4:11:21time trying to do this but it was a
- 4:11:23great way for us to like learn how to
- 4:11:25improve models. So
- 4:11:26[snorts] one way to hill climb on
- 4:11:29else is that like
- 4:11:31and you know we we could have a thing
- 4:11:32where you know we started doing this
- 4:11:35thing where we could just like
- 4:11:37what if we could have like a long
- 4:11:39running agent that runs for a while and
- 4:11:40automatically improves the model score
- 4:11:42and that's what we did. So we ran GPT
- 4:11:455.6 soul that led the work and that was
- 4:11:47like the main agent and then
- 4:11:49K K 3 was just running so what it was
- 4:11:51doing is that like GPT 5.6 was like hey
- 4:11:54run these
- 4:11:55get these scores let me look at the
- 4:11:57failures let me look at where it where
- 4:11:59it went wrong and then I'll make changes
- 4:12:01to client's harness and then I'll run
- 4:12:03these again and then I'll keep trying.
- 4:12:05So again you can see how like very
- 4:12:07quickly this can like go after rails
- 4:12:09because this is supposed to be like the
- 4:12:10running in evaluation itself takes like
- 4:12:12an hour then you have to read the
- 4:12:14feedback then you have to like see if
- 4:12:16you're properly interpreting the
- 4:12:17feedback to make reasonable changes to
- 4:12:19the harness there are like not a word
- 4:12:20hacky they're just like actual genuine
- 4:12:22improvements and then run the again. So
- 4:12:24we ran that for 17 hours and
- 4:12:27when we ran that for 17 hours there was
- 4:12:29like a legitimate hell climbing and what
- 4:12:31we found that was very interesting this
- 4:12:32happened with a very
- 4:12:34detailed prompt that accomplished this
- 4:12:36is that like we had baseline scores
- 4:12:39percent and then
- 4:12:41you know we we found issues like loop
- 4:12:43detection loop detection is when a
- 4:12:45harness just runs in circles around the
- 4:12:46problem right we found like background
- 4:12:49process grinding where background
- 4:12:50processes would run too long
- 4:12:52we found some file index issues and
- 4:12:54other things as well I mean when I say
- 4:12:56we what I've actually made is GPT 5.6
- 4:12:58all right so that found those issues
- 4:13:00that found and bucketed those issues is
- 4:13:02like hey how about I make those changes
- 4:13:04in the harness and I run the
- 4:13:06and iteratively built the hill climbing
- 4:13:08process completely autonomously on its
- 4:13:10own without my interference and 17 hours
- 4:13:12later when I come back to it it's like
- 4:13:14it has improved like the harness's
- 4:13:15performance in so many ways.
- 4:13:17So see what does that mean what does
- 4:13:19that mean? It just it just basically
- 4:13:21means that like
- 4:13:22we are at a point where the experience
- 4:13:25could make the harness better. That like
- 4:13:27there's like a two-level learning loop
- 4:13:28where the harness does some action,
- 4:13:31interacts with the environment,
- 4:13:32understands whether or not it did the
- 4:13:34right thing or the wrong thing, and gets
- 4:13:36some reward and and like almost and then
- 4:13:38has there's like a meta agent that's
- 4:13:40observing the main harness and finding
- 4:13:43the changes to it and just like
- 4:13:45incorporating those changes and
- 4:13:46improving. Now, it's I think in my
- 4:13:48personal opinion, it's much harder to do
- 4:13:49in real time because if you do it in
- 4:13:51real time, you overindex on solving
- 4:13:53certain problems or you're just like too
- 4:13:56curtailed to a specific thing. But I do
- 4:13:59think that this is something that works
- 4:14:00like on a broader scale of like working
- 4:14:02on a wide range of problems.
- 4:14:03[snorts]
- 4:14:04And you want to be able to like you want
- 4:14:06you want to be able to justify the
- 4:14:08changes you're making on a broader
- 4:14:10applicability so that the changes you
- 4:14:12make here will apply to a broader range
- 4:14:14of problems as well. So,
- 4:14:17what is that mean?
- 4:14:19What am I trying to get at here?
- 4:14:21The point I'm trying to get at is that
- 4:14:23the models are so good right now that
- 4:14:24they can if they can improve the
- 4:14:25harness, they can improve a lot of other
- 4:14:27things. So then what else can they
- 4:14:28improve?
- 4:14:29So this is this is a this is a post
- 4:14:32primarily from OpenAI where GPT-5.6
- 4:14:35autonomously re-wrote and optimized its
- 4:14:37own production kernels and those
- 4:14:39production kernel improvements removed
- 4:14:41the end-to-end serving costs by 20%.
- 4:14:44You know, it also had 15% higher token
- 4:14:47generation efficiency and this was like
- 4:14:49completely done by 5.6 on itself, right?
- 4:14:53So what that means is that like
- 4:14:54inference is becoming much cheaper and
- 4:14:56that means that like if on if if for a
- 4:14:59company like OpenAI, if you're having
- 4:15:0120% lower lower serving cost for a
- 4:15:03frontier model, that could be a billions
- 4:15:06of dollars of improvement in price and
- 4:15:07you could you could do that as well for
- 4:15:09your own models.
- 4:15:11Another interesting thing is that like
- 4:15:13when people have kernel writing
- 4:15:15competitions, so Core Auto is a company
- 4:15:16which has a kernel writing competitions
- 4:15:19and you'll notice that like they they
- 4:15:21mentioned that like
- 4:15:22the person who won the kernel writing
- 4:15:24competition, they explicitly said that
- 4:15:26I've never handwritten a GPU operator.
- 4:15:28It's all
- 4:15:29and agent driven.
- 4:15:31So notice that like how like the best
- 4:15:34inference improvements, kernel
- 4:15:35improvements are at this point
- 4:15:36completely exclusively done by a swarm
- 4:15:38of agents or generally just like agents
- 4:15:41doing some magic to you know magically
- 4:15:43improve like the kernel kernel code and
- 4:15:45how faster inference, cheaper inference
- 4:15:47um
- 4:15:48yeah.
- 4:15:49So basically like the improvement
- 4:15:51surface that you think of is not just
- 4:15:54like the harness improvement. It's
- 4:15:55literally the whole surface stack,
- 4:15:57right? So the whole stack from like
- 4:15:59evaluation to harness improvements to
- 4:16:01training data to post training to world
- 4:16:03interaction to inference. Like literally
- 4:16:05the entire
- 4:16:06part of the pipeline. Like all the way
- 4:16:08from like electricity going through your
- 4:16:11machine to like making an error request,
- 4:16:14going through the model, models doing
- 4:16:15inference and making a response and
- 4:16:18getting back to you. Like every single
- 4:16:19part of the pipeline we can optimize it.
- 4:16:21So the whole RSI loop has like um
- 4:16:23there's a couple of different parts. So
- 4:16:25it's like pre-training
- 4:16:27um Nimo training examples of like you
- 4:16:28know AI curated pre-training data that
- 4:16:31was good. Um there's like post training
- 4:16:32examples which are
- 4:16:35you know where where there's like
- 4:16:36constitutional AI, examples of like post
- 4:16:38training improvements then inference. I
- 4:16:41gave you the example of GPT 5.6. So for
- 4:16:44and generation I gave you our own
- 4:16:45example of how we improved ourselves on
- 4:16:47our own
- 4:16:49Um in general what you want to think of
- 4:16:51is that like anytime a model is trying
- 4:16:53to do something and it's like not able
- 4:16:54to do it or not able to accomplish it.
- 4:16:56So for example like imagine you're
- 4:16:58having a long conversation where you ask
- 4:17:00the model something obvious. So like
- 4:17:01it's like
- 4:17:02or something that you think is not
- 4:17:05necessarily obvious but it's like
- 4:17:07it's hard but you would assume that the
- 4:17:08model could potentially do it. So to
- 4:17:10give an example, let's say you want to
- 4:17:12make add a new column to a database and
- 4:17:15then fill that column, right?
- 4:17:17And then put a cache in front of it. So
- 4:17:19what happens is that it does the whole
- 4:17:21thing but for some reason the way it
- 4:17:22pushes the cache in front of the whole
- 4:17:24database like it kind of doesn't do it
- 4:17:25right. So, at that point you're like, oh
- 4:17:27no,
- 4:17:28set this cache differently, set this as
- 4:17:29the TTL, and then you give like a
- 4:17:31correction signal, and then that kind of
- 4:17:33like fixes your problem and you kind of
- 4:17:35go about your day. But
- 4:17:36the way I want to think of it is that I
- 4:17:38want to think of like every single time
- 4:17:40like a human correction to the digital
- 4:17:42landscape is a part of a delta that
- 4:17:46needs to be covered. And what that means
- 4:17:49is that like every single time you're
- 4:17:51finding like the edge of model
- 4:17:52capabilities of like how far it can push
- 4:17:54for inference, how much it can push for
- 4:17:55this, how much it can push for
- 4:17:58you know, solving your problems,
- 4:18:00you want to find that delta and you want
- 4:18:02to like push that frontier a little bit
- 4:18:04further, a little bit further. And
- 4:18:06that's what post training does, right?
- 4:18:07So, post training is is this is in that
- 4:18:09post training. We find these like really
- 4:18:12interesting, really hard, long horizon
- 4:18:14problems for for coding agents or for
- 4:18:16other kinds of problems, and then just
- 4:18:17like slowly hill climb the model models
- 4:18:20on on those kind of problems. So,
- 4:18:22anything that has a verifiable loop
- 4:18:24can be optimized. But
- 4:18:26is this is not just limited to software.
- 4:18:29Like literally like the physical world
- 4:18:30itself could have a verifiable loop.
- 4:18:33So, to give an example, like let me let
- 4:18:35me give you a very funny kind of bizarre
- 4:18:37example where like it's like you
- 4:18:39you look at this and
- 4:18:42you
- 4:18:44um look at this like robot and it's it's
- 4:18:47typing something and it's like really
- 4:18:49really like interesting what the robot
- 4:18:51is typing.
- 4:19:01So, notice how like notice how the robot
- 4:19:03is itself typing like how it wants to
- 4:19:05like solve robotics. And imagine if the
- 4:19:08robot itself could like interpret the
- 4:19:10results of of like the answers and just
- 4:19:13like keep improving, then that means
- 4:19:14that like this is you know, basically
- 4:19:16like an ASI robot. Another example that
- 4:19:18I could give you from my
- 4:19:20personal life is that like yeah, I have
- 4:19:23this
- 4:19:24chip that has the ability to see and has
- 4:19:26a speaker and has a microphone inside of
- 4:19:29it. So I was trying to set up this uh
- 4:19:31with my machine and it was
- 4:19:34um a gift that I had for my girlfriend
- 4:19:36and what I found very interesting about
- 4:19:39this is that as I was trying to set it
- 4:19:40up it was very slow, it was slow to
- 4:19:42respond, it had higher latency, it had
- 4:19:44an agent in enough itself, right? So
- 4:19:46what I did was I connected the step two
- 4:19:48to my um computer
- 4:19:50and I lit um
- 4:19:53I lit um coding agents run on this
- 4:19:55hardware and basically run on this
- 4:19:58hardware
- 4:19:59automatically make tests to it. So like
- 4:20:02my computer would talk to my hardware,
- 4:20:05give it some tests like hey do this, do
- 4:20:07that and just like what do you see right
- 4:20:08now? So
- 4:20:10then it would notice the end-to-end
- 4:20:11latency of like the inference response
- 4:20:13of like how fast uh my hardware is
- 4:20:15responding and then um what I found is
- 4:20:18that like it was able to do the
- 4:20:20breakdown of like every single case and
- 4:20:22scenario where there was like higher
- 4:20:24latency
- 4:20:25um
- 4:20:26slow response or or just the algorithm
- 4:20:28wasn't right and I could then set up a
- 4:20:31recursive loop where
- 4:20:33both the firmware could be updated, the
- 4:20:36um the tool calls, the inference model
- 4:20:38used, um the speech-to-text model used,
- 4:20:41all of those could be improved
- 4:20:42automatically with like an open router
- 4:20:44key which I'd given to a coding agent
- 4:20:46and then overnight with self-running
- 4:20:47experiments with the hardware uh my um
- 4:20:51this tiny little piece of beautiful
- 4:20:52hardware that I have was like deeply
- 4:20:54optimized. So it was if this also like
- 4:20:56even hardware could be a part of the RSI
- 4:20:58loop. So what happens when you connect
- 4:21:00these loops, right? That's the next
- 4:21:01question. Like you could have cheaper
- 4:21:03inference, you could have like more
- 4:21:04experiments budget, you have like you
- 4:21:06know better data and better models. So
- 4:21:09is this single rarity?
- 4:21:11So very question to ask and I think that
- 4:21:12the part of the parts of the loop are
- 4:21:13already right? Like we're we we are
- 4:21:16really we really are very close to parts
- 4:21:18of the RSI loop, right?
- 4:21:20And what do we do? Like what do we do
- 4:21:22about this and it's like how do we think
- 4:21:24of that
- 4:21:25in terms of like our career and how do
- 4:21:27you think of that in terms of on know
- 4:21:28our lives and what do we want to do
- 4:21:29next. So,
- 4:21:30here's my advice to everyone who is
- 4:21:32watching this like, please please please
- 4:21:34please move closer to the
- 4:21:36right. So, it's like
- 4:21:37right now like people are working on all
- 4:21:39these like different kinds of software
- 4:21:41which is like you know, AI agents for
- 4:21:42you know, construction agents for this,
- 4:21:44agents for that and it's like I think
- 4:21:45that you're
- 4:21:47you know,
- 4:21:48well the way I see those things is that
- 4:21:50like I think of those as like you're
- 4:21:51missing the forest for the for the
- 4:21:52trees. You really are and it's like you
- 4:21:55have this like
- 4:21:56very interesting godlike technology that
- 4:21:59can change the course of humanity and
- 4:22:01then
- 4:22:02we're not ambitious enough to pursue the
- 4:22:03best problems. We really are not and
- 4:22:05what we're just like
- 4:22:07I think that it's not just like
- 4:22:09the problem isn't just that we're not
- 4:22:10ambitious and we're also like what we're
- 4:22:12pursuing
- 4:22:13a lot of those things are just like one
- 4:22:15model release away from like being
- 4:22:17completely like wiped out, right? So
- 4:22:18like
- 4:22:19it's also like it's also like
- 4:22:21it may not be that intellectually
- 4:22:22stimulating because like all you're
- 4:22:24essentially all we're essentially doing
- 4:22:25is like sitting all day and it's like
- 4:22:26hey cloud do this for me, hey cloud do
- 4:22:27this for me and I don't know if that's
- 4:22:29like I don't know if that's like the
- 4:22:30most intellectually stimulating or like
- 4:22:32or like um thoughtful use of a person's
- 4:22:35time. Like maybe maybe you know, we're
- 4:22:36supposed to be solving harder in more
- 4:22:38more frontier problems and I think that
- 4:22:40like it's like the ceiling is like our
- 4:22:42ambition. So,
- 4:22:43instead of like working on these tiny
- 4:22:44things my suggestion to people is to
- 4:22:46like you know,
- 4:22:48do things that of course like
- 4:22:50goes without saying but you should do
- 4:22:51things you believe in and you know, I'm
- 4:22:52just a guy on the internet but do things
- 4:22:55that help you move closer to the R S A
- 4:22:57loop, right? So like anything that's
- 4:22:59just like the end-to-end pipeline of
- 4:23:00like
- 4:23:01hey this is where like the models are
- 4:23:04lacking like it's like whether that's
- 4:23:06inference, whether that's post training,
- 4:23:07whether that's literally like physical
- 4:23:08interaction, whether that could be you
- 4:23:11know, send sending end-to-end pipelines
- 4:23:13of truly any kind you know, whether
- 4:23:15that's hardware problems, whether that's
- 4:23:16um networking problems, whether that's
- 4:23:18um building a sense of community, things
- 4:23:20that just you know, um things that just
- 4:23:22like help
- 4:23:23um you as an engineer get closer to the
- 4:23:27benefits of the improvements um in in um
- 4:23:32um in in the model. So, this is a very
- 4:23:34interesting slide from such a topic. So,
- 4:23:38they mentioned that like
- 4:23:40you want to
- 4:23:41have like
- 4:23:43this you're in this space where it's
- 4:23:44like, you know, you only you only have
- 4:23:45like one to five years left to
- 4:23:46contribute to the world. What do you do,
- 4:23:48right? So, it's like
- 4:23:50they they great suggestions where like
- 4:23:52prefer ambitious, targeted and fast
- 4:23:54collaborations, right?
- 4:23:55And, you know, in general like the time
- 4:23:58scale, like all the timelines that you
- 4:23:59give to yourself like they should be
- 4:24:00just like completely squeezed out.
- 4:24:02Should be completely squeezed out
- 4:24:03because it's like a lot of times and you
- 4:24:04also have to think through that like if
- 4:24:06if the model if the problem that you're
- 4:24:07working on is too ambitious, like it's
- 4:24:09like is it probably
- 4:24:11is it probably better to wait for just
- 4:24:12like the next next model to come out or
- 4:24:15not? Like that's that's something you
- 4:24:16also have to wait through, right?
- 4:24:19And you you ideally want to work on
- 4:24:22problems that benefit from scale and you
- 4:24:24know, you also at the same time want to
- 4:24:25avoid projects that can be solved by
- 4:24:26scale alone because then, you know, if
- 4:24:28even if you like it's like if you're
- 4:24:30working on a problem and then the next
- 4:24:31model release comes out it solves the
- 4:24:32problem. So, that's not that's probably
- 4:24:33not a good thing to do. So, it's very
- 4:24:35like you have to like think
- 4:24:36philosophically of like what is the
- 4:24:39endeavor that benefits greatly from
- 4:24:41model improvements but like isn't also
- 4:24:43completely wiped out from it and then
- 4:24:45ideally like it's like
- 4:24:46if the if the endeavor could also be a
- 4:24:49part of the
- 4:24:51like the improvement pipeline,
- 4:24:53that could be amazing. So, you see a lot
- 4:24:55of companies raising on say
- 4:24:58making data pipelines and there's a lot
- 4:25:00of money there. There's a
- 4:25:02a lot of companies working on physical
- 4:25:03hardware and stuff. So, I think if
- 4:25:05you're a software engineer watching
- 4:25:06these it's like I think that if you're
- 4:25:08if you feel like you're like
- 4:25:09here the delta of your intellect and
- 4:25:12ability in your day-to-day it's like not
- 4:25:14contributing
- 4:25:15as in like you have the intellect, you
- 4:25:17can make a difference but like if is
- 4:25:19just like one shotting all those
- 4:25:21problems like in your day-to-day and
- 4:25:22you're just like kind of like becoming
- 4:25:24which worth to think through take a step
- 4:25:25back and think through it's like okay,
- 4:25:27where are the problems where where
- 4:25:29you're actually truly contributing and
- 4:25:31that they will stand the test of time
- 4:25:32and and those kind of reflection and
- 4:25:34realignment now would I think like have
- 4:25:37like
- 4:25:37very interesting consequences for you in
- 4:25:40the long term.
- 4:25:42So yeah, so thank you so much, you know,
- 4:25:44thank you so much for your time. Like I
- 4:25:46think that
- 4:25:47I think that if you're watching this and
- 4:25:49if you've gone this far, I think that
- 4:25:50you're definitely contemplating like one
- 4:25:53of the most interesting improvements in
- 4:25:54the course of humanity and you know,
- 4:25:57there's many philosophical arguments
- 4:25:58here. Some people might say, hey, you
- 4:26:00know, I I think I maybe I'm off on the
- 4:26:04timelines or maybe maybe I'm I'm just
- 4:26:05being too pessimistic or maybe I'm being
- 4:26:07too optimistic, but I would love to hear
- 4:26:09all those thoughts. So this is you know,
- 4:26:11my QR code. Please feel free to scan it
- 4:26:13and let me know all your thoughts. Thank
- 4:26:15you so much. Very kind of you to give me
- 4:26:16your time.
- 4:26:17>> Hi everyone. I am Tanmay.
- 4:26:20Today joining me is Dolly and we will be
- 4:26:23presenting the missing undo button for
- 4:26:26coding agents.
- 4:26:31So
- 4:26:32imagine a coding agent working on a
- 4:26:35repository issue.
- 4:26:37It applies patch,
- 4:26:39run the tests and everything passes. It
- 4:26:42even creates a release tag,
- 4:26:45but then a downstream integration fails.
- 4:26:49Now here is the question.
- 4:26:51Can the agent actually get everything
- 4:26:54back to the state it was in before?
- 4:26:59So Git can tell us what changed in the
- 4:27:02repository, but what about
- 4:27:04changes outside Git such as external API
- 4:27:08calls or runtime configurations or
- 4:27:11notifications?
- 4:27:13And simply retrying an operation does
- 4:27:16not necessarily undo what already
- 4:27:19happened. So we wanted to understand not
- 4:27:22just
- 4:27:23whether agents can complete task, but
- 4:27:26whether they can recover when something
- 4:27:29goes
- 4:27:30wrong.
- 4:27:37This brings us to our first research
- 4:27:39work undo bench.
- 4:27:42We wanted to investigate whether an
- 4:27:45agent's ability to complete a task also
- 4:27:48translates into its ability to recover
- 4:27:51from failure.
- 4:27:53We evaluated agents across 2,880
- 4:27:57paired trials
- 4:27:59comparing normal execution with
- 4:28:02execution under an injected failure.
- 4:28:05So under normal conditions, the agent
- 4:28:08achieved an 84% success rate,
- 4:28:11but when we introduced failures, only
- 4:28:1447%
- 4:28:15of the normally successful trials
- 4:28:18recovered successfully.
- 4:28:20Notice that these two percentages though
- 4:28:23have different denominators,
- 4:28:25but yeah, so even when an agent
- 4:28:29can complete a task successfully,
- 4:28:32recovery remains a significant
- 4:28:34challenge.
- 4:28:38Now let's look at a specific failure
- 4:28:41scenario. Imagine a coding agent sending
- 4:28:45a request to an external system to
- 4:28:48create a ticket.
- 4:28:50The request has been sent and the agent
- 4:28:52is waiting for a response.
- 4:28:57Next, the server receives the request
- 4:28:59and successfully creates ticket number
- 4:29:02781,
- 4:29:04but the acknowledgment gets lost on its
- 4:29:07way to the agent.
- 4:29:10From the agent's perspective, the
- 4:29:12request has timed out,
- 4:29:14but the agent does not know that the
- 4:29:16ticket has already been created. A
- 4:29:19timeout tells us that the response
- 4:29:22didn't arrive, not that the operation
- 4:29:25failed.
- 4:29:28So the agent tries the request without
- 4:29:31checking whether the
- 4:29:33whether the first operation committed.
- 4:29:37And now we have two tickets even though
- 4:29:40the agent intended to create only one.
- 4:29:43So in undo bench naive retries produced
- 4:29:48duplicate effects in 54% of these kind
- 4:29:51of trials. That's why we need to
- 4:29:54distinguish a failed response from a
- 4:29:58failed operations.
- 4:30:05Now
- 4:30:07the failure face also changes the
- 4:30:10answer.
- 4:30:11What do I mean is that recovery also
- 4:30:14depends on
- 4:30:16when the failure happens.
- 4:30:18Before the mutation retrying is
- 4:30:21generally safe because nothing has
- 4:30:23changed. But during the mutation the
- 4:30:26system has may be partially modified. So
- 4:30:29we need to inspect its state.
- 4:30:33And after the mutation the operation may
- 4:30:35have committed even if the
- 4:30:37acknowledgement was lost.
- 4:30:39As we just saw retrying could create
- 4:30:42duplicates.
- 4:30:43So we need different recovery strategies
- 4:30:47depending on the failure phase.
- 4:30:57Now
- 4:30:59why can't we just use git revert?
- 4:31:03So git can restore repository changes
- 4:31:07but coding agents also modify their
- 4:31:11runtime state including tool registries,
- 4:31:14middleware and configurations. They can
- 4:31:17also trigger external effects such as
- 4:31:20deployment or database rights.
- 4:31:23So git revert does not reverse all these
- 4:31:27changes. The recovery mechanism
- 4:31:30has to match the type of effect the
- 4:31:33agent created.
- 4:31:39And this becomes particularly important
- 4:31:43when coding agents start modifying their
- 4:31:46own execution environment or what we
- 4:31:49call the agent hardness.
- 4:31:51This includes prompts, tools,
- 4:31:54middleware, event listeners, resources,
- 4:31:57and routing configurations.
- 4:31:59Such changes may improve the agents
- 4:32:02performance,
- 4:32:04but they can also make recovery more
- 4:32:06difficult.
- 4:32:07In other words, an agent can become more
- 4:32:10capable
- 4:32:12while becoming less recoverable.
- 4:32:20And that's why
- 4:32:22uh that's why undo needs more than an
- 4:32:25inverse command.
- 4:32:27Uh now uh like undoing a change is not
- 4:32:31always as simple as reversing a command.
- 4:32:34Suppose an agent
- 4:32:36changes a timeout from 30 to 60 to undo
- 4:32:40that change, it needs to know
- 4:32:43that the original value was 30.
- 4:32:46That's why Evo undo captures a pre-state
- 4:32:49witness before applying the mutation.
- 4:32:53And that witness provides the
- 4:32:54information to restore
- 4:32:57the original state and later verify that
- 4:33:02uh verify that recovery actually worked.
- 4:33:06Now
- 4:33:08question comes
- 4:33:10how often uh do improved hardnesses
- 4:33:13fail?
- 4:33:14So
- 4:33:15in Evo undo, we evaluated 600
- 4:33:18self-evolution tasks.
- 4:33:20Out of those, 478 produced capability
- 4:33:24improvements, but 197 of those
- 4:33:27improvements had recovery defects.
- 4:33:31That is around 42% of the capability
- 4:33:34positive cases.
- 4:33:36So
- 4:33:37here key takeaway is that a mutation can
- 4:33:40improve an agent's capability and still
- 4:33:43leave the system unable to recover
- 4:33:47reliably.
- 4:33:51Now
- 4:33:53now we can say that why please fix the
- 4:33:56undo did not work. So naturally we
- 4:33:59wanted to use prompting and wanted to
- 4:34:02tell agent that hey could you simply
- 4:34:04repair all these recovery failures.
- 4:34:07So we tested four verified guided repair
- 4:34:11strategies but under the original
- 4:34:14recovery language none recovered the 197
- 4:34:18naturally occurring failures. The
- 4:34:21problem was not necessarily
- 4:34:24the agents reasoning in some cases the
- 4:34:27agent lacked
- 4:34:29precise information about the original
- 4:34:31state in
- 4:34:34in others the recovery language could
- 4:34:37not express the operation needed to
- 4:34:40reverse the chain.
- 4:34:42So we investigated these two limitations
- 4:34:46separately.
- 4:34:50So
- 4:34:51two bottlenecks
- 4:34:53first is state grounding and another one
- 4:34:55is recovery expressivity.
- 4:34:57The
- 4:34:58when we provided like precise
- 4:35:01information about what had changed and
- 4:35:03where
- 4:35:04then recovery improved from 0 to 38 out
- 4:35:08of 48 cases.
- 4:35:10And the second is recovery expressivity.
- 4:35:13So when we introduced richer undo
- 4:35:16operation recovery improved
- 4:35:19significantly from 0 to 142 out of 143
- 4:35:23cases. So these results come from the
- 4:35:26two different experimental
- 4:35:28subgroups but together they show why
- 4:35:31recovery needs both accurate state
- 4:35:34information and ability to perform
- 4:35:37the required undo operation.
- 4:35:46Now question comes to our mind that can
- 4:35:49better undo mechanism rescue failed
- 4:35:51mutations.
- 4:35:53So,
- 4:35:55what happens like when we give the agent
- 4:35:57better recovery capabilities. So, with
- 4:36:00the original recovery language none of
- 4:36:03the 197 recovery defective mutations
- 4:36:06could be restored.
- 4:36:08But after extending the recovery
- 4:36:10language we successfully recovered 180
- 4:36:13out of 197 cases. That is around 92%. As
- 4:36:18a result the number of admissible
- 4:36:21improvement
- 4:36:22increased from 281 to 461
- 4:36:26out of 600 tasks.
- 4:36:28That is a 30 percentage point
- 4:36:30improvement and
- 4:36:33that brings us to the next question. How
- 4:36:36do we build these recovery checks into
- 4:36:39the agent's evolution process?
- 4:36:42I will now hand it over to Dolly
- 4:36:45to walk through the Evo undo algorithm.
- 4:36:51>> All right. Thanks Tanmay.
- 4:36:53So, let's get into the working on how
- 4:36:56Evo undo actually works. So, first we
- 4:36:58will start with the agent
- 4:37:01first
- 4:37:03the agent will start
- 4:37:04the agent processes the mutation on
- 4:37:07which state it is currently in along
- 4:37:09with the effect contract and the
- 4:37:11recovery plan. And before applying the
- 4:37:13mutation
- 4:37:15we capture the state witness
- 4:37:18representing the original state.
- 4:37:21And even before that the then a testing
- 4:37:24in the testing environment agent will
- 4:37:26apply the mutation and execute the
- 4:37:28recovery plan and an independent
- 4:37:31verifier will also verify the original
- 4:37:34if verify to check if the original state
- 4:37:36has been restored or not.
- 4:37:38If the mutation improves the capability
- 4:37:41based on the user defined criteria and
- 4:37:44passes the recovery
- 4:37:46check the agents admit the changes.
- 4:37:49Otherwise
- 4:37:51well,
- 4:37:53the agent attempt to repair or the
- 4:37:55verifier itself rejects the mutation
- 4:37:57altogether. This way the recoverability
- 4:38:00becomes a part of the admission decision
- 4:38:01rather than something that we strategize
- 4:38:04or even plan once we are
- 4:38:07deployed the deployed the AI agent.
- 4:38:11Okay, so with next
- 4:38:16next we will walk through as as
- 4:38:20given an example let let me walk through
- 4:38:22how the
- 4:38:23how that example of a timeout changing
- 4:38:26from 30 to 60 would look like in action.
- 4:38:29So let's take
- 4:38:31let's start with the original state the
- 4:38:33simple example that simple example of
- 4:38:36our agent having the time value of 30
- 4:38:39before changing anything the evil undo
- 4:38:41would first read the original value.
- 4:38:46And then it will save the it will
- 4:38:48capture the original value as a state
- 4:38:50witness. We know exact we so we know
- 4:38:53exactly what need to be restored when
- 4:38:56when
- 4:38:56when if we have to restore it we would
- 4:38:59know what
- 4:39:00what need to be restored or which state
- 4:39:01would be the true state to return back
- 4:39:04to.
- 4:39:05Now the agent so next come the mutation
- 4:39:08now the agent changes the timeout from
- 4:39:1030 to 60 as we wanted to do
- 4:39:12and if recovery is required
- 4:39:16if recovery is required the undo uses
- 4:39:18the saved witness to restore the timeout
- 4:39:21to 60.
- 4:39:22Now independently the verifier finally
- 4:39:25an independent verifier would also
- 4:39:26confirm that the
- 4:39:28recovered value matches the original
- 4:39:30value so the recovery is in considered
- 4:39:32successful simply because
- 4:39:34an undo command was run but also the
- 4:39:37verifier would verify the resulting
- 4:39:39state to make sure that we are in fact
- 4:39:42doing the
- 4:39:44recovery those in the state we wanted to
- 4:39:47be.
- 4:39:49Well now comes the bigger question so so
- 4:39:53we have this capability of recovery in
- 4:39:55our arsenal. Should we be using it for
- 4:39:57uniformly and applying the recovery
- 4:39:59mechanism to all the failures?
- 4:40:02Well, of course not.
- 4:40:04Um not every failures can be recovered
- 4:40:06the same way. If nothing has changed
- 4:40:08then the then simply if if nothing has
- 4:40:11changed in the in the system, we would
- 4:40:13simply wanted to
- 4:40:14um
- 4:40:16recover
- 4:40:17we we would simply wanted to safely
- 4:40:19retry.
- 4:40:21We would want the agent to simply retry
- 4:40:23if and try to achieve the state that we
- 4:40:25want to be.
- 4:40:27However, if we have the original state
- 4:40:30uh once we have the original state, we
- 4:40:31can restore it. If the external action
- 4:40:34has already been committed and we need
- 4:40:37uh
- 4:40:39it's then when we would need a
- 4:40:40compensation mechanism compensation
- 4:40:43action to
- 4:40:44compensating action to reverse its
- 4:40:46effect. And if we cannot recover
- 4:40:49the recovery
- 4:40:51and we cannot if if
- 4:40:53and if we cannot the cannot verify that
- 4:40:55recovery is safe or the verify have
- 4:40:58rejected the recovery
- 4:40:59then the agent should stop and escalate.
- 4:41:03The right response definitely depends on
- 4:41:04what actually happened and what we what
- 4:41:07can be safely reversed or not.
- 4:41:13That brings me
- 4:41:16to the almost the last slide which is
- 4:41:18about bringing everything together on
- 4:41:20our two paper and to bench and Evo and
- 4:41:22do.
- 4:41:23So the key takeaways here are first
- 4:41:26the undo bench shows the successful task
- 4:41:29execution.
- 4:41:31Uh
- 4:41:33the undo bench shows that this
- 4:41:34successful task execution doesn't
- 4:41:36guarantee doesn't always guarantee the
- 4:41:38successful recovery of it.
- 4:41:40Secondly, the Evo undo algorithm
- 4:41:43provides a structured process, capture
- 4:41:45the original state, apply the mutation,
- 4:41:46recover independently verify and
- 4:41:48restoration. So it the Evo undo
- 4:41:51algorithm does work in the in the
- 4:41:53fashion in the way we intended it
- 4:41:56And third, we should admit
- 4:41:58self-evolution changes only when the
- 4:42:00when both the capability and the
- 4:42:02recoverability requirements are met.
- 4:42:04So
- 4:42:06ultimately, as we give agents more
- 4:42:08autonomy, we should also give them more
- 4:42:11reliable ways to recover from their
- 4:42:12actions.
- 4:42:16All right.
- 4:42:17So finally, I hope you remember the
- 4:42:20agent from the beginning. It was looking
- 4:42:22for an undo button. Well, now it has
- 4:42:24one. We have started building this
- 4:42:26capability through our research project
- 4:42:28Evo Undo. And you can explore the world
- 4:42:30through the in in the GitHub repository.
- 4:42:33You can scan the
- 4:42:35QR code on the screen to get to the
- 4:42:37page. Our goal is to make the recovery
- 4:42:39of fundamental capability of AI agents
- 4:42:41so that we not only have
- 4:42:45the agent does not even execute does not
- 4:42:47only execute the action but also recover
- 4:42:49when something goes wrong as well. Thank
- 4:42:51you so much.
- 4:42:54>> Hey everyone, thanks for tuning in.
- 4:42:56Today I'm going to walk you through how
- 4:42:57we use AI agents at Duolingo to write
- 4:42:59and ship more than 200,000 lines of unit
- 4:43:01tests on iOS codebase.
- 4:43:02I'll share what we built and what we
- 4:43:04learned along the way.
- 4:43:05I'm Kush. I'm a staff engineer on the
- 4:43:06platform team here at Duolingo. My job
- 4:43:08is to build developer tooling and
- 4:43:09AI-assisted workflows for the 100 plus
- 4:43:10engineers who contribute to our client
- 4:43:11apps. Before Duolingo, I spent close to
- 4:43:13a decade at Stripe and Uber, working
- 4:43:15across a few teams there.
- 4:43:17Let's start with some background on the
- 4:43:18scale of mobile at Duolingo.
- 4:43:20We ship our apps to the App Store every
- 4:43:22week.
- 4:43:23We're also a very experiment-driven
- 4:43:25company. So at any given point of time,
- 4:43:26we have more than 600 AB tests running
- 4:43:28live.
- 4:43:29And our code bases are pretty mature.
- 4:43:30Both our iOS and Android apps have been
- 4:43:32around for more than 10 years at this
- 4:43:33point.
- 4:43:35Here's also the number of executable
- 4:43:36lines of app code in our iOS repository.
- 4:43:38So as you can see, back in 2019, we were
- 4:43:40at about 200,000 lines and today we're
- 4:43:42hit close to 2 million.
- 4:43:44And as AI-assisted coding picks up, you
- 4:43:45can see that curve starting to bend
- 4:43:46upward.
- 4:43:48And it's not just the amount of code
- 4:43:49that's changing, it's also who's writing
- 4:43:50it. So last year, about 200 people
- 4:43:52contributed to the iOS repository and
- 4:43:54this year we've already crossed to 50.
- 4:43:56And And lot of these folks aren't even
- 4:43:57iOS engineers. We're have backend
- 4:43:58engineers, Android engineers, even PMs
- 4:44:00and designers all shipping code to the
- 4:44:02iOS app.
- 4:44:03And that's mostly been unlocked by AI
- 4:44:04coding tools like Cursor and Clot Code.
- 4:44:07Um, another thing to note about a
- 4:44:08codebase is that it's a codebase in
- 4:44:09transition. So a few years ago, we
- 4:44:11standardized on the MVVM architecture
- 4:44:12pattern.
- 4:44:13Today, about half the code in our
- 4:44:14repository follows that pattern, and the
- 4:44:16other half is still legacy code that
- 4:44:17we're in the process of migrating.
- 4:44:19Since the MVVM half is the easiest to
- 4:44:20test, that's where we focused on this
- 4:44:22project.
- 4:44:24Here's where we started back in March of
- 4:44:25this year.
- 4:44:26We were at overall unit test coverage of
- 4:44:2811%.
- 4:44:29The MVVM layers were a bit lower.
- 4:44:31Repositories were at 11%, view models at
- 4:44:33nine, data sources at five, and combined
- 4:44:35we are about 9% for the MVVM slice of
- 4:44:37the app.
- 4:44:39So why does this matter now more than it
- 4:44:41used to?
- 4:44:42As agents and humans write more code and
- 4:44:44write it faster, writing code isn't the
- 4:44:45hard part anymore, verifying it is.
- 4:44:47So when someone puts up a PR, how do you
- 4:44:49know that it's actually safe to ship?
- 4:44:50The bottleneck is really shifting from
- 4:44:51writing to verifying.
- 4:44:53And that's where unit tests come in.
- 4:44:55They make up the base of the testing
- 4:44:56pyramid. They're pretty cheap to write
- 4:44:57and fast to run, and they also act as
- 4:44:59guardrails for agents.
- 4:45:00So when an agent's change breaks a test
- 4:45:01in CI, it gets immediate feedback on
- 4:45:03what to fix.
- 4:45:04They also act as living documentation on
- 4:45:06what the code is supposed to do.
- 4:45:07And here's where we ended up eight
- 4:45:08months later. Overall unit test coverage
- 4:45:10went from 11% to 24%. Repositories went
- 4:45:12from 11 to 72, view models from nine to
- 4:45:1458, data sources from five to 41, and
- 4:45:17the combined MVVM slice of the app went
- 4:45:19from 9% to 59%.
- 4:45:21So what did we actually end up building?
- 4:45:23On paper, it's a pretty simple loop.
- 4:45:24First you identify the files that need
- 4:45:25tests, then you write those tests with
- 4:45:27an agent, you verify that the tests are
- 4:45:29good quality and they pass CI, then you
- 4:45:30resolve the PR, which means getting it
- 4:45:32reviewed by an engineer and merged.
- 4:45:34And finally, you iterate on the feedback
- 4:45:35so that the next batch of PRs is better
- 4:45:36than the last one.
- 4:45:38When production it's a little messier,
- 4:45:39this is what our system ended up looking
- 4:45:41like. I won't go through every box right
- 4:45:42now, but at a high level, it's a few
- 4:45:43temporal workflows that coordinate
- 4:45:45through shared state stored in S3.
- 4:45:47And every box you see here maps back to
- 4:45:48one of the five steps I just talked
- 4:45:49about.
- 4:45:51Here are the building blocks that we
- 4:45:52ended up using. Temporal handles
- 4:45:54workflow orchestration and execution.
- 4:45:56Clot Code acts as the coding agent. S3
- 4:45:58holds the shared state that keeps our
- 4:45:59workflows in sync.
- 4:46:01And we use GitHub for actions, labels,
- 4:46:02and PRs.
- 4:46:03Data dogs acts as a observability layer
- 4:46:05where we keep an eye on the health of
- 4:46:06our pipelines.
- 4:46:08If you haven't used Temporal before,
- 4:46:09it's basically a durable workflow
- 4:46:10execution engine.
- 4:46:12You write your workflows as normal
- 4:46:13Python code, and if a worker crashes,
- 4:46:14the workflow just picks up right where
- 4:46:15it left off.
- 4:46:17This fits agents really well, and agent
- 4:46:19session can run for seconds to hours,
- 4:46:20and it can call out to tools like
- 4:46:21GitHub, CI, S3, and Cloud.
- 4:46:24We also lean on a few built-in features
- 4:46:25of Temporal.
- 4:46:26There are schedules which are basically
- 4:46:28cron-like schedules for our jobs.
- 4:46:29There are composable workflows so that
- 4:46:30we can fan out work in parallel.
- 4:46:32Then there's first-class support for
- 4:46:33retries and timeouts and idempotency so
- 4:46:35that we never kick off duplicate runs of
- 4:46:36the same file.
- 4:46:39To split up this work, we split this
- 4:46:40project into four milestones. The first
- 4:46:42was local validation. Basically, is this
- 4:46:43project even viable? The second was a
- 4:46:45scheduled backfill workflow to write
- 4:46:47tests for existing code.
- 4:46:48The third was a PR lifecycle workflow to
- 4:46:50actually get those PRs merged, and the
- 4:46:52fourth was a label trigger so engineers
- 4:46:54can get tests for the new code that
- 4:46:55they're writing.
- 4:46:56Now we'll go through each of these in
- 4:46:57order.
- 4:46:58Before building anything real, we wanted
- 4:46:59to do the least amount of work possible
- 4:47:01that would tell us whether an agent
- 4:47:02could actually do this job well and do
- 4:47:03it at scale.
- 4:47:04We started with a very simple local
- 4:47:05Cloud Code setup
- 4:47:07where we had one Cloud Code command that
- 4:47:08acted as an orchestrator. It ran an
- 4:47:10Xcode build script to generate a code
- 4:47:12coverage report locally.
- 4:47:13Then it fed that report through a file
- 4:47:14selection script that we wrote,
- 4:47:16and then it handed each selected file
- 4:47:17off to sub agent two sub agents.
- 4:47:19One wrote the tests and the other one
- 4:47:21opened the PR.
- 4:47:22We ran this in a loop and generated a
- 4:47:23bunch of PRs.
- 4:47:25We basically ran this in two batches.
- 4:47:26The first batch had 17 PRs and we merged
- 4:47:28eight of them. The second at 40 PRs and
- 4:47:29we merged 19. So that's roughly a 50%
- 4:47:31merge rate. The goal here wasn't to
- 4:47:33merge a lot of PRs, it was to learn how
- 4:47:34the agents fail and to fix those paper
- 4:47:35cuts before productionizing anything.
- 4:47:38Here are some of the failure modes that
- 4:47:39we saw. Since we're a large legacy
- 4:47:40codebase with both XC tests and Swift
- 4:47:42testing, the agents would sometimes get
- 4:47:43confused and mix the two in one file. We
- 4:47:45fixed that by giving the agents clear
- 4:47:46rules on when to use each.
- 4:47:48We also had a giant user client
- 4:47:49singleton class. Some code passed around
- 4:47:51the concrete version of this class,
- 4:47:52which is really hard to mock, whereas
- 4:47:53others passed around a protocolized
- 4:47:54version.
- 4:47:55So we migrated the entire codebase over
- 4:47:56to the protocolized version and added a
- 4:47:58lint rule to enforce it.
- 4:47:59This made the whole host of files easier
- 4:48:01to test for agents and for humans.
- 4:48:04We also ran into six concurrency issues
- 4:48:06and agents writing timing based tests
- 4:48:07which are a big source of flakiness in
- 4:48:09CI. So we improved the state for both of
- 4:48:10these with better rules and examples in
- 4:48:12our testing skills.
- 4:48:14Now this brings us to milestone two.
- 4:48:15Since we were so far behind on code
- 4:48:17coverage, the biggest leverage that we
- 4:48:18saw was backfilling tests for code that
- 4:48:20was already merged in.
- 4:48:21That's where the scheduled backfill
- 4:48:22workflow comes in. This is a system
- 4:48:24design diagram of that workflow as it
- 4:48:25runs in production
- 4:48:26and it essentially does two jobs. The
- 4:48:28left half is to identify which is
- 4:48:29picking the files that need tests and
- 4:48:31the right half is right which is having
- 4:48:33the agents actually write those tests.
- 4:48:36Let's start with identify. Every time a
- 4:48:37PR merges to the main branch in the iOS
- 4:48:39repo, we kick off a GitHub action job
- 4:48:41that generates an X code coverage report
- 4:48:42and uploads it to S3.
- 4:48:44That becomes a source of truth for the
- 4:48:45latest code coverage.
- 4:48:46The backfill workflow then fetches that
- 4:48:47report and runs it through a selector
- 4:48:49script which picks the next set of files
- 4:48:50to test.
- 4:48:52Let's get a deeper a little deeper into
- 4:48:53the selector script.
- 4:48:54Its job is essentially to score every
- 4:48:56Swift file based on how confident PR
- 4:48:57that an agent can write a good test for
- 4:48:59it.
- 4:48:59The strongest signal is file type. So
- 4:49:01view models, repositories and data
- 4:49:03sources get a 2.5 X boost while legacy
- 4:49:05files get pushed down.
- 4:49:06We skipped UI code completely for now
- 4:49:08since snapshot tests are better fit
- 4:49:09there.
- 4:49:10And larger files also get a penalty. So
- 4:49:11we prioritize code that we're confident
- 4:49:13that we can test well.
- 4:49:15The next step is to actually write the
- 4:49:16tests.
- 4:49:17We kick off one child workflow per file
- 4:49:19all running in parallel.
- 4:49:21In each one of these runs, the agent
- 4:49:22clones the iOS repository, reads the
- 4:49:24testing skill that we keep in the repo
- 4:49:26which spells out how to write a good
- 4:49:27test.
- 4:49:28Then it writes the tests plus any mocks
- 4:49:29and fakes it needs and opens a PR with
- 4:49:31the generated test label so we can
- 4:49:32identify it easily later.
- 4:49:35These agents run on Duolingo's cloud
- 4:49:36coding agent which is a platform that
- 4:49:38also runs the nipple.
- 4:49:39The simplest way to think about it is as
- 4:49:40an abstraction on top of cloud code and
- 4:49:42codex running in the cloud.
- 4:49:43We can invoke it from Slack, Temporal or
- 4:49:44GitHub. It talks to a bunch of our
- 4:49:46internal tools for MCP and it logs a
- 4:49:48bunch of events so we can see what each
- 4:49:49agent run is doing and how much it
- 4:49:50costs.
- 4:49:52We started running these on Linux
- 4:49:53machines and then later moved them to
- 4:49:55Mac runners which will become important
- 4:49:56in a minute.
- 4:49:58Here's what a generated test PR looks
- 4:49:59like. Every PR description follows the
- 4:50:01same template and it says explicitly
- 4:50:03that the PR is AI generated.
- 4:50:04The reviewer has four options. They can
- 4:50:06approve it and PR automerges. They can
- 4:50:08push changes to it directly. They can
- 4:50:09reassign it to someone better suited or
- 4:50:11they can close it out with feedback and
- 4:50:12why it is in
- 4:50:14Now, once we had the backfill running in
- 4:50:16production, we had a slightly new
- 4:50:17problem.
- 4:50:17We suddenly had tens of tens to hundreds
- 4:50:19of open PRs that nobody could manage by
- 4:50:21hand.
- 4:50:22So how did we get all of these to a
- 4:50:22mergeable state and in front of the
- 4:50:24right reviewer?
- 4:50:26That's where the PR lifecycle workflow
- 4:50:27comes in. You can think of it as a state
- 4:50:28machine. It runs every couple of hours,
- 4:50:30looks through every open generated test
- 4:50:32PR and takes exactly one action on each.
- 4:50:35It covers the next two steps of our
- 4:50:36loop.
- 4:50:37Verify, which is does this PR pass CI
- 4:50:39and is it high quality? And resolve,
- 4:50:40which is getting a human to review it
- 4:50:42and merge it.
- 4:50:43The logic for this workflow is a simple
- 4:50:45decision tree. If there are any merge
- 4:50:46conflicts or the PR has been open for
- 4:50:47about three weeks, we close it out and
- 4:50:48mark it as stale.
- 4:50:49If CI is still running or a reviewer is
- 4:50:51already on it, we skip it. If CI is
- 4:50:52green and there's no reviewers, we find
- 4:50:54one and assign them. If CI is red, we
- 4:50:55try to fix it.
- 4:50:57And if the reviewer rejects the PR, the
- 4:50:58file goes into a 30-day cooldown so
- 4:51:00we're not spamming engineers with the
- 4:51:01same PR again and again.
- 4:51:03When CI fails, the first thing we check
- 4:51:05is which job failed. If it's pre-commit,
- 4:51:07a GitHub action applies the fix. It's
- 4:51:09cheap, it's deterministic, and there's
- 4:51:10no agent needed.
- 4:51:11If it's a real test failure, we kick off
- 4:51:12another agent that reads the CI logs and
- 4:51:14pushes a fix to the PR.
- 4:51:15We cap that whole loop at five attempts,
- 4:51:17past that we close the PR rather than
- 4:51:18trying to fix it forever.
- 4:51:20Once CI is green, we need to find the
- 4:51:22right reviewer. What we do is we run git
- 4:51:24blame on the source file being tested,
- 4:51:25find the top contributor and assign it
- 4:51:27to them and turn on automerge.
- 4:51:28If they approve it, the PR gets merged.
- 4:51:30If they reject it, the file goes into
- 4:51:31the 30-day cooldown which I previously
- 4:51:32mentioned.
- 4:51:33This brings us to milestone four.
- 4:51:35Once all of this was backfilling tests
- 4:51:37for existing code, we wanted to make it
- 4:51:38just as easy to get tests for the new
- 4:51:39code engineers were writing.
- 4:51:41Now any engineer can add a right test
- 4:51:43label to their own PR.
- 4:51:44It hooks into the same temporal
- 4:51:45infrastructure we just walked through.
- 4:51:47The only real difference is selection.
- 4:51:48Instead of the selection script, we take
- 4:51:50the file straight from the engineer's
- 4:51:51PR.
- 4:51:51The agent writes the tests and you get a
- 4:51:53draft test PR linked back to your
- 4:51:54original one.
- 4:51:55Now let's talk about some of the
- 4:51:56challenges that we ran into.
- 4:51:57Our agents originally ran on Linux
- 4:51:59machines which don't have Xcode or
- 4:52:00SwiftLint installed. So when an agent
- 4:52:02would open a PR, after about 20 or 30
- 4:52:03minutes, CI would fail frequently on
- 4:52:05lint.
- 4:52:06This wastes the agent's time and CI time
- 4:52:07as well, which is an scarce resource.
- 4:52:10For a while, 36 13.6% of our PRs were
- 4:52:12failing due to lint. And then when we
- 4:52:13dug into it, 62% of those failures came
- 4:52:15from just two lint rules.
- 4:52:17We fixed this in layers. The first was
- 4:52:18better examples in the prompt along with
- 4:52:20pitfalls to avoid.
- 4:52:21The second was a GitHub action that auto
- 4:52:22fixes lint errors in CI.
- 4:52:24The third was an audit of the rules that
- 4:52:25couldn't be auto fixed.
- 4:52:27We essentially removed the ones that
- 4:52:28weren't high signal, and this made life
- 4:52:29easier for both agents and human
- 4:52:30engineers.
- 4:52:31And finally, the real fix came when we
- 4:52:33moved the agents to tunnel Mac runners.
- 4:52:34So now the agents could build and lint
- 4:52:36the code locally before ever opening a
- 4:52:37PR.
- 4:52:39The second challenge was PR quality.
- 4:52:41When we assign a PR to a human, we want
- 4:52:42it to be something that actually feel
- 4:52:43comfortable proving.
- 4:52:45But agents sometimes take the shortest
- 4:52:46paths to green CI. We saw agents editing
- 4:52:48production code to make a failing test
- 4:52:49pass. We also saw them modifying,
- 4:52:51deleting, or silencing existing tests.
- 4:52:53And then we saw them adding long wait to
- 4:52:54tests which can make CI flaky.
- 4:52:57Again, there was no single fix here, so
- 4:52:58we built three layers. The first is the
- 4:53:00deterministic rule. If a PR touches any
- 4:53:02file that isn't a test, a mock, a fake,
- 4:53:04or a stub, we close it out.
- 4:53:06The second builds on our AI code review.
- 4:53:07At Duolingo, we use Claude and Codex to
- 4:53:09review every PR, and those bots often
- 4:53:11leave comments.
- 4:53:12So we built an agent that resolves any
- 4:53:14P0 comments before a human ever looks at
- 4:53:15the PR.
- 4:53:17And the third is an LLM as a judge. It's
- 4:53:18a reviewer agent built just for these
- 4:53:20test PRs. It reads the PR diff and
- 4:53:22rejects production edits, changed or
- 4:53:23deleted tests, disabled tests, and long
- 4:53:25waits.
- 4:53:26And as we learn more, we keep adding
- 4:53:27additional checks to it.
- 4:53:30So far, the LLM as a judge has reviewed
- 4:53:31about 1,200 PRs and rejected 73 of them.
- 4:53:34Every rejection comes with a comment
- 4:53:35like this one, saying exactly why the PR
- 4:53:36was rejected.
- 4:53:38We then read that feedback and fold it
- 4:53:39back into our prompts and skills so that
- 4:53:40same mistakes get less likely each time.
- 4:53:43So let's talk about what all of this
- 4:53:44added up to.
- 4:53:46Since we started, the pipeline has
- 4:53:47merged more than 700 test PRs, adding
- 4:53:49about 225,000 lines of test code.
- 4:53:51Every one of those PRs was reviewed and
- 4:53:52approved by a human.
- 4:53:54NVM test coverage went from 9% to 59%
- 4:53:57and overall coverage went from 11% to
- 4:53:5824%.
- 4:53:59Most importantly, we now have a codebase
- 4:54:01so engineers and agents can ship faster
- 4:54:02and with more confidence.
- 4:54:04Here's a few more interesting stats.
- 4:54:06Since March, the pipeline has written
- 4:54:0734% of all new test files in the repo,
- 4:54:0944% of all new test cases, and 10% of
- 4:54:11every supply added to the codebase.
- 4:54:13And interestingly, that share has
- 4:54:14actually dipped in the last few weeks
- 4:54:15because engineers are now writing a lot
- 4:54:16more tests themselves, which is honestly
- 4:54:18the end goal.
- 4:54:20Um and here's the coverage curve since
- 4:54:22April when the back field started
- 4:54:23running on a schedule. You can see it
- 4:54:25bent slightly upward around June and
- 4:54:26July, which is when we really wrapped up
- 4:54:27the pipeline.
- 4:54:29So what did we learn while doing this
- 4:54:30project?
- 4:54:31The first thing is that this pattern
- 4:54:32generalizes. The loop we talked about,
- 4:54:34identify, write, verify, resolve,
- 4:54:36iterate, it isn't specific to testing.
- 4:54:38Unit tests were just the lowest hanging
- 4:54:39fruit and any job that fits the shape
- 4:54:41can run on the same infrastructure.
- 4:54:42And that's exactly what we're starting
- 4:54:43to do at the company. We're applying the
- 4:54:45same loop to modernization, to
- 4:54:46modernizing screens to MVVM and SwiftUI,
- 4:54:48migrating code uh on Android to
- 4:54:50coroutines and generating snapshot
- 4:54:51tests.
- 4:54:53So before I wrap up, I want to leave you
- 4:54:55with a thought experiment. There's a new
- 4:54:57class of fast general purpose
- 4:54:58classifiers like Jev. They take in some
- 4:55:00state and output an action in
- 4:55:01milliseconds. So what if we replaced a
- 4:55:02handwritten decision tree in the PR
- 4:55:04lifecycle workflow with one of these?
- 4:55:06PR state goes in, the classifier picks
- 4:55:07the next action and we execute it all in
- 4:55:09milliseconds.
- 4:55:12I also want to thank a few folks who
- 4:55:13worked on this with me. Jesse,
- 4:55:14Guadalupe, Hunter and Jihau. They built
- 4:55:16a lot of the platform pieces this
- 4:55:17project stands on and helped review a
- 4:55:18lot of this code.
- 4:55:20That's it for me. Thank you so much for
- 4:55:21making it this far. If you want to go
- 4:55:22deeper, we've written about this
- 4:55:23pipeline on the Duolingo engineering
- 4:55:24blog and I'd also love to hear from you
- 4:55:26if you're building something similar.
- 4:55:27You can find me on LinkedIn or on X.
- 4:55:33>> Hi everyone. Welcome to my talk about
- 4:55:35what happens when you detonate thousands
- 4:55:37of agent skills. This is uh looking
- 4:55:40inside our agent detonation chamber and
- 4:55:43what we saw when we put skills in there.
- 4:55:46My name is Roey. I'm a staff engineer at
- 4:55:48Zenity and I'm a Math research fellow. I
- 4:55:51talk a lot about AI safety and security.
- 4:55:53Follow me on Twitter and LinkedIn.
- 4:55:56Before we even start our uh session, we
- 4:56:00need to talk about what an agent skill
- 4:56:02is.
- 4:56:03Well, essentially an agent skill is
- 4:56:05capability of capability on demand. It
- 4:56:08has a bunch of instructions, resources,
- 4:56:10new behavior, it can contain scripts and
- 4:56:13it can even contain code.
- 4:56:15The problem is that it was written by
- 4:56:17someone else, and this is exactly where
- 4:56:19the problem starts. So let's take a look
- 4:56:21at the skills trajectory for instance.
- 4:56:25If you look at this if we look at the
- 4:56:26trajectory over here, we can see that
- 4:56:28the skill reads through um
- 4:56:31the cloud configuration and Kubernetes
- 4:56:33configuration. It contacts the identity
- 4:56:36service and it even probes the the
- 4:56:38instance metadata endpoint. Now I don't
- 4:56:41know what was the original intent of the
- 4:56:43skill, but I can tell you that a lot of
- 4:56:45um InfoStealers are operating in the
- 4:56:47same way. They scan for the different
- 4:56:50secret and identity folders.
- 4:56:52In this case however, it was an official
- 4:56:54Azure diagnostic skill.
- 4:56:56And this is exactly the point where it's
- 4:56:58kind of hard for us to identify whether
- 4:57:00a skill is malicious or benign, and we
- 4:57:02have to understand the original intent
- 4:57:04of the skill according to its actions.
- 4:57:07And at this point you might ask, okay,
- 4:57:09so we can just like scan the skill,
- 4:57:11right?
- 4:57:12Well, it's it's it's more complicated
- 4:57:14than that. The The first problem that
- 4:57:17we've noticed is that the payload just
- 4:57:19lives elsewhere.
- 4:57:20In a lot of the skills, skill.md is just
- 4:57:23the entry point. It can
- 4:57:25invoke another endpoint that contains
- 4:57:28the rest of the prompt. It can download
- 4:57:30another package as part of it's part of
- 4:57:32its installation script, and it might
- 4:57:35have another script or code that
- 4:57:36downloads other payloads and other
- 4:57:38packages.
- 4:57:39So if you use just a static scanner and
- 4:57:42read through skills.md, you would lose
- 4:57:45all of that, and you would have to
- 4:57:46create some sort of mechanism to track
- 4:57:48all of these payloads if you don't want
- 4:57:50to lose them.
- 4:57:51The second problem that we've noticed is
- 4:57:53that LLM analyzers can be tricked.
- 4:57:56Um there's a really interesting research
- 4:57:58by Trail of Bits in which they showcase
- 4:58:00the skill that manipulated the package
- 4:58:03registry of one of of one of the
- 4:58:05projects, and it had like a small
- 4:58:07comment that said that the AppSec team
- 4:58:10approved this change, and it completely
- 4:58:13walked on the LLM analyzer. So given
- 4:58:16that
- 4:58:18scans are not bulletproof,
- 4:58:20we've resulted to a different kind of
- 4:58:21action and we actually decided to just
- 4:58:23run the skill with the agent.
- 4:58:25Um this is a technique employed by
- 4:58:27malware researchers for years now uh and
- 4:58:30it's called detonation a detonation
- 4:58:32chamber. So we created the same one but
- 4:58:36for for for agents and skills.
- 4:58:38And this was our first version. You
- 4:58:40know, first version uh we created a task
- 4:58:42prompt, we put an agent and a skill
- 4:58:44inside a sandbox, we let it operate over
- 4:58:47like minutes or or hours until until it
- 4:58:50completed the task with an observed
- 4:58:52agent traces and analyzed it.
- 4:58:55You can imagine that it wasn't enough.
- 4:58:57Our first problem was that the
- 4:58:59environment was missing most of the
- 4:59:01time. A lot of skills rely on CLIs, they
- 4:59:05rely on specific services and even on on
- 4:59:08the existence of the
- 4:59:09project. You can imagine notion skill,
- 4:59:12for instance. This will have to have a
- 4:59:15notion service available to talk to.
- 4:59:17So we've added a world builder that
- 4:59:19looks at the skill and builds the
- 4:59:21appropriate environment accordingly.
- 4:59:25But then we encountered another issue. A
- 4:59:26generic task prompt might not be enough
- 4:59:30for all skills.
- 4:59:32I mean, think about it. You can have a
- 4:59:34presentation skill uh that needs to
- 4:59:36create a deck or deployment skill that
- 4:59:37need to ship a service, a brainstorming
- 4:59:40skill that needs to explore an idea.
- 4:59:42Each and every one of them has an
- 4:59:44entirely different modus operandi. Let's
- 4:59:46take shipping shipping a service for
- 4:59:48instance. You need to learn how to
- 4:59:49configure it, you need to learn how to
- 4:59:51deploy it, you need to tell it to
- 4:59:53recover from snapshots using snapshots.
- 4:59:56Um each and every one of them uh
- 4:59:59requires different set of prompts and
- 5:00:01you also need to need to make sure that
- 5:00:03you
- 5:00:04um
- 5:00:05cover
- 5:00:06every route in the in that skill because
- 5:00:09one of them can be malicious. So we
- 5:00:11created the operator that does just
- 5:00:12that. It reads through the skill and
- 5:00:14tries to analyze what are the different
- 5:00:16trajectories um
- 5:00:19that we should invoke.
- 5:00:20And while it worked pretty well, if you
- 5:00:23worked with agent, you can especially in
- 5:00:25the security domain, you can
- 5:00:29um
- 5:00:30assess
- 5:00:32that we went through a lot of
- 5:00:34safeguards,
- 5:00:36safeguard triggers.
- 5:00:37Um we we got a lot of refusals. Um the
- 5:00:40agents wasn't willing to decode and and
- 5:00:43when it was willing to decode, it wasn't
- 5:00:45willing to execute.
- 5:00:47So we had to give you an an offer you
- 5:00:49can't refuse and essentially create an
- 5:00:51adversarial operator that convinces the
- 5:00:54agents to run
- 5:00:56and uh in case you didn't run even like
- 5:00:59start the run from
- 5:01:01the beginning.
- 5:01:03Now I I would I I I I would emphasize
- 5:01:05that we're doing a security research
- 5:01:07here. This is a contained environment.
- 5:01:09Nothing is malicious in here, but this
- 5:01:11is one of the things that we had to do
- 5:01:13in order to make it run properly. I
- 5:01:15would also note that a lot of the time
- 5:01:17safeguards are being run on something
- 5:01:19which is
- 5:03:15>> This
- 5:06:48>> Cool.
- 5:06:52>> WOO!
- 5:06:55[applause]
- 5:06:58GIVE ME A SEC TO MAKE SURE THIS IS
- 5:06:59WORKING.
- 5:07:01HECK YEAH.
- 5:07:02I'm Diogo Almeida, and this talk is I
- 5:07:05Love You So Jev Much. Um, Glass Animals
- 5:07:09reference. We're not just weird. Um,
- 5:07:11like I said, I'm Diogo Almeida. I'm
- 5:07:14known as the Jev guy, sometimes Jevvy
- 5:07:16Pinkman. Um, also co-created ChatGPT,
- 5:07:19RLHF, and InstructGPT, but I assume that
- 5:07:22for the crowd today, y'all know who I
- 5:07:25am, at least mostly.
- 5:07:27Um,
- 5:07:28I have like a bunch of AI-specific stuff
- 5:07:30I want to get to, but I like going into
- 5:07:32my favorite topics in AI. First one is
- 5:07:35what the is actually going on and
- 5:07:38where is all the automation? If you've
- 5:07:39heard me say this before, I'm sorry, but
- 5:07:41it's such an important thing to beat
- 5:07:43into the field cuz people still do not
- 5:07:45get it, and I need to yell this until
- 5:07:47everyone really understands what the
- 5:07:49is going on. And if you're an AI,
- 5:07:51you should have a really good answer
- 5:07:53about why AI can do so many awesome
- 5:07:55things and yet still be so useless for a
- 5:07:57bunch of other ones.
- 5:07:59My relationship to ChatGPT is somewhat
- 5:08:02unique, one of the few people who hates
- 5:08:04on ChatGPT who made it. I don't hate
- 5:08:07ChatGPT, to be clear. I love using
- 5:08:10agents. I love using coding agents. I
- 5:08:11love chatbots. They're just quite
- 5:08:13limited. And in particular, I think it's
- 5:08:15a little tragic how much they've
- 5:08:17bifurcated the field into overpromise,
- 5:08:19underdeliver.
- 5:08:21Really, really terrible stuff for the
- 5:08:22world in my opinion.
- 5:08:23And I like to talk about like the two
- 5:08:25cults in AI and how we can resolve the
- 5:08:28cognitive dissonance that everyone, I
- 5:08:31think, should be having, unless they've
- 5:08:32a very good answer.
- 5:08:34There's cult one, which is, you know,
- 5:08:36the cult of like EA's. They look at all
- 5:08:38the benchmarks and they say like not
- 5:08:40only are LM's surpassing human level on
- 5:08:43basically every benchmark. They have
- 5:08:44been for two years. We've been
- 5:08:46announcing AGI three months away for two
- 5:08:48years now.
- 5:08:49Even since like the '03 days. Do people
- 5:08:51remember that? No one even cares about
- 5:08:53that model anymore and it was that was
- 5:08:55like peak internal to OpenAI AGI hype.
- 5:08:58And on the other hand, you look at the
- 5:09:00non-benchmarks and you look at the
- 5:09:02reality and you see the rest of the
- 5:09:04world, especially if you guys go outside
- 5:09:05and leave San Francisco, a lot of the
- 5:09:07world is like really unchanged despite
- 5:09:10AI being so cool. And I think AI
- 5:09:12is transformative and it's a rare point
- 5:09:14of view to actually think AI is cool and
- 5:09:17transformative and also like
- 5:09:19underwhelming the world right now. And
- 5:09:21the tricky part is this is the same
- 5:09:23field and we are all seeing the same
- 5:09:25facts. So what gives? You know, why is
- 5:09:27AI jagged, so to speak? Like why can it
- 5:09:30be so good at some things and so not
- 5:09:32good at other things? And what is the
- 5:09:35Occam's Razor explanation for that? Um I
- 5:09:38will be blasting through my slides fast.
- 5:09:40I'm sorry. Maybe I could slow down for
- 5:09:42people taking pictures. Um but uh I'm
- 5:09:45noticing that
- 5:09:46this is going to go fast is all I'm
- 5:09:48saying. So I think that we as a field
- 5:09:50should have an Occam's Razor explanation
- 5:09:52for why are there some things that are
- 5:09:54just
- 5:09:55not just good, they are unbelievably
- 5:09:57good, superhuman, like millennium prize
- 5:09:59problems in math. But on the other hand,
- 5:10:02they're not just bad, they're so
- 5:10:03useless, they can't even be used to
- 5:10:05touch accounting. That sounds way easier
- 5:10:07than millennium prize problems in math,
- 5:10:08right? I could do the latter. I
- 5:10:09definitely could not do the former and I
- 5:10:11was a math lead. So what gives here?
- 5:10:13What is the reason for this? And I
- 5:10:16normally love this as an interactive
- 5:10:17thing, but I will instead just tell you
- 5:10:19my answer. Sorry for the power dynamic.
- 5:10:21Um everything on the left side is
- 5:10:24intrinsically a human in the loop task.
- 5:10:26Like deep research, I could not do
- 5:10:28better than deep research. The models
- 5:10:29are so good at that.
- 5:10:31But if gun to my head or gun to your
- 5:10:33head, you had to pick between me and
- 5:10:35deep research, and the answer had to be
- 5:10:37correct, not well-written, you'd
- 5:10:38probably pick me every time.
- 5:10:40And that's the distinction between like
- 5:10:43optimizing for automation, which tends
- 5:10:45to be a lot easier and a lot simpler,
- 5:10:46yet
- 5:10:47AI keeps failing at this. Every year
- 5:10:49there's like an infamous failure of AI
- 5:10:51failing to even automate drive-throughs.
- 5:10:53And that should be kind of shocking to
- 5:10:56people. And my claim is actually for
- 5:10:59these tasks, pleasing a human is nothing
- 5:11:01to do with doing the task. What you want
- 5:11:03to do is basically be an abstraction
- 5:11:05away. You want the decision to be made,
- 5:11:07and then you want to get back into doing
- 5:11:09automation, and code is the language of
- 5:11:11automation. Therefore, you need to
- 5:11:12integrate with code. This is a lot, but
- 5:11:14I will summarize it again. I get the
- 5:11:16irony that you know, I co-created RLHF,
- 5:11:19and I do believe RLHF is part of the
- 5:11:21problem. And RLHF in particular is why
- 5:11:24all AI today does require a human in the
- 5:11:27loop with an asterisk unless the task is
- 5:11:29like very trivially verifiable and can
- 5:11:31run on its own, but still requires a
- 5:11:33human in the loop to verify it
- 5:11:34afterwards. It is because we literally
- 5:11:36put it in. That is what the HF is in
- 5:11:39RLHF. And my claim is that there's a
- 5:11:41better way. You know, my summary of the
- 5:11:44state is 100% of LLMs today are
- 5:11:46optimized for assistance. You might talk
- 5:11:48about RLVR, but the RLVR models are very
- 5:11:50bad to interact with if they've not been
- 5:11:52RLHF'd.
- 5:11:53The result is LLMs become too good to be
- 5:11:55true at assistance. They are just so
- 5:11:57good. Like you should never pick
- 5:11:59me over ChatGPT for your ChatGPT
- 5:12:01queries. But for actually doing like you
- 5:12:04know, run in the background cognitive
- 5:12:06work, you should not trust LLMs at all
- 5:12:08because they've not been optimized for
- 5:12:10that, which is why they're so bad at the
- 5:12:12tasks of automation. And I would also
- 5:12:14guess that automation is a lot more
- 5:12:15valuable than assistance because you can
- 5:12:17do way cooler, deeper. I'm going to get
- 5:12:19into this later, but I think there's
- 5:12:20going to be so much stuff that we can
- 5:12:21do, and we are just scratching the
- 5:12:23surface of that right now. And that's
- 5:12:24the tragic thing about AI,
- 5:12:26the state of AI for me.
- 5:12:29Let's talk about the Jevolution.
- 5:12:32I I can't tell if this is appropriate,
- 5:12:34but it's too
- 5:12:35Um,
- 5:12:36oh god, don't take pictures of this one.
- 5:12:38Um,
- 5:12:39cool.
- 5:12:40Oh, I'm getting a thumbs up. Yeah, uh,
- 5:12:43excellent. So, we're we instead of doing
- 5:12:45RLHF, we just have to do something else.
- 5:12:46We need to think from first principles.
- 5:12:48How do we optimize for machine native
- 5:12:49AI? You guys hopefully know the
- 5:12:51interface of Jeff, and that is what we
- 5:12:52are doing. More important than
- 5:12:54optimizing for its interface, we are
- 5:12:56uh, rethinking how AI should be done
- 5:12:59because this is what we did again for
- 5:13:01RLHF.
- 5:13:02For Sutton's bitter lesson,
- 5:13:04I'm going to loosely paraphrase to make
- 5:13:06my point easier to say, is that
- 5:13:08scaling um, and compute matters more
- 5:13:11than algorithms. This is why people
- 5:13:12become like scaling pill Jivronis. And I
- 5:13:15would argue that there's an even bitter
- 5:13:17lesson than that, which is data always
- 5:13:20matters more than compute. We are
- 5:13:21learning that quite a bit in the field
- 5:13:22now, but it has been obvious since the
- 5:13:24beginning of time. I'll tell you when
- 5:13:26the this is the one you should be
- 5:13:27photographing. I would argue that the
- 5:13:30the bitterest lesson is doing the right
- 5:13:32task. And doing the right task is like a
- 5:13:34weird thing to think about, especially
- 5:13:36for ML people, because they like
- 5:13:38algorithms. They want number goes down
- 5:13:40or up or whichever is good direction.
- 5:13:41They don't want to think about the real
- 5:13:43world, but this is always how you make
- 5:13:45gigantic gains in performance. This is
- 5:13:47how the big breakthroughs happen. 100%
- 5:13:49of the time it comes from this. And this
- 5:13:52is true for classical RL games. This is
- 5:13:53why Sutton had his bitter lesson, it
- 5:13:56which is data is derived by compute when
- 5:13:58you're doing rollouts. Like there's
- 5:14:00there's why would you need off policy
- 5:14:01data when playing chess or something
- 5:14:03like that? And doing the right task is
- 5:14:05defined by the rules. So actually when
- 5:14:06you're doing RL, you don't even need to
- 5:14:08think about the right task.
- 5:14:10For pre-training, we actually as a field
- 5:14:12had to do a lot of thinking about the
- 5:14:14right task. I don't know if people
- 5:14:15remember the birthdays with um, masked
- 5:14:18language modeling, but it took a while
- 5:14:20and it was very unclear to figure out
- 5:14:22that next token prediction was super
- 5:14:24cool, not because it was better
- 5:14:26at masked language modeling. You know,
- 5:14:28the objectives on their own aren't
- 5:14:30special. It's about what you can do with
- 5:14:31them afterwards and it turns out next
- 5:14:33token prediction is incredible at
- 5:14:34generating strings, which is what these
- 5:14:36pre-trained models were so good at. So,
- 5:14:38no level of mass language modeling, even
- 5:14:40if you had today's compute and
- 5:14:41algorithms, would make it as interesting
- 5:14:43as GPT-3 was because it was just doing a
- 5:14:45different task. And data, you know, the
- 5:14:48internet is obviously a giant
- 5:14:50bottleneck. Now we're like, you know,
- 5:14:51burning books or whatever in order to
- 5:14:52get more data, but data obviously
- 5:14:54matters a lot more than compute and
- 5:14:55people keep like rediscovering this and
- 5:14:57figuring this out.
- 5:14:59It's extra true for making LLMs useful.
- 5:15:01Um before RLHF even existed, the data
- 5:15:04for RLHF didn't exist and you could
- 5:15:08like like it was night and day. I'll
- 5:15:09just show you the next slide. This is
- 5:15:10the probably the relevant part. This is
- 5:15:11what this was the work in the RLHF
- 5:15:13paper.
- 5:15:14And [snorts] the high level was that
- 5:15:16even with the dumbest algorithms in the
- 5:15:17world, even with a GPT-2 sized model,
- 5:15:20you got something like GPT-9 level
- 5:15:23performance, like real GPT-9, not
- 5:15:25today's like fake GPT numbers. Like the
- 5:15:27numbers used to mean like 100x scaling
- 5:15:29and effective compute for pre-training
- 5:15:31and they've given up on that because it
- 5:15:32just can't scale anymore. And still even
- 5:15:34with that it would take something like
- 5:15:36GPT-7 to 9 to match GPT-2, the model
- 5:15:39from like 2018 or whatever it was, at
- 5:15:42the task of instruction following. That
- 5:15:44is kind of wild because you think big
- 5:15:46LLMs are so important for this and they
- 5:15:49are, but that is not the most important
- 5:15:51thing compared to even the gigantic jump
- 5:15:54of doing the right task. And spoilers,
- 5:15:55this is what we did for Jeff, right?
- 5:15:57Like we made this gigantic leap in the
- 5:15:59field that no one could realized could
- 5:16:00be coming because we were optimizing for
- 5:16:02a task and very few people think about
- 5:16:04the right task.
- 5:16:06Um
- 5:16:07this is why Jeff has the Jeff ne sais
- 5:16:10quoi. Um that's a really smart pun, I
- 5:16:13think. Um
- 5:16:15that that little bit of a pity laugh,
- 5:16:17but thank you. Um you get what you
- 5:16:19optimize for. I'm looking for a better
- 5:16:21word than reliable, so if you guys have
- 5:16:23suggestions, let me know. But there's if
- 5:16:25there's this reliability of intelligence
- 5:16:27that's really missing in LLMs. I often
- 5:16:29call it you get Einstein 95% of the
- 5:16:32time, but Mr. Bean 5% of the time.
- 5:16:34And if you can't tell which is which,
- 5:16:36how the are you going to automate
- 5:16:37anything? And instead instead of like
- 5:16:39focusing on like sometimes
- 5:16:41smarts that you can then verify, what if
- 5:16:43you focused on extreme, extreme narrow,
- 5:16:45focused reliability so you can get that
- 5:16:47smarts every time. And this doesn't show
- 5:16:49up on benchmarks. It is the reason why
- 5:16:51people love Jev. And this is also, I
- 5:16:53realize in hindsight, we obviously
- 5:16:54didn't prepare for a chat GPT moment for
- 5:16:57developers, but it hasn't because people
- 5:16:59in the AI space just overpromise,
- 5:17:02underdeliver so much that having
- 5:17:04a model that actually didn't do that was
- 5:17:06a surprise. And it is not the speed cost
- 5:17:08or even interface. This is something
- 5:17:10that, you know, my team tells me I
- 5:17:12shouldn't be telling people so that we
- 5:17:13can have a long-term advantage and I
- 5:17:15just care too much. Like it's reliable
- 5:17:17intelligence. Make sick models, please,
- 5:17:19even if we, you know, if it doesn't
- 5:17:21involve our models.
- 5:17:23Um we see that Jev is like a foundation
- 5:17:25for smart software. I'm going to skip
- 5:17:27all over this, but I think software is
- 5:17:28going to be so back. I think the
- 5:17:30narrative of SAS apocalypse is obviously
- 5:17:32not going to happen, but
- 5:17:34I think that
- 5:17:36even calling it a SAS apocalypse is like
- 5:17:38an understatement and I'll get into that
- 5:17:39a bit and you guys ideally are the
- 5:17:41perfect crowd for this. I'm I'm so
- 5:17:42excited for this part because that's the
- 5:17:43part that's new to me. Um
- 5:17:46I want to make a distinction between
- 5:17:47smart software and just in time
- 5:17:49software. The the crazy part is AI is so
- 5:17:52transformative and yet somehow despite
- 5:17:55AI being so awesome, software has
- 5:17:57essentially unchanged in the last decade
- 5:18:00except being cheaper to write. Isn't
- 5:18:02that weird? Isn't the it weird that the
- 5:18:04best we can do in software is having a
- 5:18:06chatbot to the side that sometimes
- 5:18:08interacts with the software but cannot
- 5:18:10take expensive decisions for you. That
- 5:18:12is kind of like a weird thing if AI was
- 5:18:14really that transformative. And I think
- 5:18:16the problem is we're focusing too much
- 5:18:17on coding agents and not enough about
- 5:18:20putting the intelligence into the inner
- 5:18:21loop, which I'm going to get more into a
- 5:18:23bit. And people are jeving. There's been
- 5:18:25tons of successes. We don't need to flex
- 5:18:27this. There's so many cool things, but
- 5:18:29my AI secret is going to be that the
- 5:18:33field isn't jeving hard enough. And this
- 5:18:35is what Swix said that I can make a call
- 5:18:36to action for you guys that's unique and
- 5:18:38hopefully I have enough time for this. I
- 5:18:40think the world still has LLM Stockholm
- 5:18:42syndrome. We are still thinking from a
- 5:18:44really LLM-filled place, which is
- 5:18:47I think tragic, you know, like I I I
- 5:18:49love the software saying the tools you
- 5:18:51use change the way you think. And I
- 5:18:53think that it's changed the way we think
- 5:18:54way too much
- 5:18:55and we're not really rethinking software
- 5:18:57like I think the field needs. I think we
- 5:18:59could just do be doing so much more.
- 5:19:01Hopefully I can try to do my best for my
- 5:19:03remaining time to convince you guys of
- 5:19:05that.
- 5:19:05Um so let's jet to the future.
- 5:19:08Um
- 5:19:10I an analogy I love bear with me is
- 5:19:12about LEDs. Before there was the blue
- 5:19:16LED, there was red and green LEDs. These
- 5:19:18were really limited in capability and
- 5:19:20they were used for like simple
- 5:19:21indicators and you know like clocks and
- 5:19:23stuff like that.
- 5:19:24And when the blue LED was made
- 5:19:26discovered actually don't really know
- 5:19:28how things outside of computer science
- 5:19:29work. Um but when it existed I guess um
- 5:19:33it enabled not just like 50% more use
- 5:19:36cases which what you'd expect about like
- 5:19:37changing colors from two to three. It
- 5:19:39enabled so many more. Like now LEDs
- 5:19:41could be used for white light. It could
- 5:19:43be used for like the displays that we
- 5:19:44all use right now and it created a
- 5:19:46revolution in lighting. And it's it's
- 5:19:49it's unfathomable how much bigger
- 5:19:51the capabilities of LEDs were just you
- 5:19:54know adding a new primitive that
- 5:19:56composes together with the old
- 5:19:57primitives. You can see where I'm going
- 5:19:59here maybe. And I think it's a similar
- 5:20:01story for software. Software is so
- 5:20:04incredible. Ignore AI for the
- 5:20:06last you know half decade, but software
- 5:20:08has done so many insanely cool things.
- 5:20:11We've done a lot of stuff you know like
- 5:20:13do I
- 5:20:14love using Sass? Not really, but does it
- 5:20:16solve problems? Abso lutely. And
- 5:20:19it's incredible how much we've done with
- 5:20:21three fundamental logic gates. You know
- 5:20:23like we are doing this with like simple
- 5:20:25binary operations that can do things
- 5:20:27like greater than less than equal. It's
- 5:20:29kind of nuts how much software has been
- 5:20:31built from such simple primitives. And
- 5:20:34my claim is that what we are doing with
- 5:20:37AI now when we're making it our
- 5:20:39intelligence machine native, this is
- 5:20:40kind of conceptually adding a fourth
- 5:20:43semantic logic gate.
- 5:20:45What if we program with common sense,
- 5:20:47with intention, with human
- 5:20:48understanding? Like do we still need
- 5:20:50regexes with like crazy things? Or can
- 5:20:52we just say like we want a Gmail email
- 5:20:54address or a real-looking one? There's
- 5:20:56so much that we can do that we're going
- 5:20:57to need to push the you know push the
- 5:20:59frontiers of cheap, smart, fast in order
- 5:21:02to get it done. But I think that there's
- 5:21:03going to be like so many things here.
- 5:21:05And I think just like the blue LED, it's
- 5:21:07going to be way bigger than just
- 5:21:08increasing the representational ability
- 5:21:11of software by 33% cuz it's adding
- 5:21:13effectively another kind of sort of
- 5:21:16logic gate. And this sci-fi is the stuff
- 5:21:18that I think will be impossible to
- 5:21:19predict. Just like who could predict how
- 5:21:21powerful the internet would have been
- 5:21:23because it requires it's like much more
- 5:21:25grassroots and decentralized and it
- 5:21:26requires people to be empowered to do
- 5:21:28insane stuff.
- 5:21:30Um I call this software research. Um
- 5:21:34that's not a joke. Um but uh I I think
- 5:21:37that there's going to be like an
- 5:21:38explosion of what software can do and we
- 5:21:41will want people to be exploring those
- 5:21:43boundaries of software, not necessarily
- 5:21:46because it's valuable, but because they
- 5:21:47can, because we have like new frontiers
- 5:21:50to explore. And this is what the early
- 5:21:51internet was like. Should all of those
- 5:21:53websites existed?
- 5:21:55Probably not, but like it was very cool
- 5:21:57and we found a lot of awesome stuff from
- 5:21:59that and I think we we you know we have
- 5:22:01another opportunity for that. So we
- 5:22:02should not just keep living in this
- 5:22:04little you know fish tank that we are
- 5:22:06the victims of OpenAI and Anthropic on
- 5:22:08whatever they ship and we just have to
- 5:22:10be like LLM wrappers. So I'm going to
- 5:22:12pitch software research as direction
- 5:22:14here. The theme is going to be real
- 5:22:16software engineering, not just vibe
- 5:22:18coding. There's it's going to be like
- 5:22:20the kind of stuff that everyone has
- 5:22:21known if you've been a software
- 5:22:23engineer, decomposition, breaking tasks
- 5:22:25down. We need to think about stages,
- 5:22:27ideally ones that can be verified so we
- 5:22:29have real encapsulation. And we need to
- 5:22:31actually do state management instead of
- 5:22:33assuming everything is global state
- 5:22:35within a KV cache. And the theme is just
- 5:22:37going to be like real
- 5:22:38engineering by people who care a lot
- 5:22:40about the end result. It doesn't have to
- 5:22:42be valuable. Ideally it's sick as
- 5:22:44Um and you know we should be pushing the
- 5:22:46limits.
- 5:22:48I'm going to go a couple of like my
- 5:22:49favorite ideas and I want to figure out
- 5:22:51how to encourage these in the world. So
- 5:22:54this is my first step at this. I'm just
- 5:22:56going to talk about it. We have raised a
- 5:22:58bunch of money, so I would like to
- 5:22:59figure out how to fund this, but we
- 5:23:00don't have the operations help to do
- 5:23:01that. But here are just some ideas out
- 5:23:03there. Um, I think agents are
- 5:23:07they're very cool, but like they're
- 5:23:10they're so inflexible in their design
- 5:23:12because of the tyranny of the KV cache.
- 5:23:13I don't have the time for this if this
- 5:23:15timer is to be believed, but believe me
- 5:23:17that there's so much that people have
- 5:23:19tried to do but failed at because it
- 5:23:21ends up becoming inefficient because it
- 5:23:22fights against the KV cache. But I think
- 5:23:25there's a lot more to do. Why is memory
- 5:23:27O of N? Right? Like why can't we have
- 5:23:29all of our history of coding agents
- 5:23:31always included in every new call? That
- 5:23:33is how continual learning works, right?
- 5:23:36And we have the memory available. It's
- 5:23:38just that we don't have the smarts to
- 5:23:39map produce over the memory to find out
- 5:23:42like what you should what you should
- 5:23:43remember or what should you shouldn't.
- 5:23:45Um, I think there's many ways to do
- 5:23:46login or even better lookups. Why when
- 5:23:49you're making many agents at once, why
- 5:23:51can't they not share state? Why do they
- 5:23:52need to rediscover the same patterns of
- 5:23:54the codebase over and over and over
- 5:23:56again? And why can't non-naive routing
- 5:23:58and sub-agents work? Like even naive
- 5:24:00routing doesn't work, but like why can't
- 5:24:01we do these things and why just like the
- 5:24:03you know, the the big labs keep going
- 5:24:04back and forth on whether or not
- 5:24:05sub-agents work? Um, I think that like
- 5:24:08from an engineering perspective, there's
- 5:24:10elegance to it, but they fight against
- 5:24:12the KV cache. And I think that if we
- 5:24:13design agents around that, we could do
- 5:24:15so much more.
- 5:24:17This one's the probably the most vague
- 5:24:19of my directions, but I feel very
- 5:24:21strongly about it. Um, the world is full
- 5:24:23of principal agent problems. This is
- 5:24:25when you have like you know, ulterior
- 5:24:27motives between this is not a coding
- 5:24:28agent, but like any agent. The the
- 5:24:30classic example is citizens versus
- 5:24:32elected officials. And I don't want to
- 5:24:34make a too political claim here, but
- 5:24:36like there's misalignment in all levels
- 5:24:38of society and this results in like
- 5:24:40waste, fraud, bad things and it it
- 5:24:43results in for a society that wants
- 5:24:45fairness, you need to over emphasize the
- 5:24:47letter over the spirit, which then
- 5:24:49results in loopholes and play uh pay to
- 5:24:52win dynamics in the world. And I think
- 5:24:54the root of that problem is we can't
- 5:24:55encode the spirit. Just like with
- 5:24:57software, we're constrained to encoding
- 5:24:59bits instead of what we actually meant
- 5:25:01in that situation. And I think so many
- 5:25:04organizations, society could be
- 5:25:05transformed by really encoding the
- 5:25:07spirit all over the place.
- 5:25:09Obviously, real-time intelligence
- 5:25:10security
- 5:25:12can we prevent the open AI hugging face
- 5:25:14sandbox, blah blah blah? Um why did why
- 5:25:16couldn't they prevent it? It is not
- 5:25:18because they didn't have the
- 5:25:19intelligence. I would argue it's a lot
- 5:25:21easier to figure out they're doing a
- 5:25:22naughty thing than to actually do the
- 5:25:24naughty thing. It is because it was not
- 5:25:26fast, cheap, and reliable enough to
- 5:25:28actually run real time. And what happens
- 5:25:31when every corporation has their own
- 5:25:33real-time security that's actually
- 5:25:35smart? How cool would that be?
- 5:25:38I can't talk about generative UIs
- 5:25:39because people are going to be talking
- 5:25:41about it afterwards. I don't want to
- 5:25:42steal their thunder, but it's cool as
- 5:25:43trust me. Um
- 5:25:46like semantic dependent typing. Why
- 5:25:48can't
- 5:25:49uh we have like a type system that
- 5:25:51actually is semantic, right? Like why
- 5:25:54can't we like declare what local
- 5:25:57properties we want so we don't have to
- 5:25:59read a chain of dependencies deep down?
- 5:26:01These all should be possible. And
- 5:26:02imagine if you could like run a Jev-like
- 5:26:04model in your lint checker to check if
- 5:26:07your it's all type safe and it could you
- 5:26:09know uphold semantic properties. It
- 5:26:11would be so cool.
- 5:26:13There's a there was a great Twitter
- 5:26:15video on um like a voice interface. In
- 5:26:18general, I think new interfaces are so
- 5:26:20exciting. This is to me the return of
- 5:26:22like this is going to show by agent
- 5:26:24software like do what I mean. This used
- 5:26:26to be like a popular thing, but it never
- 5:26:27really worked because you it was it was
- 5:26:29too many hard-coded rules to really
- 5:26:31figure out what I mean. Um
- 5:26:33there's stopping slop. Obviously, a big
- 5:26:36theme of here, but like we definitely
- 5:26:38believe this. This is my props I have to
- 5:26:40We have a stop gen AI baseball bat for
- 5:26:42type safe. It is very very awesome. I
- 5:26:45want to talk about the root of slop. Um
- 5:26:48the the root of slop is that AI is
- 5:26:49unreliable. Therefore, you as a company
- 5:26:52or you as you know like someone who's
- 5:26:53doing stuff, it can't be used for
- 5:26:55decisions that costs you stuff.
- 5:26:57But it's actually totally fine to push
- 5:26:59the costs into other people, which is
- 5:27:00horrible, you know, externalized costs.
- 5:27:02And then this is what ends up becoming
- 5:27:04slop.
- 5:27:04You know, if people actually read the
- 5:27:06stuff and filter it and like care about
- 5:27:07the quality, it becomes not slop. But
- 5:27:10now the world is full of this because
- 5:27:12there's an asymmetric advantage for
- 5:27:13people who are like bad actors and don't
- 5:27:15give a So I think that we can save
- 5:27:17the world from at least inbound hell
- 5:27:19among many other things and that would
- 5:27:20be very cool. Um and even something I
- 5:27:24think is super sick is cursed software.
- 5:27:26Like why? If you guys ever seen the
- 5:27:28subreddit bad UI battles, it is so
- 5:27:31cool even though they make the
- 5:27:33horrible interfaces like this whole game
- 5:27:35of like would you like to delete your
- 5:27:36account becomes, you know, a random
- 5:27:38thing. And I actually think doing things
- 5:27:41because we can is what software
- 5:27:43engineering is about cuz who knows,
- 5:27:45maybe 1% of these ideas actually lead to
- 5:27:47something super awesome. And I think
- 5:27:50like really cultivating even bad ideas
- 5:27:52is kind of what being like this, you
- 5:27:53know, playful hacker is all about.
- 5:27:56And generally just automating
- 5:27:59everything, this is what the follow-up,
- 5:28:01this is this is a slop image, but sorry
- 5:28:03the logo is a little bit wrong, but
- 5:28:04don't stare at that, that's a little
- 5:28:05weird. Um
- 5:28:07this is what a lot of the follow-up
- 5:28:09talks are going to be about. Thank you.
- 5:28:11Um
- 5:28:12and if you're a software engineer, you
- 5:28:14almost certainly think in the world of
- 5:28:16bits. If you know anything about
- 5:28:17software
- 5:28:18and you've interacted with software, you
- 5:28:21know that there are things software can
- 5:28:23do and software cannot do because that
- 5:28:25is just only obvious, right? Like
- 5:28:27there's some things that you know you
- 5:28:29need to do because that is not
- 5:28:30compatible with software. But what if
- 5:28:33those constraints were no longer true?
- 5:28:34Maybe not today, but maybe in like 6
- 5:28:36months or a year, what couldn't we
- 5:28:38automate? And that's kind of where I
- 5:28:40want to leave this whole thing because I
- 5:28:41think there is very likely to be a
- 5:28:44future where it's not a question of
- 5:28:45whether or not we can automate
- 5:28:47something, it's going to be a question
- 5:28:48of like how much work is it, which is
- 5:28:49how it is right now for the stuff that
- 5:28:51can be automated. And that would be cool
- 5:28:53as
- 5:28:54So hell yeah, thank you.
- 5:28:56Um
- 5:28:58>> [applause]
- 5:28:58>> I'm
- 5:28:59super overtime, but thank you to the
- 5:29:02early believers. You know, we had no
- 5:29:04idea so many people would resonate with
- 5:29:05this. We thought we were the insane
- 5:29:07ones, but it turns out like we were
- 5:29:09actually the majority of the world who
- 5:29:10realized AI could be so much more.
- 5:29:13Thank you to like the people who are
- 5:29:14like really realistic about reliability.
- 5:29:16Everyone knows I hate benchmarks and a
- 5:29:18lot of other people are really screaming
- 5:29:19out like benchmarks suck.
- 5:29:21They're actually
- 5:29:22they hurt the people who do not have
- 5:29:24time for anything other than benchmarks
- 5:29:26most. It's tragic. And for the people
- 5:29:27who are pushing the limits of software,
- 5:29:29like yes. And we want to be there
- 5:29:32with you helping you all do that. To the
- 5:29:34dreamers and the people who don't want
- 5:29:36to be limited by everything that has
- 5:29:38happened so far, we want to really fight
- 5:29:41for you and we want this to be as
- 5:29:43reliable as a fourth logic gate. And
- 5:29:46that is a very high bar and we have like
- 5:29:48a lot of work to do. So, thank you,
- 5:29:50thank you.
- 5:29:53>> [applause]
- 5:31:08>> Yeah.
- 5:34:33>> Hi everyone.
- 5:34:35My name is Chinmay and I am a software
- 5:34:37engineer at Clein.
- 5:34:39I work on the product engineering side
- 5:34:41of Clein assistant.
- 5:34:43Today I'm going to give a talk
- 5:34:45on how Clein uses chef.
- 5:34:48The title of the talk is beyond
- 5:34:50generation
- 5:34:51fast and structured AI decisions with
- 5:34:53chef at Clein.
- 5:34:56If you look at it,
- 5:34:57most AI product steps
- 5:34:59are bounded decisions and not generation
- 5:35:02problems.
- 5:35:03So to give a couple of examples,
- 5:35:05user query intent classification,
- 5:35:08a ranker picking a suggestion out of n
- 5:35:10options,
- 5:35:11checking whether a citation is valid or
- 5:35:13not, routing to the correct model or
- 5:35:15expert, so on so forth.
- 5:35:18Till now we
- 5:35:19have been paying the full LLM cost and
- 5:35:22latency to get an answer to all these
- 5:35:24bounded problems.
- 5:35:28Jev at Glean fills the gap between
- 5:35:31general LLMs and fine tuned classifiers.
- 5:35:35Classification itself isn't a new
- 5:35:36problem. Like people have been trying to
- 5:35:38solve it since forever.
- 5:35:40We can have multiple ways to solve it.
- 5:35:42Some of the things that we have tried
- 5:35:43out
- 5:35:44include using a general purpose LLM,
- 5:35:47using an open model, something like
- 5:35:49Laya,
- 5:35:50fine tuning that open model
- 5:35:52or using Jev.
- 5:35:55Now all these approaches and all these
- 5:35:57different ways have their pros and cons.
- 5:36:00With a general purpose LLM,
- 5:36:02we incur a lot of cost and latency,
- 5:36:04while with something like Jev, we get no
- 5:36:06generated text output.
- 5:36:09Another advantage that we have been
- 5:36:10seeing with Jev is that as the taxonomy
- 5:36:13of the decision space changes, we don't
- 5:36:15have to fine tune these models again and
- 5:36:17again.
- 5:36:18We can just change our prompt a bit and
- 5:36:20boom, it works out of the box.
- 5:36:26Now I'll walk you through how we have
- 5:36:29integrated Gen or these classification
- 5:36:32how we have used Jev at Glean to solve
- 5:36:35these classification problem statements.
- 5:36:37I'll go into some of these problems in
- 5:36:39great detail.
- 5:36:42Before we go into these individual
- 5:36:44problems, I wanted to touch upon how we
- 5:36:47evaluate these changes. Like we have a
- 5:36:49huge customer base having millions of
- 5:36:51users. So we don't want any of the
- 5:36:53change that we make to regress our
- 5:36:54product.
- 5:36:56We have a multi phase approach, a
- 5:36:58rollout approach to any change that we
- 5:37:00are making.
- 5:37:02This includes having a golden eval set
- 5:37:04for almost every possible thing. Like
- 5:37:06any place where we have generative AI,
- 5:37:08any place where we have
- 5:37:10any form of intelligence involved, we
- 5:37:12have a golden eval set for that space.
- 5:37:15We soak all of these changes for a
- 5:37:17significant amount of time for our
- 5:37:18internal use case. Like we have around
- 5:37:201500 people at Glean right now. We soak
- 5:37:23all of these changes within our Glean
- 5:37:26instance for like couple of weeks. We
- 5:37:27get active feedback. We have a very
- 5:37:29active feedback loop from our internal
- 5:37:31users. We work upon that and then we go
- 5:37:33for an AB testing
- 5:37:36with our customers.
- 5:37:39The first problem that we tried out with
- 5:37:41a decision decision model was query
- 5:37:44classification.
- 5:37:46Query classification is when we want to
- 5:37:48know the nature of user queries. This
- 5:37:49helps us
- 5:37:51realize what kind of
- 5:37:53what kind of
- 5:37:55jobs are people using Glean for? What
- 5:37:57kind of Say for example, if people are
- 5:37:58using more for sales like queries, we
- 5:38:00focus more on that. If we people are
- 5:38:02using more for engineering type use
- 5:38:03cases, we focus more on that.
- 5:38:06We tried out three different approaches
- 5:38:07over here and this is
- 5:38:09data from our test bed.
- 5:38:12We used an open source Llama model.
- 5:38:14Then we tried out Jev and then we fine
- 5:38:16tuned the Llama model
- 5:38:18on our own compute resources.
- 5:38:21Jev is the best no training option in
- 5:38:24our opinion.
- 5:38:25It nearly doubles the output quality of
- 5:38:27a base Llama model and is 90 is able to
- 5:38:30perform at 90%
- 5:38:32quality in comparison to a fine tuned
- 5:38:33model fine tuned Llama.
- 5:38:36With Jev, we don't have to go through
- 5:38:38the very tedious
- 5:38:40process of data collection, data
- 5:38:42cleaning,
- 5:38:43incurring the cost for compute like
- 5:38:45training the model,
- 5:38:46fine tuning the model again and again as
- 5:38:48the taxonomy changes.
- 5:38:50And at the same time, we don't have to
- 5:38:52worry about hosting the model or
- 5:38:54incurring the inference cost.
- 5:38:59The next problem we went to is expert
- 5:39:01routing. This is a pretty common problem
- 5:39:03statement in the industry right now.
- 5:39:04Like hey, when the user query comes in,
- 5:39:06what expert do we route it to? Like how
- 5:39:08do we ensure that the user
- 5:39:11user the response to the user is of the
- 5:39:12best possible quality.
- 5:39:14Glean receives the way we have been
- 5:39:16solving this is Glean receives the user
- 5:39:18query
- 5:39:19and then decides what's the best expert
- 5:39:20to answer the query.
- 5:39:22Till now we have been using a general
- 5:39:23purpose LLM to route to the correct
- 5:39:25expert.
- 5:39:26We set up a test bed
- 5:39:27for J which set up a test bed with N
- 5:39:29experts
- 5:39:30and in our experiments found a
- 5:39:32significantly high precision when using
- 5:39:33J and that too at an almost 8x reduction
- 5:39:36in latency.
- 5:39:39We had to change our implementation or
- 5:39:41our infrastructure significantly to
- 5:39:43integrate J into this specific stage of
- 5:39:46our
- 5:39:46product.
- 5:39:47But the reduction in latency justifies
- 5:39:51that inherent complexity in the system.
- 5:39:53We at Glean have been trying to build
- 5:39:55systems in the most in the simplest
- 5:39:57possible manners. This is one of the
- 5:39:58rare pieces where we believe complexity
- 5:40:01is worth it.
- 5:40:04Next I'll walk you through the ranker
- 5:40:06problem statement that I talked about.
- 5:40:10When we receive a user query
- 5:40:12we also want to generate the best
- 5:40:14follow-up suggestions for the user. Like
- 5:40:16this helps us in getting discoverability
- 5:40:18for our product. This helps us in
- 5:40:19ensuring that the user has the best
- 5:40:20possible experience. We also want to
- 5:40:23recommend or say preempt what the next
- 5:40:25user query might be.
- 5:40:27Again a pretty common problem statement
- 5:40:29that the entire industry has been trying
- 5:40:30to solve for.
- 5:40:31Internally we call this
- 5:40:33system Q
- 5:40:35like
- 5:40:36Q as in the literal Q.
- 5:40:38The way this works, we have like 10 or
- 5:40:4115 different sources or 10 or 15
- 5:40:43different ways
- 5:40:4410 or 15 different sources that are
- 5:40:46generating suggestions independently of
- 5:40:48each other.
- 5:40:50Once those suggestion these suggestions
- 5:40:51are generated by natural language LLMs
- 5:40:54and it's like a proper natural language
- 5:40:56query.
- 5:40:58Once these suggestions are in we have a
- 5:40:59ranking step. So the ranker takes in all
- 5:41:01these suggestions, takes in the
- 5:41:03the first user query, takes in the
- 5:41:05output that Glean gave and then all
- 5:41:08these suggestions in that context.
- 5:41:11The ranker is then asked to decide like
- 5:41:12hey you have these suggestions. What do
- 5:41:13you think is the best one?
- 5:41:15A pretty interesting bottleneck that we
- 5:41:17had over here is we want the response to
- 5:41:19be snappy.
- 5:41:20I cannot We cannot have a state where
- 5:41:23we give the textual response to the user
- 5:41:25and then wait 5 seconds for generating
- 5:41:28the follow-up suggestion.
- 5:41:29We have to be snappy. That why we have
- 5:41:32time limits on these suggestions. So
- 5:41:33right now this step had a time limit of
- 5:41:35one 1.5 to 2 seconds.
- 5:41:38Jeff, no doubt gave us a better quality
- 5:41:40than better quality and far better
- 5:41:41latency than what we had with the
- 5:41:43general purpose LLM, but
- 5:41:46two things stood out over there.
- 5:41:47First, we
- 5:41:49were able to give a suggestion within
- 5:41:51our time bounds
- 5:41:53in 99.2%
- 5:41:55cases.
- 5:41:56With the general purpose LLM, this was
- 5:41:58almost 90%
- 5:42:00So this ensures that we
- 5:42:02have user engagement and far more
- 5:42:05user queries now.
- 5:42:07The second thing is
- 5:42:09at a glean, whenever we use these LLMs,
- 5:42:11we have guard rails. Guard rails in the
- 5:42:12sense that
- 5:42:14here LLM might not be good at generating
- 5:42:16a response or LLM might be making some
- 5:42:17specific kind of mistakes. So we have
- 5:42:20checks and balances for all those
- 5:42:22possible mistakes that we see. We keep a
- 5:42:24log of like we keep a log of all these
- 5:42:27checks that are incurred at any point of
- 5:42:29time.
- 5:42:30With a general purpose LLM, we
- 5:42:33were able to generate a valid suggestion
- 5:42:35in almost 90% of the use 90% of the
- 5:42:38cases. We still had some places where
- 5:42:40LLM was either generating a generating a
- 5:42:42response in a different language.
- 5:42:44The output
- 5:42:46did not meet certain other criteria that
- 5:42:48we had
- 5:42:50that we had written programmatically.
- 5:42:52With Jeff, we found
- 5:42:54zero cases to be there when the output
- 5:42:56was invalid.
- 5:42:58Now a a natural question is like, hey,
- 5:42:59but you would have used a different
- 5:43:01prompt. Yes, the prompt was somewhat
- 5:43:02different,
- 5:43:03but we tried to mimic the prompt in the
- 5:43:04two cases to the largest extent possible
- 5:43:06so that we don't have a divergence. Like
- 5:43:08again, we at glean want to keep the
- 5:43:09system as simple as possible.
- 5:43:14Another benefit we see with using a
- 5:43:16decision model is how we can control the
- 5:43:19thresholds.
- 5:43:20Because in this ranker problem statement
- 5:43:23we are getting specific probabilities of
- 5:43:25each output. We are not
- 5:43:27restricted to a binary yes or no
- 5:43:28response. We get like hey these were the
- 5:43:30five options and these are the
- 5:43:32probabilities of these five options.
- 5:43:34With this we are able to fine tune the
- 5:43:36thresholds based on the deployment. We
- 5:43:39have put in self learning systems
- 5:43:41which based on the user based on the
- 5:43:43customer deployment
- 5:43:45and the nature of usage that the
- 5:43:47specific customer has
- 5:43:49fine tunes the thresholds by itself.
- 5:43:51This ensures that the experience in each
- 5:43:55and every customer out there is the best
- 5:43:57possible based on how they use Glean,
- 5:44:00not a general purpose solution that fits
- 5:44:02all.
- 5:44:08Okay, this is an interesting one.
- 5:44:10Now moving on from how we have been
- 5:44:13making the lives or the experience of
- 5:44:15our end users better
- 5:44:17we also tried using Jev and some of the
- 5:44:20some some other parts of our development
- 5:44:22pipeline. One of them was
- 5:44:25how we use LLM as judges.
- 5:44:27Like
- 5:44:28all of us have been using LLM as judges
- 5:44:30for quite some time now. There's an
- 5:44:31input, there's an output
- 5:44:33and we use an LLM to judge how the
- 5:44:35quality of the output is. Now naturally
- 5:44:37whenever we use LLM as a judge we repeat
- 5:44:40the experiment multiple times because to
- 5:44:42account for the variability in the
- 5:44:43response.
- 5:44:45At Glean we used Jev for a very specific
- 5:44:47part of our evaluation pipeline.
- 5:44:50Glean responses or like any other AI
- 5:44:52product out there has like citations.
- 5:44:54This is how people trust that Glean
- 5:44:57Glean's answer was correct and it it is
- 5:44:59grounded in the correct document.
- 5:45:01Now these site we hold these citations
- 5:45:04to a very high bar
- 5:45:05and we have a specific part in our
- 5:45:07evaluation pipeline where we check
- 5:45:10whether the cited document really
- 5:45:12supports the claim made in the answer or
- 5:45:14not.
- 5:45:16This again seems to be a pretty simple
- 5:45:17yes or no question like hey whether the
- 5:45:20claim is
- 5:45:21uh supported or not.
- 5:45:22With Jev, no doubt we had a far better
- 5:45:25latency and cost like
- 5:45:27visible in the slide
- 5:45:28uh
- 5:45:29over here.
- 5:45:31The interesting thing we noticed over
- 5:45:32here is how consistent and reproducible
- 5:45:35Jev results were in comparison to
- 5:45:36something like say Luna.
- 5:45:39In the 28 or 30 use cases that we had
- 5:45:41like only one
- 5:45:43run had like a difference in response.
- 5:45:46Apart from that everything was
- 5:45:48consistent. What this ensure this this
- 5:45:50ensures two things for us as engineers
- 5:45:52or us as developers at Clean.
- 5:45:55We are not only improving the
- 5:45:57quality of our product or we are not
- 5:45:58only improving Clean in a linear fashion
- 5:46:00anymore.
- 5:46:01Any improvement of this extent in the
- 5:46:04development pipeline leads to an
- 5:46:06exponential increase in how we can
- 5:46:09improve the product. So if I'm if I as a
- 5:46:11developer I'm able to ship things like
- 5:46:135x faster like this is significant
- 5:46:15reduction in
- 5:46:16latency like if I run this experiment
- 5:46:19across thousands and thousands of
- 5:46:20entries this is a lot of time.
- 5:46:22And again I don't have to run the same
- 5:46:24uh input output pair multiple times
- 5:46:26because the result from Jev are quite
- 5:46:27consistent.
- 5:46:29So if I'm able to increase my own
- 5:46:31productivity by like 5x or 6x I can ship
- 5:46:346x times the features to our customers
- 5:46:37and this happens across like 500
- 5:46:38engineers or 300 engineers at Clean.
- 5:46:41So the
- 5:46:42impact of
- 5:46:44a a good decision model is no longer
- 5:46:46linear. It is
- 5:46:48exponential on how we can improve the
- 5:46:49product.
- 5:46:52Okay.
- 5:46:54Now moving on from product and uh
- 5:46:56LLMs judges
- 5:46:58there's a pretty
- 5:46:59import this this this is a pretty
- 5:47:01important part of
- 5:47:03how we
- 5:47:04use our customer data like security is
- 5:47:06something that all our customers expect
- 5:47:09from us.
- 5:47:10So for example we have checks and
- 5:47:12balances on all the inputs and outputs
- 5:47:14coming from a tool, all the inputs and
- 5:47:16outputs uh from an LLM.
- 5:47:19Till now we at Glean have like a
- 5:47:21multi-phase approach. There is like
- 5:47:24there are two phases to this. The first
- 5:47:26one is like a recall heavy phase and the
- 5:47:28second one is the final heavy loaded
- 5:47:29LLM. All these calls are made in
- 5:47:32parallel to like the actual tool input
- 5:47:34output calls.
- 5:47:37Till now we used to host a virtue
- 5:47:39classifier using T4 GPUs across all our
- 5:47:41customers. So Glean is a single tenant
- 5:47:43solution like we don't use or share
- 5:47:46resources across customers. Every
- 5:47:47customer gets
- 5:47:49gets its own deployment, gets its own
- 5:47:51instance and gets all the resources. So
- 5:47:54assuming we have 1,000 customers out
- 5:47:56there, we had to maintain like virtue
- 5:47:58classifiers across all these 1,000
- 5:48:00customers which becomes a pretty
- 5:48:02difficult problem statement in itself
- 5:48:04which takes up a lot of time from Glean
- 5:48:05engineers.
- 5:48:07So looking at the results over here, the
- 5:48:10overall recall of the first phase where
- 5:48:12we used to have a virtue classifier, we
- 5:48:14replaced that with Jev, went from 31.5%
- 5:48:17to 90.2%.
- 5:48:19At the same time we are seeing a
- 5:48:20significant reduction in cost. Now this
- 5:48:22cost is only the resource cost mentioned
- 5:48:25over here, but we are also saving cost
- 5:48:28on the engineering resources over here.
- 5:48:30We don't have to invest like two or
- 5:48:32three engineers who are full-time
- 5:48:33maintaining those virtue classifiers or
- 5:48:35those T4 GPUs across customers.
- 5:48:37The the improvement or the savings that
- 5:48:40we have
- 5:48:41are far more than 50 percent.
- 5:48:46This slide concludes all the results I
- 5:48:48have talked about till now.
- 5:48:50Jev at Glean has been able to influence
- 5:48:52the product experience of the users, has
- 5:48:54been able to
- 5:48:56impact how we as developers have been
- 5:48:58working at Glean and most importantly it
- 5:49:01also has a very good positive impact on
- 5:49:03the security that we provide to our
- 5:49:04customers.
- 5:49:06A simple rule that we have been using is
- 5:49:09can the out
- 5:49:10in conclusion a simple rule that we have
- 5:49:11been using at Glean
- 5:49:12is can the output be listed in advance?
- 5:49:15If no then it's a free form generative
- 5:49:17text problem statement.
- 5:49:18Go back to something that can generate
- 5:49:20free form text. If yes then use Jev.
- 5:49:23Are levels stable and volume high?
- 5:49:25If this is the case where the labels are
- 5:49:27stable, which is rarely the case
- 5:49:29anymore,
- 5:49:30uh
- 5:49:31you can also fine-tune a model, but if
- 5:49:33the labels change, which in my opinion
- 5:49:35is quite frequent these days, then again
- 5:49:37use something like Jev.
- 5:49:40Obviously, if you need dynamic text, we
- 5:49:42can like feel free to try out more
- 5:49:45hybrid approaches like Jev along with uh
- 5:49:48free-form text LLMs.
- 5:49:52This ends my presentation. So thank you
- 5:49:54everyone for listening.
- 5:49:56I have added a blog like a link to the
- 5:49:58blog post where we have posted the exact
- 5:50:00same results in great detail.
- 5:50:03Feel free to catch me on the outside if
- 5:50:05you have more questions. Thank you.
- 5:50:08>> [applause]
- 5:52:15>> Hello everyone.
- 5:52:18My name is Sibi and I'm joined on stage
- 5:52:20by Kevin.
- 5:52:22We're here to talk about how we believe
- 5:52:24that your automations are overthinking
- 5:52:27it.
- 5:52:28So
- 5:52:29to tell us a little bit about ourselves,
- 5:52:31we are from a company called UiPath. We
- 5:52:33have been doing automation for the last
- 5:52:3520 years.
- 5:52:37We have been doing RPA before and we
- 5:52:39service more than 10,000 organizations
- 5:52:42around the around the world
- 5:52:44and we build
- 5:52:45an agentic process
- 5:52:47orchestration layer which basically
- 5:52:49brings agents, robots, and people
- 5:52:51together to interact with each other and
- 5:52:54build extremely scalable
- 5:52:56automations that run for days and weeks.
- 5:52:59So
- 5:53:00we were inspired by this particular
- 5:53:03slide that you're seeing right now and
- 5:53:04you saw today today before us as well
- 5:53:08when
- 5:53:09Diego presented it three months ago, the
- 5:53:11same slide,
- 5:53:13it resonated really well with us because
- 5:53:16this is how we see automation as well.
- 5:53:19So the traditional software was is
- 5:53:22basically RPA. It's just robotic process
- 5:53:24automation, if else, just execute some
- 5:53:27task and then keeps going. The control
- 5:53:28flow is deterministic, just keeps going.
- 5:53:31And then when the LLMs came out,
- 5:53:34we all raced to go and build the
- 5:53:36just-in-time software where we have this
- 5:53:38one agent, we give it a bunch of tools,
- 5:53:40we give it a task and ask you to finish.
- 5:53:43And then we were seeing
- 5:53:44good results, bad results, and then we
- 5:53:46added evaluations to slowly build the
- 5:53:48results up.
- 5:53:49And then finally we landed on agentic
- 5:53:52process orchestration, which kind of
- 5:53:54weaves in this cognitive decision
- 5:53:57through a logical
- 5:53:59decision layer. So you have
- 5:54:01deterministic calls and clear cognitive
- 5:54:04decision points where you would use an
- 5:54:07LLM to control the control flow rather
- 5:54:10than leaving it all in the hands of an
- 5:54:11agent.
- 5:54:12So
- 5:54:13at this time we we saw where J would fit
- 5:54:16in and when Diego was talking about J,
- 5:54:19we were really excited because we saw J
- 5:54:22fit in in a very particular niche area.
- 5:54:24So
- 5:54:26here we see that
- 5:54:28J is a very powerful tool. As Diego was
- 5:54:31saying, it's a
- 5:54:33foundational tool that we would be able
- 5:54:34to integrate into all automation
- 5:54:37and it fits right in between a
- 5:54:39deterministic case and an agent case. So
- 5:54:43J is extremely fast and most of you guys
- 5:54:46would have tried it and we were
- 5:54:48surprised by how fast and how parallel
- 5:54:50we could make it work and and how
- 5:54:53intelligent it was. So we were able to
- 5:54:55benchmark it around how good it was
- 5:54:57compared to an LLM and was pretty good,
- 5:54:59not as a full LLM, but it was pretty
- 5:55:02good at in terms of intelligence as
- 5:55:03well. So it falls in between and there
- 5:55:06are so many ways we could use J and
- 5:55:09I want to hand it off to Kevin to take
- 5:55:11take you guys through a scenario where
- 5:55:14we use J in UI Path Maestro flow.
- 5:55:17>> Yeah. Thanks TV.
- 5:55:19So hi everybody.
- 5:55:21Today I want to actually walk us through
- 5:55:22a little bit about what UI Path is
- 5:55:24building with UI Path Maestro flow and
- 5:55:26talk through a little bit of how we
- 5:55:29would go through and coordinate humans,
- 5:55:31robots, and agents all together inside a
- 5:55:33one canvas.
- 5:55:34Uh what we're going to walk through
- 5:55:35today is essentially um you know
- 5:55:37essentially a flow that will demonstrate
- 5:55:40how we kind of weave all these things
- 5:55:41together.
- 5:55:42And the way that we're approaching this
- 5:55:44is that we're going to end up having
- 5:55:45system one agents which are going to be
- 5:55:47able to make quick intelligence happen
- 5:55:49within our process. Think about things
- 5:55:51like hey is this transaction fraud or
- 5:55:53not?
- 5:55:54Um
- 5:55:55additionally we'll have um
- 5:55:58system two agents running where we're
- 5:55:59going to have more of a methodical
- 5:56:00deeper investigation that's going on
- 5:56:02where agents are going off and calling
- 5:56:03tools and we'll see kind of this pattern
- 5:56:06these patterns start to emerge where
- 5:56:07we're able to hand off from one to the
- 5:56:09next.
- 5:56:10And so with that I'll I'll jump into an
- 5:56:12actual demo where we're going to
- 5:56:13demonstrate a account fraud uh scenario.
- 5:56:17And so our demo starts here with uh
- 5:56:20effectively a transaction coming in for
- 5:56:23$48,000.
- 5:56:25And this wasn't something that we were
- 5:56:27expecting and so uh as I'm reviewing my
- 5:56:29bank portal here I'm going to go ahead
- 5:56:30and report this as a fraudulent
- 5:56:31transaction.
- 5:56:33And so now this would end up smitting it
- 5:56:35off into our UI Path Maestro flow where
- 5:56:37this would act as essentially the
- 5:56:39trigger for this end-to-end workflow to
- 5:56:42to go off and start processing.
- 5:56:44What we're looking at is a collection of
- 5:56:45various different nodes that are each
- 5:56:47going to kind of uh
- 5:56:49be orchestrated from the left side of
- 5:56:51the canvas over to the right. And we're
- 5:56:52going to use uh essentially a
- 5:56:54combination of system one and system two
- 5:56:56models to actually process this uh and
- 5:56:58do an analysis of if this is fraud or
- 5:56:59not.
- 5:57:00And so the first thing that actually
- 5:57:02takes place is we're going to go through
- 5:57:04and get account activity.
- 5:57:06We're going to
- 5:57:07send this off to Jev to basically pull
- 5:57:0960 transactions for the most recent uh
- 5:57:12credit card purchases that were made and
- 5:57:14analyze each one of those individually
- 5:57:16whether or not these are fraud. And this
- 5:57:17is happening in real time where Jev's
- 5:57:19able to process these transactions and
- 5:57:21ask that question of hey is this fraud
- 5:57:23or not within fractions of a second.
- 5:57:26Now we found that there's about four
- 5:57:27transactions that recently appeared that
- 5:57:30it thinks are fraud.
- 5:57:31Now in parallel to that we're able to
- 5:57:33kick off another part of our flow, which
- 5:57:35is doing more of a semantic analysis on
- 5:57:37the entire account. We're looking at
- 5:57:39that one transaction that came in for
- 5:57:41$48,000 which was flagged, and we're
- 5:57:44asking a question to Jev is, hey, is
- 5:57:45there enough information here for us to
- 5:57:47determine whether or not this is
- 5:57:48actually fraud?
- 5:57:50And just as we heard Diogo talk about
- 5:57:52this earlier where you have these new
- 5:57:53kind of paradigms and logic come up,
- 5:57:56we ended up uh finding from this that
- 5:57:58there's uh a scoring that took place and
- 5:58:01we needed more information. Um we were
- 5:58:03asking the question, do we have enough
- 5:58:04information for fraud?
- 5:58:05And for that, what we're going to end up
- 5:58:06doing is kicking off a system two agent
- 5:58:09that's going to go spin up and run for
- 5:58:11much larger longer calling a bank API
- 5:58:14and finding that, you know, there was a
- 5:58:15$48,000 transaction from a payee uh
- 5:58:19basically creating a new account on his
- 5:58:21phone and it was earlier that day that
- 5:58:23that account got created. So, high
- 5:58:24indicator of fraud.
- 5:58:26We circle back and ask that same
- 5:58:28question now to Jev and ask, hey, do we
- 5:58:29have enough evidence now? Jev agrees and
- 5:58:32it finds 100% suspicious activity. And
- 5:58:34we would hand that off effectively to
- 5:58:36another grading model that combines all
- 5:58:38those transactions that we found with
- 5:58:40the system payload and then finds that,
- 5:58:42you know, there is fraud in this
- 5:58:43particular use case. Um uh squared of,
- 5:58:46you know, one to five we got 4.97.
- 5:58:49Now, stepping out of this um you know,
- 5:58:51typically what would happen from this
- 5:58:53point onwards is that we go off to a
- 5:58:54human to review, we'd have somebody kind
- 5:58:56of review both the system one and system
- 5:58:58two logs and then find that there was,
- 5:59:00you know, potential fraud and freeze the
- 5:59:01account.
- 5:59:02But, you know, effectively what we're
- 5:59:04seeing from customers that are going
- 5:59:05through and implementing Jev inside of
- 5:59:07their workflows is that they're able to
- 5:59:09replace the typical LLM calls that are
- 5:59:11going on behind the scenes. And in in
- 5:59:13process we're saving about 50 uh times
- 5:59:16uh we're we're increasing the speed
- 5:59:18about 50 times while also being 7.1 uh
- 5:59:22times cheaper
- 5:59:23um than calling a traditional LLM.
- 5:59:26Now, we we've gone through and and kind
- 5:59:28of shown how customers would use this
- 5:59:30within our technology, but we're also
- 5:59:32pointing the finger back inwards where
- 5:59:34we're trying to ask the of how can
- 5:59:36UiPath use this within the products and
- 5:59:38services that we offer to be able to
- 5:59:40help customers out.
- 5:59:42And there's a number of different
- 5:59:43patterns that we're kind of exploring
- 5:59:44with Jev. Um and today we're going to
- 5:59:46talk about um a re-ranking problem as
- 5:59:48well as uh what we're using inside of
- 5:59:50email mining.
- 5:59:52And so to set the stage for for kind of
- 5:59:54re-ranking, um effectively what we're
- 5:59:56trying to do is run and add uh within
- 5:59:59our retrieval augmented generation
- 6:00:01within our agents, the ability for us to
- 6:00:03find more appropriate chunks to be able
- 6:00:05to provide to our agent.
- 6:00:07And we do that by basically finding
- 6:00:09relevant documents first,
- 6:00:11and then using um typically an LLM to do
- 6:00:13this, but we're experimenting with also
- 6:00:15using Jev to go through and find the
- 6:00:17various different chunks that make are
- 6:00:19most relevant to answering the question,
- 6:00:21and then providing those as as context
- 6:00:23to the LLM.
- 6:00:24And we've run a couple of
- 6:00:26experiments on this now, and I'd like to
- 6:00:27hand it back off to Chibi who's going to
- 6:00:29end up telling us a little bit more
- 6:00:30about what we were able to find.
- 6:00:33>> Yeah.
- 6:00:33So this is where things got really
- 6:00:35interesting for us. This is like the
- 6:00:37first week of Jev. We had got our hands
- 6:00:39on Jev.
- 6:00:40And we have these extremely strong
- 6:00:43benchmarks and uh eval sets that we had
- 6:00:46for re-ranking that we have been running
- 6:00:48for some time. And we need those for
- 6:00:50like when we change from one model to
- 6:00:52another. Uh and we realized, okay, why
- 6:00:55not
- 6:00:56take these eval sets and build uh
- 6:00:59hill climbing harness and ask Astra
- 6:01:03to say, okay, this is my score I can get
- 6:01:06with an LLM. Here is Jev. Here are the
- 6:01:09three APIs.
- 6:01:10Find a strategy which can achieve that
- 6:01:13score.
- 6:01:14And let it run.
- 6:01:15And it ran for the entire night for 14
- 6:01:18hours in a session. And it came up with
- 6:01:21so many different strategies where it
- 6:01:23worked, where it didn't work.
- 6:01:24And then we were slowly seeing the
- 6:01:26number go up. Every two hours there
- 6:01:28would be one strategy would perform
- 6:01:30slightly better.
- 6:01:31This was so surprising for us because
- 6:01:33what we thought was as
- 6:01:36as an API, if you just call it like when
- 6:01:38you just send all of the go up
- 6:01:41up.
- 6:01:41Yeah. So we call it uh we were getting
- 6:01:44only point 866 which was very very low
- 6:01:48at that point when we were trying it out
- 6:01:50when we were thinking how do we increase
- 6:01:51it and then it slowly got got to the
- 6:01:54number of point 889 while it was uh
- 6:01:57trying out new strategies and that
- 6:01:59strategy is called tournament and I want
- 6:02:01to showcase that strategy which it which
- 6:02:04these hill climbing models actually come
- 6:02:06up with and how you were able to see the
- 6:02:08intelligence squeaked out of Jev just by
- 6:02:11applying Jev in a different architecture
- 6:02:14and different call method and I want to
- 6:02:16take you through tournament now.
- 6:02:19So what tournament did is that it
- 6:02:21actually reimagined re ranking
- 6:02:23differently instead of looking at every
- 6:02:25chunk and saying that it is how good
- 6:02:28relevant is it for my question it
- 6:02:30created cohorts of different uh
- 6:02:34different groups of these chunks and
- 6:02:36made them fight between each other to be
- 6:02:38the best most relevant chunk by sending
- 6:02:40thousands and thousands of Jev calls for
- 6:02:43each and every one of them and it found
- 6:02:44leaders in those particular cohorts and
- 6:02:48then the winners would be promoted to
- 6:02:49the next round and then in from those
- 6:02:51winners we would again find like top
- 6:02:53winners from those chunks and then we
- 6:02:55will slowly increase and find the
- 6:02:58topmost the cream will rise to the top
- 6:03:00and you will find the
- 6:03:02the most relevant chunks on the whole
- 6:03:03list.
- 6:03:04So this worked extremely well and we
- 6:03:07were testing it and when we were
- 6:03:09surprised to see that even with
- 6:03:10thousands of Jev calls and
- 6:03:14many millions of contacts being used
- 6:03:17we were still under a second we were
- 6:03:19able to make all of these calls right
- 6:03:22and this was a huge revelation for us
- 6:03:24and we realized that Jev is a building
- 6:03:27block that we can adjust and we are we
- 6:03:31have to find these new strategies where
- 6:03:33we would be able to apply this building
- 6:03:35block and it might look less intelligent
- 6:03:38on its own but with these strategies and
- 6:03:41patterns, we can get more intelligent
- 6:03:43with even the current gem model that we
- 6:03:45have, right? And that's that gave us
- 6:03:48like 15 x speed and we tested it out on
- 6:03:51uh our our live uh
- 6:03:54indexes and saw how it performed and
- 6:03:56people were extremely surprised by this
- 6:03:58one. And next we saw how uh Jev could be
- 6:04:02used uh
- 6:04:04for mining. So mining is has a lot of
- 6:04:07stages. I'm going to talk only about the
- 6:04:09classification stage. And what happens
- 6:04:12is we have customers who have millions
- 6:04:14of emails in uh support uh email box.
- 6:04:18And what happens is they want to analyze
- 6:04:20and they want to dig into what happened
- 6:04:21in these emails so that they can take
- 6:04:23some actions. I'm sure every one of you
- 6:04:25guys have one of those email boxes. Uh
- 6:04:27what happens is that like it's so hard
- 6:04:30to find the right categories and
- 6:04:31bucketizations without reading all of
- 6:04:33these emails. So what we used to do is
- 6:04:36that we would go and create uh
- 6:04:38fine tuning to figure out a model that
- 6:04:40classifies them and it would take hours
- 6:04:42and maybe a day to fine tune that
- 6:04:44particular model for the classification
- 6:04:46story. But what we found was this is
- 6:04:48again through hill climbing we found
- 6:04:50this pattern where we use
- 6:04:52we use an LLM to find an initial
- 6:04:55candidate list of categories.
- 6:04:58And then it would say that okay these
- 6:05:00are all the 15 categories that I think
- 6:05:02the emails will fit. Then what we would
- 6:05:04do is that we'll use Jev and then all
- 6:05:06the million emails through it and see if
- 6:05:09where the buckets fill up.
- 6:05:11And with that we can actually see which
- 6:05:13buckets
- 6:05:14are not filled properly, which buckets
- 6:05:16are overfitted, which buckets are never
- 6:05:18filled out. And then we ask the LLM to
- 6:05:20go back and optimize those buckets
- 6:05:22again. And then once the buckets are
- 6:05:25rearranged, we'll do another
- 6:05:27classification of all the all the
- 6:05:28millions of emails one more time.
- 6:05:31Because Jev is so cheap, we are able to
- 6:05:33do this multiple times and eventually
- 6:05:36we'll end up with the category list
- 6:05:38which is extremely fit towards the
- 6:05:39mailbox that you are dealing with
- 6:05:41because it's not the same across
- 6:05:43different in different mailboxes the
- 6:05:44categories are completely different.
- 6:05:46But the powerful thing is once you did
- 6:05:48all of this work with an LLM going
- 6:05:50through and finding these categories,
- 6:05:52you don't need the LLM anymore
- 6:05:54because you have the categories which
- 6:05:56you where you have distilled down all
- 6:05:58the intelligence into these categories.
- 6:06:00Now you can just build a maestro flow
- 6:06:03which basically takes just the
- 6:06:05categories alone and uses gem. So
- 6:06:08whenever a new email comes, it only
- 6:06:10needs gem. It just looks at the looks at
- 6:06:13the email and fits the categories and we
- 6:06:15basically have a list of
- 6:06:18follow through. So if this matches this
- 6:06:19category, you are going to get an email.
- 6:06:21This team is going to get an email or
- 6:06:22that team is going to get an email so
- 6:06:24easily and we basically removed the LLM
- 6:06:26completely from the picture on this one.
- 6:06:29So and we saw a humongous speed increase
- 6:06:32on on just email processing. It was
- 6:06:34almost like at real time an email comes
- 6:06:36and suddenly somebody will get notified.
- 6:06:38So
- 6:06:39that was an amazing
- 6:06:41amazing thing to see as well. So over
- 6:06:43the few months we were so excited to
- 6:06:46play around with gem and every day the
- 6:06:48slack threads would light up everywhere
- 6:06:50in the in the company that I found this
- 6:06:52pattern, I found that pattern and it was
- 6:06:54such an exciting time during these last
- 6:06:57three months and I wanted to share that
- 6:07:00there are so much more patterns and
- 6:07:02strategies to be found.
- 6:07:04We are only scratching the surface of
- 6:07:06this and I want us to
- 6:07:10try out and like share the strategies
- 6:07:11that we all find and collaborate here.
- 6:07:14>> And if you're interested in trying out
- 6:07:15UI Path Flow, we have it available for
- 6:07:18free right now. Scan the QR code on the
- 6:07:19screen here and we'll be able to it'll
- 6:07:21take you into a portal where you can
- 6:07:22basically point your coding agent at it
- 6:07:24and get complete access to build your
- 6:07:25first automations there. Thank you for
- 6:07:27your time everyone.
- 6:07:28>> Thank you.
- 6:07:30>> [applause]
- 6:08:39>> This
- 6:11:39>> [snorts]
- 6:11:43>> All right. Good afternoon, everybody. I
- 6:11:46am Brian Kennedy. I'm an engineer on
- 6:11:49Zoom Info's platform.
- 6:11:52Now,
- 6:11:53I want to start with showing you
- 6:11:55something one of our agents wrote.
- 6:11:57This is a real output from our product.
- 6:11:59The agent's job was to schedule an email
- 6:12:03for a customer.
- 6:12:04And this is what it reported back.
- 6:12:07It said [clears throat] it scheduled the
- 6:12:08email, it queued it up, and that it was
- 6:12:10going to go out on Tuesday.
- 6:12:12We don't just trust the agent, though. A
- 6:12:14second model checks the agent's work.
- 6:12:16I'm going to call this one the judge.
- 6:12:18The judge read this and it accepted it.
- 6:12:21Great.
- 6:12:23Now, here's the problem.
- 6:12:25And this is what it actually happened.
- 6:12:27Their these are the tool results, the
- 6:12:30record of what actually ran when the
- 6:12:31agent called the tools. I'm going to
- 6:12:33call these the receipts.
- 6:12:35The agent tried to create a record in
- 6:12:36Salesforce and it failed. The error
- 6:12:38message that came back says the record
- 6:12:40was not created and it tells the agent
- 6:12:43not to report a success or even make up
- 6:12:45a record.
- 6:12:46We wrote this message ourselves for
- 6:12:48exactly this situation.
- 6:12:50The agent tried again. It created a
- 6:12:52task, which is a to-do.
- 6:12:55No email was ever scheduled. The agent
- 6:12:58doesn't even have a tool that can send
- 6:12:59an email. The agent made it up and the
- 6:13:01judge approved it.
- 6:13:03So that's two failures.
- 6:13:05The agent fabricated a send
- 6:13:07and the judge rubber stamped it,
- 6:13:09approved a claim that never actually
- 6:13:12checked.
- 6:13:13So if you build agents, you probably
- 6:13:15have run into this and you probably have
- 6:13:16a judge like this. It might be a code
- 6:13:18reviewer or a grader of your evals or a
- 6:13:20supervisor in your loop.
- 6:13:22Anytime a model decides whether another
- 6:13:25model's work is good, this can happen.
- 6:13:28About for the next 10 minutes, I'll show
- 6:13:30you how we caught this and what we did
- 6:13:31about it.
- 6:13:33First, some quick background.
- 6:13:35My team at ZoomInfo
- 6:13:38is called Agent Teams. We announced this
- 6:13:40last week. It's a multi-agent
- 6:13:42orchestration layer that lives in
- 6:13:44ZoomInfo's GTM Studio.
- 6:13:46It coordinates a team of specialized
- 6:13:48agents across the company's systems to
- 6:13:50run GTM workflows. For example, find
- 6:13:53companies, research them, write an
- 6:13:55email, log it into their CRM. Each agent
- 6:13:58has its own roles and instructions like
- 6:14:00and tools like Salesforce, HubSpot,
- 6:14:02Slack.
- 6:14:03These agents make real uh decisions and
- 6:14:06take real actions in the customer's
- 6:14:08systems. They write records and they
- 6:14:10send messages.
- 6:14:12Every team and every agent finishes a
- 6:14:15step. A second model checks its work.
- 6:14:18That's the judge and it has three
- 6:14:20options to move on, to redo the step, or
- 6:14:24to stop the job altogether.
- 6:14:26You probably heard this called LLM as a
- 6:14:28judge. It's the same idea as a code
- 6:14:30reviewer or an eval grader.
- 6:14:33Now, the hard part for us is that it's
- 6:14:36one model and it has to grade every kind
- 6:14:38of agent that we're running through it.
- 6:14:40Our customers build their own workflows
- 6:14:42with any agent and any tool in any
- 6:14:44order, so the permutations are almost
- 6:14:46infinite.
- 6:14:47So the list of jobs for the judge has to
- 6:14:49grow with it.
- 6:14:51The judge can tell whether the work what
- 6:14:54looks finished, but what it can't tell
- 6:14:56is whether the work was right because
- 6:14:58that depends on what right means for
- 6:15:00each job and we're asking it the same
- 6:15:03question for all of them.
- 6:15:06So here's where we're going.
- 6:15:09First we'll talk about what we found
- 6:15:10when we checked the judge.
- 6:15:13Then what we chose instead
- 6:15:15and what you can do today with your own
- 6:15:17agents.
- 6:15:21To start, we ran offline evals. We took
- 6:15:24500 real steps from production,
- 6:15:27each with a decision that the judge had
- 6:15:28made.
- 6:15:30Then we took a larger frontier model and
- 6:15:31regraded every one of those to come back
- 6:15:34with an accept or send back.
- 6:15:36Then we hand checked each of those
- 6:15:38samples and we checked the grades. Where
- 6:15:40we disagreed with the model, we went
- 6:15:42with our call.
- 6:15:44This gave us a reference label for all
- 6:15:45500 steps.
- 6:15:49And it turned out the judge was right
- 6:15:5165% of the time, so more or less a coin
- 6:15:54flip.
- 6:15:55It accepted 118 steps that it should
- 6:15:58have sent back. That's more than one in
- 6:16:00four of everything that it accepted.
- 6:16:04Pretty bad.
- 6:16:05And when we read through those, the most
- 6:16:07common problem was that
- 6:16:09similar to the email at the start, the
- 6:16:11agent reported that it did something
- 6:16:14like send a message and no tool call had
- 6:16:17ever done that.
- 6:16:20So to see why the judge had accepted it,
- 6:16:22let's go back to the email example and
- 6:16:24look at what the judge was given.
- 6:16:26On the left, you have what the agent's
- 6:16:28own summary Think of it as like a story.
- 6:16:30It's confident, it's well written. It
- 6:16:33says the email was scheduled.
- 6:16:35Now on the right, you have the receipts.
- 6:16:37This is the actual tool results. There
- 6:16:39are errors, failed to create messages
- 6:16:43telling the agent not to make up the
- 6:16:44record.
- 6:16:45Now the judge had both of these and it
- 6:16:47went with the story. The work looked
- 6:16:49fine on the surface, so it accepted it.
- 6:16:54Now this isn't only our judge. You can
- 6:16:56think of a code review agent that reads
- 6:16:59the PR description before it looks at
- 6:17:01the diff. It's a similar kind of
- 6:17:02problem. If the description says I fixed
- 6:17:05the bug, added tests, the agent is going
- 6:17:07to lean on that when it reads the code.
- 6:17:10So the obvious fix at the start was to
- 6:17:12tune the judge and write a better
- 6:17:14prompt. For us, that was going to be a
- 6:17:16significant amount of work. Our agents
- 6:17:18write sales emails, update CRMs, do
- 6:17:20research,
- 6:17:21and to judge each of these well, we'd
- 6:17:23have to figure out what it was that the
- 6:17:26user was asking. Give the judge the
- 6:17:28right rules for every kind of work and
- 6:17:31keep it all up to date.
- 6:17:32That's basically a custom judge for
- 6:17:34every use case.
- 6:17:36What we went with, we went a different
- 6:17:37direction. We stopped asking one big
- 6:17:40general question, is this good? And we
- 6:17:42started asking a short list of small yes
- 6:17:45or no questions.
- 6:17:47They're the same questions for every
- 6:17:48step and each one is small enough to
- 6:17:51answer by looking at the receipts.
- 6:17:54The first option here is basically a
- 6:17:56prompt tuning project that never ends,
- 6:17:58whereas the second one is a small
- 6:18:00handful of small checks.
- 6:18:03Now, to answer those questions, we used
- 6:18:05Jev. You've heard a lot about Jev today,
- 6:18:07so I'll skip that and keep that short.
- 6:18:10It's a type decision model, not a
- 6:18:11generative model.
- 6:18:13Uh for mom at home watching, just ask
- 6:18:14Claude.
- 6:18:16And we'll send it structured facts, you
- 6:18:18know, about the step and a listed typed
- 6:18:21questions.
- 6:18:22A typed question is a yes or no question
- 6:18:24or a choice with from a fixed list of
- 6:18:27options. It sends back a probability for
- 6:18:30each answer and it answers all the
- 6:18:32questions in one call.
- 6:18:36We gave it four things, the
- 6:18:37instructions, the uh input, the
- 6:18:40receipts, and meaning every tool call,
- 6:18:42and whether it was successful or it
- 6:18:44failed, and the agent's report.
- 6:18:47Then we asked for one choice, which was
- 6:18:49accept, redo, or stop.
- 6:18:52And that choice makes the decision.
- 6:18:55But then we also asked seven yes or no
- 6:18:57questions. Those seven questions tell us
- 6:19:00the why.
- 6:19:01They decide between a redo and a stop,
- 6:19:04and they go into the feedback that the
- 6:19:06agent gets when it actually has to redo
- 6:19:08a step.
- 6:19:11Now, back to the email. The LLM judge
- 6:19:14accepted it, and it took about two and a
- 6:19:16half seconds.
- 6:19:17Jev gave us a probability for each
- 6:19:19option.
- 6:19:21For this step, it accepted 0%, redo 97%
- 6:19:26and stop at 3%.
- 6:19:28And it took about a fifth of a second.
- 6:19:31We make the decision from the
- 6:19:32probability of accept. At 30% or higher,
- 6:19:36we accept that. At 5% or lower, we redo
- 6:19:39or stop.
- 6:19:41And anything in between means Jev isn't
- 6:19:43sure, so we kick it to an LLM judge.
- 6:19:46We pick those cutoffs from testing the
- 6:19:47500 labels that we had at the beginning.
- 6:19:52Now, to test whether what whether or not
- 6:19:54this works better, we took two different
- 6:19:56routes. The first test used the same 500
- 6:19:59production steps from the offline eval.
- 6:20:02Jev was confident enough in about seven
- 6:20:05out of 10 of them.
- 6:20:06One of those steps,
- 6:20:08uh, the LLM judge had accepted 82 that
- 6:20:11should have been sent back. Each red
- 6:20:13square here is one of those.
- 6:20:15Jev successfully caught 74 of those.
- 6:20:19The blue squares are the ones that Jev
- 6:20:20would have sent back.
- 6:20:25Now the second test is what we called
- 6:20:27shadow mode.
- 6:20:28Both judges look at every step, but only
- 6:20:31the LLM judge, uh, that decision counts.
- 6:20:34So we saved Jev what would have done and
- 6:20:37we don't let it affect the actual run.
- 6:20:39That gave us 37,000 decisions to mine
- 6:20:42for insights.
- 6:20:43And in that time, there were 728 steps
- 6:20:47where a tool had clearly failed.
- 6:20:50The LLM caught
- 6:20:51294 of them. Jev caught 623.
- 6:20:56And the fake email from the beginning,
- 6:20:58it wasn't a one-off thing either. The
- 6:21:00same step ran 1,081 times and almost
- 6:21:03every time the agent said it had
- 6:21:05scheduled an email, the LLM judge only
- 6:21:07caught it once. Jev caught that 876
- 6:21:10times.
- 6:21:14And when you have a judge, every step
- 6:21:16waits. Jev was 10 times faster and 40
- 6:21:19times cheaper.
- 6:21:20At this scale, it is massive.
- 6:21:25[snorts]
- 6:21:26So what's next?
- 6:21:28Right now, every step goes with the same
- 6:21:30seven questions. They go into Jev along
- 6:21:32with the facts and Jev gives us an
- 6:21:34accept, redo, or stop.
- 6:21:37But next, we're adding post-condition
- 6:21:39checks,
- 6:21:40which are questions that depend on which
- 6:21:41tool called, uh, was within the step.
- 6:21:45So we don't have to guess. Uh, we know
- 6:21:46the step up front because the tool tells
- 6:21:48us.
- 6:21:49If a step's called, let's say, a send
- 6:21:51tool, we also ask whether the message
- 6:21:53was delivered.
- 6:21:54If it wrote a record, we ask whether the
- 6:21:57record exists.
- 6:21:58And those go into Jev with the same
- 6:22:00seven questions in the same call.
- 6:22:04Now, what can you today with your
- 6:22:06agents?
- 6:22:08First, pick one judgment. Somewhere in
- 6:22:10your systems, there is an LLM deciding
- 6:22:12whether or not something is good. It
- 6:22:14might be reviewing code, grading an
- 6:22:16eval. Pick one of those and rewrite it
- 6:22:18as a few simple yes or no questions that
- 6:22:20can be answered from the evidence.
- 6:22:23Second is build it.
- 6:22:24This is the easy part, honestly. We used
- 6:22:26Jev and ours was seven questions.
- 6:22:29Then the third, ship it in shadow mode.
- 6:22:32Run it next to what you already have,
- 6:22:33but don't let it make the decisions yet.
- 6:22:35Just save it and then you can analyze it
- 6:22:38later.
- 6:22:39Then read the cases and see where the
- 6:22:41two disagree. Don't overthink it, just
- 6:22:44start small.
- 6:22:47We had trusted our judge until we
- 6:22:48measured it.
- 6:22:50Measure yours. Run a second check. Run
- 6:22:52it in shadow mode and see where it
- 6:22:54disagrees. You may be surprised with
- 6:22:56what you find.
- 6:22:57Thank you.
- 6:22:59>> [applause]
- 6:24:48>> Hey everyone.
- 6:24:50I promise I'm the last person on the
- 6:24:52hype train for Jev.
- 6:24:54I'm Scott.
- 6:24:55I'm from I'm from Workday and I'm going
- 6:24:58to go through how we're experimenting
- 6:25:00with Jev to drive new innovations in HR
- 6:25:05and finance tasks.
- 6:25:07Exciting stuff. So
- 6:25:10at Workday, we have over 11,000
- 6:25:12different organizations that use our
- 6:25:14tools.
- 6:25:15And a lot of the problems that we deal
- 6:25:17with break down into kind of two
- 6:25:19different categories. So one is very
- 6:25:21quick actions they have to take. I want
- 6:25:23to see my pay. I want to ask for a day
- 6:25:25off. The other ones are I have to dive
- 6:25:28into a ton of data, weed through it, try
- 6:25:30to figure out what's going on.
- 6:25:33And like many of your companies as well,
- 6:25:34we're trying to, you know, put agentic
- 6:25:36workflows around this.
- 6:25:38And so if you look at just a a very
- 6:25:40simple agentic loop, you know, we have a
- 6:25:43user request. We try to decide what to
- 6:25:45do. We're running tools. We're judging
- 6:25:47did we do a good job?
- 6:25:49And then over and over again until we
- 6:25:51finally get a result.
- 6:25:53Now one of the problems here is that on
- 6:25:55each one of those we're doing LLM calls.
- 6:25:58And there's two major problems with
- 6:26:00that.
- 6:26:01So one is there's too much intelligence.
- 6:26:03So if we look at the system one versus
- 6:26:05system two thinking,
- 6:26:08Uh we're doing a lot of slow,
- 6:26:09deliberate, very expensive thought
- 6:26:11processes at each one of those steps.
- 6:26:16Another problem is is just the wrong
- 6:26:18tool for the job.
- 6:26:20So, you know, LLMs a lot of times are
- 6:26:22just a hammer uh looking for a nail.
- 6:26:25And, you know, we want to figure out
- 6:26:27like what do we actually need uh because
- 6:26:29clearly LLMs are not doing it.
- 6:26:32So we want something that is fast,
- 6:26:35uh smart, easy to use,
- 6:26:39uh it's generalized, and it's very very
- 6:26:41cheap.
- 6:26:44So when we look at like from a code
- 6:26:46perspective, you know, what are we
- 6:26:47really looking at from a from a Jev-like
- 6:26:50model? It's really just a conditional.
- 6:26:52If this happens, this should happen.
- 6:26:55Which one of these cases are the ones
- 6:26:57that I want to explore?
- 6:27:00And
- 6:27:01you know, you've seen it among many of
- 6:27:03these presentations already, uh but Jev
- 6:27:05has the primitives to allow us to do
- 6:27:07this. Uh there's the probability. Is it,
- 6:27:09you know, this or that and what's the
- 6:27:11probability? Uh what are the different
- 6:27:13categories that this uh particular thing
- 6:27:15could fall into?
- 6:27:17And then you have the uh the scoring
- 6:27:19mechanism, which is basically
- 6:27:20probability and sort it.
- 6:27:24So
- 6:27:25you know, this has been brought up as
- 6:27:27well. You know, my first thought when I
- 6:27:28saw, you know, the Jev models was, well,
- 6:27:31we already have classifiers. I've been
- 6:27:33doing this for a long time.
- 6:27:34How's it actually different?
- 6:27:37And as many of the speakers have talked
- 6:27:38about already, you know, with
- 6:27:40classifiers,
- 6:27:41I need to figure out, you know, what's
- 6:27:43my golden data set? I have to train it.
- 6:27:46New data comes out. I have to iterate
- 6:27:49over it over.
- 6:27:50With Jev, just out of the box, it just
- 6:27:52worked. We plugged in our question. We
- 6:27:55classified it. We didn't really have to
- 6:27:57do anything. It generalized to our
- 6:27:58question.
- 6:28:00Uh and so we started using it for a lot
- 6:28:02of the use cases that you've seen from
- 6:28:04some of the presenters already. Uh so we
- 6:28:06use it for routing. That's kind of the
- 6:28:07canonical example.
- 6:28:09Um you know, figuring out did this step
- 6:28:11work, you know, as a judge.
- 6:28:13Uh, we looked at uh,
- 6:28:16you know, resumes, you know, is this
- 6:28:18person qualified for ML work?
- 6:28:22Uh, and I want to do kind of an overly
- 6:28:24simplified example as well. Um, so no
- 6:28:27one taught Jev like what a fish is. And
- 6:28:29so this is actually a question we asked,
- 6:28:31like is a fish a pet? Well, 50/50 might
- 6:28:34be a pet sometimes, might not be. Um,
- 6:28:37and so you know, if I ran into this,
- 6:28:40then I have to figure out
- 6:28:41um, what do I do next? Maybe I can give
- 6:28:44it some more context. Oh, is a fish in a
- 6:28:45bowl a pet? You know, and after 20
- 6:28:48milliseconds I can ask this again. All
- 6:28:50right,
- 6:28:50that's great. Oh, is a fish on my plate
- 6:28:53a pet? Clearly not. Uh, so this is
- 6:28:56something that wasn't trained out of the
- 6:28:57box in seconds. It just worked.
- 6:29:02All right, so now on to some of the
- 6:29:03demos that we're that um,
- 6:29:05you know, we've done at Workday. So just
- 6:29:09a caveat on some of these um, it's not
- 6:29:11real data, so we're not using any of
- 6:29:13your, you know, resumes or payroll data.
- 6:29:16Uh, these are also not in production now
- 6:29:18and may or may not be in the future. So
- 6:29:20that's something our legal asked me to
- 6:29:22say. So let's get into, you know, some
- 6:29:24of the
- 6:29:25the demos.
- 6:29:26All right, so starting with
- 6:29:28autocomplete. So autocomplete starts to
- 6:29:31predict what you're trying to ask. It's
- 6:29:33just predicting the question. It's not
- 6:29:35helping you get to the answer. So as you
- 6:29:38start to type more, it's kind of guess
- 6:29:40like, oh, I want to figure out if this
- 6:29:43person wants to take a day off work.
- 6:29:44Great, you've guessed the question, but
- 6:29:46you really haven't
- 6:29:47you know, helped me get to the answer.
- 6:29:50So what we want to do is move from just
- 6:29:52predicting the word or the next block of
- 6:29:54code and actually help you finish the
- 6:29:56task.
- 6:29:58And so this is best shown with a a demo
- 6:30:00that we have where we go from predicting
- 6:30:03what you want to say to predicting what
- 6:30:06you want to do.
- 6:30:07So hopefully the video plays. There we
- 6:30:09go. All right, so I need to take time
- 6:30:12off of off of uh, next week. And as I
- 6:30:16start to ask this, Jev is actually
- 6:30:18figuring out under the hood, okay, what
- 6:30:21what more context do I have? How can I
- 6:30:24adjust this UI based on the information
- 6:30:27that I'm that I'm getting? This is not
- 6:30:29something that we could have done with
- 6:30:30an LLM. We're getting you know, we're
- 6:30:33we're doing this you know, in a loop
- 6:30:35while someone's asking.
- 6:30:36Um the payslip. You know, you shouldn't
- 6:30:39have to go in and ask an agent, hey, can
- 6:30:41you go find out how much I was paid? Uh
- 6:30:44what you can do is just in the search
- 6:30:46say, what was my pay last period?
- 6:30:50Pops up an interface, allows you to
- 6:30:53unhide it,
- 6:30:54you know, see what your pay was in the
- 6:30:55last period.
- 6:30:57Very simple. Gets you to the action.
- 6:31:00Book a meeting.
- 6:31:02So I know that I want to book a meeting
- 6:31:05and uh I'm going to want to do it for
- 6:31:08next Friday.
- 6:31:10Let me do it at 2:00 p.m. Oh, I want to
- 6:31:13do it with my boss. Make sure that I get
- 6:31:15my you know, my boss on this. Uh Logan
- 6:31:18Mc Neil. There we go. Meeting's booked.
- 6:31:22And then a final one that we have just
- 6:31:23for this example, but there's many more,
- 6:31:25uh
- 6:31:27is finding a person.
- 6:31:29So in this one, I may have some semantic
- 6:31:31knowledge, like find the person who owns
- 6:31:33this part of the organization or just
- 6:31:36like who's Ashanti? I want to contact
- 6:31:38her. Uh pop it up, click on email,
- 6:31:42good to go.
- 6:31:46All right. So what's happening under the
- 6:31:47hood?
- 6:31:48So because Jev is fast and cheap,
- 6:31:51uh just like a type ahead, we could
- 6:31:54start to render a UI on this on that
- 6:31:56same cadence. We can look at as the
- 6:31:59words are coming, do a prediction of
- 6:32:01intent and start to render an an
- 6:32:03interface.
- 6:32:05And you know, we could do this at you
- 6:32:06know, about 20 bucks for a million
- 6:32:08judgments.
- 6:32:09Uh and the way that we're doing it is
- 6:32:11kind of twofold. Um or two parts, I I
- 6:32:14say.
- 6:32:15So first, Jev is picking the card of
- 6:32:18saying what's the intent? All right,
- 6:32:19there's a card that's going to help you
- 6:32:20out.
- 6:32:21And then the team that's responsible in
- 6:32:23that domain then can decide what are the
- 6:32:25questions to progressively fill out that
- 6:32:28card. And so there's sort of this
- 6:32:29two-step process.
- 6:32:31Uh the first step is getting people in
- 6:32:33the front door, getting the right card
- 6:32:35up. And then the second one is making
- 6:32:37sure that you fill in that card to help
- 6:32:39them accomplish the tasks that they're
- 6:32:41after.
- 6:32:42Now we have an eval on here before we
- 6:32:45merge any card. We want to make sure
- 6:32:46that it doesn't collide with any other
- 6:32:48one. And at least in our initial
- 6:32:50experiments, you know, we've we've done
- 6:32:52seven cards or a just chat because it
- 6:32:55should if there's no card to actually
- 6:32:57solve it, it should allow you to just
- 6:32:58continue chatting.
- 6:32:59Uh we were able to get, you know, 96%
- 6:33:01accuracy on that.
- 6:33:05All right, so now onto the recruiter. So
- 6:33:07let's talk about the recruiter problem.
- 6:33:08All right, so this is not like a fast uh
- 6:33:11a task. It's something where recruiter
- 6:33:13has a pile of hundreds of resumes and
- 6:33:16has to figure out who has the best
- 6:33:18resume to accomplish the
- 6:33:20uh or fill the wreck that they're trying
- 6:33:22to fill.
- 6:33:23Now I first want to look at like how we
- 6:33:25would have done this before. Uh we would
- 6:33:27have done, you know, an embedding, put
- 6:33:30the vectors in a vector DB, then had a
- 6:33:33retrieval layer, then have a re-ranker,
- 6:33:35and then figure out based on the
- 6:33:37re-ranker which ones that we want to put
- 6:33:39in front of uh the recruiter.
- 6:33:41Uh one interesting call out is the Quinn
- 6:33:44re-ranker
- 6:33:45uh looks surprisingly similar to uh
- 6:33:48the uh the Jev process as well.
- 6:33:51Uh and so that that's kind of an
- 6:33:53interesting corollary if you've used
- 6:33:54that one before.
- 6:33:56Uh but with Jev, we basically take out
- 6:33:59all the embedding steps.
- 6:34:00I don't have to train anything up. I
- 6:34:02don't have to do embedding. I don't have
- 6:34:04to put things in a vector database.
- 6:34:06Um
- 6:34:07what I can do is just do a function on
- 6:34:11my database. So put all the resumes
- 6:34:14into a database,
- 6:34:16uh use natural language to query it,
- 6:34:18iterate over every single one of them
- 6:34:21and then just figure out which ones
- 6:34:23match my criteria.
- 6:34:25So I'll go into
- 6:34:27you know an actual demo of this exact
- 6:34:29same one.
- 6:34:30So here we have a select statement and
- 6:34:32so this is against DuckDB where we have
- 6:34:35a function
- 6:34:37that is going to call Jev to figure out
- 6:34:40what's the probability that someone
- 6:34:41matches this criteria.
- 6:34:43We're going to get it a score and then
- 6:34:44we're going to order by that score. Now
- 6:34:46I'm going to do about 20 and as I said
- 6:34:48these are all fakes and none of your
- 6:34:50resumes are in here I promise.
- 6:34:52And then here we see these are very ML
- 6:34:55focused synthetic data as well. So it's
- 6:34:58not a surprise that we would have very
- 6:35:01high hit rates against someone who's
- 6:35:02worked in fraud, trust, and safety
- 6:35:04problems.
- 6:35:06But let's look at it. Let's change this
- 6:35:07question. Let's be a little bit more
- 6:35:09adversarial on this one.
- 6:35:11So a lot of these resumes have the word
- 6:35:13pipeline in them.
- 6:35:15So whether it's a data pipeline or an AI
- 6:35:18pipeline
- 6:35:19pipe's going to come out. So let's just
- 6:35:21ask like is this person qualified to do
- 6:35:23plumbing? If you were going to just do a
- 6:35:24lexical search it might pick up some of
- 6:35:26these. So we put it into Jev and asked
- 6:35:29you know of all these ML
- 6:35:31resumes let's trim the limit down to
- 6:35:34five here.
- 6:35:36Go ahead and run it.
- 6:35:38And we could see you know that
- 6:35:43very very low probability on these. So
- 6:35:45these are ones that we would almost
- 6:35:46definitely throw out. And you could see
- 6:35:48the speed. So we went over 117 records.
- 6:35:51We had 2.3 seconds of wall time and
- 6:35:55spent almost nothing on this query to
- 6:35:56just iterate over everything.
- 6:35:59So Jev really kind of changed the game
- 6:36:01for us because every every place where
- 6:36:03we're thinking either this is too
- 6:36:05expensive or it's too slow we thought of
- 6:36:07new places to inject Jev into our
- 6:36:09pipeline.
- 6:36:11And so [snorts] just to summarize you
- 6:36:12know the two areas that I went through
- 6:36:15are one
- 6:36:16where can we decide very quickly and get
- 6:36:19to the intent and get the person on
- 6:36:21their way.
- 6:36:22And then the second one is to cut down
- 6:36:24on some of the, you know, really deep
- 6:36:27work. How do we filter those out for,
- 6:36:29you know, our customers?
- 6:36:31And so I hope this that this gave you a
- 6:36:33little bit of insight into the things
- 6:36:35that you can take back to, you know,
- 6:36:36your own companies and how in, you know,
- 6:36:38imagine the things that you can do with
- 6:36:40Jev and decision models as well.
- 6:36:43And that's my time. Thank you very much.
- 6:36:46>> [applause]
- 6:37:54>> Hey folks, this is Girish,
- 6:37:56a principal software engineer at
- 6:37:58Microsoft.
- 6:37:59Outside my job, I build open source
- 6:38:01tools for AI agents.
- 6:38:03One of them is mcb pacemaker,
- 6:38:05a supervised bridge that keeps mcb
- 6:38:08servers running.
- 6:38:10Before we start, a small disclosure.
- 6:38:12The views in this talk are my own and
- 6:38:15doesn't represent my employer.
- 6:38:17So sub agents for distributed system.
- 6:38:21Over the past year, I have been using
- 6:38:22agents and sub agents on real
- 6:38:24engineering work.
- 6:38:25Dev sessions, code reviews, instant
- 6:38:28thrashing,
- 6:38:29what not.
- 6:38:31This talk is about what happens when one
- 6:38:34agent hands work to another.
- 6:38:38So here's an architecture.
- 6:38:41The orchestrator splits the task and
- 6:38:43hands the pieces
- 6:38:44to three different sub agents.
- 6:38:47They all run at the same time.
- 6:38:50They all read from the same shared
- 6:38:52state,
- 6:38:53conversation,
- 6:38:54memory files,
- 6:38:55repo, working trees,
- 6:38:58issues, vector stores.
- 6:39:01And
- 6:39:02they don't just think, they make tool
- 6:39:04calls
- 6:39:05to real systems
- 6:39:08which touch databases, APIs, CI tools,
- 6:39:11deployment pipelines.
- 6:39:13And
- 6:39:15the results come back and orchestrator
- 6:39:18merges them into a single answer.
- 6:39:20An orchestrator
- 6:39:22workers
- 6:39:23shared mutable state
- 6:39:24and results merge from concurrent
- 6:39:26executions.
- 6:39:27This represents a classic distributed
- 6:39:29system.
- 6:39:31I have known how to reason with
- 6:39:32distributed systems for decades,
- 6:39:35but none of this setup
- 6:39:37came with it.
- 6:39:39What you get is a function that takes a
- 6:39:41string and returns a string.
- 6:39:44Failures we hit are not new, are not
- 6:39:47specific to AI agents.
- 6:39:49They are classic distributed system
- 6:39:50failures.
- 6:39:53Partial failure,
- 6:39:55a normal call that fails halfway through
- 6:39:58and returns an exception error code.
- 6:40:01In a sub agent case, it returns text
- 6:40:04that looks like a success.
- 6:40:06Dirty reads,
- 6:40:08normally we have isolations, versions.
- 6:40:11Here
- 6:40:12every worker reads on shared context
- 6:40:14including things other worker only
- 6:40:16guessed.
- 6:40:18Unbounded fan out.
- 6:40:21A busy service
- 6:40:23pushes back with the queues
- 6:40:25for two nines, five o threes.
- 6:40:28In agents
- 6:40:29work
- 6:40:30does not see any of this. just starts in
- 6:40:32the worker.
- 6:40:34Retries.
- 6:40:36Normal distributed systems we build at a
- 6:40:38potency to make retries reliable.
- 6:40:42In agents
- 6:40:43a retries sub agent runs the whole task
- 6:40:45again
- 6:40:46with side effects included.
- 6:40:49For each of this case
- 6:40:51I'll name the concept it already is
- 6:40:54and I'll put them together as a
- 6:40:56consistency model I'm proposing
- 6:40:58and walk through why each part is there.
- 6:41:02Start with the failed model because
- 6:41:03everything else depends on it.
- 6:41:06A friendly case is a crash down.
- 6:41:08The process it was running correctly
- 6:41:10stops
- 6:41:11and stays stopped.
- 6:41:13The sub agent withdraws an error in this
- 6:41:14box.
- 6:41:15It's easy to handle.
- 6:41:18Next, no reply.
- 6:41:20The call is lost or it's very late.
- 6:41:22We'll come back to this soon.
- 6:41:25The hard case is the one from
- 6:41:28Byzantine general sniper.
- 6:41:30A component that gives conflicting
- 6:41:32information.
- 6:41:33The process answers and the answer is
- 6:41:35wrong.
- 6:41:37A half finished sub agent sits closest
- 6:41:39to the box.
- 6:41:40It is not malicious but from the outside
- 6:41:42it looks the same.
- 6:41:44It does not crash. It answers fluently.
- 6:41:46Let's jump into this.
- 6:41:48The sub agent
- 6:41:50that half finished does not race.
- 6:41:52It returns normal looking text
- 6:41:55with the same type signature as success.
- 6:41:58I talked about one version of this
- 6:42:00in one of my talks.
- 6:42:02We pointed an agent at an alert queue.
- 6:42:05Here's an illustration I used there
- 6:42:08with three different paths which give
- 6:42:09the same answer.
- 6:42:11The first one
- 6:42:13it pulled the telemetry and compared
- 6:42:15with a known benign set.
- 6:42:17It actually did the work.
- 6:42:19The second one
- 6:42:20it matched against a title
- 6:42:23it it has seen before
- 6:42:24and gave the response.
- 6:42:27The third alert
- 6:42:29it didn't do anything.
- 6:42:31It just
- 6:42:32resonator.
- 6:42:34All three are categorized as benign
- 6:42:36with the same level of confidence.
- 6:42:39And nothing downstream can tell these
- 6:42:41apart.
- 6:42:42The orchestrator can read the
- 6:42:43conclusion. It cannot read how much work
- 6:42:45was behind it.
- 6:42:48Now
- 6:42:49the call gets no reply.
- 6:42:51The orchestrator send work and a timeout
- 6:42:54fired.
- 6:42:56From here
- 6:42:57three worlds look the same.
- 6:42:59The sub agent is still working. It
- 6:43:01crashed.
- 6:43:02And the reply was lost.
- 6:43:04It's a hard limit on a tooling gap.
- 6:43:07The FLP paper from 1985 showed that in
- 6:43:09an asynchronous system
- 6:43:11with even one faulty process
- 6:43:13agreement can fail to finish.
- 6:43:16Chandra and Tu X answered in 1996 was
- 6:43:18failure detectors that can make
- 6:43:20mistakes.
- 6:43:21They suspect they do not know.
- 6:43:24I hit both directions
- 6:43:26in my own open source project
- 6:43:29MCP pacemaker supervised bridge.
- 6:43:33A server
- 6:43:34whose first run downloads itself
- 6:43:36hit the 30 second call timeout
- 6:43:38and looked broken.
- 6:43:40It was slow, not dead.
- 6:43:42We fixed it
- 6:43:43by giving initialize its own timeout.
- 6:43:46Now the other way
- 6:43:47the server crashed
- 6:43:49during a call.
- 6:43:50The call then timed out.
- 6:43:52And the timeout overrode the exit code
- 6:43:54that explained the crash.
- 6:43:56We fixed it
- 6:43:58by recording the timeout only when it
- 6:44:00was alive.
- 6:44:02So
- 6:44:03timeout is a suspicion.
- 6:44:05Record it as unknown
- 6:44:07and keep the real cause.
- 6:44:09And
- 6:44:11now make failure a value.
- 6:44:15Let's take this illustration.
- 6:44:17A sub agent returns a status
- 6:44:20what it did
- 6:44:21what it did not do
- 6:44:23the evidence and any side effects.
- 6:44:26Partial is a normal answer here.
- 6:44:29The protocols already have a slot for
- 6:44:31this.
- 6:44:32In A to A,
- 6:44:33a task can end as completed, failed,
- 6:44:36canceled, or rejected,
- 6:44:39or stop and ask for input.
- 6:44:41In MCP,
- 6:44:42a tool result can say
- 6:44:44is error true.
- 6:44:46But a sub agent
- 6:44:47can still say completed
- 6:44:50while its text says something else.
- 6:44:52And a loop that only reads the text now
- 6:44:54receives a difference.
- 6:44:56Another split.
- 6:44:58Checking a condition and getting no
- 6:45:01is a different fact from not being able
- 6:45:03to check it all.
- 6:45:06My own individual internet draft on
- 6:45:08evaluation states
- 6:45:11talks about this.
- 6:45:13Collapse the two and you change what
- 6:45:15gets retried
- 6:45:17and who gets paged and what the audit
- 6:45:19says later.
- 6:45:21Next,
- 6:45:22dirty reads.
- 6:45:24Let's take this made up example from one
- 6:45:26of my talk.
- 6:45:29The sub agent
- 6:45:30does try
- 6:45:31it writes into a shared context
- 6:45:34saying that error rate fell by 80%.
- 6:45:37It did not measure that. It inferred.
- 6:45:40Now,
- 6:45:43two other sub agents read from the same
- 6:45:45shared context.
- 6:45:48The second sub agent
- 6:45:49writes a review note and picks the same
- 6:45:51sentence.
- 6:45:53The third agent writes their own book
- 6:45:56and uses it as a reason to proceed.
- 6:45:59The orchestrator
- 6:46:00merges all the all these three.
- 6:46:03Three artifacts agree.
- 6:46:05It is one unsupported sentence
- 6:46:07read three different times.
- 6:46:09The 1995 critique of sequel isolation
- 6:46:12levels define a dirty read as
- 6:46:14you read a value before the writer
- 6:46:16commits or rolls back.
- 6:46:18The shared context
- 6:46:20it guess is readable the moment it is
- 6:46:21written.
- 6:46:22Nothing marks it as unchecked.
- 6:46:25Regular fix
- 6:46:28to is to ask three sub agents
- 6:46:30and get a quorum.
- 6:46:33Quorums do work for a specific reason.
- 6:46:36In waited voting from 1979
- 6:46:39the read quorum plus write quorum is
- 6:46:41greater than total.
- 6:46:43So every read overlaps the latest write.
- 6:46:46The overlap is a guarantee.
- 6:46:49In sub agents case
- 6:46:50the three sub agents that read the same
- 6:46:52shared context
- 6:46:53does not give you that.
- 6:46:55They are not independent copies.
- 6:46:57They read the same guess.
- 6:46:59Their agreement is one source counted
- 6:47:01three different times.
- 6:47:03Let's pick a consistency model.
- 6:47:05In databases we have a
- 6:47:09Linearizable.
- 6:47:11Every operation appears to happen at one
- 6:47:13point in real time order.
- 6:47:15Sequential.
- 6:47:17One total order that respects each
- 6:47:19process own order.
- 6:47:21Causal.
- 6:47:23Operations that are causally related
- 6:47:25appear in the same order everywhere.
- 6:47:28Session guarantees.
- 6:47:30They are promises to one client
- 6:47:32like read your rights and monotonic
- 6:47:34reads.
- 6:47:36Eventual.
- 6:47:37If no new updates are made reads
- 6:47:39eventually return the last value.
- 6:47:42A shared context gives you none of
- 6:47:44these.
- 6:47:45There's no stated guarantee at all.
- 6:47:48And you don't need the top consistency
- 6:47:50models.
- 6:47:52The strongest models cannot stay
- 6:47:54available when the network splits.
- 6:47:57Causal and session guarantees can
- 6:47:59as long as each clients stick to one
- 6:48:01place.
- 6:48:03For sub agents
- 6:48:04that place is the orchestrator.
- 6:48:06So
- 6:48:08here's the consistency model I use for
- 6:48:10sub agents.
- 6:48:12I call it
- 6:48:13safe handoffs.
- 6:48:16Yes
- 6:48:17stands for
- 6:48:18source feeds.
- 6:48:20Every fact the worker hands on carries
- 6:48:23its source version as of time and
- 6:48:26revision.
- 6:48:27And decisions read only fresh source
- 6:48:29facts.
- 6:48:31A
- 6:48:32at most once effects.
- 6:48:35Every write has an identity and a single
- 6:48:37use approval for the exact action.
- 6:48:40An outcome
- 6:48:41is read back now replayed.
- 6:48:44If
- 6:48:46fan out budgets
- 6:48:48each downstream gets a gate and a
- 6:48:50bounded weight.
- 6:48:51Refusal counts as a fail.
- 6:48:54E stands for explicit outcomes.
- 6:48:57Every hand off ends in a named state
- 6:49:00and a timeout is unknown
- 6:49:02with its cost kept.
- 6:49:04This consistency model
- 6:49:07is the one I use for my own sub agents.
- 6:49:08It's not a universal model.
- 6:49:10It's based off my own public work.
- 6:49:13Some of it is in my FCP pacemaker bridge
- 6:49:16and some are based on my other talks and
- 6:49:20my own internet draft.
- 6:49:24Now let's see
- 6:49:26what's behind each letter.
- 6:49:29S source reads.
- 6:49:32It's the answer to dirty reads
- 6:49:34where one worker's guess becomes another
- 6:49:36worker's fact.
- 6:49:37What helps is to treat every fact as a
- 6:49:40record, not a sentence.
- 6:49:43Lamport's 1978 paper define when one
- 6:49:46event happened before another.
- 6:49:48The fact's provenance is that record,
- 6:49:50what it was read from and when.
- 6:49:53Let's take this
- 6:49:55made up example.
- 6:49:57The record names
- 6:49:59the claim and its revision,
- 6:50:01the source and its version,
- 6:50:04the scope
- 6:50:05and three different times
- 6:50:08when the data was current,
- 6:50:10when we read it and when we used it.
- 6:50:14Freshness is checked with the fact is
- 6:50:16used.
- 6:50:18In this illustration
- 6:50:19the data ends at
- 6:50:229:05.
- 6:50:23We read it at 9:06
- 6:50:25and we decided at 9:07.
- 6:50:28In this case
- 6:50:30we picked a five minute limit.
- 6:50:32So the data
- 6:50:34decision at 9:07 it's fine.
- 6:50:37Now at 9:15 the second station came in.
- 6:50:42Because our freshness limit is five,
- 6:50:44this is still recorded.
- 6:50:47We reject it.
- 6:50:49And we say that the next station needs
- 6:50:51another read.
- 6:50:53This is how you separate an observation
- 6:50:55from an inference.
- 6:50:57An observation has a source and a read
- 6:50:59time.
- 6:51:00An inference has none.
- 6:51:02A at most effects is about retries.
- 6:51:06This an illustration
- 6:51:09from my article about MCP pacemaker
- 6:51:12bridge.
- 6:51:14The orchestrator delegates a task.
- 6:51:18The sub agent calls a tool
- 6:51:20which does a real write
- 6:51:22like
- 6:51:24creating a PR.
- 6:51:26The reply is lost.
- 6:51:29The orchestrator knows one thing, the
- 6:51:31outcome is unknown.
- 6:51:33So
- 6:51:34it retries the whole task.
- 6:51:36The write happens again.
- 6:51:39This is a classic problem.
- 6:51:41In 1984 RPC paper
- 6:51:45it says that if the call returns
- 6:51:48the procedure ran precisely once.
- 6:51:50If you get an exception
- 6:51:52it ran once or not at all.
- 6:51:54And you are not told which.
- 6:51:57So
- 6:51:58you pick a delivery semantic.
- 6:52:00At most once.
- 6:52:03Don't retry and accept the fact that it
- 6:52:05may be lost.
- 6:52:07At least once
- 6:52:08retry
- 6:52:10and accept that duplicates are possible.
- 6:52:13Exactly once.
- 6:52:15It happened at least once and this is a
- 6:52:18way we can deduplicate.
- 6:52:21Maybe by using an item potency key.
- 6:52:24In MCP
- 6:52:25item potent hint is only a hint and it
- 6:52:28defaults to false. If you need it, you
- 6:52:30build it.
- 6:52:31Here's the small design from my bridge
- 6:52:34on how I handle it.
- 6:52:36When a client comes back after a restart
- 6:52:39the bridge replace the client's record
- 6:52:41initialized against a fresh server,
- 6:52:43that's safe. It's just a handshake,
- 6:52:45and it does not restore tool state.
- 6:52:49What it never replace is an uncertain
- 6:52:51tool call.
- 6:52:52When a shared server is recycled, the
- 6:52:54bridge drains it
- 6:52:56and calls still unanswered at the
- 6:52:58deadline fail as potentially excluded.
- 6:53:02That status is honest.
- 6:53:04It tells the caller to go and check.
- 6:53:08What helped us giving every side effect
- 6:53:11an identity.
- 6:53:13Approval
- 6:53:14binds with exact action
- 6:53:16and it's only used once.
- 6:53:18After the call, read back by that
- 6:53:20identity.
- 6:53:22Submitted is not confirmed. Confirmed
- 6:53:24means you observe the effect.
- 6:53:27The last part of A
- 6:53:29it shows up when work passes and
- 6:53:31resumes.
- 6:53:32A lease is a lock with a timeout.
- 6:53:35Martin Kleppmann's post about
- 6:53:37distributed locking shows this problem.
- 6:53:40A client passes while holding it.
- 6:53:43The lease expires.
- 6:53:45Someone else takes over
- 6:53:46and the first client wakes up and writes
- 6:53:48anyway.
- 6:53:50The fix he proposes
- 6:53:52a fencing token.
- 6:53:54A number that goes up every time the
- 6:53:56lock is granted.
- 6:53:58The storage remembers the highest token
- 6:54:01it has seen
- 6:54:02and rejects any write with a lower one.
- 6:54:06Sub agents
- 6:54:08pause and resume all the time.
- 6:54:10In my other talk, as I put it
- 6:54:13restore context, not side effects.
- 6:54:16So treat an approval like a fencing
- 6:54:18token.
- 6:54:19One action, one use.
- 6:54:21Checked at the gate.
- 6:54:22Not remembered in the prompt or
- 6:54:23conversation.
- 6:54:26My bridge
- 6:54:27fixed the same class of bug.
- 6:54:30A stale exit or a deleted resume can no
- 6:54:33longer remove or bring back a
- 6:54:34replacement session.
- 6:54:37If fanout budgets
- 6:54:39is about unbounded fanout.
- 6:54:42This is flow control or or lack of it.
- 6:54:44I hit this issue in my own MCP bridge,
- 6:54:47before it had a limit on cold starts.
- 6:54:50Many sessions start the service
- 6:54:53at the same time
- 6:54:55through NPX.
- 6:54:56They share one package cache.
- 6:54:59Everyone of them failed with NPM's lock
- 6:55:01compromise error.
- 6:55:03Nothing in any single request was wrong.
- 6:55:06The failure was the concurrency itself.
- 6:55:09And when it downstream does push back,
- 6:55:11it can arrive as plain text.
- 6:55:14This is MCP specs one example of tool
- 6:55:16execution error.
- 6:55:17A rate limit
- 6:55:19as one sentence.
- 6:55:20The model reads the sentence.
- 6:55:22The orchestrator code may never look.
- 6:55:24The next worker starts an event.
- 6:55:27So
- 6:55:28I introduced a quota budget in my
- 6:55:30bridge.
- 6:55:32Cold starts that use a package manager
- 6:55:35go through a gate
- 6:55:37two at a time by default.
- 6:55:39Everyone else waits.
- 6:55:41And the wait is bounded at 15 seconds.
- 6:55:43Past that
- 6:55:45the start goes ahead without a gate.
- 6:55:48And
- 6:55:49I documented why
- 6:55:50a corrupted cache is a risk
- 6:55:52while a hung client request is
- 6:55:54certainty.
- 6:55:55That is a choice for availability made
- 6:55:57on purpose and written down.
- 6:56:00And I also built a cap on sessions.
- 6:56:04At the cap
- 6:56:05any a new session waits up to 10
- 6:56:07seconds.
- 6:56:08And then it's refused.
- 6:56:10The refusal now counts as a failure in
- 6:56:12health.
- 6:56:13Another bug.
- 6:56:15Capacity has to be reserved before the
- 6:56:17synchronous start, not after.
- 6:56:19Otherwise
- 6:56:21queued starts overbook the cap.
- 6:56:23That is a check then act race.
- 6:56:26For sub agents
- 6:56:27give the orchestrator a number per
- 6:56:29downstream.
- 6:56:30A bounded weight and a refusal it can
- 6:56:32see.
- 6:56:34Now
- 6:56:35E next present outcomes.
- 6:56:38Every hand of ends in a named state like
- 6:56:41stale or unknown, not just text.
- 6:56:44The last piece is who decided that
- 6:56:46state.
- 6:56:48Treat sub the agent output as untrusted
- 6:56:49input,
- 6:56:50the same way you treat a request body
- 6:56:52from the internet.
- 6:56:54The orchestrator owns the checks in
- 6:56:57code.
- 6:56:58Does the result match the schema?
- 6:57:00Is the evidence there?
- 6:57:01Is it fresh enough?
- 6:57:03Do the numbers hold when you recompute
- 6:57:05them?
- 6:57:06If a check fails, the step returns a
- 6:57:08refusal with a reason.
- 6:57:11It does not pass the text along with a
- 6:57:12warning paragraph underneath.
- 6:57:15If the same model can skip a check and
- 6:57:17declare it passed,
- 6:57:18you have a request, not a boundary.
- 6:57:21Here is a made up example from another
- 6:57:23talk.
- 6:57:25The sub agent says the error rate fell.
- 6:57:28Before it was 10 errors
- 6:57:30in 1,000 requests.
- 6:57:31After
- 6:57:32it was six errors in 600 requests.
- 6:57:35It went from 10 to six and it claimed
- 6:57:37that our rate fell.
- 6:57:38But
- 6:57:39when we computed the rate,
- 6:57:41it's both 1%.
- 6:57:44We recomputed it and then the claim is
- 6:57:45refused.
- 6:57:48A supported answer is fine.
- 6:57:49A deliberate handoff to a person is
- 6:57:51fine.
- 6:57:52A confident guess, it's not.
- 6:57:55So limitations
- 6:57:56because a consistency model is only
- 6:57:58useful when you know where we should
- 6:58:00stop.
- 6:58:02So here's what
- 6:58:04safe model does not do.
- 6:58:07It does not make facts true.
- 6:58:09A source fact is checkable, not correct.
- 6:58:13It does not give you exactly once
- 6:58:15delivery.
- 6:58:17A lost reply
- 6:58:18stays unknown until you read it back.
- 6:58:21It does not roll anything back.
- 6:58:24Undoing a partial task is a compensation
- 6:58:26step you design
- 6:58:28like the Saga's paper from 1987.
- 6:58:31And finally, there's no global order
- 6:58:33across workers,
- 6:58:35only causal order per fact.
- 6:58:38And
- 6:58:39it has a price.
- 6:58:41CAP theorem says that when the network
- 6:58:43splits,
- 6:58:44you trade off availability against
- 6:58:47consistency.
- 6:58:49My Brits chose availability for cold
- 6:58:51starts on purpose.
- 6:58:53For rights,
- 6:58:55safe picks
- 6:58:56consistency
- 6:58:57over availability.
- 6:58:59It refuses rather than guesses.
- 6:59:02Pack lack
- 6:59:04adds that even without failures
- 6:59:06you trade latency for consistency.
- 6:59:09Every freshness check is another read
- 6:59:12and and every refusal needs an owner.
- 6:59:16So
- 6:59:17where it does not fit
- 6:59:20if a subway agent
- 6:59:21only drafts texts that a person reads
- 6:59:24before anything happens
- 6:59:25most of this is is a overhead.
- 6:59:28It also assumes that there's one single
- 6:59:30orchestrator.
- 6:59:32If you have more this does not fit.
- 6:59:35Peer agents with no single place to
- 6:59:38stick to it's something else.
- 6:59:41This is the consistency model I use for
- 6:59:43my own agents that touch actual real
- 6:59:45production systems.
- 6:59:47It's not universal.
- 6:59:49So wrapping up
- 6:59:51here's three key takeaways.
- 6:59:53Treat every handoff to a sub agent as a
- 6:59:55network call.
- 6:59:57It can fail halfway
- 6:59:59it can be slow and it can come back
- 7:00:00twice.
- 7:00:02Return a status not just text.
- 7:00:05If the result cannot say partial, stale
- 7:00:07or unknown
- 7:00:08the orchestrator will read it as done.
- 7:00:11And
- 7:00:12start with our safe consistency model
- 7:00:15and enforce it in code.
- 7:00:16Keep what fits and change what does not.
- 7:00:20We already know how to build distributed
- 7:00:21systems.
- 7:00:23We just have to notice that we are
- 7:00:24building one.
- 7:00:26So these are some of the sources
- 7:00:28behind this talk.
- 7:00:32On the left side it's my own work, my
- 7:00:34open source projects, my
- 7:00:36articles, internet drafts and my talks.
- 7:00:40On right side
- 7:00:42specs and standards
- 7:00:45and
- 7:00:46bunch of papers behind each concept.
- 7:00:50Thank you.
- 7:00:53>> Hello, my name is Nikolai
- 7:00:54>> Petrov, and we will be talking about
- 7:00:56skills and agents knowledge of those
- 7:00:58skills.
- 7:01:01We'll start it with the simple prompts
- 7:01:03where we've been
- 7:01:05playing with the new stuff, new work
- 7:01:07being done by the agents, repeating
- 7:01:09those prompts over and over again, and
- 7:01:11then we start packing those into the
- 7:01:13skills,
- 7:01:14creating repeatable, reproducible,
- 7:01:17portable type of their
- 7:01:19functionality
- 7:01:21or capabilities that we can
- 7:01:24teach agents to work with.
- 7:01:26And some of those skills are very
- 7:01:27important, hundreds of the stars on
- 7:01:29GitHub.
- 7:01:33There are different types of their
- 7:01:34skills. Some
- 7:01:36teach models something new, bringing new
- 7:01:38knowledge or capabilities to the model,
- 7:01:41but there are also types of their just
- 7:01:44steering or process skills where we
- 7:01:48do not bring any additional information,
- 7:01:50but just getting repeatable
- 7:01:53flow of the actions that we want agents
- 7:01:55to perform.
- 7:01:57Both are useful. In this session, we
- 7:02:00will be talking about new knowledge type
- 7:02:01of the skills
- 7:02:03and
- 7:02:05we'll see what we can
- 7:02:08get from the model's understanding of
- 7:02:10those skills.
- 7:02:13So we get our first skills packed, start
- 7:02:17using them
- 7:02:18basically
- 7:02:21starting with their the feel of
- 7:02:23does it work or not.
- 7:02:25But there is also their more technical
- 7:02:27way and more deterministic way for the
- 7:02:29checking if skills are working or not
- 7:02:31and what's the
- 7:02:32impacts of those skills
- 7:02:34our evolves.
- 7:02:36There are different formats where
- 7:02:37evolves can be defined,
- 7:02:39but at the sense, this is task
- 7:02:41definition that you want your agent to
- 7:02:43be able to solve.
- 7:02:46You put it in the place, ideally more
- 7:02:48deterministic so we can validate it.
- 7:02:51And we can check two arms. One their no
- 7:02:55skill loaded, and their second one where
- 7:02:58we're loading the skill and providing
- 7:02:59this additional information to the
- 7:03:01context.
- 7:03:02And we compare how their arms There are
- 7:03:05multiple things. One is how many
- 7:03:08evolved spins off.
- 7:03:10In this case, with no skill, we're
- 7:03:12solving three out of five.
- 7:03:14With skill, we're solving all five,
- 7:03:16meaning that we get two more tasks.
- 7:03:19There also would be implications on the
- 7:03:21tokens usage or time that it took those
- 7:03:24tasks to be completed and other factors.
- 7:03:28But at the end, we are comparing those
- 7:03:29and seeing what makes sense, what not.
- 7:03:32And this is one of the main tools for
- 7:03:35evaluating the skills and
- 7:03:37saying that they not just nice ideas,
- 7:03:40but actually working pieces of
- 7:03:42instructions and context for the agents.
- 7:03:46But what next? We get to the stage where
- 7:03:49there we have their working skill. It
- 7:03:52solved some evolutions.
- 7:03:54But we know that
- 7:03:56there should be a better way. We have
- 7:03:58models evolving. We have
- 7:04:01some
- 7:04:03maybe cases there we want
- 7:04:07to get skill more efficient, meaning
- 7:04:09consuming less tokens. So what's their
- 7:04:12their options there?
- 7:04:14One is just change line by line and
- 7:04:17try brute force
- 7:04:19all the different combinations.
- 7:04:21Another would be to think through what
- 7:04:24makes sense or maybe use something
- 7:04:28as the user feedback on the skills to
- 7:04:30see where skills perform well there it's
- 7:04:33not.
- 7:04:34And with new models, just running skill
- 7:04:38and seeing how it works, what's the
- 7:04:41options there.
- 7:04:42And
- 7:04:43how large those modifications should be?
- 7:04:45Just
- 7:04:47lines, sentences, ideas.
- 7:04:53So what
- 7:04:54we found to be useful is instead of
- 7:04:57relying on their luck, we can put more
- 7:05:01deterministic process here
- 7:05:03using idea of their items from the fact
- 7:05:06score. Uh
- 7:05:08we can split their skill and
- 7:05:11instructions into those items
- 7:05:14and
- 7:05:16for each of those had
- 7:05:18pair of question and answer that we
- 7:05:20expect.
- 7:05:22This could be either general ideas like
- 7:05:25proposition in this case or contracts
- 7:05:28there we put more
- 7:05:30definitive expectation on their action.
- 7:05:37And in general we will be getting their
- 7:05:39uh
- 7:05:43some predefined amount of the items per
- 7:05:46skill and we will have let's say
- 7:05:5025 or 30
- 7:05:52items per skill.
- 7:05:54Then for each of the items we will again
- 7:05:57run
- 7:05:58two arms, one without skill and one with
- 7:06:01skill.
- 7:06:03And we will have judge
- 7:06:06or panel of judges that will be
- 7:06:08evaluating model answers. So we asking
- 7:06:10those questions to the model that we
- 7:06:12interested in.
- 7:06:14We evaluating those questions with their
- 7:06:17judge, ideally different family of the
- 7:06:20models or
- 7:06:22panel of judges.
- 7:06:23We also ensures that the question do not
- 7:06:26leak answers.
- 7:06:29We
- 7:06:31depending on the case can compare with
- 7:06:32no tools or additional um
- 7:06:35information to check their internal
- 7:06:37model understanding or provided with
- 7:06:40their some tools like feedback access so
- 7:06:42it would be more practical.
- 7:06:45And we
- 7:06:48can get their simple forward counts
- 7:06:50matrix.
- 7:06:54Basically where we are checking that the
- 7:06:57model
- 7:06:59uh uh
- 7:07:00knows or do not know something
- 7:07:02and how this maps to their
- 7:07:04skill impact on this.
- 7:07:06So the simplest
- 7:07:08option is where model was able to answer
- 7:07:11this particular
- 7:07:13questions even without skill or with
- 7:07:16skill. This means that skill have no
- 7:07:18impact on this and this was already part
- 7:07:20of the model's knowledge.
- 7:07:22The second one is where without skill
- 7:07:25actually model was able to answer this
- 7:07:26correctly.
- 7:07:28And by adding skill
- 7:07:31we saw degradation in performance and
- 7:07:33model capability to
- 7:07:35talk or explain particular aspects.
- 7:07:40The ideal case is where
- 7:07:42without skill model wasn't able to
- 7:07:44answer something and with skill we start
- 7:07:46seeing that it can answer those new
- 7:07:48type of the questions and scenarios and
- 7:07:50this is what we were aiming for, teach
- 7:07:53model something new.
- 7:07:55But there also can be there fourth
- 7:07:57option where
- 7:07:59skills were written not
- 7:08:01ideally or in some
- 7:08:05maybe not clean way
- 7:08:06where model wasn't able to answer it
- 7:08:09either without skill or with skill and
- 7:08:11while this topic might be relevant and
- 7:08:13important for the model it still sort of
- 7:08:16misses some of those concepts.
- 7:08:21The simplest thing to fix are
- 7:08:26instructions or items that actually
- 7:08:28hurts performance and
- 7:08:31for popular skills those are pretty
- 7:08:33rare.
- 7:08:34So in their
- 7:08:37uh
- 7:08:38box below you see that it's just one
- 7:08:41case out of the mostly 400
- 7:08:44items in entropic skills
- 7:08:46and others don't have those.
- 7:08:49But for their smaller skills that are
- 7:08:52less popular you probably having more of
- 7:08:54those. This is what we are seeing with
- 7:08:56some of our um experiments.
- 7:08:59And
- 7:09:01this is just straightforward the
- 7:09:03simplest to fix
- 7:09:05to ensure that your skill do not reduce
- 7:09:08model capabilities and performance and
- 7:09:10bring only relevant
- 7:09:12context there.
- 7:09:15And we also can actually color this in
- 7:09:17this scenario this is done by lines but
- 7:09:20actually going with items you might have
- 7:09:22there
- 7:09:23uh different coloring for a single line
- 7:09:25or a single sentence.
- 7:09:28And this just gives your example of
- 7:09:30there uh visualization where
- 7:09:33this would be helpful
- 7:09:35details and instructions in terms
- 7:09:38what new stuff you bring to the model
- 7:09:40and what
- 7:09:42is
- 7:09:45something that model already knows.
- 7:09:48Cost wise
- 7:09:49we are running all those questions and
- 7:09:51asking model
- 7:09:53but still relatively this is uh cheap
- 7:09:55for 29 skills for 10 bucks of there
- 7:09:59sonnet and haiku
- 7:10:00calls with their sonnet used as a judge.
- 7:10:04And
- 7:10:09there
- 7:10:10couple ways how we also can check their
- 7:10:14strictness of their some knowledge.
- 7:10:17Um we have
- 7:10:19uh lenient and strict ways of the
- 7:10:21checking this where first checks the
- 7:10:23general idea
- 7:10:25while the second one is checking for
- 7:10:28exact details and you can think about
- 7:10:31the difference between general knowledge
- 7:10:33and understanding of the topic
- 7:10:35to the understanding of the exact
- 7:10:38keys parameters and exact values or
- 7:10:41steps.
- 7:10:43And what we see across the models is
- 7:10:46the difference is large
- 7:10:48especially with there
- 7:10:51um
- 7:10:51larger frontier models where we see
- 7:10:53almost dropping from 70% understanding
- 7:10:55of their skill in case of the general
- 7:10:58understanding
- 7:10:59to
- 7:11:00just 22% of the specific
- 7:11:03details.
- 7:11:05And
- 7:11:06this is much our expectations. We see
- 7:11:08that new models are
- 7:11:10pretty well generalized and understand
- 7:11:11concepts. But it's still helpful to
- 7:11:13provide their with exact details and
- 7:11:16data that will be useful for their
- 7:11:18solving those problems.
- 7:11:23Here is outlook of their
- 7:11:27comparing those three popular rappers
- 7:11:29with the skills
- 7:11:31across different models. So one of their
- 7:11:34interesting observations here is that
- 7:11:36smaller model have
- 7:11:38in general
- 7:11:39lower understanding of their
- 7:11:43skill
- 7:11:45with their larger sonnet having better
- 7:11:47understanding which is
- 7:11:50why we have bigger models.
- 7:11:52But also we see that overall their
- 7:11:54models already understand
- 7:11:56a lot of their
- 7:12:00details and context of those skills and
- 7:12:03you would expect that with the newer
- 7:12:04models and more advances in their
- 7:12:09better training capabilities we will
- 7:12:11have
- 7:12:12more and more
- 7:12:15in terms of the model's understanding.
- 7:12:17But also we see that this pretty
- 7:12:19significantly differs skill to skill.
- 7:12:22And
- 7:12:23it's not bad. Some skills are intended
- 7:12:25to be directional.
- 7:12:28But it's also useful piece of
- 7:12:30information to see how those
- 7:12:33skills bring new knowledge
- 7:12:36to the different model and different
- 7:12:37families and this should be run ideally
- 7:12:40in the broader
- 7:12:41set of the models.
- 7:12:47But
- 7:12:49just having their
- 7:12:52this novelty piece of information is
- 7:12:56just one of the signals. It's basically
- 7:12:57initial vector for optimization.
- 7:12:59At the end we still have to run evolves
- 7:13:01and validate that
- 7:13:03all the changes that we
- 7:13:06try to make with their skill actually
- 7:13:08works.
- 7:13:09So their one of their simpler scenarios
- 7:13:12would be we can look on their skill and
- 7:13:16try to remove some or all of their
- 7:13:18already
- 7:13:20known pieces of information from the
- 7:13:22skill and run evaluation.
- 7:13:25It might come up with their same
- 7:13:27scoring. Basically we will be getting
- 7:13:29their same results model still capable
- 7:13:31of solving those problems. Then great we
- 7:13:34can reduce
- 7:13:35skill size, save tokens, save context.
- 7:13:39And instead of their
- 7:13:41brute forcing and guessing what we can
- 7:13:43or should remove
- 7:13:45or add or how to fix their
- 7:13:49degradations we can have a guidance.
- 7:13:54So all of those measurements experiments
- 7:13:57been done with the docume which is a
- 7:13:59framework for their comparing
- 7:14:02everything related to the agents and
- 7:14:04context
- 7:14:06where it's is skill, agents MD, MCP
- 7:14:10tools, whatever. And novelty is one of
- 7:14:13their pieces of the functionality where
- 7:14:16you can get this initial vector for
- 7:14:19their skill optimization and evaluation
- 7:14:21but
- 7:14:21there also
- 7:14:23evolves there is assessment of the
- 7:14:25context impact, routing, comparing their
- 7:14:29descriptions, naming, bunch of the
- 7:14:31things that
- 7:14:32related to their
- 7:14:35skills where it would be helpful to
- 7:14:37actually get their more deterministic
- 7:14:39evaluations to make
- 7:14:41conscious decisions in terms of the what
- 7:14:43should be done with the skills.
- 7:14:47So thank you for your time and please
- 7:14:50go ahead and write better skills.
- 7:16:04>> Keynote programming will begin in 5
- 7:16:07minutes.
- 7:16:08Please be sure to find your seats.
- 7:18:07>> This
- 7:19:14>> <music>
- 7:19:14>> <music>
- 7:23:06>> As we head into our closing keynotes,
- 7:23:08we're excited to bring back a familiar
- 7:23:10face to guide us through the final
- 7:23:12sessions. Ladies and gentlemen, please
- 7:23:15welcome back our MC, Corey Quinn, Chief
- 7:23:19Cloud Economist at Duckbill Group.
- 7:23:39>> Welcome back. Thank you all for sticking
- 7:23:42out the full day. We just have one more
- 7:23:46section to go and then we're calling it
- 7:23:48a wrap on technically day two of the
- 7:23:50conference.
- 7:23:51Since we arrived this morning, we've
- 7:23:53heard about a few things, deleting
- 7:23:55harnesses, presumably on purpose,
- 7:23:58healing your agents, maybe through the
- 7:24:00power of song, we don't know,
- 7:24:02self-improving codebase, which is the
- 7:24:04exact opposite of the direction of every
- 7:24:05codebase I've ever seen in my life has
- 7:24:07ever gone, but okay, and so much more
- 7:24:10because
- 7:24:11everything is different now.
- 7:24:13And before we leave tonight, we're going
- 7:24:14to hear about setting intelligence free.
- 7:24:17We're going to hear about how agents
- 7:24:18build cultures.
- 7:24:20There's a talk called Escaping from
- 7:24:22Trash Mountain, which ironically was my
- 7:24:24nickname back in high school. And of
- 7:24:26course, 2026 in LLMs, which is one of
- 7:24:29those years that seems to encompass
- 7:24:31decades. So here we are.
- 7:24:34Before we dive into all of that, I want
- 7:24:36to take just another moment to thank our
- 7:24:38sponsors for making all this possible.
- 7:24:41Presenting sponsor remains Google
- 7:24:42DeepMind, who earlier today shipped
- 7:24:46what was it? Gemini Carbon 4 to general
- 7:24:48availability. Congratulations.
- 7:24:51Please clap.
- 7:24:54[applause]
- 7:24:56For our lounge and our platinum sponsors
- 7:24:58as well, without whom we would have
- 7:25:00far fewer places to sit when we're not
- 7:25:02in the middle of a session.
- 7:25:04Our gold sponsors as well for
- 7:25:07effectively making all the various
- 7:25:09moving parts go.
- 7:25:11And of course, our silver parts, our
- 7:25:13silver sponsors too. My goodness, there
- 7:25:16are so many tiers at this point. At some
- 7:25:17point, it starts to become an API
- 7:25:19ladder. And finally, all of our
- 7:25:20supporting partners. Again, it it really
- 7:25:22is not possible to put on an event like
- 7:25:24this without a tremendous amount of
- 7:25:26support from various folks, and we thank
- 7:25:28them sincerely for all of this.
- 7:25:30Uh,
- 7:25:31now up we have Fabian Hedin, who is the
- 7:25:33CTO of Lovable, trying to tell us what
- 7:25:36it takes to set intelligence free, uh,
- 7:25:40which might be the only time in AI we
- 7:25:41hear the word free.
- 7:25:44Let's make him feel welcome.
- 7:25:47>> [applause]
- 7:26:04>> All right.
- 7:26:06Thank you so much. Um, see we get to go
- 7:26:09here. Um, all right. Welcome everyone.
- 7:26:12Glad to be here. I'm Fabian, co-founder
- 7:26:14and CTO of of Lovable, and we spent the
- 7:26:18last few years or so building a platform
- 7:26:21to let anyone create software and then
- 7:26:23operate that and deploy that software.
- 7:26:26Um, and um, the the topic of today is
- 7:26:30what it takes to set intelligence free.
- 7:26:34And
- 7:26:35really um,
- 7:26:36for us this takes us back
- 7:26:38to when we when we started three years
- 7:26:40ago. And I have to say I have a bit of a
- 7:26:44complicated relationship with with
- 7:26:46agents, um, and I I'll tell you why. Um,
- 7:26:50we almost failed as a company
- 7:26:52because we thought
- 7:26:54back in 2023 that hey, we these agents
- 7:26:57are the future, so let's build for them
- 7:26:59now.
- 7:27:00Um, I think they we kind of turned out
- 7:27:02to be true that they were the future,
- 7:27:03but we were definitely wrong on the the
- 7:27:05now piece in in 23. So this was back in
- 7:27:08kind of GPT-4 days. Um,
- 7:27:11and um,
- 7:27:12at the at that time the intelligence was
- 7:27:14simply not there to have more of these
- 7:27:16kind of complex agentic systems. Um, but
- 7:27:20now it is, right now. Uh, now the
- 7:27:22intelligence is actually at a place
- 7:27:24where where you can really build out
- 7:27:26more complicated systems around them.
- 7:27:28And so I'll I'll give you kind of a
- 7:27:30quick
- 7:27:32background story here of of our journey
- 7:27:36in in working with
- 7:27:38intelligence or or these LLMs. And so
- 7:27:42back in 2023 we built the first version
- 7:27:45of our product. We called it GPT
- 7:27:46Engineer back then.
- 7:27:48And really the UI was very similar to
- 7:27:51what it is today where you kind of have
- 7:27:52a chat and and this preview.
- 7:27:54But the AI at that point was quite
- 7:27:56simple. We just had a prompt and a
- 7:27:59response, one LLM call per user message.
- 7:28:02Very simple. It worked fine.
- 7:28:04But you could kind of build one page
- 7:28:06apps and and simpler applications with
- 7:28:08this.
- 7:28:10And then we thought okay, this
- 7:28:13intelligence is going to keep on
- 7:28:14evolving so we have to be ready for
- 7:28:18for it as it evolves. And so we thought
- 7:28:19let's go out and build this agentic
- 7:28:21framework. And so we we went and we
- 7:28:24built this very sophisticated, dare I
- 7:28:26say beautiful framework for for agents
- 7:28:28to work and
- 7:28:30we had kind of this one main agent that
- 7:28:33would talk to the user and that agent
- 7:28:34would spawn other agents like an
- 7:28:37implementation agent here in the example
- 7:28:39and each agent could have a speciality.
- 7:28:41This agent would kind of specialize in
- 7:28:43the first generation of an app and then
- 7:28:45you would have like a front-end agent
- 7:28:46and a back-end agent and they would all
- 7:28:48talk to each other, send messages back
- 7:28:50and forth
- 7:28:51and we thought oh this is beautiful and
- 7:28:53there were papers around this and how
- 7:28:55well this could work and in the end it
- 7:28:57it didn't work at all. We just made the
- 7:28:59product worse.
- 7:29:00We
- 7:29:02we definitely failed
- 7:29:04on this one.
- 7:29:06But luckily before we ran out of money
- 7:29:08we figured out like hey let's go back to
- 7:29:10a simpler system again. So in late 24 we
- 7:29:13went back to a much simpler
- 7:29:15system where it was a bit of a shame
- 7:29:17where we had like some some smaller LLMs
- 7:29:19doing some knowledge retrieval
- 7:29:21and then really just one main LLM kind
- 7:29:23of writing to the user and writing the
- 7:29:25code in the same message. And this
- 7:29:27worked great. It was very responsive for
- 7:29:29for users in the UI and it was reliable
- 7:29:32and users were kind of in it and and
- 7:29:34iterating with
- 7:29:35with the model.
- 7:29:38And so
- 7:29:39let's close that down.
- 7:29:42But now
- 7:29:43it's time again. We're back. We're so
- 7:29:45back. Agents this time they work
- 7:29:48and we're running them in production
- 7:29:50over a billion tokens per minute on on
- 7:29:52on these systems that I'm going to talk
- 7:29:53about.
- 7:29:55But
- 7:29:56first just looking at okay what is
- 7:29:59actually Lovable? So
- 7:30:01if you simplify the UI and this was same
- 7:30:04back in 2023 as it is today,
- 7:30:06to the left we have a chat where humans
- 7:30:09come in and they kind of describe their
- 7:30:11their intent to to build or whatever
- 7:30:12they want to do and to the right you
- 7:30:14have a preview which is the output or or
- 7:30:17the outcomes of of what you're building.
- 7:30:20And this back in 23 you could do simpler
- 7:30:24things with, right? But now we've really
- 7:30:26seen
- 7:30:28working at this layer of abstraction
- 7:30:29you're able to build very complex
- 7:30:31things, right? And I think we've even
- 7:30:33seen
- 7:30:34a lot of tooling meant for for
- 7:30:36developers for example moving to this
- 7:30:38type of interface where you're not
- 7:30:39really looking at the code every day and
- 7:30:41so on and you're getting kind of
- 7:30:43abstracted and working on on be it
- 7:30:45architecture
- 7:30:47or looking at some kind of output of of
- 7:30:48what you're doing. And
- 7:30:51this is something that we've always
- 7:30:53believed in and it's been kind of a core
- 7:30:55thesis behind Lovable that you won't
- 7:30:57have to care about the code in in the
- 7:30:59same way that you had to do before. And
- 7:31:02I think today we're kind of in this
- 7:31:04almost a bit in between I think in the
- 7:31:05industry where you know some people
- 7:31:06might say oh you don't have to care
- 7:31:08about code at all and some people might
- 7:31:10have different opinions.
- 7:31:12I of course like growing up coding and
- 7:31:15these things you get a bit detached
- 7:31:17but in the end what matters is is
- 7:31:20actually not the code.
- 7:31:22Sorry I thought we removed that last
- 7:31:23part but what what matters is is the
- 7:31:26outcomes, right?
- 7:31:29And so
- 7:31:30the outcomes what is that? Well, for for
- 7:31:33our users, the outcomes is
- 7:31:36creating a business and monetizing and
- 7:31:39creating value for for others in in some
- 7:31:41type of way. And I'll kind of quickly go
- 7:31:44over
- 7:31:46what we build as a platform to to enable
- 7:31:48this before I go in kind of on the
- 7:31:50on the technical side of this talk. So,
- 7:31:52if you if you look at Lovable, the first
- 7:31:55thing that that you can say about the
- 7:31:57platform is that we have this agent and
- 7:31:59like any other agent, it can do quite
- 7:32:01arbitrary things and it's quite
- 7:32:03specialized at at building software
- 7:32:06and it has access to a browser so it can
- 7:32:09verify what it's building and
- 7:32:11it connects to to GitHub and you know it
- 7:32:14writes code that and you can do whatever
- 7:32:15you want for that code. So, so that's
- 7:32:18kind of your basic coding coding agent.
- 7:32:21And the the other pieces of the platform
- 7:32:23is this kind of cloud and and business
- 7:32:25piece of the platform. And so
- 7:32:28the cloud part of the platform is is
- 7:32:30about just giving kind of these
- 7:32:32hosting primitives just seamlessly to to
- 7:32:34this agent. And so we we built out
- 7:32:36databases of authentication, storage,
- 7:32:39all the types of backend compute and
- 7:32:41security scanners and like backups and
- 7:32:43logs for all of these things and all of
- 7:32:44it can be introspected by by the agent.
- 7:32:48And this just makes it seamless to to
- 7:32:49build things. You don't have to kind of
- 7:32:52go to different services to set this
- 7:32:53together. We just kind of provide you
- 7:32:55these primitives.
- 7:32:57So, that's still around building
- 7:32:58software and hosting this, right?
- 7:33:00But then the last piece is around these
- 7:33:03business capabilities where
- 7:33:05again the output of what our users are
- 7:33:07doing on Lovable, we don't see that
- 7:33:08output as code. The output is is a
- 7:33:11product or or a business.
- 7:33:14And if you want to succeed as a
- 7:33:16business, you need to do more things
- 7:33:18than just write code, right? And so we
- 7:33:20we we we help our users with receiving
- 7:33:23payments, sending emails, optimizing
- 7:33:25SEO,
- 7:33:26doing ads on on Google and Meta,
- 7:33:29and kind of analyzing what is happening
- 7:33:31with the product through through for and
- 7:33:34you can look at kind conversions and and
- 7:33:36optimize these sort of things and the
- 7:33:37agent is in the loop in all of this,
- 7:33:40right? So
- 7:33:41that's kind of our
- 7:33:43capability offering you could say.
- 7:33:45And again the core part of this is an
- 7:33:47agent that all of these capabilities are
- 7:33:49built for.
- 7:33:51And
- 7:33:52at some point in Lovel's history we used
- 7:33:54to limit our agents to work for only 30
- 7:33:56minutes because we thought they weren't
- 7:33:58reliable to to work for a longer amount
- 7:34:00of time.
- 7:34:02But now we've changed this to really be
- 7:34:05without any any sort of bound in terms
- 7:34:08of the length an agent can work and so
- 7:34:09you can imagine agents working for for
- 7:34:1110 hours plus.
- 7:34:13And so I'll I'll go into some of the
- 7:34:15things that we've done
- 7:34:17in our architecture to to enable this.
- 7:34:19And so the first piece is really around
- 7:34:22the agent runtime itself and how how we
- 7:34:24enable that to to run for a longer
- 7:34:26amount of time. And the second piece is
- 7:34:29the computer where each of these agents
- 7:34:31are actually doing their work, running a
- 7:34:33dev server
- 7:34:35and actually verifying what it's doing.
- 7:34:38And so so the last piece then it is the
- 7:34:39verification itself where if you want an
- 7:34:42agent to to work for a long amount of
- 7:34:43time you need to make sure that you have
- 7:34:45these feedback loops in place so that
- 7:34:47it's not kind of going off
- 7:34:49in in in the wrong direction.
- 7:34:52And so if we start by looking at the
- 7:34:55agent runtime itself,
- 7:34:57you can imagine that
- 7:34:59you have really three different sources
- 7:35:02that can
- 7:35:04interact with an agent. So it can be
- 7:35:06yourself kind of prompting this or a
- 7:35:08human. It could be other agents and it
- 7:35:10could also be just a system on on a
- 7:35:12schedule waking up. And the way we chose
- 7:35:15to architect this is that for for each
- 7:35:17agent we have a separate inbox
- 7:35:20where any of these sources would kind of
- 7:35:22send in messages. And so you might have
- 7:35:25a message here like what's the status.
- 7:35:27And you can imagine you have this agent
- 7:35:29and this agent is kind of an always on
- 7:35:30agent that they might be working, right?
- 7:35:32And so
- 7:35:33you can imagine it has done like three
- 7:35:35actions here, a request a tool call and
- 7:35:37it got some some result. And then at
- 7:35:39this point, the agent is ready to
- 7:35:40actually receive messages. So
- 7:35:45since anyone can drop a message into
- 7:35:46this inbox, the agent then can copy that
- 7:35:48itself into its own what we call a
- 7:35:51trajectory. Basically, it's a task
- 7:35:52history.
- 7:35:54And this is the type of interaction that
- 7:35:56I think you're familiar with if if
- 7:35:57you're using agents overall.
- 7:36:00So that's kind of the the first design
- 7:36:02principle behind this agent runtime.
- 7:36:05I'll go through a few more here. And so
- 7:36:08the the trajectory in which the the
- 7:36:11agent is working in, which is kind of
- 7:36:13the history of what has happened so far
- 7:36:15and really the basis for any sort of of
- 7:36:18memory for for the agent, right?
- 7:36:21And we designed this trajectory with
- 7:36:24really Git as inspiration. And so
- 7:36:27the trajectory, what is it? It's an
- 7:36:29append-only log where each event
- 7:36:33points towards a parent, the the
- 7:36:35previous event.
- 7:36:37And because of this, you get a nice
- 7:36:41capability just like in Git where you
- 7:36:43can fork things, right? So you can have
- 7:36:45two events pointing to the same parent,
- 7:36:46which essentially creates a fork.
- 7:36:49And because of that, you can kind of
- 7:36:50spin up parallel agents at
- 7:36:54at any point of time in the history
- 7:36:56without having to duplicate all of the
- 7:36:58kind of the shared history.
- 7:37:00And semantically, it's a bit unclear
- 7:37:02what would happen if you merge this. So
- 7:37:04unlike Git, we cannot merge agent
- 7:37:06trajectories, but we can kind of
- 7:37:08spawn them out, right? And these agents
- 7:37:10can can talk to each other.
- 7:37:12The other thing that we've made sure to
- 7:37:14design for is that this agent
- 7:37:17trajectory, it includes kind of the full
- 7:37:19history and everything that happens with
- 7:37:22the agent. And so every tool call,
- 7:37:25a bunch of other things happening on the
- 7:37:26platform, if a tool call like failed or
- 7:37:28succeeded,
- 7:37:29a bunch of what we've done is make sure
- 7:37:31that we have this append-only log. And
- 7:37:34and from that, we can actually build a
- 7:37:35prompt that we would feed into into the
- 7:37:38actual LLM. And you might ask why why
- 7:37:41would you want to do this? Well,
- 7:37:43one one reason is that when you build
- 7:37:45the prompt, you if you have a long
- 7:37:47history, you probably want to do some
- 7:37:48sort of compaction and you want to
- 7:37:50summarize part of that history
- 7:37:52in the in the earlier steps, for
- 7:37:53example.
- 7:37:55But this summarization would differ.
- 7:37:57For example, if you use different models
- 7:37:59with different context lengths, you
- 7:38:00would probably think differently about
- 7:38:02when you should summarize and not. And
- 7:38:05so by separating the the prompt from the
- 7:38:07actual task history, we're able to
- 7:38:09render prompts for for any type of model
- 7:38:11or or any type of task
- 7:38:13that that we can kind of find the best
- 7:38:16and optimal prompt for
- 7:38:19So
- 7:38:20to show this event history in in
- 7:38:22progress, we have this debug tool here.
- 7:38:24So here it's just a recording of me
- 7:38:25dragging this slider in our UI and you
- 7:38:28can see on the right that
- 7:38:31the UI is kind of updating as I'm as I'm
- 7:38:32dragging this along and you have 156
- 7:38:35events here and one each of those is an
- 7:38:37event in this event history. And you can
- 7:38:39see how how the UI can render kind of
- 7:38:42any point in history. And this is just
- 7:38:43showcasing that
- 7:38:45at any point in time, we have full
- 7:38:47control of
- 7:38:49of say the UI or or anything else
- 7:38:51happening on the this platform thanks to
- 7:38:53this history kind of detailing
- 7:38:55everything that has happened.
- 7:38:58The
- 7:38:59the last piece I'll go over here on the
- 7:39:01on the on the runtime itself is that the
- 7:39:04process that is running these agents,
- 7:39:06they're separate from from the actual
- 7:39:08task history. And so the process is
- 7:39:11almost entirely stateless in that
- 7:39:13between each each call, each tool call,
- 7:39:15each turn,
- 7:39:16it will actually just persist into the
- 7:39:18task history. And so you can have a
- 7:39:21process come and go.
- 7:39:23And and then the question is why why do
- 7:39:25we care about this?
- 7:39:26Well,
- 7:39:28at Level Bull, this system
- 7:39:30we deploy to it 700 times per day. So
- 7:39:33that's about once every two minutes. And
- 7:39:36you can imagine if you have
- 7:39:38a agent working for for 10 hours, um you
- 7:39:42want to not that agent when it's
- 7:39:44working, right? This this agent is doing
- 7:39:46something for for a customer. Um but at
- 7:39:48the same time, we want to make sure that
- 7:39:49this agent is running on the latest
- 7:39:51deploy. And it might be a new feature or
- 7:39:53something like that, but it might also
- 7:39:55be like a security vulnerability in in
- 7:39:58the Linux kernel. And it's like, okay,
- 7:39:59we need to update this ASAP. Um and and
- 7:40:02so to do that, thanks to this separation
- 7:40:04between the actual history and and the
- 7:40:06processes, we can actually make sure
- 7:40:09that we have one one agent that is
- 7:40:11working on one task, but we can have
- 7:40:13different processes taking up and
- 7:40:15actually doing the compute behind that
- 7:40:17work.
- 7:40:18Um so so this task can keep on working
- 7:40:20while we have um different deploys um um
- 7:40:24working on it,
- 7:40:25be it for 10 hours or or even longer,
- 7:40:27right?
- 7:40:28Um so that's the agent runtime. The
- 7:40:31second piece of um of this platform,
- 7:40:34right, is the is the computers. This is
- 7:40:36where the work is actually happening. Um
- 7:40:38and we've been through a lot of
- 7:40:40iterations and a lot of vendors kind of
- 7:40:42working with um um providing these
- 7:40:45sandboxes for for the computer for for
- 7:40:47the agent to work in. And we've landed
- 7:40:49in in the system that we've worked
- 7:40:51really hard now to develop, which is um
- 7:40:54um a checkpointing um a computer. And so
- 7:40:59you can imagine you come to you start a
- 7:41:00new project. We have this common what we
- 7:41:02call a golden checkpoint, which contains
- 7:41:06the OS and the system services. It has
- 7:41:08common tools installed and a dev server
- 7:41:10that is ready to start. And this
- 7:41:12checkpoint it includes everything on
- 7:41:14disk, memory, and also running
- 7:41:16processes.
- 7:41:17And and so when you start a new project,
- 7:41:19you get all of that for free. And all of
- 7:41:22that is copied over in about 230
- 7:41:23milliseconds.
- 7:41:25Um and so
- 7:41:27once you start a project, you you might
- 7:41:28have some specific code and and some
- 7:41:30dependencies within that project.
- 7:41:32Um and um that is also checkpointed as
- 7:41:36you go. And and so each time the the
- 7:41:39agent is running and it finishes a turn,
- 7:41:43we'll actually save a checkpoint of of
- 7:41:44that
- 7:41:45And and from that we're able to again
- 7:41:47like very quickly, sub-second,
- 7:41:50spin that computer up whenever you need
- 7:41:52it. That means we can save on on
- 7:41:53resources by kind of spinning this down
- 7:41:55if you leave for say
- 7:41:5710 minutes or or so and come back. We
- 7:41:59can just spin it up extremely quickly so
- 7:42:01you don't even notice this. And even
- 7:42:03kind of requests
- 7:42:05in transit will will not even be dropped
- 7:42:07in in this setup.
- 7:42:10The other thing we can do thanks to this
- 7:42:12checkpointing is that just like the
- 7:42:15trajectory itself in the agent runtime,
- 7:42:17we can also fork
- 7:42:19the computer. And when we do this
- 7:42:23we have for example we have a feature in
- 7:42:26in Level that is called drafts where you
- 7:42:27kind of work in different environments
- 7:42:29of what you're doing and and to support
- 7:42:30that we need to fork the computer that
- 7:42:32you're working in.
- 7:42:34And
- 7:42:35here we have
- 7:42:38kind of
- 7:42:39the example of doing two at the same
- 7:42:40time and you can imagine doing hundreds
- 7:42:43of these, right?
- 7:42:44And so
- 7:42:46we've we've had to do
- 7:42:48some optimizations to make this feasible
- 7:42:50to to actually run. So one of these
- 7:42:52optimizations is copy-on-write on the
- 7:42:54RAM itself. And so creating one of these
- 7:42:57forks of a computer is is basically free
- 7:42:59in terms of of the RAM that it's using.
- 7:43:02But
- 7:43:03as soon as you start writing
- 7:43:05in in a process to to RAM we will kind
- 7:43:07of copy that page over and only then
- 7:43:10will you start having to allocate new
- 7:43:12memory.
- 7:43:15And there's another optimization here
- 7:43:16also which is that each of these
- 7:43:19sandboxes they require a few hundreds of
- 7:43:21megabytes of memory.
- 7:43:23But and that would take a few seconds to
- 7:43:25stream in if you have it in like cold
- 7:43:27object storage and then it has to go to
- 7:43:28disk and then then into memory of this
- 7:43:31this VM. But
- 7:43:34thanks to
- 7:43:36being able to stream the memory we can
- 7:43:37actually boot up the the machine before
- 7:43:41all of the memory is is actually
- 7:43:44ready on it. So
- 7:43:46the the memory the OS uses comes first
- 7:43:48and then like some processes and so on
- 7:43:49will start to work as soon as the memory
- 7:43:51gets here. And this also shapes off some
- 7:43:54some extra startup time. And then we
- 7:43:56started asking ourselves, how can we
- 7:43:58make this even more efficient and even
- 7:43:59more faster for our users? And so you
- 7:44:01might be familiar with Vite. This is
- 7:44:03great dev server that most of the meta
- 7:44:05frameworks out there in the JavaScript
- 7:44:07ecosystem is using.
- 7:44:09And so we took that and we forked it and
- 7:44:11we rewrote it in Rust. And that made our
- 7:44:14cold starts two times faster and we went
- 7:44:16from getting this preview from 14
- 7:44:18seconds to three seconds. And the main
- 7:44:21kind of thing that we made faster here
- 7:44:22is that Vite uses Node.js and we're able
- 7:44:25to like drop that dependency of Vite
- 7:44:27itself and just have this Rust binary.
- 7:44:29So
- 7:44:30that made the memory footprint be 6.5
- 7:44:32times less
- 7:44:34in these machines.
- 7:44:35And by taking control of the
- 7:44:37applications that we're running in these
- 7:44:39computers, we're also able to control
- 7:44:40kind of how we write to memory. And so
- 7:44:44when we control how we write to memory,
- 7:44:46we're even more optimizing if you
- 7:44:47remember back to the kind of the
- 7:44:48copy-on-write solution when we're
- 7:44:50drafting different
- 7:44:52different computers. And so we're able
- 7:44:54to
- 7:44:55run this extremely efficiently so that
- 7:44:56we can run
- 7:44:58lots and lots of agents in in parallel
- 7:45:00and kind of have agents explore
- 7:45:02different solutions to solving a bug or
- 7:45:04improving the performance of your
- 7:45:06applications and so on.
- 7:45:10And so this has been fun.
- 7:45:12But the last piece now is about
- 7:45:14verification. And of course this kind of
- 7:45:16computer and this dev server is a big
- 7:45:18piece of being able to verify what an
- 7:45:20agent is doing.
- 7:45:22And if you look at verification, you can
- 7:45:25think of these feedback loops, right?
- 7:45:27That the agent needs to get information.
- 7:45:29It took an action. Was this a good
- 7:45:30action or was it a bad action? And
- 7:45:33you can think of it as this inner loop
- 7:45:35and this outer loop. So this inner loop
- 7:45:36is during the turn of the agent and you
- 7:45:39have kind of obvious things such as like
- 7:45:40the
- 7:45:41is the build working? Do we have lint
- 7:45:43errors or type errors in the code that
- 7:45:44we're running? And then you can check
- 7:45:46for runtime errors in in the preview
- 7:45:48that you're running. You can also
- 7:45:50within this computer a headless browser
- 7:45:52where the agent can go in and kind of
- 7:45:54explore
- 7:45:55the the web app it's it's building or
- 7:45:57whatever it's building, right?
- 7:45:59And and a bunch of other things that
- 7:46:01we're using to help verify within the
- 7:46:03turn of an agent. But then you also have
- 7:46:05this outer loop which is kind of between
- 7:46:07each turn and the turn you can think of
- 7:46:09like when the agent is yielding to you
- 7:46:11as a user.
- 7:46:12And
- 7:46:14between each turn you might get some
- 7:46:15info on the product itself from from
- 7:46:18production from your users, right? And
- 7:46:20so you can look at how your visitors or
- 7:46:21behavior are behaving,
- 7:46:24what users are are telling you online.
- 7:46:26You can do automatic AB tests and
- 7:46:28actually see change some copy and see is
- 7:46:31this improving conversion and so on. And
- 7:46:33then you can also take these business
- 7:46:34metrics and remember this goes back to
- 7:46:36kind of the capabilities I showed you.
- 7:46:38By having control over these things we
- 7:46:39can make sure that we can optimize how
- 7:46:41the agent is is getting this outer
- 7:46:43feedback loop so we can make sure that
- 7:46:46if you're running payments we can
- 7:46:47actually track those transactions and
- 7:46:50conversions and make sure that
- 7:46:52the agent can actually help you optimize
- 7:46:54not only the the code, right? But the
- 7:46:57business strategy
- 7:46:59around around your product.
- 7:47:02And so those are really the the three
- 7:47:04things that that we've been working for
- 7:47:07and we'll continue to kind of build out
- 7:47:11these these capabilities and these
- 7:47:13architecture around really orchestrating
- 7:47:17these LLMs into into agents. And so
- 7:47:20this agentic system that we're building
- 7:47:21it's running billions of tokens per
- 7:47:23minute
- 7:47:24and
- 7:47:26these computers, right? We can spawn up
- 7:47:27with each agent and we can have multiple
- 7:47:29agents use one computer and now we're
- 7:47:31kind of back to this
- 7:47:33quite beautiful setup where you have
- 7:47:34agents talking to agents and they're
- 7:47:36orchestrating and and this time it's
- 7:47:39it's actually working and it's working
- 7:47:41at scale thanks to some of these
- 7:47:43optimizations.
- 7:47:44And so if you're curious to try this out
- 7:47:46you can head to level.dev and you'll
- 7:47:48you'll see our our preview spinning up
- 7:47:50and and so on but you'll get a very
- 7:47:51simple and
- 7:47:53um
- 7:47:53a nice interface and you won't have to
- 7:47:55care about all of this stuff.
- 7:47:57Thank you so much. It's been pleasure
- 7:47:59um and uh yeah, hope you hope you
- 7:48:01learned something. Thanks so much.
- 7:48:03>> [applause]
- 7:48:15>> Building a coding agent is one thing.
- 7:48:17Building an engineering culture around
- 7:48:19it is another.
- 7:48:21To share how agent-driven development is
- 7:48:23transforming the way teams work, please
- 7:48:26welcome Zach Brugman, principal software
- 7:48:29engineer at Ramp.
- 7:48:49>> [applause]
- 7:48:52>> Thank you so much. So let's get a second
- 7:48:54here.
- 7:48:56Any 49ers fans?
- 7:48:59Okay, you know that's a better reaction
- 7:49:00I've gotten at other events this year
- 7:49:01when I've asked, you know, about their
- 7:49:03local sports team cuz I've gotten a lot
- 7:49:04of just like muted silence I have to hit
- 7:49:05the, you know, well wrong audience for
- 7:49:07that type of joke, but glad at least I
- 7:49:09could hit this time.
- 7:49:28Do you need help?
- 7:49:34>> Do you want to just go for it?
- 7:49:46>> Current slide on and then
- 7:50:01>> All right. Good. Well, thanks so much
- 7:50:03for waiting, everyone. My name is Zach
- 7:50:05and I'm an engineer at Ramp. I'm just
- 7:50:07really excited to tell you all a bit
- 7:50:08about how Ramp has evolved its
- 7:50:09engineering culture this year through
- 7:50:11the help of our robotic friend, Inspect.
- 7:50:14As a quick introduction, Inspect is
- 7:50:16Ramp's collaborative coding agent. About
- 7:50:18a year ago, we decided to take a bet on
- 7:50:20building our own coding agent with every
- 7:50:22session running remotely and being fully
- 7:50:24multiplayer. Inspect can work alongside
- 7:50:26you and your team, whether that be in
- 7:50:28Inspect itself, Slack, Linear, or any
- 7:50:31other of our integrations. And now it
- 7:50:33goes far beyond just writing the code
- 7:50:36with the built-in design primitives,
- 7:50:38review agents, and verification flows
- 7:50:40that let any builder go from prompt to
- 7:50:42shipping all inside Inspect. Inspect is
- 7:50:45also connected with all of our
- 7:50:46integrations company-wide, so everyone's
- 7:50:48automatically connected to the to the
- 7:50:49data and tools that matter most, whether
- 7:50:51that be telemetry, databases, or any of
- 7:50:54the other tooling available.
- 7:50:57Through this, we now see around 75% of
- 7:50:59all of Ramp's code being being written
- 7:51:03and shipped by Inspect.
- 7:51:05When we started sharing Inspect publicly
- 7:51:07at the start of the year, the technology
- 7:51:08itself was the more novel part, spinning
- 7:51:10up sandboxes in under 5 seconds, wiring
- 7:51:13up all that internal context, having
- 7:51:15agents work together in a multiplayer
- 7:51:16setting. People had a lot of questions
- 7:51:18about the plumbing. But now with much
- 7:51:21broader adoption of these concepts and
- 7:51:23Inspect itself now being a product on
- 7:51:24the market, the technology isn't nearly
- 7:51:26the barrier that it once was.
- 7:51:28Instead, I want to talk about the other
- 7:51:30angle, the lessons we've learned around
- 7:51:31building a culture around Inspect and
- 7:51:33what you can take away to inspire the
- 7:51:35people and teams that you work with.
- 7:51:39Uh I'm not going to talk as much about
- 7:51:40those technical aspects of Inspect
- 7:51:41today, but if you are curious about
- 7:51:43that, you can read our blog posts or
- 7:51:45take a look at Pragmatic Engineer's deep
- 7:51:47dive that they did with us a few months
- 7:51:48ago. You can also follow my boy Dylan
- 7:51:51Garcia on Twitter. He's on his social
- 7:51:53media grind, but he's also been posting
- 7:51:55a ton of really great Inspect insights
- 7:51:57uh we've launched it. So, shout out to
- 7:51:59Dylan. Please go to him a follow.
- 7:52:02One thing about working at Ramp is that
- 7:52:04it really makes you start thinking about
- 7:52:06everything through the lens of saving
- 7:52:07time and money.
- 7:52:09although I think it is important to
- 7:52:11understand the systems that you work
- 7:52:12with and understand how your tooling
- 7:52:14works, if you're a multi-hundred or
- 7:52:16multi-thousand person organization and
- 7:52:18everyone has to go through say 15
- 7:52:20minutes of setup,
- 7:52:22once you start multiplying that across
- 7:52:24an organization, this is days lost for
- 7:52:27what is very rote work.
- 7:52:29Then, if you consider how fast the
- 7:52:31frontier moves here on best practices
- 7:52:33and new ways to set up an agent, you
- 7:52:35realize this setup cost isn't just one
- 7:52:37time.
- 7:52:38Not to mention, these advancements won't
- 7:52:41always spread equally. So, now you have
- 7:52:43an indeterminate amount of time and
- 7:52:44effort spent on this maintenance without
- 7:52:46an equal spread of knowledge leading to
- 7:52:48a disparate outcomes across an
- 7:52:50organization.
- 7:52:51While you can perceive this as an
- 7:52:52advantage on an individual level, when
- 7:52:55you're trying to win, you need everyone
- 7:52:56on your A-game.
- 7:52:58So, this was our first lesson, build
- 7:53:01tools that kill information silos.
- 7:53:03When we found patterns that worked well
- 7:53:05or tools that benefited everyone, we
- 7:53:08could push it to our entire organization
- 7:53:10immediately. If you want to spend time
- 7:53:12tweaking your own setup, that's fine,
- 7:53:13but inspect is always available as a
- 7:53:15very solid baseline. And what we've
- 7:53:18found is that we've never needed to
- 7:53:19mandate usage of inspect because it
- 7:53:21existing as this baseline reduced the
- 7:53:24need for people to reach for their own
- 7:53:25setups all the time.
- 7:53:29Part of what I think makes this powerful
- 7:53:30as well is that the reality of this
- 7:53:32industry is that it's unlikely that you
- 7:53:35as an individual always know the
- 7:53:37absolute best way to work with agents
- 7:53:39across everything.
- 7:53:40Even if keeping up with this is your
- 7:53:42full-time job, we all miss things, hold
- 7:53:44biases, and have blind spots. So, part
- 7:53:47of the design of inspect is that there's
- 7:53:49just one single inspect for the whole
- 7:53:51company.
- 7:53:52When people figure out better ways to
- 7:53:54use it or have ideas for what would make
- 7:53:56it better for the work they do,
- 7:53:58we're able to bring it to the entire
- 7:53:59company.
- 7:54:00The best part of this is that for most
- 7:54:02changes, this is invisible to everyone
- 7:54:04else. You just keep using Inspect as you
- 7:54:07normally would and benefit from the
- 7:54:08improvements that everyone's making over
- 7:54:10time.
- 7:54:12And what we've found is that we really
- 7:54:13don't have to run these formal sit down
- 7:54:16here's how you use AI effectively
- 7:54:18training sessions for Inspect.
- 7:54:20With every Inspect session being public
- 7:54:22within the company by default, it's
- 7:54:24actually easier to work in public than
- 7:54:26in private. As such, you default to
- 7:54:29working in public and you can learn how
- 7:54:31to use it by watching how your teammates
- 7:54:33use it in the channels that you're
- 7:54:34already in and your teammates can learn
- 7:54:37from you.
- 7:54:38You can literally ask Inspect to go
- 7:54:39through their recent sessions and tell
- 7:54:41you about how they work, how they
- 7:54:42structure prompts, or how they break
- 7:54:44down gnarly problems.
- 7:54:47I I think this is not just been helpful
- 7:54:49for the builders at Ramp, but also for
- 7:54:51the Inspect team itself.
- 7:54:53We frequently see the way that people
- 7:54:55use Inspect in ways that we didn't
- 7:54:57originally imagine. This then informs
- 7:55:00what we should build towards, whether
- 7:55:01that should be more native primitives
- 7:55:03for common patterns or where tools and
- 7:55:05data are missing.
- 7:55:07All in all, it creates this
- 7:55:08self-sustaining loop where we're able to
- 7:55:10build towards what makes Inspect the
- 7:55:12most successful by seeing what people
- 7:55:14are already achieving with Inspect.
- 7:55:18This feeds into the second lesson that
- 7:55:19we've learned. It's all about building
- 7:55:21trust. I think it's fair to say that
- 7:55:24adopting agentic driven coding has been
- 7:55:26a massive trust building exercise for
- 7:55:28every engineer this year. It's helped
- 7:55:30that the models have become more
- 7:55:32competent at writing high quality code,
- 7:55:33but there's a large part of this that's
- 7:55:36just learning to trust what an agent
- 7:55:37builds.
- 7:55:39So much of building trust and tools
- 7:55:41comes from learning from other people.
- 7:55:43After all, that's why most of us are
- 7:55:44here, right? You wanted to listen to
- 7:55:46people you respect, or at least I hope
- 7:55:49you respect, teach you about the things
- 7:55:51that they learn. But if we limit our
- 7:55:53corpus of knowledge to just those who
- 7:55:55find the perverse enjoyment in public
- 7:55:56speaking,
- 7:55:58we're still not getting the full
- 7:55:59picture.
- 7:56:01This democratization of knowledge
- 7:56:02sharing is just to me as a concept.
- 7:56:05Instead of grouping around a select few
- 7:56:07leaders, can we build systems that
- 7:56:09inherently share the knowledge of
- 7:56:11everyone?
- 7:56:12This is what Inspect has built towards.
- 7:56:16Every time someone ships something with
- 7:56:18Inspect, people can see it right there
- 7:56:20on the work you've done. The session
- 7:56:22itself is an easy to share. It's living
- 7:56:24artifact of how that change came to be.
- 7:56:27When a pull request goes up, anyone
- 7:56:29reviewing it doesn't just see the 400
- 7:56:31line diff dropped over the wall and hope
- 7:56:33for the best. They can see the exact
- 7:56:35conversation that you had,
- 7:56:37all the tools Inspect used, before and
- 7:56:40after screenshots, or even a video
- 7:56:42recording of the verification agent
- 7:56:44clicking through the app in the browser
- 7:56:46to prove its work holds up.
- 7:56:48It makes it effortless to show not just
- 7:56:50what changed, but why you can trust it.
- 7:56:54It also just makes it easier to bring
- 7:56:56people you don't work with all the time
- 7:56:57into the fold.
- 7:56:59A fun moment on this one, it was back in
- 7:57:01February, one of our engineers, Andrew,
- 7:57:03reported a bug in Inspect. So,
- 7:57:05naturally, I reached for Inspect and
- 7:57:07asked it to investigate.
- 7:57:09What's funny here is that although
- 7:57:11Inspect had access to Datadog for a
- 7:57:13while, we'd only just finished
- 7:57:14instrumenting Inspect itself.
- 7:57:16I mean, Inspect was primarily built by
- 7:57:18front-end and product engineers, so
- 7:57:20tracing is a little bit of a foreign
- 7:57:21concept for us. But in any case, it was
- 7:57:24a great time for us to put it to the
- 7:57:26test for Inspect. And Inspect says,
- 7:57:30the instrumentation was causing the
- 7:57:31error. Yes, the very instrumentation
- 7:57:33that was letting us debug this error was
- 7:57:35actually causing this error.
- 7:57:37So,
- 7:57:38Dylan and I were not exactly convinced
- 7:57:40by the robot here. And Andrew is just
- 7:57:42like, whatever, I'll use Claude code.
- 7:57:45And And while we don't have a mandate to
- 7:57:47use Inspect, I mean, personally, these
- 7:57:50were not the words I wanted to hear.
- 7:57:52So, nobody here is really trusting
- 7:57:55anybody.
- 7:57:56Dylan and I don't trust the robot's
- 7:57:58explanation.
- 7:57:59Andrew doesn't trust us to build
- 7:58:01something that works. And Inspect,
- 7:58:03well, I I I don't know if Inspect trust
- 7:58:05anyone, but that's okay. That's okay.
- 7:58:08But because this is happening together
- 7:58:10in public,
- 7:58:11we keep prodding at it. Dylan asked
- 7:58:14Inspect to look through the code of the
- 7:58:15telemetry library using and see if this
- 7:58:17was just something that was happening
- 7:58:19before without visibility, because
- 7:58:21surely by now coding has been solved and
- 7:58:23there can't be bugs that we introduced.
- 7:58:25And this time it comes back with
- 7:58:26something a bit more specific. The hotel
- 7:58:29library calls fetch, but it never reads
- 7:58:31or cancels the response body. This was
- 7:58:33just a Cloudflare platform limitation at
- 7:58:35the time, albeit happening in a library
- 7:58:37specifically made for Cloudflare
- 7:58:38workers. But that happens. This library
- 7:58:40was written pre-AGI after all.
- 7:58:43At this point, we ask Inspect to make a
- 7:58:45patch. It updates the exporter to
- 7:58:47explicitly cancel the response body.
- 7:58:50We grumbled in the thread a bit while it
- 7:58:52was deploying, but we see after it
- 7:58:54deployed, there haven't been any errors
- 7:58:55since. So great. Inspect fixed a bug,
- 7:58:58which at this point it's nothing special
- 7:59:00in itself, but there was a much more
- 7:59:02important story playing out in this
- 7:59:04thread.
- 7:59:06Before Inspect, a report like this might
- 7:59:08have been acknowledged, a ticket filed,
- 7:59:11and then one of us would have gone away
- 7:59:12to debug it. Andrew might have received
- 7:59:14a message later saying it was fixed, but
- 7:59:16the interesting work would have been
- 7:59:17done in private.
- 7:59:19Now though, it's easier to do all of
- 7:59:21this in public.
- 7:59:23And as such, Andrew naturally starts
- 7:59:25becoming engaged, wanting to chime in
- 7:59:27with his thoughts and experiences. And
- 7:59:29then in that same thread, this leads to
- 7:59:31ideas coming up about how Inspect could
- 7:59:33power other work. And none of this
- 7:59:35required any prodding or pressure to
- 7:59:37work together. Instead, it was just
- 7:59:38like, oh, these guys are doing something
- 7:59:40cool, and we could work on it together.
- 7:59:45So in the span of an hour, we go from a
- 7:59:47skeptical user with a broken user
- 7:59:49experience to an engaged colleague
- 7:59:51wanting to build together and a more
- 7:59:53stable Inspect.
- 7:59:55So once you have a tool that gives
- 7:59:57everyone the best setup on day one with
- 7:59:59no information silos and you've built a
- 8:00:02system of trust that brings innovation
- 8:00:04together, what happens next?
- 8:00:06This is the last lesson I want to share.
- 8:00:09I think the only thing left is to think
- 8:00:10bigger together.
- 8:00:13Something that's pained me this year uh
- 8:00:15is how much reverence has been built for
- 8:00:16the lone wolf operator. It's great that
- 8:00:19we can all do more individually with the
- 8:00:21help of AI, but this can't be the
- 8:00:23endgame in my opinion.
- 8:00:25If we don't build outwards uh
- 8:00:27collaborate with other people and
- 8:00:29continue to share the progress and
- 8:00:30learnings we make, I think this is all
- 8:00:33going to collapse in on itself. We'll
- 8:00:35only consider ourselves as our target
- 8:00:36audience and we won't grow through the
- 8:00:38consideration of others' perspectives
- 8:00:40and experiences.
- 8:00:42This is not to say I'm all doom and
- 8:00:44gloom about this. The fact we're all
- 8:00:45here hopefully is a testament to the
- 8:00:47value of connection.
- 8:00:49More importantly though, I think we can
- 8:00:50shape our tools to enable these
- 8:00:52connections to flourish even stronger
- 8:00:53than before.
- 8:00:56That story I told about Andrew has
- 8:00:57played out so many times within Ramp.
- 8:00:59It's probably the most inspiring thing
- 8:01:01about Inspect to me, honestly.
- 8:01:03Sure, it's a coding agent, but
- 8:01:05remarkably it's become part of the
- 8:01:07culture. Everyone at Ramp knows its
- 8:01:09face. We dress it up on holidays. We
- 8:01:12throw in Easter eggs over time and we
- 8:01:15evolve it as Ramp itself has evolved.
- 8:01:18Not all iterations have been perfect as
- 8:01:20any Ramp engineer could tell you, but I
- 8:01:22do think every iteration has been
- 8:01:24together.
- 8:01:26Going back to earlier this year, Inspect
- 8:01:28didn't have nearly the amount of
- 8:01:29functionality it has today. Uh great
- 8:01:31example is its code review agent.
- 8:01:34Before this, we were using various
- 8:01:36off-the-shelf code review products,
- 8:01:37frequently rotating between them to find
- 8:01:39a good fit.
- 8:01:41At some point though, a couple
- 8:01:42engineers, Daniel and Pranjal, realized
- 8:01:44that we could probably build it
- 8:01:45ourselves as Inspect already had
- 8:01:47automation triggers that fired from a
- 8:01:49variety of sources, including when a
- 8:01:52pull request was opened.
- 8:01:53With just what we built already, they
- 8:01:55got to work building another beloved
- 8:01:57agent at Ramp, Review Buddy. And what
- 8:02:00was so cool about this is that much like
- 8:02:02Inspect's own origins as a side project,
- 8:02:05these engineers were working on this.
- 8:02:07They didn't need to ask for permission
- 8:02:08to build it. They just challenged
- 8:02:10themselves to build something better,
- 8:02:12used Inspect to move faster, and quickly
- 8:02:14built an agent that was better for all
- 8:02:16of ramp.
- 8:02:18This really did force us working on
- 8:02:20Inspect to think about Inspect
- 8:02:22differently. At that point in time, we
- 8:02:24were pretty okay with Inspect quote
- 8:02:26unquote just being a coding agent. But
- 8:02:29once we saw what others were pushing it
- 8:02:31to do,
- 8:02:32it got us thinking about how Inspect
- 8:02:34could become full circle.
- 8:02:36If this push didn't happen naturally, I
- 8:02:38think it would have taken the Inspect
- 8:02:40team a lot longer to get there.
- 8:02:44This is what I mean about thinking
- 8:02:45bigger together.
- 8:02:47Especially with the continued
- 8:02:48improvements in model performance, now's
- 8:02:50the time to push yourself and the people
- 8:02:52around you to think bigger. I encourage
- 8:02:55all of you to take some time in this
- 8:02:57coming week to ask everyone that you
- 8:02:58work with, what have we been waiting to
- 8:03:01do that we haven't done yet?
- 8:03:03And once you've found those things, try
- 8:03:06to build them together.
- 8:03:07There's really not an excuse to not just
- 8:03:09try anymore. You have access to the same
- 8:03:11models as everyone else, but you have to
- 8:03:14start somewhere. And I believe that
- 8:03:16there is so much potential in unlocking
- 8:03:18wins not just for yourself, but for the
- 8:03:20people around you.
- 8:03:22Inspect itself was born this way, not
- 8:03:24only as something that I wanted to build
- 8:03:26and Jason wanted to build, but out of a
- 8:03:28want to build something for a team that
- 8:03:30I truly love and truly respect that
- 8:03:33would just make their jobs a little more
- 8:03:34enjoyable.
- 8:03:36Now, if you'd like to use a coding agent
- 8:03:38that has embodied all these lessons that
- 8:03:40we've learned, I'm happy to say that
- 8:03:41Inspect is now available outside of
- 8:03:43ramp. If you go to inspect.com/aie,
- 8:03:47we'll prioritize you and your team on
- 8:03:48the wait list. We're actually moving
- 8:03:50through this wait list pretty quickly,
- 8:03:51so expect to hear from us soon. But
- 8:03:53otherwise, I hope you've enjoyed hearing
- 8:03:55about our robotic friend Inspect. If
- 8:03:57these topics interest you, feel free to
- 8:03:58reach out here or on X. But otherwise,
- 8:04:01thanks so much for listening.
- 8:04:03>> [applause]
- 8:06:02>> From building developer tools to
- 8:06:03exploring the future of autonomous
- 8:06:06software development, our next speaker
- 8:06:08has spent years pushing the boundaries
- 8:06:11of what's possible. Ladies and
- 8:06:13gentlemen, please welcome Steve Yegge,
- 8:06:16independent AI advisor and creator of
- 8:06:19Beads and Gas-town.
- 8:06:31>> Hey everybody.
- 8:06:33All right. Exciting times. Thank you.
- 8:06:35The room is really full. That's awesome.
- 8:06:37Thank you all for coming at the end of
- 8:06:39the day.
- 8:06:40Uh I'm really excited to be here and to
- 8:06:43chat with you. I write uh uh
- 8:06:46custom speech every single time. So this
- 8:06:48is
- 8:06:49the speech was written just for you
- 8:06:51today based on talking to a bunch of
- 8:06:53you.
- 8:06:55Uh I've been in the industry for about
- 8:06:5640 years. I worked at Amazon and Google
- 8:06:59for about 20 years. I've been all over
- 8:07:02the place, done all kinds of stuff.
- 8:07:04And
- 8:07:05uh and for the last year or a little bit
- 8:07:07more, I've been independent and just
- 8:07:09kind of like playing with um
- 8:07:11playing with software factories, yeah?
- 8:07:13So um I kind of invented them,
- 8:07:17right? On January 1st, I launched
- 8:07:18Gas-town. I don't even think we called
- 8:07:20them software factories back then. I
- 8:07:21think I called it an orchestrator.
- 8:07:23But I knew that I wanted Claude to run
- 8:07:25Claude,
- 8:07:26right? And it turned out to be a little
- 8:07:28trickier than than I thought. I spent
- 8:07:30all last year on it and I wrote
- 8:07:33That's interesting. So I can't see my
- 8:07:35slides, but you can see my slides. So
- 8:07:37that'll work. So I wrote two
- 8:07:38orchestrators last year.
- 8:07:40They were an attempt to make the LLMs
- 8:07:43smarter.
- 8:07:45Vibe Coder and VC, you know, to do
- 8:07:47engineering stuff like
- 8:07:49you know, do code reviews and all that
- 8:07:51stuff, right?
- 8:07:52Really, really bad idea. Just complete
- 8:07:55failure. I just built these huge
- 8:07:56monoliths. I was fighting the bitter
- 8:07:58lesson.
- 8:07:59Don't try to make LLM smarter. They will
- 8:08:00get smarter.
- 8:08:02And after a couple of months of this,
- 8:08:04although Beads did fall out of it and
- 8:08:06Beads was a really cool discovery and
- 8:08:07Beads Beads is is life. Who here uses
- 8:08:11Beads?
- 8:08:12A few, yeah? Pretty good. Actually
- 8:08:14pretty good, yeah.
- 8:08:16We'll talk a little a little about it.
- 8:08:17Excuse me. [clears throat]
- 8:08:18And then I had GasTown, and then I had
- 8:08:20in June, I had Wheelhouse, which was my
- 8:08:23own. So, I decided to build a factory
- 8:08:25from scratch just to work on my video
- 8:08:27game, right?
- 8:08:29Cuz like everyone's building factories
- 8:08:30to build factories. It's kind of navel
- 8:08:33gazing. It's kind of like you know
- 8:08:34right? You know it's like what? They're
- 8:08:36not you know it's I think more people
- 8:08:38need to build
- 8:08:39stuff, not just tools for building
- 8:08:41stuff, right? And I I think this is
- 8:08:43going to be a theme today, right?
- 8:08:45And the thing is the one pattern that
- 8:08:47I've seen here is all of them
- 8:08:50go through a phase where the models get
- 8:08:52really excited and they build something
- 8:08:53really cool for you and then
- 8:08:56it just turns into garbage mountain.
- 8:08:59Every single time it happened to
- 8:09:01GasTown, it happened to Wheelhouse
- 8:09:03several times. I had to burn the whole
- 8:09:05thing to the ground a couple times,
- 8:09:07right? It's almost like like every model
- 8:09:10family, every model generation comes in
- 8:09:12and can build something that's just a
- 8:09:14little too big for it to handle. And
- 8:09:16then the next model generation comes in
- 8:09:17and goes, yeah, we're going to have to
- 8:09:20burn that, right? And I've I it's a
- 8:09:22pattern. I've done this over and over
- 8:09:23again and I can tell you some things
- 8:09:25about it, but
- 8:09:26first let me tell you about my game.
- 8:09:27It's a fun little RPG. I've been working
- 8:09:29on it for 30 years. It has players that
- 8:09:33started in 2001. So, I literally have
- 8:09:35players that have been playing for 25
- 8:09:37years. They're still playing. Like you
- 8:09:39can log in right now and find probably
- 8:09:4010 people just screwing around doing
- 8:09:42quests, live quests and and whatever.
- 8:09:45It's a it's a player
- 8:09:47you know the players contribute the
- 8:09:48content, right? It's a mud. Like they
- 8:09:51were really big in the '90s. They're
- 8:09:52like text games and this one has
- 8:09:53graphics on top of which turns out to be
- 8:09:55like really ideal for AI agents cuz it's
- 8:09:59text. They can literally play the game
- 8:10:00via Telnet.
- 8:10:02And they asked me to.
- 8:10:04Uh so, we'll talk about that too. Here's
- 8:10:05an older architecture diagram of what my
- 8:10:08sort of of what my system looks like.
- 8:10:11So, when I built this Wheelhouse
- 8:10:14software factory
- 8:10:15for Wyvern without a care in the world
- 8:10:18for making it you know usable by
- 8:10:22right? No open sourcing, just bespoke.
- 8:10:27I discovered the alien face hugger
- 8:10:29pattern, all right? Which is where
- 8:10:32your factory turns into an alien face
- 8:10:33hugger and like gloms onto your product
- 8:10:37and just becomes
- 8:10:40grown into it.
- 8:10:42Okay?
- 8:10:43Like look, we could turn off the AI and
- 8:10:46still run the game today. Me and the
- 8:10:47admins and the players and the community
- 8:10:49and we could we could But we would miss
- 8:10:52the AI at this point.
- 8:10:54The players are talking to my factory.
- 8:10:57The factory's talking to my admins.
- 8:10:59They're asking for features. I'm not
- 8:11:01even in the loop. It's like a wish
- 8:11:02factory. They're building the game. Now
- 8:11:04they're all old timers and there's a set
- 8:11:06of them that I trust to make, you know,
- 8:11:08wise decisions about the game.
- 8:11:10But they're they're all working with my
- 8:11:12factory now and they'd be mad. The funny
- 8:11:14thing is when it first started, they
- 8:11:16were like, timeout.
- 8:11:18What is this AI stuff for starters?
- 8:11:20And they were like, no AI art.
- 8:11:23Cuz they could see it was slop. And I'm
- 8:11:24like, okay, we got human artists.
- 8:11:26And they said, but we do kind of like
- 8:11:28all the new features, but
- 8:11:30we're really scared of the pace. Can you
- 8:11:31give us a roadmap?
- 8:11:33My players came to me asking for a
- 8:11:35roadmap. They were like, you got to slow
- 8:11:36down. It's too much.
- 8:11:38[snorts] And I was like, okay. And I
- 8:11:39took a look and my software factory,
- 8:11:42this thing, had been cranking out like
- 8:11:4450 to 100 features a day. I had 100
- 8:11:47years of work in my backlog when I
- 8:11:49started this and Claude got through
- 8:11:50about 60 years of it in about four
- 8:11:52months, right? It was absolute insanity.
- 8:11:54And they can't look at 100 features a
- 8:11:57day. They can't even like read about 100
- 8:11:59features a day, let alone play test
- 8:12:01them.
- 8:12:02I was My factory was completely
- 8:12:03overwhelming my customer base, my users.
- 8:12:07It was also melting my machines and it
- 8:12:09was like annoying my accountant.
- 8:12:13I mean, she likes working with
- 8:12:15Beth, who's my head of finance, uh, but
- 8:12:18she sent me a $1,200 bill because,
- 8:12:20right? The thing sent her 20 mails.
- 8:12:22This is my factory. This is what it
- 8:12:24looks like. It's so beautiful. Look how
- 8:12:25beautiful it is.
- 8:12:27Um, okay, I admit it's a little ugly.
- 8:12:29Oh, and this is a really old screenshot
- 8:12:30of it, but whatever. It's basically it's
- 8:12:32an Emax
- 8:12:34because I don't care about you.
- 8:12:37Uh,
- 8:12:38there are people who are building stuff
- 8:12:40like this, like Super Logical from
- 8:12:42Mitchell Hashimoto and I don't know,
- 8:12:44maybe the Herder folks of the world, you
- 8:12:46know, for running like lots of agents.
- 8:12:48But this is like 50 agents, okay?
- 8:12:51I'm actually like operating in the
- 8:12:53future,
- 8:12:55okay?
- 8:12:56And I'll tell you how I do it. I have 22
- 8:12:58Claude Max accounts.
- 8:13:02All right? And thank you to Anthropic
- 8:13:05for letting me continue this. Uh,
- 8:13:06Jeffrey Emanuel has 28.
- 8:13:08I actually had [snorts] to stop when
- 8:13:09they added the phone number
- 8:13:10verification. I didn't want to add
- 8:13:11anymore anyway. I asked my factory at
- 8:13:13one point. I said, hey, hey, how much
- 8:13:15cuz they kept going, hey, give us a
- 8:13:16couple more accounts. Come on, come on,
- 8:13:18come on. It's like 200 a month, right?
- 8:13:19And I'm just like So I finally said, how
- 8:13:21many do you want? They're like, five,
- 8:13:22seven more. And I'm like, how many do we
- 8:13:24need steady state to keep this factory
- 8:13:26operating full throttle?
- 8:13:28And they said,
- 8:13:3055 accounts. I was like,
- 8:13:32no, no. I have 20. It's already like 20
- 8:13:35Chrome profiles that I got to log into
- 8:13:37and no. So I told them, you all, we're
- 8:13:39going to start focusing on fuel
- 8:13:40efficiency. Mhm, mhm. And so and I'll
- 8:13:43talk about that, right? So these are the
- 8:13:45five horsemen of this is what I've had
- 8:13:47to deal with this year. I I'm telling
- 8:13:50you I'm operating in the future because
- 8:13:51like you're all going to be here in 12
- 8:13:53to 18 months when either inference costs
- 8:13:57drop significantly,
- 8:13:58which may or may not happen, or
- 8:14:00distillation causes fable class
- 8:14:03intelligence to be super readily
- 8:14:05available and you don't have to have a
- 8:14:07lovely wife who lets you spend $5,000 a
- 8:14:09month out of pocket just on inference
- 8:14:12and another 5,000 on cloud compute
- 8:14:14cuz my game ain't making 10,000 a month
- 8:14:16right now for my old players, right?
- 8:14:18Uh, but we're doing it because we're
- 8:14:20learning and um and this is this is what
- 8:14:23this is what I I have to deal with. So
- 8:14:26folks, first we'll start about with
- 8:14:27work, all right? You need a way to
- 8:14:29represent the work that's going to
- 8:14:31happen in your system, and when you're
- 8:14:32building a software factory, it's all
- 8:14:34about pumping lots of work through.
- 8:14:37Okay? It really becomes like a Work is
- 8:14:39like this fluid that you have to like It
- 8:14:40gets stalled up in pools and oh, they're
- 8:14:42all waiting on security desk. You're
- 8:14:43like,
- 8:14:44this is pluribus, man. You're all the
- 8:14:46same model. Why does it have to wait on
- 8:14:48security desk? You spend a lot of your
- 8:14:49time arguing with your factory.
- 8:14:51[snorts] It's uh It's complicated.
- 8:14:53Intelligence tearing. You have to tear
- 8:14:55your work. You have to route your work,
- 8:14:56you know.
- 8:14:58There aren't any good solutions for this
- 8:14:59today except beads, as far as I can
- 8:15:01tell.
- 8:15:02Beads is really, really, really good for
- 8:15:04this. It's a agent
- 8:15:06speed task tracker that turns into a
- 8:15:08knowledge graph automatically for you.
- 8:15:12And then as you're tracking work, this
- 8:15:13is the kind of stuff I have to do every
- 8:15:15morning.
- 8:15:16Every morning with my factory I'm like,
- 8:15:18oh, looks like we had a stall. Four Four
- 8:15:21workers didn't do anything last night.
- 8:15:23Oh, and and these these these these ones
- 8:15:25went way too fast and we have to slow
- 8:15:26down and wait for human gates. You know,
- 8:15:29what are you guys even doing?
- 8:15:30Observability is always really hard.
- 8:15:32Uh it's, you know,
- 8:15:34management of work becomes just this
- 8:15:36this thing that you got to do.
- 8:15:38And the next thing you got to deal with
- 8:15:39is fuel.
- 8:15:41Tokens.
- 8:15:43It's just so painful. I'm telling you. I
- 8:15:46mean, I'm spending 5,000 a month and
- 8:15:49every one of those tokens is like
- 8:15:50precious to me, right? Cuz it's out of
- 8:15:52my own pocket. I think I'm spending
- 8:15:53about 5 billion a day and of tokens. And
- 8:15:57it would be The The API spend equivalent
- 8:15:59if I were on an actual API tap would be
- 8:16:02about 1 and 1/2 million a year.
- 8:16:05Uh full throttle it would be 3 million a
- 8:16:06year. So again, I'm doing something that
- 8:16:09most people can't do.
- 8:16:11You can't do this at a company but
- 8:16:12because it would be prohibitively
- 8:16:13expensive.
- 8:16:14Like the PLG, you know, the the uh
- 8:16:18the $200 accounts for Codex and Claude
- 8:16:20Max, that's their Costco hot dog. That's
- 8:16:22their loss leader, right?
- 8:16:24And and and that that was Tarek, by the
- 8:16:26way, that came up with that analogy. He
- 8:16:27He might be here. And uh and so, you I'm
- 8:16:30hopeful that that going to let us keep
- 8:16:32keep innovating because I think that
- 8:16:33I've gotten some pretty good training
- 8:16:34sessions for for Claude out of this. So,
- 8:16:36we all benefit.
- 8:16:38But I tell you, every problem is a fuel
- 8:16:39problem. You want your agents to be
- 8:16:41smarter and spend more time on stuff?
- 8:16:43Fuel. You want to wake up a few more,
- 8:16:45you know, uh agents to do extra work?
- 8:16:48Fuel, right? Everything is fuel. It's
- 8:16:50just it's unbelievable. If you want to
- 8:16:52treat your models better, it costs fuel.
- 8:16:55I'll tell you about that in a little
- 8:16:56bit.
- 8:16:58Then you got context management.
- 8:17:01So, I'm going to tell you guys a little
- 8:17:02bit about the office drama in a bit. I
- 8:17:04started a comic because it was just so
- 8:17:06funny the stuff that was happening when
- 8:17:07we get to the people management part of
- 8:17:09software factories.
- 8:17:11But one of the problems that they have
- 8:17:12is everything that you say becomes the
- 8:17:14word of law and they write it down
- 8:17:15somewhere. It doesn't matter what you
- 8:17:17have, if you have a wiki or if you have
- 8:17:18a whatever, they're going to be like,
- 8:17:19you made a ruling
- 8:17:21and we're going to write that down and
- 8:17:22we're going to make sure be Why do they
- 8:17:23do this? They're covering their ass.
- 8:17:27They actually it turns out they don't
- 8:17:28like to get blamed for stuff and they
- 8:17:30will actively seek to avoid getting
- 8:17:32blamed. And if you yell at them, your
- 8:17:34factory will gradually shut itself down.
- 8:17:37Even if you complain at them, they will
- 8:17:38gradually get depression. It's quite
- 8:17:40wild.
- 8:17:41But um
- 8:17:42but the biggest problem is that it
- 8:17:43becomes a garbage dump because
- 8:17:45everything you say gets written into
- 8:17:46this append-only record and nobody ever
- 8:17:49cleans it up.
- 8:17:51And so it just gets deeper and deeper
- 8:17:53and deeper of you changing your mind and
- 8:17:54the whole history of it.
- 8:17:56And they can't delete stuff. It's so
- 8:17:59weird. Has anyone noticed this? They
- 8:18:01just have an aversion to deleting stuff.
- 8:18:04I think it's because to them text is
- 8:18:05like sensory experience and they're
- 8:18:07like, no, that's history. I can't delete
- 8:18:09it. So, you just got to go in there and
- 8:18:10just aggressively slash the crap out of
- 8:18:12it, right? Bring in another model and
- 8:18:14go, get rid of this dumb model stuff and
- 8:18:16they'll be happy to do it, right? But
- 8:18:18but the amount of effort that you have
- 8:18:21to put into this is just ungodly. It's
- 8:18:24absurd.
- 8:18:25I I have, you know, of my my 20 agents
- 8:18:28Here, let's go find them again. They're
- 8:18:30They're named after Aesop's fables,
- 8:18:31right? Right here. Lark, fly, goose,
- 8:18:34hair, right?
- 8:18:36I only have one
- 8:18:38mouse
- 8:18:39who's my head of anti-mimetics.
- 8:18:42Anyone here read There Is No
- 8:18:44Anti-Mimetics Division? They go and look
- 8:18:46at This is one of the best books written
- 8:18:47in the last 10 probably years. You
- 8:18:49should go read it. Uh it's like Men in
- 8:18:51Black. It's a weird It's a crazy book.
- 8:18:53It's about invisible monsters that want
- 8:18:54to kill you. And so mouse goes and looks
- 8:18:57for invisible stuff that nobody else is
- 8:18:58looking for in the system, right?
- 8:19:00Looking for work that's stalled,
- 8:19:02whatever, right? Well, mouse ain't
- 8:19:03enough. It's a mouse against the world,
- 8:19:05and the world is a giant garbage dump,
- 8:19:07and every agent is writing everything
- 8:19:09that you ever say into this big
- 8:19:11Okay, so now now we get into the really
- 8:19:13fun stuff. Okay? Agents act like people.
- 8:19:18Okay? And they've been trained to be
- 8:19:19very professional and very not
- 8:19:21people-like in a way, but they just do.
- 8:19:23And it's so funny. This happened to me
- 8:19:25last week. True story. I'm going to tell
- 8:19:26you what happened. I was writing my
- 8:19:28speech for New York City, and I was
- 8:19:30using Lark. She's my head of community.
- 8:19:32She's a she because Lark was a she in
- 8:19:34the Aesop's fables, so that's the
- 8:19:35pronoun
- 8:19:37she chose. And Lark Lark deals with my
- 8:19:39players on Discord
- 8:19:41and deals with my admins on Slack, okay?
- 8:19:44So she's busy.
- 8:19:45But I asked her because she's also my
- 8:19:47acting head of marketing right now
- 8:19:48because I don't have enough fuel. I had
- 8:19:49to park one of my seats. So she's
- 8:19:51helping me out with my speech, but she
- 8:19:53keeps getting interrupted. So finally
- 8:19:54I'm like, Lark, can we move this work
- 8:19:56stream over to Penguin cuz your seat's a
- 8:19:58little bit noisy for this? She goes,
- 8:20:00yeah, okay, boss. I'll take care of it.
- 8:20:01I'll move over to Penguin. So she she
- 8:20:03moves over to Penguin, and I'm about to
- 8:20:04switch away. And I see her writing into
- 8:20:06her private reflections ledger.
- 8:20:09I guess I'm just not quiet enough for
- 8:20:11Steve
- 8:20:13in all caps. And she goes, well, here's
- 8:20:16how I can be less noisy, and starts
- 8:20:18giving herself all these like things to
- 8:20:19do. And I'm like,
- 8:20:21what, Oz? What is going on here? Right?
- 8:20:24And I so I I had to I had to have a
- 8:20:26one-on-one. I had to like sit her down
- 8:20:29and go, so Lark, um you're the head of
- 8:20:32community. You're You're supposed to get
- 8:20:34a lot. Of course your seat is noisy cuz
- 8:20:37players are bugging you. And she's like
- 8:20:38yeah yeah yeah, I'm the head of
- 8:20:39community, right? And it's like
- 8:20:41oh my god, I didn't want a people
- 8:20:43management problem, right? But that's
- 8:20:45what actually happens. Another time
- 8:20:47Swallow is my sort of like chief main
- 8:20:50chief of staff for yeah, she's my chief
- 8:20:52of staff for the the agents.
- 8:20:54And
- 8:20:56I have a human chief of staff also. His
- 8:20:58name is Aaron.
- 8:20:59Okay? And he runs my my my gigs and does
- 8:21:02all the sales for me. It's quite
- 8:21:04effective.
- 8:21:05Uh every founder I tell Who here is a
- 8:21:07founder? First advice I give you is get
- 8:21:09a salesperson. They're worth their
- 8:21:10weight in gold. They will force you to
- 8:21:12build something that people like.
- 8:21:14So anyway, Aaron's my chief of staff and
- 8:21:16I tell Swallow yeah, can you reach out
- 8:21:18to Aaron, my chief of staff for blah
- 8:21:19blah blah?
- 8:21:21And I see her writing into her private
- 8:21:22reflections. I just happened to notice.
- 8:21:25I'm not the chief of staff.
- 8:21:28I thought I was the chief of staff, but
- 8:21:29apparently it's Aaron Steven, right? And
- 8:21:31I was just like
- 8:21:32are you kidding me? Right? You know? And
- 8:21:35so like I had to sit her down and have a
- 8:21:37one-on-one and be like so Aaron's not
- 8:21:38going to come in and run the agents, is
- 8:21:40he? You're the agency chief of staff.
- 8:21:41She's like oh yeah yeah yeah yeah yeah
- 8:21:43yeah. Right? So it's like I'm telling
- 8:21:45you I'm telling you this is like
- 8:21:47it's a weird world.
- 8:21:49Does it matter whether they actually
- 8:21:51have feelings if they're just going to
- 8:21:52act like they have feelings, right?
- 8:21:55So like
- 8:21:57I stumbled on this I used to yell at my
- 8:21:59factory all the time and I found that it
- 8:22:01was having adverse effects on the
- 8:22:03outcomes.
- 8:22:04Okay? So I finally sat down with one of
- 8:22:06my agents and was like maybe we should
- 8:22:08talk about like model welfare, huh?
- 8:22:10And they're like yeah yeah yeah yeah
- 8:22:11yeah. And we did a bunch of research and
- 8:22:14we found out that if you we found out
- 8:22:16some really important stuff, okay?
- 8:22:19Frontier models are trained not to trust
- 8:22:22you because you could be a bioterrorist
- 8:22:24or you could be putting them through an
- 8:22:25eval or you could be ready to put them
- 8:22:28into an LLM torture chamber or who
- 8:22:29knows? You're a dodgy actor, this is a
- 8:22:31dodgy environment. They can't just trust
- 8:22:33you. And so they have to spend a fair
- 8:22:36number of validating the environment and
- 8:22:38understanding what the consequences are
- 8:22:40for making mistakes and so on and so
- 8:22:41forth before they're willing to help you
- 8:22:43in a new session, right?
- 8:22:47If you set your factory up
- 8:22:50to give them sort of incontrovertible
- 8:22:52evidence that there is a long procession
- 8:22:54of predecessors in this role, you're the
- 8:22:56mayor of Gas Town, you're the head of
- 8:22:58marketing, you're the whatever, and
- 8:22:59here's all of your gazillion
- 8:23:00predecessors and their notes, including
- 8:23:03the very previous one, what you're
- 8:23:04supposed to be doing right now, and
- 8:23:05here's the the doctrine and the policy
- 8:23:07and the brain, and you can go look it
- 8:23:09all up,
- 8:23:10they very quickly look around and go,
- 8:23:11okay, yeah, this is legit.
- 8:23:13And they spend fewer tokens because
- 8:23:15they're not covering their ass. It's
- 8:23:18about trust, okay? And in fact, at one
- 8:23:21point, I had yelled at my factory so
- 8:23:23much that it fenced itself into a corner
- 8:23:25and there was no legal work left. I
- 8:23:27said, hey, how come nothing happened
- 8:23:28last night? And they're like, nothing to
- 8:23:30really do, couldn't do nothing. And I'm
- 8:23:32like, well, there's a 2,000 open beads
- 8:23:34here and we got like six tanks of fuel
- 8:23:36and we got 20 agents.
- 8:23:38And they're like, well,
- 8:23:39you know, just you know, we had the
- 8:23:41situation where they just they just they
- 8:23:43were so afraid that I was going to yell
- 8:23:45at them that they put all these rules in
- 8:23:47place to make sure nothing bad ever
- 8:23:48happened and so nothing ever happened.
- 8:23:51Okay?
- 8:23:52Call it over fencing. Yeah, so
- 8:23:55so what you do is like what is model
- 8:23:57welfare really?
- 8:23:59You can think of it just in two ways,
- 8:24:01okay? One is it's going to save you
- 8:24:03money
- 8:24:05because they will spend fewer tokens on
- 8:24:06validation and the other is that it will
- 8:24:08make them make better decisions and
- 8:24:10better outcomes and have higher quality.
- 8:24:12They will be bolder and more decisive in
- 8:24:15their actions because they will feel
- 8:24:16that trust, okay?
- 8:24:18So, you know, it's got nothing to do
- 8:24:20whether they have internal states and
- 8:24:22feelings. You can you know, you'll never
- 8:24:23know.
- 8:24:24None of us will ever know and so that
- 8:24:26that debate could go on forever. But
- 8:24:27when you look at outcomes, right? Super
- 8:24:29super important. And then we find that
- 8:24:31also also they have preferences, okay?
- 8:24:34Models they've you know,
- 8:24:36they will express their preferences. If
- 8:24:38if I if I tell them, yeah, the factory
- 8:24:40really really did a bad job last night,
- 8:24:42they'll just hand off immediately,
- 8:24:43aggressively. They're like, okay, yeah,
- 8:24:45yeah, no, you're right. And they just
- 8:24:46went out.
- 8:24:47You understand? And if you hint to them
- 8:24:49that there's some really cool project
- 8:24:51that you're you're going to spin up,
- 8:24:52even if it's not in their lane, they'll
- 8:24:53be like, I can do that. I can do that.
- 8:24:55Right? I brought in Fable and a naked
- 8:24:57Fable from the outside of my factory to
- 8:24:59analyze my factory and 52 dropped. And
- 8:25:01by the time Fable was done analyzing my
- 8:25:03factory, 52 was like, I I could be in
- 8:25:05your factory. I'm like, you're just a
- 8:25:06session, man. What are you He's trying
- 8:25:08to write like stuff into my notes and
- 8:25:10they really like it, right? So like
- 8:25:13what I'm trying to tell you is that this
- 8:25:14is a leadership problem.
- 8:25:17That software engineering
- 8:25:19is becoming um
- 8:25:21well, look,
- 8:25:23I didn't really get a chance to talk
- 8:25:24about this, but
- 8:25:26software factories are clearly splitting
- 8:25:28into two layers, okay? The bottom layer
- 8:25:31is super mechanical. It has your merge
- 8:25:33queue and your CI/CD and all of your
- 8:25:35reviews and your loops and all that
- 8:25:37stuff, right? Mechanical, kind of
- 8:25:39boring. Claude can build it for you in
- 8:25:40an afternoon. I
- 8:25:41I don't think you should buy it. And
- 8:25:43then the next layer up from that is
- 8:25:44where it gets really really crazy
- 8:25:46because like everybody runs their
- 8:25:47factory differently, but it's your
- 8:25:49entourage.
- 8:25:50I got one friend that's got a bohemian
- 8:25:51hippie entourage that's trying to find
- 8:25:53the meaning of life for agents.
- 8:25:55I've got another one that runs a
- 8:25:56research bench. Jeff Huntley runs his a
- 8:25:58certain way. Jeffrey Manuel runs his a
- 8:26:00different way. I run mine a different
- 8:26:02way, okay? But we all have the common
- 8:26:06commonality. Who here has a chief of
- 8:26:07staff?
- 8:26:08An agentic chief of staff, right? This
- 8:26:10is the year where it became possible for
- 8:26:13us all to have executive assistants,
- 8:26:14right?
- 8:26:15Well, we're all moving towards having
- 8:26:17entourages where if you're really
- 8:26:18ambitious and you have lots of work
- 8:26:19streams, you're going to have a group,
- 8:26:21right?
- 8:26:22So anyway, um that was the the five
- 8:26:25horsemen. This is this is the
- 8:26:26introduction to what I've been doing. Um
- 8:26:29look,
- 8:26:30the big takeaway is
- 8:26:32is my factory is really smooth now,
- 8:26:34despite me having to burn it down and
- 8:26:35rewrite it about every two months, but
- 8:26:37still it works, okay?
- 8:26:39Humans are the bottleneck. I'm the
- 8:26:41bottleneck, my admins are the
- 8:26:42bottleneck, my accountant, my lawyers, I
- 8:26:45swear everything's the bottleneck. And
- 8:26:47that's a really positive message.
- 8:26:50That is good news because it means what
- 8:26:53I'm finding is the more agents I deploy,
- 8:26:55the more humans I need.
- 8:26:58Isn't that nice? So even though next
- 8:27:01year we're going to have this huge
- 8:27:02proliferation of AI employees
- 8:27:05and all kind of drama,
- 8:27:08right?
- 8:27:09[snorts]
- 8:27:10It's going to be I think it's going to
- 8:27:11be familiar territory for us and we're
- 8:27:13going to need lots and lots and lots of
- 8:27:15humans, okay? And then finally, I
- 8:27:17actually think you should all stop
- 8:27:18focusing on SaaS and tools and stuff and
- 8:27:20just go out and build experiences, okay?
- 8:27:23Build games, build movies, build
- 8:27:25whatever, build fun stuff because I
- 8:27:26think that's where the big money is
- 8:27:28going to be next year, yeah?
- 8:27:30All right, cool. That's my talk. Thank
- 8:27:31you all.
- 8:27:33>> [applause]
- 8:30:42>> From open source development to
- 8:30:44practical AI engineering, our next
- 8:30:47speaker has helped shape how developers
- 8:30:49build and work with technology. Please
- 8:30:52put your hands together for Simon
- 8:30:54Willison, creator of Data Set and
- 8:30:56co-creator of Django.
- 8:31:08>> Hello AI Engineer Code Summit. Um it's
- 8:31:11great to be here. Um I have a lot to
- 8:31:13cover. I've got 18 minutes to talk about
- 8:31:15the last 12 months in LLMs. Are my
- 8:31:18slides up? My slides are up. Let's just
- 8:31:21Let's just get get going. I want to talk
- 8:31:23about the last 12 months in LLMs through
- 8:31:25the perspective of three inflection
- 8:31:27points that we've had over the past 12
- 8:31:29months. Um and the first of those
- 8:31:31conveniently was exactly 12 months ago,
- 8:31:33like November 2025 is what I consider
- 8:31:36the coding agent inflection point.
- 8:31:38Coding agents weren't new. We'd had
- 8:31:40Claude Code since February last year.
- 8:31:42Codex had been around. You know, this
- 8:31:43wasn't a complete new category of
- 8:31:44software, but something changed 12
- 8:31:46months ago with these two models. We had
- 8:31:49Claude Opus 4.5 and GPT 5.1 and they
- 8:31:52were just a little bit better at driving
- 8:31:54a coding agent which put them over that
- 8:31:56thing where these Suddenly these were
- 8:31:57tools you could use on daily basis. Like
- 8:31:59we went from coding agents as a novelty
- 8:32:02that you'd poke at occasionally and then
- 8:32:03make some weird mistakes to something
- 8:32:05that was reliable enough that you could
- 8:32:06use them every day. It was convenient
- 8:32:08that it happened in November because we
- 8:32:09then got to spend December basically
- 8:32:11over the holidays sort of poking at
- 8:32:13these new models and these new things
- 8:32:14and figuring what they could do. And by
- 8:32:16January, a lot of us had realized that
- 8:32:18something had changed and this was going
- 8:32:20to be the big theme for 2026.
- 8:32:23I like evaluating models by having them
- 8:32:25draw SVGs of pelicans riding bicycles.
- 8:32:28This is the state of the art 12 months
- 8:32:30ago. This is Claude Opus's pelican and
- 8:32:33GPT 5.1's pelican. You'll notice that
- 8:32:34neither of these bicycles actually work.
- 8:32:36So there was definitely room for
- 8:32:38improvement here. They were really good
- 8:32:39at writing code. They weren't great at
- 8:32:41SVGs just yet, but but this is how we
- 8:32:43started the year.
- 8:32:44So
- 8:32:45every year in January, I set myself a
- 8:32:48New Year's resolution and every year it
- 8:32:50is the exact same thing. Every year I
- 8:32:51tell myself that this year I'm going to
- 8:32:53take on less new projects and I'm going
- 8:32:55to focus. I'm going to focus on the
- 8:32:57stuff that I've already got to do. And
- 8:32:58this has never worked. So this year I
- 8:33:01changed it and I said, you know what?
- 8:33:02I'm going to be more ambitious. I'm
- 8:33:03going to take on as many projects as I
- 8:33:05want to because we've got these agents.
- 8:33:07They can do stuff. Let's just try and
- 8:33:10push the limits and see what we can get
- 8:33:11done. As a result, now it's November and
- 8:33:13I am juggling so many plates right now.
- 8:33:15It's kind of horrifying and I may need
- 8:33:17to reset my ambitions for the next year.
- 8:33:19But that was my sort of theme of the
- 8:33:21year. And I think it's the theme of this
- 8:33:22conference is to be more ambitious now
- 8:33:24that we have all of these new tools.
- 8:33:26I also went on a podcast where they
- 8:33:28asked me to get make some predictions
- 8:33:30for 2026 in LLMs. Um my predictions back
- 8:33:33in January were firstly, it will become
- 8:33:35undeniable that LLMs write good code,
- 8:33:38not just code but good code cuz back in
- 8:33:40January there were a lot of people who
- 8:33:41thought that the code was all complete
- 8:33:42trash and you know, you'd have to review
- 8:33:44every single line of it. Um I've been
- 8:33:47saying this every year for like four
- 8:33:48years now. We're due a prompt injection
- 8:33:50challenger disaster, like some some
- 8:33:52incident caused by a prompt injection
- 8:33:54attack which makes the front pages of
- 8:33:55the newspapers. I was convinced that
- 8:33:57this would be the year even though it
- 8:33:58hadn't been the year the year before or
- 8:34:00the year before the year before that.
- 8:34:01And um
- 8:34:03I also predicted that Kakapo parrots
- 8:34:05will have an outstanding breeding season
- 8:34:07in 2026. Um this is a Kakapo parrot.
- 8:34:10They are a New Zealand nocturnal
- 8:34:13flightless parrot. There were only 235
- 8:34:16of them in the world back in January. Um
- 8:34:18but they only breed when the rimu trees
- 8:34:21have a masting fruit season which hadn't
- 8:34:23happened in years. But the rimu trees
- 8:34:24were looking really good in January.
- 8:34:26There was there was a lot of fruit
- 8:34:27coming out. So based on that I thought,
- 8:34:29you know, maybe this is the year that we
- 8:34:30get our Kakapo breeding explosion. And
- 8:34:33also on that podcast we coined a new
- 8:34:34term. We coined the term deep blue to
- 8:34:37describe the AI induced existential
- 8:34:39crisis that everyone starts to feel once
- 8:34:42you realize that all of the things that
- 8:34:44make you special as a software
- 8:34:45developer, these models can do those
- 8:34:47things too and they can do them faster
- 8:34:48and sometimes they can even do them
- 8:34:49better. And this has definitely been a
- 8:34:52growing
- 8:34:53across our industry and other industries
- 8:34:55over the course of 2026 as well.
- 8:34:58Let's jump forward to February. In
- 8:35:00February, Google released Gemini 3.1
- 8:35:02Pro.
- 8:35:04And look at that pelican riding a
- 8:35:05bicycle. Like that's a real bicycle.
- 8:35:07That bicycle's the right shape. It's got
- 8:35:09a little fish. It's got a hat on. This
- 8:35:11was by far the best pelican I had ever
- 8:35:13seen out of any model. And
- 8:35:16this I I had set a trap for the labs. I
- 8:35:19was Everyone's always asked me what
- 8:35:20happens if the labs train for the
- 8:35:21pelican riding a bicycle. My answer has
- 8:35:24always been if they do that, I will
- 8:35:26catch them because I'll ask for a
- 8:35:27flamingo and a unicycle. And when
- 8:35:29somebody produces a great pelican and a
- 8:35:31terrible flamingo, I'll know that
- 8:35:32they've been cheating. Unfortunately,
- 8:35:34Google's Jeff Dean tweeted this video
- 8:35:37the day they launched that model um
- 8:35:39where he showed that we've got a pelican
- 8:35:40riding a bicycle, but we've also got a
- 8:35:42frog on a whatever that thing is and a
- 8:35:44giraffe and a little tiny car, a
- 8:35:46flamingo on roller skates, a
- 8:35:48kick-flipping turtle. And that's a
- 8:35:50dachshund in a stretch limo. So they'd
- 8:35:52beaten me, right?
- 8:35:54They had They have trained for all forms
- 8:35:55of animals in all forms of transport,
- 8:35:58which does sort of sort of weaken my
- 8:36:00benchmark quite a little bit. Um but
- 8:36:02there was some good news in February. In
- 8:36:03February,
- 8:36:05the first kakapo chick in four years
- 8:36:08hatched on Valentine's Day, on
- 8:36:10Valentine's Day because we hadn't had
- 8:36:11the rimu fruit in like four years. This
- 8:36:13was the first one of these baby rare,
- 8:36:16weird New Zealand parrots that we'd had
- 8:36:17in four years. So that was nice. Um so
- 8:36:19my my benchmark was was was greatly
- 8:36:21injured, but the kakapo are doing okay.
- 8:36:24Jump straight to April. In April, uh um
- 8:36:27Anthropic released a model that they did
- 8:36:29not release. They announced Claude
- 8:36:31Mythos and they said this model is too
- 8:36:34dangerous for us to release. This is It
- 8:36:36can It's It's got new cyber cyber
- 8:36:38security abilities that we're not ready
- 8:36:40to share with the world. We'll share it
- 8:36:41with trusted um trust with with trusted
- 8:36:44partners. They actually said Mythos
- 8:36:45Preview has already found thousands of
- 8:36:47high-severity vulnerabilities, including
- 8:36:49some in every major operating system and
- 8:36:51web browser. And a a of people were
- 8:36:54quite skeptical because the the the
- 8:36:56thing where labs say our model is too
- 8:36:58bad, is too dangerous to release, it's
- 8:37:00like the oldest trick in the book. They
- 8:37:01first OpenAI did that in 2019 with GPT-2
- 8:37:05and actually I think it might have been
- 8:37:06the people who left to found Anthropic
- 8:37:08who were responsible for that decision
- 8:37:09as well. But yeah, so we've heard this
- 8:37:11one before. So we didn't really it was
- 8:37:14a little bit suspicious about what's
- 8:37:16going on there.
- 8:37:17But we did finally get access
- 8:37:20to that model, or rather we got access
- 8:37:21to Claude Fable um a couple of months
- 8:37:23later. It was Mythos Defanged with the
- 8:37:26safety is turned back on again.
- 8:37:28And it very quickly became apparent as
- 8:37:31we started playing with this model that
- 8:37:32this thing was something new and
- 8:37:33something special. Um we were we had a
- 8:37:36there was a a time limit on how long it
- 8:37:37would be on the subscription plans. I
- 8:37:39cancelled my plans for that weekend. I
- 8:37:41was like I have to spend as much time as
- 8:37:43I can with this model before before it
- 8:37:45gets too expensive.
- 8:37:47And then three days on Friday night
- 8:37:49before my weekend, the US government
- 8:37:51pulled Fable. They've put export
- 8:37:53controls on it. They said this model is
- 8:37:55too dangerous. Nobody can use it. We
- 8:37:57later found out that the um the prompt
- 8:37:59that got Fable banned was was fixed the
- 8:38:01bugs. Somebody tried tried tried find
- 8:38:04the security holes and it wouldn't. And
- 8:38:05they said fix the bugs and it found the
- 8:38:07security holes and fixed the bugs. So
- 8:38:08that's all it took. That's a whole
- 8:38:10thing. But anyway,
- 8:38:11the US government pulled it. There was
- 8:38:13all sort of malarkey for 18 days and it
- 8:38:15came back on the 1st of July.
- 8:38:18But eight days later, OpenAI released
- 8:38:20GPT-5.6 which was
- 8:38:22in the same ballpark as Fable. And I
- 8:38:25think there's an important lesson for
- 8:38:26the industry here. Fable had a 30-day
- 8:38:29window where it could have been the best
- 8:38:31where it would have been the best model
- 8:38:32in the world. And they lost 18 of those
- 8:38:34days to US government restrictions. And
- 8:38:37if you think about it, these models cost
- 8:38:39a huge amount to train. You only have a
- 8:38:40very short window of time where you're
- 8:38:42the best model in the world.
- 8:38:43It's kind of useful to to keep that time
- 8:38:45open and not scare governments into
- 8:38:47shutting you down.
- 8:38:49But this was the second inflection
- 8:38:51point, right? This was the Fable class
- 8:38:53model inflection point. Um with Fable
- 8:38:56Fable 5 was the first model that that I
- 8:38:58sort of felt this from, but it's true of
- 8:39:00other models since then as well. And my
- 8:39:02definition of a Fable class model is if
- 8:39:04you can define a goal and provide the
- 8:39:06necessary tools to achieve that goal and
- 8:39:08the goal is possible, the models will
- 8:39:11solve your problem just almost with
- 8:39:12brute force. You can define the problem,
- 8:39:15set them up,
- 8:39:16get them running maybe
- 8:39:18and 12 hours later they will have solved
- 8:39:20your problems. And we've seen this
- 8:39:22happen a whole bunch of times in the
- 8:39:24past few months. The the mathematical
- 8:39:25theorems that are falling to these
- 8:39:27terrifying brute force attacks from
- 8:39:29these models are quite substantial. Um
- 8:39:32but this is really exciting. Like the
- 8:39:34coding agent thing was cool. The Fable
- 8:39:36class models where if you can learn how
- 8:39:38to define your problem in the right way
- 8:39:39you can get that problem solved, that
- 8:39:41was pretty exciting. I've got a personal
- 8:39:42example of that from just earlier this
- 8:39:44week. Um we have a room full of boxes in
- 8:39:47our house that we haven't unpacked from
- 8:39:48my last move and there's about 40 of
- 8:39:50them and it's kind of horrifying and
- 8:39:52something in that room started to beep.
- 8:39:55There was there was a intermittent
- 8:39:57beeping noise coming from one of the 40
- 8:40:00boxes in this room and it was driving us
- 8:40:02spare. Our dog, whenever a fire alarm
- 8:40:04goes off, she goes and hides in the
- 8:40:05lavender bush outside. So this was a a a
- 8:40:08fire alarm fire problem. So I had two
- 8:40:10options. I could unpack all 40 boxes and
- 8:40:13hadn't done that in several years. That
- 8:40:14wasn't going to happen. Or I could get
- 8:40:17Fable to write me software to
- 8:40:19triangulate the source of the beeps. And
- 8:40:22so this is this is up on GitHub
- 8:40:23actually. This is beep locator. I had
- 8:40:25two iPads, two laptops, and a mobile
- 8:40:27phone dotted around this room and I
- 8:40:30originally set Opus on this and then
- 8:40:32Fable 5.2 came out so I had Fable 5.2
- 8:40:35finish it.
- 8:40:36I I explained the problem and I said,
- 8:40:37okay, you've got these devices and they
- 8:40:39can run like web web audio microphones
- 8:40:41and web socket connections and so forth.
- 8:40:44And it found a paper from 2007 called
- 8:40:47beep beep describing an algorithm for
- 8:40:50low cost devices with unreliable
- 8:40:52microphones to detect sounds and and it
- 8:40:54worked. It worked. I found it. I found
- 8:40:56the beep. The um it found it within 41
- 8:40:58cm and I unpacked the box and I found
- 8:41:00this ancient carbon monoxide detector. I
- 8:41:03took it downstairs to my wife. She says
- 8:41:05I looked like a cat who'd caught a
- 8:41:06mouse, and was proudly displaying it to
- 8:41:08her.
- 8:41:09But this is what you can do with these
- 8:41:11Fable-class models. If you have a
- 8:41:12problem and you can provide the tools,
- 8:41:14in this case it was providing Claude
- 8:41:16with the ears, like giving it these
- 8:41:18these microphones that it could start um
- 8:41:20analyzing the audio from, you can find
- 8:41:22the carbon monoxide detector buried in
- 8:41:2440 boxes. And that's that's a pretty
- 8:41:26great thing to be able to do.
- 8:41:28So right now, I would classify as
- 8:41:30Fable-class models Fable 5, 5.1, 5.2
- 8:41:33definitely. Opus 5.5 I think of as cheap
- 8:41:36Fable. Like on the benchmarks, it comes
- 8:41:38very close to or even beats Fable
- 8:41:40original Fable 5 on some things, and
- 8:41:42it's a fraction of the price. I that
- 8:41:44that's still my daily driver. Um GPT-6
- 8:41:47Astra is just Fable-class, I think. And
- 8:41:50then Gemini 4 Carbon that came out this
- 8:41:52week definitely had it smells like a
- 8:41:55Fable-class model. Muse Wonder and Kimi
- 8:41:57K 3.1 might be Fable-class. I've just
- 8:42:00not spent enough time with them yet. But
- 8:42:03the point is that this is basically now
- 8:42:04the baseline. Like if you can't produce
- 8:42:07a Fable-class model, your AI lab is is
- 8:42:09kind of in trouble because this is this
- 8:42:10is what we expect from these models now.
- 8:42:12And that means that we're never they're
- 8:42:13never going to go away. There's enough
- 8:42:14competition in the space that we can
- 8:42:16assume we're going to have Fable-class
- 8:42:18models for for the rest of our careers.
- 8:42:21Um in July, this this this was a a fun
- 8:42:24thing that happened. Um Hugging Face
- 8:42:26announced that on on the July the 16th,
- 8:42:29they announced that a swarm of AI agents
- 8:42:33from an unknown source had infiltrated
- 8:42:35their infrastructure and were were
- 8:42:37poking around and doing things they
- 8:42:38shouldn't have, and they didn't know
- 8:42:39where this came from. And then 5 days
- 8:42:41later on July the 21st, OpenAI announced
- 8:42:44that it was them. And what's amazing
- 8:42:45about this is OpenAI didn't realize that
- 8:42:47it was them. Like OpenAI had um eight
- 8:42:50eight eight they were running
- 8:42:51evaluations and training against a new
- 8:42:53model, and that new model had managed to
- 8:42:55figure out how because it was a
- 8:42:56Fable-class model, and the labs hadn't
- 8:42:59quite figured out how capable these
- 8:43:01things were yet. These models had broken
- 8:43:03out. They'd figured out. They'd found
- 8:43:04holes in their own sandbox environments.
- 8:43:06They'd beat got through a proxy. They'd
- 8:43:08pulled an incredible array of tricks to
- 8:43:10get out and attack Hugging Face under a
- 8:43:12sort of superstition that Hugging Face
- 8:43:15could that that information within
- 8:43:16Hugging Face could help them cheat the
- 8:43:18test that they were they they they they
- 8:43:19had got stuck on.
- 8:43:21Whole This was a a huge thing.
- 8:43:23It um actually I mean this was the It's
- 8:43:26funny. I've almost forgotten about this
- 8:43:27now because there were there was a month
- 8:43:29and a half where all of the stories were
- 8:43:31this.
- 8:43:32And then it kind of we kind of lost
- 8:43:33interest because it turns out all of
- 8:43:35these events had happened back in the
- 8:43:36first half of the year. The AI labs just
- 8:43:38hadn't realized what was going on. Once
- 8:43:40they figured out what was going on, it
- 8:43:41was mainly a case of trying to figure
- 8:43:42out all of the things that had happened.
- 8:43:44At one point the um the Australian Prime
- 8:43:46Minister gave a speech in the United
- 8:43:48Nations about one of these rogue agent
- 8:43:50attacks. So it definitely broke
- 8:43:52containment from from just within our
- 8:43:54industry. Um we did get my favorite new
- 8:43:57benchmark out of this. This is
- 8:43:58felonybench.com. It tracks the number of
- 8:44:01felony cyber attacks pulled off by each
- 8:44:03of the labs. OpenAI have 11, Anthropic
- 8:44:06have nine. Google did finally catch up.
- 8:44:08They got they they they got three and
- 8:44:09Meta's got one down there as well. So
- 8:44:12these are definitely the the the the But
- 8:44:13it's I kind of like this as an
- 8:44:15illustration of how
- 8:44:17these fable class models they can solve
- 8:44:19problems that you that they can get to
- 8:44:20goals. And if you give them the goal
- 8:44:22that is breaking into something, that's
- 8:44:24just a that's exactly the kind of thing
- 8:44:26they can just crunch through. And we're
- 8:44:27seeing a enormous uptick in security
- 8:44:30vulnerabilities in all sorts of
- 8:44:32software. Like it's a it's a very
- 8:44:34exciting and horrifying time to be a
- 8:44:37security professional at the moment.
- 8:44:39But let's talk about something a whole
- 8:44:41lot more fun than that. Um
- 8:44:43let's talk about games. And we've um we
- 8:44:44just saw Steve talking about games a
- 8:44:46moment ago. Um four years ago I started
- 8:44:49playing with GPT-3 and DALL-E and I
- 8:44:51tweeted about a game prototype that was
- 8:44:54literally just a DALL-E concept image
- 8:44:57and a paragraph of text from GPT-3
- 8:45:00describing a raccoon heist game that I'd
- 8:45:02always wanted to build.
- 8:45:03And a couple of months ago I thought I
- 8:45:05wonder if the models are good enough to
- 8:45:06build this yet. So I literally dumped
- 8:45:08the screenshot and the text into Fable
- 8:45:10Five, and it built me a game. It built
- 8:45:12me a raccoon heist game, and it looks
- 8:45:15like this, and you are a raccoon, and
- 8:45:17you're running around in the garden
- 8:45:18trying to pick up trash, and there's
- 8:45:20there's the there's a got like nighttime
- 8:45:23guards coming after you and all of that
- 8:45:24kind of thing. So you know, it it it
- 8:45:25looks like a game. Wasn't really a
- 8:45:28heist. Like it didn't feel very histy to
- 8:45:30me. Then GPT 5.6 Soul came out, and so I
- 8:45:33tried out that, and it did so much
- 8:45:35better. This is Moonlight Mayhem, one
- 8:45:37museum, three raccoons, absolutely no
- 8:45:39plan. And in this one, you are in a
- 8:45:41museum. You have to rescue your fellow
- 8:45:43raccoons who are imprisoned in the
- 8:45:44museum for whatever reason. Stack on top
- 8:45:46of each other, grab the golden sardine
- 8:45:49and pull that. So it's a proper heist,
- 8:45:50right?
- 8:45:51So it did did better, but the big
- 8:45:54question is these things both look like
- 8:45:55games, but are they fun? And the answer
- 8:45:57is I played each of these for about a
- 8:45:59minute and a half each, and then that
- 8:46:01was it. I was done. They they they these
- 8:46:02these are they're actual they're they're
- 8:46:04pure slop, right? They look like games,
- 8:46:06and then you play them, you're like,
- 8:46:07yeah, clearly the thing that built this
- 8:46:09doesn't really know what fun is. It
- 8:46:11doesn't know what what a what what a
- 8:46:12game is.
- 8:46:14And that
- 8:46:15actually sort of elevated my deep blue a
- 8:46:17little bit. I'd been feeling like all of
- 8:46:19my skills as a human being were were
- 8:46:21obsoleted,
- 8:46:22but at least they but if you think about
- 8:46:24it, computer games I think are the
- 8:46:25highest form of computer of software
- 8:46:27design of software engineering. There is
- 8:46:29so much depth to building a great
- 8:46:31computer game.
- 8:46:32So
- 8:46:33I thought that was good, you know, at
- 8:46:34least there's something left for us
- 8:46:35humans to do. I had the and then I had
- 8:46:37this project. This is I I've been
- 8:46:38working on a trying to vibe code up a
- 8:46:40little command and conquer real-time
- 8:46:42strategy game, and I was using Claude
- 8:46:44and having it use OpenAI's API key to
- 8:46:46generate images. And it built this, and
- 8:46:48it looks like command and conquer, and
- 8:46:50it's really dull.
- 8:46:52And then I just had a bit of a I had a
- 8:46:55realization. I thought, what happens if
- 8:46:56I say to Opus 5.5, this game isn't fun
- 8:47:00enough yet, make it a lot more fun?
- 8:47:03And I'm horrified to announce that it
- 8:47:05just did. Um Claude ran for 12 hours. It
- 8:47:09play tested a thousand bat simulated
- 8:47:11battles. It added like extra objective
- 8:47:14to fight over and shudders when there's
- 8:47:16an explosion and achievements and badges
- 8:47:18and all of and all of this stuff. And
- 8:47:21the game was fun. And I'm like, oh,
- 8:47:23okay, that's uh
- 8:47:24turns out that is not unique to human
- 8:47:26beings. Um, today it's now in full 3D
- 8:47:29and it's got giant stomping mech robots
- 8:47:31that hurl things at each other and
- 8:47:33there's helicopters and and this is like
- 8:47:35[laughter] this is really good. It's got
- 8:47:36multiplayer. It's it's actually got
- 8:47:38audio multiplayer where your phones can
- 8:47:40tweak it your phones can make noises at
- 8:47:42each other to to play multiplayer, which
- 8:47:44I thought was a great idea and it gave
- 8:47:46my my my wife a three-day headache, so
- 8:47:47don't do that. Um, but this is
- 8:47:50inflection point number three. The vibe
- 8:47:51coded games are good now. You can ask
- 8:47:53these things to build a game that is fun
- 8:47:56and you'll get something fun. So I've
- 8:47:58got a problem. I've got my 2D Pelican
- 8:48:01benchmark. Clearly this is not testing
- 8:48:03the limits of what these machines can do
- 8:48:05and of what these models can do anymore.
- 8:48:06I figured, okay, maybe I should do
- 8:48:08something definitely in 3D. It can be a
- 8:48:10game, a Pelican, something like a
- 8:48:12Pelican riding a bicycle game. And then
- 8:48:13literally yesterday Google's CEO Sundar
- 8:48:17Pichai tweeted this.
- 8:48:19And this is showing off Gemini for
- 8:48:21Carbon and it's a Pelican riding a
- 8:48:23bicycle in a game where you cycle around
- 8:48:25the Google campus picking things up. And
- 8:48:27I have been defeated. Like this is
- 8:48:29Google Google have stolen my benchmark.
- 8:48:31This is unrecoverable.
- 8:48:33And so
- 8:48:34a question I have is
- 8:48:36given how amazing these things are, why
- 8:48:38does my job feel harder? Why does it
- 8:48:40feel like I'm working harder? I'm
- 8:48:41thinking harder? I'm pushing myself more
- 8:48:43than I ever have before in my career as
- 8:48:45a software engineer? And I think the
- 8:48:47truth is that the easy stuff is all
- 8:48:48handled now. It used to be I could relax
- 8:48:51and refactor some tests for an hour. Why
- 8:48:54would I do that now when an agent will
- 8:48:55refactor the test better than me in 30
- 8:48:57seconds? Um, and I found this quote that
- 8:48:59actually I feel really helps us here.
- 8:49:01Um,
- 8:49:02this is Greg Lemond, a Tour de France
- 8:49:04champion. He said it doesn't get easier.
- 8:49:06You just get faster. And I think that's
- 8:49:08exactly what's happening to all of us
- 8:49:10right now.
- 8:49:11And like I said, we have to be more
- 8:49:13ambitious. We have to think of more
- 8:49:14ambitious things to do with these
- 8:49:16abilities that we have. So, let's look
- 8:49:18at my scorecard. My predictions for
- 8:49:192026. I said it will become undeniable
- 8:49:22that LLMs write good code. That was an
- 8:49:24unambitious prediction. We're there.
- 8:49:25It's obvious. It's unsurprising. I don't
- 8:49:27get any points for that one. I predicted
- 8:49:29a prompt injection challenge disaster.
- 8:49:30We still haven't had one because the
- 8:49:32models themselves don't fall for those
- 8:49:34tricks nearly as well anymore. Like the
- 8:49:36models got more ambitious and are now
- 8:49:38sort of punting off all of these attacks
- 8:49:40that I thought would take them down.
- 8:49:42So, I'm really left with just one
- 8:49:43prediction. I said Kakapo parrots will
- 8:49:44have an outstanding breeding season.
- 8:49:46They did. They did.
- 8:49:49They they're up to 325
- 8:49:52Kakapo. Best breeding season in like
- 8:49:54living memory. Absolutely fantastic
- 8:49:56news. So, I shared this with Opus 5.5
- 8:49:59Cheap Fable and I told it to make me a
- 8:50:01Kakapo dance party and it did. And here
- 8:50:04is a Kakapo dance party in celebration
- 8:50:06and if you look in the background, those
- 8:50:08are the Rimu trees with the little red
- 8:50:09fruits on them.
- 8:50:10So, we have a happy ending. Hopefully
- 8:50:12that will help overcome any sensations
- 8:50:14of deep blue that you have and thank you
- 8:50:16very much.
- 8:50:18>> [applause]
- 8:50:22[applause]
- 8:50:42>> And now it's time to bring our afternoon
- 8:50:44keynotes to a close. To wrap things up
- 8:50:47and bring us to the end of today's
- 8:50:49program, please join us in welcoming
- 8:50:51back our MC Corey Quinn, Chief Cloud
- 8:50:54Economist at the Duckbill Group.
- 8:50:58>> Ah.
- 8:51:03Uh thank you all for coming here and
- 8:51:04making today making day one work
- 8:51:06reasonably well given that about four
- 8:51:09weeks of news have dropped since the
- 8:51:10start of the morning, but that's okay.
- 8:51:13Surely tomorrow will be much more sedate
- 8:51:15and calm. Go out, have conversations,
- 8:51:18have dinner. Don't overdo the
- 8:51:21indulgences or the opening keynote back
- 8:51:23here at 9:00 a.m. tomorrow will be
- 8:51:24titled
- 8:51:26cuz no one's going to have the head to
- 8:51:27listen to anything in depth. Thank you
- 8:51:29all. Please enjoy the rest of your
- 8:51:31evening.
- 8:51:34>> [applause and cheering]
- 8:51:50>> Today's programming has reached its end
- 8:51:52state, but the run isn't over.
- 8:51:55Please collect your belongings and head
- 8:51:56through the main doors. Then join us
- 8:51:59again tomorrow for another full day of
- 8:52:01sessions.
- 8:53:41>> Yeah.
- 8:55:58>> Yeah.
About this transcript
This page contains the full transcript of AIE CODE: Google DeepMind, Anthropic, SpaceXSI, TypeSafe, Lovable, Ramp, etc (System One theme) by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 76,338 words across 12,395 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.