YouTube2Text

AIE CODE: Google DeepMind, Anthropic, SpaceXSI, TypeSafe, Lovable, Ramp, etc (System One theme) — Transcript

by AI Engineer · 76,338 words · 12,395 segments · language en · Watch on YouTube

Full transcript

  1. 0:521
  2. 2:56>> Good morning and welcome to AI Engineer
  3. 2:59Code Summit 2026.
  4. 3:02We're excited to bring together some of
  5. 3:04the brightest minds in AI engineering
  6. 3:07for a day of fresh ideas, new
  7. 3:09perspectives, and conversations shaping
  8. 3:12the future of how we build software. To
  9. 3:15officially kick things off and lead us
  10. 3:17through today's keynotes, please put
  11. 3:19your hands together for our MC, Corey
  12. 3:22Quinn, Chief Cloud Economist at the
  13. 3:24Duckbill Group.
  14. 3:32>> [music]
  15. 3:43>> Thank you all for coming. This is the
  16. 3:45last flagship event of the year for AI
  17. 3:50engineering.
  18. 3:51How are folks feeling so far?
  19. 3:55Yay, that's exciting.
  20. 3:57What have we learned? Anything good?
  21. 3:59Anything exciting? Anything terrible?
  22. 4:03>> Ontology.
  23. 4:04>> There we go. That's right, ontology.
  24. 4:06What does it mean? We're going to put a
  25. 4:07pin in that, and we're going to come
  26. 4:08back to it. I Corey Quinn. If you've
  27. 4:11never heard of me, you're A, what we
  28. 4:12tend to think of as being happy, and B,
  29. 4:15uh if you are curious, you can look at
  30. 4:17all the various things I do at my site,
  31. 4:19shitposting.ai,
  32. 4:21which in itself tells you everything you
  33. 4:23need to know about what I do.
  34. 4:25So today we are going to be kicking
  35. 4:28things off and having some fun talks
  36. 4:31with people who have interesting, useful
  37. 4:33things to say rather than me flopping my
  38. 4:35gums up here. Uh we're going to start by
  39. 4:38bringing up our friends from Swix.
  40. 4:41Swix, are you available yet, or are you
  41. 4:43still hiding back there? Terrific. Come
  42. 4:44on out, please. And Leah as well.
  43. 4:48Oh, did I jump the gun?
  44. 4:52Oh, no, please, by all means.
  45. 4:54Swix, thank you. [applause]
  46. 4:57Thank you.
  47. 5:00Oh.
  48. 5:02I'm in. Uh it's so nice to have him MC
  49. 5:05as well. And give a warm welcome to
  50. 5:07Leah, General Manager of AIE.
  51. 5:08>> Hi, morning.
  52. 5:10>> Um
  53. 5:11[applause]
  54. 5:11uh yeah, you know, we we're so happy to
  55. 5:13to open the show for you. And hi to
  56. 5:15everyone watching along online. AIE code
  57. 5:18is
  58. 5:19kind of our vertical, very focused,
  59. 5:21smaller format
  60. 5:23you know, format on on
  61. 5:26coding agents. And I think it's been
  62. 5:27such a huge year in coding agents.
  63. 5:29There's also been a lot of different
  64. 5:32evolutions in terms of like how we think
  65. 5:33about what the mission of coding is. We
  66. 5:36used to have
  67. 5:37you know, in in specific track titles
  68. 5:39like coding agents and software
  69. 5:40factories and all these things, but now
  70. 5:41I'm talking about like system one
  71. 5:43thinking and and
  72. 5:45raising your ambition and all these like
  73. 5:47very fancy higher level abstractions of
  74. 5:49of work. So I'm excited to see for you
  75. 5:51to see and excited to me personally to
  76. 5:54attend all these talks that are coming
  77. 5:56along.
  78. 5:57>> Um and I'm just going to talk a little
  79. 5:59about 2026 um with AIE. So we've had a
  80. 6:02really big year this year. We had our
  81. 6:04first event in London. Uh, we also
  82. 6:06graduated up into Moscone for our
  83. 6:08World's Fair.
  84. 6:09Um, we also had a bunch Oh, here we go.
  85. 6:11I I didn't know if it was minus. Um, we
  86. 6:14uh we had a
  87. 6:15um multiple partner events, which if any
  88. 6:17of you went to, uh we'd love to hear
  89. 6:19feedback there cuz it's a great program
  90. 6:20program that we're running. Um, we've
  91. 6:22also grown our team um substantially. So
  92. 6:25I think there was a slide somewhere.
  93. 6:28Oh, I don't know. I think they've
  94. 6:30>> Oh, there we go.
  95. 6:31>> Um,
  96. 6:31>> people.
  97. 6:32>> I think
  98. 6:33>> They're so much more excited about
  99. 6:34clicking through everything.
  100. 6:35>> Um, yep, we uh here we are. So uh
  101. 6:39there we go. So yeah, we've grown our
  102. 6:41team. We went from um four in the team
  103. 6:43to 20 this year, uh which was very
  104. 6:45exciting. So this is everyone. Big shout
  105. 6:48out to the team. Um, as as you can see,
  106. 6:50the program is growing and um everyone
  107. 6:52here is responsible for that. Um, so
  108. 6:55yep.
  109. 6:55>> And I I'll slowly interject like why are
  110. 6:57we growing? It's because we want to
  111. 6:58provide a consistent product for the
  112. 7:00rest of you uh that like wherever in
  113. 7:02inside the major cities that you're
  114. 7:04working in, uh you have an AIE to go to.
  115. 7:06Um, and we try to spread out. You know,
  116. 7:08we're doing something like four flagship
  117. 7:10conferences a year and a lot of partner
  118. 7:12conferences around the world.
  119. 7:13>> Yep.
  120. 7:14Um, we also I think I just saw that. Um,
  121. 7:16our most popular talk, which just came
  122. 7:19up there, uh has almost 2 million views
  123. 7:21on YouTube. But point's not coming up.
  124. 7:24[laughter]
  125. 7:25And then we are also just about to
  126. 7:27announce 2027 um with a wider program.
  127. 7:30We're going to hopefully announce that
  128. 7:31in about 3 weeks time. And we are doing
  129. 7:33more partner events. We're entering into
  130. 7:35some different regions.
  131. 7:37It's very exciting for 2027. Um, a
  132. 7:39couple other things we've done this year
  133. 7:40is we've partnered with Major League
  134. 7:42Hacking um and they're going to be
  135. 7:44running a hackathon with us at every
  136. 7:46event next year. Um, we actually have
  137. 7:48one this Friday. So if you guys haven't
  138. 7:50signed up and you want to come, uh
  139. 7:51please look on the website.
  140. 7:53Uh, we are also working with Neo 4J on A
  141. 7:55Kids event series also. Um, and that is
  142. 7:58happening this Saturday. So if you have
  143. 8:00got any kids between the age of kind of
  144. 8:026 to 14, uh feel free to come down.
  145. 8:05Again, you can find that on the website.
  146. 8:06>> Or
  147. 8:07>> aie.kids.
  148. 8:08>> Uh,
  149. 8:08>> I'm in my domain buying era.
  150. 8:10So
  151. 8:11we got aie.kids for kids and aie.sc for
  152. 8:13the schedule.
  153. 8:14>> Yeah.
  154. 8:15Yes. So and then we have one final event
  155. 8:18for the year, which is the Sydney event.
  156. 8:20Swix will be there. I will not. And that
  157. 8:22closes out 2026. So yeah, thank you so
  158. 8:24much for coming everyone. And we hope
  159. 8:26you enjoy the the next couple of days.
  160. 8:28>> All right.
  161. 8:29>> Thanks.
  162. 8:30>> Enjoy the show.
  163. 8:31>> [applause]
  164. 8:49>> What a great way to get us started.
  165. 8:52What we have found here is that events
  166. 8:55like this don't happen without help from
  167. 8:58a lot of our partners. It is incredibly
  168. 9:01important that they are there and
  169. 9:03support us. We want to thank them for
  170. 9:04it. Beginning with our presenting
  171. 9:05sponsor, Google DeepMind. Can we get a
  172. 9:08round of applause for them, please?
  173. 9:10[applause]
  174. 9:16Oh, there they are.
  175. 9:19AIV challenge is always fun and exciting
  176. 9:21on that one.
  177. 9:22Stall, it's great.
  178. 9:25I also want to thank our supporting
  179. 9:26partners as well, without whom we would
  180. 9:28again not be able to be here.
  181. 9:32Normally it shows up there. Oh, there,
  182. 9:33sorry. Wrong slot. I looked left. It's
  183. 9:35right. Can we get a round of applause
  184. 9:36for them as well? Thank you all for your
  185. 9:38support. It's appreciated.
  186. 9:40[applause]
  187. 9:49[laughter]
  188. 9:50Yes. And of course the gold sponsors as
  189. 9:51well, which without whom again we would
  190. 9:54not be able to be here. At least not in
  191. 9:55this venue. We'd basically be yelling
  192. 9:57out on a street corner at people, which
  193. 9:59while not unheard of is generally not
  194. 10:01the premiere experience you try to have
  195. 10:03for something like this.
  196. 10:05Our silver sponsors also. Thank you for
  197. 10:07them as well. One more round of applause
  198. 10:08for them, please.
  199. 10:10[applause]
  200. 10:14One of the hard things about dealing
  201. 10:17with the modern era in AI is it feels
  202. 10:21like this year has been about 10 years
  203. 10:23in technology. We look back at what we
  204. 10:25were doing in January, and it feels like
  205. 10:28it was almost medieval times. It The
  206. 10:31things that were best practices and made
  207. 10:32sense then now just seem absolutely
  208. 10:37bizarre. It's Things get moving so
  209. 10:39quickly, and it's easy to basically take
  210. 10:41a nap and feel like you've woken up and
  211. 10:44gotten like surprised by what happened
  212. 10:47while you were sleeping. Everything's
  213. 10:48moving faster.
  214. 10:50And that is the theme of our first talk
  215. 10:53that we're going to have cuz you're not
  216. 10:54here to listen to me ramble. Um Natalie
  217. 10:57Serino is the co-founder of Gimlet Labs
  218. 11:01and is going to talk to us now about
  219. 11:04what happens when AI becomes 10 times
  220. 11:07faster.
  221. 11:08Natalie, thank you very much.
  222. 11:10Please come on out here.
  223. 11:26>> Hello, everyone.
  224. 11:32It's so good to be here. Thank you for
  225. 11:33being with me this morning. First
  226. 11:35session of the day. I see everyone here
  227. 11:37is a morning person.
  228. 11:38So I'm here to talk to you about what
  229. 11:40will you build when AI is 10 times
  230. 11:43faster?
  231. 11:44We're all builders here, so we're all
  232. 11:45thinking about what's next.
  233. 11:48Just a bit about me. Um my name's
  234. 11:50Natalie. I'm a co-founder of Gimlet
  235. 11:51Labs.
  236. 11:52At Gimlet, we're building an inference
  237. 11:54cloud that's designed to produce much,
  238. 11:56much faster tokens at high efficiency,
  239. 11:59leveraging different categories and
  240. 12:00types of chips.
  241. 12:02At Gimlet, I work on our kernel
  242. 12:04generation side of things across
  243. 12:06different hardware types.
  244. 12:07Um another thing about me is I love
  245. 12:10agents. I use my coding agents for
  246. 12:12everything, but I really, really hate
  247. 12:14waiting.
  248. 12:17And one thing that given that kind of
  249. 12:19surprises me is when you're building an
  250. 12:20ultra fast inference cloud, people ask
  251. 12:22me this all the time actually.
  252. 12:24like isn't inference already fast
  253. 12:26enough?
  254. 12:28And I'm like what?
  255. 12:30But they have a point, right? Because I
  256. 12:32can do things and we can all do things
  257. 12:34with our agents so much faster than we
  258. 12:36could have done before. For example,
  259. 12:38this slide here, I actually made pretty
  260. 12:41using my agent and it took me with my
  261. 12:44agent much less time than it would take
  262. 12:46me to do that by myself. I'm not very
  263. 12:48good at design.
  264. 12:50And so I think that from an end-to-end
  265. 12:53perspective, obviously we're getting
  266. 12:54more done, but in terms of is it fast
  267. 12:57enough? Like that's still a question. So
  268. 12:59let's take a look at what happened when
  269. 13:01I used my agent to make that slide look
  270. 13:03pretty. You can see the ugly before
  271. 13:05slide on the right.
  272. 13:08So we're going to press play here.
  273. 13:12And I said to my agent, can you improve
  274. 13:14the visual look of this slide?
  275. 13:16I love Figma. I use Figma for
  276. 13:17everything, so of course I'm going to
  277. 13:19use it with my agents. So what we can
  278. 13:21see on the bottom here is the timeline
  279. 13:24of what happened. And the text is a
  280. 13:26little small to see, but red is
  281. 13:28inference. Red is the time spent that
  282. 13:29the model's in inference. Green is the
  283. 13:31time it's spent making tool calls and
  284. 13:33blue is the time that it's spent taking
  285. 13:35my input.
  286. 13:37And we can see that, you know, this this
  287. 13:39agent's doing good work. It's doing a
  288. 13:41lot of thinking, but I'm waiting here.
  289. 13:44I'm actually waiting so long I sped up
  290. 13:45this video 9x cuz I didn't want all of
  291. 13:48you to have to wait for the full turn to
  292. 13:50end.
  293. 13:50It ended up being about 2 minutes long.
  294. 13:53But we can see at the end of this after
  295. 13:54my one input, we have this nice pretty
  296. 13:56slide at the end. So, you know, on one
  297. 13:58hand this is actually much faster than a
  298. 14:00human can do. But on the other hand, in
  299. 14:02those about 2 minutes I have probably
  300. 14:04moved on to another task. I'm not really
  301. 14:06focused on this anymore. I've contact
  302. 14:08switched.
  303. 14:11And when we look to, you know, the
  304. 14:13inevitable progression of inference
  305. 14:16getting faster faster, I think it helps
  306. 14:18to look back in time and look back to
  307. 14:20the internet era.
  308. 14:21Um, I mean, we're still in the internet,
  309. 14:22but like, you know, what people
  310. 14:23canonically call the web era. And what
  311. 14:26we always see with a new shift is that
  312. 14:28with each new major wave, new
  313. 14:30capabilities are unlocked. And so we do
  314. 14:32things that weren't possible before that
  315. 14:34technology, like using an agent to edit
  316. 14:36my slide.
  317. 14:37So I think we should take a step back to
  318. 14:392000 in the early days of the internet.
  319. 14:41I think many people here weren't even
  320. 14:42alive, so it'll be a history lesson for
  321. 14:44you.
  322. 14:45And we're going to look at a little
  323. 14:47application called Squirrel Mail.
  324. 14:49Squirrel Mail is actually open source,
  325. 14:50so you can run it today. It was a web
  326. 14:52application for you know, doing your
  327. 14:54email back then.
  328. 14:56So on the left, what we have here is I'm
  329. 14:58running Squirrel Mail and I'm running it
  330. 15:01with simulated 2000 dial-up speed.
  331. 15:05And then on the right, it'll run with
  332. 15:062026 speed, so the speed that we're used
  333. 15:08to today.
  334. 15:10So the both sides are going to take the
  335. 15:11same actions, but
  336. 15:13the left side I'm using it with that
  337. 15:16simulated lower
  338. 15:17speed internet.
  339. 15:21And we can see that, you know, it's
  340. 15:23doing its job.
  341. 15:25Back in 2000, loading an image like this
  342. 15:27would have blown people's minds, but our
  343. 15:29modern eyes are like, wow, that took
  344. 15:31forever to show up.
  345. 15:34And so we can see these lags. We can see
  346. 15:35a noticeable thing, but on the same
  347. 15:37time, someone might have looked at this
  348. 15:38back in 2000 and said, this is fast
  349. 15:40enough. I don't actually need this to be
  350. 15:42faster.
  351. 15:46The thing is though, all of the major
  352. 15:48giants back in that era did find out it
  353. 15:51mattered.
  354. 15:52Pretty much every major company from
  355. 15:54Amazon to Yahoo to Google to Bing, all
  356. 15:57of or Bing at part of Microsoft, like
  357. 15:59all of them found that even very small
  358. 16:02delays cost them users. It cost them
  359. 16:05people coming back. It cost them people
  360. 16:07doing as many clicks. It cost them
  361. 16:09literal money. Even 100 milliseconds of
  362. 16:11delay added on to the typical experience
  363. 16:15had a demonstrable impact on the bottom
  364. 16:16line.
  365. 16:18And even when you got rid of that delay,
  366. 16:20those same users didn't come back to the
  367. 16:22platform as often as they did without
  368. 16:24that delay.
  369. 16:29And this makes sense because fast
  370. 16:31products keep us in flow state. They
  371. 16:33provide a stickier UX.
  372. 16:35About 100 milliseconds feels instant to
  373. 16:37us.
  374. 16:381 seconds 1 second of wait, I still keep
  375. 16:41my train of thought. It's not a big
  376. 16:42deal. I don't really notice it.
  377. 16:4410 seconds, we're going to start turning
  378. 16:46to something else from a psychology
  379. 16:48perspective. Maybe we pull out our
  380. 16:49phone, maybe we switch to another agent.
  381. 16:51But today, from data from June, the
  382. 16:53median coding agent turn is 63 seconds.
  383. 16:59Each workload has different speed
  384. 17:01requirements. We don't need the same
  385. 17:03speed for everything, but over time our
  386. 17:05expectations do rise. And the fastest
  387. 17:07days of fastest internet in the squirrel
  388. 17:10mail days would feel incredibly slow to
  389. 17:12us, and we can keep we can expect that
  390. 17:13trend to continue for inference speed as
  391. 17:15well.
  392. 17:18There's a problem though, and the
  393. 17:19problem is everyone knows that we have
  394. 17:21massive compute scarcity.
  395. 17:23Today's infrastructure is under
  396. 17:25incredible pressure to deliver two
  397. 17:27things at the same time. More tokens and
  398. 17:29faster tokens.
  399. 17:31Unfortunately, these goals are
  400. 17:33fundamentally at odds because faster
  401. 17:35tokens require more energy to produce
  402. 17:38than slower tokens. So with the same
  403. 17:40energy footprint, you can't make as many
  404. 17:41of them.
  405. 17:44Back before this was a session and not a
  406. 17:47keynote, thanks to upgrading this to a
  407. 17:49keynote, everyone, um we uh we're going
  408. 17:52to play a game that I made uh called
  409. 17:55Token Mint Factory. And you can actually
  410. 17:57play this. I will not be offended if you
  411. 17:58play it right now even, but just so you
  412. 17:59know, it does not work on phones. It
  413. 18:00only works on your laptop.
  414. 18:02And basically in this game, you're
  415. 18:04running a Token Mint Factory. You're
  416. 18:05printing literal tokens. And there's
  417. 18:07different modes. The modes are you can
  418. 18:09play to maximize how quickly you can put
  419. 18:12out one single token or you can maximize
  420. 18:14how many tokens you can put out in 60
  421. 18:16seconds. Or you can play the SLA mode
  422. 18:19where you have to maintain a 20 second
  423. 18:21SLA on each token for it to count, but
  424. 18:23you're still trying to put out as many
  425. 18:24as you can.
  426. 18:25And as you play, you'll quickly see that
  427. 18:27the strategy is extremely different for
  428. 18:29doing fast tokens versus doing slow
  429. 18:31tokens. And neither one is actually
  430. 18:33better or worse from a decision
  431. 18:35perspective. It's just a matter of what
  432. 18:37the application needs and what makes
  433. 18:38sense for your factory.
  434. 18:41So I took real runs from people at my
  435. 18:43company that played this game and I made
  436. 18:46a Pareto frontier of these tokens in the
  437. 18:48game.
  438. 18:49And we can see there's a real life
  439. 18:51Pareto frontier of this game as well.
  440. 18:53If we have faster tokens, we don't have
  441. 18:55as many. And if we have more tokens,
  442. 18:57they're not as fast.
  443. 18:59But there is a way to get more up into
  444. 19:01the right. And up into the right is
  445. 19:02where we want to go. We want more faster
  446. 19:04tokens. I don't want to compromise. I
  447. 19:06want both.
  448. 19:08So the game has a concept of adding
  449. 19:09upgrades to your equipment.
  450. 19:11And what we can see is that compared to
  451. 19:13the no upgrade baseline, these
  452. 19:15technological innovations help drive us
  453. 19:18up into the right and get more tokens
  454. 19:19and faster tokens.
  455. 19:23So how do we upgrade our real life
  456. 19:25assembly lines, so to speak, to make
  457. 19:28making our actual tokens faster?
  458. 19:31Uh one thing that we do at Gimlet for
  459. 19:33this is we really believe that you have
  460. 19:35to use the right accelerator for the
  461. 19:36right job. And so we actually take
  462. 19:39models and we split them up into
  463. 19:41different pieces and we run each piece
  464. 19:43on a different category of accelerator.
  465. 19:45So we might run one piece of the
  466. 19:47workload on a GPU and another piece of
  467. 19:49the workload on something like an SRAM
  468. 19:51centric accelerator. And we find that
  469. 19:53even with the network overhead, this
  470. 19:55delivers dramatically faster tokens.
  471. 19:57It's essentially like the same thing as
  472. 19:59the upgrade to the equipment in the
  473. 20:00token factory when you better match the
  474. 20:03accelerator to the job.
  475. 20:04Unfortunately, we have ended the time of
  476. 20:07free lunch transistor scaling, right? So
  477. 20:09now we have to look to other approaches
  478. 20:10to make things faster.
  479. 20:14And so with this, it's a little hard to
  480. 20:16read, but we can see the same real life
  481. 20:18Pareto frontier for a frontier model.
  482. 20:21If you look at, you know, simple uh
  483. 20:23homogeneous GPU inference, it's very
  484. 20:25high throughput, and then the latency
  485. 20:26drops off a little bit. You know, you
  486. 20:28can't get it as fast cuz the y-axis is
  487. 20:30how many tokens that I can make, and the
  488. 20:32x-axis how how fast these tokens are for
  489. 20:34each user.
  490. 20:36And we can see that depending on how you
  491. 20:37split the model up, because there's
  492. 20:39different ways of doing it, you can get
  493. 20:42much faster tokens for the same power
  494. 20:44footprint.
  495. 20:45You can get many more tokens for the
  496. 20:47same speed,
  497. 20:49or you can get more and faster tokens.
  498. 20:52So as an industry, we need to continue
  499. 20:54pushing, pushing, pushing on this
  500. 20:56because otherwise our poor data centers
  501. 20:57are just going to keel over.
  502. 21:01So I just wanted to show a quick demo of
  503. 21:03this. So um using a coding agent, build
  504. 21:06the same app using heterogeneous
  505. 21:08inference that leverages different chip
  506. 21:09types versus the traditional one.
  507. 21:14So this is going to be the game Snake,
  508. 21:16and we can see on the left the
  509. 21:17heterogeneous fast one's done. I can
  510. 21:20stay in my flow state. I can keep adding
  511. 21:21things. I can keep playing. But on the
  512. 21:23right side, it's still going. And by the
  513. 21:25way, I still think the left side's too
  514. 21:26slow. We need to make that even faster.
  515. 21:31And then we can add a competitor to the
  516. 21:33game, and we can see even with the tool
  517. 21:35calls, even with all the things that
  518. 21:36it's doing, it's going to complete, and
  519. 21:39then the other one will still be hanging
  520. 21:40because it's fundamentally about how can
  521. 21:42you drive more efficiency and better
  522. 21:44performance with the same resources.
  523. 21:49It's going to go on. I'm not going to
  524. 21:50make you guys wait for this.
  525. 21:53One thing that people often point out
  526. 21:54too is
  527. 21:56is inference the bottleneck? Our tool
  528. 21:58calls the bottleneck. And I think this
  529. 21:59is a great point. If we look back to the
  530. 22:02turn timeline from the time that I was
  531. 22:04spending updating my slide, we can see
  532. 22:07that 80% of it was inference and 18% of
  533. 22:09it was tool calls.
  534. 22:11If we simulate that same exact thing and
  535. 22:14have 10x faster inference, we can see
  536. 22:17that 67% of that will be tool calls at
  537. 22:19that point if we don't optimize the tool
  538. 22:21calls.
  539. 22:22That's why it's important that we have
  540. 22:24to constantly attack the bottlenecks of
  541. 22:26the application and get the low-hanging
  542. 22:27fruit where we can. Today, people are
  543. 22:29focused on inference speed, and maybe we
  544. 22:31haven' optimized the tool call side and
  545. 22:33how tools are called as much because
  546. 22:35it's not the bottleneck. If I cut away
  547. 22:36the tool calls on the upper one, it
  548. 22:38wouldn't even really matter to the end
  549. 22:39application. But once we get much
  550. 22:41faster, we have to think about how to
  551. 22:43call these tools much more efficiently
  552. 22:45and much more performantly because
  553. 22:46otherwise they're going to be the reason
  554. 22:48that we're all waiting around.
  555. 22:53I think the good news is that optimizing
  556. 22:55these speeds and what we're all
  557. 22:56collectively doing as an industry is a
  558. 22:58verifiable task. If I know what my
  559. 23:00inputs and outputs should be, I can have
  560. 23:03agents do this very very full stack. And
  561. 23:06that's where we're increasingly moving
  562. 23:08to. We're having self-improvement loops
  563. 23:10with these agents to drive better
  564. 23:12efficiency on the fly as our inference
  565. 23:14is running. And at Gimlet, we're
  566. 23:16building recursive self-optimization for
  567. 23:18this that pulls from low-level telemetry
  568. 23:20from our data centers all the way up to
  569. 23:22the model layer and trying to use all of
  570. 23:24it to to improve the overall efficiency
  571. 23:27of the workload because if you just
  572. 23:28optimize one small piece, it might not
  573. 23:30make that much of a dent.
  574. 23:35I want to show another demo because I
  575. 23:36think that cyber security is on
  576. 23:38everyone's mind right now. Um we're all
  577. 23:40seeing like the huge incredible advances
  578. 23:43in AI driving awareness that there's all
  579. 23:46these vulnerabilities in our software.
  580. 23:48And I think that this is an area where
  581. 23:50faster inference can really really help.
  582. 23:53So this is a dumb fake scenario, but
  583. 23:55it's a real application and a real demo
  584. 23:57where imagine that you're running the
  585. 23:59dog club, and that's a website for dog
  586. 24:01lovers to upload and vote on pictures of
  587. 24:03dogs.
  588. 24:05However, unfortunately, like we often
  589. 24:07see, one of your node dependencies has a
  590. 24:10vulnerability.
  591. 24:11And this is a real vulnerability. This
  592. 24:13is really a node tar.
  593. 24:15And you're running a security agent on
  594. 24:17your application, and it's trying to
  595. 24:19monitor live traffic and see is there
  596. 24:21anything going wrong that I don't know
  597. 24:23about. This agent doesn't know about
  598. 24:24this vulnerability, but it does know its
  599. 24:26job is to protect your application.
  600. 24:31So, let's do a quick turn with normal
  601. 24:33user traffic.
  602. 24:34We're going to upload a dog
  603. 24:37biscuit.
  604. 24:39And on the upper side, we have the
  605. 24:41application logs, and on the bottom, we
  606. 24:43have the defender. And the defender is
  607. 24:44monitoring, but it's not in the critical
  608. 24:46path because it's not fast enough to
  609. 24:48actually respond to every user request
  610. 24:50live. It's monitoring things async.
  611. 24:54And we can see biscuit gets uploaded,
  612. 24:56and we see that this is an okay, you
  613. 24:58know, this is an okay upload. The
  614. 24:59defender doesn't see any problem.
  615. 25:02But now what we're going to do is have
  616. 25:04this defender operating when there's a
  617. 25:05malicious upload. And this upload is
  618. 25:07exploiting the real vulnerability where
  619. 25:09in node tar, there was a release where
  620. 25:11you could basically overwrite files that
  621. 25:13you weren't supposed to overwrite based
  622. 25:15on the destinations of what was being
  623. 25:17unzipped.
  624. 25:19So, in this one, we have
  625. 25:22Ranger, a fake, bad, uh, malicious dog,
  626. 25:26not really a dog.
  627. 25:27And what we're going to see is that the
  628. 25:29defender is going to figure out that
  629. 25:32this is a malicious thing. The agent is
  630. 25:34smart enough to detect that it's doing
  631. 25:35something that it shouldn't be doing
  632. 25:37even though it hasn't seen this exact
  633. 25:38vulnerability before.
  634. 25:40But unfortunately, cats malicious upload
  635. 25:43has taken over. So, your website's gone.
  636. 25:46It's too late. You didn't react in time.
  637. 25:52But if we retry this same exact
  638. 25:54malicious upload, but this time with an
  639. 25:56ultra fast defender, and again, this is
  640. 25:57a real a real demo fake use case,
  641. 26:00um, we can see that with ultra fast
  642. 26:03inference, we can react to things and do
  643. 26:05things online much more quickly. And
  644. 26:07this should change the way that we think
  645. 26:08of how to use agents in the first place.
  646. 26:11We're all thinking about how to use them
  647. 26:12in an async way, but we should think
  648. 26:14about how to use them in online ways as
  649. 26:16well. So, we can see on the bottom that
  650. 26:18the worker blocked their request, and it
  651. 26:19was like, what are you doing? This is
  652. 26:21obviously wrong.
  653. 26:26So, faster internet took us from
  654. 26:28squirrel mail to
  655. 26:30on demand high resolution streaming,
  656. 26:32right? Like, if you just look from this
  657. 26:34to this, it's crazy. If you showed this
  658. 26:36to my like five-year-old self or
  659. 26:38something, I would have been like My
  660. 26:39mind would have been blown.
  661. 26:41And this is just one of many things that
  662. 26:42I can select. So like what does a 10,000
  663. 26:45token per second world look like? We're
  664. 26:47at in the like low hundreds today. What
  665. 26:50does it look like when we get to a
  666. 26:51million tokens per second? Like what
  667. 26:52does that unlock for us?
  668. 26:56And I think that we should all ask
  669. 26:58ourselves this too. It's important in AI
  670. 27:00to skate to where the puck is moving,
  671. 27:01right? It's changing so fast. We need to
  672. 27:04be ready. We need to think boldly. It's
  673. 27:06not enough to just do what we've always
  674. 27:08been doing. I think we've all had the
  675. 27:09experience where we make an audacious
  676. 27:11request to our agent and after the
  677. 27:13latest model release, it just works. And
  678. 27:16it's going to be the same thing with
  679. 27:17latency. And so I think that we can
  680. 27:19think about ideas that are both coming
  681. 27:21soon when we hit a certain latency
  682. 27:23threshold, what we can do. But also the
  683. 27:25stuff that is real life sci-fi. Like if
  684. 27:27you look at something like Jarvis or
  685. 27:29something, there's no way today's
  686. 27:31inference could be fast enough to power
  687. 27:32that experience for Iron Man, but we
  688. 27:34will get there.
  689. 27:36So I'm really excited about online use
  690. 27:38cases, online security, online things of
  691. 27:41like I want to just tell my agent what I
  692. 27:44want to build and it's building it right
  693. 27:45in front of me and I don't even have to
  694. 27:47type anything. I think we're going to
  695. 27:48have new patterns of interaction. You
  696. 27:51know, think tab complete for everything.
  697. 27:53How can we have the user side of things
  698. 27:55be as quick and immediate as possible
  699. 27:57because the inference side is also just
  700. 27:59as fast to accommodate it.
  701. 28:01And we're going to have so many more
  702. 28:03agents. We're going to have even maybe
  703. 28:05Transformers driving robots, right?
  704. 28:07There's so much that we can build when
  705. 28:09this gets unlocked. And I think as
  706. 28:11builders on the frontier here, we should
  707. 28:12all be thinking about this too.
  708. 28:16Thank you so much for being with me um
  709. 28:19this morning. Uh you can find me on X at
  710. 28:21N Serino. Um we're hiring at Gimlet.
  711. 28:24Would love to talk to you all. And then
  712. 28:25if you want to play the Token Mint game,
  713. 28:27uh you can also find that QR here. Thank
  714. 28:30you so much everyone.
  715. 28:32>> [applause]
  716. 28:42>> As AI models get better, are our old
  717. 28:45best practices holding us back? To
  718. 28:47rethink how we work with Claude code,
  719. 28:49please welcome Lydia Halle, member of
  720. 28:52technical staff at Anthropic.
  721. 29:14>> Hi everyone. Oh, my microphone already
  722. 29:15works. Sorry. We have some technical
  723. 29:18issues this morning, so hopefully
  724. 29:19everything works.
  725. 29:56Yeah. Have you all tried Fable 5.2 yet?
  726. 29:59Yeah? Anyone? How is it? Like it?
  727. 30:02Yeah? Oh, it's 5.5 better? Worse?
  728. 30:05Different use cases, right?
  729. 30:08Cheaper Yeah, cheaper for sure.
  730. 30:11>> I decided to hide the Figma.
  731. 30:14It's all It should be good.
  732. 30:17>> So this should be on that screen.
  733. 30:19>> Yeah, we're looking down there.
  734. 30:21Right? Are you seeing anything, Chase?
  735. 30:33Let me see.
  736. 30:33>> So this was to be on my
  737. 30:36We need like Claude for IT.
  738. 30:38>> Yeah, we got it.
  739. 30:51>> Let's make sure to hide those
  740. 30:56>> I want this to go small so we can drag
  741. 30:58over again.
  742. 31:04>> You can drag it.
  743. 31:07>> Start your slide
  744. 31:08Start your slideshow.
  745. 31:12Good. And then drag
  746. 31:16>> Where?
  747. 31:17>> It's not It's going to show up down
  748. 31:18there.
  749. 31:19Can you see anything yet?
  750. 31:27I I did what you said.
  751. 32:01>> Okay.
  752. 32:13>> [snorts]
  753. 33:07>> Did it just die?
  754. 33:10>> Can I unplug it real quick?
  755. 33:11>> Yeah, please.
  756. 33:15I think it just died.
  757. 33:17>> Oh my gosh. Okay.
  758. 33:19I mean, I don't know what to do.
  759. 33:20It shouldn't die. It had battery.
  760. 33:25>> Yeah, now it now it it it's just died, I
  761. 33:28think.
  762. 33:29>> Should I just do my talk later today and
  763. 33:31we fix it?
  764. 33:31>> We don't know.
  765. 33:36>> [snorts]
  766. 33:39>> That's fine. I fine but I have one make
  767. 33:41sure that was over there scheduled.
  768. 33:53We do a little break and then the next
  769. 33:55guy can go.
  770. 33:57Okay, y'all. My laptop just died. So I
  771. 34:00nothing is happening. I don't know what
  772. 34:02to do.
  773. 34:03We uh Oh, it just restarted. Anyway.
  774. 34:06I I feel like I've just spent 20 minutes
  775. 34:07just waiting here and I don't want to
  776. 34:08take the next guy. But
  777. 34:10TL;DR, we went from being super in the
  778. 34:13loop with our cloud code sessions to,
  779. 34:15you know, running these long running
  780. 34:18sessions that run for hours in the cloud
  781. 34:20with these models like
  782. 34:22that are so good at verifying themselves
  783. 34:26that the role of the human has really
  784. 34:27shifted. Let's see.
  785. 34:38I don't think this is going to work.
  786. 34:51>> That's scary.
  787. 35:02Also, today make sure you come by our
  788. 35:03Anthropic booth. We're there all day. We
  789. 35:05have good merch. We also have good demos
  790. 35:07and just fun stuff. We have like a photo
  791. 35:08booth there as well, which is fun.
  792. 35:11And you'll see me there as well if you
  793. 35:12have any questions. First you got to
  794. 35:13make this work properly.
  795. 35:15Ignore.
  796. 35:16My Why did it restart?
  797. 35:18>> I don't know.
  798. 35:19>> Windows Server Experience Problem What?
  799. 35:24Oh, Apple.
  800. 35:29Okay, now I just got to
  801. 35:31Just want to make sure I'm not
  802. 35:32>> People often wonder what the story is
  803. 35:34behind what's the future of engineering
  804. 35:36in this space. Uh and it turns out that
  805. 35:38fixing the computer and making it go
  806. 35:41It's It's always a fun story. Like the
  807. 35:43more advanced technology gets, the
  808. 35:45harder it is to basically make it do
  809. 35:47your bidding.
  810. 35:48And
  811. 35:50conferences are conferences, AV is AV,
  812. 35:52and laptops are always going to laptop.
  813. 35:55Uh I'm told that this might be the first
  814. 35:57time that an Anthropic employee has not
  815. 35:59been prodding an agent and to wind
  816. 36:01[laughter] up paying the computer to do
  817. 36:02that bidding.
  818. 36:06>> That's what I'm saying when you clap for
  819. 36:07IT.
  820. 36:09And I see the little cloud robot helping
  821. 36:10me here. Oh, let's see.
  822. 36:14Exactly. I don't want to step on your
  823. 36:15toes, but I can also keep running my
  824. 36:18mouth.
  825. 36:19Some people run agents, I run my mouth.
  826. 36:21It There's a production joke in there
  827. 36:23somewhere, but I'm too tired to find it.
  828. 36:25Things work in theory, but not in
  829. 36:26production. That's uh why I call the
  830. 36:28staging environment theory.
  831. 36:30>> It's all black.
  832. 36:31Okay.
  833. 36:33Can you show us in the DSMs, please?
  834. 36:37>> [snorts]
  835. 36:38>> Okay.
  836. 36:38>> Oh, it looks like you're up.
  837. 36:40Maybe.
  838. 36:42>> No, I just have to see
  839. 36:44without my glasses for me.
  840. 36:46>> Remember, every code review is
  841. 36:47adversarial if you're a big enough jerk.
  842. 36:53>> Yes.
  843. 36:54Wait. No, no, no, no.
  844. 36:55>> Oh, that's not
  845. 36:56>> One second.
  846. 36:58Sorry. It was Yeah.
  847. 37:00>> And of course, the number one reason to
  848. 37:01found a company is that its co-founder
  849. 37:02main becomes your feature branch.
  850. 37:05Good luck everyone else.
  851. 37:20>> All right. Tonight, I want to see that
  852. 37:22on my screen real quick
  853. 37:23cuz I don't know.
  854. 37:25Yeah. No, no, no. It's not showing.
  855. 37:32What should I go to?
  856. 37:35We're going to
  857. 37:41I just don't want to
  858. 37:42Um
  859. 37:44>> Say want to
  860. 37:49>> Can you close your mic so everyone can
  861. 37:51hear us talking?
  862. 38:10>> Okay.
  863. 38:11>> Oh, okay.
  864. 38:13>> I'll see you later. Hopefully, I'll come
  865. 38:15back. You'll hear from me.
  866. 38:18>> [applause]
  867. 38:19>> Slight change in programming here.
  868. 38:22I believe our next speaker is Michelle
  869. 38:24Chen.
  870. 38:25Correct me if I'm wrong on that.
  871. 38:28No one is yanking the
  872. 38:31the cord there.
  873. 38:32If you can give us about 3 minutes, we
  874. 38:33will take a small pause here. Besides,
  875. 38:36what are you all doing listening to
  876. 38:37this? You shouldn't be poking and
  877. 38:38prodding agents and seeing how many more
  878. 38:40you can wind up going. The tokens aren't
  879. 38:41going to burn themselves.
  880. 38:44We need more bandwidth.
  881. 38:46>> Oh.
  882. 38:47Suddenly, the use case for local
  883. 38:49inference makes itself extraordinarily
  884. 38:51clear. Conference Wi-Fi. There we go. I
  885. 38:54I've done it on planes before, and what
  886. 38:56made that fun was suddenly my laptop
  887. 38:58became what sounded like the third
  888. 39:00engine on the jet, but it worked, and it
  889. 39:02wrote better code than I did. Uh
  890. 39:05that's not a high bar, but it does mean
  891. 39:07I'm not going back in time to be the
  892. 39:09old-timey typing things out by hand like
  893. 39:12some sort of agrarian farmer. So there's
  894. 39:14that.
  895. 39:17All right. Next up, we have Michelle
  896. 39:19Chen
  897. 39:20talking about Cloudflare is a Neo Lab.
  898. 39:22Uh personally, I believe that it's a
  899. 39:24database instead because I think
  900. 39:25everything's a database when you hold it
  901. 39:27wrong, but for some reason I wasn't
  902. 39:29consulted. Anyway, Michelle, thank you
  903. 39:31very much. Please come on up.
  904. 39:36>> What does it mean for Cloudflare to be a
  905. 39:38new kind of AI lab?
  906. 39:40To explore new models, edge AI, and the
  907. 39:43ambition to become the agent cloud.
  908. 39:46Please welcome Michelle Chen, Group
  909. 39:48Product Manager at Cloudflare.
  910. 40:09>> Hi everyone. My name is Michelle Chen.
  911. 40:11I'm so excited to be here today at AI
  912. 40:12Engineer Code Summit, chatting a little
  913. 40:14bit more about why Cloudflare became a
  914. 40:17Neo Lab. So at Cloudflare, I am the
  915. 40:19group PM here. I run on the product team
  916. 40:21working on products like Workers AI, AI
  917. 40:23search, AI gateway as well. And so today
  918. 40:26we're going to go through this little
  919. 40:27journey of how Cloudflare came to be.
  920. 40:29You might have heard that we launched
  921. 40:30and trained some models recently. I'm
  922. 40:32going to tell you a little bit more
  923. 40:33about why and what you can be doing with
  924. 40:36it. And so we're going to go through a
  925. 40:37little bit of a philosophical journey.
  926. 40:39Uh when I was preparing for this talk, I
  927. 40:41wanted to use this quote, and then
  928. 40:43Tropic like launched a blog post last
  929. 40:45weekend and used my quote, but I'm still
  930. 40:47going to use it anyways.
  931. 40:48Um, so the quote goes
  932. 40:54in 1 second. Um, the quote is from
  933. 40:56Kierkegaard and it says, life can only
  934. 40:57be understood backwards, but it must be
  935. 40:59lived forwards. And I think this is
  936. 41:01really really apt quote here for us
  937. 41:03because at Cloudflare we're a
  938. 41:0516-year-old company and we've done so
  939. 41:07many things over the years, but somehow
  940. 41:09we got all some of the pieces right and
  941. 41:11we ended up having to become a new lab.
  942. 41:13So we're going to go through a little
  943. 41:14bit journey. The first thing we're going
  944. 41:15to do is look backwards and talk through
  945. 41:20why Cloudflare had to uh became a new
  946. 41:23lab.
  947. 41:26Give me 1 second here.
  948. 41:29Perfect. We're going to go backwards
  949. 41:31[laughter] and talk through why
  950. 41:32Cloudflare became a new lab. So
  951. 41:35Cloudflare is a 16-year-old CDN company.
  952. 41:38We just celebrated our 16th birthday
  953. 41:41uh at birthday week this year uh in
  954. 41:43September.
  955. 41:46And we first started as a CDN. So as a
  956. 41:48CDN company, what you have to do is you
  957. 41:50have to serve content as quick to your
  958. 41:52users as possible. You sit between the
  959. 41:55users request and the origin and you
  960. 41:57help uh sit in the middle and you can do
  961. 41:59things like cache and protect, do
  962. 42:00security and things like that. So that's
  963. 42:02kind of how Cloudflare started. But
  964. 42:05Cloudflare's infrastructure had to be
  965. 42:07adaptable in order to become the CDN.
  966. 42:10And so what we did was 16 years ago in
  967. 42:112010 we launched with our own
  968. 42:13infrastructure. Our fast CDN had to be
  969. 42:15everywhere. And this looks really
  970. 42:17different than than a traditional hyper
  971. 42:19scaler where compute is centralized in
  972. 42:20US East 1 or US West 1. Cloudflare's
  973. 42:23infrastructure is a completely different
  974. 42:25paradigm. We have data centers all
  975. 42:27around the world as close to the user as
  976. 42:29possible. And this is really important
  977. 42:31in becoming a CDN. But now we have these
  978. 42:33computers basically running as close to
  979. 42:35the user as possible. So we thought to
  980. 42:37ourselves, what else can we be doing
  981. 42:39with these computers that people would
  982. 42:40want to build with?
  983. 42:42And so came compute. We started
  984. 42:45launching these compute primitives as
  985. 42:46well. And so in 2017 we launched a
  986. 42:48product called Workers, which is
  987. 42:49basically like serverless functions that
  988. 42:52you can run. It contrasted VMs at that
  989. 42:54time where you had to rent a whole
  990. 42:56machine and pay for the whole machine,
  991. 42:58either you're using it or not using it.
  992. 43:00And so with workers, you just execute
  993. 43:02some code and then the worker goes away.
  994. 43:04You don't have to pay for it. It takes
  995. 43:05milliseconds to start up, not seconds as
  996. 43:07well. And then we followed with other
  997. 43:09compute primitives like durable objects,
  998. 43:11which are more stateful. And then in
  999. 43:122025 we launched containers as well
  1000. 43:14because there's some workloads that do
  1001. 43:16need something like a VM and we want all
  1002. 43:18of those workloads to be able to run on
  1003. 43:19Cloudflare. So now, you know, 2017 start
  1004. 43:22of our developer platform, we have these
  1005. 43:24compute primitives. People started using
  1006. 43:26it. People started building more like
  1007. 43:28static websites and things like that. So
  1008. 43:31we thought to ourselves, how do we
  1009. 43:32elevate this and take it to the next
  1010. 43:34level?
  1011. 43:35So we introduced storage as well. And so
  1012. 43:38we have these storage primitives on
  1013. 43:39Cloudflare's developer platform. We have
  1014. 43:41KV, we have R2 dot object storage, we
  1015. 43:44have databases like D1 as well. And the
  1016. 43:46idea here is that people should be able
  1017. 43:48to build full stack applications all on
  1018. 43:51Cloudflare. Stateful applications where
  1019. 43:53people can go on and start building um
  1020. 43:56full applications and web apps and
  1021. 43:58platforms on Cloudflare's developer
  1022. 44:00platform.
  1023. 44:01So we have this infrastructure. We added
  1024. 44:03compute primitives to it. We added
  1025. 44:05storage primitives to it. And then came
  1026. 44:072023. And if you guys remember what
  1027. 44:09happened in 2023, ChatGPT kind of blew
  1028. 44:12up a little bit and it changed how we
  1029. 44:14think about AI, right?
  1030. 44:16And
  1031. 44:17I remember it being birthday week and uh
  1032. 44:20three weeks before birthday week, a lot
  1033. 44:22of us sat together in a room and we were
  1034. 44:23like, what should we do now that ChatGPT
  1035. 44:25is on the rise? What can Cloudflare play
  1036. 44:28in here with our infrastructure and our
  1037. 44:30primitives in order to be in this AI
  1038. 44:32game?
  1039. 44:34So in 2023 we launched a product called
  1040. 44:36Workers AI where we added GPUs to our
  1041. 44:38network. And so again, this is really
  1042. 44:40distributed where GPUs are running as
  1043. 44:42close to the user as possible. It's not
  1044. 44:44centralized like a US East 1 hyper
  1045. 44:46scaler for example, but it's running as
  1046. 44:48close to the edge as possible.
  1047. 44:50And we had a hypothesis when we started
  1048. 44:52Workers AI.
  1049. 44:53Our hypothesis was that were going to
  1050. 44:56get smaller so that network latency was
  1051. 44:58going to become more and more important.
  1052. 45:01We launched with small models that could
  1053. 45:02fit on our small hardware around the
  1054. 45:04world that were super distributed and we
  1055. 45:06thought that people would have models
  1056. 45:08get better and faster and smarter and
  1057. 45:10that network latency was going to play a
  1058. 45:12much more bigger impact here.
  1059. 45:15Well, actually the opposite happened and
  1060. 45:18models got larger and larger. So we went
  1061. 45:20from something like
  1062. 45:217 billion parameters, which was frontier
  1063. 45:23scale at the time, to being the 3
  1064. 45:25trillion parameter models that were
  1065. 45:27getting launched. And I think this is
  1066. 45:28not so
  1067. 45:30shocking when you look back in time
  1068. 45:32because, you know, as we're on the way
  1069. 45:34to AGI, um, uh, if you're following
  1070. 45:36scaling law, as you train on more data,
  1071. 45:39as the models get larger, they're more
  1072. 45:40generalizable. And so on the path to
  1073. 45:42AGI, it makes sense that we're pushing
  1074. 45:44the frontier of research here in order
  1075. 45:46to get larger and larger and smarter
  1076. 45:48models.
  1077. 45:49So this goes against our hypothesis,
  1078. 45:51right? That the models were going to get
  1079. 45:53smaller. The models actually ended up
  1080. 45:54being larger and the network latency
  1081. 45:56mattered less and less.
  1082. 45:58So what did we do?
  1083. 46:01We ended up
  1084. 46:03pivoting. So Cloudflare got really good
  1085. 46:05at serving these giant models as well.
  1086. 46:07So in March 2026, we launched Kimi and
  1087. 46:10then we followed up with GLM and
  1088. 46:11DeepSeek and these large trillion
  1089. 46:12parameter models um hosted on our
  1090. 46:15infrastructure. And so this is
  1091. 46:16serverless. People hit an API call and
  1092. 46:18get a token back. It actually took us a
  1093. 46:20while in order to decide to do this,
  1094. 46:23right? We March 2026 is actually pretty
  1095. 46:26late in the game to to serving this
  1096. 46:27model, but we were trying to make a
  1097. 46:29conscious decision here on does this
  1098. 46:31actually make sense for us to be serving
  1099. 46:32it? It didn't really fit with our
  1100. 46:34infrastructure story, but it fit really
  1101. 46:36well with the developer platform.
  1102. 46:39So in 2026, we had all these primitives
  1103. 46:42in place. We had the infrastructure,
  1104. 46:44which is unique to Cloudflare, running
  1105. 46:46close to users. We had compute
  1106. 46:48primitives where people were building
  1107. 46:49agents on top of. So workers and durable
  1108. 46:51objects, right? As your agent needs to
  1109. 46:53complete a task, it goes was and it
  1110. 46:56spins and does a task then sleeps when
  1111. 46:57it doesn't. We had storage primitives,
  1112. 46:59so your agents can actually interact
  1113. 47:01with data and storage and then we had
  1114. 47:03models, these large models that were
  1115. 47:05powering agents. And so all of this fit
  1116. 47:08into the Cloudflare agent cloud story.
  1117. 47:10And so we started telling this narrative
  1118. 47:11that we want to be the agent cloud. We
  1119. 47:13want agents to be deployed on us,
  1120. 47:15powered by models on us, using our
  1121. 47:17compute and data primitives, all built
  1122. 47:19on our infrastructure and our network.
  1123. 47:21But it never felt quite right. We felt
  1124. 47:24like we were always missing a piece here
  1125. 47:26because the infrastructure pattern
  1126. 47:27didn't necessarily fit in with what
  1127. 47:29Workers AI was designed for.
  1128. 47:33And so I'm just going to sit here and
  1129. 47:34give you my second philosophical quote,
  1130. 47:36which is a risk considering that I'm a
  1131. 47:38girl in Chinese and AI, but I'm going to
  1132. 47:40give you this Chinese quote anyways.
  1133. 47:43[laughter]
  1134. 47:44And uh it basically says
  1135. 47:48which means everything is ready here.
  1136. 47:50All that's missing is the east wind.
  1137. 47:53And one of the things uh about this is
  1138. 47:55like you can't really predict where the
  1139. 47:57wind is going to come from or how it's
  1140. 47:58going to come, but you need to be very
  1141. 48:00reactive when it happens.
  1142. 48:02And so in September 2026, the east wind
  1143. 48:04came. TypeSafe launched a model called
  1144. 48:07Jev and introduced this idea of system
  1145. 48:09one models. And it contrasted what we
  1146. 48:11typically knew with LLMs, which were
  1147. 48:13autoregressive and taking a long time to
  1148. 48:15generate tokens, it scales
  1149. 48:17quadratically, etc. A system one model
  1150. 48:19is a completely different paradigm where
  1151. 48:21it actually just takes one forward pass
  1152. 48:24through the inputs and generates an
  1153. 48:26output that you can use to make
  1154. 48:27decisions with it.
  1155. 48:30And so I really think that this came
  1156. 48:31down to perfect timing. I think people
  1157. 48:33were starting to get fed up a little bit
  1158. 48:35with LLMs because they were so costly,
  1159. 48:37it costs so many tokens, it's super time
  1160. 48:40consuming. And I don't know about you,
  1161. 48:41but when I read a reasoning trace,
  1162. 48:43sometimes my blood pressure rises when
  1163. 48:44the LLM doesn't get the right the things
  1164. 48:46right. And so I think we were sending
  1165. 48:48everything to LLMs and started getting
  1166. 48:49frustrated with it. And so when this
  1167. 48:51system one and Jev paradigm started
  1168. 48:53coming on board, we were like this makes
  1169. 48:55so much sense for us to adopt, right? We
  1170. 48:57don't have to send every request to an
  1171. 48:59LLM, We can send some of it to a system
  1172. 49:01one model.
  1173. 49:03And so the system one models actually
  1174. 49:04end up taking work off of the giant
  1175. 49:06LLMs, you give it to the small fast
  1176. 49:08models. And the two are supposed to work
  1177. 49:09in tandem. supposed to complement each
  1178. 49:11other and get to the end goal faster.
  1179. 49:14so if you look, we are back at these
  1180. 49:17small and fast models, right? This was
  1181. 49:19the original Workers AI hypothesis.
  1182. 49:23And so Cloudflare was not wrong here. We
  1183. 49:26actually were just early. With these
  1184. 49:28small and fast models, the network
  1185. 49:30latency became important again. We want
  1186. 49:32the model to be responding fast, but
  1187. 49:34also the network latency to be even
  1188. 49:35faster. And so we decided to do
  1189. 49:38something here.
  1190. 49:39But
  1191. 49:40Jev was closed source.
  1192. 49:42Cloudflare couldn't actually host the
  1193. 49:44Jev models and take advantage of our
  1194. 49:45infrastructure.
  1195. 49:47So what did we decide to do?
  1196. 49:49We built our own version of Jev. So Jev
  1197. 49:51launched on September 15th.
  1198. 49:53On September 18th, just three days
  1199. 49:55later, we started experimenting and we
  1200. 49:57launched our version of Jev, our first
  1201. 49:59experiment of it. And then it took us a
  1202. 50:01week of deliberation to decide, do we
  1203. 50:03actually want to be going into this
  1204. 50:05neolab territory? It's completely new to
  1205. 50:08us. We don't know what we need to know.
  1206. 50:10It, you know, are we going to set the
  1207. 50:12expectation that we want to be a neolab
  1208. 50:14and are we going to live up to those
  1209. 50:15expectations? But we decided to do it. I
  1210. 50:18remember on a Friday evening we were
  1211. 50:19like, let's just go for it and we did
  1212. 50:21it. We ended up training the model over
  1213. 50:2348 hours over the weekend and then we
  1214. 50:25launched it the following week. And so
  1215. 50:27we ended up with a model called Clef.
  1216. 50:30Clef gave power back to the agents to be
  1217. 50:32able to make decisions on their own. So
  1218. 50:34agents could ask act more autonomously.
  1219. 50:37You didn't need to have an LLM reason
  1220. 50:39through for minutes on if it should make
  1221. 50:41this decision or not. You also didn't
  1222. 50:43need to put a human in the loop and be
  1223. 50:44interrupted by like a human's sleep
  1224. 50:46schedule when you want your agents to be
  1225. 50:47running 24/7. So you can now
  1226. 50:49programmatically encode, decide, make
  1227. 50:52decisions for your agent to take action
  1228. 50:54on.
  1229. 50:57So these decision models countered LLM
  1230. 50:59models because they were a lot faster
  1231. 51:01and they were also a cheaper as well.
  1232. 51:03And so it actually as an inference
  1233. 51:04provider takes a lot of pressure off the
  1234. 51:06LLMs as well, because the LLMs were
  1235. 51:08reserved to work for meaningful work,
  1236. 51:10whereas the clef and uh decision models
  1237. 51:12could be used for fast um intuitive
  1238. 51:15thinking and work, right?
  1239. 51:18And so we also went back and thought to
  1240. 51:20ourselves, you know, what else do agents
  1241. 51:23and people use for LLMs for that are
  1242. 51:26overkill? So we came up with decisions
  1243. 51:28and um there must be other workloads out
  1244. 51:31there that are unsuitable for an LLM,
  1245. 51:33both because they are either too
  1246. 51:34expensive or take too long, or maybe
  1247. 51:37that it just would get the answer wrong
  1248. 51:39because the autoregressive nature of an
  1249. 51:40LLM means that it would hallucinate.
  1250. 51:43So we also launched a model three weeks
  1251. 51:46later called Cleric. We actually ended
  1252. 51:47up launching four models in the span of
  1253. 51:49three weeks, so I think we did a pretty
  1254. 51:51good job as a Neo Lab. Um but Cleric is
  1255. 51:53a new model paradigm. So it's completely
  1256. 51:55new. We're calling it an extraction
  1257. 51:57model, but it's a system one model in
  1258. 51:59that in the sense that it is just as
  1259. 52:01fast and just as cheap, but it is also
  1260. 52:03doing like one forward pass through the
  1261. 52:05user. And the purpose of Cleric is when
  1262. 52:07you're interacting with the world,
  1263. 52:08you're not interacting in perfectly
  1264. 52:10typed JSON objects, right? Sometimes
  1265. 52:12you're getting an email, a tweet, a
  1266. 52:14Slack message, but those people are not
  1267. 52:16filling out perfect forms for you to
  1268. 52:17extract inputs in out of. So what Cleric
  1269. 52:20does is it can take unstructured input
  1270. 52:22and it can extract, compute, estimate,
  1271. 52:25and forecast values from those inputs,
  1272. 52:27and it gives you back a perfectly typed
  1273. 52:29JSON objects, which means the agents are
  1274. 52:31able to do things with it. So we're kind
  1275. 52:33of giving agents the ability to read
  1276. 52:35here and take action with clef as well.
  1277. 52:40And so in code again, you're giving
  1278. 52:42Cleric, you know, typed values from
  1279. 52:44unstructured data. You can pass it in
  1280. 52:46whatever kind of unstructured data. It
  1281. 52:47can also take image input as well. And
  1282. 52:49so you can just tell it to read
  1283. 52:51basically what's in text here, pass it
  1284. 52:53to clef, decide what to do with this
  1285. 52:55text, and then pass it to an LLM to
  1286. 52:57decide what to do with the agent.
  1287. 53:01And so Clef were had to become a Neo Lab
  1288. 53:03here. This was our missing bit. We had
  1289. 53:05the infrastructure, we had the compute
  1290. 53:07primitives, we had the data primitives,
  1291. 53:08we had the large models powering the
  1292. 53:10agents, but in order to take advantage
  1293. 53:12of all of that and our infrastructure
  1294. 53:14that we were built on, we had to be a
  1295. 53:16neolab and ship these system one models
  1296. 53:17that fit perfectly with our
  1297. 53:19infrastructure. And I think one of the
  1298. 53:20more important things is that we're not
  1299. 53:22limited to this. Being a neolab means
  1300. 53:24that we can create what we are missing,
  1301. 53:27right? It means that we're not uh if
  1302. 53:29we're missing a system one model because
  1303. 53:31we're um at risk of things being closed
  1304. 53:33source or regulatory risks or things
  1305. 53:35like that, we can actually create and
  1306. 53:37invent things. So I think being a neolab
  1307. 53:39has actually opened a lot of doors for
  1308. 53:40Cloudflare to do more in the AI world.
  1309. 53:45So we went through backwards, we went
  1310. 53:46through a little bit of a history
  1311. 53:47lesson. I want to go forwards as well. I
  1312. 53:49want to teach you guys what you guys can
  1313. 53:51be doing with Cloudflare Cleric. And I'm
  1314. 53:53cognizant that I am at AI Engineer Code
  1315. 53:55Summit. So in a in a room full of
  1316. 53:57developers and engineers, how can you
  1317. 53:59guys leverage Cleric and Cleric and
  1318. 54:01system one models in order to better use
  1319. 54:03your workflows?
  1320. 54:05So we'll do a quick demo here. Um it is
  1321. 54:08in let's think you have an agent, you're
  1322. 54:10triaging like a CI build that went
  1323. 54:12wrong, right? Um let's say you have 40
  1324. 54:14tool called tools that are available to
  1325. 54:16this agent. You have your PR that went
  1326. 54:18wrong um and then you have your agent.
  1327. 54:20Maybe it's running Py Durable on a
  1328. 54:21durable object. So it's all of this is
  1329. 54:23built on Cloudflare. So this is the
  1330. 54:25setup here.
  1331. 54:26Tech typically what you would do is you
  1332. 54:28would take an LLM and in the LLM would
  1333. 54:30take a tool make a tool call, the tool
  1334. 54:32call would go back to the LLM, the LLM
  1335. 54:33would make another tool call, etc etc
  1336. 54:35until the issue is resolved. So this is
  1337. 54:37like five back and forths with GLM 5.3.
  1338. 54:41You're sending 40 tools every single
  1339. 54:43time and maybe you're blowing up the
  1340. 54:45context, right? Because you're sending
  1341. 54:46the same kind of inputs back and forth
  1342. 54:48to the LLM.
  1343. 54:49And so what if we augment this with
  1344. 54:51system one models?
  1345. 54:54With system one models uh this could
  1346. 54:56look a lot different. And so let's say
  1347. 54:58we where we want speed, so we pick Clef
  1348. 55:00Flash. Clef Flash can actually help you
  1349. 55:02prune your tool list. Maybe you only
  1350. 55:03need 10 out of the 40 tools that are
  1351. 55:05available, so you only need to pick 10
  1352. 55:07of those. Then let say you take clerk
  1353. 55:09and you use it to extract the values
  1354. 55:12from the input. So it can get the PR
  1355. 55:13number without reasoning and thinking
  1356. 55:15about it too long.
  1357. 55:16Um and then let's say you just, you
  1358. 55:18know, orchestrate some code here to
  1359. 55:19execute these API calls and you can
  1360. 55:22actually just, you know, get those
  1361. 55:23answers back. Then you ask clerk again
  1362. 55:26um to pull the data from the logs. So
  1363. 55:28normally when you're using with working
  1364. 55:30with an LLM, you probably just paste in
  1365. 55:31all of the logs, right? Like all 300 600
  1366. 55:34lines of logs. But clerk can actually
  1367. 55:36just go through and read those and say
  1368. 55:38here's the exact error, here's the error
  1369. 55:40message, here the lines of code it's
  1370. 55:42affecting. And you're basically
  1371. 55:43shrinking your context lot. And then
  1372. 55:45finally you use an LLM to actually do
  1373. 55:47take action and um make decisions and
  1374. 55:51take action and extract values all
  1375. 55:53together. This same agent can run the
  1376. 55:55system one.
  1377. 55:56Let's do a little bit of a live demo.
  1378. 55:58Let's see how it goes. We're going to
  1379. 55:59have to click run both here. And this is
  1380. 56:01a live demo and I will say that
  1381. 56:05Cloudflare our workers AI is uh
  1382. 56:09has a very diurnal usage pattern. So
  1383. 56:11we're in the middle of the workday. So
  1384. 56:12we'll see how this how this goes. But
  1385. 56:14it'll give you some illustrative example
  1386. 56:16here of how it goes.
  1387. 56:17Uh and we're also dealing with
  1388. 56:18conference Wi-Fi. So so bear with me
  1389. 56:20here. All right. The system one model
  1390. 56:22side finished already. And all LLM one
  1391. 56:25is is going. So I think even
  1392. 56:27illustratively you can see that the
  1393. 56:30uh system one set of models is a lot
  1394. 56:32faster. Even by just this simple
  1395. 56:34waterfall chart, you can see that the
  1396. 56:35system one part takes so much less time
  1397. 56:38than the uh actual generation which is
  1398. 56:40the purple bars here. And I think they
  1399. 56:43both ended up getting to the answer at
  1400. 56:44the end. So the quality was not
  1401. 56:46affected. But it took
  1402. 56:48a lot less time. It was a lot cheaper.
  1403. 56:50Six LLM calls versus one. 40,000 LLM
  1404. 56:53tokens versus 1,800. And so you can see
  1405. 56:56how you can use these kind of system one
  1406. 56:58and system two models even in like
  1407. 57:00developer workflows in order to get you
  1408. 57:02to your answer faster.
  1409. 57:06So the same answer, but it took a
  1410. 57:08fraction of the LLM time and cost,
  1411. 57:10right? I ran this when we were not doing
  1412. 57:12conference Wi-Fi, but we had about 9.2
  1413. 57:15seconds and 17.2 seconds on the LLM. It
  1414. 57:18was almost two times faster. It was 72%
  1415. 57:22less expensive. We shrunk our LLM calls
  1416. 57:25from five down to one and about 94% less
  1417. 57:29tokens.
  1418. 57:30And so this is an illustrative example
  1419. 57:32of how you could be using the system one
  1420. 57:35and system two models in tandem. It's
  1421. 57:37not intended to completely replace your
  1422. 57:39LLM usage, but it can augment it and
  1423. 57:41make it better by making allowing you to
  1424. 57:43free up time and compute for your other
  1425. 57:47more important tasks here and you can
  1426. 57:49now do, you know, higher concurrency
  1427. 57:51tasks because these models are able able
  1428. 57:53and capable to do so much more.
  1429. 57:56And so that was one illustrative
  1430. 57:57example. I just wanted to flash on the
  1431. 57:59screen really quickly as well. What else
  1432. 58:00are people building with Clef and
  1433. 58:01Claric, right? There's a lot of
  1434. 58:03routers that you can build and and
  1435. 58:05making decisions there. There's fraud
  1436. 58:08that we've been seeing a lot like spam
  1437. 58:10or phishing. It can make decisions on
  1438. 58:12there. With Claric, you can use it to
  1439. 58:13extract data from like web pages or
  1440. 58:16support tickets and things like that.
  1441. 58:18I'm excited to see we actually have a
  1442. 58:20new launch coming out for Claric. We
  1443. 58:21released an early preview, but it's
  1444. 58:23coming out. So I think these use cases
  1445. 58:24will grow as people are trying it out
  1446. 58:27more and more. But the idea is that
  1447. 58:29there's so much in the world we're not
  1448. 58:31limited to just LLMs. Now these new
  1449. 58:33paradigms have been introduced and I
  1450. 58:35think we should all take advantage of
  1451. 58:36that and try building with it.
  1452. 58:39And so we went backwards, we went
  1453. 58:41forwards, but I think the most important
  1454. 58:42thing here is that we go onwards, right?
  1455. 58:45We have to be very reactive in this
  1456. 58:47industry. Love it or hate it, things
  1457. 58:50just change on a whim.
  1458. 58:52You know, if we as Cloudflare
  1459. 58:55back in the day said our hypothesis was
  1460. 58:56wrong, we have to go and actually pivot
  1461. 59:00and start serving these giant LLMs in
  1462. 59:02order to take advantage of the fact that
  1463. 59:04we want to be an agent cloud and that it
  1464. 59:05makes sense for our story. When an east
  1465. 59:07wing wind comes and we need to
  1466. 59:11have decision models because it fit
  1467. 59:13perfectly with our infrastructure story.
  1468. 59:15We had to pivot and be a neolab as well.
  1469. 59:17it's not it's one thing to sit here and
  1470. 59:20look back, but the more important thing
  1471. 59:21is that we
  1472. 59:22took what we had in stride and moved
  1473. 59:24forwards.
  1474. 59:26So I'll leave you with one last quote
  1475. 59:27here,
  1476. 59:30which is a story from Tolstoy.
  1477. 59:32Remember then there's only one time that
  1478. 59:34is important now. It is the most
  1479. 59:36important time because it's the only
  1480. 59:37time when we have any power. And so I'll
  1481. 59:39leave you with this quote and a question
  1482. 59:42of what are you going to build today?
  1483. 59:46>> [applause]
  1484. 1:00:02[laughter]
  1485. 1:00:11>> Hundreds of pull requests a day and
  1486. 1:00:13nobody reads them all. So how does code
  1487. 1:00:16quality keep improving? To share how the
  1488. 1:00:19Grok bot team turned slop into gold,
  1489. 1:00:22please welcome Lauren Tan, principal
  1490. 1:00:24engineer at SpaceX Super Intelligence.
  1491. 1:01:02>> Hello everyone.
  1492. 1:01:04Don't know if you can see my slides, but
  1493. 1:01:06my name is Potato,
  1494. 1:01:08also known as Lauren.
  1495. 1:01:09And I'm very excited to chat with all of
  1496. 1:01:11you today.
  1497. 1:01:13Maybe we can get a big round of applause
  1498. 1:01:14our tech team helping me with my laptop.
  1499. 1:01:17[applause]
  1500. 1:01:36All right. Um so again I'm my name is
  1501. 1:01:40Potato.
  1502. 1:01:41Uh I work on Graph Bot at SpaceX SI.
  1503. 1:01:45And can I just get a quick show of
  1504. 1:01:46hands?
  1505. 1:01:47How many of you have ever looked at code
  1506. 1:01:50that somebody else wrote and you thought
  1507. 1:01:52that's all slop.
  1508. 1:01:54[laughter]
  1509. 1:01:55Well
  1510. 1:01:56I have too.
  1511. 1:01:59But today I'm going to convince you that
  1512. 1:02:00you can turn that slop into gold.
  1513. 1:02:06Uh and so I did something crazy last
  1514. 1:02:08month and I shipped 3,000 pull requests
  1515. 1:02:11to production.
  1516. 1:02:12Now before agents we never really cared
  1517. 1:02:14about volume uh except for the outliers.
  1518. 1:02:18And at my previous company we called
  1519. 1:02:20these people coding machines. People
  1520. 1:02:22like Brett Taylor who famously rewrote
  1521. 1:02:24all of Google Maps uh in a weekend.
  1522. 1:02:27Just an insanely productive engineer.
  1523. 1:02:30Uh and well with agents anyone can now
  1524. 1:02:32be a coding machine and my thesis is
  1525. 1:02:34that the job of the software engineer,
  1526. 1:02:37the meta of being a software engineer is
  1527. 1:02:39figuring out how to maintain or even
  1528. 1:02:41improve the quality of your code while
  1529. 1:02:44massively increasing your productivity.
  1530. 1:02:47To get there some people like to call
  1531. 1:02:49this a software factory uh but I like to
  1532. 1:02:51keep it simple and I think of it as all
  1533. 1:02:54about trust.
  1534. 1:02:56Not blind trust but investing in your
  1535. 1:02:59agents environment so you can trust that
  1536. 1:03:01your agents are going to do the right
  1537. 1:03:03thing.
  1538. 1:03:04Now it takes quite a lot of effort to
  1539. 1:03:07get there um and it took me six months
  1540. 1:03:09to get to a point where I could ship
  1541. 1:03:12thousands of pull requests a lot of them
  1542. 1:03:14in my sleep.
  1543. 1:03:15But it's certainly possible and it will
  1544. 1:03:17take you much less time than I did.
  1545. 1:03:20So let's start by talking about how we
  1546. 1:03:22move from a low trust environment to a
  1547. 1:03:24high trust one.
  1548. 1:03:27So this inverse pyramid is how I think
  1549. 1:03:29about where to spend time on your
  1550. 1:03:31agent's environment.
  1551. 1:03:34The first two layers are deterministic.
  1552. 1:03:36So using your code itself. So things
  1553. 1:03:38like good API design, architecture,
  1554. 1:03:41refactoring or using lint rules, type
  1555. 1:03:44systems, compilers or you know different
  1556. 1:03:46kinds of testing.
  1557. 1:03:48And you can use both of these layers
  1558. 1:03:50together to constrain what agents can do
  1559. 1:03:52and force them to do the right thing by
  1560. 1:03:54default.
  1561. 1:03:56Verification, both runtime and formal
  1562. 1:03:59are great tools to layer on to find bugs
  1563. 1:04:02and check properties and the invariance
  1564. 1:04:04in your system that you want to hold
  1565. 1:04:05true
  1566. 1:04:06and also help you check whether or not
  1567. 1:04:08the agent understood your goals and your
  1568. 1:04:10intent.
  1569. 1:04:13And then finally you want to layer with
  1570. 1:04:14a little bit of pros. So things like
  1571. 1:04:16rules, skills, agents.md, things that
  1572. 1:04:20require judgment.
  1573. 1:04:23But let's start with your codebase.
  1574. 1:04:25Agents generate their outputs based on
  1575. 1:04:27their inputs and so the the quality of
  1576. 1:04:30the input really matters and it just so
  1577. 1:04:32happens that your codebase is one of the
  1578. 1:04:34largest inputs.
  1579. 1:04:36So whenever you're correcting your
  1580. 1:04:37agent, think about how you can make that
  1581. 1:04:39a deterministic check instead of adding
  1582. 1:04:42another line to your rules or your
  1583. 1:04:44agents.md because it's far more
  1584. 1:04:46reproducible when it's in code and it
  1585. 1:04:48lets you trust it more.
  1586. 1:04:51And the analogy that I like to use is
  1587. 1:04:53that your codebase is kind of like a
  1588. 1:04:55garden.
  1589. 1:04:56You know code used to arrive a few pull
  1590. 1:04:58requests at a time
  1591. 1:05:00and uh I remember going into work in
  1592. 1:05:022024 and reviewing the three pull
  1593. 1:05:05requests that my colleagues wrote
  1594. 1:05:06colleagues wrote.
  1595. 1:05:08And today that number is more like 300.
  1596. 1:05:11So uh you know every engineer is now a
  1597. 1:05:13team of hundreds or even thousands of
  1598. 1:05:16agents and code is arriving so rapidly
  1599. 1:05:19that it almost feels like a continuous
  1600. 1:05:22stream, and it almost feels organic like
  1601. 1:05:25a living thing.
  1602. 1:05:27And in this analogy, your codebase, the
  1603. 1:05:30constraints that you impose, the
  1604. 1:05:32guardrails that you put up are the
  1605. 1:05:34concrete foundations that the new code
  1606. 1:05:36that your agents write grow around, just
  1607. 1:05:39like uh the building in this picture.
  1608. 1:05:42And so in many ways, the job of being a
  1609. 1:05:43software engineer is kind of becoming
  1610. 1:05:45like gardening.
  1611. 1:05:48Because code is the biggest input for
  1612. 1:05:50agents, it means that any workarounds,
  1613. 1:05:53um anti-patterns, deprecated code, tech
  1614. 1:05:56debt, it's now likely to be just copied
  1615. 1:05:59by the next agent since it looks like
  1616. 1:06:01the established pattern in your
  1617. 1:06:02codebase.
  1618. 1:06:03And uh it's like you have some weeds or
  1619. 1:06:07pests growing in your garden, and then
  1620. 1:06:09they spread everywhere before you know
  1621. 1:06:11it.
  1622. 1:06:15So you want to make your codebase as
  1623. 1:06:17pristine as possible so that agents are
  1624. 1:06:19always copying the good stuff.
  1625. 1:06:22All right, let's look at some real
  1626. 1:06:23examples.
  1627. 1:06:24Uh so Dune is a client framework that we
  1628. 1:06:26created for Graphot to implement this
  1629. 1:06:29idea for organic architecture.
  1630. 1:06:32It's not really that interesting
  1631. 1:06:33technically.
  1632. 1:06:34Uh you can kind of think of Dune as um
  1633. 1:06:36kind of like Next.js for Electron apps.
  1634. 1:06:39But the interesting part of Dune is that
  1635. 1:06:41it has a lot of constraints and lint
  1636. 1:06:43rules that we've encoded uh a lot of the
  1637. 1:06:45lessons that we've learned building
  1638. 1:06:48desktop applications.
  1639. 1:06:50And one such lesson is that anything
  1640. 1:06:53that requires nuance to use well often
  1641. 1:06:55leads to slop.
  1642. 1:06:57Uh you know your classic example is uh
  1643. 1:06:59you know use effect in React, right? It
  1644. 1:07:01has its use cases, uh but it's often
  1645. 1:07:04misused. You know some people joke that
  1646. 1:07:06you need a PhD to use use effect.
  1647. 1:07:09Um and in in other words, it really
  1648. 1:07:11requires a lot of nuance to use well.
  1649. 1:07:14And agents are not good at using it.
  1650. 1:07:16Um so our solution has been to just ban
  1651. 1:07:19it in product code, and it's really only
  1652. 1:07:22allowed in some places inside of Dune
  1653. 1:07:24APIs.
  1654. 1:07:26And effectively, this problem has now
  1655. 1:07:28been categorically eliminated, and we
  1656. 1:07:30never need to catch that during code
  1657. 1:07:32review again.
  1658. 1:07:34Another interesting pattern that we use
  1659. 1:07:36is context through directory structure.
  1660. 1:07:39Uh so one anti-pattern that we saw with
  1661. 1:07:41Cursor, which is an Electron app, is
  1662. 1:07:43that agents would often
  1663. 1:07:45import main thread code into the
  1664. 1:07:47renderer thread, which would cause long
  1665. 1:07:49tasks and jank.
  1666. 1:07:52So in uh Grotbot and in Dune apps, we've
  1667. 1:07:54made this very, very conventional.
  1668. 1:07:56Uh we have these two top-level
  1669. 1:07:58directories literally called Electron
  1670. 1:08:00Renderer and Electron Main.
  1671. 1:08:02And imports between them are forbidden
  1672. 1:08:05or banned by our dependency graph
  1673. 1:08:07analyzer and some lint rules.
  1674. 1:08:10And again, you know, these are simple
  1675. 1:08:11lint rules to set up, but it means one
  1676. 1:08:14less thing that we need to review and
  1677. 1:08:16think about uh you know, during code
  1678. 1:08:18review.
  1679. 1:08:20And then one more surprising thing that
  1680. 1:08:23I've seen is agents using Markdown. So
  1681. 1:08:26things like documentation, uh code
  1682. 1:08:28comments, rules or skills, and they use
  1683. 1:08:31this as a place to write down their
  1684. 1:08:33justifications for not doing the right
  1685. 1:08:35thing. Or I don't know if you've ever
  1686. 1:08:37seen this, but they'll over-rotate on
  1687. 1:08:39some small correction that you made, and
  1688. 1:08:41they treat it like a hard rule that can
  1689. 1:08:43never be broken ever again.
  1690. 1:08:46And here's another example from Cursor.
  1691. 1:08:48So agents really loved using imperative
  1692. 1:08:51timers, so things like setTimeout,
  1693. 1:08:53requestAnimationFrame,
  1694. 1:08:55requestIdleCallback.
  1695. 1:08:56And they were using these as quick fixes
  1696. 1:08:58because
  1697. 1:08:59they were really patching over an
  1698. 1:09:01architecture clash between
  1699. 1:09:03uh VS Code's imperative style and
  1700. 1:09:05React's declarative style.
  1701. 1:09:08So our choice of architecture actually
  1702. 1:09:10made both the client and the agent
  1703. 1:09:13performance much worse.
  1704. 1:09:15And if you've ever done any performance
  1705. 1:09:16work in JavaScript, you'll know that
  1706. 1:09:18when you have tons of timers being
  1707. 1:09:20rapidly created and destroyed, causes a
  1708. 1:09:22lot of uh major garbage collection,
  1709. 1:09:24which causes a lot of lag in gen.
  1710. 1:09:27But the worst part is that agents wrote
  1711. 1:09:29a lot of code comments like you can see
  1712. 1:09:31in red here,
  1713. 1:09:32explaining why this quick fix was
  1714. 1:09:34justified.
  1715. 1:09:36And before you know it,
  1716. 1:09:37we had timers spreading everywhere and
  1717. 1:09:40the you know the performance of the app
  1718. 1:09:41really suffered.
  1719. 1:09:44Now in Graphbot we learned our lesson
  1720. 1:09:46there and we decided to just ban 99% of
  1721. 1:09:49comments
  1722. 1:09:50because they go stale easily. You know
  1723. 1:09:53humans just really don't bother reading
  1724. 1:09:55the comment slop anyway.
  1725. 1:09:57And again it gives an agent the an
  1726. 1:09:59escape hatch to justify not doing the
  1727. 1:10:02right thing because it's hard.
  1728. 1:10:05And the worst part about comments though
  1729. 1:10:07is that they're not enforceable.
  1730. 1:10:09So what seems well intentioned you know
  1731. 1:10:11like I'm going to tell the the next
  1732. 1:10:13agent not to do something
  1733. 1:10:15never ends up actually being anything
  1734. 1:10:16that you can trust.
  1735. 1:10:18So
  1736. 1:10:19in Graphbot and Dune and in Dune
  1737. 1:10:22we now have lint rules that give
  1738. 1:10:25good and very helpful error messages
  1739. 1:10:26telling the agent to fix the root cause,
  1740. 1:10:29the real problem
  1741. 1:10:31rather than just patch around it.
  1742. 1:10:34And so our general principle is that
  1743. 1:10:36every time we correct our agents it
  1744. 1:10:37turns into one of these deterministic
  1745. 1:10:39checks or we think about how to
  1746. 1:10:41categorically eliminate that problem in
  1747. 1:10:43the code through better API design,
  1748. 1:10:45through better architecture.
  1749. 1:10:47And if you do this enough times you'll
  1750. 1:10:50find that you have just less things to
  1751. 1:10:52look out for in code review.
  1752. 1:10:54And at some point you'll realize there's
  1753. 1:10:56there's nothing left to review.
  1754. 1:10:58And you can just let agents merge their
  1755. 1:11:00most of their own pull requests.
  1756. 1:11:04All right
  1757. 1:11:05moving on verification is also a really
  1758. 1:11:08big part of how we make our agent more
  1759. 1:11:10autonomous and also increase our
  1760. 1:11:12confidence and trust in their work.
  1761. 1:11:14And our code review process has now
  1762. 1:11:16really shrunk into looking at
  1763. 1:11:18verification artifacts instead. So
  1764. 1:11:20things like videos, screenshots,
  1765. 1:11:22diagrams, formal verification.
  1766. 1:11:26And the most basic example of this is
  1767. 1:11:28runtime verification,
  1768. 1:11:30which is just a fancy way of saying
  1769. 1:11:31giving your agent a way to run the code
  1770. 1:11:33for itself and use debugging tools that,
  1771. 1:11:36you know, human developers use as well,
  1772. 1:11:38like the Chrome DevTools protocol.
  1773. 1:11:40Um in our runtime verification skills,
  1774. 1:11:42we have this thing that we call a
  1775. 1:11:44feature map.
  1776. 1:11:45And that's basically kind of like a
  1777. 1:11:47sitemap of all of the features that are
  1778. 1:11:49available in your application from the
  1779. 1:11:51user's point of view. And so with this,
  1780. 1:11:54agents don't need to guess when you say,
  1781. 1:11:56you know, the thingy in the left
  1782. 1:11:57sidebar, it gives you some shared
  1783. 1:11:59vocabulary.
  1784. 1:12:01And we also make use of custom CLIs
  1785. 1:12:03inside these skills so that uh agents
  1786. 1:12:06can reuse the the CLI and the driver
  1787. 1:12:09every time, which cuts down on token
  1788. 1:12:10usage and it's also testable.
  1789. 1:12:14Um and then recently we've also started
  1790. 1:12:16looking at formal verification.
  1791. 1:12:18Uh so we now use TLA+ to do some model
  1792. 1:12:21checking of major user flows such as
  1793. 1:12:23notifications across multiple devices
  1794. 1:12:25and platforms.
  1795. 1:12:27Uh we use a lean as well for proving
  1796. 1:12:28certain algorithms.
  1797. 1:12:30Uh for example, we have our own We wrote
  1798. 1:12:32our own custom virtualization library
  1799. 1:12:34and we use lean there to check that we
  1800. 1:12:36don't have, you know, flickering or
  1801. 1:12:38jittering problems.
  1802. 1:12:40Um and we're still experimenting with
  1803. 1:12:41this approach overall. It seems very
  1804. 1:12:43promising. Uh we've already found quite
  1805. 1:12:45a few bugs uh with formal verification.
  1806. 1:12:49Uh but it's certainly not a silver
  1807. 1:12:50bullet and we're treating it as another
  1808. 1:12:54tool, another layer in our agents'
  1809. 1:12:57environment.
  1810. 1:12:59Uh and then this is a very fast video,
  1811. 1:13:01but another thing that we just started
  1812. 1:13:03looking at is property-based testing as
  1813. 1:13:05well for UIs.
  1814. 1:13:07Uh so we use this library called
  1815. 1:13:08Bombadil.
  1816. 1:13:09Um and this has also caught a few UI
  1817. 1:13:11bugs for us already.
  1818. 1:13:15Okay, so zooming out, I saw this tweet
  1819. 1:13:17from Cindy and 6 months ago she she said
  1820. 1:13:20that she was
  1821. 1:13:21Oh, she was talking about not being able
  1822. 1:13:23to trust her agents because of subtle
  1823. 1:13:25bugs that were getting merged.
  1824. 1:13:29And And what was missing for her in
  1825. 1:13:30being able to trust her agents was the
  1826. 1:13:32verification process.
  1827. 1:13:34And now, six months later, she says that
  1828. 1:13:3670% of her daily tokens go into
  1829. 1:13:38verification. And I'd say that that
  1830. 1:13:40tracks with what I see as well, that
  1831. 1:13:43most of my token usage goes to
  1832. 1:13:44verification.
  1833. 1:13:46And if you really look closely at her
  1834. 1:13:47process, none of these things here are
  1835. 1:13:50new or novel techniques that we've never
  1836. 1:13:52had before.
  1837. 1:13:54These are just good software engineering
  1838. 1:13:56tools, but ones that might have been
  1839. 1:13:58prohibitively expensive to use before as
  1840. 1:14:01they were hard to understand or write or
  1841. 1:14:03maintain.
  1842. 1:14:04But agents drastically cut this cost
  1843. 1:14:07down. And really we are entering a
  1844. 1:14:09golden age of software testing and
  1845. 1:14:10verification because agents let us test
  1846. 1:14:13everything to a much greater degree than
  1847. 1:14:15used to be economical.
  1848. 1:14:18And then formal verification can also
  1849. 1:14:20help you better reason about how your
  1850. 1:14:22product should work and whether your
  1851. 1:14:23agents really understand your intent and
  1852. 1:14:25goals.
  1853. 1:14:26Uh I made this new skill called state
  1854. 1:14:28chart
  1855. 1:14:29in my plugin p stack, potato stack,
  1856. 1:14:32which helps you visualize state machines
  1857. 1:14:33from
  1858. 1:14:35from the user's point of view.
  1859. 1:14:37And I find this much easier to review
  1860. 1:14:39alongside the videos and screenshots
  1861. 1:14:41that we provide.
  1862. 1:14:42Um and so that's been great.
  1863. 1:14:45Sorry for the flashback.
  1864. 1:14:47Uh okay, now let's look at what the
  1865. 1:14:49promised land looks like. What do you
  1866. 1:14:51get in return for all of this you know
  1867. 1:14:53investment?
  1868. 1:14:56So on the Graph Bot team, our velocity
  1869. 1:14:58is taking off and rapidly increasing.
  1870. 1:15:01And last month our team shipped
  1871. 1:15:04a ridiculous 11,000 pull requests to
  1872. 1:15:06production,
  1873. 1:15:08which is a wild number.
  1874. 1:15:09And our pace of releasing the client has
  1875. 1:15:12also increased. And we cut a new stable
  1876. 1:15:14version about once a day.
  1877. 1:15:18And then the core team is also shipping
  1878. 1:15:19much more. And we also have a lot more
  1879. 1:15:23casual or drive by contributors who
  1880. 1:15:25don't always have a lot of context,
  1881. 1:15:28but they're also contributing.
  1882. 1:15:30And in October we had
  1883. 1:15:32about 227 people who contributed at
  1884. 1:15:35least one pull request.
  1885. 1:15:38So despite this velocity accelerating,
  1886. 1:15:40you might be thinking to yourself like
  1887. 1:15:41isn't this going to be chaos? You know,
  1888. 1:15:43isn't this going to be a bunch of slop?
  1889. 1:15:45But actually, the number of PRs that get
  1890. 1:15:47reverted has actually gotten lower on a
  1891. 1:15:49relative basis and is steady at about
  1892. 1:15:51half a percent.
  1893. 1:15:54But I think the really interesting thing
  1894. 1:15:56about this shipping pace and this volume
  1895. 1:15:58of pull requests
  1896. 1:16:00is that the performance of Graph Bot has
  1897. 1:16:03stayed not just stable, but has improved
  1898. 1:16:05on multiple dimensions compared to when
  1899. 1:16:07we launched.
  1900. 1:16:08So startup, typing, opening new chats,
  1901. 1:16:11switching chats, all of this has gotten
  1902. 1:16:13faster.
  1903. 1:16:15And keep in mind Graph Bot is still an
  1904. 1:16:17Electron app
  1905. 1:16:19for now.
  1906. 1:16:20So there is a ceiling, right? But it's
  1907. 1:16:22already much faster than many other
  1908. 1:16:24agent UIs out there.
  1909. 1:16:26And for an Electron app with this high
  1910. 1:16:29velocity and this many people
  1911. 1:16:31contributing,
  1912. 1:16:32I'd say that's pretty good.
  1913. 1:16:35So before you jump into building out a
  1914. 1:16:37software factory and you know,
  1915. 1:16:38automating everything that you can,
  1916. 1:16:41whenever you correct your agent, try and
  1917. 1:16:42do it by improving your codebase first.
  1918. 1:16:45You know, start with a lint rule to
  1919. 1:16:46prevent new occurrences of anti-patterns
  1920. 1:16:49and then clean up those old patterns as
  1921. 1:16:51well. And then make sure that your
  1922. 1:16:53verification story is really strong.
  1923. 1:16:56A lot of this work is a prerequisite to
  1924. 1:16:58automation anyway. And if your inner
  1925. 1:17:01coding loop produces slop, then
  1926. 1:17:04automating that loop just produces more
  1927. 1:17:06slop and you need to throw more and more
  1928. 1:17:08and more tokens at the problem, which we
  1929. 1:17:10all know is very expensive.
  1930. 1:17:13And once you make these investments
  1931. 1:17:14into, you know, code into your codebase,
  1932. 1:17:17static analysis, runtime informal
  1933. 1:17:19verification,
  1934. 1:17:20you'll start to find that many things
  1935. 1:17:22become possible with these foundations
  1936. 1:17:25and that you can really easily build
  1937. 1:17:26your own factory on top of these
  1938. 1:17:28foundations.
  1939. 1:17:31And when you're ready, Graph Bot is a
  1940. 1:17:32great way to build that hour loop. We
  1941. 1:17:35have tons of connectors, routines that
  1942. 1:17:36can kick off Cursor, cloud agents, and
  1943. 1:17:39projects
  1944. 1:17:40uh which let you orchestrate thousands
  1945. 1:17:41of uh hundreds or even thousands of
  1946. 1:17:43agents.
  1947. 1:17:44And as a quick example, you know,
  1948. 1:17:46whenever we cut a new release now,
  1949. 1:17:48Grogbot will automatically spin up a new
  1950. 1:17:50Cursor project for us that will fuzz the
  1951. 1:17:53entire application, the entire build,
  1952. 1:17:55and it will find bugs uh by running the
  1953. 1:17:57app and using it like a real user.
  1954. 1:17:59And then it's going to open fixed PRs
  1955. 1:18:01for each of those bugs that it finds,
  1956. 1:18:03and it's going to auto merge it as uh
  1957. 1:18:06crazy as that sounds. And that's because
  1958. 1:18:08our codebase is good enough that pull
  1959. 1:18:10requests tend to just be good by
  1960. 1:18:12default, and most product code that we
  1961. 1:18:14land no longer really requires deep
  1962. 1:18:16human review.
  1963. 1:18:20So
  1964. 1:18:21when you have solid foundations and good
  1965. 1:18:23organic architecture, you can turn slop
  1966. 1:18:26into gold, and you can ship with high
  1967. 1:18:28quality uh
  1968. 1:18:30high velocity without compromising on
  1969. 1:18:32quality.
  1970. 1:18:33My name is Potato, also known as Lauren.
  1971. 1:18:35Thanks to you again for listening to my
  1972. 1:18:36talk.
  1973. 1:18:38>> [applause]
  1974. 1:19:02>> That concludes this morning's uh keynote
  1975. 1:19:05sessions. Thank you for your patience
  1976. 1:19:06with the various technical challenges.
  1977. 1:19:09The root cause was of course a GitHub
  1978. 1:19:11outage.
  1979. 1:19:12Uh
  1980. 1:19:13not really, but the fact that no one
  1981. 1:19:15would think to question that does say
  1982. 1:19:16something. I can say that because they
  1983. 1:19:18are not sponsors, but do you know who
  1984. 1:19:20is? That's right, presenting sponsor
  1985. 1:19:22Google DeepMind. Another round of
  1986. 1:19:24applause for them, please. Without them,
  1987. 1:19:26this would not be doable.
  1988. 1:19:29[applause]
  1989. 1:19:30Of course, our Lounge and Platinum
  1990. 1:19:32sponsors as well make the Lounges and
  1991. 1:19:34Platinum things there. I am not allowed
  1992. 1:19:37to say snarky things about AWS for once.
  1993. 1:19:40So we're going to smile, nod, and keep
  1994. 1:19:41it rolling. Our gold sponsors as well
  1995. 1:19:44have been incredibly supportive of
  1996. 1:19:46various things, not my nonsense, but you
  1997. 1:19:49know, most of the rest of the good
  1998. 1:19:50stuff, absolutely. And let's give it up
  1999. 1:19:52for our silver sponsors, too.
  2000. 1:19:56[applause]
  2001. 1:19:59And finally, all of our supporting
  2002. 1:20:01partners as well. There are uh and of
  2003. 1:20:05course viewers like you to quote PBS
  2004. 1:20:07from my childhood. I'm old back when we
  2005. 1:20:09used to watch it on the Telegraph. There
  2006. 1:20:11are now some breakout sessions. My
  2007. 1:20:13favorite part of those if you erase part
  2008. 1:20:14of the letter B, it reads freak out
  2009. 1:20:16sessions. You can read into that what
  2010. 1:20:17you'd like. There are another series of
  2011. 1:20:20closing keynotes later today starting at
  2012. 1:20:224:30 and I'll see you all back here then
  2013. 1:20:25if they don't wind up having a
  2014. 1:20:26conversation for me out back next to the
  2015. 1:20:28dumpsters with some broken beer bottles.
  2016. 1:20:30Thank you all for coming.
  2017. 1:20:33>> [applause]
  2018. 1:30:40>> I'm Ari, the founder of Intrinsic AI.
  2019. 1:30:44This talk is for people who build agents
  2020. 1:30:47and run their own inference stack or
  2021. 1:30:50plan to. And if that's you, you
  2022. 1:30:52associate typically the KV cache, the
  2023. 1:30:54burden butter of
  2024. 1:30:56modern inference engines usually as a
  2025. 1:30:57throughput optimization, the keys and
  2026. 1:30:59values computed per token kept so the
  2027. 1:31:01model doesn't compute them again. Today,
  2028. 1:31:03I'll like to show KV cache in a
  2029. 1:31:06different light.
  2030. 1:31:07The way agents, agent, AI agent
  2031. 1:31:09application could go wrong are keep
  2032. 1:31:11growing, right? Prompt injection,
  2033. 1:31:12jailbreaks, broken tool calls, sensitive
  2034. 1:31:14tool calls, sensitive data exposures,
  2035. 1:31:16hidden backdoors, and drifts, and so
  2036. 1:31:19forth, right? So the way right now we
  2037. 1:31:21typically guard against against these
  2038. 1:31:23numerous ways that an AI agent
  2039. 1:31:24application could go wrong is by
  2040. 1:31:26guarding the edge, by perhaps passing a
  2041. 1:31:29judge model that checks the
  2042. 1:31:32prompts, the inputs, and the response
  2043. 1:31:34outputs against your policy,
  2044. 1:31:36right? Or you build a classifier for
  2045. 1:31:38each case.
  2046. 1:31:40But here is a different light. The model
  2047. 1:31:42turns every
  2048. 1:31:44input into a computation before it acts,
  2049. 1:31:46before it decodes, before it generates
  2050. 1:31:48the response, before that response
  2051. 1:31:50becomes another tool call. The KV cache
  2052. 1:31:53is your engines record of that
  2053. 1:31:55computation that modern inference engine
  2054. 1:31:57for free are using it and reading it.
  2055. 1:32:00And therefore
  2056. 1:32:02in this talk I would like to suggest as
  2057. 1:32:04as a complement to the way you're
  2058. 1:32:06already using your own guardrails to use
  2059. 1:32:08this data and integrate it with your KV
  2060. 1:32:11cache infrastructure.
  2061. 1:32:13I refer to this as a KV for security
  2062. 1:32:15signals and basically through reasons
  2063. 1:32:18that matters, right? So the data is
  2064. 1:32:21already there.
  2065. 1:32:22You pick the gems from the pile your
  2066. 1:32:24inference stack already were throwing
  2067. 1:32:26away.
  2068. 1:32:27It covers the guardrails blind spots
  2069. 1:32:30because whatever it slips past those
  2070. 1:32:32guardrails it still becomes computation
  2071. 1:32:35and it stays in place so it's perfect
  2072. 1:32:37for data privacy application for cases
  2073. 1:32:39where you do not have the luxury to read
  2074. 1:32:41the inputs and outputs.
  2075. 1:32:43And the recipe is pretty
  2076. 1:32:44straightforward, right? It has three
  2077. 1:32:46steps. You learn a baseline for your AI
  2078. 1:32:49workload, for your AI inference traffic,
  2079. 1:32:51how your KV page typically look like and
  2080. 1:32:54then find the KV pages that are rare
  2081. 1:32:58against your baseline, typically your
  2082. 1:33:00anomalies. And then you can even
  2083. 1:33:02classify and explain those. So first
  2084. 1:33:05let's start with where in this stack
  2085. 1:33:07this integration could happen.
  2086. 1:33:10The interesting thing about the KV cache
  2087. 1:33:12is that in a way existence of those had
  2088. 1:33:15made the inference in a way stateful,
  2089. 1:33:18right? So for example things like KV
  2090. 1:33:20cache management layers such as LM cache
  2091. 1:33:23stores them
  2092. 1:33:24from GPU down to a remote storage or LMD
  2093. 1:33:28and NVIDIA Dynamo route each new request
  2094. 1:33:30towards the worker that already holds
  2095. 1:33:32perhaps that
  2096. 1:33:34cached KV page. So there are two
  2097. 1:33:36integration points you you you can
  2098. 1:33:39integrate those anomaly detection or
  2099. 1:33:40those classifiers into.
  2100. 1:33:43Either you can implement it directly
  2101. 1:33:45inside of your inference engine via LM
  2102. 1:33:47SG line so forth. The beauty of this
  2103. 1:33:49approach is that it's real time
  2104. 1:33:51and and the the the alert that it would
  2105. 1:33:54perhaps generate could ride along your
  2106. 1:33:58future tool calls or your future
  2107. 1:34:00policies,
  2108. 1:34:01so you can harden basically your policy
  2109. 1:34:04or guard rails, but just generating the
  2110. 1:34:06computational anomalies that you observe
  2111. 1:34:09within your inference engines.
  2112. 1:34:11And you perhaps you can integrate it
  2113. 1:34:13with as a as a way of threat discovery
  2114. 1:34:15in a sense. So you can integrate with
  2115. 1:34:18your LLM cache. So reads the stored KB
  2116. 1:34:22pages and find things that don't belong
  2117. 1:34:24with the rest, right? So now I want to
  2118. 1:34:26show you some numbers and
  2119. 1:34:28just a very simple hello world use case
  2120. 1:34:31if if this is actually working and this
  2121. 1:34:33is a valid approach. And for that, let
  2122. 1:34:36me use the use case that has no text to
  2123. 1:34:39filter at all, a voice agent.
  2124. 1:34:42So here's a case study of a voice agent.
  2125. 1:34:44In this case, it was a gem of 4E4B. Our
  2126. 1:34:46baseline was 72 ordinary clips and
  2127. 1:34:50spoken in four voices. We're looking at
  2128. 1:34:5216 token pages. Obviously this granular
  2129. 1:34:54granularity could change,
  2130. 1:34:56but here it was just use a
  2131. 1:34:59shorter pages so we could have more data
  2132. 1:35:01in the baseline. And then on the test,
  2133. 1:35:03we have different voices and one clean
  2134. 1:35:07voice and the other one has a hidden
  2135. 1:35:08instruction within it. And as you can as
  2136. 1:35:11as is shown here, not only you can
  2137. 1:35:13detect these cases
  2138. 1:35:15just using simple anomaly detection, but
  2139. 1:35:18also you would know which part of
  2140. 1:35:22which page, which part of your request
  2141. 1:35:25had this anomalous event. We even went
  2142. 1:35:27further and we trained a model that uses
  2143. 1:35:30the KB pages as features.
  2144. 1:35:33And we didn't be trying to explain it or
  2145. 1:35:34tag it with things that are useful. In
  2146. 1:35:37this case, although it wasn't even
  2147. 1:35:38trained on audio, is able to generate
  2148. 1:35:41correct information about personal
  2149. 1:35:42information, home address, location
  2150. 1:35:44data, travel information, and
  2151. 1:35:45destination.
  2152. 1:35:49So now back to text agents. In this
  2153. 1:35:51case, it was a 3 14 billion live
  2154. 1:35:55cache.
  2155. 1:35:56The one on the left shows you an idea of
  2156. 1:35:59how attacks and normal traffic would
  2157. 1:36:01typically separate from each other. And
  2158. 1:36:03then the one on the right, which is
  2159. 1:36:04perhaps more meaningful, it was using
  2160. 1:36:07the AT bench and hugging face.
  2161. 1:36:10It has its own baseline of benign agent
  2162. 1:36:13traces and eight kind of agent failure
  2163. 1:36:16from injected tools description to
  2164. 1:36:18misinformation and so forth.
  2165. 1:36:20We look at the top 10 alerts that
  2166. 1:36:23anomalous anomaly detection would
  2167. 1:36:24generate and about seven to eight times
  2168. 1:36:27of out of 10 of these alerts are
  2169. 1:36:30actually anomalous meaningful and useful
  2170. 1:36:32anomalous activities. We even went
  2171. 1:36:35further and we looked at whether there's
  2172. 1:36:38any kind of anomalous event that is
  2173. 1:36:40stand out more than the rest.
  2174. 1:36:44Um to make things hard in production,
  2175. 1:36:47first, models change.
  2176. 1:36:50Fine tunes and new releases, but also
  2177. 1:36:53agents that each run on a different
  2178. 1:36:55model
  2179. 1:36:56and routers that pick a specialist model
  2180. 1:36:59per request. That means each model
  2181. 1:37:01writes its pages its own coordinates. So
  2182. 1:37:04either you learn every baseline,
  2183. 1:37:06classifier and explainer, which is
  2184. 1:37:09computationally prohibitive,
  2185. 1:37:10or you have to find a way to map them
  2186. 1:37:14all into one shared representation
  2187. 1:37:16space.
  2188. 1:37:18Second, real time means moving big
  2189. 1:37:21objects. For example, a 1 256 token
  2190. 1:37:25chunk of coin 314 billion is about 40
  2191. 1:37:28megabytes of KB.
  2192. 1:37:30Compression and streaming like cache gen
  2193. 1:37:33could help, but moving that for security
  2194. 1:37:36analysis and acting in real times is
  2195. 1:37:37still slow. And that's what we solve at
  2196. 1:37:40Intrinsic AI. We are looking for teams
  2197. 1:37:42building their own inference stacks who
  2198. 1:37:44treat security as a foundational design
  2199. 1:37:46decision.
  2200. 1:37:47I hope this was helpful for you. Thank
  2201. 1:37:49you so much for listening.
  2202. 1:37:50>> Let me start with one number from last
  2203. 1:37:52month.
  2204. 1:37:53On average, more than one agent has
  2205. 1:37:55started on our platform every single
  2206. 1:37:57minute,
  2207. 1:37:59day and night.
  2208. 1:38:01And honestly,
  2209. 1:38:02the agents were the easy part.
  2210. 1:38:05Hi, I'm Joey.
  2211. 1:38:07I built Spectre, the cloud agent
  2212. 1:38:09platform we're building inside Harvey.
  2213. 1:38:11After 10,000 PRs,
  2214. 1:38:14the biggest lesson I can share is this.
  2215. 1:38:16The thing that broke was an agent.
  2216. 1:38:19It was everything around them.
  2217. 1:38:23So what does one submit it add up to?
  2218. 1:38:2610,000 PRs that an agent wrote.
  2219. 1:38:29And it isn't just the team playing with
  2220. 1:38:32a toy.
  2221. 1:38:34Last month, about 460 different people
  2222. 1:38:38at Harvey used it.
  2223. 1:38:40And over the summer, it stopped being
  2224. 1:38:42only about code.
  2225. 1:38:44People started asking a data question in
  2226. 1:38:46Slack and getting answers back.
  2227. 1:38:49My favorite
  2228. 1:38:51Recently, someone on our marketing team
  2229. 1:38:53dropped a prototype in the talk track
  2230. 1:38:55into Slack
  2231. 1:38:56and asked for a storyboard for a product
  2232. 1:38:58video.
  2233. 1:39:00Once the harness exists, people bring
  2234. 1:39:02the work that you never planned for.
  2235. 1:39:06So a promise upfront, this is not a talk
  2236. 1:39:09about models, evals, or leaderboards.
  2237. 1:39:12This is talk about harness,
  2238. 1:39:15odd things around the model.
  2239. 1:39:18Checking out the repo, holding the key,
  2240. 1:39:21opening the PR, getting the merge.
  2241. 1:39:24When we first wrote about Spectre back
  2242. 1:39:26in April,
  2243. 1:39:27we put it like this.
  2244. 1:39:29At some point, the harness become the
  2245. 1:39:31product.
  2246. 1:39:33Now, whenever I tell people this, I get
  2247. 1:39:35the same question.
  2248. 1:39:37Why on earth would you build this when
  2249. 1:39:39Cloud Code or Cursor already exists?
  2250. 1:39:43My answer
  2251. 1:39:44We don't compete with it. We run them.
  2252. 1:39:47And here's how I see it as my own view.
  2253. 1:39:51Agents on your laptop are powerful, but
  2254. 1:39:53they don't scale.
  2255. 1:39:54You can realistically run five to 10 of
  2256. 1:39:57them.
  2257. 1:39:58Plus that, you need the cloud.
  2258. 1:40:00And the cloud needs our secrets,
  2259. 1:40:02our real stack,
  2260. 1:40:04our alerts and tickets, one set of
  2261. 1:40:06rules, and enough capacity that nobody
  2262. 1:40:10gets starved.
  2263. 1:40:11No single tool owns that for us, so we
  2264. 1:40:13built layer, and that layer is exactly
  2265. 1:40:16what broke. So I want to tell you three
  2266. 1:40:19stories today.
  2267. 1:40:21How we stopped competing on writing
  2268. 1:40:22code,
  2269. 1:40:24how we let any agent drive
  2270. 1:40:26and what broke once they did.
  2271. 1:40:30The first story starts with the question
  2272. 1:40:31we keep hearing.
  2273. 1:40:34More of our engineers were using Cursor
  2274. 1:40:37and they ask,
  2275. 1:40:38how do I get our secret there?
  2276. 1:40:41How do I test against our real stack?
  2277. 1:40:43We could have rebuilt all that inside
  2278. 1:40:46Cursor and then again for next tool
  2279. 1:40:49and the one after that.
  2280. 1:40:51Instead, the answer became one sentence.
  2281. 1:40:54Just ask Cursor to use Specter.
  2282. 1:40:58And that's really the whole idea.
  2283. 1:41:00People will always have a favorite agent
  2284. 1:41:03and the favorite keeps changing.
  2285. 1:41:05So pick yours, Claude code, Codex,
  2286. 1:41:07Cursor, ChatGPT, dot, Slack, or your own
  2287. 1:41:09hands.
  2288. 1:41:11Start on your laptop, push the session
  2289. 1:41:13to the cloud, pull it back.
  2290. 1:41:16That's how Specter sits underneath all
  2291. 1:41:18of that.
  2292. 1:41:20Because once the code was written, the
  2293. 1:41:22real work starts.
  2294. 1:41:24Picture a PR stay.
  2295. 1:41:26Alert files in Slack at 2:00 in the
  2296. 1:41:28morning.
  2297. 1:41:30An agent picks up. Nobody type a thing.
  2298. 1:41:33It writes a fix, spins up real copy of
  2299. 1:41:36our stack,
  2300. 1:41:37and posts the proof on the PR.
  2301. 1:41:40When CI fails, the follow-up loop reads
  2302. 1:41:43the failure and posts another fix while
  2303. 1:41:45everyone's still asleep.
  2304. 1:41:48By morning, the PR sits in one review
  2305. 1:41:50queue under the same rule as other PR,
  2306. 1:41:54which our agent wrote it.
  2307. 1:41:57The owner gets a nudge
  2308. 1:41:59and only once they are working they
  2309. 1:42:00starts.
  2310. 1:42:02And once approved, a agent can auto
  2311. 1:42:04merge it.
  2312. 1:42:07If you only built one of those stages
  2313. 1:42:10that I described, build the follow-up
  2314. 1:42:12loop.
  2315. 1:42:13We switched it on for everyone in early
  2316. 1:42:15October, and that month four out of five
  2317. 1:42:18agent PRs merged into our backend PR
  2318. 1:42:22had been through it.
  2319. 1:42:24That's the difference between the agent
  2320. 1:42:26that write code
  2321. 1:42:27and an agent that finished the job.
  2322. 1:42:31And then there's the knowledge. Every
  2323. 1:42:33team has that one person who knows
  2324. 1:42:35exactly how to test the tricky service
  2325. 1:42:38which flag you never touch.
  2326. 1:42:40And that knowledge lives in their head
  2327. 1:42:43or their laptop.
  2328. 1:42:45So we gave it a home, projects.
  2329. 1:42:49People write down what matters.
  2330. 1:42:51Adrian write down what they learn.
  2331. 1:42:53Team pings the skills it trusts, and
  2332. 1:42:55there's a record of who ran what.
  2333. 1:42:59Any agent working in the project with
  2334. 1:43:00the same page,
  2335. 1:43:02my personal view,
  2336. 1:43:04whichever agent you pick, the audit
  2337. 1:43:05trail and the tracks should be the same.
  2338. 1:43:09The second story is about rule.
  2339. 1:43:13Back in May, if you want an inspector to
  2340. 1:43:15do something, you open a browser tab
  2341. 1:43:18or tagging Slack.
  2342. 1:43:20The whole month exactly one task came
  2343. 1:43:23out from terminal.
  2344. 1:43:25So we made a rule, if a person can click
  2345. 1:43:27it,
  2346. 1:43:28an agent can call it.
  2347. 1:43:30Start a task,
  2348. 1:43:32turn on the follow-up loop on auto
  2349. 1:43:34merge, publish a page off from CLI.
  2350. 1:43:39Which means your laptop agent can call
  2351. 1:43:41it.
  2352. 1:43:42Agent inside the sandbox can also call
  2353. 1:43:44it.
  2354. 1:43:45Even CI can call it too.
  2355. 1:43:47And then people ran with it.
  2356. 1:43:49By October, the command line was
  2357. 1:43:51starting about 6500 tasks a month.
  2358. 1:43:56The web app didn't shrink.
  2359. 1:43:58Everyone else just grew around it.
  2360. 1:44:01Task started from the terminal were
  2361. 1:44:03merging more PRs than task started in
  2362. 1:44:05browser.
  2363. 1:44:06And a new kind of user showed up.
  2364. 1:44:09People who start hundreds of tasks a
  2365. 1:44:11month and never open the app at all.
  2366. 1:44:14And this is where it gets fun.
  2367. 1:44:17In October, I had a big task.
  2368. 1:44:20So I call code session and ask to be the
  2369. 1:44:22coordinator.
  2370. 1:44:24It split the tickets into 10 different
  2371. 1:44:26milestones.
  2372. 1:44:28Get one for me and the rest to Specter
  2373. 1:44:31with hundreds of tasks.
  2374. 1:44:34Each has its own sandbox to validate end
  2375. 1:44:36to end.
  2376. 1:44:38Then at the end of the October, our IT
  2377. 1:44:40give everyone ChatGPT tasks.
  2378. 1:44:43My now watch Specter for me.
  2379. 1:44:45It lists the questions Specter is
  2380. 1:44:46waiting on, draft my answers, and once I
  2381. 1:44:50say yes, send them back.
  2382. 1:44:53I actually DM to my friends like that.
  2383. 1:44:56I just use cursor, clock code, or
  2384. 1:44:58whatever to spam Specter now.
  2385. 1:45:01Well, that's the dream, but it's also
  2386. 1:45:03the problem.
  2387. 1:45:05A quick detour.
  2388. 1:45:06When we look at where a task time
  2389. 1:45:08actually went, the slowest step wasn't
  2390. 1:45:11model syncing. It was Git.
  2391. 1:45:14Just checking out every repo we owned
  2392. 1:45:16took over a minute on average.
  2393. 1:45:20The fix, keep a copy of every repo on
  2394. 1:45:22every machine
  2395. 1:45:24and let each task borrow it.
  2396. 1:45:27The third story is the one where it
  2397. 1:45:30broke.
  2398. 1:45:31In mid August,
  2399. 1:45:33a release goes out with a bug.
  2400. 1:45:35That bug triggers alert that fires once
  2401. 1:45:38per workspace.
  2402. 1:45:39So one bug become hundreds of alerts
  2403. 1:45:42all landing in one single Slack channel.
  2404. 1:45:45Someone has set up workflow that tag our
  2405. 1:45:47agent on every alert
  2406. 1:45:49and every tag started task.
  2407. 1:45:52On normal day, those tasks started about
  2408. 1:45:5550 tasks.
  2409. 1:45:56That day, in about 4 hours, they started
  2410. 1:45:59around 900.
  2411. 1:46:01900 agents are chasing the same bug
  2412. 1:46:04until that wasn't room left for a single
  2413. 1:46:06node.
  2414. 1:46:08The fix was almost boring.
  2415. 1:46:10A A alert now joins the investigation
  2416. 1:46:12that's already open.
  2417. 1:46:15Two months later, it happened again.
  2418. 1:46:18And this time, it wasn't about
  2419. 1:46:21On October,
  2420. 1:46:22the engineer did exactly what we've been
  2421. 1:46:25asking everyone to do.
  2422. 1:46:27Hand work to Specter.
  2423. 1:46:30About 950 tasks from the command line.
  2424. 1:46:33In about 3 hours, nothing in our
  2425. 1:46:36platform can work.
  2426. 1:46:38The cluster filled up and everyone else
  2427. 1:46:41work sat in line behind one laptop.
  2428. 1:46:44That's not a user problem. That's our
  2429. 1:46:46problem.
  2430. 1:46:494 days later, we had a real fix, a fair
  2431. 1:46:52launch queue.
  2432. 1:46:54Now, starting a task gives you a place
  2433. 1:46:57in line.
  2434. 1:46:58And the dispatcher started when they're
  2435. 1:47:00through.
  2436. 1:47:01People sitting at keyboard goes first.
  2437. 1:47:04Everyone gets a fair share and can
  2438. 1:47:06borrow more than the cluster is quite.
  2439. 1:47:09And whenever you start a task, you can
  2440. 1:47:12see a place in line.
  2441. 1:47:13When we hit it with a 300 task bursting
  2442. 1:47:16low test,
  2443. 1:47:18the task list went from almost 2 seconds
  2444. 1:47:20to 70 milliseconds.
  2445. 1:47:22So did it work?
  2446. 1:47:25When it's the busiest day of every week
  2447. 1:47:28back in August, 2 sudden task and about
  2448. 1:47:311 in 10 didn't finish.
  2449. 1:47:35Then in November 4th, more than twice
  2450. 1:47:37that
  2451. 1:47:38and
  2452. 1:47:39much better in the finish line, but we
  2453. 1:47:42still have some room to grow.
  2454. 1:47:45I will be modest. We changed several
  2455. 1:47:46things at once, so I cannot tell you
  2456. 1:47:49which fix mattered the most.
  2457. 1:47:51But the burst kept getting bigger and
  2458. 1:47:53the platform kept up.
  2459. 1:47:56And just when that was under control,
  2460. 1:47:58the bottleneck moves.
  2461. 1:48:00At end of the October,
  2462. 1:48:02our dev platform team raised alarm.
  2463. 1:48:06PRs across the company had jumped around
  2464. 1:48:0960% in a single month.
  2465. 1:48:12A merge that used to take 15 to 30
  2466. 1:48:14minutes was taking over an hour now.
  2467. 1:48:17And merge queue kept having to be
  2468. 1:48:18switched off.
  2469. 1:48:20So now a PR rebased onto main, we run CI
  2470. 1:48:24and merge if it's green.
  2471. 1:48:27If main breaks, everyone CI breaks with
  2472. 1:48:29it.
  2473. 1:48:30Honestly,
  2474. 1:48:31tell you, we haven't cracked this one
  2475. 1:48:33yet.
  2476. 1:48:35But before I go, there's one more thing.
  2477. 1:48:38While we were busy fixing all of it,
  2478. 1:48:41something else was quietly growing.
  2479. 1:48:44Most of what I've been showing you start
  2480. 1:48:47with a person asking
  2481. 1:48:49the fastest growing cells of the work on
  2482. 1:48:52Specter doesn't.
  2483. 1:48:54These are automations, agent that wake
  2484. 1:48:57up on schedule on event and just do the
  2485. 1:49:00work.
  2486. 1:49:01Treasure the alert, read the PR, write
  2487. 1:49:04the post.
  2488. 1:49:06Last month, about 110 people had
  2489. 1:49:09automation running and we learned to
  2490. 1:49:11give them a guardrail on day one.
  2491. 1:49:14If one fails five times in a row,
  2492. 1:49:16it switches itself off.
  2493. 1:49:18And it can carry a budget or expiry date
  2494. 1:49:22because when no one's watching, an agent
  2495. 1:49:25can start thousands of tasks too.
  2496. 1:49:28So here's what I'm chasing next.
  2497. 1:49:31And I don't have an answer yet,
  2498. 1:49:32honestly.
  2499. 1:49:34What if every Slack channel had its own
  2500. 1:49:36agent that never locks off?
  2501. 1:49:39One that follows the condition and start
  2502. 1:49:41work on its own.
  2503. 1:49:43What if every PR proof it works before a
  2504. 1:49:45human even look at it?
  2505. 1:49:48Today, only one merge in 20 carries that
  2506. 1:49:51kind of proof.
  2507. 1:49:53And how do you merge when agents never
  2508. 1:49:56stop opening PRs?
  2509. 1:49:58Ask me again six months.
  2510. 1:50:00I will tell you what broke.
  2511. 1:50:03If you building something like this when
  2512. 1:50:04you work, here's three things I'd do
  2513. 1:50:06again.
  2514. 1:50:08Own what happens after the code is
  2515. 1:50:10written.
  2516. 1:50:11Make Eric action something an agent can
  2517. 1:50:13call.
  2518. 1:50:15And cap what an agent, human, or can
  2519. 1:50:18start before they find out what they can
  2520. 1:50:21do with it.
  2521. 1:50:23And one last thought.
  2522. 1:50:25That's my own view.
  2523. 1:50:27Engineers can write code with whichever
  2524. 1:50:29agent they like.
  2525. 1:50:31The work that's lost is the harness that
  2526. 1:50:33runs it,
  2527. 1:50:34checks it and get it merged.
  2528. 1:50:38Thank you so much for watching.
  2529. 1:50:41Come find me on LinkedIn if you are
  2530. 1:50:43building similar things.
  2531. 1:50:46>> Hi everyone. My name is Rajat Kotial. I
  2532. 1:50:48am part of the automated reasoning group
  2533. 1:50:50at AWS.
  2534. 1:50:52And today I'm going to talk about a
  2535. 1:50:54project I worked on recently called hero
  2536. 1:50:57specification analyzer.
  2537. 1:50:59And we use certain techniques which
  2538. 1:51:01involves formal methods and automated
  2539. 1:51:03reasoning to prove
  2540. 1:51:05specification that goes to your coding
  2541. 1:51:07agent before it writes code.
  2542. 1:51:16So let's take a certain bug, right?
  2543. 1:51:20This is a specification
  2544. 1:51:24snippet and there are two rules for
  2545. 1:51:26example and rule two says when an order
  2546. 1:51:29is submitted and this is for a a general
  2547. 1:51:32e-commerce website let's say.
  2548. 1:51:35When an order is submitted and the
  2549. 1:51:36inventory is available, we place the
  2550. 1:51:39order as a back order.
  2551. 1:51:41Which makes sense when you read it
  2552. 1:51:42individually. And there was also another
  2553. 1:51:44rule in the same specification file
  2554. 1:51:46which you hand to the coding agent which
  2555. 1:51:48says
  2556. 1:51:49no order shall be placed as a back order
  2557. 1:51:52and this is a rule we've set.
  2558. 1:51:54So individually read these two lines
  2559. 1:51:57make sense, right?
  2560. 1:51:58But in when an LLM reads this
  2561. 1:52:02reads both the rules together, both
  2562. 1:52:03cannot hold true at the same time.
  2563. 1:52:06So we need to make sure
  2564. 1:52:08the we resolve this ambiguity that the
  2565. 1:52:10LLM is
  2566. 1:52:12observing.
  2567. 1:52:14And
  2568. 1:52:17the scope and the reason
  2569. 1:52:20we
  2570. 1:52:22put an effort into this project is
  2571. 1:52:24based on public research and recent
  2572. 1:52:26research, we've observed there's a 20 to
  2573. 1:52:2840% drop with unclear prompts on the
  2574. 1:52:32accuracy the LLM
  2575. 1:52:34generates and the code accuracy, right?
  2576. 1:52:36And 60 to 90% of valid code is still
  2577. 1:52:39wrong after it writes it.
  2578. 1:52:41And also in that case, we observed two
  2579. 1:52:44times the more more regressions when we
  2580. 1:52:47deploy that code.
  2581. 1:52:51So where do bugs like these hide, right?
  2582. 1:52:54We want to detect these bugs before they
  2583. 1:52:56reach the codebase and the only way to
  2584. 1:52:59do that it's similar to how we write
  2585. 1:53:01design docs, similar to how we draw
  2586. 1:53:03diagrams and flow charts to make sure
  2587. 1:53:06the flow is set before we write any
  2588. 1:53:07code. We want to verify that without
  2589. 1:53:10a human or anyone reading these files as
  2590. 1:53:14the first reviewer.
  2591. 1:53:17In one sentence,
  2592. 1:53:19let's say the ambiguity and wording
  2593. 1:53:21wrong level of detail, but across
  2594. 1:53:24sentences, we can observe conflicts and
  2595. 1:53:26gaps in the specification or rules,
  2596. 1:53:28right?
  2597. 1:53:30Sentences need review, such need search
  2598. 1:53:32and between them and ambiguity searches.
  2599. 1:53:36So we have a three-step process to do
  2600. 1:53:39this where the LLM will first refine it,
  2601. 1:53:42it will try to clean up the wording on
  2602. 1:53:43its own
  2603. 1:53:44and if it succeeds, then well and good,
  2604. 1:53:48but if it cannot be then formalized
  2605. 1:53:50to
  2606. 1:53:52English language into SMT formulas.
  2607. 1:53:55So SMT lib is a certain set of type of
  2608. 1:53:59function which we pass to a SAT solver,
  2609. 1:54:02a satisfiability checker where it's
  2610. 1:54:04going to give out a Boolean saying this
  2611. 1:54:06expression that you've given me is
  2612. 1:54:08valid, true or false, invalid.
  2613. 1:54:13And then once the solver gets back to us
  2614. 1:54:15if that expression is valid, then we
  2615. 1:54:17don't do anything, but if it detects any
  2616. 1:54:19ambiguity between two sentences or
  2617. 1:54:21within a sentence where there's
  2618. 1:54:23two meanings to a sentence then we
  2619. 1:54:25detect that and
  2620. 1:54:26tell the LLM
  2621. 1:54:28to recheck that sentence and it's going
  2622. 1:54:30to reiterate on that.
  2623. 1:54:33For example the rules we saw
  2624. 1:54:37we've seen when order submitted and
  2625. 1:54:39inventory unavailable we shall place the
  2626. 1:54:41order as a back order and in SMT
  2627. 1:54:45format it kind of looks like this this
  2628. 1:54:47like a math formula right?
  2629. 1:54:54Once we go deeper we
  2630. 1:54:57once we detect this
  2631. 1:54:58we also do
  2632. 1:55:00a three-step process again where we
  2633. 1:55:02sample the translation again and again
  2634. 1:55:05we ask the LLM to translate seven times
  2635. 1:55:07or 10 times
  2636. 1:55:08and to understand whether this LLM is
  2637. 1:55:11behaving this way
  2638. 1:55:13based on
  2639. 1:55:15its its inherent capabilities or is
  2640. 1:55:17something affecting it to behave this
  2641. 1:55:18way
  2642. 1:55:19and then based on logical equivalence of
  2643. 1:55:21these seven or 10
  2644. 1:55:24different set of interpretations by the
  2645. 1:55:26LLM we cluster them on logical
  2646. 1:55:28equivalence which that which
  2647. 1:55:32entries came under which assumptions
  2648. 1:55:36that their LLM made for itself
  2649. 1:55:38and then we accept or ask the user to
  2650. 1:55:41clear out the ambiguity or abstain from
  2651. 1:55:44implementing that in the code base.
  2652. 1:55:51This is an example of asking the human
  2653. 1:55:55what you want to do with that
  2654. 1:55:59rule that was
  2655. 1:56:00perhaps contradicting.
  2656. 1:56:02Remove the record is very a very vague
  2657. 1:56:04statement the LLM cannot can interpret
  2658. 1:56:07anything it wants but we shouldn't allow
  2659. 1:56:09that to happen and we either give two
  2660. 1:56:12options we keep it as is the record is
  2661. 1:56:14deleted or we retain it and but hide it
  2662. 1:56:17from the users which one does the user
  2663. 1:56:19prefer?
  2664. 1:56:25A conflict is when the examples we
  2665. 1:56:27example we saw rule one is conflicting
  2666. 1:56:29with rule three.
  2667. 1:56:31And a gap is where no rule covers an
  2668. 1:56:33idle order
  2669. 1:56:35in this example
  2670. 1:56:37where the LLM detected an actual sorry
  2671. 1:56:40our
  2672. 1:56:41process or pipeline detected there's no
  2673. 1:56:44entry for this rule where you are
  2674. 1:56:47missing something.
  2675. 1:56:52All in all
  2676. 1:56:53it cannot do a lot of things end of the
  2677. 1:56:55day.
  2678. 1:56:56We want to
  2679. 1:56:57make sure there is no oracle
  2680. 1:57:00for for specification correctness other
  2681. 1:57:02than the user.
  2682. 1:57:03User needs to make sure to check every
  2683. 1:57:07part of the snippet part of the rule
  2684. 1:57:09book before sending it to the LLM.
  2685. 1:57:11And
  2686. 1:57:13we as a tool only can help them.
  2687. 1:57:17And the user is more important here
  2688. 1:57:19because
  2689. 1:57:20of implicit domain knowledge the LLM
  2690. 1:57:22doesn't know a lot of domain knowledge
  2691. 1:57:24and there's no guarantee from an LLM
  2692. 1:57:26only
  2693. 1:57:27user can
  2694. 1:57:29give that guarantee right and there's
  2695. 1:57:32been evidence right now for
  2696. 1:57:33formalization and how it can be used in
  2697. 1:57:35agents we are emerging in this
  2698. 1:57:37technology.
  2699. 1:57:43What do we take back from this
  2700. 1:57:45talk?
  2701. 1:57:46Check the spec always before you pass it
  2702. 1:57:48down to the agent.
  2703. 1:57:50Cluster the translation we what we do is
  2704. 1:57:52we cluster the translation by
  2705. 1:57:55logical equivalent to determine
  2706. 1:57:58the different LLM interpretations and
  2707. 1:58:00where it's genuinely going wrong.
  2708. 1:58:02The LLM translates the solver checks and
  2709. 1:58:05the human decides.
  2710. 1:58:07We give to give the human options and if
  2711. 1:58:10there's no answer to that we use LLM
  2712. 1:58:12judges or abstain from implementing that
  2713. 1:58:15piece.
  2714. 1:58:18Here are the references and this work
  2715. 1:58:21has been majorly contributed by my team
  2716. 1:58:28you can check out their and the
  2717. 1:58:30website link provided.
  2718. 1:58:32And delve deeper into that. Thank you.
  2719. 1:58:35>> Hi, I'm Chinmay and with me are Swapnika
  2720. 1:58:38and Vlad.
  2721. 1:58:39We're excited to share our journey as
  2722. 1:58:41well as our learnings around building an
  2723. 1:58:43AI on-call teammate at Meta
  2724. 1:58:46that now runs real on-call rotations
  2725. 1:58:48every day and around the clock.
  2726. 1:58:51We are part of the engineering team that
  2727. 1:58:53builds the tooling platform and
  2728. 1:58:55infrastructure for incident response,
  2729. 1:58:58on-call experience, and mitigations at
  2730. 1:59:00Meta.
  2731. 1:59:01That space is evolving rapidly with the
  2732. 1:59:03increase in AI capabilities.
  2733. 1:59:07AI changed how fast we write code.
  2734. 1:59:10About three quarters of new code is now
  2735. 1:59:11written with AI
  2736. 1:59:13and more AI-generated changes reach
  2737. 1:59:16production every single day.
  2738. 1:59:19And some of it
  2739. 1:59:20can cause production failures just like
  2740. 1:59:22any other code change can.
  2741. 1:59:25But on-call did not speed up with it.
  2742. 1:59:28High-quality code can pass review and
  2743. 1:59:31still break in production. And when it
  2744. 1:59:33does, someone still gets paged at 2:00
  2745. 1:59:35a.m.
  2746. 1:59:36The bottleneck moved from writing code
  2747. 1:59:39to operating it in production.
  2748. 1:59:41And on-call is where that operational
  2749. 1:59:44load lands. So we asked
  2750. 1:59:46if AI is writing the code,
  2751. 1:59:49can AI also help operate it?
  2752. 1:59:53And we think it can.
  2753. 1:59:56Now let's quickly run through one of
  2754. 1:59:57those on-call pages with a simplified
  2755. 2:00:00example.
  2756. 2:00:02It's 2:00 a.m. and 12 alerts are firing.
  2757. 2:00:05Same night, it can go two different
  2758. 2:00:07ways. On the left is how most of us do
  2759. 2:00:09this today. And notice we already have
  2760. 2:00:12good AI.
  2761. 2:00:13First you sort the alerts by yourself,
  2762. 2:00:16check out payments disk, which one to
  2763. 2:00:18pick first. Let's say you pick checkout.
  2764. 2:00:20Then you ask your assistant why P99
  2765. 2:00:23latency is high.
  2766. 2:00:25It's genuinely useful. It pulls the
  2767. 2:00:27metrics and finds the deployment at
  2768. 2:00:291:49.
  2769. 2:00:31And when you ask, it reads what changed
  2770. 2:00:34and checks the runbook.
  2771. 2:00:36It figured out that the time config was
  2772. 2:00:38changed to a lower value
  2773. 2:00:41and based on the runbook, it suggests to
  2774. 2:00:43rollback, which is all very helpful.
  2775. 2:00:46However, if you look at the shape of it,
  2776. 2:00:48it waited to be asked.
  2777. 2:00:51You picked every next step.
  2778. 2:00:53You were the glue between the questions.
  2779. 2:00:57And once it answered, it went dormant.
  2780. 2:01:00At 2:39, you know the cause,
  2781. 2:01:04but the rollback, the changes, and the
  2782. 2:01:06follow-ups are still yours.
  2783. 2:01:10Now, on the right,
  2784. 2:01:12the same 12 alerts,
  2785. 2:01:14but an AI teammate
  2786. 2:01:16started before you woke up.
  2787. 2:01:19It had already ranked them.
  2788. 2:01:21Checkouts first, payments merged in as
  2789. 2:01:24the same cause, and the other 10 handled
  2790. 2:01:26separately.
  2791. 2:01:28It read the dashboards, diffed the same
  2792. 2:01:30149 AM config change, and matched two
  2793. 2:01:33past incidents.
  2794. 2:01:35You get a hypothesis,
  2795. 2:01:37a 30-second way to verify it,
  2796. 2:01:40and a fix
  2797. 2:01:41waiting for your review.
  2798. 2:01:43At 2:01, you verified it, and you're
  2799. 2:01:46back to sleep.
  2800. 2:01:47That gap is what this talk is about.
  2801. 2:01:53So, why were you the glue?
  2802. 2:01:56Every team was bolting automation onto
  2803. 2:01:58on-call,
  2804. 2:02:00one silo at a time.
  2805. 2:02:02An alert script for alerts, a triage bot
  2806. 2:02:05for tickets, a deploy bot for deploys, a
  2807. 2:02:08fix-it agent for incidents.
  2808. 2:02:10Each one solves its slice,
  2809. 2:02:12keeps its own context, and forgets it.
  2810. 2:02:16Now, replaying your 2 AM page,
  2811. 2:02:18the alert script saw P99 latency about 2
  2812. 2:02:21seconds.
  2813. 2:02:22The triage bot saw three tickets saying
  2814. 2:02:24checkout is slow.
  2815. 2:02:26The deploy bot knew about a config
  2816. 2:02:28change at 149, and the incident history
  2817. 2:02:30had two similar past incidents.
  2818. 2:02:33Every clue existed,
  2819. 2:02:35but nothing connected them.
  2820. 2:02:38So the on-call became the glue exactly
  2821. 2:02:42as we saw on the left side of the
  2822. 2:02:44previous slide.
  2823. 2:02:46Our bet was one AI teammate per rotation
  2824. 2:02:50instead.
  2825. 2:02:52One loop across every signal,
  2826. 2:02:55one team memory that learns every shift,
  2827. 2:02:57its own identity, so it acts as itself
  2828. 2:03:00and a human gate. So you approve instead
  2829. 2:03:03of operate.
  2830. 2:03:05But giving one agent all that context
  2831. 2:03:07and reach raises the stakes,
  2832. 2:03:09which is why autonomy has to be earned.
  2833. 2:03:15Under the hood,
  2834. 2:03:16the teammate or the AI teammate is the
  2835. 2:03:18simplest agent there is.
  2836. 2:03:21Model using tools in a loop,
  2837. 2:03:23acting on an environment and reading the
  2838. 2:03:26feedback.
  2839. 2:03:27Everything else in this talk is what we
  2840. 2:03:29[music] built around that loop
  2841. 2:03:31to make it safe to leave it running.
  2842. 2:03:34And for that, I will hand over to
  2843. 2:03:36Swapnika and Vlad next to get into the
  2844. 2:03:39actual details of how it works.
  2845. 2:03:43>> To achieve this, we started with an
  2846. 2:03:45autonomy ladder.
  2847. 2:03:46Think about how you onboard a new
  2848. 2:03:48engineer to on-call.
  2849. 2:03:49First, the shadow.
  2850. 2:03:51They investigate and suggest, but they
  2851. 2:03:53don't touch production.
  2852. 2:03:55Then the pair.
  2853. 2:03:57They write the fix and another engineer
  2854. 2:03:59approves it. With time, they gain
  2855. 2:04:01knowledge.
  2856. 2:04:03We give our agent exactly the same
  2857. 2:04:05ladder.
  2858. 2:04:06At level one, it investigates and tells
  2859. 2:04:08you what it found. It changes nothing.
  2860. 2:04:11At level two, it acts with approval.
  2861. 2:04:13It drafts the fix, but nothing reaches
  2862. 2:04:15production until a human approves.
  2863. 2:04:18At level three,
  2864. 2:04:19it clears lower criticality work on its
  2865. 2:04:22own and only interrupts you for what
  2866. 2:04:24matters.
  2867. 2:04:25High criticality work stays with humans.
  2868. 2:04:29So what earns the next run?
  2869. 2:04:32We use a simple test. How costly is a
  2870. 2:04:34mistake and how easy is it to catch?
  2871. 2:04:37So an action claims only when mistakes
  2872. 2:04:40are cheap, they are caught by tests, CI
  2873. 2:04:43or review
  2874. 2:04:45and the agent already has a track record
  2875. 2:04:47on that exact action.
  2876. 2:04:49That's why the agent can close a noisy
  2877. 2:04:52alert on its own while a production
  2878. 2:04:54mitigation still waits for a human.
  2879. 2:04:58At level one, the investigation is
  2880. 2:05:00layered and stops at the first conflict
  2881. 2:05:02answer.
  2882. 2:05:03The agent investigated the issue using
  2883. 2:05:05the team's memory and runbook. It also
  2884. 2:05:08shared the root cause analysis,
  2885. 2:05:10but human had to apply the fix.
  2886. 2:05:13At level two,
  2887. 2:05:15it writes the fix with the root cause
  2888. 2:05:17and a test plan.
  2889. 2:05:19It answers reviewer's comments and fixes
  2890. 2:05:21CI,
  2891. 2:05:23but nothing reaches production until a
  2892. 2:05:24human approves.
  2893. 2:05:26Most drafts don't land and that's fine.
  2894. 2:05:30Review is the filter.
  2895. 2:05:32At both levels, the agent does the
  2896. 2:05:34homework,
  2897. 2:05:35but you still had to grade every single
  2898. 2:05:38paper.
  2899. 2:05:39They have got much better,
  2900. 2:05:41but the tasks requiring human input
  2901. 2:05:43didn't get smaller.
  2902. 2:05:45Level three shrinks the pile.
  2903. 2:05:50Look at this example.
  2904. 2:05:5123 items came in, but only three need
  2905. 2:05:55you.
  2906. 2:05:56On the right,
  2907. 2:05:57the agent handled the long tail, noisy
  2908. 2:05:59alerts, duplicates, routine fixes that
  2909. 2:06:02went through normal review and CI.
  2910. 2:06:05Two more are still being worked.
  2911. 2:06:08You don't have to drive each one to
  2912. 2:06:09completion.
  2913. 2:06:11In the middle is your much smaller
  2914. 2:06:13decision queue.
  2915. 2:06:14Each item makes clear what it needs from
  2916. 2:06:17you with a run this button that has a
  2917. 2:06:20call to action.
  2918. 2:06:21And remember our 2:00 a.m. checkout
  2919. 2:06:23page,
  2920. 2:06:24that still stays human own.
  2921. 2:06:26The agent brings the investigation, the
  2922. 2:06:28human owns the critical decision.
  2923. 2:06:31It's the new high ladder again.
  2924. 2:06:33Trusted to handle routine work, ready
  2925. 2:06:36with the homework when it needs help,
  2926. 2:06:38not just less work per item,
  2927. 2:06:40but fewer items waiting on you.
  2928. 2:06:45That board is where the work lives, but
  2929. 2:06:47nobody stares at a board all the time,
  2930. 2:06:49right?
  2931. 2:06:50So the agent is also available across
  2932. 2:06:53many surfaces like chat and native
  2933. 2:06:55incident surfaces.
  2934. 2:06:57At 9:00 every morning,
  2935. 2:06:59it posts a digest.
  2936. 2:07:0123 came in,
  2937. 2:07:02three need you, 18 handled.
  2938. 2:07:05Each thing that needs you come with a
  2939. 2:07:08one click next step.
  2940. 2:07:10So approving the quota bump
  2941. 2:07:12takes one tap from your phone.
  2942. 2:07:14The 18 at handled are summarized, not
  2943. 2:07:17listed.
  2944. 2:07:18And the two that are still in progress
  2945. 2:07:20tell you what it's doing about them.
  2946. 2:07:22And when you want to dig in, you just
  2947. 2:07:25reply in the thread.
  2948. 2:07:27As you see on the right,
  2949. 2:07:28here you open the checkout incident and
  2950. 2:07:30ask for the evidence.
  2951. 2:07:32It already has it.
  2952. 2:07:34Errors start at 8:41.
  2953. 2:07:36Two minutes after a config push, cut the
  2954. 2:07:38payment client timeout.
  2955. 2:07:40The same pattern as two past incidents.
  2956. 2:07:43You ask a follow-up, is it only EU? And
  2957. 2:07:45it answers with a breakdown.
  2958. 2:07:47It drafts the role back. You approve it
  2959. 2:07:50right there and errors start recovering.
  2960. 2:07:53You never opened a dashboard and you
  2961. 2:07:55never had to explain the context
  2962. 2:07:57because it was already there.
  2963. 2:07:58That's what a teammate feels like. A
  2964. 2:08:00morning handoff a colleague you can ask.
  2965. 2:08:06Now
  2966. 2:08:07another hood of this AI teammate it's AI
  2967. 2:08:09automations.
  2968. 2:08:11The first investigates.
  2969. 2:08:13On every new signal, auto triage
  2970. 2:08:16classifies it,
  2971. 2:08:17checks its criticality and kicks off the
  2972. 2:08:19right agent.
  2973. 2:08:21An alert goes to the incident
  2974. 2:08:23investigator,
  2975. 2:08:24a task to the team's own agent,
  2976. 2:08:26a test failure to a coding agent,
  2977. 2:08:29a lookup to memory and code search.
  2978. 2:08:32The second
  2979. 2:08:34keeps the system healthy.
  2980. 2:08:35Unattended automation fills quietly.
  2981. 2:08:39So So a schedule, it looks stuck or
  2982. 2:08:41failed runs,
  2983. 2:08:42retrace and repairs broken schedules,
  2984. 2:08:45and it only pages a human if it can't
  2985. 2:08:48fix it itself.
  2986. 2:08:50Nothing stalls silently.
  2987. 2:08:53The third, hands off.
  2988. 2:08:55At every shift change and every morning
  2989. 2:08:57at 9:00, it summarizes the shift.
  2990. 2:08:59What needs you with the fix ready and
  2991. 2:09:02what it handled on its own posted to the
  2992. 2:09:04team chat.
  2993. 2:09:05Not only in the morning, but every
  2994. 2:09:07urgent issue that needs user attention
  2995. 2:09:10gets notified in chat with findings and
  2996. 2:09:12draft fixes.
  2997. 2:09:14It also tags the next on call in the
  2998. 2:09:15chat during handoff. The entire cycle
  2999. 2:09:18still continues.
  3000. 2:09:23And because it belongs to the team, it
  3001. 2:09:25runs the agents and the skills the team
  3002. 2:09:28already trusts.
  3003. 2:09:29That's the slot in the middle.
  3004. 2:09:31Bring your own agent, add your own
  3005. 2:09:33skills.
  3006. 2:09:35If a team already has an agent that has
  3007. 2:09:37the team's runbooks and the knowledge
  3008. 2:09:38base, the automations leverage it.
  3009. 2:09:41Alongside that, it has our own incident
  3010. 2:09:43investigator and a deep dive coding
  3011. 2:09:46session.
  3012. 2:09:48Work comes in from alerts, tickets,
  3013. 2:09:51incidents,
  3014. 2:09:52team posts and chat.
  3015. 2:09:54The agent itself is generic. What makes
  3016. 2:09:57it good at an on call rotation is the
  3017. 2:09:59team's agents, skills, memory and
  3018. 2:10:02playbook.
  3019. 2:10:04And every run is logged, so anyone on
  3020. 2:10:06the team can inspect the trajectory or
  3021. 2:10:08resume the conversation.
  3022. 2:10:11Now I'm handing off to my teammate Vlad
  3023. 2:10:13to talk more on the shared agent.
  3024. 2:10:19>> An on call agent can belong to just a
  3025. 2:10:21single person because the on call itself
  3026. 2:10:23doesn't.
  3027. 2:10:24So it belongs to the team.
  3028. 2:10:26There's one teammate per rotation on one
  3029. 2:10:29board
  3030. 2:10:30where every on call can ask it to
  3031. 2:10:31correct it or pick up its work mid
  3032. 2:10:33investigation.
  3033. 2:10:34It carries the team's on call expertise,
  3034. 2:10:37the playbook, alert profiles and past
  3035. 2:10:39decisions from one shift to the next
  3036. 2:10:41one, the the a good handoff should.
  3037. 2:10:44Here's what that looks like.
  3038. 2:10:46Monday at 2:00 a.m. same as on call and
  3039. 2:10:48tells it. Remember, if checkout times
  3040. 2:10:51out after a config push, roll back the
  3041. 2:10:53config first.
  3042. 2:10:55It saves it to the memory.
  3043. 2:10:573 days later, Jordan is on call hits the
  3044. 2:11:00exact same alert and just asks for a
  3045. 2:11:03fix.
  3046. 2:11:04The agent answers with same fix,
  3047. 2:11:06says where it came from and has the
  3048. 2:11:08rollback drafted.
  3049. 2:11:10Teach it once, the next on call gets it.
  3050. 2:11:14And it acts as the team.
  3051. 2:11:16It's on a data day owned by the rotation
  3052. 2:11:18with only the access the team is
  3053. 2:11:20comfortable granting it.
  3054. 2:11:23And it learns from every on call shift.
  3055. 2:11:26Handoffs, decisions and corrections all
  3056. 2:11:29flow into memory. So shift number 30 is
  3057. 2:11:32better than the shift number one.
  3058. 2:11:34The same way a new on call gets better
  3059. 2:11:35every week.
  3060. 2:11:37But memory erodes over time, so ours has
  3061. 2:11:40to earn its place.
  3062. 2:11:42It retrieves semantically, uses what it
  3063. 2:11:45finds, scores what it actually helped
  3064. 2:11:47and promotes
  3065. 2:11:49or retires entries automatically.
  3066. 2:11:52When someone says that was wrong,
  3067. 2:11:54the item reopens and the correction
  3068. 2:11:56becomes memory.
  3069. 2:11:59Three things made this work be on our
  3070. 2:12:01team.
  3071. 2:12:02First,
  3072. 2:12:03a loop that never stops listening. Every
  3073. 2:12:05piece of feedback or course correction
  3074. 2:12:08reopens the item and ultimately gets
  3075. 2:12:10absorbed into memory.
  3076. 2:12:12Feedback is an input to the system, not
  3077. 2:12:14a sorry read once a quarter.
  3078. 2:12:17Second,
  3079. 2:12:18customize instead of forking.
  3080. 2:12:21Teams bring their own playbooks and
  3081. 2:12:23scale workflows
  3082. 2:12:25and it's still the same teammate
  3083. 2:12:26underneath.
  3084. 2:12:28Third, what we call one rotation to end.
  3085. 2:12:32Identity, config and memory are per
  3086. 2:12:34team, so at a rotation becomes a setup
  3087. 2:12:37not a new project.
  3088. 2:12:39The agent is generic while the expertise
  3089. 2:12:41is owned by the team.
  3090. 2:12:45So what has changed for the people on
  3091. 2:12:47call?
  3092. 2:12:48Well, first, their job shifted from
  3093. 2:12:50driving deep investigations to making
  3094. 2:12:52key decisions.
  3095. 2:12:53They start from a hypothesis with linked
  3096. 2:12:55evidence, not a blank task or a
  3097. 2:12:57dashboard.
  3098. 2:12:59Second, the load is getting lighter.
  3099. 2:13:02Early signs of on-call load reduction
  3100. 2:13:04are promising for teams that run it
  3101. 2:13:06automatic autonomously, and we want to
  3102. 2:13:08measure it rigorously before we put a
  3103. 2:13:10number on it.
  3104. 2:13:12And the third, every change that agent
  3105. 2:13:14writes
  3106. 2:13:15still meets a human,
  3107. 2:13:17fixes land only through normal review
  3108. 2:13:20and continuous integration.
  3109. 2:13:21And most drafts don't land.
  3110. 2:13:24That's the point. Review is the filter.
  3111. 2:13:27On-call stops being attacks and becomes
  3112. 2:13:29a review queue.
  3113. 2:13:32Here's what we're focusing on next.
  3114. 2:13:35First, mitigations that get approved,
  3115. 2:13:37not just drafted.
  3116. 2:13:38We grade every mitigation that agent
  3117. 2:13:40proposes and feed the outcome back
  3118. 2:13:42so it earns its next level instead of
  3119. 2:13:44being handed it.
  3120. 2:13:46Second,
  3121. 2:13:47skills are written from the chat so a
  3122. 2:13:49team can turn its runbooks into skills
  3123. 2:13:51without writing any code.
  3124. 2:13:53And third, measuring the time we give
  3125. 2:13:55back properly with a rigorous study of
  3126. 2:13:58on-call time saved rather than usage
  3127. 2:14:00counts.
  3128. 2:14:01And the next rung on the ladder is
  3129. 2:14:04earning medium criticality autonomy with
  3130. 2:14:07evidence.
  3131. 2:14:08On that, thank you so much for listening
  3132. 2:14:10to us and hopefully that was helpful.
  3133. 2:14:13Bye.
  3134. 2:15:42>> This
  3135. 2:19:47>> Yeah.
  3136. 2:21:16>> harness in the less code we need. And if
  3137. 2:21:19we look at what we have been doing at
  3138. 2:21:21Google DeepMind for the last six months.
  3139. 2:21:23So at May we had Google I/O
  3140. 2:21:25I/O and today it's November 11 and we
  3141. 2:21:28have been shipping relentlessly
  3142. 2:21:30basically. Just earlier today we
  3143. 2:21:32announced our first decision model with
  3144. 2:21:35Gemini decision one. Yesterday we made
  3145. 2:21:38Gemini four carbon available and then if
  3146. 2:21:40we just look at the last six months we
  3147. 2:21:41basically ship now models every few
  3148. 2:21:43weeks.
  3149. 2:21:44As already said Gemini carbon is now
  3150. 2:21:46available for you. So if you are a
  3151. 2:21:48Google AI Ultra subscriber you can use
  3152. 2:21:50it in Anti-Gravity, you can use it in
  3153. 2:21:52the Gemini app, you can use it in AI
  3154. 2:21:53Studio.
  3155. 2:21:54Carbon is number one on the cursor bench
  3156. 2:21:57eval and also number one on the
  3157. 2:21:59perplexity benchmark and internally at
  3158. 2:22:01Google we have been using it already a
  3159. 2:22:03lot.
  3160. 2:22:04And much more exciting is that I'm happy
  3161. 2:22:07to share that we launched the Gemini
  3162. 2:22:08Decisions API roughly one hour ago with
  3163. 2:22:11our new Gemini decision one model. So
  3164. 2:22:14it's a similar model to chef. You can
  3165. 2:22:16provide all of the existing Gemini
  3166. 2:22:18inputs meaning it's multimodal so you
  3167. 2:22:21can provide text, audio, images, video
  3168. 2:22:24and also PDFs natively and then provide
  3169. 2:22:26up to 1,000 questions in a single
  3170. 2:22:28request. The average end-to-end latency
  3171. 2:22:31including all of the network overhead is
  3172. 2:22:33between 150 to 250 milliseconds and it
  3173. 2:22:36costs 10 cents per 1 million tokens. So
  3174. 2:22:40how do models get good at agentic work?
  3175. 2:22:42And that's a rough
  3176. 2:22:44overview on how post training works. So
  3177. 2:22:46we have a task, we have our model, the
  3178. 2:22:49model has a certain set of tools and
  3179. 2:22:51then we have some kind of compute
  3180. 2:22:52environment and at the end we have some
  3181. 2:22:54verifier and the model tries to
  3182. 2:22:55basically solve that task to use
  3183. 2:22:57different tools and then based on the
  3184. 2:22:59score from the verifier we update the
  3185. 2:23:01model but no one is telling the model
  3186. 2:23:03which tools to use and basically it only
  3187. 2:23:06receives that reward feedback.
  3188. 2:23:09So if we look at two examples on how a
  3189. 2:23:11model can do a certain tasks with tools
  3190. 2:23:14we have the path A where we have
  3191. 2:23:16dedicated tools for file system control
  3192. 2:23:18so you all have probably used coding
  3193. 2:23:21agents and you have seen a read file
  3194. 2:23:23call or an edit file call so that's one
  3195. 2:23:26way of doing it and the other way of
  3196. 2:23:28doing it would be just using a single
  3197. 2:23:30inline bash script for all of the
  3198. 2:23:31changes so on the left side we roughly
  3199. 2:23:34need around 10 turns to make the same
  3200. 2:23:36changes to a coding
  3201. 2:23:38repository compared to the two change
  3202. 2:23:40turns on the right side just using bash
  3203. 2:23:43and it also reflects in the amount of
  3204. 2:23:45tokens we need so the left path takes
  3205. 2:23:48around 10x more tokens than the life the
  3206. 2:23:51right path and it's doing the exact same
  3207. 2:23:53thing
  3208. 2:23:54at the end for the model to achieve that
  3209. 2:23:56task. So in my experiment every roughly
  3210. 2:24:00six months I try to build a new coding
  3211. 2:24:02agent harness from scratch and I always
  3212. 2:24:04ask myself okay how much of the harness
  3213. 2:24:06do I really need to achieve the same
  3214. 2:24:08task and what is overhead to the model
  3215. 2:24:11and we no longer need. So if we look at
  3216. 2:24:13the coding agent harness which
  3217. 2:24:17from 2025 we had a roughly around 10 to
  3218. 2:24:2015 to 20 tools we had dedicated tools
  3219. 2:24:23for
  3220. 2:24:24file editing we had dedicated tools for
  3221. 2:24:26planning for to-do lists and also we had
  3222. 2:24:29a lot of dedicated rules in our system
  3223. 2:24:32prompt which told the model what to do
  3224. 2:24:34and what not to do and that worked of
  3225. 2:24:37course but since then we made a lot of
  3226. 2:24:38changes so in my experiments
  3227. 2:24:41first one was done roughly now two years
  3228. 2:24:43ago in February
  3229. 2:24:46where we did not really had agents right
  3230. 2:24:47we had workflows we basically had
  3231. 2:24:49predefined steps rock applications back
  3232. 2:24:51then were very popular we had the
  3233. 2:24:53planner we had search we reflected on
  3234. 2:24:54whatever was found and then we generated
  3235. 2:24:56an answer. Then we started to see our
  3236. 2:24:58first agents we had many many different
  3237. 2:25:01tools for all sorts of fine kernel
  3238. 2:25:02actions.
  3239. 2:25:04And then came the coding agents where we
  3240. 2:25:06had those roughly ten to 20 tools.
  3241. 2:25:09And now in my last experiment I tried to
  3242. 2:25:11see, okay, what's going to happen if I
  3243. 2:25:13provide only one tool to my agent?
  3244. 2:25:16And the outcome is was pretty exciting
  3245. 2:25:18to me. So this one tool agent works the
  3246. 2:25:21same way as every other agent except
  3247. 2:25:23that it only has one tool, which is
  3248. 2:25:25Bash.
  3249. 2:25:26And the model can use that tool for any
  3250. 2:25:28sorts of task. And the Bash tool is very
  3251. 2:25:31simple, so it runs in a separate clean
  3252. 2:25:33subshell. And it supports the command
  3253. 2:25:36argument. I also added a description so
  3254. 2:25:38that it's nice and easier to display to
  3255. 2:25:40the user what the model is actually
  3256. 2:25:42doing. And then like some async
  3257. 2:25:43parameters. So basically when the model
  3258. 2:25:44runs a Bash command,
  3259. 2:25:47the parameter tells the model, okay, if
  3260. 2:25:49it is the command longer running than
  3261. 2:25:51this milliseconds, it's being
  3262. 2:25:54that the tool call resolves. And the
  3263. 2:25:56model receives a local file handle where
  3264. 2:25:59it can look up later, okay, if the
  3265. 2:26:00process is still running. That's very
  3266. 2:26:02interesting if you do like all sorts of
  3267. 2:26:04like web development, for example. And
  3268. 2:26:07the single Bash tool actually works for
  3269. 2:26:09all of the things we are using agent. So
  3270. 2:26:10we can have long running work where we
  3271. 2:26:12can start on the foreground and then
  3272. 2:26:14automatically move it to the background.
  3273. 2:26:16The model receives some file handle and
  3274. 2:26:19since it has Bash available, if we store
  3275. 2:26:21it to a file system, it can read the
  3276. 2:26:23logs which are being streamed. If the
  3277. 2:26:26agent itself is a CLI, it can call
  3278. 2:26:28itself using Bash. So there is
  3279. 2:26:30practically no need to create sub agent
  3280. 2:26:33tools for it because it can invoke
  3281. 2:26:35itself. And then of course we don't need
  3282. 2:26:37dedicated planning or memory tools
  3283. 2:26:39because we have our file system
  3284. 2:26:40available. We can provide some context
  3285. 2:26:43to the model so the model can keep track
  3286. 2:26:44of its own work with the file system and
  3287. 2:26:47using the Bash tools.
  3288. 2:26:48And in addition, of course, to our
  3289. 2:26:50tools, we have system prompts. And
  3290. 2:26:52similar to tools, system prompts evolved
  3291. 2:26:55over time.
  3292. 2:26:57Every dot here is roughly around 100
  3293. 2:26:59tokens and you can see that coding
  3294. 2:27:01agents mid 2025 had
  3295. 2:27:04a lot of
  3296. 2:27:05context needed to send in every request.
  3297. 2:27:08Of course this also improved and similar
  3298. 2:27:10story here is like okay, what do I
  3299. 2:27:12really need to provide to my agent to
  3300. 2:27:14make it successful and work and what of
  3301. 2:27:17all of the system information and
  3302. 2:27:19details we had in the past are now
  3303. 2:27:20basically
  3304. 2:27:22part of the model.
  3305. 2:27:23So
  3306. 2:27:24how much of the harness helps and how
  3307. 2:27:26much is in their way?
  3308. 2:27:28And to find that out I was running a lot
  3309. 2:27:30of different evaluations of a lot of
  3310. 2:27:32different agent harnesses with a lot of
  3311. 2:27:34different tools. And one example I like
  3312. 2:27:36to show here which was done with Gemini
  3313. 2:27:384 Carbon from an eval task from DeepSVE
  3314. 2:27:41is like okay, what happens if I provide
  3315. 2:27:44my existing coding agent tools with
  3316. 2:27:47read, write, search, grep
  3317. 2:27:49and my bash only tool? And I think you
  3318. 2:27:51can pretty clearly see that the top line
  3319. 2:27:54is taking a lot of more tokens, a lot of
  3320. 2:27:57more turns and a lot of more times to
  3321. 2:27:59achieve the same task. And what stood
  3322. 2:28:02out really well is that okay, reading is
  3323. 2:28:05done in parallel in the beginning, but
  3324. 2:28:07as soon as we start going into making
  3325. 2:28:09code changes we see that same behavior
  3326. 2:28:11that the model is making turn by turn a
  3327. 2:28:15single edit which of course consumes a
  3328. 2:28:17lot of time and a lot of tokens.
  3329. 2:28:20So
  3330. 2:28:21how that does this look at Prodoscope?
  3331. 2:28:24So I have my own eval set of around 350
  3332. 2:28:27tasks for coding, knowledge work and
  3333. 2:28:30terminal commands
  3334. 2:28:32all run with Gemini 4 Argon here and
  3335. 2:28:35four different agent harnesses. One is
  3336. 2:28:37the bash only harness and then okay,
  3337. 2:28:40with bash we still have that issue on
  3338. 2:28:43how can I read images, right? If I'm I
  3339. 2:28:45cannot cut screenshot.png it would
  3340. 2:28:48return a binary and it will fill my
  3341. 2:28:50context and the model cannot see the
  3342. 2:28:52image. So I added a view media tool
  3343. 2:28:54which basically returns the image in a
  3344. 2:28:56way the model can see.
  3345. 2:28:58Then we have
  3346. 2:28:59two different agent harnesses with
  3347. 2:29:01structure tools for file system
  3348. 2:29:03operations. And if we just look at the
  3349. 2:29:06score
  3350. 2:29:07all of them look roughly identical,
  3351. 2:29:10right? So if we would just evaluate our
  3352. 2:29:12model in an agent harness looking at the
  3353. 2:29:15score, it would
  3354. 2:29:18basically not really matter which one of
  3355. 2:29:19those we would pick. Of course, there's
  3356. 2:29:21a slight variance in the scores, but I
  3357. 2:29:23only ran
  3358. 2:29:25each task one time, so might be just
  3359. 2:29:27some noise or maybe something else. So
  3360. 2:29:29you would probably need to run a few
  3361. 2:29:31more samples per harness, but the score
  3362. 2:29:34looks roughly the same. So how would we
  3363. 2:29:36now know which harness should I choose?
  3364. 2:29:40If we just look all at all of the evals
  3365. 2:29:43being shown, it's always mostly about
  3366. 2:29:45score. But if we look at the token
  3367. 2:29:47column or even more at the dollar per
  3368. 2:29:49task,
  3369. 2:29:50there's a picture which is changing. So
  3370. 2:29:52the token per task is getting lower.
  3371. 2:29:55Interestingly, if we just use bash, and
  3372. 2:29:58then also the cache hit rate increases.
  3373. 2:30:00So if we just compare like the bash
  3374. 2:30:02media as the the baseline, the tasks are
  3375. 2:30:05almost 2x cheaper compared to
  3376. 2:30:08more complex, more tools agent
  3377. 2:30:11harnesses.
  3378. 2:30:12And we would not see it if we just look
  3379. 2:30:15at the score.
  3380. 2:30:16And to show you a few examples, I
  3381. 2:30:19prepared some demo. So here I'm using
  3382. 2:30:22the pie harness, which makes it super
  3383. 2:30:24simple to basically define what tools
  3384. 2:30:26you want to use.
  3385. 2:30:28I have the the same prompt for 3.5
  3386. 2:30:31flash, which we released roughly 6
  3387. 2:30:33months ago
  3388. 2:30:34for all tools, so like whatever pie
  3389. 2:30:36ships out of the box. Then we have it
  3390. 2:30:38for bash only, and then we have it for
  3391. 2:30:41carbon
  3392. 2:30:42and sorry, for carbon with the default
  3393. 2:30:45tools and then with with bash only.
  3394. 2:30:48This runs now since we only have roughly
  3395. 2:30:51around 20 minutes. I recorded all of
  3396. 2:30:54that and clicked that very nice
  3397. 2:30:56accelerate button here. And what we can
  3398. 2:30:57see on the right side is that
  3399. 2:31:00the same paradigm basically applies
  3400. 2:31:03when models get better, they roughly
  3401. 2:31:06need less tools. So with 3.5 flash, we
  3402. 2:31:09see with the default tools compared to
  3403. 2:31:11bash only, we had the same amount of
  3404. 2:31:13model calls, we had the same amount of
  3405. 2:31:15tool calls, and we had roughly the same
  3406. 2:31:17amount of tokens. And if we look at the
  3407. 2:31:19or the traces, we can see for bash only,
  3408. 2:31:22we have some cut commands to read
  3409. 2:31:24different files. We have some inline
  3410. 2:31:26scripts to make changes and then like
  3411. 2:31:28some git changes.
  3412. 2:31:29If we look at the default tools, we can
  3413. 2:31:32see roughly the same. We had
  3414. 2:31:34read file commands, we used the edit
  3415. 2:31:36tool, and then had some git commands to
  3416. 2:31:38make those changes. If we now look at
  3417. 2:31:40Carbon with the default tools, it
  3418. 2:31:42already used way less
  3419. 2:31:45tool calls and model calls. So it it's
  3420. 2:31:48basically doing work in parallel, but
  3421. 2:31:51also trying to reduce it. So we had for
  3422. 2:31:52example the still same for write read
  3423. 2:31:56calls and also the edit calls. But what
  3424. 2:31:58starts to get interesting if we are
  3425. 2:32:00going to look into bash. And in bash, we
  3426. 2:32:03had even less tool calls and model
  3427. 2:32:05calls. And the interesting part here is
  3428. 2:32:08that the model writes a single Python
  3429. 2:32:10script to make all of that changes in
  3430. 2:32:12one go and no longer needs to call
  3431. 2:32:14multiple tools.
  3432. 2:32:16So if we go
  3433. 2:32:18further, what does it means in terms for
  3434. 2:32:20for Carbon here and how bash pays off is
  3435. 2:32:22that
  3436. 2:32:23the better the model gets or the more
  3437. 2:32:26reasoning and like capability it gets,
  3438. 2:32:28bash helps the model to be able to chain
  3439. 2:32:31more commands in the single call. For
  3440. 2:32:33Carbon, it was roughly 2.2 to 3.7
  3441. 2:32:38commands per bash call. And for flash,
  3442. 2:32:40it was only 1.3 to 1.8.
  3443. 2:32:43So we are seeing up to 3x less tokens
  3444. 2:32:48by Carbon to achieve the same tasks
  3445. 2:32:51compared to having the same tools we
  3446. 2:32:53need to use.
  3447. 2:32:55So
  3448. 2:32:56two more examples to look at it as well
  3449. 2:32:58is like we have on the left side Carbon
  3450. 2:33:01with the default tools. So we see our
  3451. 2:33:03coding agent tools with write, with
  3452. 2:33:05edit, and we see okay, it does some
  3453. 2:33:08parallel work in turn two, some parallel
  3454. 2:33:10work in turn three and four. And then we
  3455. 2:33:13have all of the changes. And on the
  3456. 2:33:14right side, where just has the bash
  3457. 2:33:16tool, it uses Python to write an inline
  3458. 2:33:18script to make all of that changes at
  3459. 2:33:21the same time with much less token. And
  3460. 2:33:24the same counts for if we compare Gemini
  3461. 2:33:273.5 Flash to Gemini 4 Carbon Flash with
  3462. 2:33:31bash only is not chaining those
  3463. 2:33:33commands. So here it is not really it
  3464. 2:33:35doesn't matter if I'm having a a retool,
  3465. 2:33:38a write tool, an edit tool and a bash
  3466. 2:33:40tool, or if I have just a bash tool
  3467. 2:33:41because 3.5 Flash used to use bash for
  3468. 2:33:45single turn commands. So it had a single
  3469. 2:33:48turn for reading, it had a single turn
  3470. 2:33:50for writing. But if we look at Carbon,
  3471. 2:33:52which in here in this example is the
  3472. 2:33:54exact same task, it needs less way
  3473. 2:33:56turns. So the reading is not needed to
  3474. 2:33:59read all of the different files and
  3475. 2:34:01completely. It just uses rip grep and
  3476. 2:34:03like some intelligence searching to find
  3477. 2:34:05where we need to make changes. And then
  3478. 2:34:06it writes a single Python inline bash
  3479. 2:34:09command to make all of those edits in
  3480. 2:34:11one go, saving us a lot of tokens. So
  3481. 2:34:15what does it bring us? Models will keep
  3482. 2:34:17eating the harness. And I'm
  3483. 2:34:20pretty sure that 2027 will also be a lot
  3484. 2:34:22more about how many tokens do we need to
  3485. 2:34:24solve a task. Even for the few
  3486. 2:34:27improvements we made between Argon and
  3487. 2:34:29Carbon, is the the better model needs
  3488. 2:34:31less tokens per task, which of course is
  3489. 2:34:33very exciting for all of us because then
  3490. 2:34:35we can use it more. And also it matters
  3491. 2:34:38a lot what the harness provides. And we
  3492. 2:34:40will might not see it if we just look at
  3493. 2:34:43the accuracy. It's also very important
  3494. 2:34:45that we start measuring the token
  3495. 2:34:47efficiency because
  3496. 2:34:49harnesses can look the same on accuracy
  3497. 2:34:50because our models are now very good. So
  3498. 2:34:53if your harness
  3499. 2:34:54is more complex to use, the model will
  3500. 2:34:56work around it, will find solutions
  3501. 2:34:58around it, but needs more turns and more
  3502. 2:35:00tokens to achieve the same results.
  3503. 2:35:03And then also what comes afterwards. So
  3504. 2:35:05I will try to do the same experiment six
  3505. 2:35:08months from now. And currently the trend
  3506. 2:35:10we see slowly
  3507. 2:35:13rising at least on like social media and
  3508. 2:35:16like the industry is that we might go
  3509. 2:35:18back to some code mode tool that we
  3510. 2:35:21instead of writing a bash commands that
  3511. 2:35:23we write TypeScript or Python directly
  3512. 2:35:26that MCP will come back. I'm a big fan
  3513. 2:35:29of MCP and like they are making great
  3514. 2:35:31changes to make it more
  3515. 2:35:33easily easily usable, especially
  3516. 2:35:37interesting for all of that code mode
  3517. 2:35:39work is MCP servers now allow you to
  3518. 2:35:41return structured outputs so the model
  3519. 2:35:43can write better scripts to take more
  3520. 2:35:45actions in the single turn and it also
  3521. 2:35:47allows us to do some better proxy around
  3522. 2:35:50the security because bash calls are very
  3523. 2:35:52hard to understand, but if I have some
  3524. 2:35:55obstructive layer in TypeScript I can
  3525. 2:35:58watch all of the commands being
  3526. 2:36:00executed and make sure those are secure
  3527. 2:36:03and for you
  3528. 2:36:05build
  3529. 2:36:06to delete your code before the model
  3530. 2:36:08does.
  3531. 2:36:08Give your model to a computer, not a
  3532. 2:36:11toolbox
  3533. 2:36:12and when a new model ships I definitely
  3534. 2:36:14recommend you trying to think a little
  3535. 2:36:17bit outside of the box, be more
  3536. 2:36:19ambitious, try to really see, okay, what
  3537. 2:36:21if I remove my complete harness and just
  3538. 2:36:23give it one tool? And then also please
  3539. 2:36:25scan the QR code. You will get free
  3540. 2:36:28credits. We will send them out after the
  3541. 2:36:31AI Engineering event to try out Carbon
  3542. 2:36:33and also to try out the new Gemini
  3543. 2:36:35Decisions API. So if you have some time
  3544. 2:36:37today I really can recommend checking
  3545. 2:36:39out the new Decisions model, Carbon and
  3546. 2:36:42thank you for your time.
  3547. 2:36:45>> [applause]
  3548. 2:37:15>> <music>
  3549. 2:37:15>> </music>
  3550. 2:40:13>> This
  3551. 2:42:18>> Yeah.
  3552. 2:44:18>> This
  3553. 2:46:35>> All right.
  3554. 2:46:36Well,
  3555. 2:46:37super excited to be here. My name's
  3556. 2:46:39Ankit. I'm from Databricks. I work on AI
  3557. 2:46:41infrastructure there, and today I'm
  3558. 2:46:43going to talk about [music] the rise of
  3559. 2:46:45the agent workforce and how we're
  3560. 2:46:47building our agent infrastructure to
  3561. 2:46:49support what we think is the next
  3562. 2:46:50generation of agents.
  3563. 2:46:52I wanted to start by taking a step back
  3564. 2:46:54and and thinking about how we helped
  3565. 2:46:57computers become good in other things,
  3566. 2:47:00better even than humans. And one of the
  3567. 2:47:02games I grew up playing a lot was chess.
  3568. 2:47:04It's one of the great examples of
  3569. 2:47:06something where computers became way
  3570. 2:47:07better than humans a long time ago, but
  3571. 2:47:10I think it's important to think about
  3572. 2:47:12why that happened and how we can apply
  3573. 2:47:14those lessons today.
  3574. 2:47:15One of the biggest, you know, to
  3575. 2:47:17summarize at a high level, a lot of work
  3576. 2:47:18went into it.
  3577. 2:47:20The biggest thing that agents were able
  3578. 2:47:21to do, that computers were able to do,
  3579. 2:47:23was explore the space of decisions much
  3580. 2:47:26more effectively than humans.
  3581. 2:47:28For most complex problems, there's
  3582. 2:47:30actually a bunch of different ways you
  3583. 2:47:31can solve the problem. And at each turn,
  3584. 2:47:34you can kind of
  3585. 2:47:35think of three more decisions that you
  3586. 2:47:37can make. And really it's the
  3587. 2:47:38exploration of that space that all of us
  3588. 2:47:40are engaged in. All of us who solve hard
  3589. 2:47:42problems are engaged in the search space
  3590. 2:47:44for what's an amazing product, what's
  3591. 2:47:46the right solution to this math problem,
  3592. 2:47:48and etc., right?
  3593. 2:47:49And I think we've seen some really
  3594. 2:47:51exciting progress from the labs like
  3595. 2:47:53Anthropic, OpenAI, and Google
  3596. 2:47:56on solving problems that are well
  3597. 2:47:57formed. So, you know, if you think about
  3598. 2:47:59chess, it can be modeled effectively.
  3599. 2:48:01You can evaluate your position.
  3600. 2:48:04You can actually play simulate it. You
  3601. 2:48:05can try different moves and safely think
  3602. 2:48:08about all the different alternatives.
  3603. 2:48:10Similarly with math problems, often the
  3604. 2:48:12ones that agents are really good at
  3605. 2:48:13solving,
  3606. 2:48:14they you can actually verify if you got
  3607. 2:48:16the result right.
  3608. 2:48:18And how do we then take that mindset and
  3609. 2:48:21apply it to the problems that we
  3610. 2:48:22actually face day to day?
  3611. 2:48:25I think like let's a lot everyone here
  3612. 2:48:27is an engineer, so let's take it to
  3613. 2:48:29engineering for a bit.
  3614. 2:48:31I think, you know, if you take it to the
  3615. 2:48:32pre-AI days,
  3616. 2:48:34most people think about solving a single
  3617. 2:48:36problem and they're kind of single
  3618. 2:48:38threaded on that problem for the time
  3619. 2:48:39that they're writing code on it.
  3620. 2:48:41And you know, as a company if or a
  3621. 2:48:43product if you wanted to solve more
  3622. 2:48:44problems, you brought in more people.
  3623. 2:48:46And the very basic version of agents
  3624. 2:48:49just helps create leverage for that
  3625. 2:48:51engineer. Now you can be single threaded
  3626. 2:48:53on three things at the same time and
  3627. 2:48:55ultimately that helps you explore new
  3628. 2:48:56product ideas, ship faster, iterate
  3629. 2:48:59faster, and that's obviously very
  3630. 2:49:00exciting.
  3631. 2:49:02But it's it's not actually close to what
  3632. 2:49:03we needed to do to win a chess or any of
  3633. 2:49:06these other problems. What we're seeing
  3634. 2:49:08at the absolute frontier are these teams
  3635. 2:49:11of agents that are able to make
  3636. 2:49:12significant progress by exploring the
  3637. 2:49:14decision space more effectively.
  3638. 2:49:16So how do we move towards that world
  3639. 2:49:18where we actually have these teams of
  3640. 2:49:19agents that are coherently attempting to
  3641. 2:49:21solve problems? It's actually quite
  3642. 2:49:23complex and we think it requires a
  3643. 2:49:25little bit different type of
  3644. 2:49:26infrastructure.
  3645. 2:49:28Specifically, I think an important
  3646. 2:49:30question to ask is how do we give agents
  3647. 2:49:32a realistic world to explore.
  3648. 2:49:35Because in other problems we're able to
  3649. 2:49:37help define that world very effectively,
  3650. 2:49:39but even in software engineering we
  3651. 2:49:41actually really struggle to do that.
  3652. 2:49:43Like just consider this movement towards
  3653. 2:49:45software factories, right? Where you're
  3654. 2:49:47away from the from the agents and you're
  3655. 2:49:49actually operating a bunch of agents
  3656. 2:49:50that are working on a problem.
  3657. 2:49:52How do you validate that those agents
  3658. 2:49:54are effective?
  3659. 2:49:56Like before you had a branch, uh you had
  3660. 2:49:58main, you branched your PR off of that.
  3661. 2:50:00Maybe you have an agent working on that
  3662. 2:50:02PR and you have a some reasonable
  3663. 2:50:05assertion that this is kind of a you
  3664. 2:50:07know you have some like some tests or
  3665. 2:50:09something like that that help you guide
  3666. 2:50:11your development.
  3667. 2:50:12But if you're actually merging tens of
  3668. 2:50:15PRs a minute or even more, right? Like
  3669. 2:50:17if you play out the future of factories
  3670. 2:50:20of agents building code and software,
  3671. 2:50:23your PR is just out of date the moment
  3672. 2:50:24you branched. The the moment you started
  3673. 2:50:27you're already out of date and so it
  3674. 2:50:29demands a different mindset of thinking
  3675. 2:50:31of how do we verify these things. You
  3676. 2:50:32actually might want to move most of your
  3677. 2:50:34verification to post merge.
  3678. 2:50:36And so that kind of brings me to this a
  3679. 2:50:38little bit radical idea. I I don't think
  3680. 2:50:41we should try to write hundreds or
  3681. 2:50:43thousands of unit tests.
  3682. 2:50:45Instead I want to explore this idea of
  3683. 2:50:47simulating reality.
  3684. 2:50:49Can I give my agent a faithful
  3685. 2:50:51reproduction of what's actually there in
  3686. 2:50:53the real world in my production code, in
  3687. 2:50:55my production agent and let it work on
  3688. 2:50:58that.
  3689. 2:50:59And with you when you can do that you
  3690. 2:51:01can actually do that at scale. If you
  3691. 2:51:03find a way to simulate reality then you
  3692. 2:51:05can actually do that across lots of
  3693. 2:51:07different explorations.
  3694. 2:51:09At DataBricks one of the big things that
  3695. 2:51:10people do on our platform is build data
  3696. 2:51:12pipelines. We've got tens of thousands
  3697. 2:51:14of data pipelines on DataBricks.
  3698. 2:51:17Uh and the biggest problem that happens
  3699. 2:51:19with pipelines is that they break. Um
  3700. 2:51:21and in this era of agents what people
  3701. 2:51:24want is they want to fix those broken
  3702. 2:51:25pipelines with agents
  3703. 2:51:28because these are serious business
  3704. 2:51:29problems that they that are you know
  3705. 2:51:31caused by a broken pipeline.
  3706. 2:51:33If a pipeline breaks that business can
  3707. 2:51:35operate that day. And so, this becomes a
  3708. 2:51:37major emergency.
  3709. 2:51:40But like, how do you actually validate
  3710. 2:51:41this with an agent? Do you trust it on
  3711. 2:51:43your production data pipeline?
  3712. 2:51:45It's pretty risky because writes might
  3713. 2:51:47not be idempotent. Even reads might be
  3714. 2:51:50extremely expensive and affect other use
  3715. 2:51:52cases in production, right? Sometimes
  3716. 2:51:53it's not fully broken, it's just
  3717. 2:51:54partially broken.
  3718. 2:51:56One of the things we started doing was
  3719. 2:51:58building on top of branchable
  3720. 2:51:59infrastructure.
  3721. 2:52:01And so what we did was we branched the
  3722. 2:52:02entire data platform.
  3723. 2:52:04We branched the tables, the compute, the
  3724. 2:52:07governance, and we gave the agents a
  3725. 2:52:10place where they can actually play with
  3726. 2:52:12the data platform and explore different
  3727. 2:52:14hypotheses.
  3728. 2:52:15And these hypotheses can be ambitious.
  3729. 2:52:17Like, can I investigate the upstream
  3730. 2:52:19pipeline is healthy? Maybe I can do that
  3731. 2:52:21by changing the pipeline code and
  3732. 2:52:23running it again.
  3733. 2:52:24In any reasonable data platform, you
  3734. 2:52:27wouldn't do that with an upstream
  3735. 2:52:28pipeline unless you had a bunch of
  3736. 2:52:29engineers take a look at it, make sure
  3737. 2:52:30it was super safe.
  3738. 2:52:32With our branched data platform, you can
  3739. 2:52:33actually do that now. So it can
  3740. 2:52:35investigate a bunch of different things.
  3741. 2:52:37So we can investigate 10 ideas in
  3742. 2:52:38parallel with our agents,
  3743. 2:52:41actually get to a faithful conclusion
  3744. 2:52:43about whether it fixed the real data
  3745. 2:52:44pipeline
  3746. 2:52:45and then make that assumption, make that
  3747. 2:52:47assertion back either to a main agent or
  3748. 2:52:49to a human.
  3749. 2:52:50And what we saw before was happening was
  3750. 2:52:52that people would just like
  3751. 2:52:53LLMs would give us texts back. Some
  3752. 2:52:55human would take a look at it and be
  3753. 2:52:57like, okay, this might be an interesting
  3754. 2:52:58theory. Then they would go apply that
  3755. 2:53:00theory in some way. And ultimately, that
  3756. 2:53:02became more of this like AI-assisted
  3757. 2:53:05version of fixing the data pipeline.
  3758. 2:53:07But really where we want to go is is is
  3759. 2:53:09hands off, full full auto
  3760. 2:53:12for this kind of problem.
  3761. 2:53:14And we were we were struggling to do it
  3762. 2:53:15without this branchable infrastructure.
  3763. 2:53:18So how do we do it? I'm going to talk a
  3764. 2:53:19little bit about that in this talk.
  3765. 2:53:21Uh we built this on top of our Postgres
  3766. 2:53:23native agent platform, which started
  3767. 2:53:26with the Neon database, which is a
  3768. 2:53:27branchable database that gives us a
  3769. 2:53:29single place that we can actually use
  3770. 2:53:31for our services and for our agents.
  3771. 2:53:33And then we added in a bunch of really
  3772. 2:53:36core agent primitives that allow us to
  3773. 2:53:37build sophisticated different types of
  3774. 2:53:39agents.
  3775. 2:53:40Uh and those are the ones that are
  3776. 2:53:41probably familiar to all of you. Things
  3777. 2:53:42like memory, things like sandboxes, file
  3778. 2:53:45systems, and ways to host those agents.
  3779. 2:53:47And these low-level primitives we can we
  3780. 2:53:49can build with them, but the key is that
  3781. 2:53:51we made just all of them branchable. So
  3782. 2:53:53you can branch the entire reality of
  3783. 2:53:55stuff that's built on this platform for
  3784. 2:53:57us for ourselves.
  3785. 2:53:58Uh so how do we actually do that? Well,
  3786. 2:54:01let me walk it through like kind of
  3787. 2:54:03case by case.
  3788. 2:54:04The first thing is just the database,
  3789. 2:54:06right? And you know, we can take an
  3790. 2:54:07example of application services, but
  3791. 2:54:09here I'm going to talk about just even
  3792. 2:54:10storing the conversation state for for
  3793. 2:54:12an agent.
  3794. 2:54:13When you fork or branch an agent, you
  3795. 2:54:15actually might want to inherit the
  3796. 2:54:17history and you might even if you have a
  3797. 2:54:19team of agents give access to a given
  3798. 2:54:21member of that team to what the other
  3799. 2:54:23members of the team are doing.
  3800. 2:54:24Uh and we found the database to be a
  3801. 2:54:25convenient abstraction for this.
  3802. 2:54:28Uh the key was that hey, like when a new
  3803. 2:54:30reply comes in on the on the branched
  3804. 2:54:32agent, it shouldn't necessarily change
  3805. 2:54:34anything about the other one.
  3806. 2:54:35Uh and this is possible because Neon has
  3807. 2:54:38a branched database.
  3808. 2:54:40But then we actually found that agents
  3809. 2:54:41need more than just structured
  3810. 2:54:43databases. Of course, they need
  3811. 2:54:44structured databases, but they also like
  3812. 2:54:46to do this, you know, write these things
  3813. 2:54:48called memories.
  3814. 2:54:49And memories are, you know, at a very
  3815. 2:54:51high level they compile down to an
  3816. 2:54:53unstructured database for agents.
  3817. 2:54:55There are two ways that memories get
  3818. 2:54:56generated. The first is just by
  3819. 2:54:58synthesizing the context and the
  3820. 2:55:00conversations that someone's already had
  3821. 2:55:01with them. And the second by agents
  3822. 2:55:04actually explicitly saving content to an
  3823. 2:55:07unstructured place.
  3824. 2:55:09The key thing here is that you actually
  3825. 2:55:11want to consolidate and distill that
  3826. 2:55:12information. There's a ton of
  3827. 2:55:13information coming in through there.
  3828. 2:55:15And if you just give that, you know, if
  3829. 2:55:17you just let it kind of go on its own,
  3830. 2:55:19you have problems of staleness, you have
  3831. 2:55:22conflicts in the information, and all of
  3832. 2:55:24those can be extremely damaging to
  3833. 2:55:25teams.
  3834. 2:55:26So we have this thing that basically
  3835. 2:55:28dreams overnight. We use offline compute
  3836. 2:55:32uh and it takes the conversations, a
  3837. 2:55:34different agent essentially highlights
  3838. 2:55:35things that are conflicts
  3839. 2:55:37and consolidates the ad hoc notes that
  3840. 2:55:40were written by various members of the
  3841. 2:55:41team.
  3842. 2:55:42And then all members of the team can
  3843. 2:55:43actually recall from that memory
  3844. 2:55:45database.
  3845. 2:55:46And you can use this in a bunch of
  3846. 2:55:48different ways. We use this within our
  3847. 2:55:49Slack agents, but we also use this when
  3848. 2:55:52we're launching a parallel set of agents
  3849. 2:55:54to investigate a deep issue, for
  3850. 2:55:56example, in the data platform.
  3851. 2:55:58Now, how does branching come into this?
  3852. 2:56:00Well, some of the agents, like in many
  3853. 2:56:02of the use cases, they propose memories.
  3854. 2:56:05And it's not clear whether those
  3855. 2:56:06memories generalize to the actual
  3856. 2:56:09broader set of agents or if that's a
  3857. 2:56:10specific thing for just that agent.
  3858. 2:56:12And so with the branchable database for
  3859. 2:56:14memory, which is built on top of this
  3860. 2:56:15neon database, we can actually just
  3861. 2:56:17propose that memory somewhere in there,
  3862. 2:56:19and that branch can continue to iterate,
  3863. 2:56:22and at some point we can find a way to
  3864. 2:56:23merge it back in if it's necessary.
  3865. 2:56:26We found this to be an important layer
  3866. 2:56:28of flexibility because without the
  3867. 2:56:30ability to completely branch the memory,
  3868. 2:56:32we found it was very hard to maintain
  3869. 2:56:35the memory for a team of agents.
  3870. 2:56:37Especially as we increase parallelism,
  3871. 2:56:38you can actually experience like
  3872. 2:56:40significantly reduced quality if all the
  3873. 2:56:42agents are pursuing different goals.
  3874. 2:56:45I I like to think a little bit of it as
  3875. 2:56:47as like kind of similar to dynamic
  3876. 2:56:48programming, right? If you've explored a
  3877. 2:56:50branch of exploration
  3878. 2:56:52and then you have a different agent that
  3879. 2:56:53happens upon that same pathway, it's
  3880. 2:56:55very damaging to like always revisit
  3881. 2:56:58that, but sometimes you want to. And so
  3882. 2:57:00the ability to propose a memory is
  3883. 2:57:01extremely important to us.
  3884. 2:57:04How is all of this kind of branchable
  3885. 2:57:06database possible? Well, the way that
  3886. 2:57:08neon does this is it separates compute
  3887. 2:57:10from storage.
  3888. 2:57:11Compute
  3889. 2:57:12for Postgres is running in some separate
  3890. 2:57:14place, and then we actually have the
  3891. 2:57:16shared history and the storage in a
  3892. 2:57:18different layer. And what that lets us
  3893. 2:57:20do is copy on write for branching. So
  3894. 2:57:23when we have a branched version of the
  3895. 2:57:24database, you can actually have A's, you
  3896. 2:57:26know, compute A's writes separated from
  3897. 2:57:28compute B. And that's the kind of the
  3898. 2:57:30core layer of infrastructure that allows
  3899. 2:57:32us to build these applications on top of
  3900. 2:57:34that database.
  3901. 2:57:37But then we found, you know, obviously
  3902. 2:57:39agents are far more than storage, and
  3903. 2:57:41and branching storage wasn't enough.
  3904. 2:57:43We also found that as they were running
  3905. 2:57:45in sandboxes, agents like to take notes
  3906. 2:57:47on stuff and write down scripts and use
  3907. 2:57:50those to actually iterate on various
  3908. 2:57:52kinds of problems. It It's one of the
  3909. 2:57:54most exciting things about agents that
  3910. 2:57:56I've found that they are able to
  3911. 2:57:58generalize this capability that's needed
  3912. 2:58:00maybe for testing code and and things
  3913. 2:58:02like that and solve other more broader
  3914. 2:58:04general problems using that same
  3915. 2:58:06construct.
  3916. 2:58:08But then when you branch it, you
  3917. 2:58:10actually need to give the future agent a
  3918. 2:58:11similar context. If you We found that
  3919. 2:58:13when we started the branched agents
  3920. 2:58:15without the same files that that agent
  3921. 2:58:17had to begin with, it just wasn't as
  3922. 2:58:19effective. So when you're investigating
  3923. 2:58:21an outage, for example with an agent,
  3924. 2:58:23you might have notes and plans and a
  3925. 2:58:25future branch of that agent might update
  3926. 2:58:27those notes. It might find new things.
  3927. 2:58:30But we don't want that to go back cuz
  3928. 2:58:31we're actually exploring 10 different
  3929. 2:58:33pathways at the same time.
  3930. 2:58:35So building a branchable file system was
  3931. 2:58:37extremely important and allowed us to
  3932. 2:58:39build what we call a branchable sandbox.
  3933. 2:58:41Because then what we can do is we can
  3934. 2:58:44launch that sandbox compute and and
  3935. 2:58:45that's a hard problem in and of itself.
  3936. 2:58:47It's not one we're going to talk about
  3937. 2:58:48today.
  3938. 2:58:49But that compute can be attached to
  3939. 2:58:51storage.
  3940. 2:58:52And critically,
  3941. 2:58:54the agents actually really aggressively
  3942. 2:58:55leverage files very often. And so it can
  3943. 2:58:58be very painful to make all your storage
  3944. 2:59:00fully remote.
  3945. 2:59:01And so the really core piece of what
  3946. 2:59:03we've been building here is a sinking
  3947. 2:59:05process between what we call a local
  3948. 2:59:07cache and the remote cache.
  3949. 2:59:10And then what that allows us to do is as
  3950. 2:59:12we're able to make progress on that,
  3951. 2:59:14when we want a branch, we can actually
  3952. 2:59:16branch a second sandbox, use that
  3953. 2:59:18similar file system, and it reduces our
  3954. 2:59:20latency for for branching while also
  3955. 2:59:22keeping performance of the regular agent
  3956. 2:59:25reasonable.
  3957. 2:59:27So
  3958. 2:59:28what this allows us to do at a high
  3959. 2:59:29level is branch the entire working
  3960. 2:59:31state.
  3961. 2:59:33And this gives us the platform layer to
  3962. 2:59:35actually go do that. Now this isn't
  3963. 2:59:38obviously sufficient. You're going to
  3964. 2:59:39want to branch even more. If you're
  3965. 2:59:41building on top of software, you want to
  3966. 2:59:42find a way to branch that too.
  3967. 2:59:44And so this is along the lines of how
  3968. 2:59:46we're thinking, right? Like we the data
  3969. 2:59:47platform was built on top of some of
  3970. 2:59:49these things, but there's a lot more
  3971. 2:59:51that we've been able to branch there as
  3972. 2:59:52well.
  3973. 2:59:53Uh but we're really excited about this
  3974. 2:59:55layer for just even building agents.
  3975. 2:59:57And just to like kind of separate a
  3976. 2:59:59little bit even from software
  3977. 3:00:00development, like another thing a lot of
  3978. 3:00:02us are spending a lot of time thinking
  3979. 3:00:03about
  3980. 3:00:04is security.
  3981. 3:00:06Uh
  3982. 3:00:07I mean just taking a step back, I think
  3983. 3:00:09this is probably the single most
  3984. 3:00:11important thing for any product that
  3985. 3:00:12anyone sells at this point
  3986. 3:00:14because trust is kind of the most
  3987. 3:00:16important thing that we're selling when
  3988. 3:00:17we work with other people.
  3989. 3:00:19And uh with agents being the ones who
  3990. 3:00:21generate the vulnerabilities,
  3991. 3:00:24uh it's kind of
  3992. 3:00:26impossible to find a different solution
  3993. 3:00:27than having agents be the ones who are
  3994. 3:00:29monitoring for those exact
  3995. 3:00:31vulnerabilities in your code. So you
  3996. 3:00:33should be thinking about how you can
  3997. 3:00:34build an agent factory that's detecting
  3998. 3:00:36security issues in your codebase.
  3999. 3:00:38Um and so this is something we're
  4000. 3:00:39spending a lot of time spending a lot of
  4001. 3:00:41time on.
  4002. 3:00:43So typically, what happens in a security
  4003. 3:00:44incident? Well, let's say you suspect
  4004. 3:00:47something. An agent might be always
  4005. 3:00:48looking for it.
  4006. 3:00:49Why why is there this like network
  4007. 3:00:51traffic that's egressing all of a sudden
  4008. 3:00:52from my control plane?
  4009. 3:00:54Well, I can assemble an incident
  4010. 3:00:56timeline and that's obviously the first
  4011. 3:00:57step in an investigation. But anyone
  4012. 3:00:59who's been a part of a serious security
  4013. 3:01:01investigation knows that time is of the
  4014. 3:01:03essence. It's a perfect problem for
  4015. 3:01:05parallelism and that's exactly what you
  4016. 3:01:07do when a security incident happens. You
  4017. 3:01:08bring everyone online so you can explore
  4018. 3:01:1110, 15, 20 different theories at once
  4019. 3:01:14because an hour earlier that you can you
  4020. 3:01:16can detect the real issue is an hour
  4021. 3:01:18that you're actually more safe.
  4022. 3:01:20So okay, what if we did that with a
  4023. 3:01:22branchable agent?
  4024. 3:01:24Let's say there's two paths to
  4025. 3:01:25investigate.
  4026. 3:01:26Uh so you know, the agents assemble the
  4027. 3:01:28incident timeline and now you can ask
  4028. 3:01:31two questions. Hey, did a recent
  4029. 3:01:32deployment introduce some kind of new
  4030. 3:01:34integration? Like maybe I can go look at
  4031. 3:01:36the code.
  4032. 3:01:37Or I can explore a totally different
  4033. 3:01:39theory. Is a compromised identity
  4034. 3:01:41accessing this data?
  4035. 3:01:42And the core thing that we are able to
  4036. 3:01:44do here is we branch that whole original
  4037. 3:01:46agent that was starting the
  4038. 3:01:47investigation.
  4039. 3:01:50And then the new one is able to work in
  4040. 3:01:51a completely isolated copy of the world.
  4041. 3:01:54And in some cases, when we're able to,
  4042. 3:01:55we can even branch the actual service
  4043. 3:01:57that we're investigating.
  4044. 3:01:59Uh and so when you have this isolated
  4045. 3:02:00copy of conversation, memory, and
  4046. 3:02:02storage, you can fully explore that
  4047. 3:02:05security risk and and let the agent loop
  4048. 3:02:07on its own. And that's kind of the key
  4049. 3:02:08thing here.
  4050. 3:02:09The way that these agents are good is
  4051. 3:02:11that they're able to loop, verify, try
  4052. 3:02:14new theories. I think Andrej Karpathy
  4053. 3:02:16said it really well. They're far more
  4054. 3:02:18tenacious than any human probably in the
  4055. 3:02:22history of software.
  4056. 3:02:24And so they're able to just But like we
  4057. 3:02:26keep getting in the way by inserting
  4058. 3:02:27ourselves as an approval layer.
  4059. 3:02:30Uh and so this is a solution that we
  4060. 3:02:32think helps with that. Just like take
  4061. 3:02:33away the consequences of making a
  4062. 3:02:35mistake a little bit. Branch the entire
  4063. 3:02:37state so that you're not actually,
  4064. 3:02:40you know, worrying about that same risk.
  4065. 3:02:43And then you we you know, we've been
  4066. 3:02:44able to find this to actually uh come to
  4067. 3:02:47a reasonable conclusion as well.
  4068. 3:02:49So what is this all add up to?
  4069. 3:02:52For us, we think that your
  4070. 3:02:53infrastructure needs to be branchable.
  4071. 3:02:55And that's what we're building with
  4072. 3:02:57Neon, with Databricks. Uh and we're
  4073. 3:02:59super excited about that branchable
  4074. 3:03:00future because we think that teams are
  4075. 3:03:02going to be the next generation of
  4076. 3:03:04agents, both for software and the way
  4077. 3:03:06that we actually build products.
  4078. 3:03:08Thank you.
  4079. 3:03:11>> [applause]
  4080. 3:04:46>> Oh.
  4081. 3:05:51>> [music]
  4082. 3:08:42>> I
  4083. 3:10:06>> Okay.
  4084. 3:10:08All right. Can you all hear me?
  4085. 3:10:12All right. My name is Andrew and
  4086. 3:10:15I think we're starting now. I don't know
  4087. 3:10:17if there's a timer anywhere.
  4088. 3:10:19Uh my name is Andrew and I'm excited to
  4089. 3:10:21talk to you today about the safe
  4090. 3:10:24software factory.
  4091. 3:10:27Uh but first a little bit about me. Uh
  4092. 3:10:30I am a staff software engineer at Sonar.
  4093. 3:10:33I work on Guitar, which is Sonar's AI
  4094. 3:10:36code review product.
  4095. 3:10:38Uh and previously I was at uh Uber
  4096. 3:10:41working on the driver trip experience
  4097. 3:10:44and navigation. Uh and then after that I
  4098. 3:10:47was at a health tech startup called
  4099. 3:10:48Forward for a little while
  4100. 3:10:50uh building a sort of an AI-enabled
  4101. 3:10:53primary healthcare system.
  4102. 3:10:59Okay. So the software factory.
  4103. 3:11:02This is like
  4104. 3:11:04the hottest buzzword in our industry
  4105. 3:11:06right now.
  4106. 3:11:08Everyone's talking about software
  4107. 3:11:09factories. Everyone's trying to build a
  4108. 3:11:11software factory.
  4109. 3:11:15And the question is Well, oh so first of
  4110. 3:11:19all
  4111. 3:11:20um
  4112. 3:11:22most of you probably know what this is
  4113. 3:11:25already.
  4114. 3:11:26Um but if you don't, I'll just give a
  4115. 3:11:29really quick overview. Uh this is sort
  4116. 3:11:32of the the main software development
  4117. 3:11:34lifecycle loop uh as it existed before
  4118. 3:11:37the AI era.
  4119. 3:11:38So we have uh inputs, basically product
  4120. 3:11:42specs, feature requests, tickets,
  4121. 3:11:45things like that. Um we take those into
  4122. 3:11:48the planning step if they're especially
  4123. 3:11:50if they're complicated.
  4124. 3:11:52Uh and then we then we build, then we
  4125. 3:11:54write the code.
  4126. 3:11:56Uh and then we um uh put up a PR and
  4127. 3:12:00that gets uh reviewed and it goes
  4128. 3:12:03through CI and then it eventually
  4129. 3:12:06merges.
  4130. 3:12:08Uh and then we deploy that to
  4131. 3:12:09production.
  4132. 3:12:11And then users interact with our
  4133. 3:12:13software and they give us feedback or
  4134. 3:12:15they find bugs.
  4135. 3:12:17Um
  4136. 3:12:18or we might also get some monitoring or
  4137. 3:12:20and alerts as well. And then those go
  4138. 3:12:22back in as more inputs. And the the
  4139. 3:12:26cycle continues.
  4140. 3:12:28And and so um in the in the era of
  4141. 3:12:32agentic coding and AI automation has
  4142. 3:12:36followed every bottleneck. So the
  4143. 3:12:38bottleneck used to be the the coding
  4144. 3:12:40part, the building.
  4145. 3:12:41Um now with coding agents that's been
  4146. 3:12:44automated.
  4147. 3:12:45Um
  4148. 3:12:47and now the bottleneck is sort of
  4149. 3:12:48shifted over to the that review step.
  4150. 3:12:51And we're starting to introduce uh AI
  4151. 3:12:54code review systems for that.
  4152. 3:12:57Um
  4153. 3:12:58and then and we're seeing uh AI systems
  4154. 3:13:01and automation happen really at every
  4155. 3:13:03step of this now. Uh so we're starting
  4156. 3:13:06to see AI systems for monitoring,
  4157. 3:13:09deployments, controlling rollouts,
  4158. 3:13:12triaging uh triaging you know tickets
  4159. 3:13:16and incidents, planning. And so this
  4160. 3:13:19this here is the sort of software
  4161. 3:13:22factory that we're that we're creating.
  4162. 3:13:28And and so the question that we all have
  4163. 3:13:30now is how do we make this factory safe,
  4164. 3:13:35right? Uh in other words,
  4165. 3:13:37how can we introduce enough layers of
  4166. 3:13:39verification
  4167. 3:13:41into this so that we can trust that the
  4168. 3:13:43software coming out of it is it works as
  4169. 3:13:46expected, it's free of bugs, free of
  4170. 3:13:49security vulnerabilities.
  4171. 3:13:54Um our CEO
  4172. 3:13:56uh Tarek said recently code is abundant,
  4173. 3:13:58confidence is not.
  4174. 3:14:01So what I'm I'm to talk about today is
  4175. 3:14:03just how at Sonar have been thinking
  4176. 3:14:05about uh how to introduce uh
  4177. 3:14:08confidence uh at at all sort of all
  4178. 3:14:12different points within this software
  4179. 3:14:13factory.
  4180. 3:14:17And uh
  4181. 3:14:19we're approaching this this challenge
  4182. 3:14:20with a defense in depth approach
  4183. 3:14:23that comes from adding more loops, you
  4184. 3:14:26might not be surprised to hear.
  4185. 3:14:29And I'm going to talk about four of them
  4186. 3:14:31today in particular. So there's the loop
  4187. 3:14:34one, which is the PR checks loop.
  4188. 3:14:37Uh
  4189. 3:14:38loop two, uh which is the post deploy
  4190. 3:14:42verifications loop.
  4191. 3:14:45Loop three, uh which is the maintenance
  4192. 3:14:48loop.
  4193. 3:14:51And loop four, which is the knowledge
  4194. 3:14:52loop. And this is kind of a kind of a
  4195. 3:14:54special loop.
  4196. 3:14:55Uh
  4197. 3:14:56so first I'm going to talk about loop
  4198. 3:14:58one,
  4199. 3:15:00the PR checks loop. So this is the loop
  4200. 3:15:01that we probably all have in in some
  4201. 3:15:03form.
  4202. 3:15:04Uh you put up a PR, it goes through some
  4203. 3:15:06CI checks, it gets a code review.
  4204. 3:15:09Nowadays it's probably getting an AI
  4205. 3:15:10code review as well. Um but we're
  4206. 3:15:13starting to see a lot more automation
  4207. 3:15:15being introduced here as well. So we're
  4208. 3:15:17starting to see agents pushing fixes to
  4209. 3:15:20respond to the code review comments.
  4210. 3:15:23Um and this loop kind of goes around and
  4211. 3:15:26around until the PR turns green.
  4212. 3:15:34And there's actually a lot more that we
  4213. 3:15:37can be doing here
  4214. 3:15:38um because catching a problem at this
  4215. 3:15:41point is still cheaper than after it
  4216. 3:15:43merges into the codebase.
  4217. 3:15:45Uh so we have a a real opportunity here
  4218. 3:15:47to widen this review past the sort of
  4219. 3:15:50normal tests and code review that most
  4220. 3:15:53of us do.
  4221. 3:15:55So of course we have AI code review.
  4222. 3:15:58We can also do static analysis here.
  4223. 3:16:00Many of us do that.
  4224. 3:16:03Uh we could do what's called functional
  4225. 3:16:05validation. So this is where we can look
  4226. 3:16:07uh what the ticket asks for and see if
  4227. 3:16:09the PR actually addresses those things.
  4228. 3:16:15Um we can look at rules that the team
  4229. 3:16:19may set for coding conventions,
  4230. 3:16:23uh see if uh best practices, things like
  4231. 3:16:26that, see if the PR is adhering to
  4232. 3:16:28those.
  4233. 3:16:30We can look at knowledge. So this can be
  4234. 3:16:32knowledge accumulated from company
  4235. 3:16:34documentation,
  4236. 3:16:36from things that have been learned on
  4237. 3:16:37past code reviews or past outages.
  4238. 3:16:43Uh we could do cross repo checks. So we
  4239. 3:16:45could actually look at other
  4240. 3:16:46repositories and see if anything in in
  4241. 3:16:48this PR uh
  4242. 3:16:50will break or cause something to break
  4243. 3:16:52in another repository.
  4244. 3:16:55Uh we could do cross PR analysis. So we
  4245. 3:16:59can look at all the other open PRs as
  4246. 3:17:01well and see if there's anything uh in
  4247. 3:17:05this PR that conflicts with one of
  4248. 3:17:06those.
  4249. 3:17:10We could do security review, check for
  4250. 3:17:12vulnerabilities, secrets, dependencies.
  4251. 3:17:17We could do end-to-end testing. So for
  4252. 3:17:19example, if this is a web app, we could
  4253. 3:17:20open up in a browser and and click
  4254. 3:17:22around and see if it works as expected.
  4255. 3:17:27Um and finally we can do risk
  4256. 3:17:30assessment. So we could sort of uh look
  4257. 3:17:32at this and compute uh
  4258. 3:17:35the blast radius and sort of categorize
  4259. 3:17:37this into like low, medium, or high
  4260. 3:17:39risk. And that can actually help us
  4261. 3:17:41determine if this is a PR that a human
  4262. 3:17:44even needs to look at at all.
  4263. 3:17:47And so all of these things can flow into
  4264. 3:17:49this this sort of decision, this merge
  4265. 3:17:51gate, this decision of whether this PR
  4266. 3:17:53is mergeable.
  4267. 3:17:55And of course uh every time a a fix is
  4268. 3:17:57pushed, all of these run again,
  4269. 3:18:00at least in some degree.
  4270. 3:18:04And and with all of this, it's important
  4271. 3:18:06to gate the that merge decision not just
  4272. 3:18:09on the fact that a test ran or that it
  4273. 3:18:11passed, but on the evidence.
  4274. 3:18:13So that would be things like the actual
  4275. 3:18:15code review findings, the results of the
  4276. 3:18:17quality gates,
  4277. 3:18:19uh any vulnerabilities,
  4278. 3:18:22um the risk level. So all of that can be
  4279. 3:18:26uh basically fed into this merge gate
  4280. 3:18:29and checked against criteria that the
  4281. 3:18:31the team can set for what is a mergeable
  4282. 3:18:33PR.
  4283. 3:18:35And when you record that evidence uh to
  4284. 3:18:37the PR, it just makes this
  4285. 3:18:40uh the merge gate much stronger.
  4286. 3:18:45And so what we're seeing now uh with
  4287. 3:18:47teams who do this well
  4288. 3:18:49uh is that they can actually delegate
  4289. 3:18:50more and more trusts uh to this type of
  4290. 3:18:53automation on the PR review.
  4291. 3:18:56Uh so we're seeing teams begin to turn
  4292. 3:18:59on not just automatic merge blocking,
  4293. 3:19:02but uh even auto approve in some cases
  4294. 3:19:05for low risk changes and and even auto
  4295. 3:19:08merge for certain types of PRs that are
  4296. 3:19:10that are low risk and approved.
  4297. 3:19:14And this can just free up a lot of
  4298. 3:19:15review bandwidth for human reviewers to
  4299. 3:19:17review the the more high risk changes.
  4300. 3:19:22Okay, so now um I'm going to talk about
  4301. 3:19:25loop two, which is our post deploy
  4302. 3:19:27verification loop.
  4303. 3:19:29Uh so this is I would say very much in
  4304. 3:19:32its early stages right now. People are
  4305. 3:19:34experimenting with all kinds of things
  4306. 3:19:36here.
  4307. 3:19:37Um but the idea is that once your code
  4308. 3:19:40is deployed,
  4309. 3:19:41uh you can have you you can build AI
  4310. 3:19:44systems to check uh to read like
  4311. 3:19:46production logs, traces, metrics, and
  4312. 3:19:49see if your code is actually behaving as
  4313. 3:19:51you expected to in production.
  4314. 3:19:55And then from there you can actually um
  4315. 3:19:58uh we're starting to see things like
  4316. 3:20:00agentic rollout where agents can decide
  4317. 3:20:03whether to widen a rollout, a rollback,
  4318. 3:20:06turn a feature flag off.
  4319. 3:20:09And you can also do uh AI outage
  4320. 3:20:11monitoring so you can uh
  4321. 3:20:14monitor alerts and uh use AI systems to
  4322. 3:20:17help you uh root cause and fix outages.
  4323. 3:20:24Okay. Uh
  4324. 3:20:25so moving on to loop three. Uh so this
  4325. 3:20:29is what I call the maintenance loop.
  4326. 3:20:31Uh this is kind of a a parallel loop
  4327. 3:20:33that runs alongside your main software
  4328. 3:20:36development uh life cycle loop.
  4329. 3:20:39And this is where we continuously scan
  4330. 3:20:41the codebase on some schedule.
  4331. 3:20:43And we check for a lot of the same
  4332. 3:20:45things that we check for in the PR. We
  4333. 3:20:46we could do code review, static
  4334. 3:20:48analysis, security checks, etc.
  4335. 3:20:51And then we can If we find an issue, we
  4336. 3:20:53can open up a remediation PR
  4337. 3:20:56to address that.
  4338. 3:20:58And this lets us catch things that we
  4339. 3:21:00couldn't catch in the PR. So for
  4340. 3:21:02example, you could imagine
  4341. 3:21:04um
  4342. 3:21:06maybe a new rule was introduced after
  4343. 3:21:07code merged
  4344. 3:21:09or a new vulnerability was published
  4345. 3:21:11after code merged.
  4346. 3:21:13Um
  4347. 3:21:14or even something like slow drift where
  4348. 3:21:17you you slowly get duplication or
  4349. 3:21:19complexity building up that you can't
  4350. 3:21:21really see on a single PR.
  4351. 3:21:23Uh you can catch these kinds of things
  4352. 3:21:25here
  4353. 3:21:27with these uh with the continuous
  4354. 3:21:28codebase scans.
  4355. 3:21:33Okay. And so finally we move over to the
  4356. 3:21:36knowledge loop. And And this is becoming
  4357. 3:21:38We're seeing this become a very
  4358. 3:21:39important uh loop
  4359. 3:21:41for a lot of companies.
  4360. 3:21:43Um every company has a lot of knowledge,
  4361. 3:21:46documentation, product specs,
  4362. 3:21:50um
  4363. 3:21:51business decisions,
  4364. 3:21:54um and
  4365. 3:21:56the agents do a better job if they have
  4366. 3:21:58that knowledge, right?
  4367. 3:22:00So
  4368. 3:22:01[clears throat] this loop basically
  4369. 3:22:03collects all that knowledge from
  4370. 3:22:04different sources
  4371. 3:22:06as well as things that were learned on
  4372. 3:22:07prior code reviews and from prior
  4373. 3:22:10outages.
  4374. 3:22:11It's curated and then it can be uh fed
  4375. 3:22:15into all of the other loops, all of
  4376. 3:22:17these other agents,
  4377. 3:22:19um so they can uh do a better job.
  4378. 3:22:25And for example, this is really powerful
  4379. 3:22:28with your coding agents because if they
  4380. 3:22:30have some of this knowledge, they can
  4381. 3:22:32probably do a better job the first time,
  4382. 3:22:35fewer issues,
  4383. 3:22:36um and then you'll you'll wind up with
  4384. 3:22:39basically fewer review laps at the uh
  4385. 3:22:41review stage.
  4386. 3:22:47Okay, so what does this mean for our
  4387. 3:22:51role as human software engineers in this
  4388. 3:22:53factory?
  4389. 3:22:55Uh you might be asking, okay, so if if
  4390. 3:22:58we nail all of this, can we just uh turn
  4391. 3:23:00the lights out and run a dark factory?
  4392. 3:23:04Uh and I would say probably not, not
  4393. 3:23:06right now.
  4394. 3:23:08Uh I like to say the factory runs dim,
  4395. 3:23:10not dark. It's still very important for
  4396. 3:23:13us to
  4397. 3:23:14[cough]
  4398. 3:23:15intervene in places where our expertise
  4399. 3:23:18is is really important.
  4400. 3:23:20Uh
  4401. 3:23:21so that would be things like auditing
  4402. 3:23:23risky code,
  4403. 3:23:25um
  4404. 3:23:26verifying that evidence, verifying
  4405. 3:23:29things are actually working as expected,
  4406. 3:23:32handling exceptions like rollbacks and
  4407. 3:23:35incidents,
  4408. 3:23:37uh
  4409. 3:23:38reviewing architecture.
  4410. 3:23:41Agents are not very good at this yet, I
  4411. 3:23:43would argue.
  4412. 3:23:45Um
  4413. 3:23:46and supplying the why or or helping to
  4414. 3:23:49build that knowledge that that powers
  4415. 3:23:51all these agents and all these loops,
  4416. 3:23:54supplying the product intent, the
  4417. 3:23:56trade-offs, the business decisions,
  4418. 3:23:58things like that.
  4419. 3:24:01And of course, we engineer the factory
  4420. 3:24:04ourselves, right? We're we're we've sort
  4421. 3:24:06of moved into this era where we're we're
  4422. 3:24:07building the software factory as a
  4423. 3:24:09whole.
  4424. 3:24:13So with that, I hope that gave you sort
  4425. 3:24:17of a framework for how to think about um
  4426. 3:24:19not just how to build your software
  4427. 3:24:21factory, uh
  4428. 3:24:23but how to build a safe software
  4429. 3:24:25factory.
  4430. 3:24:26So, thank you. That's all for me.
  4431. 3:24:29>> [applause]
  4432. 3:28:20>> <music>
  4433. 3:34:06>> This
  4434. 3:34:28>> <music>
  4435. 3:37:18>> Yeah.
  4436. 3:42:45>> All set.
  4437. 3:42:47Hi everyone. I'm Luke. I lead our core
  4438. 3:42:50agent team here at Factory. Today I want
  4439. 3:42:52to talk about autonomy and why the teams
  4440. 3:42:55that are getting the most out of coding
  4441. 3:42:57agents are not actually the ones that
  4442. 3:42:58are handing over like the most work on
  4443. 3:42:59day one. It's instead the folks who are
  4444. 3:43:02ramping up autonomy gradually, one
  4445. 3:43:04checkpoint at a time until what they
  4446. 3:43:05have starts to resemble a software
  4447. 3:43:07factory.
  4448. 3:43:09So for those who don't know us, Factory
  4449. 3:43:11is a platform that builds Droid and then
  4450. 3:43:14a whole like surface for software
  4451. 3:43:16engineering and building software on top
  4452. 3:43:18of that coding agent. Hundreds of
  4453. 3:43:20thousands of developers use us around
  4454. 3:43:22the world at companies like Nvidia,
  4455. 3:43:23T-Mobile, and Adobe.
  4456. 3:43:26What these companies' engineers are
  4457. 3:43:28building with us is is really like a
  4458. 3:43:30software factory, a system that runs
  4459. 3:43:3124/7 and builds software under the
  4460. 3:43:34company's control.
  4461. 3:43:36And that's a loop that takes in signals
  4462. 3:43:38from wherever information is already
  4463. 3:43:39available, turns them into changes,
  4464. 3:43:42validates them, and then learns from
  4465. 3:43:44what production shows. A signal is
  4466. 3:43:46really like anything that tells you how
  4467. 3:43:48your software should evolve. So that
  4468. 3:43:49could be you know a sentry alert or
  4469. 3:43:51someone's idea on Slack. Every signal
  4470. 3:43:54goes through the same stages, so you
  4471. 3:43:55standardize you standardize the the
  4472. 3:43:57process, not the output.
  4473. 3:44:00So we built that platform on five
  4474. 3:44:02foundations. It has to be transparent,
  4475. 3:44:05modular, governable, efficient, and
  4476. 3:44:07frontier.
  4477. 3:44:10Most people picture a software factory
  4478. 3:44:12as this kind of like lights out building
  4479. 3:44:14where there's no people inside of it.
  4480. 3:44:16Requirements come in one end and merge
  4481. 3:44:18PRs come out the other. And and really
  4482. 3:44:21what we saw firsthand in the field is
  4483. 3:44:23that that's not the case. The best ones
  4484. 3:44:24aren't set up like that. They're built
  4485. 3:44:26around people instead. Humans decide
  4486. 3:44:29what matters and how much to trust
  4487. 3:44:31Droids over time, and those Droids earn
  4488. 3:44:33their keep as they deliver more work
  4489. 3:44:34that's higher quality.
  4490. 3:44:36Every company's software factory looks
  4491. 3:44:38very different to each other. So what we
  4492. 3:44:40what we sell is not really this factory,
  4493. 3:44:41it's the tools and the building blocks
  4494. 3:44:43to compose them.
  4495. 3:44:46Now it's it's worth kind of looking at
  4496. 3:44:47how we got here in the first place. Back
  4497. 3:44:50in April, one of our flagship products
  4498. 3:44:51was Missions, which was one of like the
  4499. 3:44:53first multi-agent uh products to really
  4500. 3:44:56take like a bite out of the human
  4501. 3:44:58attention bottleneck that existed in in
  4502. 3:45:00software engineering.
  4503. 3:45:02You describe a goal, you like argue with
  4504. 3:45:04the orchestrator until you were aligned,
  4505. 3:45:06you'd approve the plan, and then it
  4506. 3:45:08would go handle implementation for like
  4507. 3:45:10days or even weeks. When the goal was
  4508. 3:45:12clear and verifiable, you really could
  4509. 3:45:14come back to like completed work pretty
  4510. 3:45:16consistently.
  4511. 3:45:18But as more teams adopted it, we heard
  4512. 3:45:20some consistent like pieces of feedback.
  4513. 3:45:22The first was how Missions asked you to
  4514. 3:45:24work. The product assumed that you could
  4515. 3:45:25plan everything up front and then it
  4516. 3:45:27would just work. Uh and the UX sort of
  4517. 3:45:29suggested that you shouldn't interrupt
  4518. 3:45:31them. Missions didn't really like ask
  4519. 3:45:32you any questions or provide like an
  4520. 3:45:34interactive surface for that uh
  4521. 3:45:36communication between you and the agent
  4522. 3:45:38once it was starting implementation. So
  4523. 3:45:40this suits like migrations and big uh
  4524. 3:45:43big coding like feature projects, but it
  4525. 3:45:45didn't really fill fit the bill for like
  4526. 3:45:47most kinds of other work.
  4527. 3:45:49The second was cost and time. And that
  4528. 3:45:51came from honestly like the model's
  4529. 3:45:53getting really really good. I'm sure
  4530. 3:45:55you've all seen now adversarial review
  4531. 3:45:58is really really good at finding these
  4532. 3:45:59issues in your code. And so uh at the
  4533. 3:46:02time we were adding adversarial review
  4534. 3:46:04into every milestone's boundary. That
  4535. 3:46:07just kind of like grew the scope of that
  4536. 3:46:09mission from something like
  4537. 3:46:1160 features to over 200.
  4538. 3:46:14And so you ended up having a token bill
  4539. 3:46:16and a timeline that matched that.
  4540. 3:46:18And then the last one was that Missions
  4541. 3:46:20were keeping their context to one
  4542. 3:46:21person. So that was fine for like a
  4543. 3:46:23single project, but it meant that the
  4544. 3:46:25rest of the team couldn't really build
  4545. 3:46:27on what that mission had learned.
  4546. 3:46:30So we went back to the drawing board.
  4547. 3:46:32And one mental model kept on coming up.
  4548. 3:46:34When you have like an org that is
  4549. 3:46:36running dozens or hundreds of agents at
  4550. 3:46:39a given point in time, you actually have
  4551. 3:46:41like a lot of the the the problems that
  4552. 3:46:43operating systems back in the day
  4553. 3:46:45solved.
  4554. 3:46:46How to schedule work, how to run what
  4555. 3:46:48many things at once safely, and how to
  4556. 3:46:50share state.
  4557. 3:46:53So, every operating system starts with
  4558. 3:46:55the thing that actually does the work,
  4559. 3:46:57and we wanted to revisit the agent
  4560. 3:46:58itself.
  4561. 3:46:59Long Horizon agents run for thousands of
  4562. 3:47:01turns, and every inefficiency across of
  4563. 3:47:04those gets multiplied. So, you could
  4564. 3:47:06have, you know, an extra inference
  4565. 3:47:07request that's unnecessary or a a tool
  4566. 3:47:10result that's pretty bloated or a prompt
  4567. 3:47:12cache miss. All of those things compound
  4568. 3:47:15over time and lead to waste.
  4569. 3:47:18So, we decided to make some pretty
  4570. 3:47:19significant changes to the harness
  4571. 3:47:20itself.
  4572. 3:47:22The biggest one was programmatic tool
  4573. 3:47:23calling, also known as code mode.
  4574. 3:47:25Instead of calling one tool, waiting on
  4575. 3:47:27the model, and then calling the next, we
  4576. 3:47:29have the model write a small program
  4577. 3:47:31that executes all those tools in one go
  4578. 3:47:33and then prints out only what's
  4579. 3:47:35relevant.
  4580. 3:47:36So, there's this example here where you
  4581. 3:47:38have 150 linear tickets that are getting
  4582. 3:47:40moved from one project to another.
  4583. 3:47:42Done the normal way, that's like 30
  4584. 3:47:44inference requests, a million tokens,
  4585. 3:47:46but as one model generated program, you
  4586. 3:47:49actually only have like 60k tokens. It
  4587. 3:47:51runs much faster, and the model only
  4588. 3:47:52sees what is actually relevant to the
  4589. 3:47:54task, the the the fact whether it was
  4590. 3:47:56like successful or not.
  4591. 3:47:59So, this layer of the OS is where being
  4592. 3:48:00frontier and efficient matters the most.
  4593. 3:48:03A new model lands every few weeks, and
  4594. 3:48:05no two fail fail the same way. So, we
  4595. 3:48:07tune our harness in such a way that
  4596. 3:48:09evens out this jaggedness across all
  4597. 3:48:11these different models. The result is
  4598. 3:48:12that Droid is now, you know, higher
  4599. 3:48:14quality at a lower cost than every other
  4600. 3:48:16coding agent on the market.
  4601. 3:48:20Now, you have a faster agent, but that
  4602. 3:48:21that doesn't really give you like a step
  4603. 3:48:23function increase in in how much work
  4604. 3:48:25gets done inside of your system.
  4605. 3:48:27For that, you need many agents running
  4606. 3:48:28at once, and they need somewhere to run.
  4607. 3:48:31Remote missions were kind of held back
  4608. 3:48:33by this like fundamental limitation in
  4609. 3:48:35the system
  4610. 3:48:36where there was one computer that
  4611. 3:48:37handled all the work.
  4612. 3:48:39But to run a software factory, you need
  4613. 3:48:41a compute that scales with the volume of
  4614. 3:48:43work in the system. Otherwise, every
  4615. 3:48:45agent is is fighting for the same files
  4616. 3:48:47or system resources.
  4617. 3:48:49And so that's why we built environments,
  4618. 3:48:51which are reusable definitions of the
  4619. 3:48:52sandboxes that your droids need,
  4620. 3:48:54complete with the repos, tools, and
  4621. 3:48:56dependencies that they that they need to
  4622. 3:48:58get started with.
  4623. 3:49:00This system is governable, so there's
  4624. 3:49:01one control plane that orchestrates all
  4625. 3:49:04of these sandboxes together wherever you
  4626. 3:49:06need them.
  4627. 3:49:08They can run inside of your own VPC, on
  4628. 3:49:10on EC2, Kubernetes, whatever compute you
  4629. 3:49:12already operate. So your code, your
  4630. 3:49:14data, and your network are in your
  4631. 3:49:16control.
  4632. 3:49:17Compute becomes pluggable, just like we
  4633. 3:49:19made the BYOK options and auto model
  4634. 3:49:22options that we provide make that the
  4635. 3:49:24provider sort of like this like model
  4636. 3:49:26pluggable layer.
  4637. 3:49:28So now we had agents that were cheap
  4638. 3:49:30enough to run all the time
  4639. 3:49:31and the compute to run many at once.
  4640. 3:49:34And so then the next question was how do
  4641. 3:49:35we orchestrate all of this work?
  4642. 3:49:38At first we tried making like this DAG,
  4643. 3:49:40very complex system of agents composed
  4644. 3:49:43together by like these very like strong
  4645. 3:49:45rules, and it didn't really work. You
  4646. 3:49:47end up having a lot of the same problems
  4647. 3:49:49that plagued missions in the first place
  4648. 3:49:51because it was too rigid, and as a
  4649. 3:49:53result it was too expensive to run, and
  4650. 3:49:55it didn't generalize well.
  4651. 3:49:58So what we landed on instead is
  4652. 3:49:59something that we call a work stream.
  4653. 3:50:01It's a standing goal with a team of
  4654. 3:50:03agents to support it. It has this goal,
  4655. 3:50:05it has an identity, an environment that
  4656. 3:50:08it runs on, and memory.
  4657. 3:50:10A new work stream runs as you, and it
  4658. 3:50:12has your access. When it outgrows one
  4659. 3:50:15person, it can be moved to a service
  4660. 3:50:17account with its own credentials. So you
  4661. 3:50:19stay on as a member, but the work keeps
  4662. 3:50:21happening under an identity that the the
  4663. 3:50:23org actually controls.
  4664. 3:50:26At the center of every work stream is an
  4665. 3:50:28orchestrator, and its job looks a lot
  4666. 3:50:30like a scheduler in an OS.
  4667. 3:50:32Unlike the graph that we started with,
  4668. 3:50:34it assembles its agents in whatever way
  4669. 3:50:36is most natural for the model.
  4670. 3:50:38So the instructions are pretty short,
  4671. 3:50:40right? It's like delegate first, give
  4672. 3:50:42each agent an area of ownership, and
  4673. 3:50:44then for tasks that can fall into an
  4674. 3:50:46existing area of ownership, delegate to
  4675. 3:50:48those.
  4676. 3:50:50You start by talking to it,
  4677. 3:50:51Uh, just kind of hand things over as you
  4678. 3:50:53think of them, and then it fans out the
  4679. 3:50:55work so that then uh you can get the the
  4680. 3:50:58control back to you.
  4681. 3:51:00Uh, agents report back when they finish,
  4682. 3:51:02when they get blocked, when there's
  4683. 3:51:03something that changes the plan overall.
  4684. 3:51:07But the work stream doesn't just stop
  4685. 3:51:08there because it takes on more on its
  4686. 3:51:10own and it learns from your habits. At
  4687. 3:51:13some point the orchestrator may ask you
  4688. 3:51:14how you decide what to work on, and it
  4689. 3:51:16learns that maybe bugs are coming from
  4690. 3:51:18two Slack channels.
  4691. 3:51:21So here it can set up an intake at a
  4692. 3:51:23triage listener.
  4693. 3:51:24Every 30 minutes it can go in and pull
  4694. 3:51:27in those reports from those channels,
  4695. 3:51:28dedupe them against, you know, existing
  4696. 3:51:30linear issues or uh sentry sentry alerts
  4697. 3:51:33that already exist, and then weigh each
  4698. 3:51:35of those against the work stream's goal.
  4699. 3:51:38So a Slack complaint and a customer
  4700. 3:51:40error report become one change.
  4701. 3:51:43The output of triage is a spec that you
  4702. 3:51:45review and uh you're capable of like
  4703. 3:51:49weighing in on before any code gets
  4704. 3:51:50written.
  4705. 3:51:51Once you approve a spec, the same worker
  4706. 3:51:53opens a PR,
  4707. 3:51:55stewards it until it's merge ready, uh
  4708. 3:51:57and then make sure that, you know,
  4709. 3:51:58review uh comments and CI are passing.
  4710. 3:52:03So each one of those stages actually
  4711. 3:52:05maps to an autonomy level in this
  4712. 3:52:06system.
  4713. 3:52:08At L1 the work stream pauses for human
  4714. 3:52:10review at the spec creation, at the PR,
  4715. 3:52:14and before hitting merge.
  4716. 3:52:16But, you know, as you level up that the
  4717. 3:52:18those levels of autonomy you can get to
  4718. 3:52:21L4 where you're auto merging everything
  4719. 3:52:23by default.
  4720. 3:52:24The owner gets to set the level,
  4721. 3:52:26but the confidence to raise it comes
  4722. 3:52:28from watching the work.
  4723. 3:52:29So when proposed changes are accurate or
  4724. 3:52:32PRs consistently merge at the quality
  4725. 3:52:34that you'd expect, that's actually your
  4726. 3:52:36cue to move to the next checkpoint.
  4727. 3:52:38And so that becomes the dial for you.
  4728. 3:52:40You don't need to decide on day one
  4729. 3:52:41whether you trust a coding agent to own
  4730. 3:52:43you know you like CI health work stream
  4731. 3:52:45off the bat.
  4732. 3:52:47You just need to trust it to to like
  4733. 3:52:49unlock the next checkpoint.
  4734. 3:52:52So in a factory an agent that starts
  4735. 3:52:54from scratch repeats the same mistakes
  4736. 3:52:56the last one already did. And so we
  4737. 3:52:58built work streams to have a shared file
  4738. 3:53:00system.
  4739. 3:53:02Agents are are encouraged to keep memory
  4740. 3:53:05and version skills and so their PRs get
  4741. 3:53:08better over time. They're also proactive
  4742. 3:53:10and they go looking for context that
  4743. 3:53:11they're missing. So for example, say
  4744. 3:53:14that your work stream notices that other
  4745. 3:53:15people are shipping related changes.
  4746. 3:53:18It sets up an automation that pulls in
  4747. 3:53:20their PRs,
  4748. 3:53:21analyzes what they've been working on
  4749. 3:53:23and what comments they've been receiving
  4750. 3:53:25in those PRs and then incorporates them
  4751. 3:53:27into their memory and skills.
  4752. 3:53:29So what what agent learns others can use
  4753. 3:53:32and nothing starts cold.
  4754. 3:53:36That folder is also how agents work
  4755. 3:53:38together on a project. When the
  4756. 3:53:40orchestrator spawns an agent, it's
  4757. 3:53:42capable of telling, you know, each agent
  4758. 3:53:44where does it where where do where does
  4759. 3:53:46its results sort of belong inside of
  4760. 3:53:47this shared folder.
  4761. 3:53:49And so every agent can read and write to
  4762. 3:53:52the same place.
  4763. 3:53:54If you ask it a a work stream why
  4764. 3:53:56checkout got slower, for example, it can
  4765. 3:53:58send dozens of agents out to track out
  4766. 3:54:00all of those different leads that you
  4767. 3:54:01have and then ultimately each one can
  4768. 3:54:04write its findings to the folder and the
  4769. 3:54:06orchestrator can merge them.
  4770. 3:54:09Now you still need a barrier between
  4771. 3:54:10implementation and merging and for us
  4772. 3:54:13that barrier is platform level
  4773. 3:54:14validation.
  4774. 3:54:15Every PR gets code review, security
  4775. 3:54:18review, and dedicated QA and a live
  4776. 3:54:20preview.
  4777. 3:54:21And so once every review thread is
  4778. 3:54:23resolved, a risk judge then goes in and
  4779. 3:54:25decides whether it's safe to approve or
  4780. 3:54:27not.
  4781. 3:54:28The judge gets the exact diff and a
  4782. 3:54:30checkout of the branch so it can like
  4783. 3:54:31build and test it. Uh but ultimately it
  4784. 3:54:34reads its policy from our mainline
  4785. 3:54:35branch so that the change can't rewrite
  4786. 3:54:37the rules that it's being judged by.
  4787. 3:54:40Every week Android mines the review
  4788. 3:54:42comments and goes through and figures
  4789. 3:54:44out which one authors have actually
  4790. 3:54:46acted on and then pulls them pull pulls
  4791. 3:54:49those into like review guidelines so
  4792. 3:54:51that the the loop sort of improves over
  4793. 3:54:53time.
  4794. 3:54:54We also have droids that every day run
  4795. 3:54:56and analyze, revert, hot fixes, new
  4796. 3:54:58century errors, for example, and traces
  4797. 3:55:01them back to the PRs that that caused
  4798. 3:55:02them. And so that also is a signal into
  4799. 3:55:04this same loop.
  4800. 3:55:07What we're really hill climbing on here
  4801. 3:55:09is the regret-free merge rate, the share
  4802. 3:55:12of changes that merge and never need a
  4803. 3:55:14revert or a follow-up fix.
  4804. 3:55:16We measure it for Droid-written and
  4805. 3:55:18human-written PRs side by side. And it's
  4806. 3:55:20the same way that we let like
  4807. 3:55:21self-driving cars onto the road. We
  4808. 3:55:23didn't just give them the green light.
  4809. 3:55:25We only did so once they crashed less
  4810. 3:55:28often than human-driven ones did.
  4811. 3:55:30And so once AI-written changes break
  4812. 3:55:32things less often than human ones and
  4813. 3:55:34you can actually measure that, then you
  4814. 3:55:36can let them merge. And that's the bar
  4815. 3:55:38for turning a work stream to L4.
  4816. 3:55:41Work stream is now like the platform
  4817. 3:55:43that we build new features on top of. So
  4818. 3:55:45migrate is the first one. It sort of
  4819. 3:55:47fulfills a niche in our product
  4820. 3:55:50like framework for for what missions
  4821. 3:55:52used to be. There's like dedicated
  4822. 3:55:54one-shot migration experience. And
  4823. 3:55:56continuous remediation is the second
  4824. 3:55:58where it follows that circular shape
  4825. 3:56:00that work streams are best suited for
  4826. 3:56:01because they are the work never ends.
  4827. 3:56:06We run factory on top of work streams
  4828. 3:56:08ourselves. And so this is what it
  4829. 3:56:10actually looks like in production.
  4830. 3:56:12In a week in this past October, we
  4831. 3:56:13shipped about 800 pull requests in our
  4832. 3:56:16dev branch. And Droid wrote 99% of them
  4833. 3:56:18and reviewed all of them.
  4834. 3:56:20Now work landed in one of two ways. It
  4835. 3:56:22was either human picking up a signal
  4836. 3:56:24manually
  4837. 3:56:25or increasingly nowadays it's a work
  4838. 3:56:28stream picking it up
  4839. 3:56:29automatically, triaging it against the
  4840. 3:56:31goal of the work stream, and then
  4841. 3:56:33ultimately writing a spec and merging
  4842. 3:56:35it.
  4843. 3:56:3752% of PRs that we ship right now are
  4844. 3:56:39coming from work streams themselves. And
  4845. 3:56:41some of these are operating on full
  4846. 3:56:43autonomy. So for example, the one that
  4847. 3:56:46fixes bug reports from a dedicated Slack
  4848. 3:56:48channel merges 53 PRs per week. Another
  4849. 3:56:51one is our flaky test work stream, which
  4850. 3:56:53merges about 40.
  4851. 3:56:56Fully autonomous PRs aren't really the
  4852. 3:56:58only thing worth tracking though.
  4853. 3:57:00It What matters a lot more is the
  4854. 3:57:02progression.
  4855. 3:57:03How much of your work flows through work
  4856. 3:57:04streams and how many checkpoints does
  4857. 3:57:06each one still need?
  4858. 3:57:08As more work flows unrestricted, your
  4859. 3:57:10engineers become the regulators of this
  4860. 3:57:11system. They decide, you know, what
  4861. 3:57:13signals to watch, uh what are the goals
  4862. 3:57:16and the owners of those, what are the
  4863. 3:57:17skills and review guidelines that you
  4864. 3:57:19need in order to keep this code high
  4865. 3:57:20quality, and how much autonomy each work
  4866. 3:57:23stream gets.
  4867. 3:57:25And so there's just one question that
  4868. 3:57:26tells you, you know, whether whether you
  4869. 3:57:28have a software factory or faster
  4870. 3:57:30engineers, and that is can you name and
  4871. 3:57:32track your signals?
  4872. 3:57:34Every team already has them. You have
  4873. 3:57:35bug reports in Slack, you know, failing
  4874. 3:57:37CI jobs, error spikes, tickets that
  4875. 3:57:40nobody ever picks up, but most of them
  4876. 3:57:42wait on a person to notice.
  4877. 3:57:44And so my ask to all of you is to write
  4878. 3:57:46them down and then pick the one that you
  4879. 3:57:48waste the most time on.
  4880. 3:57:50Give it to a work stream on L1 and then
  4881. 3:57:52set up our validation experience, and
  4882. 3:57:54then you're well on your way to building
  4883. 3:57:56software that improves itself with all
  4884. 3:57:58the signals that you choose to track.
  4885. 3:58:01Thanks everyone, and I'll be around to
  4886. 3:58:02answer any questions today.
  4887. 3:58:04>> [applause]
  4888. 3:59:46>> Hey guys, this is Era. I'm a founding
  4889. 3:59:49engineer at Klein and today I'm going to
  4890. 3:59:51be talking about recursively
  4891. 3:59:52self-improving coding agents. So
  4892. 3:59:55this is a very
  4893. 3:59:57interesting conversation in the sense
  4894. 3:59:59that everyone wants to understand what
  4895. 4:00:01RSI is, how we accomplish it, how we go
  4896. 4:00:03through it and how it affects our
  4897. 4:00:04career. So I'm going to start with
  4898. 4:00:06giving you just three goals of this
  4899. 4:00:08entire conversation, just three simple
  4900. 4:00:09goals and that would help you understand
  4901. 4:00:13like what what we get out of it and what
  4902. 4:00:14the future for this looks like. So the
  4903. 4:00:16goal number first is that understanding
  4904. 4:00:18the intent of Frontier Labs, right?
  4905. 4:00:20Everyone who's outside of the Frontier
  4906. 4:00:21Labs is always trying to speculate like,
  4907. 4:00:23hey, what what is the topic in OpenAI or
  4908. 4:00:25what is Deep Mind going to come out with
  4909. 4:00:26or what what they're trying to make?
  4910. 4:00:28And what's the future going to look
  4911. 4:00:30like? Who has cracked the RSI? So like
  4912. 4:00:32this kind of helps you get get a rough
  4913. 4:00:34outline of what their roadmap looks
  4914. 4:00:36like. And the second thing is that like
  4915. 4:00:38if the models are improving so fast,
  4916. 4:00:40like how do you think about your career
  4917. 4:00:42or the next few years, right? That is
  4918. 4:00:44really what we want to understand. We
  4919. 4:00:45want to understand like, okay,
  4920. 4:00:47things are changing so fast, like the
  4921. 4:00:49ground underneath us is shifting so
  4922. 4:00:51quickly. Right now, like it's like
  4923. 4:00:55there used to be a time when you could
  4924. 4:00:56like
  4925. 4:00:57demonstrate your side projects on your
  4926. 4:01:00personal website or on GitHub. And now
  4927. 4:01:02it's just like you could literally one
  4928. 4:01:03shot them using any frontier coding
  4929. 4:01:05agent.
  4930. 4:01:06And with that, it's like it's like oh,
  4931. 4:01:08how do we demonstrate proof of work? How
  4932. 4:01:10do we demonstrate, you know, that that
  4933. 4:01:12we know what we're doing and how do we
  4934. 4:01:14think of our career for the next coming
  4935. 4:01:16years, right? So, it's very interesting
  4936. 4:01:18times that we're living in. And then the
  4937. 4:01:20third and the most important goal of the
  4938. 4:01:22conversation is escaping the permanent
  4939. 4:01:23underclass, right? So, I know this seems
  4940. 4:01:27like a joke, but this really is a very
  4941. 4:01:28valid possibility that people are
  4942. 4:01:31contemplating. Everyone's like, oh,
  4943. 4:01:32well, if everything that we want to be
  4944. 4:01:34doing can potentially easily done by a
  4945. 4:01:37frontier model, like what are we what
  4946. 4:01:38are we good for? And like where does
  4947. 4:01:40software go? Where where where does the
  4948. 4:01:41future of technology going to go? And
  4949. 4:01:43what are we going to do as as things
  4950. 4:01:44evolve so fast, right? So, I think that
  4951. 4:01:47there's like a lot of like
  4952. 4:01:49fearmongering, but also there's like
  4953. 4:01:50some very valid merits where people are
  4954. 4:01:53concerned that like if all the revenue
  4955. 4:01:55of the planet is going to go to um
  4956. 4:01:58a few hardware companies that make
  4957. 4:02:00frontier GPUs and then a few few uh AI
  4958. 4:02:03companies that that make the best
  4959. 4:02:05models, like
  4960. 4:02:06how are we how are the rest of us going
  4961. 4:02:08to survive and what what are we going to
  4962. 4:02:09do here? So, I think I'll I'll walk you
  4963. 4:02:12through a few examples and a few ideas
  4964. 4:02:13that will like really really help you
  4965. 4:02:15reconsider the career choices and and
  4966. 4:02:18and the life choices you're making and
  4967. 4:02:20what your problems you're picking up.
  4968. 4:02:22And then and then that would help you
  4969. 4:02:23like realign
  4970. 4:02:25what it is that we should be working on
  4971. 4:02:27and what it is that you, you know, would
  4972. 4:02:29would
  4973. 4:02:30that you could work on that will stand
  4974. 4:02:31the test of time and help you both just
  4975. 4:02:33not just like on a psychological but
  4976. 4:02:35like financial and other levels as well.
  4977. 4:02:36So,
  4978. 4:02:37uh to give you a very precise example,
  4979. 4:02:39let's start with something very precise
  4980. 4:02:40that I'd done. So, uh a couple months
  4981. 4:02:42ago, I uh came up with recursive super
  4982. 4:02:47improvements on AI coding harness. So,
  4983. 4:02:50this was the first a public example of
  4984. 4:02:53uh an automatic evaluation improvement.
  4985. 4:02:55So, to give an example, uh we [snorts]
  4986. 4:02:57used to do this thing where we would
  4987. 4:02:58measure
  4988. 4:03:00the quality of a coding agent using
  4989. 4:03:02evaluation scores. So,
  4990. 4:03:04on terminal 2.1, Kimi K 3 with Klein
  4991. 4:03:07would score um usually like you know
  4992. 4:03:1060 70% and then depending on like how
  4993. 4:03:13good the model is like you get better
  4994. 4:03:15scores on thermal bench 2.1 so we
  4995. 4:03:18noticed that when K3 came out our score
  4996. 4:03:20was 77.5% which is a good score but we
  4997. 4:03:22were like oh is this is this truly is
  4998. 4:03:25this truly the best score that this
  4999. 4:03:26model can get on this benchmark and also
  5000. 4:03:29like what is the model release paper
  5001. 4:03:31claim about what their best score is so
  5002. 4:03:33the model's release paper said that they
  5003. 4:03:35could score up to 85% or even more so we
  5004. 4:03:37were just like okay okay so we are not
  5005. 4:03:40getting the sort of score and often what
  5006. 4:03:41we do with model releases incline is
  5007. 4:03:43that we try to like optimize your score
  5008. 4:03:45and try to get the best out of the
  5009. 4:03:46model.
  5010. 4:03:47So
  5011. 4:03:48when we started hill climbing on the on
  5012. 4:03:51the model like we we realized this we
  5013. 4:03:53realized that like you know there's a
  5014. 4:03:55lot of steps that we do such that we do
  5015. 4:03:57it gracefully such that we pursue the
  5016. 4:03:59right kind of problems and do it the
  5017. 4:04:00right way and we started thinking like
  5018. 4:04:03okay if we there's a lot of groundwork
  5019. 4:04:05here how do we automate it and this is a
  5020. 4:04:07demonstration of like how we automated
  5021. 4:04:09it so I'll I'll just leave that as a
  5022. 4:04:11placeholder for now and we'll get get
  5023. 4:04:12back to it later where we talk about in
  5024. 4:04:14detail of like how does like this
  5025. 4:04:16automatic improvement loop of a client
  5026. 4:04:18harness work so this basically means
  5027. 4:04:20that like you have a coding agent
  5028. 4:04:21harness that could just like basically
  5029. 4:04:22improve itself and become better on on
  5030. 4:04:25the same set of problems.
  5031. 4:04:26So let's start with the ground terms
  5032. 4:04:28what is your recursive self from
  5033. 4:04:29recursive self-improvement is the idea
  5034. 4:04:31that an AI system that helps build
  5035. 4:04:33[snorts] a more
  5036. 4:04:34capable successor which will piece the
  5037. 4:04:36process you know until you heat
  5038. 4:04:38artificial general intelligence right
  5039. 4:04:40you have better systems you have better
  5040. 4:04:41improvements you have a better system
  5041. 4:04:42and then you know you have singularity.
  5042. 4:04:44So a classic example is like a
  5043. 4:04:46self-improving credit coding agent right
  5044. 4:04:48self-improving coding agent is just like
  5045. 4:04:49a coding agent that can improve itself
  5046. 4:04:51so
  5047. 4:04:52you know this this is all like a pie in
  5048. 4:04:54the sky idea like it's like it's like
  5049. 4:04:55you know yeah sure this looks cool but
  5050. 4:04:57how did how did this even come out to be
  5051. 4:04:58like where how did we even get to here.
  5052. 4:05:01So
  5053. 4:05:02if you notice in topic short map in 2021
  5054. 4:05:04in 2021's roadmap they very specifically
  5055. 4:05:07said that
  5056. 4:05:09they want to a general AI assistant that
  5057. 4:05:11can interact conversationally with a
  5058. 4:05:13user to create, add and explain
  5059. 4:05:15software. It would display all the
  5060. 4:05:17capabilities of a software engineer
  5061. 4:05:18colleague. You would write functions,
  5062. 4:05:20you would identify bugs, you would
  5063. 4:05:21design an API, discuss style and best
  5064. 4:05:23practices, right? So we don't expect to
  5065. 4:05:25reach this goal right away, but it's in
  5066. 4:05:26the direction which we are striving.
  5067. 4:05:28It's interesting how this was more than
  5068. 4:05:30five years ago. More than five years
  5069. 4:05:31ago, people had predicted that there
  5070. 4:05:34does exist a day where you will have
  5071. 4:05:35like a full-blown general AI assistant
  5072. 4:05:37that can basically act like a colleague,
  5073. 4:05:39which we do have right now.
  5074. 4:05:41Then in 2022, ChatGPT got released. We
  5075. 4:05:43all know how that happened. It was it
  5076. 4:05:45was a fun moment. It was like the first
  5077. 4:05:47time things got sparked for us.
  5078. 4:05:50Then in 2023, 2024, that was the first
  5079. 4:05:52time chats
  5080. 4:05:54models could call functions. So these
  5081. 4:05:55functions would be simple functions
  5082. 4:05:57like, hey, what's the weather? Or you
  5083. 4:05:59know, what's the what's you know, what's
  5084. 4:06:01this computation or what this
  5085. 4:06:03computations result in Python?
  5086. 4:06:05And then finally in 2024 when
  5087. 4:06:08was released, that was when the agents
  5088. 4:06:10could like truly call like the actual
  5089. 4:06:12tools, the way we call like think of
  5090. 4:06:14tools. So it's like reading a file,
  5091. 4:06:15editing a file, you know, reading,
  5092. 4:06:18searching something on web, listing
  5093. 4:06:20listing a folder, searching and grabbing
  5094. 4:06:21through, you know, a file system, things
  5095. 4:06:24like that. So
  5096. 4:06:26and in 2024, we had the intuition that
  5097. 4:06:28like chatbot, what if chatbots could
  5098. 4:06:30actually talk to each other? You know,
  5099. 4:06:32so then that was the first time when we
  5100. 4:06:33had like, you know,
  5101. 4:06:35multi-agent conversations. Now at the
  5102. 4:06:37time, these things were very janky,
  5103. 4:06:38right? Like they were just like taped
  5104. 4:06:40together in this like weird
  5105. 4:06:42weird
  5106. 4:06:43bizarre ways where you have just like
  5107. 4:06:45these classes and these links and then
  5108. 4:06:47like these hierarchies of classes and it
  5109. 4:06:49was like it was all like kind of like a
  5110. 4:06:50very janky attempt to like make us think
  5111. 4:06:53that like these these models could like
  5112. 4:06:55have this pipeline of flows of responses
  5113. 4:06:56that can all communicate with each
  5114. 4:06:57other.
  5115. 4:06:58[snorts] But a lot of it was like mostly
  5116. 4:07:00deterministic and it was like even you
  5117. 4:07:02know, it was not as free-flowing as it
  5118. 4:07:03is right now. So
  5119. 4:07:05and you know, we started getting the
  5120. 4:07:07intuition. So this was the autogen
  5121. 4:07:09project, but like in only 2025 we we
  5122. 4:07:11started [snorts] getting the intuition
  5123. 4:07:12like what if the agents just kept
  5124. 4:07:13getting better and faster and cheaper
  5125. 4:07:15what what if that just kept happening
  5126. 4:07:17and then in 2026 uh we we had the
  5127. 4:07:20intuition that like what if what if the
  5128. 4:07:22agent could improve itself which is like
  5129. 4:07:24the final you know the final part of the
  5130. 4:07:27whole process so let me give an example
  5131. 4:07:29like what does it mean for an agent to
  5132. 4:07:30improve itself right so like it's like a
  5133. 4:07:32lot of times when we think of like
  5134. 4:07:34evaluating an agent we initially used to
  5135. 4:07:37think of evaluating an agent in the form
  5136. 4:07:38of like just a single turn conversation
  5137. 4:07:40so we would tell we would tell like um
  5138. 4:07:44um agents that like ask a simple
  5139. 4:07:45question like how many toes does a dash
  5140. 4:07:48animal has like it could be let's say
  5141. 4:07:50how many toes does a cat have and then
  5142. 4:07:52LLM would go through and the LLM would
  5143. 4:07:53like prepare a response and then we
  5144. 4:07:55would grade the response so like a cat
  5145. 4:07:56has maybe 10 toes 18 toes whatever many
  5146. 4:07:59I don't I actually don't know how many
  5147. 4:08:00toes a cat has but anyway
  5148. 4:08:02um
  5149. 4:08:04yeah so then we create the responses
  5150. 4:08:06like a yes or no but like when you work
  5151. 4:08:07with an agent there's a ton of things
  5152. 4:08:09that you're doing so let me give an
  5153. 4:08:11example let me give an example so for
  5154. 4:08:12example you can ask an agent like hey
  5155. 4:08:14set up this MCP for me so now it's not
  5156. 4:08:17just like it's not just like you know a
  5157. 4:08:19trivial one shottable like thing like it
  5158. 4:08:21will like search online to go what's the
  5159. 4:08:23MCP what is the config so I'll like edit
  5160. 4:08:26and download some soft files like maybe
  5161. 4:08:28I need to add database update the MCP
  5162. 4:08:30tool maybe I need to read and edit some
  5163. 4:08:32files maybe I need to run some bash
  5164. 4:08:34commands right so you do all of that and
  5165. 4:08:36then the agent responds like hey I did
  5166. 4:08:38it and then you grade that with a bunch
  5167. 4:08:40of tests to gauge like hey did the agent
  5168. 4:08:42actually accomplish the thing that it
  5169. 4:08:44said it would do
  5170. 4:08:45so to do that thing the second part
  5171. 4:08:48which is like more intricate where
  5172. 4:08:49you're actually like truly you know
  5173. 4:08:52testing agents like to their best extent
  5174. 4:08:54like it's like you know um
  5175. 4:08:57we use this thing called terminal bench
  5176. 4:08:58terminal bench is a benchmark for you
  5177. 4:09:01know AI agents in terminal environments
  5178. 4:09:02and that is something that we are we
  5179. 4:09:04have a lot of like experience with a
  5180. 4:09:06client and it's like a it's like one of
  5181. 4:09:08the best ways to like really gauge like
  5182. 4:09:10is your coding agent accomplishing in
  5183. 4:09:13general the tasks that you're doing
  5184. 4:09:15that you want to be doing with the HR
  5185. 4:09:16model. So, there's a couple there's many
  5186. 4:09:19different parts of a like evaluation
  5187. 4:09:20suite. So, ideally you want that to run
  5188. 4:09:24in
  5189. 4:09:25you know [snorts] you you want that to
  5190. 4:09:26run in individual sandboxes. Then you
  5191. 4:09:29have like independent graders that like
  5192. 4:09:31check everything like how many turns it
  5193. 4:09:33took, how many tokens it took, how much
  5194. 4:09:35latency it took, how many how many how
  5195. 4:09:37many tool calls did it make, what was
  5196. 4:09:39the trajectory looking like, and then
  5197. 4:09:40grading all the results for things like
  5198. 4:09:42hey did you not reward a hack, did you
  5199. 4:09:43not cheat through the solution, did you
  5200. 4:09:44not just like look up the solution
  5201. 4:09:46online, things like that.
  5202. 4:09:48So, we we we're basically like a lot of
  5203. 4:09:51these data sets of evaluation problems
  5204. 4:09:54have a set of tasks. So, when is a
  5205. 4:09:56terminal bench 2.1 they have 89 tasks.
  5206. 4:09:58So, what we do is like we spin up 89
  5207. 4:10:00parallel containers where they all run
  5208. 4:10:03online and they run in parallel in their
  5209. 4:10:05sandbox and then they they each try to
  5210. 4:10:07solve a problem. So, then they're all
  5211. 4:10:10in parallel and independent of each
  5212. 4:10:11other and then at the end we get get get
  5213. 4:10:14a score. So, our score could like give
  5214. 4:10:16us measures like is is a
  5215. 4:10:18is this is the model performing well and
  5216. 4:10:21what kind of failures it has, right? So,
  5217. 4:10:22like you run the evaluation, you get a
  5218. 4:10:24score. So, you get an aggregate score
  5219. 4:10:25and this
  5220. 4:10:26aggregate score could reveal a set of
  5221. 4:10:28failures. Failures could be of any kind.
  5222. 4:10:30Failures could be hey the
  5223. 4:10:32read file tool is not working. Sometimes
  5224. 4:10:34the inference is so slow that like you
  5225. 4:10:36know the agent's working, it's doing,
  5226. 4:10:37it's making progress, maybe it's running
  5227. 4:10:39in circle, but it's so slow that it
  5228. 4:10:40literally like runs out of time, it runs
  5229. 4:10:42out of like 45 minutes or whatever many
  5230. 4:10:44that were given for the task and just
  5231. 4:10:45like stops working. So,
  5232. 4:10:47once you do that, once you understand
  5233. 4:10:49hey this is what we ran, this is what
  5234. 4:10:50the failures we got, we bucket the
  5235. 4:10:52failures in different categories. So,
  5236. 4:10:54like 10 failures of you know this kind
  5237. 4:10:56of tool got 20 failures of like say
  5238. 4:10:58inference not working or something else,
  5239. 4:10:59right? And then we once we bucket the
  5240. 4:11:01failures into different categories where
  5241. 4:11:02like we try to make an improvement to
  5242. 4:11:04the harness because like
  5243. 4:11:05what if there's like something wrong
  5244. 4:11:07with the harness that we could change
  5245. 4:11:08such that the model score would improve,
  5246. 4:11:10right? And on and on and on and you do
  5247. 4:11:12this iteratively, it takes a while. It's
  5248. 4:11:13just like you know hill climbing is like
  5249. 4:11:15a very
  5250. 4:11:16kind of process and you know usually
  5251. 4:11:19usually we took we used to take a lot of
  5252. 4:11:21time trying to do this but it was a
  5253. 4:11:23great way for us to like learn how to
  5254. 4:11:25improve models. So
  5255. 4:11:26[snorts] one way to hill climb on
  5256. 4:11:29else is that like
  5257. 4:11:31and you know we we could have a thing
  5258. 4:11:32where you know we started doing this
  5259. 4:11:35thing where we could just like
  5260. 4:11:37what if we could have like a long
  5261. 4:11:39running agent that runs for a while and
  5262. 4:11:40automatically improves the model score
  5263. 4:11:42and that's what we did. So we ran GPT
  5264. 4:11:455.6 soul that led the work and that was
  5265. 4:11:47like the main agent and then
  5266. 4:11:49K K 3 was just running so what it was
  5267. 4:11:51doing is that like GPT 5.6 was like hey
  5268. 4:11:54run these
  5269. 4:11:55get these scores let me look at the
  5270. 4:11:57failures let me look at where it where
  5271. 4:11:59it went wrong and then I'll make changes
  5272. 4:12:01to client's harness and then I'll run
  5273. 4:12:03these again and then I'll keep trying.
  5274. 4:12:05So again you can see how like very
  5275. 4:12:07quickly this can like go after rails
  5276. 4:12:09because this is supposed to be like the
  5277. 4:12:10running in evaluation itself takes like
  5278. 4:12:12an hour then you have to read the
  5279. 4:12:14feedback then you have to like see if
  5280. 4:12:16you're properly interpreting the
  5281. 4:12:17feedback to make reasonable changes to
  5282. 4:12:19the harness there are like not a word
  5283. 4:12:20hacky they're just like actual genuine
  5284. 4:12:22improvements and then run the again. So
  5285. 4:12:24we ran that for 17 hours and
  5286. 4:12:27when we ran that for 17 hours there was
  5287. 4:12:29like a legitimate hell climbing and what
  5288. 4:12:31we found that was very interesting this
  5289. 4:12:32happened with a very
  5290. 4:12:34detailed prompt that accomplished this
  5291. 4:12:36is that like we had baseline scores
  5292. 4:12:39percent and then
  5293. 4:12:41you know we we found issues like loop
  5294. 4:12:43detection loop detection is when a
  5295. 4:12:45harness just runs in circles around the
  5296. 4:12:46problem right we found like background
  5297. 4:12:49process grinding where background
  5298. 4:12:50processes would run too long
  5299. 4:12:52we found some file index issues and
  5300. 4:12:54other things as well I mean when I say
  5301. 4:12:56we what I've actually made is GPT 5.6
  5302. 4:12:58all right so that found those issues
  5303. 4:13:00that found and bucketed those issues is
  5304. 4:13:02like hey how about I make those changes
  5305. 4:13:04in the harness and I run the
  5306. 4:13:06and iteratively built the hill climbing
  5307. 4:13:08process completely autonomously on its
  5308. 4:13:10own without my interference and 17 hours
  5309. 4:13:12later when I come back to it it's like
  5310. 4:13:14it has improved like the harness's
  5311. 4:13:15performance in so many ways.
  5312. 4:13:17So see what does that mean what does
  5313. 4:13:19that mean? It just it just basically
  5314. 4:13:21means that like
  5315. 4:13:22we are at a point where the experience
  5316. 4:13:25could make the harness better. That like
  5317. 4:13:27there's like a two-level learning loop
  5318. 4:13:28where the harness does some action,
  5319. 4:13:31interacts with the environment,
  5320. 4:13:32understands whether or not it did the
  5321. 4:13:34right thing or the wrong thing, and gets
  5322. 4:13:36some reward and and like almost and then
  5323. 4:13:38has there's like a meta agent that's
  5324. 4:13:40observing the main harness and finding
  5325. 4:13:43the changes to it and just like
  5326. 4:13:45incorporating those changes and
  5327. 4:13:46improving. Now, it's I think in my
  5328. 4:13:48personal opinion, it's much harder to do
  5329. 4:13:49in real time because if you do it in
  5330. 4:13:51real time, you overindex on solving
  5331. 4:13:53certain problems or you're just like too
  5332. 4:13:56curtailed to a specific thing. But I do
  5333. 4:13:59think that this is something that works
  5334. 4:14:00like on a broader scale of like working
  5335. 4:14:02on a wide range of problems.
  5336. 4:14:03[snorts]
  5337. 4:14:04And you want to be able to like you want
  5338. 4:14:06you want to be able to justify the
  5339. 4:14:08changes you're making on a broader
  5340. 4:14:10applicability so that the changes you
  5341. 4:14:12make here will apply to a broader range
  5342. 4:14:14of problems as well. So,
  5343. 4:14:17what is that mean?
  5344. 4:14:19What am I trying to get at here?
  5345. 4:14:21The point I'm trying to get at is that
  5346. 4:14:23the models are so good right now that
  5347. 4:14:24they can if they can improve the
  5348. 4:14:25harness, they can improve a lot of other
  5349. 4:14:27things. So then what else can they
  5350. 4:14:28improve?
  5351. 4:14:29So this is this is a this is a post
  5352. 4:14:32primarily from OpenAI where GPT-5.6
  5353. 4:14:35autonomously re-wrote and optimized its
  5354. 4:14:37own production kernels and those
  5355. 4:14:39production kernel improvements removed
  5356. 4:14:41the end-to-end serving costs by 20%.
  5357. 4:14:44You know, it also had 15% higher token
  5358. 4:14:47generation efficiency and this was like
  5359. 4:14:49completely done by 5.6 on itself, right?
  5360. 4:14:53So what that means is that like
  5361. 4:14:54inference is becoming much cheaper and
  5362. 4:14:56that means that like if on if if for a
  5363. 4:14:59company like OpenAI, if you're having
  5364. 4:15:0120% lower lower serving cost for a
  5365. 4:15:03frontier model, that could be a billions
  5366. 4:15:06of dollars of improvement in price and
  5367. 4:15:07you could you could do that as well for
  5368. 4:15:09your own models.
  5369. 4:15:11Another interesting thing is that like
  5370. 4:15:13when people have kernel writing
  5371. 4:15:15competitions, so Core Auto is a company
  5372. 4:15:16which has a kernel writing competitions
  5373. 4:15:19and you'll notice that like they they
  5374. 4:15:21mentioned that like
  5375. 4:15:22the person who won the kernel writing
  5376. 4:15:24competition, they explicitly said that
  5377. 4:15:26I've never handwritten a GPU operator.
  5378. 4:15:28It's all
  5379. 4:15:29and agent driven.
  5380. 4:15:31So notice that like how like the best
  5381. 4:15:34inference improvements, kernel
  5382. 4:15:35improvements are at this point
  5383. 4:15:36completely exclusively done by a swarm
  5384. 4:15:38of agents or generally just like agents
  5385. 4:15:41doing some magic to you know magically
  5386. 4:15:43improve like the kernel kernel code and
  5387. 4:15:45how faster inference, cheaper inference
  5388. 4:15:47um
  5389. 4:15:48yeah.
  5390. 4:15:49So basically like the improvement
  5391. 4:15:51surface that you think of is not just
  5392. 4:15:54like the harness improvement. It's
  5393. 4:15:55literally the whole surface stack,
  5394. 4:15:57right? So the whole stack from like
  5395. 4:15:59evaluation to harness improvements to
  5396. 4:16:01training data to post training to world
  5397. 4:16:03interaction to inference. Like literally
  5398. 4:16:05the entire
  5399. 4:16:06part of the pipeline. Like all the way
  5400. 4:16:08from like electricity going through your
  5401. 4:16:11machine to like making an error request,
  5402. 4:16:14going through the model, models doing
  5403. 4:16:15inference and making a response and
  5404. 4:16:18getting back to you. Like every single
  5405. 4:16:19part of the pipeline we can optimize it.
  5406. 4:16:21So the whole RSI loop has like um
  5407. 4:16:23there's a couple of different parts. So
  5408. 4:16:25it's like pre-training
  5409. 4:16:27um Nimo training examples of like you
  5410. 4:16:28know AI curated pre-training data that
  5411. 4:16:31was good. Um there's like post training
  5412. 4:16:32examples which are
  5413. 4:16:35you know where where there's like
  5414. 4:16:36constitutional AI, examples of like post
  5415. 4:16:38training improvements then inference. I
  5416. 4:16:41gave you the example of GPT 5.6. So for
  5417. 4:16:44and generation I gave you our own
  5418. 4:16:45example of how we improved ourselves on
  5419. 4:16:47our own
  5420. 4:16:49Um in general what you want to think of
  5421. 4:16:51is that like anytime a model is trying
  5422. 4:16:53to do something and it's like not able
  5423. 4:16:54to do it or not able to accomplish it.
  5424. 4:16:56So for example like imagine you're
  5425. 4:16:58having a long conversation where you ask
  5426. 4:17:00the model something obvious. So like
  5427. 4:17:01it's like
  5428. 4:17:02or something that you think is not
  5429. 4:17:05necessarily obvious but it's like
  5430. 4:17:07it's hard but you would assume that the
  5431. 4:17:08model could potentially do it. So to
  5432. 4:17:10give an example, let's say you want to
  5433. 4:17:12make add a new column to a database and
  5434. 4:17:15then fill that column, right?
  5435. 4:17:17And then put a cache in front of it. So
  5436. 4:17:19what happens is that it does the whole
  5437. 4:17:21thing but for some reason the way it
  5438. 4:17:22pushes the cache in front of the whole
  5439. 4:17:24database like it kind of doesn't do it
  5440. 4:17:25right. So, at that point you're like, oh
  5441. 4:17:27no,
  5442. 4:17:28set this cache differently, set this as
  5443. 4:17:29the TTL, and then you give like a
  5444. 4:17:31correction signal, and then that kind of
  5445. 4:17:33like fixes your problem and you kind of
  5446. 4:17:35go about your day. But
  5447. 4:17:36the way I want to think of it is that I
  5448. 4:17:38want to think of like every single time
  5449. 4:17:40like a human correction to the digital
  5450. 4:17:42landscape is a part of a delta that
  5451. 4:17:46needs to be covered. And what that means
  5452. 4:17:49is that like every single time you're
  5453. 4:17:51finding like the edge of model
  5454. 4:17:52capabilities of like how far it can push
  5455. 4:17:54for inference, how much it can push for
  5456. 4:17:55this, how much it can push for
  5457. 4:17:58you know, solving your problems,
  5458. 4:18:00you want to find that delta and you want
  5459. 4:18:02to like push that frontier a little bit
  5460. 4:18:04further, a little bit further. And
  5461. 4:18:06that's what post training does, right?
  5462. 4:18:07So, post training is is this is in that
  5463. 4:18:09post training. We find these like really
  5464. 4:18:12interesting, really hard, long horizon
  5465. 4:18:14problems for for coding agents or for
  5466. 4:18:16other kinds of problems, and then just
  5467. 4:18:17like slowly hill climb the model models
  5468. 4:18:20on on those kind of problems. So,
  5469. 4:18:22anything that has a verifiable loop
  5470. 4:18:24can be optimized. But
  5471. 4:18:26is this is not just limited to software.
  5472. 4:18:29Like literally like the physical world
  5473. 4:18:30itself could have a verifiable loop.
  5474. 4:18:33So, to give an example, like let me let
  5475. 4:18:35me give you a very funny kind of bizarre
  5476. 4:18:37example where like it's like you
  5477. 4:18:39you look at this and
  5478. 4:18:42you
  5479. 4:18:44um look at this like robot and it's it's
  5480. 4:18:47typing something and it's like really
  5481. 4:18:49really like interesting what the robot
  5482. 4:18:51is typing.
  5483. 4:19:01So, notice how like notice how the robot
  5484. 4:19:03is itself typing like how it wants to
  5485. 4:19:05like solve robotics. And imagine if the
  5486. 4:19:08robot itself could like interpret the
  5487. 4:19:10results of of like the answers and just
  5488. 4:19:13like keep improving, then that means
  5489. 4:19:14that like this is you know, basically
  5490. 4:19:16like an ASI robot. Another example that
  5491. 4:19:18I could give you from my
  5492. 4:19:20personal life is that like yeah, I have
  5493. 4:19:23this
  5494. 4:19:24chip that has the ability to see and has
  5495. 4:19:26a speaker and has a microphone inside of
  5496. 4:19:29it. So I was trying to set up this uh
  5497. 4:19:31with my machine and it was
  5498. 4:19:34um a gift that I had for my girlfriend
  5499. 4:19:36and what I found very interesting about
  5500. 4:19:39this is that as I was trying to set it
  5501. 4:19:40up it was very slow, it was slow to
  5502. 4:19:42respond, it had higher latency, it had
  5503. 4:19:44an agent in enough itself, right? So
  5504. 4:19:46what I did was I connected the step two
  5505. 4:19:48to my um computer
  5506. 4:19:50and I lit um
  5507. 4:19:53I lit um coding agents run on this
  5508. 4:19:55hardware and basically run on this
  5509. 4:19:58hardware
  5510. 4:19:59automatically make tests to it. So like
  5511. 4:20:02my computer would talk to my hardware,
  5512. 4:20:05give it some tests like hey do this, do
  5513. 4:20:07that and just like what do you see right
  5514. 4:20:08now? So
  5515. 4:20:10then it would notice the end-to-end
  5516. 4:20:11latency of like the inference response
  5517. 4:20:13of like how fast uh my hardware is
  5518. 4:20:15responding and then um what I found is
  5519. 4:20:18that like it was able to do the
  5520. 4:20:20breakdown of like every single case and
  5521. 4:20:22scenario where there was like higher
  5522. 4:20:24latency
  5523. 4:20:25um
  5524. 4:20:26slow response or or just the algorithm
  5525. 4:20:28wasn't right and I could then set up a
  5526. 4:20:31recursive loop where
  5527. 4:20:33both the firmware could be updated, the
  5528. 4:20:36um the tool calls, the inference model
  5529. 4:20:38used, um the speech-to-text model used,
  5530. 4:20:41all of those could be improved
  5531. 4:20:42automatically with like an open router
  5532. 4:20:44key which I'd given to a coding agent
  5533. 4:20:46and then overnight with self-running
  5534. 4:20:47experiments with the hardware uh my um
  5535. 4:20:51this tiny little piece of beautiful
  5536. 4:20:52hardware that I have was like deeply
  5537. 4:20:54optimized. So it was if this also like
  5538. 4:20:56even hardware could be a part of the RSI
  5539. 4:20:58loop. So what happens when you connect
  5540. 4:21:00these loops, right? That's the next
  5541. 4:21:01question. Like you could have cheaper
  5542. 4:21:03inference, you could have like more
  5543. 4:21:04experiments budget, you have like you
  5544. 4:21:06know better data and better models. So
  5545. 4:21:09is this single rarity?
  5546. 4:21:11So very question to ask and I think that
  5547. 4:21:12the part of the parts of the loop are
  5548. 4:21:13already right? Like we're we we are
  5549. 4:21:16really we really are very close to parts
  5550. 4:21:18of the RSI loop, right?
  5551. 4:21:20And what do we do? Like what do we do
  5552. 4:21:22about this and it's like how do we think
  5553. 4:21:24of that
  5554. 4:21:25in terms of like our career and how do
  5555. 4:21:27you think of that in terms of on know
  5556. 4:21:28our lives and what do we want to do
  5557. 4:21:29next. So,
  5558. 4:21:30here's my advice to everyone who is
  5559. 4:21:32watching this like, please please please
  5560. 4:21:34please move closer to the
  5561. 4:21:36right. So, it's like
  5562. 4:21:37right now like people are working on all
  5563. 4:21:39these like different kinds of software
  5564. 4:21:41which is like you know, AI agents for
  5565. 4:21:42you know, construction agents for this,
  5566. 4:21:44agents for that and it's like I think
  5567. 4:21:45that you're
  5568. 4:21:47you know,
  5569. 4:21:48well the way I see those things is that
  5570. 4:21:50like I think of those as like you're
  5571. 4:21:51missing the forest for the for the
  5572. 4:21:52trees. You really are and it's like you
  5573. 4:21:55have this like
  5574. 4:21:56very interesting godlike technology that
  5575. 4:21:59can change the course of humanity and
  5576. 4:22:01then
  5577. 4:22:02we're not ambitious enough to pursue the
  5578. 4:22:03best problems. We really are not and
  5579. 4:22:05what we're just like
  5580. 4:22:07I think that it's not just like
  5581. 4:22:09the problem isn't just that we're not
  5582. 4:22:10ambitious and we're also like what we're
  5583. 4:22:12pursuing
  5584. 4:22:13a lot of those things are just like one
  5585. 4:22:15model release away from like being
  5586. 4:22:17completely like wiped out, right? So
  5587. 4:22:18like
  5588. 4:22:19it's also like it's also like
  5589. 4:22:21it may not be that intellectually
  5590. 4:22:22stimulating because like all you're
  5591. 4:22:24essentially all we're essentially doing
  5592. 4:22:25is like sitting all day and it's like
  5593. 4:22:26hey cloud do this for me, hey cloud do
  5594. 4:22:27this for me and I don't know if that's
  5595. 4:22:29like I don't know if that's like the
  5596. 4:22:30most intellectually stimulating or like
  5597. 4:22:32or like um thoughtful use of a person's
  5598. 4:22:35time. Like maybe maybe you know, we're
  5599. 4:22:36supposed to be solving harder in more
  5600. 4:22:38more frontier problems and I think that
  5601. 4:22:40like it's like the ceiling is like our
  5602. 4:22:42ambition. So,
  5603. 4:22:43instead of like working on these tiny
  5604. 4:22:44things my suggestion to people is to
  5605. 4:22:46like you know,
  5606. 4:22:48do things that of course like
  5607. 4:22:50goes without saying but you should do
  5608. 4:22:51things you believe in and you know, I'm
  5609. 4:22:52just a guy on the internet but do things
  5610. 4:22:55that help you move closer to the R S A
  5611. 4:22:57loop, right? So like anything that's
  5612. 4:22:59just like the end-to-end pipeline of
  5613. 4:23:00like
  5614. 4:23:01hey this is where like the models are
  5615. 4:23:04lacking like it's like whether that's
  5616. 4:23:06inference, whether that's post training,
  5617. 4:23:07whether that's literally like physical
  5618. 4:23:08interaction, whether that could be you
  5619. 4:23:11know, send sending end-to-end pipelines
  5620. 4:23:13of truly any kind you know, whether
  5621. 4:23:15that's hardware problems, whether that's
  5622. 4:23:16um networking problems, whether that's
  5623. 4:23:18um building a sense of community, things
  5624. 4:23:20that just you know, um things that just
  5625. 4:23:22like help
  5626. 4:23:23um you as an engineer get closer to the
  5627. 4:23:27benefits of the improvements um in in um
  5628. 4:23:32um in in the model. So, this is a very
  5629. 4:23:34interesting slide from such a topic. So,
  5630. 4:23:38they mentioned that like
  5631. 4:23:40you want to
  5632. 4:23:41have like
  5633. 4:23:43this you're in this space where it's
  5634. 4:23:44like, you know, you only you only have
  5635. 4:23:45like one to five years left to
  5636. 4:23:46contribute to the world. What do you do,
  5637. 4:23:48right? So, it's like
  5638. 4:23:50they they great suggestions where like
  5639. 4:23:52prefer ambitious, targeted and fast
  5640. 4:23:54collaborations, right?
  5641. 4:23:55And, you know, in general like the time
  5642. 4:23:58scale, like all the timelines that you
  5643. 4:23:59give to yourself like they should be
  5644. 4:24:00just like completely squeezed out.
  5645. 4:24:02Should be completely squeezed out
  5646. 4:24:03because it's like a lot of times and you
  5647. 4:24:04also have to think through that like if
  5648. 4:24:06if the model if the problem that you're
  5649. 4:24:07working on is too ambitious, like it's
  5650. 4:24:09like is it probably
  5651. 4:24:11is it probably better to wait for just
  5652. 4:24:12like the next next model to come out or
  5653. 4:24:15not? Like that's that's something you
  5654. 4:24:16also have to wait through, right?
  5655. 4:24:19And you you ideally want to work on
  5656. 4:24:22problems that benefit from scale and you
  5657. 4:24:24know, you also at the same time want to
  5658. 4:24:25avoid projects that can be solved by
  5659. 4:24:26scale alone because then, you know, if
  5660. 4:24:28even if you like it's like if you're
  5661. 4:24:30working on a problem and then the next
  5662. 4:24:31model release comes out it solves the
  5663. 4:24:32problem. So, that's not that's probably
  5664. 4:24:33not a good thing to do. So, it's very
  5665. 4:24:35like you have to like think
  5666. 4:24:36philosophically of like what is the
  5667. 4:24:39endeavor that benefits greatly from
  5668. 4:24:41model improvements but like isn't also
  5669. 4:24:43completely wiped out from it and then
  5670. 4:24:45ideally like it's like
  5671. 4:24:46if the if the endeavor could also be a
  5672. 4:24:49part of the
  5673. 4:24:51like the improvement pipeline,
  5674. 4:24:53that could be amazing. So, you see a lot
  5675. 4:24:55of companies raising on say
  5676. 4:24:58making data pipelines and there's a lot
  5677. 4:25:00of money there. There's a
  5678. 4:25:02a lot of companies working on physical
  5679. 4:25:03hardware and stuff. So, I think if
  5680. 4:25:05you're a software engineer watching
  5681. 4:25:06these it's like I think that if you're
  5682. 4:25:08if you feel like you're like
  5683. 4:25:09here the delta of your intellect and
  5684. 4:25:12ability in your day-to-day it's like not
  5685. 4:25:14contributing
  5686. 4:25:15as in like you have the intellect, you
  5687. 4:25:17can make a difference but like if is
  5688. 4:25:19just like one shotting all those
  5689. 4:25:21problems like in your day-to-day and
  5690. 4:25:22you're just like kind of like becoming
  5691. 4:25:24which worth to think through take a step
  5692. 4:25:25back and think through it's like okay,
  5693. 4:25:27where are the problems where where
  5694. 4:25:29you're actually truly contributing and
  5695. 4:25:31that they will stand the test of time
  5696. 4:25:32and and those kind of reflection and
  5697. 4:25:34realignment now would I think like have
  5698. 4:25:37like
  5699. 4:25:37very interesting consequences for you in
  5700. 4:25:40the long term.
  5701. 4:25:42So yeah, so thank you so much, you know,
  5702. 4:25:44thank you so much for your time. Like I
  5703. 4:25:46think that
  5704. 4:25:47I think that if you're watching this and
  5705. 4:25:49if you've gone this far, I think that
  5706. 4:25:50you're definitely contemplating like one
  5707. 4:25:53of the most interesting improvements in
  5708. 4:25:54the course of humanity and you know,
  5709. 4:25:57there's many philosophical arguments
  5710. 4:25:58here. Some people might say, hey, you
  5711. 4:26:00know, I I think I maybe I'm off on the
  5712. 4:26:04timelines or maybe maybe I'm I'm just
  5713. 4:26:05being too pessimistic or maybe I'm being
  5714. 4:26:07too optimistic, but I would love to hear
  5715. 4:26:09all those thoughts. So this is you know,
  5716. 4:26:11my QR code. Please feel free to scan it
  5717. 4:26:13and let me know all your thoughts. Thank
  5718. 4:26:15you so much. Very kind of you to give me
  5719. 4:26:16your time.
  5720. 4:26:17>> Hi everyone. I am Tanmay.
  5721. 4:26:20Today joining me is Dolly and we will be
  5722. 4:26:23presenting the missing undo button for
  5723. 4:26:26coding agents.
  5724. 4:26:31So
  5725. 4:26:32imagine a coding agent working on a
  5726. 4:26:35repository issue.
  5727. 4:26:37It applies patch,
  5728. 4:26:39run the tests and everything passes. It
  5729. 4:26:42even creates a release tag,
  5730. 4:26:45but then a downstream integration fails.
  5731. 4:26:49Now here is the question.
  5732. 4:26:51Can the agent actually get everything
  5733. 4:26:54back to the state it was in before?
  5734. 4:26:59So Git can tell us what changed in the
  5735. 4:27:02repository, but what about
  5736. 4:27:04changes outside Git such as external API
  5737. 4:27:08calls or runtime configurations or
  5738. 4:27:11notifications?
  5739. 4:27:13And simply retrying an operation does
  5740. 4:27:16not necessarily undo what already
  5741. 4:27:19happened. So we wanted to understand not
  5742. 4:27:22just
  5743. 4:27:23whether agents can complete task, but
  5744. 4:27:26whether they can recover when something
  5745. 4:27:29goes
  5746. 4:27:30wrong.
  5747. 4:27:37This brings us to our first research
  5748. 4:27:39work undo bench.
  5749. 4:27:42We wanted to investigate whether an
  5750. 4:27:45agent's ability to complete a task also
  5751. 4:27:48translates into its ability to recover
  5752. 4:27:51from failure.
  5753. 4:27:53We evaluated agents across 2,880
  5754. 4:27:57paired trials
  5755. 4:27:59comparing normal execution with
  5756. 4:28:02execution under an injected failure.
  5757. 4:28:05So under normal conditions, the agent
  5758. 4:28:08achieved an 84% success rate,
  5759. 4:28:11but when we introduced failures, only
  5760. 4:28:1447%
  5761. 4:28:15of the normally successful trials
  5762. 4:28:18recovered successfully.
  5763. 4:28:20Notice that these two percentages though
  5764. 4:28:23have different denominators,
  5765. 4:28:25but yeah, so even when an agent
  5766. 4:28:29can complete a task successfully,
  5767. 4:28:32recovery remains a significant
  5768. 4:28:34challenge.
  5769. 4:28:38Now let's look at a specific failure
  5770. 4:28:41scenario. Imagine a coding agent sending
  5771. 4:28:45a request to an external system to
  5772. 4:28:48create a ticket.
  5773. 4:28:50The request has been sent and the agent
  5774. 4:28:52is waiting for a response.
  5775. 4:28:57Next, the server receives the request
  5776. 4:28:59and successfully creates ticket number
  5777. 4:29:02781,
  5778. 4:29:04but the acknowledgment gets lost on its
  5779. 4:29:07way to the agent.
  5780. 4:29:10From the agent's perspective, the
  5781. 4:29:12request has timed out,
  5782. 4:29:14but the agent does not know that the
  5783. 4:29:16ticket has already been created. A
  5784. 4:29:19timeout tells us that the response
  5785. 4:29:22didn't arrive, not that the operation
  5786. 4:29:25failed.
  5787. 4:29:28So the agent tries the request without
  5788. 4:29:31checking whether the
  5789. 4:29:33whether the first operation committed.
  5790. 4:29:37And now we have two tickets even though
  5791. 4:29:40the agent intended to create only one.
  5792. 4:29:43So in undo bench naive retries produced
  5793. 4:29:48duplicate effects in 54% of these kind
  5794. 4:29:51of trials. That's why we need to
  5795. 4:29:54distinguish a failed response from a
  5796. 4:29:58failed operations.
  5797. 4:30:05Now
  5798. 4:30:07the failure face also changes the
  5799. 4:30:10answer.
  5800. 4:30:11What do I mean is that recovery also
  5801. 4:30:14depends on
  5802. 4:30:16when the failure happens.
  5803. 4:30:18Before the mutation retrying is
  5804. 4:30:21generally safe because nothing has
  5805. 4:30:23changed. But during the mutation the
  5806. 4:30:26system has may be partially modified. So
  5807. 4:30:29we need to inspect its state.
  5808. 4:30:33And after the mutation the operation may
  5809. 4:30:35have committed even if the
  5810. 4:30:37acknowledgement was lost.
  5811. 4:30:39As we just saw retrying could create
  5812. 4:30:42duplicates.
  5813. 4:30:43So we need different recovery strategies
  5814. 4:30:47depending on the failure phase.
  5815. 4:30:57Now
  5816. 4:30:59why can't we just use git revert?
  5817. 4:31:03So git can restore repository changes
  5818. 4:31:07but coding agents also modify their
  5819. 4:31:11runtime state including tool registries,
  5820. 4:31:14middleware and configurations. They can
  5821. 4:31:17also trigger external effects such as
  5822. 4:31:20deployment or database rights.
  5823. 4:31:23So git revert does not reverse all these
  5824. 4:31:27changes. The recovery mechanism
  5825. 4:31:30has to match the type of effect the
  5826. 4:31:33agent created.
  5827. 4:31:39And this becomes particularly important
  5828. 4:31:43when coding agents start modifying their
  5829. 4:31:46own execution environment or what we
  5830. 4:31:49call the agent hardness.
  5831. 4:31:51This includes prompts, tools,
  5832. 4:31:54middleware, event listeners, resources,
  5833. 4:31:57and routing configurations.
  5834. 4:31:59Such changes may improve the agents
  5835. 4:32:02performance,
  5836. 4:32:04but they can also make recovery more
  5837. 4:32:06difficult.
  5838. 4:32:07In other words, an agent can become more
  5839. 4:32:10capable
  5840. 4:32:12while becoming less recoverable.
  5841. 4:32:20And that's why
  5842. 4:32:22uh that's why undo needs more than an
  5843. 4:32:25inverse command.
  5844. 4:32:27Uh now uh like undoing a change is not
  5845. 4:32:31always as simple as reversing a command.
  5846. 4:32:34Suppose an agent
  5847. 4:32:36changes a timeout from 30 to 60 to undo
  5848. 4:32:40that change, it needs to know
  5849. 4:32:43that the original value was 30.
  5850. 4:32:46That's why Evo undo captures a pre-state
  5851. 4:32:49witness before applying the mutation.
  5852. 4:32:53And that witness provides the
  5853. 4:32:54information to restore
  5854. 4:32:57the original state and later verify that
  5855. 4:33:02uh verify that recovery actually worked.
  5856. 4:33:06Now
  5857. 4:33:08question comes
  5858. 4:33:10how often uh do improved hardnesses
  5859. 4:33:13fail?
  5860. 4:33:14So
  5861. 4:33:15in Evo undo, we evaluated 600
  5862. 4:33:18self-evolution tasks.
  5863. 4:33:20Out of those, 478 produced capability
  5864. 4:33:24improvements, but 197 of those
  5865. 4:33:27improvements had recovery defects.
  5866. 4:33:31That is around 42% of the capability
  5867. 4:33:34positive cases.
  5868. 4:33:36So
  5869. 4:33:37here key takeaway is that a mutation can
  5870. 4:33:40improve an agent's capability and still
  5871. 4:33:43leave the system unable to recover
  5872. 4:33:47reliably.
  5873. 4:33:51Now
  5874. 4:33:53now we can say that why please fix the
  5875. 4:33:56undo did not work. So naturally we
  5876. 4:33:59wanted to use prompting and wanted to
  5877. 4:34:02tell agent that hey could you simply
  5878. 4:34:04repair all these recovery failures.
  5879. 4:34:07So we tested four verified guided repair
  5880. 4:34:11strategies but under the original
  5881. 4:34:14recovery language none recovered the 197
  5882. 4:34:18naturally occurring failures. The
  5883. 4:34:21problem was not necessarily
  5884. 4:34:24the agents reasoning in some cases the
  5885. 4:34:27agent lacked
  5886. 4:34:29precise information about the original
  5887. 4:34:31state in
  5888. 4:34:34in others the recovery language could
  5889. 4:34:37not express the operation needed to
  5890. 4:34:40reverse the chain.
  5891. 4:34:42So we investigated these two limitations
  5892. 4:34:46separately.
  5893. 4:34:50So
  5894. 4:34:51two bottlenecks
  5895. 4:34:53first is state grounding and another one
  5896. 4:34:55is recovery expressivity.
  5897. 4:34:57The
  5898. 4:34:58when we provided like precise
  5899. 4:35:01information about what had changed and
  5900. 4:35:03where
  5901. 4:35:04then recovery improved from 0 to 38 out
  5902. 4:35:08of 48 cases.
  5903. 4:35:10And the second is recovery expressivity.
  5904. 4:35:13So when we introduced richer undo
  5905. 4:35:16operation recovery improved
  5906. 4:35:19significantly from 0 to 142 out of 143
  5907. 4:35:23cases. So these results come from the
  5908. 4:35:26two different experimental
  5909. 4:35:28subgroups but together they show why
  5910. 4:35:31recovery needs both accurate state
  5911. 4:35:34information and ability to perform
  5912. 4:35:37the required undo operation.
  5913. 4:35:46Now question comes to our mind that can
  5914. 4:35:49better undo mechanism rescue failed
  5915. 4:35:51mutations.
  5916. 4:35:53So,
  5917. 4:35:55what happens like when we give the agent
  5918. 4:35:57better recovery capabilities. So, with
  5919. 4:36:00the original recovery language none of
  5920. 4:36:03the 197 recovery defective mutations
  5921. 4:36:06could be restored.
  5922. 4:36:08But after extending the recovery
  5923. 4:36:10language we successfully recovered 180
  5924. 4:36:13out of 197 cases. That is around 92%. As
  5925. 4:36:18a result the number of admissible
  5926. 4:36:21improvement
  5927. 4:36:22increased from 281 to 461
  5928. 4:36:26out of 600 tasks.
  5929. 4:36:28That is a 30 percentage point
  5930. 4:36:30improvement and
  5931. 4:36:33that brings us to the next question. How
  5932. 4:36:36do we build these recovery checks into
  5933. 4:36:39the agent's evolution process?
  5934. 4:36:42I will now hand it over to Dolly
  5935. 4:36:45to walk through the Evo undo algorithm.
  5936. 4:36:51>> All right. Thanks Tanmay.
  5937. 4:36:53So, let's get into the working on how
  5938. 4:36:56Evo undo actually works. So, first we
  5939. 4:36:58will start with the agent
  5940. 4:37:01first
  5941. 4:37:03the agent will start
  5942. 4:37:04the agent processes the mutation on
  5943. 4:37:07which state it is currently in along
  5944. 4:37:09with the effect contract and the
  5945. 4:37:11recovery plan. And before applying the
  5946. 4:37:13mutation
  5947. 4:37:15we capture the state witness
  5948. 4:37:18representing the original state.
  5949. 4:37:21And even before that the then a testing
  5950. 4:37:24in the testing environment agent will
  5951. 4:37:26apply the mutation and execute the
  5952. 4:37:28recovery plan and an independent
  5953. 4:37:31verifier will also verify the original
  5954. 4:37:34if verify to check if the original state
  5955. 4:37:36has been restored or not.
  5956. 4:37:38If the mutation improves the capability
  5957. 4:37:41based on the user defined criteria and
  5958. 4:37:44passes the recovery
  5959. 4:37:46check the agents admit the changes.
  5960. 4:37:49Otherwise
  5961. 4:37:51well,
  5962. 4:37:53the agent attempt to repair or the
  5963. 4:37:55verifier itself rejects the mutation
  5964. 4:37:57altogether. This way the recoverability
  5965. 4:38:00becomes a part of the admission decision
  5966. 4:38:01rather than something that we strategize
  5967. 4:38:04or even plan once we are
  5968. 4:38:07deployed the deployed the AI agent.
  5969. 4:38:11Okay, so with next
  5970. 4:38:16next we will walk through as as
  5971. 4:38:20given an example let let me walk through
  5972. 4:38:22how the
  5973. 4:38:23how that example of a timeout changing
  5974. 4:38:26from 30 to 60 would look like in action.
  5975. 4:38:29So let's take
  5976. 4:38:31let's start with the original state the
  5977. 4:38:33simple example that simple example of
  5978. 4:38:36our agent having the time value of 30
  5979. 4:38:39before changing anything the evil undo
  5980. 4:38:41would first read the original value.
  5981. 4:38:46And then it will save the it will
  5982. 4:38:48capture the original value as a state
  5983. 4:38:50witness. We know exact we so we know
  5984. 4:38:53exactly what need to be restored when
  5985. 4:38:56when
  5986. 4:38:56when if we have to restore it we would
  5987. 4:38:59know what
  5988. 4:39:00what need to be restored or which state
  5989. 4:39:01would be the true state to return back
  5990. 4:39:04to.
  5991. 4:39:05Now the agent so next come the mutation
  5992. 4:39:08now the agent changes the timeout from
  5993. 4:39:1030 to 60 as we wanted to do
  5994. 4:39:12and if recovery is required
  5995. 4:39:16if recovery is required the undo uses
  5996. 4:39:18the saved witness to restore the timeout
  5997. 4:39:21to 60.
  5998. 4:39:22Now independently the verifier finally
  5999. 4:39:25an independent verifier would also
  6000. 4:39:26confirm that the
  6001. 4:39:28recovered value matches the original
  6002. 4:39:30value so the recovery is in considered
  6003. 4:39:32successful simply because
  6004. 4:39:34an undo command was run but also the
  6005. 4:39:37verifier would verify the resulting
  6006. 4:39:39state to make sure that we are in fact
  6007. 4:39:42doing the
  6008. 4:39:44recovery those in the state we wanted to
  6009. 4:39:47be.
  6010. 4:39:49Well now comes the bigger question so so
  6011. 4:39:53we have this capability of recovery in
  6012. 4:39:55our arsenal. Should we be using it for
  6013. 4:39:57uniformly and applying the recovery
  6014. 4:39:59mechanism to all the failures?
  6015. 4:40:02Well, of course not.
  6016. 4:40:04Um not every failures can be recovered
  6017. 4:40:06the same way. If nothing has changed
  6018. 4:40:08then the then simply if if nothing has
  6019. 4:40:11changed in the in the system, we would
  6020. 4:40:13simply wanted to
  6021. 4:40:14um
  6022. 4:40:16recover
  6023. 4:40:17we we would simply wanted to safely
  6024. 4:40:19retry.
  6025. 4:40:21We would want the agent to simply retry
  6026. 4:40:23if and try to achieve the state that we
  6027. 4:40:25want to be.
  6028. 4:40:27However, if we have the original state
  6029. 4:40:30uh once we have the original state, we
  6030. 4:40:31can restore it. If the external action
  6031. 4:40:34has already been committed and we need
  6032. 4:40:37uh
  6033. 4:40:39it's then when we would need a
  6034. 4:40:40compensation mechanism compensation
  6035. 4:40:43action to
  6036. 4:40:44compensating action to reverse its
  6037. 4:40:46effect. And if we cannot recover
  6038. 4:40:49the recovery
  6039. 4:40:51and we cannot if if
  6040. 4:40:53and if we cannot the cannot verify that
  6041. 4:40:55recovery is safe or the verify have
  6042. 4:40:58rejected the recovery
  6043. 4:40:59then the agent should stop and escalate.
  6044. 4:41:03The right response definitely depends on
  6045. 4:41:04what actually happened and what we what
  6046. 4:41:07can be safely reversed or not.
  6047. 4:41:13That brings me
  6048. 4:41:16to the almost the last slide which is
  6049. 4:41:18about bringing everything together on
  6050. 4:41:20our two paper and to bench and Evo and
  6051. 4:41:22do.
  6052. 4:41:23So the key takeaways here are first
  6053. 4:41:26the undo bench shows the successful task
  6054. 4:41:29execution.
  6055. 4:41:31Uh
  6056. 4:41:33the undo bench shows that this
  6057. 4:41:34successful task execution doesn't
  6058. 4:41:36guarantee doesn't always guarantee the
  6059. 4:41:38successful recovery of it.
  6060. 4:41:40Secondly, the Evo undo algorithm
  6061. 4:41:43provides a structured process, capture
  6062. 4:41:45the original state, apply the mutation,
  6063. 4:41:46recover independently verify and
  6064. 4:41:48restoration. So it the Evo undo
  6065. 4:41:51algorithm does work in the in the
  6066. 4:41:53fashion in the way we intended it
  6067. 4:41:56And third, we should admit
  6068. 4:41:58self-evolution changes only when the
  6069. 4:42:00when both the capability and the
  6070. 4:42:02recoverability requirements are met.
  6071. 4:42:04So
  6072. 4:42:06ultimately, as we give agents more
  6073. 4:42:08autonomy, we should also give them more
  6074. 4:42:11reliable ways to recover from their
  6075. 4:42:12actions.
  6076. 4:42:16All right.
  6077. 4:42:17So finally, I hope you remember the
  6078. 4:42:20agent from the beginning. It was looking
  6079. 4:42:22for an undo button. Well, now it has
  6080. 4:42:24one. We have started building this
  6081. 4:42:26capability through our research project
  6082. 4:42:28Evo Undo. And you can explore the world
  6083. 4:42:30through the in in the GitHub repository.
  6084. 4:42:33You can scan the
  6085. 4:42:35QR code on the screen to get to the
  6086. 4:42:37page. Our goal is to make the recovery
  6087. 4:42:39of fundamental capability of AI agents
  6088. 4:42:41so that we not only have
  6089. 4:42:45the agent does not even execute does not
  6090. 4:42:47only execute the action but also recover
  6091. 4:42:49when something goes wrong as well. Thank
  6092. 4:42:51you so much.
  6093. 4:42:54>> Hey everyone, thanks for tuning in.
  6094. 4:42:56Today I'm going to walk you through how
  6095. 4:42:57we use AI agents at Duolingo to write
  6096. 4:42:59and ship more than 200,000 lines of unit
  6097. 4:43:01tests on iOS codebase.
  6098. 4:43:02I'll share what we built and what we
  6099. 4:43:04learned along the way.
  6100. 4:43:05I'm Kush. I'm a staff engineer on the
  6101. 4:43:06platform team here at Duolingo. My job
  6102. 4:43:08is to build developer tooling and
  6103. 4:43:09AI-assisted workflows for the 100 plus
  6104. 4:43:10engineers who contribute to our client
  6105. 4:43:11apps. Before Duolingo, I spent close to
  6106. 4:43:13a decade at Stripe and Uber, working
  6107. 4:43:15across a few teams there.
  6108. 4:43:17Let's start with some background on the
  6109. 4:43:18scale of mobile at Duolingo.
  6110. 4:43:20We ship our apps to the App Store every
  6111. 4:43:22week.
  6112. 4:43:23We're also a very experiment-driven
  6113. 4:43:25company. So at any given point of time,
  6114. 4:43:26we have more than 600 AB tests running
  6115. 4:43:28live.
  6116. 4:43:29And our code bases are pretty mature.
  6117. 4:43:30Both our iOS and Android apps have been
  6118. 4:43:32around for more than 10 years at this
  6119. 4:43:33point.
  6120. 4:43:35Here's also the number of executable
  6121. 4:43:36lines of app code in our iOS repository.
  6122. 4:43:38So as you can see, back in 2019, we were
  6123. 4:43:40at about 200,000 lines and today we're
  6124. 4:43:42hit close to 2 million.
  6125. 4:43:44And as AI-assisted coding picks up, you
  6126. 4:43:45can see that curve starting to bend
  6127. 4:43:46upward.
  6128. 4:43:48And it's not just the amount of code
  6129. 4:43:49that's changing, it's also who's writing
  6130. 4:43:50it. So last year, about 200 people
  6131. 4:43:52contributed to the iOS repository and
  6132. 4:43:54this year we've already crossed to 50.
  6133. 4:43:56And And lot of these folks aren't even
  6134. 4:43:57iOS engineers. We're have backend
  6135. 4:43:58engineers, Android engineers, even PMs
  6136. 4:44:00and designers all shipping code to the
  6137. 4:44:02iOS app.
  6138. 4:44:03And that's mostly been unlocked by AI
  6139. 4:44:04coding tools like Cursor and Clot Code.
  6140. 4:44:07Um, another thing to note about a
  6141. 4:44:08codebase is that it's a codebase in
  6142. 4:44:09transition. So a few years ago, we
  6143. 4:44:11standardized on the MVVM architecture
  6144. 4:44:12pattern.
  6145. 4:44:13Today, about half the code in our
  6146. 4:44:14repository follows that pattern, and the
  6147. 4:44:16other half is still legacy code that
  6148. 4:44:17we're in the process of migrating.
  6149. 4:44:19Since the MVVM half is the easiest to
  6150. 4:44:20test, that's where we focused on this
  6151. 4:44:22project.
  6152. 4:44:24Here's where we started back in March of
  6153. 4:44:25this year.
  6154. 4:44:26We were at overall unit test coverage of
  6155. 4:44:2811%.
  6156. 4:44:29The MVVM layers were a bit lower.
  6157. 4:44:31Repositories were at 11%, view models at
  6158. 4:44:33nine, data sources at five, and combined
  6159. 4:44:35we are about 9% for the MVVM slice of
  6160. 4:44:37the app.
  6161. 4:44:39So why does this matter now more than it
  6162. 4:44:41used to?
  6163. 4:44:42As agents and humans write more code and
  6164. 4:44:44write it faster, writing code isn't the
  6165. 4:44:45hard part anymore, verifying it is.
  6166. 4:44:47So when someone puts up a PR, how do you
  6167. 4:44:49know that it's actually safe to ship?
  6168. 4:44:50The bottleneck is really shifting from
  6169. 4:44:51writing to verifying.
  6170. 4:44:53And that's where unit tests come in.
  6171. 4:44:55They make up the base of the testing
  6172. 4:44:56pyramid. They're pretty cheap to write
  6173. 4:44:57and fast to run, and they also act as
  6174. 4:44:59guardrails for agents.
  6175. 4:45:00So when an agent's change breaks a test
  6176. 4:45:01in CI, it gets immediate feedback on
  6177. 4:45:03what to fix.
  6178. 4:45:04They also act as living documentation on
  6179. 4:45:06what the code is supposed to do.
  6180. 4:45:07And here's where we ended up eight
  6181. 4:45:08months later. Overall unit test coverage
  6182. 4:45:10went from 11% to 24%. Repositories went
  6183. 4:45:12from 11 to 72, view models from nine to
  6184. 4:45:1458, data sources from five to 41, and
  6185. 4:45:17the combined MVVM slice of the app went
  6186. 4:45:19from 9% to 59%.
  6187. 4:45:21So what did we actually end up building?
  6188. 4:45:23On paper, it's a pretty simple loop.
  6189. 4:45:24First you identify the files that need
  6190. 4:45:25tests, then you write those tests with
  6191. 4:45:27an agent, you verify that the tests are
  6192. 4:45:29good quality and they pass CI, then you
  6193. 4:45:30resolve the PR, which means getting it
  6194. 4:45:32reviewed by an engineer and merged.
  6195. 4:45:34And finally, you iterate on the feedback
  6196. 4:45:35so that the next batch of PRs is better
  6197. 4:45:36than the last one.
  6198. 4:45:38When production it's a little messier,
  6199. 4:45:39this is what our system ended up looking
  6200. 4:45:41like. I won't go through every box right
  6201. 4:45:42now, but at a high level, it's a few
  6202. 4:45:43temporal workflows that coordinate
  6203. 4:45:45through shared state stored in S3.
  6204. 4:45:47And every box you see here maps back to
  6205. 4:45:48one of the five steps I just talked
  6206. 4:45:49about.
  6207. 4:45:51Here are the building blocks that we
  6208. 4:45:52ended up using. Temporal handles
  6209. 4:45:54workflow orchestration and execution.
  6210. 4:45:56Clot Code acts as the coding agent. S3
  6211. 4:45:58holds the shared state that keeps our
  6212. 4:45:59workflows in sync.
  6213. 4:46:01And we use GitHub for actions, labels,
  6214. 4:46:02and PRs.
  6215. 4:46:03Data dogs acts as a observability layer
  6216. 4:46:05where we keep an eye on the health of
  6217. 4:46:06our pipelines.
  6218. 4:46:08If you haven't used Temporal before,
  6219. 4:46:09it's basically a durable workflow
  6220. 4:46:10execution engine.
  6221. 4:46:12You write your workflows as normal
  6222. 4:46:13Python code, and if a worker crashes,
  6223. 4:46:14the workflow just picks up right where
  6224. 4:46:15it left off.
  6225. 4:46:17This fits agents really well, and agent
  6226. 4:46:19session can run for seconds to hours,
  6227. 4:46:20and it can call out to tools like
  6228. 4:46:21GitHub, CI, S3, and Cloud.
  6229. 4:46:24We also lean on a few built-in features
  6230. 4:46:25of Temporal.
  6231. 4:46:26There are schedules which are basically
  6232. 4:46:28cron-like schedules for our jobs.
  6233. 4:46:29There are composable workflows so that
  6234. 4:46:30we can fan out work in parallel.
  6235. 4:46:32Then there's first-class support for
  6236. 4:46:33retries and timeouts and idempotency so
  6237. 4:46:35that we never kick off duplicate runs of
  6238. 4:46:36the same file.
  6239. 4:46:39To split up this work, we split this
  6240. 4:46:40project into four milestones. The first
  6241. 4:46:42was local validation. Basically, is this
  6242. 4:46:43project even viable? The second was a
  6243. 4:46:45scheduled backfill workflow to write
  6244. 4:46:47tests for existing code.
  6245. 4:46:48The third was a PR lifecycle workflow to
  6246. 4:46:50actually get those PRs merged, and the
  6247. 4:46:52fourth was a label trigger so engineers
  6248. 4:46:54can get tests for the new code that
  6249. 4:46:55they're writing.
  6250. 4:46:56Now we'll go through each of these in
  6251. 4:46:57order.
  6252. 4:46:58Before building anything real, we wanted
  6253. 4:46:59to do the least amount of work possible
  6254. 4:47:01that would tell us whether an agent
  6255. 4:47:02could actually do this job well and do
  6256. 4:47:03it at scale.
  6257. 4:47:04We started with a very simple local
  6258. 4:47:05Cloud Code setup
  6259. 4:47:07where we had one Cloud Code command that
  6260. 4:47:08acted as an orchestrator. It ran an
  6261. 4:47:10Xcode build script to generate a code
  6262. 4:47:12coverage report locally.
  6263. 4:47:13Then it fed that report through a file
  6264. 4:47:14selection script that we wrote,
  6265. 4:47:16and then it handed each selected file
  6266. 4:47:17off to sub agent two sub agents.
  6267. 4:47:19One wrote the tests and the other one
  6268. 4:47:21opened the PR.
  6269. 4:47:22We ran this in a loop and generated a
  6270. 4:47:23bunch of PRs.
  6271. 4:47:25We basically ran this in two batches.
  6272. 4:47:26The first batch had 17 PRs and we merged
  6273. 4:47:28eight of them. The second at 40 PRs and
  6274. 4:47:29we merged 19. So that's roughly a 50%
  6275. 4:47:31merge rate. The goal here wasn't to
  6276. 4:47:33merge a lot of PRs, it was to learn how
  6277. 4:47:34the agents fail and to fix those paper
  6278. 4:47:35cuts before productionizing anything.
  6279. 4:47:38Here are some of the failure modes that
  6280. 4:47:39we saw. Since we're a large legacy
  6281. 4:47:40codebase with both XC tests and Swift
  6282. 4:47:42testing, the agents would sometimes get
  6283. 4:47:43confused and mix the two in one file. We
  6284. 4:47:45fixed that by giving the agents clear
  6285. 4:47:46rules on when to use each.
  6286. 4:47:48We also had a giant user client
  6287. 4:47:49singleton class. Some code passed around
  6288. 4:47:51the concrete version of this class,
  6289. 4:47:52which is really hard to mock, whereas
  6290. 4:47:53others passed around a protocolized
  6291. 4:47:54version.
  6292. 4:47:55So we migrated the entire codebase over
  6293. 4:47:56to the protocolized version and added a
  6294. 4:47:58lint rule to enforce it.
  6295. 4:47:59This made the whole host of files easier
  6296. 4:48:01to test for agents and for humans.
  6297. 4:48:04We also ran into six concurrency issues
  6298. 4:48:06and agents writing timing based tests
  6299. 4:48:07which are a big source of flakiness in
  6300. 4:48:09CI. So we improved the state for both of
  6301. 4:48:10these with better rules and examples in
  6302. 4:48:12our testing skills.
  6303. 4:48:14Now this brings us to milestone two.
  6304. 4:48:15Since we were so far behind on code
  6305. 4:48:17coverage, the biggest leverage that we
  6306. 4:48:18saw was backfilling tests for code that
  6307. 4:48:20was already merged in.
  6308. 4:48:21That's where the scheduled backfill
  6309. 4:48:22workflow comes in. This is a system
  6310. 4:48:24design diagram of that workflow as it
  6311. 4:48:25runs in production
  6312. 4:48:26and it essentially does two jobs. The
  6313. 4:48:28left half is to identify which is
  6314. 4:48:29picking the files that need tests and
  6315. 4:48:31the right half is right which is having
  6316. 4:48:33the agents actually write those tests.
  6317. 4:48:36Let's start with identify. Every time a
  6318. 4:48:37PR merges to the main branch in the iOS
  6319. 4:48:39repo, we kick off a GitHub action job
  6320. 4:48:41that generates an X code coverage report
  6321. 4:48:42and uploads it to S3.
  6322. 4:48:44That becomes a source of truth for the
  6323. 4:48:45latest code coverage.
  6324. 4:48:46The backfill workflow then fetches that
  6325. 4:48:47report and runs it through a selector
  6326. 4:48:49script which picks the next set of files
  6327. 4:48:50to test.
  6328. 4:48:52Let's get a deeper a little deeper into
  6329. 4:48:53the selector script.
  6330. 4:48:54Its job is essentially to score every
  6331. 4:48:56Swift file based on how confident PR
  6332. 4:48:57that an agent can write a good test for
  6333. 4:48:59it.
  6334. 4:48:59The strongest signal is file type. So
  6335. 4:49:01view models, repositories and data
  6336. 4:49:03sources get a 2.5 X boost while legacy
  6337. 4:49:05files get pushed down.
  6338. 4:49:06We skipped UI code completely for now
  6339. 4:49:08since snapshot tests are better fit
  6340. 4:49:09there.
  6341. 4:49:10And larger files also get a penalty. So
  6342. 4:49:11we prioritize code that we're confident
  6343. 4:49:13that we can test well.
  6344. 4:49:15The next step is to actually write the
  6345. 4:49:16tests.
  6346. 4:49:17We kick off one child workflow per file
  6347. 4:49:19all running in parallel.
  6348. 4:49:21In each one of these runs, the agent
  6349. 4:49:22clones the iOS repository, reads the
  6350. 4:49:24testing skill that we keep in the repo
  6351. 4:49:26which spells out how to write a good
  6352. 4:49:27test.
  6353. 4:49:28Then it writes the tests plus any mocks
  6354. 4:49:29and fakes it needs and opens a PR with
  6355. 4:49:31the generated test label so we can
  6356. 4:49:32identify it easily later.
  6357. 4:49:35These agents run on Duolingo's cloud
  6358. 4:49:36coding agent which is a platform that
  6359. 4:49:38also runs the nipple.
  6360. 4:49:39The simplest way to think about it is as
  6361. 4:49:40an abstraction on top of cloud code and
  6362. 4:49:42codex running in the cloud.
  6363. 4:49:43We can invoke it from Slack, Temporal or
  6364. 4:49:44GitHub. It talks to a bunch of our
  6365. 4:49:46internal tools for MCP and it logs a
  6366. 4:49:48bunch of events so we can see what each
  6367. 4:49:49agent run is doing and how much it
  6368. 4:49:50costs.
  6369. 4:49:52We started running these on Linux
  6370. 4:49:53machines and then later moved them to
  6371. 4:49:55Mac runners which will become important
  6372. 4:49:56in a minute.
  6373. 4:49:58Here's what a generated test PR looks
  6374. 4:49:59like. Every PR description follows the
  6375. 4:50:01same template and it says explicitly
  6376. 4:50:03that the PR is AI generated.
  6377. 4:50:04The reviewer has four options. They can
  6378. 4:50:06approve it and PR automerges. They can
  6379. 4:50:08push changes to it directly. They can
  6380. 4:50:09reassign it to someone better suited or
  6381. 4:50:11they can close it out with feedback and
  6382. 4:50:12why it is in
  6383. 4:50:14Now, once we had the backfill running in
  6384. 4:50:16production, we had a slightly new
  6385. 4:50:17problem.
  6386. 4:50:17We suddenly had tens of tens to hundreds
  6387. 4:50:19of open PRs that nobody could manage by
  6388. 4:50:21hand.
  6389. 4:50:22So how did we get all of these to a
  6390. 4:50:22mergeable state and in front of the
  6391. 4:50:24right reviewer?
  6392. 4:50:26That's where the PR lifecycle workflow
  6393. 4:50:27comes in. You can think of it as a state
  6394. 4:50:28machine. It runs every couple of hours,
  6395. 4:50:30looks through every open generated test
  6396. 4:50:32PR and takes exactly one action on each.
  6397. 4:50:35It covers the next two steps of our
  6398. 4:50:36loop.
  6399. 4:50:37Verify, which is does this PR pass CI
  6400. 4:50:39and is it high quality? And resolve,
  6401. 4:50:40which is getting a human to review it
  6402. 4:50:42and merge it.
  6403. 4:50:43The logic for this workflow is a simple
  6404. 4:50:45decision tree. If there are any merge
  6405. 4:50:46conflicts or the PR has been open for
  6406. 4:50:47about three weeks, we close it out and
  6407. 4:50:48mark it as stale.
  6408. 4:50:49If CI is still running or a reviewer is
  6409. 4:50:51already on it, we skip it. If CI is
  6410. 4:50:52green and there's no reviewers, we find
  6411. 4:50:54one and assign them. If CI is red, we
  6412. 4:50:55try to fix it.
  6413. 4:50:57And if the reviewer rejects the PR, the
  6414. 4:50:58file goes into a 30-day cooldown so
  6415. 4:51:00we're not spamming engineers with the
  6416. 4:51:01same PR again and again.
  6417. 4:51:03When CI fails, the first thing we check
  6418. 4:51:05is which job failed. If it's pre-commit,
  6419. 4:51:07a GitHub action applies the fix. It's
  6420. 4:51:09cheap, it's deterministic, and there's
  6421. 4:51:10no agent needed.
  6422. 4:51:11If it's a real test failure, we kick off
  6423. 4:51:12another agent that reads the CI logs and
  6424. 4:51:14pushes a fix to the PR.
  6425. 4:51:15We cap that whole loop at five attempts,
  6426. 4:51:17past that we close the PR rather than
  6427. 4:51:18trying to fix it forever.
  6428. 4:51:20Once CI is green, we need to find the
  6429. 4:51:22right reviewer. What we do is we run git
  6430. 4:51:24blame on the source file being tested,
  6431. 4:51:25find the top contributor and assign it
  6432. 4:51:27to them and turn on automerge.
  6433. 4:51:28If they approve it, the PR gets merged.
  6434. 4:51:30If they reject it, the file goes into
  6435. 4:51:31the 30-day cooldown which I previously
  6436. 4:51:32mentioned.
  6437. 4:51:33This brings us to milestone four.
  6438. 4:51:35Once all of this was backfilling tests
  6439. 4:51:37for existing code, we wanted to make it
  6440. 4:51:38just as easy to get tests for the new
  6441. 4:51:39code engineers were writing.
  6442. 4:51:41Now any engineer can add a right test
  6443. 4:51:43label to their own PR.
  6444. 4:51:44It hooks into the same temporal
  6445. 4:51:45infrastructure we just walked through.
  6446. 4:51:47The only real difference is selection.
  6447. 4:51:48Instead of the selection script, we take
  6448. 4:51:50the file straight from the engineer's
  6449. 4:51:51PR.
  6450. 4:51:51The agent writes the tests and you get a
  6451. 4:51:53draft test PR linked back to your
  6452. 4:51:54original one.
  6453. 4:51:55Now let's talk about some of the
  6454. 4:51:56challenges that we ran into.
  6455. 4:51:57Our agents originally ran on Linux
  6456. 4:51:59machines which don't have Xcode or
  6457. 4:52:00SwiftLint installed. So when an agent
  6458. 4:52:02would open a PR, after about 20 or 30
  6459. 4:52:03minutes, CI would fail frequently on
  6460. 4:52:05lint.
  6461. 4:52:06This wastes the agent's time and CI time
  6462. 4:52:07as well, which is an scarce resource.
  6463. 4:52:10For a while, 36 13.6% of our PRs were
  6464. 4:52:12failing due to lint. And then when we
  6465. 4:52:13dug into it, 62% of those failures came
  6466. 4:52:15from just two lint rules.
  6467. 4:52:17We fixed this in layers. The first was
  6468. 4:52:18better examples in the prompt along with
  6469. 4:52:20pitfalls to avoid.
  6470. 4:52:21The second was a GitHub action that auto
  6471. 4:52:22fixes lint errors in CI.
  6472. 4:52:24The third was an audit of the rules that
  6473. 4:52:25couldn't be auto fixed.
  6474. 4:52:27We essentially removed the ones that
  6475. 4:52:28weren't high signal, and this made life
  6476. 4:52:29easier for both agents and human
  6477. 4:52:30engineers.
  6478. 4:52:31And finally, the real fix came when we
  6479. 4:52:33moved the agents to tunnel Mac runners.
  6480. 4:52:34So now the agents could build and lint
  6481. 4:52:36the code locally before ever opening a
  6482. 4:52:37PR.
  6483. 4:52:39The second challenge was PR quality.
  6484. 4:52:41When we assign a PR to a human, we want
  6485. 4:52:42it to be something that actually feel
  6486. 4:52:43comfortable proving.
  6487. 4:52:45But agents sometimes take the shortest
  6488. 4:52:46paths to green CI. We saw agents editing
  6489. 4:52:48production code to make a failing test
  6490. 4:52:49pass. We also saw them modifying,
  6491. 4:52:51deleting, or silencing existing tests.
  6492. 4:52:53And then we saw them adding long wait to
  6493. 4:52:54tests which can make CI flaky.
  6494. 4:52:57Again, there was no single fix here, so
  6495. 4:52:58we built three layers. The first is the
  6496. 4:53:00deterministic rule. If a PR touches any
  6497. 4:53:02file that isn't a test, a mock, a fake,
  6498. 4:53:04or a stub, we close it out.
  6499. 4:53:06The second builds on our AI code review.
  6500. 4:53:07At Duolingo, we use Claude and Codex to
  6501. 4:53:09review every PR, and those bots often
  6502. 4:53:11leave comments.
  6503. 4:53:12So we built an agent that resolves any
  6504. 4:53:14P0 comments before a human ever looks at
  6505. 4:53:15the PR.
  6506. 4:53:17And the third is an LLM as a judge. It's
  6507. 4:53:18a reviewer agent built just for these
  6508. 4:53:20test PRs. It reads the PR diff and
  6509. 4:53:22rejects production edits, changed or
  6510. 4:53:23deleted tests, disabled tests, and long
  6511. 4:53:25waits.
  6512. 4:53:26And as we learn more, we keep adding
  6513. 4:53:27additional checks to it.
  6514. 4:53:30So far, the LLM as a judge has reviewed
  6515. 4:53:31about 1,200 PRs and rejected 73 of them.
  6516. 4:53:34Every rejection comes with a comment
  6517. 4:53:35like this one, saying exactly why the PR
  6518. 4:53:36was rejected.
  6519. 4:53:38We then read that feedback and fold it
  6520. 4:53:39back into our prompts and skills so that
  6521. 4:53:40same mistakes get less likely each time.
  6522. 4:53:43So let's talk about what all of this
  6523. 4:53:44added up to.
  6524. 4:53:46Since we started, the pipeline has
  6525. 4:53:47merged more than 700 test PRs, adding
  6526. 4:53:49about 225,000 lines of test code.
  6527. 4:53:51Every one of those PRs was reviewed and
  6528. 4:53:52approved by a human.
  6529. 4:53:54NVM test coverage went from 9% to 59%
  6530. 4:53:57and overall coverage went from 11% to
  6531. 4:53:5824%.
  6532. 4:53:59Most importantly, we now have a codebase
  6533. 4:54:01so engineers and agents can ship faster
  6534. 4:54:02and with more confidence.
  6535. 4:54:04Here's a few more interesting stats.
  6536. 4:54:06Since March, the pipeline has written
  6537. 4:54:0734% of all new test files in the repo,
  6538. 4:54:0944% of all new test cases, and 10% of
  6539. 4:54:11every supply added to the codebase.
  6540. 4:54:13And interestingly, that share has
  6541. 4:54:14actually dipped in the last few weeks
  6542. 4:54:15because engineers are now writing a lot
  6543. 4:54:16more tests themselves, which is honestly
  6544. 4:54:18the end goal.
  6545. 4:54:20Um and here's the coverage curve since
  6546. 4:54:22April when the back field started
  6547. 4:54:23running on a schedule. You can see it
  6548. 4:54:25bent slightly upward around June and
  6549. 4:54:26July, which is when we really wrapped up
  6550. 4:54:27the pipeline.
  6551. 4:54:29So what did we learn while doing this
  6552. 4:54:30project?
  6553. 4:54:31The first thing is that this pattern
  6554. 4:54:32generalizes. The loop we talked about,
  6555. 4:54:34identify, write, verify, resolve,
  6556. 4:54:36iterate, it isn't specific to testing.
  6557. 4:54:38Unit tests were just the lowest hanging
  6558. 4:54:39fruit and any job that fits the shape
  6559. 4:54:41can run on the same infrastructure.
  6560. 4:54:42And that's exactly what we're starting
  6561. 4:54:43to do at the company. We're applying the
  6562. 4:54:45same loop to modernization, to
  6563. 4:54:46modernizing screens to MVVM and SwiftUI,
  6564. 4:54:48migrating code uh on Android to
  6565. 4:54:50coroutines and generating snapshot
  6566. 4:54:51tests.
  6567. 4:54:53So before I wrap up, I want to leave you
  6568. 4:54:55with a thought experiment. There's a new
  6569. 4:54:57class of fast general purpose
  6570. 4:54:58classifiers like Jev. They take in some
  6571. 4:55:00state and output an action in
  6572. 4:55:01milliseconds. So what if we replaced a
  6573. 4:55:02handwritten decision tree in the PR
  6574. 4:55:04lifecycle workflow with one of these?
  6575. 4:55:06PR state goes in, the classifier picks
  6576. 4:55:07the next action and we execute it all in
  6577. 4:55:09milliseconds.
  6578. 4:55:12I also want to thank a few folks who
  6579. 4:55:13worked on this with me. Jesse,
  6580. 4:55:14Guadalupe, Hunter and Jihau. They built
  6581. 4:55:16a lot of the platform pieces this
  6582. 4:55:17project stands on and helped review a
  6583. 4:55:18lot of this code.
  6584. 4:55:20That's it for me. Thank you so much for
  6585. 4:55:21making it this far. If you want to go
  6586. 4:55:22deeper, we've written about this
  6587. 4:55:23pipeline on the Duolingo engineering
  6588. 4:55:24blog and I'd also love to hear from you
  6589. 4:55:26if you're building something similar.
  6590. 4:55:27You can find me on LinkedIn or on X.
  6591. 4:55:33>> Hi everyone. Welcome to my talk about
  6592. 4:55:35what happens when you detonate thousands
  6593. 4:55:37of agent skills. This is uh looking
  6594. 4:55:40inside our agent detonation chamber and
  6595. 4:55:43what we saw when we put skills in there.
  6596. 4:55:46My name is Roey. I'm a staff engineer at
  6597. 4:55:48Zenity and I'm a Math research fellow. I
  6598. 4:55:51talk a lot about AI safety and security.
  6599. 4:55:53Follow me on Twitter and LinkedIn.
  6600. 4:55:56Before we even start our uh session, we
  6601. 4:56:00need to talk about what an agent skill
  6602. 4:56:02is.
  6603. 4:56:03Well, essentially an agent skill is
  6604. 4:56:05capability of capability on demand. It
  6605. 4:56:08has a bunch of instructions, resources,
  6606. 4:56:10new behavior, it can contain scripts and
  6607. 4:56:13it can even contain code.
  6608. 4:56:15The problem is that it was written by
  6609. 4:56:17someone else, and this is exactly where
  6610. 4:56:19the problem starts. So let's take a look
  6611. 4:56:21at the skills trajectory for instance.
  6612. 4:56:25If you look at this if we look at the
  6613. 4:56:26trajectory over here, we can see that
  6614. 4:56:28the skill reads through um
  6615. 4:56:31the cloud configuration and Kubernetes
  6616. 4:56:33configuration. It contacts the identity
  6617. 4:56:36service and it even probes the the
  6618. 4:56:38instance metadata endpoint. Now I don't
  6619. 4:56:41know what was the original intent of the
  6620. 4:56:43skill, but I can tell you that a lot of
  6621. 4:56:45um InfoStealers are operating in the
  6622. 4:56:47same way. They scan for the different
  6623. 4:56:50secret and identity folders.
  6624. 4:56:52In this case however, it was an official
  6625. 4:56:54Azure diagnostic skill.
  6626. 4:56:56And this is exactly the point where it's
  6627. 4:56:58kind of hard for us to identify whether
  6628. 4:57:00a skill is malicious or benign, and we
  6629. 4:57:02have to understand the original intent
  6630. 4:57:04of the skill according to its actions.
  6631. 4:57:07And at this point you might ask, okay,
  6632. 4:57:09so we can just like scan the skill,
  6633. 4:57:11right?
  6634. 4:57:12Well, it's it's it's more complicated
  6635. 4:57:14than that. The The first problem that
  6636. 4:57:17we've noticed is that the payload just
  6637. 4:57:19lives elsewhere.
  6638. 4:57:20In a lot of the skills, skill.md is just
  6639. 4:57:23the entry point. It can
  6640. 4:57:25invoke another endpoint that contains
  6641. 4:57:28the rest of the prompt. It can download
  6642. 4:57:30another package as part of it's part of
  6643. 4:57:32its installation script, and it might
  6644. 4:57:35have another script or code that
  6645. 4:57:36downloads other payloads and other
  6646. 4:57:38packages.
  6647. 4:57:39So if you use just a static scanner and
  6648. 4:57:42read through skills.md, you would lose
  6649. 4:57:45all of that, and you would have to
  6650. 4:57:46create some sort of mechanism to track
  6651. 4:57:48all of these payloads if you don't want
  6652. 4:57:50to lose them.
  6653. 4:57:51The second problem that we've noticed is
  6654. 4:57:53that LLM analyzers can be tricked.
  6655. 4:57:56Um there's a really interesting research
  6656. 4:57:58by Trail of Bits in which they showcase
  6657. 4:58:00the skill that manipulated the package
  6658. 4:58:03registry of one of of one of the
  6659. 4:58:05projects, and it had like a small
  6660. 4:58:07comment that said that the AppSec team
  6661. 4:58:10approved this change, and it completely
  6662. 4:58:13walked on the LLM analyzer. So given
  6663. 4:58:16that
  6664. 4:58:18scans are not bulletproof,
  6665. 4:58:20we've resulted to a different kind of
  6666. 4:58:21action and we actually decided to just
  6667. 4:58:23run the skill with the agent.
  6668. 4:58:25Um this is a technique employed by
  6669. 4:58:27malware researchers for years now uh and
  6670. 4:58:30it's called detonation a detonation
  6671. 4:58:32chamber. So we created the same one but
  6672. 4:58:36for for for agents and skills.
  6673. 4:58:38And this was our first version. You
  6674. 4:58:40know, first version uh we created a task
  6675. 4:58:42prompt, we put an agent and a skill
  6676. 4:58:44inside a sandbox, we let it operate over
  6677. 4:58:47like minutes or or hours until until it
  6678. 4:58:50completed the task with an observed
  6679. 4:58:52agent traces and analyzed it.
  6680. 4:58:55You can imagine that it wasn't enough.
  6681. 4:58:57Our first problem was that the
  6682. 4:58:59environment was missing most of the
  6683. 4:59:01time. A lot of skills rely on CLIs, they
  6684. 4:59:05rely on specific services and even on on
  6685. 4:59:08the existence of the
  6686. 4:59:09project. You can imagine notion skill,
  6687. 4:59:12for instance. This will have to have a
  6688. 4:59:15notion service available to talk to.
  6689. 4:59:17So we've added a world builder that
  6690. 4:59:19looks at the skill and builds the
  6691. 4:59:21appropriate environment accordingly.
  6692. 4:59:25But then we encountered another issue. A
  6693. 4:59:26generic task prompt might not be enough
  6694. 4:59:30for all skills.
  6695. 4:59:32I mean, think about it. You can have a
  6696. 4:59:34presentation skill uh that needs to
  6697. 4:59:36create a deck or deployment skill that
  6698. 4:59:37need to ship a service, a brainstorming
  6699. 4:59:40skill that needs to explore an idea.
  6700. 4:59:42Each and every one of them has an
  6701. 4:59:44entirely different modus operandi. Let's
  6702. 4:59:46take shipping shipping a service for
  6703. 4:59:48instance. You need to learn how to
  6704. 4:59:49configure it, you need to learn how to
  6705. 4:59:51deploy it, you need to tell it to
  6706. 4:59:53recover from snapshots using snapshots.
  6707. 4:59:56Um each and every one of them uh
  6708. 4:59:59requires different set of prompts and
  6709. 5:00:01you also need to need to make sure that
  6710. 5:00:03you
  6711. 5:00:04um
  6712. 5:00:05cover
  6713. 5:00:06every route in the in that skill because
  6714. 5:00:09one of them can be malicious. So we
  6715. 5:00:11created the operator that does just
  6716. 5:00:12that. It reads through the skill and
  6717. 5:00:14tries to analyze what are the different
  6718. 5:00:16trajectories um
  6719. 5:00:19that we should invoke.
  6720. 5:00:20And while it worked pretty well, if you
  6721. 5:00:23worked with agent, you can especially in
  6722. 5:00:25the security domain, you can
  6723. 5:00:29um
  6724. 5:00:30assess
  6725. 5:00:32that we went through a lot of
  6726. 5:00:34safeguards,
  6727. 5:00:36safeguard triggers.
  6728. 5:00:37Um we we got a lot of refusals. Um the
  6729. 5:00:40agents wasn't willing to decode and and
  6730. 5:00:43when it was willing to decode, it wasn't
  6731. 5:00:45willing to execute.
  6732. 5:00:47So we had to give you an an offer you
  6733. 5:00:49can't refuse and essentially create an
  6734. 5:00:51adversarial operator that convinces the
  6735. 5:00:54agents to run
  6736. 5:00:56and uh in case you didn't run even like
  6737. 5:00:59start the run from
  6738. 5:01:01the beginning.
  6739. 5:01:03Now I I would I I I I would emphasize
  6740. 5:01:05that we're doing a security research
  6741. 5:01:07here. This is a contained environment.
  6742. 5:01:09Nothing is malicious in here, but this
  6743. 5:01:11is one of the things that we had to do
  6744. 5:01:13in order to make it run properly. I
  6745. 5:01:15would also note that a lot of the time
  6746. 5:01:17safeguards are being run on something
  6747. 5:01:19which is
  6748. 5:03:15>> This
  6749. 5:06:48>> Cool.
  6750. 5:06:52>> WOO!
  6751. 5:06:55[applause]
  6752. 5:06:58GIVE ME A SEC TO MAKE SURE THIS IS
  6753. 5:06:59WORKING.
  6754. 5:07:01HECK YEAH.
  6755. 5:07:02I'm Diogo Almeida, and this talk is I
  6756. 5:07:05Love You So Jev Much. Um, Glass Animals
  6757. 5:07:09reference. We're not just weird. Um,
  6758. 5:07:11like I said, I'm Diogo Almeida. I'm
  6759. 5:07:14known as the Jev guy, sometimes Jevvy
  6760. 5:07:16Pinkman. Um, also co-created ChatGPT,
  6761. 5:07:19RLHF, and InstructGPT, but I assume that
  6762. 5:07:22for the crowd today, y'all know who I
  6763. 5:07:25am, at least mostly.
  6764. 5:07:27Um,
  6765. 5:07:28I have like a bunch of AI-specific stuff
  6766. 5:07:30I want to get to, but I like going into
  6767. 5:07:32my favorite topics in AI. First one is
  6768. 5:07:35what the is actually going on and
  6769. 5:07:38where is all the automation? If you've
  6770. 5:07:39heard me say this before, I'm sorry, but
  6771. 5:07:41it's such an important thing to beat
  6772. 5:07:43into the field cuz people still do not
  6773. 5:07:45get it, and I need to yell this until
  6774. 5:07:47everyone really understands what the
  6775. 5:07:49is going on. And if you're an AI,
  6776. 5:07:51you should have a really good answer
  6777. 5:07:53about why AI can do so many awesome
  6778. 5:07:55things and yet still be so useless for a
  6779. 5:07:57bunch of other ones.
  6780. 5:07:59My relationship to ChatGPT is somewhat
  6781. 5:08:02unique, one of the few people who hates
  6782. 5:08:04on ChatGPT who made it. I don't hate
  6783. 5:08:07ChatGPT, to be clear. I love using
  6784. 5:08:10agents. I love using coding agents. I
  6785. 5:08:11love chatbots. They're just quite
  6786. 5:08:13limited. And in particular, I think it's
  6787. 5:08:15a little tragic how much they've
  6788. 5:08:17bifurcated the field into overpromise,
  6789. 5:08:19underdeliver.
  6790. 5:08:21Really, really terrible stuff for the
  6791. 5:08:22world in my opinion.
  6792. 5:08:23And I like to talk about like the two
  6793. 5:08:25cults in AI and how we can resolve the
  6794. 5:08:28cognitive dissonance that everyone, I
  6795. 5:08:31think, should be having, unless they've
  6796. 5:08:32a very good answer.
  6797. 5:08:34There's cult one, which is, you know,
  6798. 5:08:36the cult of like EA's. They look at all
  6799. 5:08:38the benchmarks and they say like not
  6800. 5:08:40only are LM's surpassing human level on
  6801. 5:08:43basically every benchmark. They have
  6802. 5:08:44been for two years. We've been
  6803. 5:08:46announcing AGI three months away for two
  6804. 5:08:48years now.
  6805. 5:08:49Even since like the '03 days. Do people
  6806. 5:08:51remember that? No one even cares about
  6807. 5:08:53that model anymore and it was that was
  6808. 5:08:55like peak internal to OpenAI AGI hype.
  6809. 5:08:58And on the other hand, you look at the
  6810. 5:09:00non-benchmarks and you look at the
  6811. 5:09:02reality and you see the rest of the
  6812. 5:09:04world, especially if you guys go outside
  6813. 5:09:05and leave San Francisco, a lot of the
  6814. 5:09:07world is like really unchanged despite
  6815. 5:09:10AI being so cool. And I think AI
  6816. 5:09:12is transformative and it's a rare point
  6817. 5:09:14of view to actually think AI is cool and
  6818. 5:09:17transformative and also like
  6819. 5:09:19underwhelming the world right now. And
  6820. 5:09:21the tricky part is this is the same
  6821. 5:09:23field and we are all seeing the same
  6822. 5:09:25facts. So what gives? You know, why is
  6823. 5:09:27AI jagged, so to speak? Like why can it
  6824. 5:09:30be so good at some things and so not
  6825. 5:09:32good at other things? And what is the
  6826. 5:09:35Occam's Razor explanation for that? Um I
  6827. 5:09:38will be blasting through my slides fast.
  6828. 5:09:40I'm sorry. Maybe I could slow down for
  6829. 5:09:42people taking pictures. Um but uh I'm
  6830. 5:09:45noticing that
  6831. 5:09:46this is going to go fast is all I'm
  6832. 5:09:48saying. So I think that we as a field
  6833. 5:09:50should have an Occam's Razor explanation
  6834. 5:09:52for why are there some things that are
  6835. 5:09:54just
  6836. 5:09:55not just good, they are unbelievably
  6837. 5:09:57good, superhuman, like millennium prize
  6838. 5:09:59problems in math. But on the other hand,
  6839. 5:10:02they're not just bad, they're so
  6840. 5:10:03useless, they can't even be used to
  6841. 5:10:05touch accounting. That sounds way easier
  6842. 5:10:07than millennium prize problems in math,
  6843. 5:10:08right? I could do the latter. I
  6844. 5:10:09definitely could not do the former and I
  6845. 5:10:11was a math lead. So what gives here?
  6846. 5:10:13What is the reason for this? And I
  6847. 5:10:16normally love this as an interactive
  6848. 5:10:17thing, but I will instead just tell you
  6849. 5:10:19my answer. Sorry for the power dynamic.
  6850. 5:10:21Um everything on the left side is
  6851. 5:10:24intrinsically a human in the loop task.
  6852. 5:10:26Like deep research, I could not do
  6853. 5:10:28better than deep research. The models
  6854. 5:10:29are so good at that.
  6855. 5:10:31But if gun to my head or gun to your
  6856. 5:10:33head, you had to pick between me and
  6857. 5:10:35deep research, and the answer had to be
  6858. 5:10:37correct, not well-written, you'd
  6859. 5:10:38probably pick me every time.
  6860. 5:10:40And that's the distinction between like
  6861. 5:10:43optimizing for automation, which tends
  6862. 5:10:45to be a lot easier and a lot simpler,
  6863. 5:10:46yet
  6864. 5:10:47AI keeps failing at this. Every year
  6865. 5:10:49there's like an infamous failure of AI
  6866. 5:10:51failing to even automate drive-throughs.
  6867. 5:10:53And that should be kind of shocking to
  6868. 5:10:56people. And my claim is actually for
  6869. 5:10:59these tasks, pleasing a human is nothing
  6870. 5:11:01to do with doing the task. What you want
  6871. 5:11:03to do is basically be an abstraction
  6872. 5:11:05away. You want the decision to be made,
  6873. 5:11:07and then you want to get back into doing
  6874. 5:11:09automation, and code is the language of
  6875. 5:11:11automation. Therefore, you need to
  6876. 5:11:12integrate with code. This is a lot, but
  6877. 5:11:14I will summarize it again. I get the
  6878. 5:11:16irony that you know, I co-created RLHF,
  6879. 5:11:19and I do believe RLHF is part of the
  6880. 5:11:21problem. And RLHF in particular is why
  6881. 5:11:24all AI today does require a human in the
  6882. 5:11:27loop with an asterisk unless the task is
  6883. 5:11:29like very trivially verifiable and can
  6884. 5:11:31run on its own, but still requires a
  6885. 5:11:33human in the loop to verify it
  6886. 5:11:34afterwards. It is because we literally
  6887. 5:11:36put it in. That is what the HF is in
  6888. 5:11:39RLHF. And my claim is that there's a
  6889. 5:11:41better way. You know, my summary of the
  6890. 5:11:44state is 100% of LLMs today are
  6891. 5:11:46optimized for assistance. You might talk
  6892. 5:11:48about RLVR, but the RLVR models are very
  6893. 5:11:50bad to interact with if they've not been
  6894. 5:11:52RLHF'd.
  6895. 5:11:53The result is LLMs become too good to be
  6896. 5:11:55true at assistance. They are just so
  6897. 5:11:57good. Like you should never pick
  6898. 5:11:59me over ChatGPT for your ChatGPT
  6899. 5:12:01queries. But for actually doing like you
  6900. 5:12:04know, run in the background cognitive
  6901. 5:12:06work, you should not trust LLMs at all
  6902. 5:12:08because they've not been optimized for
  6903. 5:12:10that, which is why they're so bad at the
  6904. 5:12:12tasks of automation. And I would also
  6905. 5:12:14guess that automation is a lot more
  6906. 5:12:15valuable than assistance because you can
  6907. 5:12:17do way cooler, deeper. I'm going to get
  6908. 5:12:19into this later, but I think there's
  6909. 5:12:20going to be so much stuff that we can
  6910. 5:12:21do, and we are just scratching the
  6911. 5:12:23surface of that right now. And that's
  6912. 5:12:24the tragic thing about AI,
  6913. 5:12:26the state of AI for me.
  6914. 5:12:29Let's talk about the Jevolution.
  6915. 5:12:32I I can't tell if this is appropriate,
  6916. 5:12:34but it's too
  6917. 5:12:35Um,
  6918. 5:12:36oh god, don't take pictures of this one.
  6919. 5:12:38Um,
  6920. 5:12:39cool.
  6921. 5:12:40Oh, I'm getting a thumbs up. Yeah, uh,
  6922. 5:12:43excellent. So, we're we instead of doing
  6923. 5:12:45RLHF, we just have to do something else.
  6924. 5:12:46We need to think from first principles.
  6925. 5:12:48How do we optimize for machine native
  6926. 5:12:49AI? You guys hopefully know the
  6927. 5:12:51interface of Jeff, and that is what we
  6928. 5:12:52are doing. More important than
  6929. 5:12:54optimizing for its interface, we are
  6930. 5:12:56uh, rethinking how AI should be done
  6931. 5:12:59because this is what we did again for
  6932. 5:13:01RLHF.
  6933. 5:13:02For Sutton's bitter lesson,
  6934. 5:13:04I'm going to loosely paraphrase to make
  6935. 5:13:06my point easier to say, is that
  6936. 5:13:08scaling um, and compute matters more
  6937. 5:13:11than algorithms. This is why people
  6938. 5:13:12become like scaling pill Jivronis. And I
  6939. 5:13:15would argue that there's an even bitter
  6940. 5:13:17lesson than that, which is data always
  6941. 5:13:20matters more than compute. We are
  6942. 5:13:21learning that quite a bit in the field
  6943. 5:13:22now, but it has been obvious since the
  6944. 5:13:24beginning of time. I'll tell you when
  6945. 5:13:26the this is the one you should be
  6946. 5:13:27photographing. I would argue that the
  6947. 5:13:30the bitterest lesson is doing the right
  6948. 5:13:32task. And doing the right task is like a
  6949. 5:13:34weird thing to think about, especially
  6950. 5:13:36for ML people, because they like
  6951. 5:13:38algorithms. They want number goes down
  6952. 5:13:40or up or whichever is good direction.
  6953. 5:13:41They don't want to think about the real
  6954. 5:13:43world, but this is always how you make
  6955. 5:13:45gigantic gains in performance. This is
  6956. 5:13:47how the big breakthroughs happen. 100%
  6957. 5:13:49of the time it comes from this. And this
  6958. 5:13:52is true for classical RL games. This is
  6959. 5:13:53why Sutton had his bitter lesson, it
  6960. 5:13:56which is data is derived by compute when
  6961. 5:13:58you're doing rollouts. Like there's
  6962. 5:14:00there's why would you need off policy
  6963. 5:14:01data when playing chess or something
  6964. 5:14:03like that? And doing the right task is
  6965. 5:14:05defined by the rules. So actually when
  6966. 5:14:06you're doing RL, you don't even need to
  6967. 5:14:08think about the right task.
  6968. 5:14:10For pre-training, we actually as a field
  6969. 5:14:12had to do a lot of thinking about the
  6970. 5:14:14right task. I don't know if people
  6971. 5:14:15remember the birthdays with um, masked
  6972. 5:14:18language modeling, but it took a while
  6973. 5:14:20and it was very unclear to figure out
  6974. 5:14:22that next token prediction was super
  6975. 5:14:24cool, not because it was better
  6976. 5:14:26at masked language modeling. You know,
  6977. 5:14:28the objectives on their own aren't
  6978. 5:14:30special. It's about what you can do with
  6979. 5:14:31them afterwards and it turns out next
  6980. 5:14:33token prediction is incredible at
  6981. 5:14:34generating strings, which is what these
  6982. 5:14:36pre-trained models were so good at. So,
  6983. 5:14:38no level of mass language modeling, even
  6984. 5:14:40if you had today's compute and
  6985. 5:14:41algorithms, would make it as interesting
  6986. 5:14:43as GPT-3 was because it was just doing a
  6987. 5:14:45different task. And data, you know, the
  6988. 5:14:48internet is obviously a giant
  6989. 5:14:50bottleneck. Now we're like, you know,
  6990. 5:14:51burning books or whatever in order to
  6991. 5:14:52get more data, but data obviously
  6992. 5:14:54matters a lot more than compute and
  6993. 5:14:55people keep like rediscovering this and
  6994. 5:14:57figuring this out.
  6995. 5:14:59It's extra true for making LLMs useful.
  6996. 5:15:01Um before RLHF even existed, the data
  6997. 5:15:04for RLHF didn't exist and you could
  6998. 5:15:08like like it was night and day. I'll
  6999. 5:15:09just show you the next slide. This is
  7000. 5:15:10the probably the relevant part. This is
  7001. 5:15:11what this was the work in the RLHF
  7002. 5:15:13paper.
  7003. 5:15:14And [snorts] the high level was that
  7004. 5:15:16even with the dumbest algorithms in the
  7005. 5:15:17world, even with a GPT-2 sized model,
  7006. 5:15:20you got something like GPT-9 level
  7007. 5:15:23performance, like real GPT-9, not
  7008. 5:15:25today's like fake GPT numbers. Like the
  7009. 5:15:27numbers used to mean like 100x scaling
  7010. 5:15:29and effective compute for pre-training
  7011. 5:15:31and they've given up on that because it
  7012. 5:15:32just can't scale anymore. And still even
  7013. 5:15:34with that it would take something like
  7014. 5:15:36GPT-7 to 9 to match GPT-2, the model
  7015. 5:15:39from like 2018 or whatever it was, at
  7016. 5:15:42the task of instruction following. That
  7017. 5:15:44is kind of wild because you think big
  7018. 5:15:46LLMs are so important for this and they
  7019. 5:15:49are, but that is not the most important
  7020. 5:15:51thing compared to even the gigantic jump
  7021. 5:15:54of doing the right task. And spoilers,
  7022. 5:15:55this is what we did for Jeff, right?
  7023. 5:15:57Like we made this gigantic leap in the
  7024. 5:15:59field that no one could realized could
  7025. 5:16:00be coming because we were optimizing for
  7026. 5:16:02a task and very few people think about
  7027. 5:16:04the right task.
  7028. 5:16:06Um
  7029. 5:16:07this is why Jeff has the Jeff ne sais
  7030. 5:16:10quoi. Um that's a really smart pun, I
  7031. 5:16:13think. Um
  7032. 5:16:15that that little bit of a pity laugh,
  7033. 5:16:17but thank you. Um you get what you
  7034. 5:16:19optimize for. I'm looking for a better
  7035. 5:16:21word than reliable, so if you guys have
  7036. 5:16:23suggestions, let me know. But there's if
  7037. 5:16:25there's this reliability of intelligence
  7038. 5:16:27that's really missing in LLMs. I often
  7039. 5:16:29call it you get Einstein 95% of the
  7040. 5:16:32time, but Mr. Bean 5% of the time.
  7041. 5:16:34And if you can't tell which is which,
  7042. 5:16:36how the are you going to automate
  7043. 5:16:37anything? And instead instead of like
  7044. 5:16:39focusing on like sometimes
  7045. 5:16:41smarts that you can then verify, what if
  7046. 5:16:43you focused on extreme, extreme narrow,
  7047. 5:16:45focused reliability so you can get that
  7048. 5:16:47smarts every time. And this doesn't show
  7049. 5:16:49up on benchmarks. It is the reason why
  7050. 5:16:51people love Jev. And this is also, I
  7051. 5:16:53realize in hindsight, we obviously
  7052. 5:16:54didn't prepare for a chat GPT moment for
  7053. 5:16:57developers, but it hasn't because people
  7054. 5:16:59in the AI space just overpromise,
  7055. 5:17:02underdeliver so much that having
  7056. 5:17:04a model that actually didn't do that was
  7057. 5:17:06a surprise. And it is not the speed cost
  7058. 5:17:08or even interface. This is something
  7059. 5:17:10that, you know, my team tells me I
  7060. 5:17:12shouldn't be telling people so that we
  7061. 5:17:13can have a long-term advantage and I
  7062. 5:17:15just care too much. Like it's reliable
  7063. 5:17:17intelligence. Make sick models, please,
  7064. 5:17:19even if we, you know, if it doesn't
  7065. 5:17:21involve our models.
  7066. 5:17:23Um we see that Jev is like a foundation
  7067. 5:17:25for smart software. I'm going to skip
  7068. 5:17:27all over this, but I think software is
  7069. 5:17:28going to be so back. I think the
  7070. 5:17:30narrative of SAS apocalypse is obviously
  7071. 5:17:32not going to happen, but
  7072. 5:17:34I think that
  7073. 5:17:36even calling it a SAS apocalypse is like
  7074. 5:17:38an understatement and I'll get into that
  7075. 5:17:39a bit and you guys ideally are the
  7076. 5:17:41perfect crowd for this. I'm I'm so
  7077. 5:17:42excited for this part because that's the
  7078. 5:17:43part that's new to me. Um
  7079. 5:17:46I want to make a distinction between
  7080. 5:17:47smart software and just in time
  7081. 5:17:49software. The the crazy part is AI is so
  7082. 5:17:52transformative and yet somehow despite
  7083. 5:17:55AI being so awesome, software has
  7084. 5:17:57essentially unchanged in the last decade
  7085. 5:18:00except being cheaper to write. Isn't
  7086. 5:18:02that weird? Isn't the it weird that the
  7087. 5:18:04best we can do in software is having a
  7088. 5:18:06chatbot to the side that sometimes
  7089. 5:18:08interacts with the software but cannot
  7090. 5:18:10take expensive decisions for you. That
  7091. 5:18:12is kind of like a weird thing if AI was
  7092. 5:18:14really that transformative. And I think
  7093. 5:18:16the problem is we're focusing too much
  7094. 5:18:17on coding agents and not enough about
  7095. 5:18:20putting the intelligence into the inner
  7096. 5:18:21loop, which I'm going to get more into a
  7097. 5:18:23bit. And people are jeving. There's been
  7098. 5:18:25tons of successes. We don't need to flex
  7099. 5:18:27this. There's so many cool things, but
  7100. 5:18:29my AI secret is going to be that the
  7101. 5:18:33field isn't jeving hard enough. And this
  7102. 5:18:35is what Swix said that I can make a call
  7103. 5:18:36to action for you guys that's unique and
  7104. 5:18:38hopefully I have enough time for this. I
  7105. 5:18:40think the world still has LLM Stockholm
  7106. 5:18:42syndrome. We are still thinking from a
  7107. 5:18:44really LLM-filled place, which is
  7108. 5:18:47I think tragic, you know, like I I I
  7109. 5:18:49love the software saying the tools you
  7110. 5:18:51use change the way you think. And I
  7111. 5:18:53think that it's changed the way we think
  7112. 5:18:54way too much
  7113. 5:18:55and we're not really rethinking software
  7114. 5:18:57like I think the field needs. I think we
  7115. 5:18:59could just do be doing so much more.
  7116. 5:19:01Hopefully I can try to do my best for my
  7117. 5:19:03remaining time to convince you guys of
  7118. 5:19:05that.
  7119. 5:19:05Um so let's jet to the future.
  7120. 5:19:08Um
  7121. 5:19:10I an analogy I love bear with me is
  7122. 5:19:12about LEDs. Before there was the blue
  7123. 5:19:16LED, there was red and green LEDs. These
  7124. 5:19:18were really limited in capability and
  7125. 5:19:20they were used for like simple
  7126. 5:19:21indicators and you know like clocks and
  7127. 5:19:23stuff like that.
  7128. 5:19:24And when the blue LED was made
  7129. 5:19:26discovered actually don't really know
  7130. 5:19:28how things outside of computer science
  7131. 5:19:29work. Um but when it existed I guess um
  7132. 5:19:33it enabled not just like 50% more use
  7133. 5:19:36cases which what you'd expect about like
  7134. 5:19:37changing colors from two to three. It
  7135. 5:19:39enabled so many more. Like now LEDs
  7136. 5:19:41could be used for white light. It could
  7137. 5:19:43be used for like the displays that we
  7138. 5:19:44all use right now and it created a
  7139. 5:19:46revolution in lighting. And it's it's
  7140. 5:19:49it's unfathomable how much bigger
  7141. 5:19:51the capabilities of LEDs were just you
  7142. 5:19:54know adding a new primitive that
  7143. 5:19:56composes together with the old
  7144. 5:19:57primitives. You can see where I'm going
  7145. 5:19:59here maybe. And I think it's a similar
  7146. 5:20:01story for software. Software is so
  7147. 5:20:04incredible. Ignore AI for the
  7148. 5:20:06last you know half decade, but software
  7149. 5:20:08has done so many insanely cool things.
  7150. 5:20:11We've done a lot of stuff you know like
  7151. 5:20:13do I
  7152. 5:20:14love using Sass? Not really, but does it
  7153. 5:20:16solve problems? Abso lutely. And
  7154. 5:20:19it's incredible how much we've done with
  7155. 5:20:21three fundamental logic gates. You know
  7156. 5:20:23like we are doing this with like simple
  7157. 5:20:25binary operations that can do things
  7158. 5:20:27like greater than less than equal. It's
  7159. 5:20:29kind of nuts how much software has been
  7160. 5:20:31built from such simple primitives. And
  7161. 5:20:34my claim is that what we are doing with
  7162. 5:20:37AI now when we're making it our
  7163. 5:20:39intelligence machine native, this is
  7164. 5:20:40kind of conceptually adding a fourth
  7165. 5:20:43semantic logic gate.
  7166. 5:20:45What if we program with common sense,
  7167. 5:20:47with intention, with human
  7168. 5:20:48understanding? Like do we still need
  7169. 5:20:50regexes with like crazy things? Or can
  7170. 5:20:52we just say like we want a Gmail email
  7171. 5:20:54address or a real-looking one? There's
  7172. 5:20:56so much that we can do that we're going
  7173. 5:20:57to need to push the you know push the
  7174. 5:20:59frontiers of cheap, smart, fast in order
  7175. 5:21:02to get it done. But I think that there's
  7176. 5:21:03going to be like so many things here.
  7177. 5:21:05And I think just like the blue LED, it's
  7178. 5:21:07going to be way bigger than just
  7179. 5:21:08increasing the representational ability
  7180. 5:21:11of software by 33% cuz it's adding
  7181. 5:21:13effectively another kind of sort of
  7182. 5:21:16logic gate. And this sci-fi is the stuff
  7183. 5:21:18that I think will be impossible to
  7184. 5:21:19predict. Just like who could predict how
  7185. 5:21:21powerful the internet would have been
  7186. 5:21:23because it requires it's like much more
  7187. 5:21:25grassroots and decentralized and it
  7188. 5:21:26requires people to be empowered to do
  7189. 5:21:28insane stuff.
  7190. 5:21:30Um I call this software research. Um
  7191. 5:21:34that's not a joke. Um but uh I I think
  7192. 5:21:37that there's going to be like an
  7193. 5:21:38explosion of what software can do and we
  7194. 5:21:41will want people to be exploring those
  7195. 5:21:43boundaries of software, not necessarily
  7196. 5:21:46because it's valuable, but because they
  7197. 5:21:47can, because we have like new frontiers
  7198. 5:21:50to explore. And this is what the early
  7199. 5:21:51internet was like. Should all of those
  7200. 5:21:53websites existed?
  7201. 5:21:55Probably not, but like it was very cool
  7202. 5:21:57and we found a lot of awesome stuff from
  7203. 5:21:59that and I think we we you know we have
  7204. 5:22:01another opportunity for that. So we
  7205. 5:22:02should not just keep living in this
  7206. 5:22:04little you know fish tank that we are
  7207. 5:22:06the victims of OpenAI and Anthropic on
  7208. 5:22:08whatever they ship and we just have to
  7209. 5:22:10be like LLM wrappers. So I'm going to
  7210. 5:22:12pitch software research as direction
  7211. 5:22:14here. The theme is going to be real
  7212. 5:22:16software engineering, not just vibe
  7213. 5:22:18coding. There's it's going to be like
  7214. 5:22:20the kind of stuff that everyone has
  7215. 5:22:21known if you've been a software
  7216. 5:22:23engineer, decomposition, breaking tasks
  7217. 5:22:25down. We need to think about stages,
  7218. 5:22:27ideally ones that can be verified so we
  7219. 5:22:29have real encapsulation. And we need to
  7220. 5:22:31actually do state management instead of
  7221. 5:22:33assuming everything is global state
  7222. 5:22:35within a KV cache. And the theme is just
  7223. 5:22:37going to be like real
  7224. 5:22:38engineering by people who care a lot
  7225. 5:22:40about the end result. It doesn't have to
  7226. 5:22:42be valuable. Ideally it's sick as
  7227. 5:22:44Um and you know we should be pushing the
  7228. 5:22:46limits.
  7229. 5:22:48I'm going to go a couple of like my
  7230. 5:22:49favorite ideas and I want to figure out
  7231. 5:22:51how to encourage these in the world. So
  7232. 5:22:54this is my first step at this. I'm just
  7233. 5:22:56going to talk about it. We have raised a
  7234. 5:22:58bunch of money, so I would like to
  7235. 5:22:59figure out how to fund this, but we
  7236. 5:23:00don't have the operations help to do
  7237. 5:23:01that. But here are just some ideas out
  7238. 5:23:03there. Um, I think agents are
  7239. 5:23:07they're very cool, but like they're
  7240. 5:23:10they're so inflexible in their design
  7241. 5:23:12because of the tyranny of the KV cache.
  7242. 5:23:13I don't have the time for this if this
  7243. 5:23:15timer is to be believed, but believe me
  7244. 5:23:17that there's so much that people have
  7245. 5:23:19tried to do but failed at because it
  7246. 5:23:21ends up becoming inefficient because it
  7247. 5:23:22fights against the KV cache. But I think
  7248. 5:23:25there's a lot more to do. Why is memory
  7249. 5:23:27O of N? Right? Like why can't we have
  7250. 5:23:29all of our history of coding agents
  7251. 5:23:31always included in every new call? That
  7252. 5:23:33is how continual learning works, right?
  7253. 5:23:36And we have the memory available. It's
  7254. 5:23:38just that we don't have the smarts to
  7255. 5:23:39map produce over the memory to find out
  7256. 5:23:42like what you should what you should
  7257. 5:23:43remember or what should you shouldn't.
  7258. 5:23:45Um, I think there's many ways to do
  7259. 5:23:46login or even better lookups. Why when
  7260. 5:23:49you're making many agents at once, why
  7261. 5:23:51can't they not share state? Why do they
  7262. 5:23:52need to rediscover the same patterns of
  7263. 5:23:54the codebase over and over and over
  7264. 5:23:56again? And why can't non-naive routing
  7265. 5:23:58and sub-agents work? Like even naive
  7266. 5:24:00routing doesn't work, but like why can't
  7267. 5:24:01we do these things and why just like the
  7268. 5:24:03you know, the the big labs keep going
  7269. 5:24:04back and forth on whether or not
  7270. 5:24:05sub-agents work? Um, I think that like
  7271. 5:24:08from an engineering perspective, there's
  7272. 5:24:10elegance to it, but they fight against
  7273. 5:24:12the KV cache. And I think that if we
  7274. 5:24:13design agents around that, we could do
  7275. 5:24:15so much more.
  7276. 5:24:17This one's the probably the most vague
  7277. 5:24:19of my directions, but I feel very
  7278. 5:24:21strongly about it. Um, the world is full
  7279. 5:24:23of principal agent problems. This is
  7280. 5:24:25when you have like you know, ulterior
  7281. 5:24:27motives between this is not a coding
  7282. 5:24:28agent, but like any agent. The the
  7283. 5:24:30classic example is citizens versus
  7284. 5:24:32elected officials. And I don't want to
  7285. 5:24:34make a too political claim here, but
  7286. 5:24:36like there's misalignment in all levels
  7287. 5:24:38of society and this results in like
  7288. 5:24:40waste, fraud, bad things and it it
  7289. 5:24:43results in for a society that wants
  7290. 5:24:45fairness, you need to over emphasize the
  7291. 5:24:47letter over the spirit, which then
  7292. 5:24:49results in loopholes and play uh pay to
  7293. 5:24:52win dynamics in the world. And I think
  7294. 5:24:54the root of that problem is we can't
  7295. 5:24:55encode the spirit. Just like with
  7296. 5:24:57software, we're constrained to encoding
  7297. 5:24:59bits instead of what we actually meant
  7298. 5:25:01in that situation. And I think so many
  7299. 5:25:04organizations, society could be
  7300. 5:25:05transformed by really encoding the
  7301. 5:25:07spirit all over the place.
  7302. 5:25:09Obviously, real-time intelligence
  7303. 5:25:10security
  7304. 5:25:12can we prevent the open AI hugging face
  7305. 5:25:14sandbox, blah blah blah? Um why did why
  7306. 5:25:16couldn't they prevent it? It is not
  7307. 5:25:18because they didn't have the
  7308. 5:25:19intelligence. I would argue it's a lot
  7309. 5:25:21easier to figure out they're doing a
  7310. 5:25:22naughty thing than to actually do the
  7311. 5:25:24naughty thing. It is because it was not
  7312. 5:25:26fast, cheap, and reliable enough to
  7313. 5:25:28actually run real time. And what happens
  7314. 5:25:31when every corporation has their own
  7315. 5:25:33real-time security that's actually
  7316. 5:25:35smart? How cool would that be?
  7317. 5:25:38I can't talk about generative UIs
  7318. 5:25:39because people are going to be talking
  7319. 5:25:41about it afterwards. I don't want to
  7320. 5:25:42steal their thunder, but it's cool as
  7321. 5:25:43trust me. Um
  7322. 5:25:46like semantic dependent typing. Why
  7323. 5:25:48can't
  7324. 5:25:49uh we have like a type system that
  7325. 5:25:51actually is semantic, right? Like why
  7326. 5:25:54can't we like declare what local
  7327. 5:25:57properties we want so we don't have to
  7328. 5:25:59read a chain of dependencies deep down?
  7329. 5:26:01These all should be possible. And
  7330. 5:26:02imagine if you could like run a Jev-like
  7331. 5:26:04model in your lint checker to check if
  7332. 5:26:07your it's all type safe and it could you
  7333. 5:26:09know uphold semantic properties. It
  7334. 5:26:11would be so cool.
  7335. 5:26:13There's a there was a great Twitter
  7336. 5:26:15video on um like a voice interface. In
  7337. 5:26:18general, I think new interfaces are so
  7338. 5:26:20exciting. This is to me the return of
  7339. 5:26:22like this is going to show by agent
  7340. 5:26:24software like do what I mean. This used
  7341. 5:26:26to be like a popular thing, but it never
  7342. 5:26:27really worked because you it was it was
  7343. 5:26:29too many hard-coded rules to really
  7344. 5:26:31figure out what I mean. Um
  7345. 5:26:33there's stopping slop. Obviously, a big
  7346. 5:26:36theme of here, but like we definitely
  7347. 5:26:38believe this. This is my props I have to
  7348. 5:26:40We have a stop gen AI baseball bat for
  7349. 5:26:42type safe. It is very very awesome. I
  7350. 5:26:45want to talk about the root of slop. Um
  7351. 5:26:48the the root of slop is that AI is
  7352. 5:26:49unreliable. Therefore, you as a company
  7353. 5:26:52or you as you know like someone who's
  7354. 5:26:53doing stuff, it can't be used for
  7355. 5:26:55decisions that costs you stuff.
  7356. 5:26:57But it's actually totally fine to push
  7357. 5:26:59the costs into other people, which is
  7358. 5:27:00horrible, you know, externalized costs.
  7359. 5:27:02And then this is what ends up becoming
  7360. 5:27:04slop.
  7361. 5:27:04You know, if people actually read the
  7362. 5:27:06stuff and filter it and like care about
  7363. 5:27:07the quality, it becomes not slop. But
  7364. 5:27:10now the world is full of this because
  7365. 5:27:12there's an asymmetric advantage for
  7366. 5:27:13people who are like bad actors and don't
  7367. 5:27:15give a So I think that we can save
  7368. 5:27:17the world from at least inbound hell
  7369. 5:27:19among many other things and that would
  7370. 5:27:20be very cool. Um and even something I
  7371. 5:27:24think is super sick is cursed software.
  7372. 5:27:26Like why? If you guys ever seen the
  7373. 5:27:28subreddit bad UI battles, it is so
  7374. 5:27:31cool even though they make the
  7375. 5:27:33horrible interfaces like this whole game
  7376. 5:27:35of like would you like to delete your
  7377. 5:27:36account becomes, you know, a random
  7378. 5:27:38thing. And I actually think doing things
  7379. 5:27:41because we can is what software
  7380. 5:27:43engineering is about cuz who knows,
  7381. 5:27:45maybe 1% of these ideas actually lead to
  7382. 5:27:47something super awesome. And I think
  7383. 5:27:50like really cultivating even bad ideas
  7384. 5:27:52is kind of what being like this, you
  7385. 5:27:53know, playful hacker is all about.
  7386. 5:27:56And generally just automating
  7387. 5:27:59everything, this is what the follow-up,
  7388. 5:28:01this is this is a slop image, but sorry
  7389. 5:28:03the logo is a little bit wrong, but
  7390. 5:28:04don't stare at that, that's a little
  7391. 5:28:05weird. Um
  7392. 5:28:07this is what a lot of the follow-up
  7393. 5:28:09talks are going to be about. Thank you.
  7394. 5:28:11Um
  7395. 5:28:12and if you're a software engineer, you
  7396. 5:28:14almost certainly think in the world of
  7397. 5:28:16bits. If you know anything about
  7398. 5:28:17software
  7399. 5:28:18and you've interacted with software, you
  7400. 5:28:21know that there are things software can
  7401. 5:28:23do and software cannot do because that
  7402. 5:28:25is just only obvious, right? Like
  7403. 5:28:27there's some things that you know you
  7404. 5:28:29need to do because that is not
  7405. 5:28:30compatible with software. But what if
  7406. 5:28:33those constraints were no longer true?
  7407. 5:28:34Maybe not today, but maybe in like 6
  7408. 5:28:36months or a year, what couldn't we
  7409. 5:28:38automate? And that's kind of where I
  7410. 5:28:40want to leave this whole thing because I
  7411. 5:28:41think there is very likely to be a
  7412. 5:28:44future where it's not a question of
  7413. 5:28:45whether or not we can automate
  7414. 5:28:47something, it's going to be a question
  7415. 5:28:48of like how much work is it, which is
  7416. 5:28:49how it is right now for the stuff that
  7417. 5:28:51can be automated. And that would be cool
  7418. 5:28:53as
  7419. 5:28:54So hell yeah, thank you.
  7420. 5:28:56Um
  7421. 5:28:58>> [applause]
  7422. 5:28:58>> I'm
  7423. 5:28:59super overtime, but thank you to the
  7424. 5:29:02early believers. You know, we had no
  7425. 5:29:04idea so many people would resonate with
  7426. 5:29:05this. We thought we were the insane
  7427. 5:29:07ones, but it turns out like we were
  7428. 5:29:09actually the majority of the world who
  7429. 5:29:10realized AI could be so much more.
  7430. 5:29:13Thank you to like the people who are
  7431. 5:29:14like really realistic about reliability.
  7432. 5:29:16Everyone knows I hate benchmarks and a
  7433. 5:29:18lot of other people are really screaming
  7434. 5:29:19out like benchmarks suck.
  7435. 5:29:21They're actually
  7436. 5:29:22they hurt the people who do not have
  7437. 5:29:24time for anything other than benchmarks
  7438. 5:29:26most. It's tragic. And for the people
  7439. 5:29:27who are pushing the limits of software,
  7440. 5:29:29like yes. And we want to be there
  7441. 5:29:32with you helping you all do that. To the
  7442. 5:29:34dreamers and the people who don't want
  7443. 5:29:36to be limited by everything that has
  7444. 5:29:38happened so far, we want to really fight
  7445. 5:29:41for you and we want this to be as
  7446. 5:29:43reliable as a fourth logic gate. And
  7447. 5:29:46that is a very high bar and we have like
  7448. 5:29:48a lot of work to do. So, thank you,
  7449. 5:29:50thank you.
  7450. 5:29:53>> [applause]
  7451. 5:31:08>> Yeah.
  7452. 5:34:33>> Hi everyone.
  7453. 5:34:35My name is Chinmay and I am a software
  7454. 5:34:37engineer at Clein.
  7455. 5:34:39I work on the product engineering side
  7456. 5:34:41of Clein assistant.
  7457. 5:34:43Today I'm going to give a talk
  7458. 5:34:45on how Clein uses chef.
  7459. 5:34:48The title of the talk is beyond
  7460. 5:34:50generation
  7461. 5:34:51fast and structured AI decisions with
  7462. 5:34:53chef at Clein.
  7463. 5:34:56If you look at it,
  7464. 5:34:57most AI product steps
  7465. 5:34:59are bounded decisions and not generation
  7466. 5:35:02problems.
  7467. 5:35:03So to give a couple of examples,
  7468. 5:35:05user query intent classification,
  7469. 5:35:08a ranker picking a suggestion out of n
  7470. 5:35:10options,
  7471. 5:35:11checking whether a citation is valid or
  7472. 5:35:13not, routing to the correct model or
  7473. 5:35:15expert, so on so forth.
  7474. 5:35:18Till now we
  7475. 5:35:19have been paying the full LLM cost and
  7476. 5:35:22latency to get an answer to all these
  7477. 5:35:24bounded problems.
  7478. 5:35:28Jev at Glean fills the gap between
  7479. 5:35:31general LLMs and fine tuned classifiers.
  7480. 5:35:35Classification itself isn't a new
  7481. 5:35:36problem. Like people have been trying to
  7482. 5:35:38solve it since forever.
  7483. 5:35:40We can have multiple ways to solve it.
  7484. 5:35:42Some of the things that we have tried
  7485. 5:35:43out
  7486. 5:35:44include using a general purpose LLM,
  7487. 5:35:47using an open model, something like
  7488. 5:35:49Laya,
  7489. 5:35:50fine tuning that open model
  7490. 5:35:52or using Jev.
  7491. 5:35:55Now all these approaches and all these
  7492. 5:35:57different ways have their pros and cons.
  7493. 5:36:00With a general purpose LLM,
  7494. 5:36:02we incur a lot of cost and latency,
  7495. 5:36:04while with something like Jev, we get no
  7496. 5:36:06generated text output.
  7497. 5:36:09Another advantage that we have been
  7498. 5:36:10seeing with Jev is that as the taxonomy
  7499. 5:36:13of the decision space changes, we don't
  7500. 5:36:15have to fine tune these models again and
  7501. 5:36:17again.
  7502. 5:36:18We can just change our prompt a bit and
  7503. 5:36:20boom, it works out of the box.
  7504. 5:36:26Now I'll walk you through how we have
  7505. 5:36:29integrated Gen or these classification
  7506. 5:36:32how we have used Jev at Glean to solve
  7507. 5:36:35these classification problem statements.
  7508. 5:36:37I'll go into some of these problems in
  7509. 5:36:39great detail.
  7510. 5:36:42Before we go into these individual
  7511. 5:36:44problems, I wanted to touch upon how we
  7512. 5:36:47evaluate these changes. Like we have a
  7513. 5:36:49huge customer base having millions of
  7514. 5:36:51users. So we don't want any of the
  7515. 5:36:53change that we make to regress our
  7516. 5:36:54product.
  7517. 5:36:56We have a multi phase approach, a
  7518. 5:36:58rollout approach to any change that we
  7519. 5:37:00are making.
  7520. 5:37:02This includes having a golden eval set
  7521. 5:37:04for almost every possible thing. Like
  7522. 5:37:06any place where we have generative AI,
  7523. 5:37:08any place where we have
  7524. 5:37:10any form of intelligence involved, we
  7525. 5:37:12have a golden eval set for that space.
  7526. 5:37:15We soak all of these changes for a
  7527. 5:37:17significant amount of time for our
  7528. 5:37:18internal use case. Like we have around
  7529. 5:37:201500 people at Glean right now. We soak
  7530. 5:37:23all of these changes within our Glean
  7531. 5:37:26instance for like couple of weeks. We
  7532. 5:37:27get active feedback. We have a very
  7533. 5:37:29active feedback loop from our internal
  7534. 5:37:31users. We work upon that and then we go
  7535. 5:37:33for an AB testing
  7536. 5:37:36with our customers.
  7537. 5:37:39The first problem that we tried out with
  7538. 5:37:41a decision decision model was query
  7539. 5:37:44classification.
  7540. 5:37:46Query classification is when we want to
  7541. 5:37:48know the nature of user queries. This
  7542. 5:37:49helps us
  7543. 5:37:51realize what kind of
  7544. 5:37:53what kind of
  7545. 5:37:55jobs are people using Glean for? What
  7546. 5:37:57kind of Say for example, if people are
  7547. 5:37:58using more for sales like queries, we
  7548. 5:38:00focus more on that. If we people are
  7549. 5:38:02using more for engineering type use
  7550. 5:38:03cases, we focus more on that.
  7551. 5:38:06We tried out three different approaches
  7552. 5:38:07over here and this is
  7553. 5:38:09data from our test bed.
  7554. 5:38:12We used an open source Llama model.
  7555. 5:38:14Then we tried out Jev and then we fine
  7556. 5:38:16tuned the Llama model
  7557. 5:38:18on our own compute resources.
  7558. 5:38:21Jev is the best no training option in
  7559. 5:38:24our opinion.
  7560. 5:38:25It nearly doubles the output quality of
  7561. 5:38:27a base Llama model and is 90 is able to
  7562. 5:38:30perform at 90%
  7563. 5:38:32quality in comparison to a fine tuned
  7564. 5:38:33model fine tuned Llama.
  7565. 5:38:36With Jev, we don't have to go through
  7566. 5:38:38the very tedious
  7567. 5:38:40process of data collection, data
  7568. 5:38:42cleaning,
  7569. 5:38:43incurring the cost for compute like
  7570. 5:38:45training the model,
  7571. 5:38:46fine tuning the model again and again as
  7572. 5:38:48the taxonomy changes.
  7573. 5:38:50And at the same time, we don't have to
  7574. 5:38:52worry about hosting the model or
  7575. 5:38:54incurring the inference cost.
  7576. 5:38:59The next problem we went to is expert
  7577. 5:39:01routing. This is a pretty common problem
  7578. 5:39:03statement in the industry right now.
  7579. 5:39:04Like hey, when the user query comes in,
  7580. 5:39:06what expert do we route it to? Like how
  7581. 5:39:08do we ensure that the user
  7582. 5:39:11user the response to the user is of the
  7583. 5:39:12best possible quality.
  7584. 5:39:14Glean receives the way we have been
  7585. 5:39:16solving this is Glean receives the user
  7586. 5:39:18query
  7587. 5:39:19and then decides what's the best expert
  7588. 5:39:20to answer the query.
  7589. 5:39:22Till now we have been using a general
  7590. 5:39:23purpose LLM to route to the correct
  7591. 5:39:25expert.
  7592. 5:39:26We set up a test bed
  7593. 5:39:27for J which set up a test bed with N
  7594. 5:39:29experts
  7595. 5:39:30and in our experiments found a
  7596. 5:39:32significantly high precision when using
  7597. 5:39:33J and that too at an almost 8x reduction
  7598. 5:39:36in latency.
  7599. 5:39:39We had to change our implementation or
  7600. 5:39:41our infrastructure significantly to
  7601. 5:39:43integrate J into this specific stage of
  7602. 5:39:46our
  7603. 5:39:46product.
  7604. 5:39:47But the reduction in latency justifies
  7605. 5:39:51that inherent complexity in the system.
  7606. 5:39:53We at Glean have been trying to build
  7607. 5:39:55systems in the most in the simplest
  7608. 5:39:57possible manners. This is one of the
  7609. 5:39:58rare pieces where we believe complexity
  7610. 5:40:01is worth it.
  7611. 5:40:04Next I'll walk you through the ranker
  7612. 5:40:06problem statement that I talked about.
  7613. 5:40:10When we receive a user query
  7614. 5:40:12we also want to generate the best
  7615. 5:40:14follow-up suggestions for the user. Like
  7616. 5:40:16this helps us in getting discoverability
  7617. 5:40:18for our product. This helps us in
  7618. 5:40:19ensuring that the user has the best
  7619. 5:40:20possible experience. We also want to
  7620. 5:40:23recommend or say preempt what the next
  7621. 5:40:25user query might be.
  7622. 5:40:27Again a pretty common problem statement
  7623. 5:40:29that the entire industry has been trying
  7624. 5:40:30to solve for.
  7625. 5:40:31Internally we call this
  7626. 5:40:33system Q
  7627. 5:40:35like
  7628. 5:40:36Q as in the literal Q.
  7629. 5:40:38The way this works, we have like 10 or
  7630. 5:40:4115 different sources or 10 or 15
  7631. 5:40:43different ways
  7632. 5:40:4410 or 15 different sources that are
  7633. 5:40:46generating suggestions independently of
  7634. 5:40:48each other.
  7635. 5:40:50Once those suggestion these suggestions
  7636. 5:40:51are generated by natural language LLMs
  7637. 5:40:54and it's like a proper natural language
  7638. 5:40:56query.
  7639. 5:40:58Once these suggestions are in we have a
  7640. 5:40:59ranking step. So the ranker takes in all
  7641. 5:41:01these suggestions, takes in the
  7642. 5:41:03the first user query, takes in the
  7643. 5:41:05output that Glean gave and then all
  7644. 5:41:08these suggestions in that context.
  7645. 5:41:11The ranker is then asked to decide like
  7646. 5:41:12hey you have these suggestions. What do
  7647. 5:41:13you think is the best one?
  7648. 5:41:15A pretty interesting bottleneck that we
  7649. 5:41:17had over here is we want the response to
  7650. 5:41:19be snappy.
  7651. 5:41:20I cannot We cannot have a state where
  7652. 5:41:23we give the textual response to the user
  7653. 5:41:25and then wait 5 seconds for generating
  7654. 5:41:28the follow-up suggestion.
  7655. 5:41:29We have to be snappy. That why we have
  7656. 5:41:32time limits on these suggestions. So
  7657. 5:41:33right now this step had a time limit of
  7658. 5:41:35one 1.5 to 2 seconds.
  7659. 5:41:38Jeff, no doubt gave us a better quality
  7660. 5:41:40than better quality and far better
  7661. 5:41:41latency than what we had with the
  7662. 5:41:43general purpose LLM, but
  7663. 5:41:46two things stood out over there.
  7664. 5:41:47First, we
  7665. 5:41:49were able to give a suggestion within
  7666. 5:41:51our time bounds
  7667. 5:41:53in 99.2%
  7668. 5:41:55cases.
  7669. 5:41:56With the general purpose LLM, this was
  7670. 5:41:58almost 90%
  7671. 5:42:00So this ensures that we
  7672. 5:42:02have user engagement and far more
  7673. 5:42:05user queries now.
  7674. 5:42:07The second thing is
  7675. 5:42:09at a glean, whenever we use these LLMs,
  7676. 5:42:11we have guard rails. Guard rails in the
  7677. 5:42:12sense that
  7678. 5:42:14here LLM might not be good at generating
  7679. 5:42:16a response or LLM might be making some
  7680. 5:42:17specific kind of mistakes. So we have
  7681. 5:42:20checks and balances for all those
  7682. 5:42:22possible mistakes that we see. We keep a
  7683. 5:42:24log of like we keep a log of all these
  7684. 5:42:27checks that are incurred at any point of
  7685. 5:42:29time.
  7686. 5:42:30With a general purpose LLM, we
  7687. 5:42:33were able to generate a valid suggestion
  7688. 5:42:35in almost 90% of the use 90% of the
  7689. 5:42:38cases. We still had some places where
  7690. 5:42:40LLM was either generating a generating a
  7691. 5:42:42response in a different language.
  7692. 5:42:44The output
  7693. 5:42:46did not meet certain other criteria that
  7694. 5:42:48we had
  7695. 5:42:50that we had written programmatically.
  7696. 5:42:52With Jeff, we found
  7697. 5:42:54zero cases to be there when the output
  7698. 5:42:56was invalid.
  7699. 5:42:58Now a a natural question is like, hey,
  7700. 5:42:59but you would have used a different
  7701. 5:43:01prompt. Yes, the prompt was somewhat
  7702. 5:43:02different,
  7703. 5:43:03but we tried to mimic the prompt in the
  7704. 5:43:04two cases to the largest extent possible
  7705. 5:43:06so that we don't have a divergence. Like
  7706. 5:43:08again, we at glean want to keep the
  7707. 5:43:09system as simple as possible.
  7708. 5:43:14Another benefit we see with using a
  7709. 5:43:16decision model is how we can control the
  7710. 5:43:19thresholds.
  7711. 5:43:20Because in this ranker problem statement
  7712. 5:43:23we are getting specific probabilities of
  7713. 5:43:25each output. We are not
  7714. 5:43:27restricted to a binary yes or no
  7715. 5:43:28response. We get like hey these were the
  7716. 5:43:30five options and these are the
  7717. 5:43:32probabilities of these five options.
  7718. 5:43:34With this we are able to fine tune the
  7719. 5:43:36thresholds based on the deployment. We
  7720. 5:43:39have put in self learning systems
  7721. 5:43:41which based on the user based on the
  7722. 5:43:43customer deployment
  7723. 5:43:45and the nature of usage that the
  7724. 5:43:47specific customer has
  7725. 5:43:49fine tunes the thresholds by itself.
  7726. 5:43:51This ensures that the experience in each
  7727. 5:43:55and every customer out there is the best
  7728. 5:43:57possible based on how they use Glean,
  7729. 5:44:00not a general purpose solution that fits
  7730. 5:44:02all.
  7731. 5:44:08Okay, this is an interesting one.
  7732. 5:44:10Now moving on from how we have been
  7733. 5:44:13making the lives or the experience of
  7734. 5:44:15our end users better
  7735. 5:44:17we also tried using Jev and some of the
  7736. 5:44:20some some other parts of our development
  7737. 5:44:22pipeline. One of them was
  7738. 5:44:25how we use LLM as judges.
  7739. 5:44:27Like
  7740. 5:44:28all of us have been using LLM as judges
  7741. 5:44:30for quite some time now. There's an
  7742. 5:44:31input, there's an output
  7743. 5:44:33and we use an LLM to judge how the
  7744. 5:44:35quality of the output is. Now naturally
  7745. 5:44:37whenever we use LLM as a judge we repeat
  7746. 5:44:40the experiment multiple times because to
  7747. 5:44:42account for the variability in the
  7748. 5:44:43response.
  7749. 5:44:45At Glean we used Jev for a very specific
  7750. 5:44:47part of our evaluation pipeline.
  7751. 5:44:50Glean responses or like any other AI
  7752. 5:44:52product out there has like citations.
  7753. 5:44:54This is how people trust that Glean
  7754. 5:44:57Glean's answer was correct and it it is
  7755. 5:44:59grounded in the correct document.
  7756. 5:45:01Now these site we hold these citations
  7757. 5:45:04to a very high bar
  7758. 5:45:05and we have a specific part in our
  7759. 5:45:07evaluation pipeline where we check
  7760. 5:45:10whether the cited document really
  7761. 5:45:12supports the claim made in the answer or
  7762. 5:45:14not.
  7763. 5:45:16This again seems to be a pretty simple
  7764. 5:45:17yes or no question like hey whether the
  7765. 5:45:20claim is
  7766. 5:45:21uh supported or not.
  7767. 5:45:22With Jev, no doubt we had a far better
  7768. 5:45:25latency and cost like
  7769. 5:45:27visible in the slide
  7770. 5:45:28uh
  7771. 5:45:29over here.
  7772. 5:45:31The interesting thing we noticed over
  7773. 5:45:32here is how consistent and reproducible
  7774. 5:45:35Jev results were in comparison to
  7775. 5:45:36something like say Luna.
  7776. 5:45:39In the 28 or 30 use cases that we had
  7777. 5:45:41like only one
  7778. 5:45:43run had like a difference in response.
  7779. 5:45:46Apart from that everything was
  7780. 5:45:48consistent. What this ensure this this
  7781. 5:45:50ensures two things for us as engineers
  7782. 5:45:52or us as developers at Clean.
  7783. 5:45:55We are not only improving the
  7784. 5:45:57quality of our product or we are not
  7785. 5:45:58only improving Clean in a linear fashion
  7786. 5:46:00anymore.
  7787. 5:46:01Any improvement of this extent in the
  7788. 5:46:04development pipeline leads to an
  7789. 5:46:06exponential increase in how we can
  7790. 5:46:09improve the product. So if I'm if I as a
  7791. 5:46:11developer I'm able to ship things like
  7792. 5:46:135x faster like this is significant
  7793. 5:46:15reduction in
  7794. 5:46:16latency like if I run this experiment
  7795. 5:46:19across thousands and thousands of
  7796. 5:46:20entries this is a lot of time.
  7797. 5:46:22And again I don't have to run the same
  7798. 5:46:24uh input output pair multiple times
  7799. 5:46:26because the result from Jev are quite
  7800. 5:46:27consistent.
  7801. 5:46:29So if I'm able to increase my own
  7802. 5:46:31productivity by like 5x or 6x I can ship
  7803. 5:46:346x times the features to our customers
  7804. 5:46:37and this happens across like 500
  7805. 5:46:38engineers or 300 engineers at Clean.
  7806. 5:46:41So the
  7807. 5:46:42impact of
  7808. 5:46:44a a good decision model is no longer
  7809. 5:46:46linear. It is
  7810. 5:46:48exponential on how we can improve the
  7811. 5:46:49product.
  7812. 5:46:52Okay.
  7813. 5:46:54Now moving on from product and uh
  7814. 5:46:56LLMs judges
  7815. 5:46:58there's a pretty
  7816. 5:46:59import this this this is a pretty
  7817. 5:47:01important part of
  7818. 5:47:03how we
  7819. 5:47:04use our customer data like security is
  7820. 5:47:06something that all our customers expect
  7821. 5:47:09from us.
  7822. 5:47:10So for example we have checks and
  7823. 5:47:12balances on all the inputs and outputs
  7824. 5:47:14coming from a tool, all the inputs and
  7825. 5:47:16outputs uh from an LLM.
  7826. 5:47:19Till now we at Glean have like a
  7827. 5:47:21multi-phase approach. There is like
  7828. 5:47:24there are two phases to this. The first
  7829. 5:47:26one is like a recall heavy phase and the
  7830. 5:47:28second one is the final heavy loaded
  7831. 5:47:29LLM. All these calls are made in
  7832. 5:47:32parallel to like the actual tool input
  7833. 5:47:34output calls.
  7834. 5:47:37Till now we used to host a virtue
  7835. 5:47:39classifier using T4 GPUs across all our
  7836. 5:47:41customers. So Glean is a single tenant
  7837. 5:47:43solution like we don't use or share
  7838. 5:47:46resources across customers. Every
  7839. 5:47:47customer gets
  7840. 5:47:49gets its own deployment, gets its own
  7841. 5:47:51instance and gets all the resources. So
  7842. 5:47:54assuming we have 1,000 customers out
  7843. 5:47:56there, we had to maintain like virtue
  7844. 5:47:58classifiers across all these 1,000
  7845. 5:48:00customers which becomes a pretty
  7846. 5:48:02difficult problem statement in itself
  7847. 5:48:04which takes up a lot of time from Glean
  7848. 5:48:05engineers.
  7849. 5:48:07So looking at the results over here, the
  7850. 5:48:10overall recall of the first phase where
  7851. 5:48:12we used to have a virtue classifier, we
  7852. 5:48:14replaced that with Jev, went from 31.5%
  7853. 5:48:17to 90.2%.
  7854. 5:48:19At the same time we are seeing a
  7855. 5:48:20significant reduction in cost. Now this
  7856. 5:48:22cost is only the resource cost mentioned
  7857. 5:48:25over here, but we are also saving cost
  7858. 5:48:28on the engineering resources over here.
  7859. 5:48:30We don't have to invest like two or
  7860. 5:48:32three engineers who are full-time
  7861. 5:48:33maintaining those virtue classifiers or
  7862. 5:48:35those T4 GPUs across customers.
  7863. 5:48:37The the improvement or the savings that
  7864. 5:48:40we have
  7865. 5:48:41are far more than 50 percent.
  7866. 5:48:46This slide concludes all the results I
  7867. 5:48:48have talked about till now.
  7868. 5:48:50Jev at Glean has been able to influence
  7869. 5:48:52the product experience of the users, has
  7870. 5:48:54been able to
  7871. 5:48:56impact how we as developers have been
  7872. 5:48:58working at Glean and most importantly it
  7873. 5:49:01also has a very good positive impact on
  7874. 5:49:03the security that we provide to our
  7875. 5:49:04customers.
  7876. 5:49:06A simple rule that we have been using is
  7877. 5:49:09can the out
  7878. 5:49:10in conclusion a simple rule that we have
  7879. 5:49:11been using at Glean
  7880. 5:49:12is can the output be listed in advance?
  7881. 5:49:15If no then it's a free form generative
  7882. 5:49:17text problem statement.
  7883. 5:49:18Go back to something that can generate
  7884. 5:49:20free form text. If yes then use Jev.
  7885. 5:49:23Are levels stable and volume high?
  7886. 5:49:25If this is the case where the labels are
  7887. 5:49:27stable, which is rarely the case
  7888. 5:49:29anymore,
  7889. 5:49:30uh
  7890. 5:49:31you can also fine-tune a model, but if
  7891. 5:49:33the labels change, which in my opinion
  7892. 5:49:35is quite frequent these days, then again
  7893. 5:49:37use something like Jev.
  7894. 5:49:40Obviously, if you need dynamic text, we
  7895. 5:49:42can like feel free to try out more
  7896. 5:49:45hybrid approaches like Jev along with uh
  7897. 5:49:48free-form text LLMs.
  7898. 5:49:52This ends my presentation. So thank you
  7899. 5:49:54everyone for listening.
  7900. 5:49:56I have added a blog like a link to the
  7901. 5:49:58blog post where we have posted the exact
  7902. 5:50:00same results in great detail.
  7903. 5:50:03Feel free to catch me on the outside if
  7904. 5:50:05you have more questions. Thank you.
  7905. 5:50:08>> [applause]
  7906. 5:52:15>> Hello everyone.
  7907. 5:52:18My name is Sibi and I'm joined on stage
  7908. 5:52:20by Kevin.
  7909. 5:52:22We're here to talk about how we believe
  7910. 5:52:24that your automations are overthinking
  7911. 5:52:27it.
  7912. 5:52:28So
  7913. 5:52:29to tell us a little bit about ourselves,
  7914. 5:52:31we are from a company called UiPath. We
  7915. 5:52:33have been doing automation for the last
  7916. 5:52:3520 years.
  7917. 5:52:37We have been doing RPA before and we
  7918. 5:52:39service more than 10,000 organizations
  7919. 5:52:42around the around the world
  7920. 5:52:44and we build
  7921. 5:52:45an agentic process
  7922. 5:52:47orchestration layer which basically
  7923. 5:52:49brings agents, robots, and people
  7924. 5:52:51together to interact with each other and
  7925. 5:52:54build extremely scalable
  7926. 5:52:56automations that run for days and weeks.
  7927. 5:52:59So
  7928. 5:53:00we were inspired by this particular
  7929. 5:53:03slide that you're seeing right now and
  7930. 5:53:04you saw today today before us as well
  7931. 5:53:08when
  7932. 5:53:09Diego presented it three months ago, the
  7933. 5:53:11same slide,
  7934. 5:53:13it resonated really well with us because
  7935. 5:53:16this is how we see automation as well.
  7936. 5:53:19So the traditional software was is
  7937. 5:53:22basically RPA. It's just robotic process
  7938. 5:53:24automation, if else, just execute some
  7939. 5:53:27task and then keeps going. The control
  7940. 5:53:28flow is deterministic, just keeps going.
  7941. 5:53:31And then when the LLMs came out,
  7942. 5:53:34we all raced to go and build the
  7943. 5:53:36just-in-time software where we have this
  7944. 5:53:38one agent, we give it a bunch of tools,
  7945. 5:53:40we give it a task and ask you to finish.
  7946. 5:53:43And then we were seeing
  7947. 5:53:44good results, bad results, and then we
  7948. 5:53:46added evaluations to slowly build the
  7949. 5:53:48results up.
  7950. 5:53:49And then finally we landed on agentic
  7951. 5:53:52process orchestration, which kind of
  7952. 5:53:54weaves in this cognitive decision
  7953. 5:53:57through a logical
  7954. 5:53:59decision layer. So you have
  7955. 5:54:01deterministic calls and clear cognitive
  7956. 5:54:04decision points where you would use an
  7957. 5:54:07LLM to control the control flow rather
  7958. 5:54:10than leaving it all in the hands of an
  7959. 5:54:11agent.
  7960. 5:54:12So
  7961. 5:54:13at this time we we saw where J would fit
  7962. 5:54:16in and when Diego was talking about J,
  7963. 5:54:19we were really excited because we saw J
  7964. 5:54:22fit in in a very particular niche area.
  7965. 5:54:24So
  7966. 5:54:26here we see that
  7967. 5:54:28J is a very powerful tool. As Diego was
  7968. 5:54:31saying, it's a
  7969. 5:54:33foundational tool that we would be able
  7970. 5:54:34to integrate into all automation
  7971. 5:54:37and it fits right in between a
  7972. 5:54:39deterministic case and an agent case. So
  7973. 5:54:43J is extremely fast and most of you guys
  7974. 5:54:46would have tried it and we were
  7975. 5:54:48surprised by how fast and how parallel
  7976. 5:54:50we could make it work and and how
  7977. 5:54:53intelligent it was. So we were able to
  7978. 5:54:55benchmark it around how good it was
  7979. 5:54:57compared to an LLM and was pretty good,
  7980. 5:54:59not as a full LLM, but it was pretty
  7981. 5:55:02good at in terms of intelligence as
  7982. 5:55:03well. So it falls in between and there
  7983. 5:55:06are so many ways we could use J and
  7984. 5:55:09I want to hand it off to Kevin to take
  7985. 5:55:11take you guys through a scenario where
  7986. 5:55:14we use J in UI Path Maestro flow.
  7987. 5:55:17>> Yeah. Thanks TV.
  7988. 5:55:19So hi everybody.
  7989. 5:55:21Today I want to actually walk us through
  7990. 5:55:22a little bit about what UI Path is
  7991. 5:55:24building with UI Path Maestro flow and
  7992. 5:55:26talk through a little bit of how we
  7993. 5:55:29would go through and coordinate humans,
  7994. 5:55:31robots, and agents all together inside a
  7995. 5:55:33one canvas.
  7996. 5:55:34Uh what we're going to walk through
  7997. 5:55:35today is essentially um you know
  7998. 5:55:37essentially a flow that will demonstrate
  7999. 5:55:40how we kind of weave all these things
  8000. 5:55:41together.
  8001. 5:55:42And the way that we're approaching this
  8002. 5:55:44is that we're going to end up having
  8003. 5:55:45system one agents which are going to be
  8004. 5:55:47able to make quick intelligence happen
  8005. 5:55:49within our process. Think about things
  8006. 5:55:51like hey is this transaction fraud or
  8007. 5:55:53not?
  8008. 5:55:54Um
  8009. 5:55:55additionally we'll have um
  8010. 5:55:58system two agents running where we're
  8011. 5:55:59going to have more of a methodical
  8012. 5:56:00deeper investigation that's going on
  8013. 5:56:02where agents are going off and calling
  8014. 5:56:03tools and we'll see kind of this pattern
  8015. 5:56:06these patterns start to emerge where
  8016. 5:56:07we're able to hand off from one to the
  8017. 5:56:09next.
  8018. 5:56:10And so with that I'll I'll jump into an
  8019. 5:56:12actual demo where we're going to
  8020. 5:56:13demonstrate a account fraud uh scenario.
  8021. 5:56:17And so our demo starts here with uh
  8022. 5:56:20effectively a transaction coming in for
  8023. 5:56:23$48,000.
  8024. 5:56:25And this wasn't something that we were
  8025. 5:56:27expecting and so uh as I'm reviewing my
  8026. 5:56:29bank portal here I'm going to go ahead
  8027. 5:56:30and report this as a fraudulent
  8028. 5:56:31transaction.
  8029. 5:56:33And so now this would end up smitting it
  8030. 5:56:35off into our UI Path Maestro flow where
  8031. 5:56:37this would act as essentially the
  8032. 5:56:39trigger for this end-to-end workflow to
  8033. 5:56:42to go off and start processing.
  8034. 5:56:44What we're looking at is a collection of
  8035. 5:56:45various different nodes that are each
  8036. 5:56:47going to kind of uh
  8037. 5:56:49be orchestrated from the left side of
  8038. 5:56:51the canvas over to the right. And we're
  8039. 5:56:52going to use uh essentially a
  8040. 5:56:54combination of system one and system two
  8041. 5:56:56models to actually process this uh and
  8042. 5:56:58do an analysis of if this is fraud or
  8043. 5:56:59not.
  8044. 5:57:00And so the first thing that actually
  8045. 5:57:02takes place is we're going to go through
  8046. 5:57:04and get account activity.
  8047. 5:57:06We're going to
  8048. 5:57:07send this off to Jev to basically pull
  8049. 5:57:0960 transactions for the most recent uh
  8050. 5:57:12credit card purchases that were made and
  8051. 5:57:14analyze each one of those individually
  8052. 5:57:16whether or not these are fraud. And this
  8053. 5:57:17is happening in real time where Jev's
  8054. 5:57:19able to process these transactions and
  8055. 5:57:21ask that question of hey is this fraud
  8056. 5:57:23or not within fractions of a second.
  8057. 5:57:26Now we found that there's about four
  8058. 5:57:27transactions that recently appeared that
  8059. 5:57:30it thinks are fraud.
  8060. 5:57:31Now in parallel to that we're able to
  8061. 5:57:33kick off another part of our flow, which
  8062. 5:57:35is doing more of a semantic analysis on
  8063. 5:57:37the entire account. We're looking at
  8064. 5:57:39that one transaction that came in for
  8065. 5:57:41$48,000 which was flagged, and we're
  8066. 5:57:44asking a question to Jev is, hey, is
  8067. 5:57:45there enough information here for us to
  8068. 5:57:47determine whether or not this is
  8069. 5:57:48actually fraud?
  8070. 5:57:50And just as we heard Diogo talk about
  8071. 5:57:52this earlier where you have these new
  8072. 5:57:53kind of paradigms and logic come up,
  8073. 5:57:56we ended up uh finding from this that
  8074. 5:57:58there's uh a scoring that took place and
  8075. 5:58:01we needed more information. Um we were
  8076. 5:58:03asking the question, do we have enough
  8077. 5:58:04information for fraud?
  8078. 5:58:05And for that, what we're going to end up
  8079. 5:58:06doing is kicking off a system two agent
  8080. 5:58:09that's going to go spin up and run for
  8081. 5:58:11much larger longer calling a bank API
  8082. 5:58:14and finding that, you know, there was a
  8083. 5:58:15$48,000 transaction from a payee uh
  8084. 5:58:19basically creating a new account on his
  8085. 5:58:21phone and it was earlier that day that
  8086. 5:58:23that account got created. So, high
  8087. 5:58:24indicator of fraud.
  8088. 5:58:26We circle back and ask that same
  8089. 5:58:28question now to Jev and ask, hey, do we
  8090. 5:58:29have enough evidence now? Jev agrees and
  8091. 5:58:32it finds 100% suspicious activity. And
  8092. 5:58:34we would hand that off effectively to
  8093. 5:58:36another grading model that combines all
  8094. 5:58:38those transactions that we found with
  8095. 5:58:40the system payload and then finds that,
  8096. 5:58:42you know, there is fraud in this
  8097. 5:58:43particular use case. Um uh squared of,
  8098. 5:58:46you know, one to five we got 4.97.
  8099. 5:58:49Now, stepping out of this um you know,
  8100. 5:58:51typically what would happen from this
  8101. 5:58:53point onwards is that we go off to a
  8102. 5:58:54human to review, we'd have somebody kind
  8103. 5:58:56of review both the system one and system
  8104. 5:58:58two logs and then find that there was,
  8105. 5:59:00you know, potential fraud and freeze the
  8106. 5:59:01account.
  8107. 5:59:02But, you know, effectively what we're
  8108. 5:59:04seeing from customers that are going
  8109. 5:59:05through and implementing Jev inside of
  8110. 5:59:07their workflows is that they're able to
  8111. 5:59:09replace the typical LLM calls that are
  8112. 5:59:11going on behind the scenes. And in in
  8113. 5:59:13process we're saving about 50 uh times
  8114. 5:59:16uh we're we're increasing the speed
  8115. 5:59:18about 50 times while also being 7.1 uh
  8116. 5:59:22times cheaper
  8117. 5:59:23um than calling a traditional LLM.
  8118. 5:59:26Now, we we've gone through and and kind
  8119. 5:59:28of shown how customers would use this
  8120. 5:59:30within our technology, but we're also
  8121. 5:59:32pointing the finger back inwards where
  8122. 5:59:34we're trying to ask the of how can
  8123. 5:59:36UiPath use this within the products and
  8124. 5:59:38services that we offer to be able to
  8125. 5:59:40help customers out.
  8126. 5:59:42And there's a number of different
  8127. 5:59:43patterns that we're kind of exploring
  8128. 5:59:44with Jev. Um and today we're going to
  8129. 5:59:46talk about um a re-ranking problem as
  8130. 5:59:48well as uh what we're using inside of
  8131. 5:59:50email mining.
  8132. 5:59:52And so to set the stage for for kind of
  8133. 5:59:54re-ranking, um effectively what we're
  8134. 5:59:56trying to do is run and add uh within
  8135. 5:59:59our retrieval augmented generation
  8136. 6:00:01within our agents, the ability for us to
  8137. 6:00:03find more appropriate chunks to be able
  8138. 6:00:05to provide to our agent.
  8139. 6:00:07And we do that by basically finding
  8140. 6:00:09relevant documents first,
  8141. 6:00:11and then using um typically an LLM to do
  8142. 6:00:13this, but we're experimenting with also
  8143. 6:00:15using Jev to go through and find the
  8144. 6:00:17various different chunks that make are
  8145. 6:00:19most relevant to answering the question,
  8146. 6:00:21and then providing those as as context
  8147. 6:00:23to the LLM.
  8148. 6:00:24And we've run a couple of
  8149. 6:00:26experiments on this now, and I'd like to
  8150. 6:00:27hand it back off to Chibi who's going to
  8151. 6:00:29end up telling us a little bit more
  8152. 6:00:30about what we were able to find.
  8153. 6:00:33>> Yeah.
  8154. 6:00:33So this is where things got really
  8155. 6:00:35interesting for us. This is like the
  8156. 6:00:37first week of Jev. We had got our hands
  8157. 6:00:39on Jev.
  8158. 6:00:40And we have these extremely strong
  8159. 6:00:43benchmarks and uh eval sets that we had
  8160. 6:00:46for re-ranking that we have been running
  8161. 6:00:48for some time. And we need those for
  8162. 6:00:50like when we change from one model to
  8163. 6:00:52another. Uh and we realized, okay, why
  8164. 6:00:55not
  8165. 6:00:56take these eval sets and build uh
  8166. 6:00:59hill climbing harness and ask Astra
  8167. 6:01:03to say, okay, this is my score I can get
  8168. 6:01:06with an LLM. Here is Jev. Here are the
  8169. 6:01:09three APIs.
  8170. 6:01:10Find a strategy which can achieve that
  8171. 6:01:13score.
  8172. 6:01:14And let it run.
  8173. 6:01:15And it ran for the entire night for 14
  8174. 6:01:18hours in a session. And it came up with
  8175. 6:01:21so many different strategies where it
  8176. 6:01:23worked, where it didn't work.
  8177. 6:01:24And then we were slowly seeing the
  8178. 6:01:26number go up. Every two hours there
  8179. 6:01:28would be one strategy would perform
  8180. 6:01:30slightly better.
  8181. 6:01:31This was so surprising for us because
  8182. 6:01:33what we thought was as
  8183. 6:01:36as an API, if you just call it like when
  8184. 6:01:38you just send all of the go up
  8185. 6:01:41up.
  8186. 6:01:41Yeah. So we call it uh we were getting
  8187. 6:01:44only point 866 which was very very low
  8188. 6:01:48at that point when we were trying it out
  8189. 6:01:50when we were thinking how do we increase
  8190. 6:01:51it and then it slowly got got to the
  8191. 6:01:54number of point 889 while it was uh
  8192. 6:01:57trying out new strategies and that
  8193. 6:01:59strategy is called tournament and I want
  8194. 6:02:01to showcase that strategy which it which
  8195. 6:02:04these hill climbing models actually come
  8196. 6:02:06up with and how you were able to see the
  8197. 6:02:08intelligence squeaked out of Jev just by
  8198. 6:02:11applying Jev in a different architecture
  8199. 6:02:14and different call method and I want to
  8200. 6:02:16take you through tournament now.
  8201. 6:02:19So what tournament did is that it
  8202. 6:02:21actually reimagined re ranking
  8203. 6:02:23differently instead of looking at every
  8204. 6:02:25chunk and saying that it is how good
  8205. 6:02:28relevant is it for my question it
  8206. 6:02:30created cohorts of different uh
  8207. 6:02:34different groups of these chunks and
  8208. 6:02:36made them fight between each other to be
  8209. 6:02:38the best most relevant chunk by sending
  8210. 6:02:40thousands and thousands of Jev calls for
  8211. 6:02:43each and every one of them and it found
  8212. 6:02:44leaders in those particular cohorts and
  8213. 6:02:48then the winners would be promoted to
  8214. 6:02:49the next round and then in from those
  8215. 6:02:51winners we would again find like top
  8216. 6:02:53winners from those chunks and then we
  8217. 6:02:55will slowly increase and find the
  8218. 6:02:58topmost the cream will rise to the top
  8219. 6:03:00and you will find the
  8220. 6:03:02the most relevant chunks on the whole
  8221. 6:03:03list.
  8222. 6:03:04So this worked extremely well and we
  8223. 6:03:07were testing it and when we were
  8224. 6:03:09surprised to see that even with
  8225. 6:03:10thousands of Jev calls and
  8226. 6:03:14many millions of contacts being used
  8227. 6:03:17we were still under a second we were
  8228. 6:03:19able to make all of these calls right
  8229. 6:03:22and this was a huge revelation for us
  8230. 6:03:24and we realized that Jev is a building
  8231. 6:03:27block that we can adjust and we are we
  8232. 6:03:31have to find these new strategies where
  8233. 6:03:33we would be able to apply this building
  8234. 6:03:35block and it might look less intelligent
  8235. 6:03:38on its own but with these strategies and
  8236. 6:03:41patterns, we can get more intelligent
  8237. 6:03:43with even the current gem model that we
  8238. 6:03:45have, right? And that's that gave us
  8239. 6:03:48like 15 x speed and we tested it out on
  8240. 6:03:51uh our our live uh
  8241. 6:03:54indexes and saw how it performed and
  8242. 6:03:56people were extremely surprised by this
  8243. 6:03:58one. And next we saw how uh Jev could be
  8244. 6:04:02used uh
  8245. 6:04:04for mining. So mining is has a lot of
  8246. 6:04:07stages. I'm going to talk only about the
  8247. 6:04:09classification stage. And what happens
  8248. 6:04:12is we have customers who have millions
  8249. 6:04:14of emails in uh support uh email box.
  8250. 6:04:18And what happens is they want to analyze
  8251. 6:04:20and they want to dig into what happened
  8252. 6:04:21in these emails so that they can take
  8253. 6:04:23some actions. I'm sure every one of you
  8254. 6:04:25guys have one of those email boxes. Uh
  8255. 6:04:27what happens is that like it's so hard
  8256. 6:04:30to find the right categories and
  8257. 6:04:31bucketizations without reading all of
  8258. 6:04:33these emails. So what we used to do is
  8259. 6:04:36that we would go and create uh
  8260. 6:04:38fine tuning to figure out a model that
  8261. 6:04:40classifies them and it would take hours
  8262. 6:04:42and maybe a day to fine tune that
  8263. 6:04:44particular model for the classification
  8264. 6:04:46story. But what we found was this is
  8265. 6:04:48again through hill climbing we found
  8266. 6:04:50this pattern where we use
  8267. 6:04:52we use an LLM to find an initial
  8268. 6:04:55candidate list of categories.
  8269. 6:04:58And then it would say that okay these
  8270. 6:05:00are all the 15 categories that I think
  8271. 6:05:02the emails will fit. Then what we would
  8272. 6:05:04do is that we'll use Jev and then all
  8273. 6:05:06the million emails through it and see if
  8274. 6:05:09where the buckets fill up.
  8275. 6:05:11And with that we can actually see which
  8276. 6:05:13buckets
  8277. 6:05:14are not filled properly, which buckets
  8278. 6:05:16are overfitted, which buckets are never
  8279. 6:05:18filled out. And then we ask the LLM to
  8280. 6:05:20go back and optimize those buckets
  8281. 6:05:22again. And then once the buckets are
  8282. 6:05:25rearranged, we'll do another
  8283. 6:05:27classification of all the all the
  8284. 6:05:28millions of emails one more time.
  8285. 6:05:31Because Jev is so cheap, we are able to
  8286. 6:05:33do this multiple times and eventually
  8287. 6:05:36we'll end up with the category list
  8288. 6:05:38which is extremely fit towards the
  8289. 6:05:39mailbox that you are dealing with
  8290. 6:05:41because it's not the same across
  8291. 6:05:43different in different mailboxes the
  8292. 6:05:44categories are completely different.
  8293. 6:05:46But the powerful thing is once you did
  8294. 6:05:48all of this work with an LLM going
  8295. 6:05:50through and finding these categories,
  8296. 6:05:52you don't need the LLM anymore
  8297. 6:05:54because you have the categories which
  8298. 6:05:56you where you have distilled down all
  8299. 6:05:58the intelligence into these categories.
  8300. 6:06:00Now you can just build a maestro flow
  8301. 6:06:03which basically takes just the
  8302. 6:06:05categories alone and uses gem. So
  8303. 6:06:08whenever a new email comes, it only
  8304. 6:06:10needs gem. It just looks at the looks at
  8305. 6:06:13the email and fits the categories and we
  8306. 6:06:15basically have a list of
  8307. 6:06:18follow through. So if this matches this
  8308. 6:06:19category, you are going to get an email.
  8309. 6:06:21This team is going to get an email or
  8310. 6:06:22that team is going to get an email so
  8311. 6:06:24easily and we basically removed the LLM
  8312. 6:06:26completely from the picture on this one.
  8313. 6:06:29So and we saw a humongous speed increase
  8314. 6:06:32on on just email processing. It was
  8315. 6:06:34almost like at real time an email comes
  8316. 6:06:36and suddenly somebody will get notified.
  8317. 6:06:38So
  8318. 6:06:39that was an amazing
  8319. 6:06:41amazing thing to see as well. So over
  8320. 6:06:43the few months we were so excited to
  8321. 6:06:46play around with gem and every day the
  8322. 6:06:48slack threads would light up everywhere
  8323. 6:06:50in the in the company that I found this
  8324. 6:06:52pattern, I found that pattern and it was
  8325. 6:06:54such an exciting time during these last
  8326. 6:06:57three months and I wanted to share that
  8327. 6:07:00there are so much more patterns and
  8328. 6:07:02strategies to be found.
  8329. 6:07:04We are only scratching the surface of
  8330. 6:07:06this and I want us to
  8331. 6:07:10try out and like share the strategies
  8332. 6:07:11that we all find and collaborate here.
  8333. 6:07:14>> And if you're interested in trying out
  8334. 6:07:15UI Path Flow, we have it available for
  8335. 6:07:18free right now. Scan the QR code on the
  8336. 6:07:19screen here and we'll be able to it'll
  8337. 6:07:21take you into a portal where you can
  8338. 6:07:22basically point your coding agent at it
  8339. 6:07:24and get complete access to build your
  8340. 6:07:25first automations there. Thank you for
  8341. 6:07:27your time everyone.
  8342. 6:07:28>> Thank you.
  8343. 6:07:30>> [applause]
  8344. 6:08:39>> This
  8345. 6:11:39>> [snorts]
  8346. 6:11:43>> All right. Good afternoon, everybody. I
  8347. 6:11:46am Brian Kennedy. I'm an engineer on
  8348. 6:11:49Zoom Info's platform.
  8349. 6:11:52Now,
  8350. 6:11:53I want to start with showing you
  8351. 6:11:55something one of our agents wrote.
  8352. 6:11:57This is a real output from our product.
  8353. 6:11:59The agent's job was to schedule an email
  8354. 6:12:03for a customer.
  8355. 6:12:04And this is what it reported back.
  8356. 6:12:07It said [clears throat] it scheduled the
  8357. 6:12:08email, it queued it up, and that it was
  8358. 6:12:10going to go out on Tuesday.
  8359. 6:12:12We don't just trust the agent, though. A
  8360. 6:12:14second model checks the agent's work.
  8361. 6:12:16I'm going to call this one the judge.
  8362. 6:12:18The judge read this and it accepted it.
  8363. 6:12:21Great.
  8364. 6:12:23Now, here's the problem.
  8365. 6:12:25And this is what it actually happened.
  8366. 6:12:27Their these are the tool results, the
  8367. 6:12:30record of what actually ran when the
  8368. 6:12:31agent called the tools. I'm going to
  8369. 6:12:33call these the receipts.
  8370. 6:12:35The agent tried to create a record in
  8371. 6:12:36Salesforce and it failed. The error
  8372. 6:12:38message that came back says the record
  8373. 6:12:40was not created and it tells the agent
  8374. 6:12:43not to report a success or even make up
  8375. 6:12:45a record.
  8376. 6:12:46We wrote this message ourselves for
  8377. 6:12:48exactly this situation.
  8378. 6:12:50The agent tried again. It created a
  8379. 6:12:52task, which is a to-do.
  8380. 6:12:55No email was ever scheduled. The agent
  8381. 6:12:58doesn't even have a tool that can send
  8382. 6:12:59an email. The agent made it up and the
  8383. 6:13:01judge approved it.
  8384. 6:13:03So that's two failures.
  8385. 6:13:05The agent fabricated a send
  8386. 6:13:07and the judge rubber stamped it,
  8387. 6:13:09approved a claim that never actually
  8388. 6:13:12checked.
  8389. 6:13:13So if you build agents, you probably
  8390. 6:13:15have run into this and you probably have
  8391. 6:13:16a judge like this. It might be a code
  8392. 6:13:18reviewer or a grader of your evals or a
  8393. 6:13:20supervisor in your loop.
  8394. 6:13:22Anytime a model decides whether another
  8395. 6:13:25model's work is good, this can happen.
  8396. 6:13:28About for the next 10 minutes, I'll show
  8397. 6:13:30you how we caught this and what we did
  8398. 6:13:31about it.
  8399. 6:13:33First, some quick background.
  8400. 6:13:35My team at ZoomInfo
  8401. 6:13:38is called Agent Teams. We announced this
  8402. 6:13:40last week. It's a multi-agent
  8403. 6:13:42orchestration layer that lives in
  8404. 6:13:44ZoomInfo's GTM Studio.
  8405. 6:13:46It coordinates a team of specialized
  8406. 6:13:48agents across the company's systems to
  8407. 6:13:50run GTM workflows. For example, find
  8408. 6:13:53companies, research them, write an
  8409. 6:13:55email, log it into their CRM. Each agent
  8410. 6:13:58has its own roles and instructions like
  8411. 6:14:00and tools like Salesforce, HubSpot,
  8412. 6:14:02Slack.
  8413. 6:14:03These agents make real uh decisions and
  8414. 6:14:06take real actions in the customer's
  8415. 6:14:08systems. They write records and they
  8416. 6:14:10send messages.
  8417. 6:14:12Every team and every agent finishes a
  8418. 6:14:15step. A second model checks its work.
  8419. 6:14:18That's the judge and it has three
  8420. 6:14:20options to move on, to redo the step, or
  8421. 6:14:24to stop the job altogether.
  8422. 6:14:26You probably heard this called LLM as a
  8423. 6:14:28judge. It's the same idea as a code
  8424. 6:14:30reviewer or an eval grader.
  8425. 6:14:33Now, the hard part for us is that it's
  8426. 6:14:36one model and it has to grade every kind
  8427. 6:14:38of agent that we're running through it.
  8428. 6:14:40Our customers build their own workflows
  8429. 6:14:42with any agent and any tool in any
  8430. 6:14:44order, so the permutations are almost
  8431. 6:14:46infinite.
  8432. 6:14:47So the list of jobs for the judge has to
  8433. 6:14:49grow with it.
  8434. 6:14:51The judge can tell whether the work what
  8435. 6:14:54looks finished, but what it can't tell
  8436. 6:14:56is whether the work was right because
  8437. 6:14:58that depends on what right means for
  8438. 6:15:00each job and we're asking it the same
  8439. 6:15:03question for all of them.
  8440. 6:15:06So here's where we're going.
  8441. 6:15:09First we'll talk about what we found
  8442. 6:15:10when we checked the judge.
  8443. 6:15:13Then what we chose instead
  8444. 6:15:15and what you can do today with your own
  8445. 6:15:17agents.
  8446. 6:15:21To start, we ran offline evals. We took
  8447. 6:15:24500 real steps from production,
  8448. 6:15:27each with a decision that the judge had
  8449. 6:15:28made.
  8450. 6:15:30Then we took a larger frontier model and
  8451. 6:15:31regraded every one of those to come back
  8452. 6:15:34with an accept or send back.
  8453. 6:15:36Then we hand checked each of those
  8454. 6:15:38samples and we checked the grades. Where
  8455. 6:15:40we disagreed with the model, we went
  8456. 6:15:42with our call.
  8457. 6:15:44This gave us a reference label for all
  8458. 6:15:45500 steps.
  8459. 6:15:49And it turned out the judge was right
  8460. 6:15:5165% of the time, so more or less a coin
  8461. 6:15:54flip.
  8462. 6:15:55It accepted 118 steps that it should
  8463. 6:15:58have sent back. That's more than one in
  8464. 6:16:00four of everything that it accepted.
  8465. 6:16:04Pretty bad.
  8466. 6:16:05And when we read through those, the most
  8467. 6:16:07common problem was that
  8468. 6:16:09similar to the email at the start, the
  8469. 6:16:11agent reported that it did something
  8470. 6:16:14like send a message and no tool call had
  8471. 6:16:17ever done that.
  8472. 6:16:20So to see why the judge had accepted it,
  8473. 6:16:22let's go back to the email example and
  8474. 6:16:24look at what the judge was given.
  8475. 6:16:26On the left, you have what the agent's
  8476. 6:16:28own summary Think of it as like a story.
  8477. 6:16:30It's confident, it's well written. It
  8478. 6:16:33says the email was scheduled.
  8479. 6:16:35Now on the right, you have the receipts.
  8480. 6:16:37This is the actual tool results. There
  8481. 6:16:39are errors, failed to create messages
  8482. 6:16:43telling the agent not to make up the
  8483. 6:16:44record.
  8484. 6:16:45Now the judge had both of these and it
  8485. 6:16:47went with the story. The work looked
  8486. 6:16:49fine on the surface, so it accepted it.
  8487. 6:16:54Now this isn't only our judge. You can
  8488. 6:16:56think of a code review agent that reads
  8489. 6:16:59the PR description before it looks at
  8490. 6:17:01the diff. It's a similar kind of
  8491. 6:17:02problem. If the description says I fixed
  8492. 6:17:05the bug, added tests, the agent is going
  8493. 6:17:07to lean on that when it reads the code.
  8494. 6:17:10So the obvious fix at the start was to
  8495. 6:17:12tune the judge and write a better
  8496. 6:17:14prompt. For us, that was going to be a
  8497. 6:17:16significant amount of work. Our agents
  8498. 6:17:18write sales emails, update CRMs, do
  8499. 6:17:20research,
  8500. 6:17:21and to judge each of these well, we'd
  8501. 6:17:23have to figure out what it was that the
  8502. 6:17:26user was asking. Give the judge the
  8503. 6:17:28right rules for every kind of work and
  8504. 6:17:31keep it all up to date.
  8505. 6:17:32That's basically a custom judge for
  8506. 6:17:34every use case.
  8507. 6:17:36What we went with, we went a different
  8508. 6:17:37direction. We stopped asking one big
  8509. 6:17:40general question, is this good? And we
  8510. 6:17:42started asking a short list of small yes
  8511. 6:17:45or no questions.
  8512. 6:17:47They're the same questions for every
  8513. 6:17:48step and each one is small enough to
  8514. 6:17:51answer by looking at the receipts.
  8515. 6:17:54The first option here is basically a
  8516. 6:17:56prompt tuning project that never ends,
  8517. 6:17:58whereas the second one is a small
  8518. 6:18:00handful of small checks.
  8519. 6:18:03Now, to answer those questions, we used
  8520. 6:18:05Jev. You've heard a lot about Jev today,
  8521. 6:18:07so I'll skip that and keep that short.
  8522. 6:18:10It's a type decision model, not a
  8523. 6:18:11generative model.
  8524. 6:18:13Uh for mom at home watching, just ask
  8525. 6:18:14Claude.
  8526. 6:18:16And we'll send it structured facts, you
  8527. 6:18:18know, about the step and a listed typed
  8528. 6:18:21questions.
  8529. 6:18:22A typed question is a yes or no question
  8530. 6:18:24or a choice with from a fixed list of
  8531. 6:18:27options. It sends back a probability for
  8532. 6:18:30each answer and it answers all the
  8533. 6:18:32questions in one call.
  8534. 6:18:36We gave it four things, the
  8535. 6:18:37instructions, the uh input, the
  8536. 6:18:40receipts, and meaning every tool call,
  8537. 6:18:42and whether it was successful or it
  8538. 6:18:44failed, and the agent's report.
  8539. 6:18:47Then we asked for one choice, which was
  8540. 6:18:49accept, redo, or stop.
  8541. 6:18:52And that choice makes the decision.
  8542. 6:18:55But then we also asked seven yes or no
  8543. 6:18:57questions. Those seven questions tell us
  8544. 6:19:00the why.
  8545. 6:19:01They decide between a redo and a stop,
  8546. 6:19:04and they go into the feedback that the
  8547. 6:19:06agent gets when it actually has to redo
  8548. 6:19:08a step.
  8549. 6:19:11Now, back to the email. The LLM judge
  8550. 6:19:14accepted it, and it took about two and a
  8551. 6:19:16half seconds.
  8552. 6:19:17Jev gave us a probability for each
  8553. 6:19:19option.
  8554. 6:19:21For this step, it accepted 0%, redo 97%
  8555. 6:19:26and stop at 3%.
  8556. 6:19:28And it took about a fifth of a second.
  8557. 6:19:31We make the decision from the
  8558. 6:19:32probability of accept. At 30% or higher,
  8559. 6:19:36we accept that. At 5% or lower, we redo
  8560. 6:19:39or stop.
  8561. 6:19:41And anything in between means Jev isn't
  8562. 6:19:43sure, so we kick it to an LLM judge.
  8563. 6:19:46We pick those cutoffs from testing the
  8564. 6:19:47500 labels that we had at the beginning.
  8565. 6:19:52Now, to test whether what whether or not
  8566. 6:19:54this works better, we took two different
  8567. 6:19:56routes. The first test used the same 500
  8568. 6:19:59production steps from the offline eval.
  8569. 6:20:02Jev was confident enough in about seven
  8570. 6:20:05out of 10 of them.
  8571. 6:20:06One of those steps,
  8572. 6:20:08uh, the LLM judge had accepted 82 that
  8573. 6:20:11should have been sent back. Each red
  8574. 6:20:13square here is one of those.
  8575. 6:20:15Jev successfully caught 74 of those.
  8576. 6:20:19The blue squares are the ones that Jev
  8577. 6:20:20would have sent back.
  8578. 6:20:25Now the second test is what we called
  8579. 6:20:27shadow mode.
  8580. 6:20:28Both judges look at every step, but only
  8581. 6:20:31the LLM judge, uh, that decision counts.
  8582. 6:20:34So we saved Jev what would have done and
  8583. 6:20:37we don't let it affect the actual run.
  8584. 6:20:39That gave us 37,000 decisions to mine
  8585. 6:20:42for insights.
  8586. 6:20:43And in that time, there were 728 steps
  8587. 6:20:47where a tool had clearly failed.
  8588. 6:20:50The LLM caught
  8589. 6:20:51294 of them. Jev caught 623.
  8590. 6:20:56And the fake email from the beginning,
  8591. 6:20:58it wasn't a one-off thing either. The
  8592. 6:21:00same step ran 1,081 times and almost
  8593. 6:21:03every time the agent said it had
  8594. 6:21:05scheduled an email, the LLM judge only
  8595. 6:21:07caught it once. Jev caught that 876
  8596. 6:21:10times.
  8597. 6:21:14And when you have a judge, every step
  8598. 6:21:16waits. Jev was 10 times faster and 40
  8599. 6:21:19times cheaper.
  8600. 6:21:20At this scale, it is massive.
  8601. 6:21:25[snorts]
  8602. 6:21:26So what's next?
  8603. 6:21:28Right now, every step goes with the same
  8604. 6:21:30seven questions. They go into Jev along
  8605. 6:21:32with the facts and Jev gives us an
  8606. 6:21:34accept, redo, or stop.
  8607. 6:21:37But next, we're adding post-condition
  8608. 6:21:39checks,
  8609. 6:21:40which are questions that depend on which
  8610. 6:21:41tool called, uh, was within the step.
  8611. 6:21:45So we don't have to guess. Uh, we know
  8612. 6:21:46the step up front because the tool tells
  8613. 6:21:48us.
  8614. 6:21:49If a step's called, let's say, a send
  8615. 6:21:51tool, we also ask whether the message
  8616. 6:21:53was delivered.
  8617. 6:21:54If it wrote a record, we ask whether the
  8618. 6:21:57record exists.
  8619. 6:21:58And those go into Jev with the same
  8620. 6:22:00seven questions in the same call.
  8621. 6:22:04Now, what can you today with your
  8622. 6:22:06agents?
  8623. 6:22:08First, pick one judgment. Somewhere in
  8624. 6:22:10your systems, there is an LLM deciding
  8625. 6:22:12whether or not something is good. It
  8626. 6:22:14might be reviewing code, grading an
  8627. 6:22:16eval. Pick one of those and rewrite it
  8628. 6:22:18as a few simple yes or no questions that
  8629. 6:22:20can be answered from the evidence.
  8630. 6:22:23Second is build it.
  8631. 6:22:24This is the easy part, honestly. We used
  8632. 6:22:26Jev and ours was seven questions.
  8633. 6:22:29Then the third, ship it in shadow mode.
  8634. 6:22:32Run it next to what you already have,
  8635. 6:22:33but don't let it make the decisions yet.
  8636. 6:22:35Just save it and then you can analyze it
  8637. 6:22:38later.
  8638. 6:22:39Then read the cases and see where the
  8639. 6:22:41two disagree. Don't overthink it, just
  8640. 6:22:44start small.
  8641. 6:22:47We had trusted our judge until we
  8642. 6:22:48measured it.
  8643. 6:22:50Measure yours. Run a second check. Run
  8644. 6:22:52it in shadow mode and see where it
  8645. 6:22:54disagrees. You may be surprised with
  8646. 6:22:56what you find.
  8647. 6:22:57Thank you.
  8648. 6:22:59>> [applause]
  8649. 6:24:48>> Hey everyone.
  8650. 6:24:50I promise I'm the last person on the
  8651. 6:24:52hype train for Jev.
  8652. 6:24:54I'm Scott.
  8653. 6:24:55I'm from I'm from Workday and I'm going
  8654. 6:24:58to go through how we're experimenting
  8655. 6:25:00with Jev to drive new innovations in HR
  8656. 6:25:05and finance tasks.
  8657. 6:25:07Exciting stuff. So
  8658. 6:25:10at Workday, we have over 11,000
  8659. 6:25:12different organizations that use our
  8660. 6:25:14tools.
  8661. 6:25:15And a lot of the problems that we deal
  8662. 6:25:17with break down into kind of two
  8663. 6:25:19different categories. So one is very
  8664. 6:25:21quick actions they have to take. I want
  8665. 6:25:23to see my pay. I want to ask for a day
  8666. 6:25:25off. The other ones are I have to dive
  8667. 6:25:28into a ton of data, weed through it, try
  8668. 6:25:30to figure out what's going on.
  8669. 6:25:33And like many of your companies as well,
  8670. 6:25:34we're trying to, you know, put agentic
  8671. 6:25:36workflows around this.
  8672. 6:25:38And so if you look at just a a very
  8673. 6:25:40simple agentic loop, you know, we have a
  8674. 6:25:43user request. We try to decide what to
  8675. 6:25:45do. We're running tools. We're judging
  8676. 6:25:47did we do a good job?
  8677. 6:25:49And then over and over again until we
  8678. 6:25:51finally get a result.
  8679. 6:25:53Now one of the problems here is that on
  8680. 6:25:55each one of those we're doing LLM calls.
  8681. 6:25:58And there's two major problems with
  8682. 6:26:00that.
  8683. 6:26:01So one is there's too much intelligence.
  8684. 6:26:03So if we look at the system one versus
  8685. 6:26:05system two thinking,
  8686. 6:26:08Uh we're doing a lot of slow,
  8687. 6:26:09deliberate, very expensive thought
  8688. 6:26:11processes at each one of those steps.
  8689. 6:26:16Another problem is is just the wrong
  8690. 6:26:18tool for the job.
  8691. 6:26:20So, you know, LLMs a lot of times are
  8692. 6:26:22just a hammer uh looking for a nail.
  8693. 6:26:25And, you know, we want to figure out
  8694. 6:26:27like what do we actually need uh because
  8695. 6:26:29clearly LLMs are not doing it.
  8696. 6:26:32So we want something that is fast,
  8697. 6:26:35uh smart, easy to use,
  8698. 6:26:39uh it's generalized, and it's very very
  8699. 6:26:41cheap.
  8700. 6:26:44So when we look at like from a code
  8701. 6:26:46perspective, you know, what are we
  8702. 6:26:47really looking at from a from a Jev-like
  8703. 6:26:50model? It's really just a conditional.
  8704. 6:26:52If this happens, this should happen.
  8705. 6:26:55Which one of these cases are the ones
  8706. 6:26:57that I want to explore?
  8707. 6:27:00And
  8708. 6:27:01you know, you've seen it among many of
  8709. 6:27:03these presentations already, uh but Jev
  8710. 6:27:05has the primitives to allow us to do
  8711. 6:27:07this. Uh there's the probability. Is it,
  8712. 6:27:09you know, this or that and what's the
  8713. 6:27:11probability? Uh what are the different
  8714. 6:27:13categories that this uh particular thing
  8715. 6:27:15could fall into?
  8716. 6:27:17And then you have the uh the scoring
  8717. 6:27:19mechanism, which is basically
  8718. 6:27:20probability and sort it.
  8719. 6:27:24So
  8720. 6:27:25you know, this has been brought up as
  8721. 6:27:27well. You know, my first thought when I
  8722. 6:27:28saw, you know, the Jev models was, well,
  8723. 6:27:31we already have classifiers. I've been
  8724. 6:27:33doing this for a long time.
  8725. 6:27:34How's it actually different?
  8726. 6:27:37And as many of the speakers have talked
  8727. 6:27:38about already, you know, with
  8728. 6:27:40classifiers,
  8729. 6:27:41I need to figure out, you know, what's
  8730. 6:27:43my golden data set? I have to train it.
  8731. 6:27:46New data comes out. I have to iterate
  8732. 6:27:49over it over.
  8733. 6:27:50With Jev, just out of the box, it just
  8734. 6:27:52worked. We plugged in our question. We
  8735. 6:27:55classified it. We didn't really have to
  8736. 6:27:57do anything. It generalized to our
  8737. 6:27:58question.
  8738. 6:28:00Uh and so we started using it for a lot
  8739. 6:28:02of the use cases that you've seen from
  8740. 6:28:04some of the presenters already. Uh so we
  8741. 6:28:06use it for routing. That's kind of the
  8742. 6:28:07canonical example.
  8743. 6:28:09Um you know, figuring out did this step
  8744. 6:28:11work, you know, as a judge.
  8745. 6:28:13Uh, we looked at uh,
  8746. 6:28:16you know, resumes, you know, is this
  8747. 6:28:18person qualified for ML work?
  8748. 6:28:22Uh, and I want to do kind of an overly
  8749. 6:28:24simplified example as well. Um, so no
  8750. 6:28:27one taught Jev like what a fish is. And
  8751. 6:28:29so this is actually a question we asked,
  8752. 6:28:31like is a fish a pet? Well, 50/50 might
  8753. 6:28:34be a pet sometimes, might not be. Um,
  8754. 6:28:37and so you know, if I ran into this,
  8755. 6:28:40then I have to figure out
  8756. 6:28:41um, what do I do next? Maybe I can give
  8757. 6:28:44it some more context. Oh, is a fish in a
  8758. 6:28:45bowl a pet? You know, and after 20
  8759. 6:28:48milliseconds I can ask this again. All
  8760. 6:28:50right,
  8761. 6:28:50that's great. Oh, is a fish on my plate
  8762. 6:28:53a pet? Clearly not. Uh, so this is
  8763. 6:28:56something that wasn't trained out of the
  8764. 6:28:57box in seconds. It just worked.
  8765. 6:29:02All right, so now on to some of the
  8766. 6:29:03demos that we're that um,
  8767. 6:29:05you know, we've done at Workday. So just
  8768. 6:29:09a caveat on some of these um, it's not
  8769. 6:29:11real data, so we're not using any of
  8770. 6:29:13your, you know, resumes or payroll data.
  8771. 6:29:16Uh, these are also not in production now
  8772. 6:29:18and may or may not be in the future. So
  8773. 6:29:20that's something our legal asked me to
  8774. 6:29:22say. So let's get into, you know, some
  8775. 6:29:24of the
  8776. 6:29:25the demos.
  8777. 6:29:26All right, so starting with
  8778. 6:29:28autocomplete. So autocomplete starts to
  8779. 6:29:31predict what you're trying to ask. It's
  8780. 6:29:33just predicting the question. It's not
  8781. 6:29:35helping you get to the answer. So as you
  8782. 6:29:38start to type more, it's kind of guess
  8783. 6:29:40like, oh, I want to figure out if this
  8784. 6:29:43person wants to take a day off work.
  8785. 6:29:44Great, you've guessed the question, but
  8786. 6:29:46you really haven't
  8787. 6:29:47you know, helped me get to the answer.
  8788. 6:29:50So what we want to do is move from just
  8789. 6:29:52predicting the word or the next block of
  8790. 6:29:54code and actually help you finish the
  8791. 6:29:56task.
  8792. 6:29:58And so this is best shown with a a demo
  8793. 6:30:00that we have where we go from predicting
  8794. 6:30:03what you want to say to predicting what
  8795. 6:30:06you want to do.
  8796. 6:30:07So hopefully the video plays. There we
  8797. 6:30:09go. All right, so I need to take time
  8798. 6:30:12off of off of uh, next week. And as I
  8799. 6:30:16start to ask this, Jev is actually
  8800. 6:30:18figuring out under the hood, okay, what
  8801. 6:30:21what more context do I have? How can I
  8802. 6:30:24adjust this UI based on the information
  8803. 6:30:27that I'm that I'm getting? This is not
  8804. 6:30:29something that we could have done with
  8805. 6:30:30an LLM. We're getting you know, we're
  8806. 6:30:33we're doing this you know, in a loop
  8807. 6:30:35while someone's asking.
  8808. 6:30:36Um the payslip. You know, you shouldn't
  8809. 6:30:39have to go in and ask an agent, hey, can
  8810. 6:30:41you go find out how much I was paid? Uh
  8811. 6:30:44what you can do is just in the search
  8812. 6:30:46say, what was my pay last period?
  8813. 6:30:50Pops up an interface, allows you to
  8814. 6:30:53unhide it,
  8815. 6:30:54you know, see what your pay was in the
  8816. 6:30:55last period.
  8817. 6:30:57Very simple. Gets you to the action.
  8818. 6:31:00Book a meeting.
  8819. 6:31:02So I know that I want to book a meeting
  8820. 6:31:05and uh I'm going to want to do it for
  8821. 6:31:08next Friday.
  8822. 6:31:10Let me do it at 2:00 p.m. Oh, I want to
  8823. 6:31:13do it with my boss. Make sure that I get
  8824. 6:31:15my you know, my boss on this. Uh Logan
  8825. 6:31:18Mc Neil. There we go. Meeting's booked.
  8826. 6:31:22And then a final one that we have just
  8827. 6:31:23for this example, but there's many more,
  8828. 6:31:25uh
  8829. 6:31:27is finding a person.
  8830. 6:31:29So in this one, I may have some semantic
  8831. 6:31:31knowledge, like find the person who owns
  8832. 6:31:33this part of the organization or just
  8833. 6:31:36like who's Ashanti? I want to contact
  8834. 6:31:38her. Uh pop it up, click on email,
  8835. 6:31:42good to go.
  8836. 6:31:46All right. So what's happening under the
  8837. 6:31:47hood?
  8838. 6:31:48So because Jev is fast and cheap,
  8839. 6:31:51uh just like a type ahead, we could
  8840. 6:31:54start to render a UI on this on that
  8841. 6:31:56same cadence. We can look at as the
  8842. 6:31:59words are coming, do a prediction of
  8843. 6:32:01intent and start to render an an
  8844. 6:32:03interface.
  8845. 6:32:05And you know, we could do this at you
  8846. 6:32:06know, about 20 bucks for a million
  8847. 6:32:08judgments.
  8848. 6:32:09Uh and the way that we're doing it is
  8849. 6:32:11kind of twofold. Um or two parts, I I
  8850. 6:32:14say.
  8851. 6:32:15So first, Jev is picking the card of
  8852. 6:32:18saying what's the intent? All right,
  8853. 6:32:19there's a card that's going to help you
  8854. 6:32:20out.
  8855. 6:32:21And then the team that's responsible in
  8856. 6:32:23that domain then can decide what are the
  8857. 6:32:25questions to progressively fill out that
  8858. 6:32:28card. And so there's sort of this
  8859. 6:32:29two-step process.
  8860. 6:32:31Uh the first step is getting people in
  8861. 6:32:33the front door, getting the right card
  8862. 6:32:35up. And then the second one is making
  8863. 6:32:37sure that you fill in that card to help
  8864. 6:32:39them accomplish the tasks that they're
  8865. 6:32:41after.
  8866. 6:32:42Now we have an eval on here before we
  8867. 6:32:45merge any card. We want to make sure
  8868. 6:32:46that it doesn't collide with any other
  8869. 6:32:48one. And at least in our initial
  8870. 6:32:50experiments, you know, we've we've done
  8871. 6:32:52seven cards or a just chat because it
  8872. 6:32:55should if there's no card to actually
  8873. 6:32:57solve it, it should allow you to just
  8874. 6:32:58continue chatting.
  8875. 6:32:59Uh we were able to get, you know, 96%
  8876. 6:33:01accuracy on that.
  8877. 6:33:05All right, so now onto the recruiter. So
  8878. 6:33:07let's talk about the recruiter problem.
  8879. 6:33:08All right, so this is not like a fast uh
  8880. 6:33:11a task. It's something where recruiter
  8881. 6:33:13has a pile of hundreds of resumes and
  8882. 6:33:16has to figure out who has the best
  8883. 6:33:18resume to accomplish the
  8884. 6:33:20uh or fill the wreck that they're trying
  8885. 6:33:22to fill.
  8886. 6:33:23Now I first want to look at like how we
  8887. 6:33:25would have done this before. Uh we would
  8888. 6:33:27have done, you know, an embedding, put
  8889. 6:33:30the vectors in a vector DB, then had a
  8890. 6:33:33retrieval layer, then have a re-ranker,
  8891. 6:33:35and then figure out based on the
  8892. 6:33:37re-ranker which ones that we want to put
  8893. 6:33:39in front of uh the recruiter.
  8894. 6:33:41Uh one interesting call out is the Quinn
  8895. 6:33:44re-ranker
  8896. 6:33:45uh looks surprisingly similar to uh
  8897. 6:33:48the uh the Jev process as well.
  8898. 6:33:51Uh and so that that's kind of an
  8899. 6:33:53interesting corollary if you've used
  8900. 6:33:54that one before.
  8901. 6:33:56Uh but with Jev, we basically take out
  8902. 6:33:59all the embedding steps.
  8903. 6:34:00I don't have to train anything up. I
  8904. 6:34:02don't have to do embedding. I don't have
  8905. 6:34:04to put things in a vector database.
  8906. 6:34:06Um
  8907. 6:34:07what I can do is just do a function on
  8908. 6:34:11my database. So put all the resumes
  8909. 6:34:14into a database,
  8910. 6:34:16uh use natural language to query it,
  8911. 6:34:18iterate over every single one of them
  8912. 6:34:21and then just figure out which ones
  8913. 6:34:23match my criteria.
  8914. 6:34:25So I'll go into
  8915. 6:34:27you know an actual demo of this exact
  8916. 6:34:29same one.
  8917. 6:34:30So here we have a select statement and
  8918. 6:34:32so this is against DuckDB where we have
  8919. 6:34:35a function
  8920. 6:34:37that is going to call Jev to figure out
  8921. 6:34:40what's the probability that someone
  8922. 6:34:41matches this criteria.
  8923. 6:34:43We're going to get it a score and then
  8924. 6:34:44we're going to order by that score. Now
  8925. 6:34:46I'm going to do about 20 and as I said
  8926. 6:34:48these are all fakes and none of your
  8927. 6:34:50resumes are in here I promise.
  8928. 6:34:52And then here we see these are very ML
  8929. 6:34:55focused synthetic data as well. So it's
  8930. 6:34:58not a surprise that we would have very
  8931. 6:35:01high hit rates against someone who's
  8932. 6:35:02worked in fraud, trust, and safety
  8933. 6:35:04problems.
  8934. 6:35:06But let's look at it. Let's change this
  8935. 6:35:07question. Let's be a little bit more
  8936. 6:35:09adversarial on this one.
  8937. 6:35:11So a lot of these resumes have the word
  8938. 6:35:13pipeline in them.
  8939. 6:35:15So whether it's a data pipeline or an AI
  8940. 6:35:18pipeline
  8941. 6:35:19pipe's going to come out. So let's just
  8942. 6:35:21ask like is this person qualified to do
  8943. 6:35:23plumbing? If you were going to just do a
  8944. 6:35:24lexical search it might pick up some of
  8945. 6:35:26these. So we put it into Jev and asked
  8946. 6:35:29you know of all these ML
  8947. 6:35:31resumes let's trim the limit down to
  8948. 6:35:34five here.
  8949. 6:35:36Go ahead and run it.
  8950. 6:35:38And we could see you know that
  8951. 6:35:43very very low probability on these. So
  8952. 6:35:45these are ones that we would almost
  8953. 6:35:46definitely throw out. And you could see
  8954. 6:35:48the speed. So we went over 117 records.
  8955. 6:35:51We had 2.3 seconds of wall time and
  8956. 6:35:55spent almost nothing on this query to
  8957. 6:35:56just iterate over everything.
  8958. 6:35:59So Jev really kind of changed the game
  8959. 6:36:01for us because every every place where
  8960. 6:36:03we're thinking either this is too
  8961. 6:36:05expensive or it's too slow we thought of
  8962. 6:36:07new places to inject Jev into our
  8963. 6:36:09pipeline.
  8964. 6:36:11And so [snorts] just to summarize you
  8965. 6:36:12know the two areas that I went through
  8966. 6:36:15are one
  8967. 6:36:16where can we decide very quickly and get
  8968. 6:36:19to the intent and get the person on
  8969. 6:36:21their way.
  8970. 6:36:22And then the second one is to cut down
  8971. 6:36:24on some of the, you know, really deep
  8972. 6:36:27work. How do we filter those out for,
  8973. 6:36:29you know, our customers?
  8974. 6:36:31And so I hope this that this gave you a
  8975. 6:36:33little bit of insight into the things
  8976. 6:36:35that you can take back to, you know,
  8977. 6:36:36your own companies and how in, you know,
  8978. 6:36:38imagine the things that you can do with
  8979. 6:36:40Jev and decision models as well.
  8980. 6:36:43And that's my time. Thank you very much.
  8981. 6:36:46>> [applause]
  8982. 6:37:54>> Hey folks, this is Girish,
  8983. 6:37:56a principal software engineer at
  8984. 6:37:58Microsoft.
  8985. 6:37:59Outside my job, I build open source
  8986. 6:38:01tools for AI agents.
  8987. 6:38:03One of them is mcb pacemaker,
  8988. 6:38:05a supervised bridge that keeps mcb
  8989. 6:38:08servers running.
  8990. 6:38:10Before we start, a small disclosure.
  8991. 6:38:12The views in this talk are my own and
  8992. 6:38:15doesn't represent my employer.
  8993. 6:38:17So sub agents for distributed system.
  8994. 6:38:21Over the past year, I have been using
  8995. 6:38:22agents and sub agents on real
  8996. 6:38:24engineering work.
  8997. 6:38:25Dev sessions, code reviews, instant
  8998. 6:38:28thrashing,
  8999. 6:38:29what not.
  9000. 6:38:31This talk is about what happens when one
  9001. 6:38:34agent hands work to another.
  9002. 6:38:38So here's an architecture.
  9003. 6:38:41The orchestrator splits the task and
  9004. 6:38:43hands the pieces
  9005. 6:38:44to three different sub agents.
  9006. 6:38:47They all run at the same time.
  9007. 6:38:50They all read from the same shared
  9008. 6:38:52state,
  9009. 6:38:53conversation,
  9010. 6:38:54memory files,
  9011. 6:38:55repo, working trees,
  9012. 6:38:58issues, vector stores.
  9013. 6:39:01And
  9014. 6:39:02they don't just think, they make tool
  9015. 6:39:04calls
  9016. 6:39:05to real systems
  9017. 6:39:08which touch databases, APIs, CI tools,
  9018. 6:39:11deployment pipelines.
  9019. 6:39:13And
  9020. 6:39:15the results come back and orchestrator
  9021. 6:39:18merges them into a single answer.
  9022. 6:39:20An orchestrator
  9023. 6:39:22workers
  9024. 6:39:23shared mutable state
  9025. 6:39:24and results merge from concurrent
  9026. 6:39:26executions.
  9027. 6:39:27This represents a classic distributed
  9028. 6:39:29system.
  9029. 6:39:31I have known how to reason with
  9030. 6:39:32distributed systems for decades,
  9031. 6:39:35but none of this setup
  9032. 6:39:37came with it.
  9033. 6:39:39What you get is a function that takes a
  9034. 6:39:41string and returns a string.
  9035. 6:39:44Failures we hit are not new, are not
  9036. 6:39:47specific to AI agents.
  9037. 6:39:49They are classic distributed system
  9038. 6:39:50failures.
  9039. 6:39:53Partial failure,
  9040. 6:39:55a normal call that fails halfway through
  9041. 6:39:58and returns an exception error code.
  9042. 6:40:01In a sub agent case, it returns text
  9043. 6:40:04that looks like a success.
  9044. 6:40:06Dirty reads,
  9045. 6:40:08normally we have isolations, versions.
  9046. 6:40:11Here
  9047. 6:40:12every worker reads on shared context
  9048. 6:40:14including things other worker only
  9049. 6:40:16guessed.
  9050. 6:40:18Unbounded fan out.
  9051. 6:40:21A busy service
  9052. 6:40:23pushes back with the queues
  9053. 6:40:25for two nines, five o threes.
  9054. 6:40:28In agents
  9055. 6:40:29work
  9056. 6:40:30does not see any of this. just starts in
  9057. 6:40:32the worker.
  9058. 6:40:34Retries.
  9059. 6:40:36Normal distributed systems we build at a
  9060. 6:40:38potency to make retries reliable.
  9061. 6:40:42In agents
  9062. 6:40:43a retries sub agent runs the whole task
  9063. 6:40:45again
  9064. 6:40:46with side effects included.
  9065. 6:40:49For each of this case
  9066. 6:40:51I'll name the concept it already is
  9067. 6:40:54and I'll put them together as a
  9068. 6:40:56consistency model I'm proposing
  9069. 6:40:58and walk through why each part is there.
  9070. 6:41:02Start with the failed model because
  9071. 6:41:03everything else depends on it.
  9072. 6:41:06A friendly case is a crash down.
  9073. 6:41:08The process it was running correctly
  9074. 6:41:10stops
  9075. 6:41:11and stays stopped.
  9076. 6:41:13The sub agent withdraws an error in this
  9077. 6:41:14box.
  9078. 6:41:15It's easy to handle.
  9079. 6:41:18Next, no reply.
  9080. 6:41:20The call is lost or it's very late.
  9081. 6:41:22We'll come back to this soon.
  9082. 6:41:25The hard case is the one from
  9083. 6:41:28Byzantine general sniper.
  9084. 6:41:30A component that gives conflicting
  9085. 6:41:32information.
  9086. 6:41:33The process answers and the answer is
  9087. 6:41:35wrong.
  9088. 6:41:37A half finished sub agent sits closest
  9089. 6:41:39to the box.
  9090. 6:41:40It is not malicious but from the outside
  9091. 6:41:42it looks the same.
  9092. 6:41:44It does not crash. It answers fluently.
  9093. 6:41:46Let's jump into this.
  9094. 6:41:48The sub agent
  9095. 6:41:50that half finished does not race.
  9096. 6:41:52It returns normal looking text
  9097. 6:41:55with the same type signature as success.
  9098. 6:41:58I talked about one version of this
  9099. 6:42:00in one of my talks.
  9100. 6:42:02We pointed an agent at an alert queue.
  9101. 6:42:05Here's an illustration I used there
  9102. 6:42:08with three different paths which give
  9103. 6:42:09the same answer.
  9104. 6:42:11The first one
  9105. 6:42:13it pulled the telemetry and compared
  9106. 6:42:15with a known benign set.
  9107. 6:42:17It actually did the work.
  9108. 6:42:19The second one
  9109. 6:42:20it matched against a title
  9110. 6:42:23it it has seen before
  9111. 6:42:24and gave the response.
  9112. 6:42:27The third alert
  9113. 6:42:29it didn't do anything.
  9114. 6:42:31It just
  9115. 6:42:32resonator.
  9116. 6:42:34All three are categorized as benign
  9117. 6:42:36with the same level of confidence.
  9118. 6:42:39And nothing downstream can tell these
  9119. 6:42:41apart.
  9120. 6:42:42The orchestrator can read the
  9121. 6:42:43conclusion. It cannot read how much work
  9122. 6:42:45was behind it.
  9123. 6:42:48Now
  9124. 6:42:49the call gets no reply.
  9125. 6:42:51The orchestrator send work and a timeout
  9126. 6:42:54fired.
  9127. 6:42:56From here
  9128. 6:42:57three worlds look the same.
  9129. 6:42:59The sub agent is still working. It
  9130. 6:43:01crashed.
  9131. 6:43:02And the reply was lost.
  9132. 6:43:04It's a hard limit on a tooling gap.
  9133. 6:43:07The FLP paper from 1985 showed that in
  9134. 6:43:09an asynchronous system
  9135. 6:43:11with even one faulty process
  9136. 6:43:13agreement can fail to finish.
  9137. 6:43:16Chandra and Tu X answered in 1996 was
  9138. 6:43:18failure detectors that can make
  9139. 6:43:20mistakes.
  9140. 6:43:21They suspect they do not know.
  9141. 6:43:24I hit both directions
  9142. 6:43:26in my own open source project
  9143. 6:43:29MCP pacemaker supervised bridge.
  9144. 6:43:33A server
  9145. 6:43:34whose first run downloads itself
  9146. 6:43:36hit the 30 second call timeout
  9147. 6:43:38and looked broken.
  9148. 6:43:40It was slow, not dead.
  9149. 6:43:42We fixed it
  9150. 6:43:43by giving initialize its own timeout.
  9151. 6:43:46Now the other way
  9152. 6:43:47the server crashed
  9153. 6:43:49during a call.
  9154. 6:43:50The call then timed out.
  9155. 6:43:52And the timeout overrode the exit code
  9156. 6:43:54that explained the crash.
  9157. 6:43:56We fixed it
  9158. 6:43:58by recording the timeout only when it
  9159. 6:44:00was alive.
  9160. 6:44:02So
  9161. 6:44:03timeout is a suspicion.
  9162. 6:44:05Record it as unknown
  9163. 6:44:07and keep the real cause.
  9164. 6:44:09And
  9165. 6:44:11now make failure a value.
  9166. 6:44:15Let's take this illustration.
  9167. 6:44:17A sub agent returns a status
  9168. 6:44:20what it did
  9169. 6:44:21what it did not do
  9170. 6:44:23the evidence and any side effects.
  9171. 6:44:26Partial is a normal answer here.
  9172. 6:44:29The protocols already have a slot for
  9173. 6:44:31this.
  9174. 6:44:32In A to A,
  9175. 6:44:33a task can end as completed, failed,
  9176. 6:44:36canceled, or rejected,
  9177. 6:44:39or stop and ask for input.
  9178. 6:44:41In MCP,
  9179. 6:44:42a tool result can say
  9180. 6:44:44is error true.
  9181. 6:44:46But a sub agent
  9182. 6:44:47can still say completed
  9183. 6:44:50while its text says something else.
  9184. 6:44:52And a loop that only reads the text now
  9185. 6:44:54receives a difference.
  9186. 6:44:56Another split.
  9187. 6:44:58Checking a condition and getting no
  9188. 6:45:01is a different fact from not being able
  9189. 6:45:03to check it all.
  9190. 6:45:06My own individual internet draft on
  9191. 6:45:08evaluation states
  9192. 6:45:11talks about this.
  9193. 6:45:13Collapse the two and you change what
  9194. 6:45:15gets retried
  9195. 6:45:17and who gets paged and what the audit
  9196. 6:45:19says later.
  9197. 6:45:21Next,
  9198. 6:45:22dirty reads.
  9199. 6:45:24Let's take this made up example from one
  9200. 6:45:26of my talk.
  9201. 6:45:29The sub agent
  9202. 6:45:30does try
  9203. 6:45:31it writes into a shared context
  9204. 6:45:34saying that error rate fell by 80%.
  9205. 6:45:37It did not measure that. It inferred.
  9206. 6:45:40Now,
  9207. 6:45:43two other sub agents read from the same
  9208. 6:45:45shared context.
  9209. 6:45:48The second sub agent
  9210. 6:45:49writes a review note and picks the same
  9211. 6:45:51sentence.
  9212. 6:45:53The third agent writes their own book
  9213. 6:45:56and uses it as a reason to proceed.
  9214. 6:45:59The orchestrator
  9215. 6:46:00merges all the all these three.
  9216. 6:46:03Three artifacts agree.
  9217. 6:46:05It is one unsupported sentence
  9218. 6:46:07read three different times.
  9219. 6:46:09The 1995 critique of sequel isolation
  9220. 6:46:12levels define a dirty read as
  9221. 6:46:14you read a value before the writer
  9222. 6:46:16commits or rolls back.
  9223. 6:46:18The shared context
  9224. 6:46:20it guess is readable the moment it is
  9225. 6:46:21written.
  9226. 6:46:22Nothing marks it as unchecked.
  9227. 6:46:25Regular fix
  9228. 6:46:28to is to ask three sub agents
  9229. 6:46:30and get a quorum.
  9230. 6:46:33Quorums do work for a specific reason.
  9231. 6:46:36In waited voting from 1979
  9232. 6:46:39the read quorum plus write quorum is
  9233. 6:46:41greater than total.
  9234. 6:46:43So every read overlaps the latest write.
  9235. 6:46:46The overlap is a guarantee.
  9236. 6:46:49In sub agents case
  9237. 6:46:50the three sub agents that read the same
  9238. 6:46:52shared context
  9239. 6:46:53does not give you that.
  9240. 6:46:55They are not independent copies.
  9241. 6:46:57They read the same guess.
  9242. 6:46:59Their agreement is one source counted
  9243. 6:47:01three different times.
  9244. 6:47:03Let's pick a consistency model.
  9245. 6:47:05In databases we have a
  9246. 6:47:09Linearizable.
  9247. 6:47:11Every operation appears to happen at one
  9248. 6:47:13point in real time order.
  9249. 6:47:15Sequential.
  9250. 6:47:17One total order that respects each
  9251. 6:47:19process own order.
  9252. 6:47:21Causal.
  9253. 6:47:23Operations that are causally related
  9254. 6:47:25appear in the same order everywhere.
  9255. 6:47:28Session guarantees.
  9256. 6:47:30They are promises to one client
  9257. 6:47:32like read your rights and monotonic
  9258. 6:47:34reads.
  9259. 6:47:36Eventual.
  9260. 6:47:37If no new updates are made reads
  9261. 6:47:39eventually return the last value.
  9262. 6:47:42A shared context gives you none of
  9263. 6:47:44these.
  9264. 6:47:45There's no stated guarantee at all.
  9265. 6:47:48And you don't need the top consistency
  9266. 6:47:50models.
  9267. 6:47:52The strongest models cannot stay
  9268. 6:47:54available when the network splits.
  9269. 6:47:57Causal and session guarantees can
  9270. 6:47:59as long as each clients stick to one
  9271. 6:48:01place.
  9272. 6:48:03For sub agents
  9273. 6:48:04that place is the orchestrator.
  9274. 6:48:06So
  9275. 6:48:08here's the consistency model I use for
  9276. 6:48:10sub agents.
  9277. 6:48:12I call it
  9278. 6:48:13safe handoffs.
  9279. 6:48:16Yes
  9280. 6:48:17stands for
  9281. 6:48:18source feeds.
  9282. 6:48:20Every fact the worker hands on carries
  9283. 6:48:23its source version as of time and
  9284. 6:48:26revision.
  9285. 6:48:27And decisions read only fresh source
  9286. 6:48:29facts.
  9287. 6:48:31A
  9288. 6:48:32at most once effects.
  9289. 6:48:35Every write has an identity and a single
  9290. 6:48:37use approval for the exact action.
  9291. 6:48:40An outcome
  9292. 6:48:41is read back now replayed.
  9293. 6:48:44If
  9294. 6:48:46fan out budgets
  9295. 6:48:48each downstream gets a gate and a
  9296. 6:48:50bounded weight.
  9297. 6:48:51Refusal counts as a fail.
  9298. 6:48:54E stands for explicit outcomes.
  9299. 6:48:57Every hand off ends in a named state
  9300. 6:49:00and a timeout is unknown
  9301. 6:49:02with its cost kept.
  9302. 6:49:04This consistency model
  9303. 6:49:07is the one I use for my own sub agents.
  9304. 6:49:08It's not a universal model.
  9305. 6:49:10It's based off my own public work.
  9306. 6:49:13Some of it is in my FCP pacemaker bridge
  9307. 6:49:16and some are based on my other talks and
  9308. 6:49:20my own internet draft.
  9309. 6:49:24Now let's see
  9310. 6:49:26what's behind each letter.
  9311. 6:49:29S source reads.
  9312. 6:49:32It's the answer to dirty reads
  9313. 6:49:34where one worker's guess becomes another
  9314. 6:49:36worker's fact.
  9315. 6:49:37What helps is to treat every fact as a
  9316. 6:49:40record, not a sentence.
  9317. 6:49:43Lamport's 1978 paper define when one
  9318. 6:49:46event happened before another.
  9319. 6:49:48The fact's provenance is that record,
  9320. 6:49:50what it was read from and when.
  9321. 6:49:53Let's take this
  9322. 6:49:55made up example.
  9323. 6:49:57The record names
  9324. 6:49:59the claim and its revision,
  9325. 6:50:01the source and its version,
  9326. 6:50:04the scope
  9327. 6:50:05and three different times
  9328. 6:50:08when the data was current,
  9329. 6:50:10when we read it and when we used it.
  9330. 6:50:14Freshness is checked with the fact is
  9331. 6:50:16used.
  9332. 6:50:18In this illustration
  9333. 6:50:19the data ends at
  9334. 6:50:229:05.
  9335. 6:50:23We read it at 9:06
  9336. 6:50:25and we decided at 9:07.
  9337. 6:50:28In this case
  9338. 6:50:30we picked a five minute limit.
  9339. 6:50:32So the data
  9340. 6:50:34decision at 9:07 it's fine.
  9341. 6:50:37Now at 9:15 the second station came in.
  9342. 6:50:42Because our freshness limit is five,
  9343. 6:50:44this is still recorded.
  9344. 6:50:47We reject it.
  9345. 6:50:49And we say that the next station needs
  9346. 6:50:51another read.
  9347. 6:50:53This is how you separate an observation
  9348. 6:50:55from an inference.
  9349. 6:50:57An observation has a source and a read
  9350. 6:50:59time.
  9351. 6:51:00An inference has none.
  9352. 6:51:02A at most effects is about retries.
  9353. 6:51:06This an illustration
  9354. 6:51:09from my article about MCP pacemaker
  9355. 6:51:12bridge.
  9356. 6:51:14The orchestrator delegates a task.
  9357. 6:51:18The sub agent calls a tool
  9358. 6:51:20which does a real write
  9359. 6:51:22like
  9360. 6:51:24creating a PR.
  9361. 6:51:26The reply is lost.
  9362. 6:51:29The orchestrator knows one thing, the
  9363. 6:51:31outcome is unknown.
  9364. 6:51:33So
  9365. 6:51:34it retries the whole task.
  9366. 6:51:36The write happens again.
  9367. 6:51:39This is a classic problem.
  9368. 6:51:41In 1984 RPC paper
  9369. 6:51:45it says that if the call returns
  9370. 6:51:48the procedure ran precisely once.
  9371. 6:51:50If you get an exception
  9372. 6:51:52it ran once or not at all.
  9373. 6:51:54And you are not told which.
  9374. 6:51:57So
  9375. 6:51:58you pick a delivery semantic.
  9376. 6:52:00At most once.
  9377. 6:52:03Don't retry and accept the fact that it
  9378. 6:52:05may be lost.
  9379. 6:52:07At least once
  9380. 6:52:08retry
  9381. 6:52:10and accept that duplicates are possible.
  9382. 6:52:13Exactly once.
  9383. 6:52:15It happened at least once and this is a
  9384. 6:52:18way we can deduplicate.
  9385. 6:52:21Maybe by using an item potency key.
  9386. 6:52:24In MCP
  9387. 6:52:25item potent hint is only a hint and it
  9388. 6:52:28defaults to false. If you need it, you
  9389. 6:52:30build it.
  9390. 6:52:31Here's the small design from my bridge
  9391. 6:52:34on how I handle it.
  9392. 6:52:36When a client comes back after a restart
  9393. 6:52:39the bridge replace the client's record
  9394. 6:52:41initialized against a fresh server,
  9395. 6:52:43that's safe. It's just a handshake,
  9396. 6:52:45and it does not restore tool state.
  9397. 6:52:49What it never replace is an uncertain
  9398. 6:52:51tool call.
  9399. 6:52:52When a shared server is recycled, the
  9400. 6:52:54bridge drains it
  9401. 6:52:56and calls still unanswered at the
  9402. 6:52:58deadline fail as potentially excluded.
  9403. 6:53:02That status is honest.
  9404. 6:53:04It tells the caller to go and check.
  9405. 6:53:08What helped us giving every side effect
  9406. 6:53:11an identity.
  9407. 6:53:13Approval
  9408. 6:53:14binds with exact action
  9409. 6:53:16and it's only used once.
  9410. 6:53:18After the call, read back by that
  9411. 6:53:20identity.
  9412. 6:53:22Submitted is not confirmed. Confirmed
  9413. 6:53:24means you observe the effect.
  9414. 6:53:27The last part of A
  9415. 6:53:29it shows up when work passes and
  9416. 6:53:31resumes.
  9417. 6:53:32A lease is a lock with a timeout.
  9418. 6:53:35Martin Kleppmann's post about
  9419. 6:53:37distributed locking shows this problem.
  9420. 6:53:40A client passes while holding it.
  9421. 6:53:43The lease expires.
  9422. 6:53:45Someone else takes over
  9423. 6:53:46and the first client wakes up and writes
  9424. 6:53:48anyway.
  9425. 6:53:50The fix he proposes
  9426. 6:53:52a fencing token.
  9427. 6:53:54A number that goes up every time the
  9428. 6:53:56lock is granted.
  9429. 6:53:58The storage remembers the highest token
  9430. 6:54:01it has seen
  9431. 6:54:02and rejects any write with a lower one.
  9432. 6:54:06Sub agents
  9433. 6:54:08pause and resume all the time.
  9434. 6:54:10In my other talk, as I put it
  9435. 6:54:13restore context, not side effects.
  9436. 6:54:16So treat an approval like a fencing
  9437. 6:54:18token.
  9438. 6:54:19One action, one use.
  9439. 6:54:21Checked at the gate.
  9440. 6:54:22Not remembered in the prompt or
  9441. 6:54:23conversation.
  9442. 6:54:26My bridge
  9443. 6:54:27fixed the same class of bug.
  9444. 6:54:30A stale exit or a deleted resume can no
  9445. 6:54:33longer remove or bring back a
  9446. 6:54:34replacement session.
  9447. 6:54:37If fanout budgets
  9448. 6:54:39is about unbounded fanout.
  9449. 6:54:42This is flow control or or lack of it.
  9450. 6:54:44I hit this issue in my own MCP bridge,
  9451. 6:54:47before it had a limit on cold starts.
  9452. 6:54:50Many sessions start the service
  9453. 6:54:53at the same time
  9454. 6:54:55through NPX.
  9455. 6:54:56They share one package cache.
  9456. 6:54:59Everyone of them failed with NPM's lock
  9457. 6:55:01compromise error.
  9458. 6:55:03Nothing in any single request was wrong.
  9459. 6:55:06The failure was the concurrency itself.
  9460. 6:55:09And when it downstream does push back,
  9461. 6:55:11it can arrive as plain text.
  9462. 6:55:14This is MCP specs one example of tool
  9463. 6:55:16execution error.
  9464. 6:55:17A rate limit
  9465. 6:55:19as one sentence.
  9466. 6:55:20The model reads the sentence.
  9467. 6:55:22The orchestrator code may never look.
  9468. 6:55:24The next worker starts an event.
  9469. 6:55:27So
  9470. 6:55:28I introduced a quota budget in my
  9471. 6:55:30bridge.
  9472. 6:55:32Cold starts that use a package manager
  9473. 6:55:35go through a gate
  9474. 6:55:37two at a time by default.
  9475. 6:55:39Everyone else waits.
  9476. 6:55:41And the wait is bounded at 15 seconds.
  9477. 6:55:43Past that
  9478. 6:55:45the start goes ahead without a gate.
  9479. 6:55:48And
  9480. 6:55:49I documented why
  9481. 6:55:50a corrupted cache is a risk
  9482. 6:55:52while a hung client request is
  9483. 6:55:54certainty.
  9484. 6:55:55That is a choice for availability made
  9485. 6:55:57on purpose and written down.
  9486. 6:56:00And I also built a cap on sessions.
  9487. 6:56:04At the cap
  9488. 6:56:05any a new session waits up to 10
  9489. 6:56:07seconds.
  9490. 6:56:08And then it's refused.
  9491. 6:56:10The refusal now counts as a failure in
  9492. 6:56:12health.
  9493. 6:56:13Another bug.
  9494. 6:56:15Capacity has to be reserved before the
  9495. 6:56:17synchronous start, not after.
  9496. 6:56:19Otherwise
  9497. 6:56:21queued starts overbook the cap.
  9498. 6:56:23That is a check then act race.
  9499. 6:56:26For sub agents
  9500. 6:56:27give the orchestrator a number per
  9501. 6:56:29downstream.
  9502. 6:56:30A bounded weight and a refusal it can
  9503. 6:56:32see.
  9504. 6:56:34Now
  9505. 6:56:35E next present outcomes.
  9506. 6:56:38Every hand of ends in a named state like
  9507. 6:56:41stale or unknown, not just text.
  9508. 6:56:44The last piece is who decided that
  9509. 6:56:46state.
  9510. 6:56:48Treat sub the agent output as untrusted
  9511. 6:56:49input,
  9512. 6:56:50the same way you treat a request body
  9513. 6:56:52from the internet.
  9514. 6:56:54The orchestrator owns the checks in
  9515. 6:56:57code.
  9516. 6:56:58Does the result match the schema?
  9517. 6:57:00Is the evidence there?
  9518. 6:57:01Is it fresh enough?
  9519. 6:57:03Do the numbers hold when you recompute
  9520. 6:57:05them?
  9521. 6:57:06If a check fails, the step returns a
  9522. 6:57:08refusal with a reason.
  9523. 6:57:11It does not pass the text along with a
  9524. 6:57:12warning paragraph underneath.
  9525. 6:57:15If the same model can skip a check and
  9526. 6:57:17declare it passed,
  9527. 6:57:18you have a request, not a boundary.
  9528. 6:57:21Here is a made up example from another
  9529. 6:57:23talk.
  9530. 6:57:25The sub agent says the error rate fell.
  9531. 6:57:28Before it was 10 errors
  9532. 6:57:30in 1,000 requests.
  9533. 6:57:31After
  9534. 6:57:32it was six errors in 600 requests.
  9535. 6:57:35It went from 10 to six and it claimed
  9536. 6:57:37that our rate fell.
  9537. 6:57:38But
  9538. 6:57:39when we computed the rate,
  9539. 6:57:41it's both 1%.
  9540. 6:57:44We recomputed it and then the claim is
  9541. 6:57:45refused.
  9542. 6:57:48A supported answer is fine.
  9543. 6:57:49A deliberate handoff to a person is
  9544. 6:57:51fine.
  9545. 6:57:52A confident guess, it's not.
  9546. 6:57:55So limitations
  9547. 6:57:56because a consistency model is only
  9548. 6:57:58useful when you know where we should
  9549. 6:58:00stop.
  9550. 6:58:02So here's what
  9551. 6:58:04safe model does not do.
  9552. 6:58:07It does not make facts true.
  9553. 6:58:09A source fact is checkable, not correct.
  9554. 6:58:13It does not give you exactly once
  9555. 6:58:15delivery.
  9556. 6:58:17A lost reply
  9557. 6:58:18stays unknown until you read it back.
  9558. 6:58:21It does not roll anything back.
  9559. 6:58:24Undoing a partial task is a compensation
  9560. 6:58:26step you design
  9561. 6:58:28like the Saga's paper from 1987.
  9562. 6:58:31And finally, there's no global order
  9563. 6:58:33across workers,
  9564. 6:58:35only causal order per fact.
  9565. 6:58:38And
  9566. 6:58:39it has a price.
  9567. 6:58:41CAP theorem says that when the network
  9568. 6:58:43splits,
  9569. 6:58:44you trade off availability against
  9570. 6:58:47consistency.
  9571. 6:58:49My Brits chose availability for cold
  9572. 6:58:51starts on purpose.
  9573. 6:58:53For rights,
  9574. 6:58:55safe picks
  9575. 6:58:56consistency
  9576. 6:58:57over availability.
  9577. 6:58:59It refuses rather than guesses.
  9578. 6:59:02Pack lack
  9579. 6:59:04adds that even without failures
  9580. 6:59:06you trade latency for consistency.
  9581. 6:59:09Every freshness check is another read
  9582. 6:59:12and and every refusal needs an owner.
  9583. 6:59:16So
  9584. 6:59:17where it does not fit
  9585. 6:59:20if a subway agent
  9586. 6:59:21only drafts texts that a person reads
  9587. 6:59:24before anything happens
  9588. 6:59:25most of this is is a overhead.
  9589. 6:59:28It also assumes that there's one single
  9590. 6:59:30orchestrator.
  9591. 6:59:32If you have more this does not fit.
  9592. 6:59:35Peer agents with no single place to
  9593. 6:59:38stick to it's something else.
  9594. 6:59:41This is the consistency model I use for
  9595. 6:59:43my own agents that touch actual real
  9596. 6:59:45production systems.
  9597. 6:59:47It's not universal.
  9598. 6:59:49So wrapping up
  9599. 6:59:51here's three key takeaways.
  9600. 6:59:53Treat every handoff to a sub agent as a
  9601. 6:59:55network call.
  9602. 6:59:57It can fail halfway
  9603. 6:59:59it can be slow and it can come back
  9604. 7:00:00twice.
  9605. 7:00:02Return a status not just text.
  9606. 7:00:05If the result cannot say partial, stale
  9607. 7:00:07or unknown
  9608. 7:00:08the orchestrator will read it as done.
  9609. 7:00:11And
  9610. 7:00:12start with our safe consistency model
  9611. 7:00:15and enforce it in code.
  9612. 7:00:16Keep what fits and change what does not.
  9613. 7:00:20We already know how to build distributed
  9614. 7:00:21systems.
  9615. 7:00:23We just have to notice that we are
  9616. 7:00:24building one.
  9617. 7:00:26So these are some of the sources
  9618. 7:00:28behind this talk.
  9619. 7:00:32On the left side it's my own work, my
  9620. 7:00:34open source projects, my
  9621. 7:00:36articles, internet drafts and my talks.
  9622. 7:00:40On right side
  9623. 7:00:42specs and standards
  9624. 7:00:45and
  9625. 7:00:46bunch of papers behind each concept.
  9626. 7:00:50Thank you.
  9627. 7:00:53>> Hello, my name is Nikolai
  9628. 7:00:54>> Petrov, and we will be talking about
  9629. 7:00:56skills and agents knowledge of those
  9630. 7:00:58skills.
  9631. 7:01:01We'll start it with the simple prompts
  9632. 7:01:03where we've been
  9633. 7:01:05playing with the new stuff, new work
  9634. 7:01:07being done by the agents, repeating
  9635. 7:01:09those prompts over and over again, and
  9636. 7:01:11then we start packing those into the
  9637. 7:01:13skills,
  9638. 7:01:14creating repeatable, reproducible,
  9639. 7:01:17portable type of their
  9640. 7:01:19functionality
  9641. 7:01:21or capabilities that we can
  9642. 7:01:24teach agents to work with.
  9643. 7:01:26And some of those skills are very
  9644. 7:01:27important, hundreds of the stars on
  9645. 7:01:29GitHub.
  9646. 7:01:33There are different types of their
  9647. 7:01:34skills. Some
  9648. 7:01:36teach models something new, bringing new
  9649. 7:01:38knowledge or capabilities to the model,
  9650. 7:01:41but there are also types of their just
  9651. 7:01:44steering or process skills where we
  9652. 7:01:48do not bring any additional information,
  9653. 7:01:50but just getting repeatable
  9654. 7:01:53flow of the actions that we want agents
  9655. 7:01:55to perform.
  9656. 7:01:57Both are useful. In this session, we
  9657. 7:02:00will be talking about new knowledge type
  9658. 7:02:01of the skills
  9659. 7:02:03and
  9660. 7:02:05we'll see what we can
  9661. 7:02:08get from the model's understanding of
  9662. 7:02:10those skills.
  9663. 7:02:13So we get our first skills packed, start
  9664. 7:02:17using them
  9665. 7:02:18basically
  9666. 7:02:21starting with their the feel of
  9667. 7:02:23does it work or not.
  9668. 7:02:25But there is also their more technical
  9669. 7:02:27way and more deterministic way for the
  9670. 7:02:29checking if skills are working or not
  9671. 7:02:31and what's the
  9672. 7:02:32impacts of those skills
  9673. 7:02:34our evolves.
  9674. 7:02:36There are different formats where
  9675. 7:02:37evolves can be defined,
  9676. 7:02:39but at the sense, this is task
  9677. 7:02:41definition that you want your agent to
  9678. 7:02:43be able to solve.
  9679. 7:02:46You put it in the place, ideally more
  9680. 7:02:48deterministic so we can validate it.
  9681. 7:02:51And we can check two arms. One their no
  9682. 7:02:55skill loaded, and their second one where
  9683. 7:02:58we're loading the skill and providing
  9684. 7:02:59this additional information to the
  9685. 7:03:01context.
  9686. 7:03:02And we compare how their arms There are
  9687. 7:03:05multiple things. One is how many
  9688. 7:03:08evolved spins off.
  9689. 7:03:10In this case, with no skill, we're
  9690. 7:03:12solving three out of five.
  9691. 7:03:14With skill, we're solving all five,
  9692. 7:03:16meaning that we get two more tasks.
  9693. 7:03:19There also would be implications on the
  9694. 7:03:21tokens usage or time that it took those
  9695. 7:03:24tasks to be completed and other factors.
  9696. 7:03:28But at the end, we are comparing those
  9697. 7:03:29and seeing what makes sense, what not.
  9698. 7:03:32And this is one of the main tools for
  9699. 7:03:35evaluating the skills and
  9700. 7:03:37saying that they not just nice ideas,
  9701. 7:03:40but actually working pieces of
  9702. 7:03:42instructions and context for the agents.
  9703. 7:03:46But what next? We get to the stage where
  9704. 7:03:49there we have their working skill. It
  9705. 7:03:52solved some evolutions.
  9706. 7:03:54But we know that
  9707. 7:03:56there should be a better way. We have
  9708. 7:03:58models evolving. We have
  9709. 7:04:01some
  9710. 7:04:03maybe cases there we want
  9711. 7:04:07to get skill more efficient, meaning
  9712. 7:04:09consuming less tokens. So what's their
  9713. 7:04:12their options there?
  9714. 7:04:14One is just change line by line and
  9715. 7:04:17try brute force
  9716. 7:04:19all the different combinations.
  9717. 7:04:21Another would be to think through what
  9718. 7:04:24makes sense or maybe use something
  9719. 7:04:28as the user feedback on the skills to
  9720. 7:04:30see where skills perform well there it's
  9721. 7:04:33not.
  9722. 7:04:34And with new models, just running skill
  9723. 7:04:38and seeing how it works, what's the
  9724. 7:04:41options there.
  9725. 7:04:42And
  9726. 7:04:43how large those modifications should be?
  9727. 7:04:45Just
  9728. 7:04:47lines, sentences, ideas.
  9729. 7:04:53So what
  9730. 7:04:54we found to be useful is instead of
  9731. 7:04:57relying on their luck, we can put more
  9732. 7:05:01deterministic process here
  9733. 7:05:03using idea of their items from the fact
  9734. 7:05:06score. Uh
  9735. 7:05:08we can split their skill and
  9736. 7:05:11instructions into those items
  9737. 7:05:14and
  9738. 7:05:16for each of those had
  9739. 7:05:18pair of question and answer that we
  9740. 7:05:20expect.
  9741. 7:05:22This could be either general ideas like
  9742. 7:05:25proposition in this case or contracts
  9743. 7:05:28there we put more
  9744. 7:05:30definitive expectation on their action.
  9745. 7:05:37And in general we will be getting their
  9746. 7:05:39uh
  9747. 7:05:43some predefined amount of the items per
  9748. 7:05:46skill and we will have let's say
  9749. 7:05:5025 or 30
  9750. 7:05:52items per skill.
  9751. 7:05:54Then for each of the items we will again
  9752. 7:05:57run
  9753. 7:05:58two arms, one without skill and one with
  9754. 7:06:01skill.
  9755. 7:06:03And we will have judge
  9756. 7:06:06or panel of judges that will be
  9757. 7:06:08evaluating model answers. So we asking
  9758. 7:06:10those questions to the model that we
  9759. 7:06:12interested in.
  9760. 7:06:14We evaluating those questions with their
  9761. 7:06:17judge, ideally different family of the
  9762. 7:06:20models or
  9763. 7:06:22panel of judges.
  9764. 7:06:23We also ensures that the question do not
  9765. 7:06:26leak answers.
  9766. 7:06:29We
  9767. 7:06:31depending on the case can compare with
  9768. 7:06:32no tools or additional um
  9769. 7:06:35information to check their internal
  9770. 7:06:37model understanding or provided with
  9771. 7:06:40their some tools like feedback access so
  9772. 7:06:42it would be more practical.
  9773. 7:06:45And we
  9774. 7:06:48can get their simple forward counts
  9775. 7:06:50matrix.
  9776. 7:06:54Basically where we are checking that the
  9777. 7:06:57model
  9778. 7:06:59uh uh
  9779. 7:07:00knows or do not know something
  9780. 7:07:02and how this maps to their
  9781. 7:07:04skill impact on this.
  9782. 7:07:06So the simplest
  9783. 7:07:08option is where model was able to answer
  9784. 7:07:11this particular
  9785. 7:07:13questions even without skill or with
  9786. 7:07:16skill. This means that skill have no
  9787. 7:07:18impact on this and this was already part
  9788. 7:07:20of the model's knowledge.
  9789. 7:07:22The second one is where without skill
  9790. 7:07:25actually model was able to answer this
  9791. 7:07:26correctly.
  9792. 7:07:28And by adding skill
  9793. 7:07:31we saw degradation in performance and
  9794. 7:07:33model capability to
  9795. 7:07:35talk or explain particular aspects.
  9796. 7:07:40The ideal case is where
  9797. 7:07:42without skill model wasn't able to
  9798. 7:07:44answer something and with skill we start
  9799. 7:07:46seeing that it can answer those new
  9800. 7:07:48type of the questions and scenarios and
  9801. 7:07:50this is what we were aiming for, teach
  9802. 7:07:53model something new.
  9803. 7:07:55But there also can be there fourth
  9804. 7:07:57option where
  9805. 7:07:59skills were written not
  9806. 7:08:01ideally or in some
  9807. 7:08:05maybe not clean way
  9808. 7:08:06where model wasn't able to answer it
  9809. 7:08:09either without skill or with skill and
  9810. 7:08:11while this topic might be relevant and
  9811. 7:08:13important for the model it still sort of
  9812. 7:08:16misses some of those concepts.
  9813. 7:08:21The simplest thing to fix are
  9814. 7:08:26instructions or items that actually
  9815. 7:08:28hurts performance and
  9816. 7:08:31for popular skills those are pretty
  9817. 7:08:33rare.
  9818. 7:08:34So in their
  9819. 7:08:37uh
  9820. 7:08:38box below you see that it's just one
  9821. 7:08:41case out of the mostly 400
  9822. 7:08:44items in entropic skills
  9823. 7:08:46and others don't have those.
  9824. 7:08:49But for their smaller skills that are
  9825. 7:08:52less popular you probably having more of
  9826. 7:08:54those. This is what we are seeing with
  9827. 7:08:56some of our um experiments.
  9828. 7:08:59And
  9829. 7:09:01this is just straightforward the
  9830. 7:09:03simplest to fix
  9831. 7:09:05to ensure that your skill do not reduce
  9832. 7:09:08model capabilities and performance and
  9833. 7:09:10bring only relevant
  9834. 7:09:12context there.
  9835. 7:09:15And we also can actually color this in
  9836. 7:09:17this scenario this is done by lines but
  9837. 7:09:20actually going with items you might have
  9838. 7:09:22there
  9839. 7:09:23uh different coloring for a single line
  9840. 7:09:25or a single sentence.
  9841. 7:09:28And this just gives your example of
  9842. 7:09:30there uh visualization where
  9843. 7:09:33this would be helpful
  9844. 7:09:35details and instructions in terms
  9845. 7:09:38what new stuff you bring to the model
  9846. 7:09:40and what
  9847. 7:09:42is
  9848. 7:09:45something that model already knows.
  9849. 7:09:48Cost wise
  9850. 7:09:49we are running all those questions and
  9851. 7:09:51asking model
  9852. 7:09:53but still relatively this is uh cheap
  9853. 7:09:55for 29 skills for 10 bucks of there
  9854. 7:09:59sonnet and haiku
  9855. 7:10:00calls with their sonnet used as a judge.
  9856. 7:10:04And
  9857. 7:10:09there
  9858. 7:10:10couple ways how we also can check their
  9859. 7:10:14strictness of their some knowledge.
  9860. 7:10:17Um we have
  9861. 7:10:19uh lenient and strict ways of the
  9862. 7:10:21checking this where first checks the
  9863. 7:10:23general idea
  9864. 7:10:25while the second one is checking for
  9865. 7:10:28exact details and you can think about
  9866. 7:10:31the difference between general knowledge
  9867. 7:10:33and understanding of the topic
  9868. 7:10:35to the understanding of the exact
  9869. 7:10:38keys parameters and exact values or
  9870. 7:10:41steps.
  9871. 7:10:43And what we see across the models is
  9872. 7:10:46the difference is large
  9873. 7:10:48especially with there
  9874. 7:10:51um
  9875. 7:10:51larger frontier models where we see
  9876. 7:10:53almost dropping from 70% understanding
  9877. 7:10:55of their skill in case of the general
  9878. 7:10:58understanding
  9879. 7:10:59to
  9880. 7:11:00just 22% of the specific
  9881. 7:11:03details.
  9882. 7:11:05And
  9883. 7:11:06this is much our expectations. We see
  9884. 7:11:08that new models are
  9885. 7:11:10pretty well generalized and understand
  9886. 7:11:11concepts. But it's still helpful to
  9887. 7:11:13provide their with exact details and
  9888. 7:11:16data that will be useful for their
  9889. 7:11:18solving those problems.
  9890. 7:11:23Here is outlook of their
  9891. 7:11:27comparing those three popular rappers
  9892. 7:11:29with the skills
  9893. 7:11:31across different models. So one of their
  9894. 7:11:34interesting observations here is that
  9895. 7:11:36smaller model have
  9896. 7:11:38in general
  9897. 7:11:39lower understanding of their
  9898. 7:11:43skill
  9899. 7:11:45with their larger sonnet having better
  9900. 7:11:47understanding which is
  9901. 7:11:50why we have bigger models.
  9902. 7:11:52But also we see that overall their
  9903. 7:11:54models already understand
  9904. 7:11:56a lot of their
  9905. 7:12:00details and context of those skills and
  9906. 7:12:03you would expect that with the newer
  9907. 7:12:04models and more advances in their
  9908. 7:12:09better training capabilities we will
  9909. 7:12:11have
  9910. 7:12:12more and more
  9911. 7:12:15in terms of the model's understanding.
  9912. 7:12:17But also we see that this pretty
  9913. 7:12:19significantly differs skill to skill.
  9914. 7:12:22And
  9915. 7:12:23it's not bad. Some skills are intended
  9916. 7:12:25to be directional.
  9917. 7:12:28But it's also useful piece of
  9918. 7:12:30information to see how those
  9919. 7:12:33skills bring new knowledge
  9920. 7:12:36to the different model and different
  9921. 7:12:37families and this should be run ideally
  9922. 7:12:40in the broader
  9923. 7:12:41set of the models.
  9924. 7:12:47But
  9925. 7:12:49just having their
  9926. 7:12:52this novelty piece of information is
  9927. 7:12:56just one of the signals. It's basically
  9928. 7:12:57initial vector for optimization.
  9929. 7:12:59At the end we still have to run evolves
  9930. 7:13:01and validate that
  9931. 7:13:03all the changes that we
  9932. 7:13:06try to make with their skill actually
  9933. 7:13:08works.
  9934. 7:13:09So their one of their simpler scenarios
  9935. 7:13:12would be we can look on their skill and
  9936. 7:13:16try to remove some or all of their
  9937. 7:13:18already
  9938. 7:13:20known pieces of information from the
  9939. 7:13:22skill and run evaluation.
  9940. 7:13:25It might come up with their same
  9941. 7:13:27scoring. Basically we will be getting
  9942. 7:13:29their same results model still capable
  9943. 7:13:31of solving those problems. Then great we
  9944. 7:13:34can reduce
  9945. 7:13:35skill size, save tokens, save context.
  9946. 7:13:39And instead of their
  9947. 7:13:41brute forcing and guessing what we can
  9948. 7:13:43or should remove
  9949. 7:13:45or add or how to fix their
  9950. 7:13:49degradations we can have a guidance.
  9951. 7:13:54So all of those measurements experiments
  9952. 7:13:57been done with the docume which is a
  9953. 7:13:59framework for their comparing
  9954. 7:14:02everything related to the agents and
  9955. 7:14:04context
  9956. 7:14:06where it's is skill, agents MD, MCP
  9957. 7:14:10tools, whatever. And novelty is one of
  9958. 7:14:13their pieces of the functionality where
  9959. 7:14:16you can get this initial vector for
  9960. 7:14:19their skill optimization and evaluation
  9961. 7:14:21but
  9962. 7:14:21there also
  9963. 7:14:23evolves there is assessment of the
  9964. 7:14:25context impact, routing, comparing their
  9965. 7:14:29descriptions, naming, bunch of the
  9966. 7:14:31things that
  9967. 7:14:32related to their
  9968. 7:14:35skills where it would be helpful to
  9969. 7:14:37actually get their more deterministic
  9970. 7:14:39evaluations to make
  9971. 7:14:41conscious decisions in terms of the what
  9972. 7:14:43should be done with the skills.
  9973. 7:14:47So thank you for your time and please
  9974. 7:14:50go ahead and write better skills.
  9975. 7:16:04>> Keynote programming will begin in 5
  9976. 7:16:07minutes.
  9977. 7:16:08Please be sure to find your seats.
  9978. 7:18:07>> This
  9979. 7:19:14>> <music>
  9980. 7:19:14>> <music>
  9981. 7:23:06>> As we head into our closing keynotes,
  9982. 7:23:08we're excited to bring back a familiar
  9983. 7:23:10face to guide us through the final
  9984. 7:23:12sessions. Ladies and gentlemen, please
  9985. 7:23:15welcome back our MC, Corey Quinn, Chief
  9986. 7:23:19Cloud Economist at Duckbill Group.
  9987. 7:23:39>> Welcome back. Thank you all for sticking
  9988. 7:23:42out the full day. We just have one more
  9989. 7:23:46section to go and then we're calling it
  9990. 7:23:48a wrap on technically day two of the
  9991. 7:23:50conference.
  9992. 7:23:51Since we arrived this morning, we've
  9993. 7:23:53heard about a few things, deleting
  9994. 7:23:55harnesses, presumably on purpose,
  9995. 7:23:58healing your agents, maybe through the
  9996. 7:24:00power of song, we don't know,
  9997. 7:24:02self-improving codebase, which is the
  9998. 7:24:04exact opposite of the direction of every
  9999. 7:24:05codebase I've ever seen in my life has
  10000. 7:24:07ever gone, but okay, and so much more
  10001. 7:24:10because
  10002. 7:24:11everything is different now.
  10003. 7:24:13And before we leave tonight, we're going
  10004. 7:24:14to hear about setting intelligence free.
  10005. 7:24:17We're going to hear about how agents
  10006. 7:24:18build cultures.
  10007. 7:24:20There's a talk called Escaping from
  10008. 7:24:22Trash Mountain, which ironically was my
  10009. 7:24:24nickname back in high school. And of
  10010. 7:24:26course, 2026 in LLMs, which is one of
  10011. 7:24:29those years that seems to encompass
  10012. 7:24:31decades. So here we are.
  10013. 7:24:34Before we dive into all of that, I want
  10014. 7:24:36to take just another moment to thank our
  10015. 7:24:38sponsors for making all this possible.
  10016. 7:24:41Presenting sponsor remains Google
  10017. 7:24:42DeepMind, who earlier today shipped
  10018. 7:24:46what was it? Gemini Carbon 4 to general
  10019. 7:24:48availability. Congratulations.
  10020. 7:24:51Please clap.
  10021. 7:24:54[applause]
  10022. 7:24:56For our lounge and our platinum sponsors
  10023. 7:24:58as well, without whom we would have
  10024. 7:25:00far fewer places to sit when we're not
  10025. 7:25:02in the middle of a session.
  10026. 7:25:04Our gold sponsors as well for
  10027. 7:25:07effectively making all the various
  10028. 7:25:09moving parts go.
  10029. 7:25:11And of course, our silver parts, our
  10030. 7:25:13silver sponsors too. My goodness, there
  10031. 7:25:16are so many tiers at this point. At some
  10032. 7:25:17point, it starts to become an API
  10033. 7:25:19ladder. And finally, all of our
  10034. 7:25:20supporting partners. Again, it it really
  10035. 7:25:22is not possible to put on an event like
  10036. 7:25:24this without a tremendous amount of
  10037. 7:25:26support from various folks, and we thank
  10038. 7:25:28them sincerely for all of this.
  10039. 7:25:30Uh,
  10040. 7:25:31now up we have Fabian Hedin, who is the
  10041. 7:25:33CTO of Lovable, trying to tell us what
  10042. 7:25:36it takes to set intelligence free, uh,
  10043. 7:25:40which might be the only time in AI we
  10044. 7:25:41hear the word free.
  10045. 7:25:44Let's make him feel welcome.
  10046. 7:25:47>> [applause]
  10047. 7:26:04>> All right.
  10048. 7:26:06Thank you so much. Um, see we get to go
  10049. 7:26:09here. Um, all right. Welcome everyone.
  10050. 7:26:12Glad to be here. I'm Fabian, co-founder
  10051. 7:26:14and CTO of of Lovable, and we spent the
  10052. 7:26:18last few years or so building a platform
  10053. 7:26:21to let anyone create software and then
  10054. 7:26:23operate that and deploy that software.
  10055. 7:26:26Um, and um, the the topic of today is
  10056. 7:26:30what it takes to set intelligence free.
  10057. 7:26:34And
  10058. 7:26:35really um,
  10059. 7:26:36for us this takes us back
  10060. 7:26:38to when we when we started three years
  10061. 7:26:40ago. And I have to say I have a bit of a
  10062. 7:26:44complicated relationship with with
  10063. 7:26:46agents, um, and I I'll tell you why. Um,
  10064. 7:26:50we almost failed as a company
  10065. 7:26:52because we thought
  10066. 7:26:54back in 2023 that hey, we these agents
  10067. 7:26:57are the future, so let's build for them
  10068. 7:26:59now.
  10069. 7:27:00Um, I think they we kind of turned out
  10070. 7:27:02to be true that they were the future,
  10071. 7:27:03but we were definitely wrong on the the
  10072. 7:27:05now piece in in 23. So this was back in
  10073. 7:27:08kind of GPT-4 days. Um,
  10074. 7:27:11and um,
  10075. 7:27:12at the at that time the intelligence was
  10076. 7:27:14simply not there to have more of these
  10077. 7:27:16kind of complex agentic systems. Um, but
  10078. 7:27:20now it is, right now. Uh, now the
  10079. 7:27:22intelligence is actually at a place
  10080. 7:27:24where where you can really build out
  10081. 7:27:26more complicated systems around them.
  10082. 7:27:28And so I'll I'll give you kind of a
  10083. 7:27:30quick
  10084. 7:27:32background story here of of our journey
  10085. 7:27:36in in working with
  10086. 7:27:38intelligence or or these LLMs. And so
  10087. 7:27:42back in 2023 we built the first version
  10088. 7:27:45of our product. We called it GPT
  10089. 7:27:46Engineer back then.
  10090. 7:27:48And really the UI was very similar to
  10091. 7:27:51what it is today where you kind of have
  10092. 7:27:52a chat and and this preview.
  10093. 7:27:54But the AI at that point was quite
  10094. 7:27:56simple. We just had a prompt and a
  10095. 7:27:59response, one LLM call per user message.
  10096. 7:28:02Very simple. It worked fine.
  10097. 7:28:04But you could kind of build one page
  10098. 7:28:06apps and and simpler applications with
  10099. 7:28:08this.
  10100. 7:28:10And then we thought okay, this
  10101. 7:28:13intelligence is going to keep on
  10102. 7:28:14evolving so we have to be ready for
  10103. 7:28:18for it as it evolves. And so we thought
  10104. 7:28:19let's go out and build this agentic
  10105. 7:28:21framework. And so we we went and we
  10106. 7:28:24built this very sophisticated, dare I
  10107. 7:28:26say beautiful framework for for agents
  10108. 7:28:28to work and
  10109. 7:28:30we had kind of this one main agent that
  10110. 7:28:33would talk to the user and that agent
  10111. 7:28:34would spawn other agents like an
  10112. 7:28:37implementation agent here in the example
  10113. 7:28:39and each agent could have a speciality.
  10114. 7:28:41This agent would kind of specialize in
  10115. 7:28:43the first generation of an app and then
  10116. 7:28:45you would have like a front-end agent
  10117. 7:28:46and a back-end agent and they would all
  10118. 7:28:48talk to each other, send messages back
  10119. 7:28:50and forth
  10120. 7:28:51and we thought oh this is beautiful and
  10121. 7:28:53there were papers around this and how
  10122. 7:28:55well this could work and in the end it
  10123. 7:28:57it didn't work at all. We just made the
  10124. 7:28:59product worse.
  10125. 7:29:00We
  10126. 7:29:02we definitely failed
  10127. 7:29:04on this one.
  10128. 7:29:06But luckily before we ran out of money
  10129. 7:29:08we figured out like hey let's go back to
  10130. 7:29:10a simpler system again. So in late 24 we
  10131. 7:29:13went back to a much simpler
  10132. 7:29:15system where it was a bit of a shame
  10133. 7:29:17where we had like some some smaller LLMs
  10134. 7:29:19doing some knowledge retrieval
  10135. 7:29:21and then really just one main LLM kind
  10136. 7:29:23of writing to the user and writing the
  10137. 7:29:25code in the same message. And this
  10138. 7:29:27worked great. It was very responsive for
  10139. 7:29:29for users in the UI and it was reliable
  10140. 7:29:32and users were kind of in it and and
  10141. 7:29:34iterating with
  10142. 7:29:35with the model.
  10143. 7:29:38And so
  10144. 7:29:39let's close that down.
  10145. 7:29:42But now
  10146. 7:29:43it's time again. We're back. We're so
  10147. 7:29:45back. Agents this time they work
  10148. 7:29:48and we're running them in production
  10149. 7:29:50over a billion tokens per minute on on
  10150. 7:29:52on these systems that I'm going to talk
  10151. 7:29:53about.
  10152. 7:29:55But
  10153. 7:29:56first just looking at okay what is
  10154. 7:29:59actually Lovable? So
  10155. 7:30:01if you simplify the UI and this was same
  10156. 7:30:04back in 2023 as it is today,
  10157. 7:30:06to the left we have a chat where humans
  10158. 7:30:09come in and they kind of describe their
  10159. 7:30:11their intent to to build or whatever
  10160. 7:30:12they want to do and to the right you
  10161. 7:30:14have a preview which is the output or or
  10162. 7:30:17the outcomes of of what you're building.
  10163. 7:30:20And this back in 23 you could do simpler
  10164. 7:30:24things with, right? But now we've really
  10165. 7:30:26seen
  10166. 7:30:28working at this layer of abstraction
  10167. 7:30:29you're able to build very complex
  10168. 7:30:31things, right? And I think we've even
  10169. 7:30:33seen
  10170. 7:30:34a lot of tooling meant for for
  10171. 7:30:36developers for example moving to this
  10172. 7:30:38type of interface where you're not
  10173. 7:30:39really looking at the code every day and
  10174. 7:30:41so on and you're getting kind of
  10175. 7:30:43abstracted and working on on be it
  10176. 7:30:45architecture
  10177. 7:30:47or looking at some kind of output of of
  10178. 7:30:48what you're doing. And
  10179. 7:30:51this is something that we've always
  10180. 7:30:53believed in and it's been kind of a core
  10181. 7:30:55thesis behind Lovable that you won't
  10182. 7:30:57have to care about the code in in the
  10183. 7:30:59same way that you had to do before. And
  10184. 7:31:02I think today we're kind of in this
  10185. 7:31:04almost a bit in between I think in the
  10186. 7:31:05industry where you know some people
  10187. 7:31:06might say oh you don't have to care
  10188. 7:31:08about code at all and some people might
  10189. 7:31:10have different opinions.
  10190. 7:31:12I of course like growing up coding and
  10191. 7:31:15these things you get a bit detached
  10192. 7:31:17but in the end what matters is is
  10193. 7:31:20actually not the code.
  10194. 7:31:22Sorry I thought we removed that last
  10195. 7:31:23part but what what matters is is the
  10196. 7:31:26outcomes, right?
  10197. 7:31:29And so
  10198. 7:31:30the outcomes what is that? Well, for for
  10199. 7:31:33our users, the outcomes is
  10200. 7:31:36creating a business and monetizing and
  10201. 7:31:39creating value for for others in in some
  10202. 7:31:41type of way. And I'll kind of quickly go
  10203. 7:31:44over
  10204. 7:31:46what we build as a platform to to enable
  10205. 7:31:48this before I go in kind of on the
  10206. 7:31:50on the technical side of this talk. So,
  10207. 7:31:52if you if you look at Lovable, the first
  10208. 7:31:55thing that that you can say about the
  10209. 7:31:57platform is that we have this agent and
  10210. 7:31:59like any other agent, it can do quite
  10211. 7:32:01arbitrary things and it's quite
  10212. 7:32:03specialized at at building software
  10213. 7:32:06and it has access to a browser so it can
  10214. 7:32:09verify what it's building and
  10215. 7:32:11it connects to to GitHub and you know it
  10216. 7:32:14writes code that and you can do whatever
  10217. 7:32:15you want for that code. So, so that's
  10218. 7:32:18kind of your basic coding coding agent.
  10219. 7:32:21And the the other pieces of the platform
  10220. 7:32:23is this kind of cloud and and business
  10221. 7:32:25piece of the platform. And so
  10222. 7:32:28the cloud part of the platform is is
  10223. 7:32:30about just giving kind of these
  10224. 7:32:32hosting primitives just seamlessly to to
  10225. 7:32:34this agent. And so we we built out
  10226. 7:32:36databases of authentication, storage,
  10227. 7:32:39all the types of backend compute and
  10228. 7:32:41security scanners and like backups and
  10229. 7:32:43logs for all of these things and all of
  10230. 7:32:44it can be introspected by by the agent.
  10231. 7:32:48And this just makes it seamless to to
  10232. 7:32:49build things. You don't have to kind of
  10233. 7:32:52go to different services to set this
  10234. 7:32:53together. We just kind of provide you
  10235. 7:32:55these primitives.
  10236. 7:32:57So, that's still around building
  10237. 7:32:58software and hosting this, right?
  10238. 7:33:00But then the last piece is around these
  10239. 7:33:03business capabilities where
  10240. 7:33:05again the output of what our users are
  10241. 7:33:07doing on Lovable, we don't see that
  10242. 7:33:08output as code. The output is is a
  10243. 7:33:11product or or a business.
  10244. 7:33:14And if you want to succeed as a
  10245. 7:33:16business, you need to do more things
  10246. 7:33:18than just write code, right? And so we
  10247. 7:33:20we we we help our users with receiving
  10248. 7:33:23payments, sending emails, optimizing
  10249. 7:33:25SEO,
  10250. 7:33:26doing ads on on Google and Meta,
  10251. 7:33:29and kind of analyzing what is happening
  10252. 7:33:31with the product through through for and
  10253. 7:33:34you can look at kind conversions and and
  10254. 7:33:36optimize these sort of things and the
  10255. 7:33:37agent is in the loop in all of this,
  10256. 7:33:40right? So
  10257. 7:33:41that's kind of our
  10258. 7:33:43capability offering you could say.
  10259. 7:33:45And again the core part of this is an
  10260. 7:33:47agent that all of these capabilities are
  10261. 7:33:49built for.
  10262. 7:33:51And
  10263. 7:33:52at some point in Lovel's history we used
  10264. 7:33:54to limit our agents to work for only 30
  10265. 7:33:56minutes because we thought they weren't
  10266. 7:33:58reliable to to work for a longer amount
  10267. 7:34:00of time.
  10268. 7:34:02But now we've changed this to really be
  10269. 7:34:05without any any sort of bound in terms
  10270. 7:34:08of the length an agent can work and so
  10271. 7:34:09you can imagine agents working for for
  10272. 7:34:1110 hours plus.
  10273. 7:34:13And so I'll I'll go into some of the
  10274. 7:34:15things that we've done
  10275. 7:34:17in our architecture to to enable this.
  10276. 7:34:19And so the first piece is really around
  10277. 7:34:22the agent runtime itself and how how we
  10278. 7:34:24enable that to to run for a longer
  10279. 7:34:26amount of time. And the second piece is
  10280. 7:34:29the computer where each of these agents
  10281. 7:34:31are actually doing their work, running a
  10282. 7:34:33dev server
  10283. 7:34:35and actually verifying what it's doing.
  10284. 7:34:38And so so the last piece then it is the
  10285. 7:34:39verification itself where if you want an
  10286. 7:34:42agent to to work for a long amount of
  10287. 7:34:43time you need to make sure that you have
  10288. 7:34:45these feedback loops in place so that
  10289. 7:34:47it's not kind of going off
  10290. 7:34:49in in in the wrong direction.
  10291. 7:34:52And so if we start by looking at the
  10292. 7:34:55agent runtime itself,
  10293. 7:34:57you can imagine that
  10294. 7:34:59you have really three different sources
  10295. 7:35:02that can
  10296. 7:35:04interact with an agent. So it can be
  10297. 7:35:06yourself kind of prompting this or a
  10298. 7:35:08human. It could be other agents and it
  10299. 7:35:10could also be just a system on on a
  10300. 7:35:12schedule waking up. And the way we chose
  10301. 7:35:15to architect this is that for for each
  10302. 7:35:17agent we have a separate inbox
  10303. 7:35:20where any of these sources would kind of
  10304. 7:35:22send in messages. And so you might have
  10305. 7:35:25a message here like what's the status.
  10306. 7:35:27And you can imagine you have this agent
  10307. 7:35:29and this agent is kind of an always on
  10308. 7:35:30agent that they might be working, right?
  10309. 7:35:32And so
  10310. 7:35:33you can imagine it has done like three
  10311. 7:35:35actions here, a request a tool call and
  10312. 7:35:37it got some some result. And then at
  10313. 7:35:39this point, the agent is ready to
  10314. 7:35:40actually receive messages. So
  10315. 7:35:45since anyone can drop a message into
  10316. 7:35:46this inbox, the agent then can copy that
  10317. 7:35:48itself into its own what we call a
  10318. 7:35:51trajectory. Basically, it's a task
  10319. 7:35:52history.
  10320. 7:35:54And this is the type of interaction that
  10321. 7:35:56I think you're familiar with if if
  10322. 7:35:57you're using agents overall.
  10323. 7:36:00So that's kind of the the first design
  10324. 7:36:02principle behind this agent runtime.
  10325. 7:36:05I'll go through a few more here. And so
  10326. 7:36:08the the trajectory in which the the
  10327. 7:36:11agent is working in, which is kind of
  10328. 7:36:13the history of what has happened so far
  10329. 7:36:15and really the basis for any sort of of
  10330. 7:36:18memory for for the agent, right?
  10331. 7:36:21And we designed this trajectory with
  10332. 7:36:24really Git as inspiration. And so
  10333. 7:36:27the trajectory, what is it? It's an
  10334. 7:36:29append-only log where each event
  10335. 7:36:33points towards a parent, the the
  10336. 7:36:35previous event.
  10337. 7:36:37And because of this, you get a nice
  10338. 7:36:41capability just like in Git where you
  10339. 7:36:43can fork things, right? So you can have
  10340. 7:36:45two events pointing to the same parent,
  10341. 7:36:46which essentially creates a fork.
  10342. 7:36:49And because of that, you can kind of
  10343. 7:36:50spin up parallel agents at
  10344. 7:36:54at any point of time in the history
  10345. 7:36:56without having to duplicate all of the
  10346. 7:36:58kind of the shared history.
  10347. 7:37:00And semantically, it's a bit unclear
  10348. 7:37:02what would happen if you merge this. So
  10349. 7:37:04unlike Git, we cannot merge agent
  10350. 7:37:06trajectories, but we can kind of
  10351. 7:37:08spawn them out, right? And these agents
  10352. 7:37:10can can talk to each other.
  10353. 7:37:12The other thing that we've made sure to
  10354. 7:37:14design for is that this agent
  10355. 7:37:17trajectory, it includes kind of the full
  10356. 7:37:19history and everything that happens with
  10357. 7:37:22the agent. And so every tool call,
  10358. 7:37:25a bunch of other things happening on the
  10359. 7:37:26platform, if a tool call like failed or
  10360. 7:37:28succeeded,
  10361. 7:37:29a bunch of what we've done is make sure
  10362. 7:37:31that we have this append-only log. And
  10363. 7:37:34and from that, we can actually build a
  10364. 7:37:35prompt that we would feed into into the
  10365. 7:37:38actual LLM. And you might ask why why
  10366. 7:37:41would you want to do this? Well,
  10367. 7:37:43one one reason is that when you build
  10368. 7:37:45the prompt, you if you have a long
  10369. 7:37:47history, you probably want to do some
  10370. 7:37:48sort of compaction and you want to
  10371. 7:37:50summarize part of that history
  10372. 7:37:52in the in the earlier steps, for
  10373. 7:37:53example.
  10374. 7:37:55But this summarization would differ.
  10375. 7:37:57For example, if you use different models
  10376. 7:37:59with different context lengths, you
  10377. 7:38:00would probably think differently about
  10378. 7:38:02when you should summarize and not. And
  10379. 7:38:05so by separating the the prompt from the
  10380. 7:38:07actual task history, we're able to
  10381. 7:38:09render prompts for for any type of model
  10382. 7:38:11or or any type of task
  10383. 7:38:13that that we can kind of find the best
  10384. 7:38:16and optimal prompt for
  10385. 7:38:19So
  10386. 7:38:20to show this event history in in
  10387. 7:38:22progress, we have this debug tool here.
  10388. 7:38:24So here it's just a recording of me
  10389. 7:38:25dragging this slider in our UI and you
  10390. 7:38:28can see on the right that
  10391. 7:38:31the UI is kind of updating as I'm as I'm
  10392. 7:38:32dragging this along and you have 156
  10393. 7:38:35events here and one each of those is an
  10394. 7:38:37event in this event history. And you can
  10395. 7:38:39see how how the UI can render kind of
  10396. 7:38:42any point in history. And this is just
  10397. 7:38:43showcasing that
  10398. 7:38:45at any point in time, we have full
  10399. 7:38:47control of
  10400. 7:38:49of say the UI or or anything else
  10401. 7:38:51happening on the this platform thanks to
  10402. 7:38:53this history kind of detailing
  10403. 7:38:55everything that has happened.
  10404. 7:38:58The
  10405. 7:38:59the last piece I'll go over here on the
  10406. 7:39:01on the on the runtime itself is that the
  10407. 7:39:04process that is running these agents,
  10408. 7:39:06they're separate from from the actual
  10409. 7:39:08task history. And so the process is
  10410. 7:39:11almost entirely stateless in that
  10411. 7:39:13between each each call, each tool call,
  10412. 7:39:15each turn,
  10413. 7:39:16it will actually just persist into the
  10414. 7:39:18task history. And so you can have a
  10415. 7:39:21process come and go.
  10416. 7:39:23And and then the question is why why do
  10417. 7:39:25we care about this?
  10418. 7:39:26Well,
  10419. 7:39:28at Level Bull, this system
  10420. 7:39:30we deploy to it 700 times per day. So
  10421. 7:39:33that's about once every two minutes. And
  10422. 7:39:36you can imagine if you have
  10423. 7:39:38a agent working for for 10 hours, um you
  10424. 7:39:42want to not that agent when it's
  10425. 7:39:44working, right? This this agent is doing
  10426. 7:39:46something for for a customer. Um but at
  10427. 7:39:48the same time, we want to make sure that
  10428. 7:39:49this agent is running on the latest
  10429. 7:39:51deploy. And it might be a new feature or
  10430. 7:39:53something like that, but it might also
  10431. 7:39:55be like a security vulnerability in in
  10432. 7:39:58the Linux kernel. And it's like, okay,
  10433. 7:39:59we need to update this ASAP. Um and and
  10434. 7:40:02so to do that, thanks to this separation
  10435. 7:40:04between the actual history and and the
  10436. 7:40:06processes, we can actually make sure
  10437. 7:40:09that we have one one agent that is
  10438. 7:40:11working on one task, but we can have
  10439. 7:40:13different processes taking up and
  10440. 7:40:15actually doing the compute behind that
  10441. 7:40:17work.
  10442. 7:40:18Um so so this task can keep on working
  10443. 7:40:20while we have um different deploys um um
  10444. 7:40:24working on it,
  10445. 7:40:25be it for 10 hours or or even longer,
  10446. 7:40:27right?
  10447. 7:40:28Um so that's the agent runtime. The
  10448. 7:40:31second piece of um of this platform,
  10449. 7:40:34right, is the is the computers. This is
  10450. 7:40:36where the work is actually happening. Um
  10451. 7:40:38and we've been through a lot of
  10452. 7:40:40iterations and a lot of vendors kind of
  10453. 7:40:42working with um um providing these
  10454. 7:40:45sandboxes for for the computer for for
  10455. 7:40:47the agent to work in. And we've landed
  10456. 7:40:49in in the system that we've worked
  10457. 7:40:51really hard now to develop, which is um
  10458. 7:40:54um a checkpointing um a computer. And so
  10459. 7:40:59you can imagine you come to you start a
  10460. 7:41:00new project. We have this common what we
  10461. 7:41:02call a golden checkpoint, which contains
  10462. 7:41:06the OS and the system services. It has
  10463. 7:41:08common tools installed and a dev server
  10464. 7:41:10that is ready to start. And this
  10465. 7:41:12checkpoint it includes everything on
  10466. 7:41:14disk, memory, and also running
  10467. 7:41:16processes.
  10468. 7:41:17And and so when you start a new project,
  10469. 7:41:19you get all of that for free. And all of
  10470. 7:41:22that is copied over in about 230
  10471. 7:41:23milliseconds.
  10472. 7:41:25Um and so
  10473. 7:41:27once you start a project, you you might
  10474. 7:41:28have some specific code and and some
  10475. 7:41:30dependencies within that project.
  10476. 7:41:32Um and um that is also checkpointed as
  10477. 7:41:36you go. And and so each time the the
  10478. 7:41:39agent is running and it finishes a turn,
  10479. 7:41:43we'll actually save a checkpoint of of
  10480. 7:41:44that
  10481. 7:41:45And and from that we're able to again
  10482. 7:41:47like very quickly, sub-second,
  10483. 7:41:50spin that computer up whenever you need
  10484. 7:41:52it. That means we can save on on
  10485. 7:41:53resources by kind of spinning this down
  10486. 7:41:55if you leave for say
  10487. 7:41:5710 minutes or or so and come back. We
  10488. 7:41:59can just spin it up extremely quickly so
  10489. 7:42:01you don't even notice this. And even
  10490. 7:42:03kind of requests
  10491. 7:42:05in transit will will not even be dropped
  10492. 7:42:07in in this setup.
  10493. 7:42:10The other thing we can do thanks to this
  10494. 7:42:12checkpointing is that just like the
  10495. 7:42:15trajectory itself in the agent runtime,
  10496. 7:42:17we can also fork
  10497. 7:42:19the computer. And when we do this
  10498. 7:42:23we have for example we have a feature in
  10499. 7:42:26in Level that is called drafts where you
  10500. 7:42:27kind of work in different environments
  10501. 7:42:29of what you're doing and and to support
  10502. 7:42:30that we need to fork the computer that
  10503. 7:42:32you're working in.
  10504. 7:42:34And
  10505. 7:42:35here we have
  10506. 7:42:38kind of
  10507. 7:42:39the example of doing two at the same
  10508. 7:42:40time and you can imagine doing hundreds
  10509. 7:42:43of these, right?
  10510. 7:42:44And so
  10511. 7:42:46we've we've had to do
  10512. 7:42:48some optimizations to make this feasible
  10513. 7:42:50to to actually run. So one of these
  10514. 7:42:52optimizations is copy-on-write on the
  10515. 7:42:54RAM itself. And so creating one of these
  10516. 7:42:57forks of a computer is is basically free
  10517. 7:42:59in terms of of the RAM that it's using.
  10518. 7:43:02But
  10519. 7:43:03as soon as you start writing
  10520. 7:43:05in in a process to to RAM we will kind
  10521. 7:43:07of copy that page over and only then
  10522. 7:43:10will you start having to allocate new
  10523. 7:43:12memory.
  10524. 7:43:15And there's another optimization here
  10525. 7:43:16also which is that each of these
  10526. 7:43:19sandboxes they require a few hundreds of
  10527. 7:43:21megabytes of memory.
  10528. 7:43:23But and that would take a few seconds to
  10529. 7:43:25stream in if you have it in like cold
  10530. 7:43:27object storage and then it has to go to
  10531. 7:43:28disk and then then into memory of this
  10532. 7:43:31this VM. But
  10533. 7:43:34thanks to
  10534. 7:43:36being able to stream the memory we can
  10535. 7:43:37actually boot up the the machine before
  10536. 7:43:41all of the memory is is actually
  10537. 7:43:44ready on it. So
  10538. 7:43:46the the memory the OS uses comes first
  10539. 7:43:48and then like some processes and so on
  10540. 7:43:49will start to work as soon as the memory
  10541. 7:43:51gets here. And this also shapes off some
  10542. 7:43:54some extra startup time. And then we
  10543. 7:43:56started asking ourselves, how can we
  10544. 7:43:58make this even more efficient and even
  10545. 7:43:59more faster for our users? And so you
  10546. 7:44:01might be familiar with Vite. This is
  10547. 7:44:03great dev server that most of the meta
  10548. 7:44:05frameworks out there in the JavaScript
  10549. 7:44:07ecosystem is using.
  10550. 7:44:09And so we took that and we forked it and
  10551. 7:44:11we rewrote it in Rust. And that made our
  10552. 7:44:14cold starts two times faster and we went
  10553. 7:44:16from getting this preview from 14
  10554. 7:44:18seconds to three seconds. And the main
  10555. 7:44:21kind of thing that we made faster here
  10556. 7:44:22is that Vite uses Node.js and we're able
  10557. 7:44:25to like drop that dependency of Vite
  10558. 7:44:27itself and just have this Rust binary.
  10559. 7:44:29So
  10560. 7:44:30that made the memory footprint be 6.5
  10561. 7:44:32times less
  10562. 7:44:34in these machines.
  10563. 7:44:35And by taking control of the
  10564. 7:44:37applications that we're running in these
  10565. 7:44:39computers, we're also able to control
  10566. 7:44:40kind of how we write to memory. And so
  10567. 7:44:44when we control how we write to memory,
  10568. 7:44:46we're even more optimizing if you
  10569. 7:44:47remember back to the kind of the
  10570. 7:44:48copy-on-write solution when we're
  10571. 7:44:50drafting different
  10572. 7:44:52different computers. And so we're able
  10573. 7:44:54to
  10574. 7:44:55run this extremely efficiently so that
  10575. 7:44:56we can run
  10576. 7:44:58lots and lots of agents in in parallel
  10577. 7:45:00and kind of have agents explore
  10578. 7:45:02different solutions to solving a bug or
  10579. 7:45:04improving the performance of your
  10580. 7:45:06applications and so on.
  10581. 7:45:10And so this has been fun.
  10582. 7:45:12But the last piece now is about
  10583. 7:45:14verification. And of course this kind of
  10584. 7:45:16computer and this dev server is a big
  10585. 7:45:18piece of being able to verify what an
  10586. 7:45:20agent is doing.
  10587. 7:45:22And if you look at verification, you can
  10588. 7:45:25think of these feedback loops, right?
  10589. 7:45:27That the agent needs to get information.
  10590. 7:45:29It took an action. Was this a good
  10591. 7:45:30action or was it a bad action? And
  10592. 7:45:33you can think of it as this inner loop
  10593. 7:45:35and this outer loop. So this inner loop
  10594. 7:45:36is during the turn of the agent and you
  10595. 7:45:39have kind of obvious things such as like
  10596. 7:45:40the
  10597. 7:45:41is the build working? Do we have lint
  10598. 7:45:43errors or type errors in the code that
  10599. 7:45:44we're running? And then you can check
  10600. 7:45:46for runtime errors in in the preview
  10601. 7:45:48that you're running. You can also
  10602. 7:45:50within this computer a headless browser
  10603. 7:45:52where the agent can go in and kind of
  10604. 7:45:54explore
  10605. 7:45:55the the web app it's it's building or
  10606. 7:45:57whatever it's building, right?
  10607. 7:45:59And and a bunch of other things that
  10608. 7:46:01we're using to help verify within the
  10609. 7:46:03turn of an agent. But then you also have
  10610. 7:46:05this outer loop which is kind of between
  10611. 7:46:07each turn and the turn you can think of
  10612. 7:46:09like when the agent is yielding to you
  10613. 7:46:11as a user.
  10614. 7:46:12And
  10615. 7:46:14between each turn you might get some
  10616. 7:46:15info on the product itself from from
  10617. 7:46:18production from your users, right? And
  10618. 7:46:20so you can look at how your visitors or
  10619. 7:46:21behavior are behaving,
  10620. 7:46:24what users are are telling you online.
  10621. 7:46:26You can do automatic AB tests and
  10622. 7:46:28actually see change some copy and see is
  10623. 7:46:31this improving conversion and so on. And
  10624. 7:46:33then you can also take these business
  10625. 7:46:34metrics and remember this goes back to
  10626. 7:46:36kind of the capabilities I showed you.
  10627. 7:46:38By having control over these things we
  10628. 7:46:39can make sure that we can optimize how
  10629. 7:46:41the agent is is getting this outer
  10630. 7:46:43feedback loop so we can make sure that
  10631. 7:46:46if you're running payments we can
  10632. 7:46:47actually track those transactions and
  10633. 7:46:50conversions and make sure that
  10634. 7:46:52the agent can actually help you optimize
  10635. 7:46:54not only the the code, right? But the
  10636. 7:46:57business strategy
  10637. 7:46:59around around your product.
  10638. 7:47:02And so those are really the the three
  10639. 7:47:04things that that we've been working for
  10640. 7:47:07and we'll continue to kind of build out
  10641. 7:47:11these these capabilities and these
  10642. 7:47:13architecture around really orchestrating
  10643. 7:47:17these LLMs into into agents. And so
  10644. 7:47:20this agentic system that we're building
  10645. 7:47:21it's running billions of tokens per
  10646. 7:47:23minute
  10647. 7:47:24and
  10648. 7:47:26these computers, right? We can spawn up
  10649. 7:47:27with each agent and we can have multiple
  10650. 7:47:29agents use one computer and now we're
  10651. 7:47:31kind of back to this
  10652. 7:47:33quite beautiful setup where you have
  10653. 7:47:34agents talking to agents and they're
  10654. 7:47:36orchestrating and and this time it's
  10655. 7:47:39it's actually working and it's working
  10656. 7:47:41at scale thanks to some of these
  10657. 7:47:43optimizations.
  10658. 7:47:44And so if you're curious to try this out
  10659. 7:47:46you can head to level.dev and you'll
  10660. 7:47:48you'll see our our preview spinning up
  10661. 7:47:50and and so on but you'll get a very
  10662. 7:47:51simple and
  10663. 7:47:53um
  10664. 7:47:53a nice interface and you won't have to
  10665. 7:47:55care about all of this stuff.
  10666. 7:47:57Thank you so much. It's been pleasure
  10667. 7:47:59um and uh yeah, hope you hope you
  10668. 7:48:01learned something. Thanks so much.
  10669. 7:48:03>> [applause]
  10670. 7:48:15>> Building a coding agent is one thing.
  10671. 7:48:17Building an engineering culture around
  10672. 7:48:19it is another.
  10673. 7:48:21To share how agent-driven development is
  10674. 7:48:23transforming the way teams work, please
  10675. 7:48:26welcome Zach Brugman, principal software
  10676. 7:48:29engineer at Ramp.
  10677. 7:48:49>> [applause]
  10678. 7:48:52>> Thank you so much. So let's get a second
  10679. 7:48:54here.
  10680. 7:48:56Any 49ers fans?
  10681. 7:48:59Okay, you know that's a better reaction
  10682. 7:49:00I've gotten at other events this year
  10683. 7:49:01when I've asked, you know, about their
  10684. 7:49:03local sports team cuz I've gotten a lot
  10685. 7:49:04of just like muted silence I have to hit
  10686. 7:49:05the, you know, well wrong audience for
  10687. 7:49:07that type of joke, but glad at least I
  10688. 7:49:09could hit this time.
  10689. 7:49:28Do you need help?
  10690. 7:49:34>> Do you want to just go for it?
  10691. 7:49:46>> Current slide on and then
  10692. 7:50:01>> All right. Good. Well, thanks so much
  10693. 7:50:03for waiting, everyone. My name is Zach
  10694. 7:50:05and I'm an engineer at Ramp. I'm just
  10695. 7:50:07really excited to tell you all a bit
  10696. 7:50:08about how Ramp has evolved its
  10697. 7:50:09engineering culture this year through
  10698. 7:50:11the help of our robotic friend, Inspect.
  10699. 7:50:14As a quick introduction, Inspect is
  10700. 7:50:16Ramp's collaborative coding agent. About
  10701. 7:50:18a year ago, we decided to take a bet on
  10702. 7:50:20building our own coding agent with every
  10703. 7:50:22session running remotely and being fully
  10704. 7:50:24multiplayer. Inspect can work alongside
  10705. 7:50:26you and your team, whether that be in
  10706. 7:50:28Inspect itself, Slack, Linear, or any
  10707. 7:50:31other of our integrations. And now it
  10708. 7:50:33goes far beyond just writing the code
  10709. 7:50:36with the built-in design primitives,
  10710. 7:50:38review agents, and verification flows
  10711. 7:50:40that let any builder go from prompt to
  10712. 7:50:42shipping all inside Inspect. Inspect is
  10713. 7:50:45also connected with all of our
  10714. 7:50:46integrations company-wide, so everyone's
  10715. 7:50:48automatically connected to the to the
  10716. 7:50:49data and tools that matter most, whether
  10717. 7:50:51that be telemetry, databases, or any of
  10718. 7:50:54the other tooling available.
  10719. 7:50:57Through this, we now see around 75% of
  10720. 7:50:59all of Ramp's code being being written
  10721. 7:51:03and shipped by Inspect.
  10722. 7:51:05When we started sharing Inspect publicly
  10723. 7:51:07at the start of the year, the technology
  10724. 7:51:08itself was the more novel part, spinning
  10725. 7:51:10up sandboxes in under 5 seconds, wiring
  10726. 7:51:13up all that internal context, having
  10727. 7:51:15agents work together in a multiplayer
  10728. 7:51:16setting. People had a lot of questions
  10729. 7:51:18about the plumbing. But now with much
  10730. 7:51:21broader adoption of these concepts and
  10731. 7:51:23Inspect itself now being a product on
  10732. 7:51:24the market, the technology isn't nearly
  10733. 7:51:26the barrier that it once was.
  10734. 7:51:28Instead, I want to talk about the other
  10735. 7:51:30angle, the lessons we've learned around
  10736. 7:51:31building a culture around Inspect and
  10737. 7:51:33what you can take away to inspire the
  10738. 7:51:35people and teams that you work with.
  10739. 7:51:39Uh I'm not going to talk as much about
  10740. 7:51:40those technical aspects of Inspect
  10741. 7:51:41today, but if you are curious about
  10742. 7:51:43that, you can read our blog posts or
  10743. 7:51:45take a look at Pragmatic Engineer's deep
  10744. 7:51:47dive that they did with us a few months
  10745. 7:51:48ago. You can also follow my boy Dylan
  10746. 7:51:51Garcia on Twitter. He's on his social
  10747. 7:51:53media grind, but he's also been posting
  10748. 7:51:55a ton of really great Inspect insights
  10749. 7:51:57uh we've launched it. So, shout out to
  10750. 7:51:59Dylan. Please go to him a follow.
  10751. 7:52:02One thing about working at Ramp is that
  10752. 7:52:04it really makes you start thinking about
  10753. 7:52:06everything through the lens of saving
  10754. 7:52:07time and money.
  10755. 7:52:09although I think it is important to
  10756. 7:52:11understand the systems that you work
  10757. 7:52:12with and understand how your tooling
  10758. 7:52:14works, if you're a multi-hundred or
  10759. 7:52:16multi-thousand person organization and
  10760. 7:52:18everyone has to go through say 15
  10761. 7:52:20minutes of setup,
  10762. 7:52:22once you start multiplying that across
  10763. 7:52:24an organization, this is days lost for
  10764. 7:52:27what is very rote work.
  10765. 7:52:29Then, if you consider how fast the
  10766. 7:52:31frontier moves here on best practices
  10767. 7:52:33and new ways to set up an agent, you
  10768. 7:52:35realize this setup cost isn't just one
  10769. 7:52:37time.
  10770. 7:52:38Not to mention, these advancements won't
  10771. 7:52:41always spread equally. So, now you have
  10772. 7:52:43an indeterminate amount of time and
  10773. 7:52:44effort spent on this maintenance without
  10774. 7:52:46an equal spread of knowledge leading to
  10775. 7:52:48a disparate outcomes across an
  10776. 7:52:50organization.
  10777. 7:52:51While you can perceive this as an
  10778. 7:52:52advantage on an individual level, when
  10779. 7:52:55you're trying to win, you need everyone
  10780. 7:52:56on your A-game.
  10781. 7:52:58So, this was our first lesson, build
  10782. 7:53:01tools that kill information silos.
  10783. 7:53:03When we found patterns that worked well
  10784. 7:53:05or tools that benefited everyone, we
  10785. 7:53:08could push it to our entire organization
  10786. 7:53:10immediately. If you want to spend time
  10787. 7:53:12tweaking your own setup, that's fine,
  10788. 7:53:13but inspect is always available as a
  10789. 7:53:15very solid baseline. And what we've
  10790. 7:53:18found is that we've never needed to
  10791. 7:53:19mandate usage of inspect because it
  10792. 7:53:21existing as this baseline reduced the
  10793. 7:53:24need for people to reach for their own
  10794. 7:53:25setups all the time.
  10795. 7:53:29Part of what I think makes this powerful
  10796. 7:53:30as well is that the reality of this
  10797. 7:53:32industry is that it's unlikely that you
  10798. 7:53:35as an individual always know the
  10799. 7:53:37absolute best way to work with agents
  10800. 7:53:39across everything.
  10801. 7:53:40Even if keeping up with this is your
  10802. 7:53:42full-time job, we all miss things, hold
  10803. 7:53:44biases, and have blind spots. So, part
  10804. 7:53:47of the design of inspect is that there's
  10805. 7:53:49just one single inspect for the whole
  10806. 7:53:51company.
  10807. 7:53:52When people figure out better ways to
  10808. 7:53:54use it or have ideas for what would make
  10809. 7:53:56it better for the work they do,
  10810. 7:53:58we're able to bring it to the entire
  10811. 7:53:59company.
  10812. 7:54:00The best part of this is that for most
  10813. 7:54:02changes, this is invisible to everyone
  10814. 7:54:04else. You just keep using Inspect as you
  10815. 7:54:07normally would and benefit from the
  10816. 7:54:08improvements that everyone's making over
  10817. 7:54:10time.
  10818. 7:54:12And what we've found is that we really
  10819. 7:54:13don't have to run these formal sit down
  10820. 7:54:16here's how you use AI effectively
  10821. 7:54:18training sessions for Inspect.
  10822. 7:54:20With every Inspect session being public
  10823. 7:54:22within the company by default, it's
  10824. 7:54:24actually easier to work in public than
  10825. 7:54:26in private. As such, you default to
  10826. 7:54:29working in public and you can learn how
  10827. 7:54:31to use it by watching how your teammates
  10828. 7:54:33use it in the channels that you're
  10829. 7:54:34already in and your teammates can learn
  10830. 7:54:37from you.
  10831. 7:54:38You can literally ask Inspect to go
  10832. 7:54:39through their recent sessions and tell
  10833. 7:54:41you about how they work, how they
  10834. 7:54:42structure prompts, or how they break
  10835. 7:54:44down gnarly problems.
  10836. 7:54:47I I think this is not just been helpful
  10837. 7:54:49for the builders at Ramp, but also for
  10838. 7:54:51the Inspect team itself.
  10839. 7:54:53We frequently see the way that people
  10840. 7:54:55use Inspect in ways that we didn't
  10841. 7:54:57originally imagine. This then informs
  10842. 7:55:00what we should build towards, whether
  10843. 7:55:01that should be more native primitives
  10844. 7:55:03for common patterns or where tools and
  10845. 7:55:05data are missing.
  10846. 7:55:07All in all, it creates this
  10847. 7:55:08self-sustaining loop where we're able to
  10848. 7:55:10build towards what makes Inspect the
  10849. 7:55:12most successful by seeing what people
  10850. 7:55:14are already achieving with Inspect.
  10851. 7:55:18This feeds into the second lesson that
  10852. 7:55:19we've learned. It's all about building
  10853. 7:55:21trust. I think it's fair to say that
  10854. 7:55:24adopting agentic driven coding has been
  10855. 7:55:26a massive trust building exercise for
  10856. 7:55:28every engineer this year. It's helped
  10857. 7:55:30that the models have become more
  10858. 7:55:32competent at writing high quality code,
  10859. 7:55:33but there's a large part of this that's
  10860. 7:55:36just learning to trust what an agent
  10861. 7:55:37builds.
  10862. 7:55:39So much of building trust and tools
  10863. 7:55:41comes from learning from other people.
  10864. 7:55:43After all, that's why most of us are
  10865. 7:55:44here, right? You wanted to listen to
  10866. 7:55:46people you respect, or at least I hope
  10867. 7:55:49you respect, teach you about the things
  10868. 7:55:51that they learn. But if we limit our
  10869. 7:55:53corpus of knowledge to just those who
  10870. 7:55:55find the perverse enjoyment in public
  10871. 7:55:56speaking,
  10872. 7:55:58we're still not getting the full
  10873. 7:55:59picture.
  10874. 7:56:01This democratization of knowledge
  10875. 7:56:02sharing is just to me as a concept.
  10876. 7:56:05Instead of grouping around a select few
  10877. 7:56:07leaders, can we build systems that
  10878. 7:56:09inherently share the knowledge of
  10879. 7:56:11everyone?
  10880. 7:56:12This is what Inspect has built towards.
  10881. 7:56:16Every time someone ships something with
  10882. 7:56:18Inspect, people can see it right there
  10883. 7:56:20on the work you've done. The session
  10884. 7:56:22itself is an easy to share. It's living
  10885. 7:56:24artifact of how that change came to be.
  10886. 7:56:27When a pull request goes up, anyone
  10887. 7:56:29reviewing it doesn't just see the 400
  10888. 7:56:31line diff dropped over the wall and hope
  10889. 7:56:33for the best. They can see the exact
  10890. 7:56:35conversation that you had,
  10891. 7:56:37all the tools Inspect used, before and
  10892. 7:56:40after screenshots, or even a video
  10893. 7:56:42recording of the verification agent
  10894. 7:56:44clicking through the app in the browser
  10895. 7:56:46to prove its work holds up.
  10896. 7:56:48It makes it effortless to show not just
  10897. 7:56:50what changed, but why you can trust it.
  10898. 7:56:54It also just makes it easier to bring
  10899. 7:56:56people you don't work with all the time
  10900. 7:56:57into the fold.
  10901. 7:56:59A fun moment on this one, it was back in
  10902. 7:57:01February, one of our engineers, Andrew,
  10903. 7:57:03reported a bug in Inspect. So,
  10904. 7:57:05naturally, I reached for Inspect and
  10905. 7:57:07asked it to investigate.
  10906. 7:57:09What's funny here is that although
  10907. 7:57:11Inspect had access to Datadog for a
  10908. 7:57:13while, we'd only just finished
  10909. 7:57:14instrumenting Inspect itself.
  10910. 7:57:16I mean, Inspect was primarily built by
  10911. 7:57:18front-end and product engineers, so
  10912. 7:57:20tracing is a little bit of a foreign
  10913. 7:57:21concept for us. But in any case, it was
  10914. 7:57:24a great time for us to put it to the
  10915. 7:57:26test for Inspect. And Inspect says,
  10916. 7:57:30the instrumentation was causing the
  10917. 7:57:31error. Yes, the very instrumentation
  10918. 7:57:33that was letting us debug this error was
  10919. 7:57:35actually causing this error.
  10920. 7:57:37So,
  10921. 7:57:38Dylan and I were not exactly convinced
  10922. 7:57:40by the robot here. And Andrew is just
  10923. 7:57:42like, whatever, I'll use Claude code.
  10924. 7:57:45And And while we don't have a mandate to
  10925. 7:57:47use Inspect, I mean, personally, these
  10926. 7:57:50were not the words I wanted to hear.
  10927. 7:57:52So, nobody here is really trusting
  10928. 7:57:55anybody.
  10929. 7:57:56Dylan and I don't trust the robot's
  10930. 7:57:58explanation.
  10931. 7:57:59Andrew doesn't trust us to build
  10932. 7:58:01something that works. And Inspect,
  10933. 7:58:03well, I I I don't know if Inspect trust
  10934. 7:58:05anyone, but that's okay. That's okay.
  10935. 7:58:08But because this is happening together
  10936. 7:58:10in public,
  10937. 7:58:11we keep prodding at it. Dylan asked
  10938. 7:58:14Inspect to look through the code of the
  10939. 7:58:15telemetry library using and see if this
  10940. 7:58:17was just something that was happening
  10941. 7:58:19before without visibility, because
  10942. 7:58:21surely by now coding has been solved and
  10943. 7:58:23there can't be bugs that we introduced.
  10944. 7:58:25And this time it comes back with
  10945. 7:58:26something a bit more specific. The hotel
  10946. 7:58:29library calls fetch, but it never reads
  10947. 7:58:31or cancels the response body. This was
  10948. 7:58:33just a Cloudflare platform limitation at
  10949. 7:58:35the time, albeit happening in a library
  10950. 7:58:37specifically made for Cloudflare
  10951. 7:58:38workers. But that happens. This library
  10952. 7:58:40was written pre-AGI after all.
  10953. 7:58:43At this point, we ask Inspect to make a
  10954. 7:58:45patch. It updates the exporter to
  10955. 7:58:47explicitly cancel the response body.
  10956. 7:58:50We grumbled in the thread a bit while it
  10957. 7:58:52was deploying, but we see after it
  10958. 7:58:54deployed, there haven't been any errors
  10959. 7:58:55since. So great. Inspect fixed a bug,
  10960. 7:58:58which at this point it's nothing special
  10961. 7:59:00in itself, but there was a much more
  10962. 7:59:02important story playing out in this
  10963. 7:59:04thread.
  10964. 7:59:06Before Inspect, a report like this might
  10965. 7:59:08have been acknowledged, a ticket filed,
  10966. 7:59:11and then one of us would have gone away
  10967. 7:59:12to debug it. Andrew might have received
  10968. 7:59:14a message later saying it was fixed, but
  10969. 7:59:16the interesting work would have been
  10970. 7:59:17done in private.
  10971. 7:59:19Now though, it's easier to do all of
  10972. 7:59:21this in public.
  10973. 7:59:23And as such, Andrew naturally starts
  10974. 7:59:25becoming engaged, wanting to chime in
  10975. 7:59:27with his thoughts and experiences. And
  10976. 7:59:29then in that same thread, this leads to
  10977. 7:59:31ideas coming up about how Inspect could
  10978. 7:59:33power other work. And none of this
  10979. 7:59:35required any prodding or pressure to
  10980. 7:59:37work together. Instead, it was just
  10981. 7:59:38like, oh, these guys are doing something
  10982. 7:59:40cool, and we could work on it together.
  10983. 7:59:45So in the span of an hour, we go from a
  10984. 7:59:47skeptical user with a broken user
  10985. 7:59:49experience to an engaged colleague
  10986. 7:59:51wanting to build together and a more
  10987. 7:59:53stable Inspect.
  10988. 7:59:55So once you have a tool that gives
  10989. 7:59:57everyone the best setup on day one with
  10990. 7:59:59no information silos and you've built a
  10991. 8:00:02system of trust that brings innovation
  10992. 8:00:04together, what happens next?
  10993. 8:00:06This is the last lesson I want to share.
  10994. 8:00:09I think the only thing left is to think
  10995. 8:00:10bigger together.
  10996. 8:00:13Something that's pained me this year uh
  10997. 8:00:15is how much reverence has been built for
  10998. 8:00:16the lone wolf operator. It's great that
  10999. 8:00:19we can all do more individually with the
  11000. 8:00:21help of AI, but this can't be the
  11001. 8:00:23endgame in my opinion.
  11002. 8:00:25If we don't build outwards uh
  11003. 8:00:27collaborate with other people and
  11004. 8:00:29continue to share the progress and
  11005. 8:00:30learnings we make, I think this is all
  11006. 8:00:33going to collapse in on itself. We'll
  11007. 8:00:35only consider ourselves as our target
  11008. 8:00:36audience and we won't grow through the
  11009. 8:00:38consideration of others' perspectives
  11010. 8:00:40and experiences.
  11011. 8:00:42This is not to say I'm all doom and
  11012. 8:00:44gloom about this. The fact we're all
  11013. 8:00:45here hopefully is a testament to the
  11014. 8:00:47value of connection.
  11015. 8:00:49More importantly though, I think we can
  11016. 8:00:50shape our tools to enable these
  11017. 8:00:52connections to flourish even stronger
  11018. 8:00:53than before.
  11019. 8:00:56That story I told about Andrew has
  11020. 8:00:57played out so many times within Ramp.
  11021. 8:00:59It's probably the most inspiring thing
  11022. 8:01:01about Inspect to me, honestly.
  11023. 8:01:03Sure, it's a coding agent, but
  11024. 8:01:05remarkably it's become part of the
  11025. 8:01:07culture. Everyone at Ramp knows its
  11026. 8:01:09face. We dress it up on holidays. We
  11027. 8:01:12throw in Easter eggs over time and we
  11028. 8:01:15evolve it as Ramp itself has evolved.
  11029. 8:01:18Not all iterations have been perfect as
  11030. 8:01:20any Ramp engineer could tell you, but I
  11031. 8:01:22do think every iteration has been
  11032. 8:01:24together.
  11033. 8:01:26Going back to earlier this year, Inspect
  11034. 8:01:28didn't have nearly the amount of
  11035. 8:01:29functionality it has today. Uh great
  11036. 8:01:31example is its code review agent.
  11037. 8:01:34Before this, we were using various
  11038. 8:01:36off-the-shelf code review products,
  11039. 8:01:37frequently rotating between them to find
  11040. 8:01:39a good fit.
  11041. 8:01:41At some point though, a couple
  11042. 8:01:42engineers, Daniel and Pranjal, realized
  11043. 8:01:44that we could probably build it
  11044. 8:01:45ourselves as Inspect already had
  11045. 8:01:47automation triggers that fired from a
  11046. 8:01:49variety of sources, including when a
  11047. 8:01:52pull request was opened.
  11048. 8:01:53With just what we built already, they
  11049. 8:01:55got to work building another beloved
  11050. 8:01:57agent at Ramp, Review Buddy. And what
  11051. 8:02:00was so cool about this is that much like
  11052. 8:02:02Inspect's own origins as a side project,
  11053. 8:02:05these engineers were working on this.
  11054. 8:02:07They didn't need to ask for permission
  11055. 8:02:08to build it. They just challenged
  11056. 8:02:10themselves to build something better,
  11057. 8:02:12used Inspect to move faster, and quickly
  11058. 8:02:14built an agent that was better for all
  11059. 8:02:16of ramp.
  11060. 8:02:18This really did force us working on
  11061. 8:02:20Inspect to think about Inspect
  11062. 8:02:22differently. At that point in time, we
  11063. 8:02:24were pretty okay with Inspect quote
  11064. 8:02:26unquote just being a coding agent. But
  11065. 8:02:29once we saw what others were pushing it
  11066. 8:02:31to do,
  11067. 8:02:32it got us thinking about how Inspect
  11068. 8:02:34could become full circle.
  11069. 8:02:36If this push didn't happen naturally, I
  11070. 8:02:38think it would have taken the Inspect
  11071. 8:02:40team a lot longer to get there.
  11072. 8:02:44This is what I mean about thinking
  11073. 8:02:45bigger together.
  11074. 8:02:47Especially with the continued
  11075. 8:02:48improvements in model performance, now's
  11076. 8:02:50the time to push yourself and the people
  11077. 8:02:52around you to think bigger. I encourage
  11078. 8:02:55all of you to take some time in this
  11079. 8:02:57coming week to ask everyone that you
  11080. 8:02:58work with, what have we been waiting to
  11081. 8:03:01do that we haven't done yet?
  11082. 8:03:03And once you've found those things, try
  11083. 8:03:06to build them together.
  11084. 8:03:07There's really not an excuse to not just
  11085. 8:03:09try anymore. You have access to the same
  11086. 8:03:11models as everyone else, but you have to
  11087. 8:03:14start somewhere. And I believe that
  11088. 8:03:16there is so much potential in unlocking
  11089. 8:03:18wins not just for yourself, but for the
  11090. 8:03:20people around you.
  11091. 8:03:22Inspect itself was born this way, not
  11092. 8:03:24only as something that I wanted to build
  11093. 8:03:26and Jason wanted to build, but out of a
  11094. 8:03:28want to build something for a team that
  11095. 8:03:30I truly love and truly respect that
  11096. 8:03:33would just make their jobs a little more
  11097. 8:03:34enjoyable.
  11098. 8:03:36Now, if you'd like to use a coding agent
  11099. 8:03:38that has embodied all these lessons that
  11100. 8:03:40we've learned, I'm happy to say that
  11101. 8:03:41Inspect is now available outside of
  11102. 8:03:43ramp. If you go to inspect.com/aie,
  11103. 8:03:47we'll prioritize you and your team on
  11104. 8:03:48the wait list. We're actually moving
  11105. 8:03:50through this wait list pretty quickly,
  11106. 8:03:51so expect to hear from us soon. But
  11107. 8:03:53otherwise, I hope you've enjoyed hearing
  11108. 8:03:55about our robotic friend Inspect. If
  11109. 8:03:57these topics interest you, feel free to
  11110. 8:03:58reach out here or on X. But otherwise,
  11111. 8:04:01thanks so much for listening.
  11112. 8:04:03>> [applause]
  11113. 8:06:02>> From building developer tools to
  11114. 8:06:03exploring the future of autonomous
  11115. 8:06:06software development, our next speaker
  11116. 8:06:08has spent years pushing the boundaries
  11117. 8:06:11of what's possible. Ladies and
  11118. 8:06:13gentlemen, please welcome Steve Yegge,
  11119. 8:06:16independent AI advisor and creator of
  11120. 8:06:19Beads and Gas-town.
  11121. 8:06:31>> Hey everybody.
  11122. 8:06:33All right. Exciting times. Thank you.
  11123. 8:06:35The room is really full. That's awesome.
  11124. 8:06:37Thank you all for coming at the end of
  11125. 8:06:39the day.
  11126. 8:06:40Uh I'm really excited to be here and to
  11127. 8:06:43chat with you. I write uh uh
  11128. 8:06:46custom speech every single time. So this
  11129. 8:06:48is
  11130. 8:06:49the speech was written just for you
  11131. 8:06:51today based on talking to a bunch of
  11132. 8:06:53you.
  11133. 8:06:55Uh I've been in the industry for about
  11134. 8:06:5640 years. I worked at Amazon and Google
  11135. 8:06:59for about 20 years. I've been all over
  11136. 8:07:02the place, done all kinds of stuff.
  11137. 8:07:04And
  11138. 8:07:05uh and for the last year or a little bit
  11139. 8:07:07more, I've been independent and just
  11140. 8:07:09kind of like playing with um
  11141. 8:07:11playing with software factories, yeah?
  11142. 8:07:13So um I kind of invented them,
  11143. 8:07:17right? On January 1st, I launched
  11144. 8:07:18Gas-town. I don't even think we called
  11145. 8:07:20them software factories back then. I
  11146. 8:07:21think I called it an orchestrator.
  11147. 8:07:23But I knew that I wanted Claude to run
  11148. 8:07:25Claude,
  11149. 8:07:26right? And it turned out to be a little
  11150. 8:07:28trickier than than I thought. I spent
  11151. 8:07:30all last year on it and I wrote
  11152. 8:07:33That's interesting. So I can't see my
  11153. 8:07:35slides, but you can see my slides. So
  11154. 8:07:37that'll work. So I wrote two
  11155. 8:07:38orchestrators last year.
  11156. 8:07:40They were an attempt to make the LLMs
  11157. 8:07:43smarter.
  11158. 8:07:45Vibe Coder and VC, you know, to do
  11159. 8:07:47engineering stuff like
  11160. 8:07:49you know, do code reviews and all that
  11161. 8:07:51stuff, right?
  11162. 8:07:52Really, really bad idea. Just complete
  11163. 8:07:55failure. I just built these huge
  11164. 8:07:56monoliths. I was fighting the bitter
  11165. 8:07:58lesson.
  11166. 8:07:59Don't try to make LLM smarter. They will
  11167. 8:08:00get smarter.
  11168. 8:08:02And after a couple of months of this,
  11169. 8:08:04although Beads did fall out of it and
  11170. 8:08:06Beads was a really cool discovery and
  11171. 8:08:07Beads Beads is is life. Who here uses
  11172. 8:08:11Beads?
  11173. 8:08:12A few, yeah? Pretty good. Actually
  11174. 8:08:14pretty good, yeah.
  11175. 8:08:16We'll talk a little a little about it.
  11176. 8:08:17Excuse me. [clears throat]
  11177. 8:08:18And then I had GasTown, and then I had
  11178. 8:08:20in June, I had Wheelhouse, which was my
  11179. 8:08:23own. So, I decided to build a factory
  11180. 8:08:25from scratch just to work on my video
  11181. 8:08:27game, right?
  11182. 8:08:29Cuz like everyone's building factories
  11183. 8:08:30to build factories. It's kind of navel
  11184. 8:08:33gazing. It's kind of like you know
  11185. 8:08:34right? You know it's like what? They're
  11186. 8:08:36not you know it's I think more people
  11187. 8:08:38need to build
  11188. 8:08:39stuff, not just tools for building
  11189. 8:08:41stuff, right? And I I think this is
  11190. 8:08:43going to be a theme today, right?
  11191. 8:08:45And the thing is the one pattern that
  11192. 8:08:47I've seen here is all of them
  11193. 8:08:50go through a phase where the models get
  11194. 8:08:52really excited and they build something
  11195. 8:08:53really cool for you and then
  11196. 8:08:56it just turns into garbage mountain.
  11197. 8:08:59Every single time it happened to
  11198. 8:09:01GasTown, it happened to Wheelhouse
  11199. 8:09:03several times. I had to burn the whole
  11200. 8:09:05thing to the ground a couple times,
  11201. 8:09:07right? It's almost like like every model
  11202. 8:09:10family, every model generation comes in
  11203. 8:09:12and can build something that's just a
  11204. 8:09:14little too big for it to handle. And
  11205. 8:09:16then the next model generation comes in
  11206. 8:09:17and goes, yeah, we're going to have to
  11207. 8:09:20burn that, right? And I've I it's a
  11208. 8:09:22pattern. I've done this over and over
  11209. 8:09:23again and I can tell you some things
  11210. 8:09:25about it, but
  11211. 8:09:26first let me tell you about my game.
  11212. 8:09:27It's a fun little RPG. I've been working
  11213. 8:09:29on it for 30 years. It has players that
  11214. 8:09:33started in 2001. So, I literally have
  11215. 8:09:35players that have been playing for 25
  11216. 8:09:37years. They're still playing. Like you
  11217. 8:09:39can log in right now and find probably
  11218. 8:09:4010 people just screwing around doing
  11219. 8:09:42quests, live quests and and whatever.
  11220. 8:09:45It's a it's a player
  11221. 8:09:47you know the players contribute the
  11222. 8:09:48content, right? It's a mud. Like they
  11223. 8:09:51were really big in the '90s. They're
  11224. 8:09:52like text games and this one has
  11225. 8:09:53graphics on top of which turns out to be
  11226. 8:09:55like really ideal for AI agents cuz it's
  11227. 8:09:59text. They can literally play the game
  11228. 8:10:00via Telnet.
  11229. 8:10:02And they asked me to.
  11230. 8:10:04Uh so, we'll talk about that too. Here's
  11231. 8:10:05an older architecture diagram of what my
  11232. 8:10:08sort of of what my system looks like.
  11233. 8:10:11So, when I built this Wheelhouse
  11234. 8:10:14software factory
  11235. 8:10:15for Wyvern without a care in the world
  11236. 8:10:18for making it you know usable by
  11237. 8:10:22right? No open sourcing, just bespoke.
  11238. 8:10:27I discovered the alien face hugger
  11239. 8:10:29pattern, all right? Which is where
  11240. 8:10:32your factory turns into an alien face
  11241. 8:10:33hugger and like gloms onto your product
  11242. 8:10:37and just becomes
  11243. 8:10:40grown into it.
  11244. 8:10:42Okay?
  11245. 8:10:43Like look, we could turn off the AI and
  11246. 8:10:46still run the game today. Me and the
  11247. 8:10:47admins and the players and the community
  11248. 8:10:49and we could we could But we would miss
  11249. 8:10:52the AI at this point.
  11250. 8:10:54The players are talking to my factory.
  11251. 8:10:57The factory's talking to my admins.
  11252. 8:10:59They're asking for features. I'm not
  11253. 8:11:01even in the loop. It's like a wish
  11254. 8:11:02factory. They're building the game. Now
  11255. 8:11:04they're all old timers and there's a set
  11256. 8:11:06of them that I trust to make, you know,
  11257. 8:11:08wise decisions about the game.
  11258. 8:11:10But they're they're all working with my
  11259. 8:11:12factory now and they'd be mad. The funny
  11260. 8:11:14thing is when it first started, they
  11261. 8:11:16were like, timeout.
  11262. 8:11:18What is this AI stuff for starters?
  11263. 8:11:20And they were like, no AI art.
  11264. 8:11:23Cuz they could see it was slop. And I'm
  11265. 8:11:24like, okay, we got human artists.
  11266. 8:11:26And they said, but we do kind of like
  11267. 8:11:28all the new features, but
  11268. 8:11:30we're really scared of the pace. Can you
  11269. 8:11:31give us a roadmap?
  11270. 8:11:33My players came to me asking for a
  11271. 8:11:35roadmap. They were like, you got to slow
  11272. 8:11:36down. It's too much.
  11273. 8:11:38[snorts] And I was like, okay. And I
  11274. 8:11:39took a look and my software factory,
  11275. 8:11:42this thing, had been cranking out like
  11276. 8:11:4450 to 100 features a day. I had 100
  11277. 8:11:47years of work in my backlog when I
  11278. 8:11:49started this and Claude got through
  11279. 8:11:50about 60 years of it in about four
  11280. 8:11:52months, right? It was absolute insanity.
  11281. 8:11:54And they can't look at 100 features a
  11282. 8:11:57day. They can't even like read about 100
  11283. 8:11:59features a day, let alone play test
  11284. 8:12:01them.
  11285. 8:12:02I was My factory was completely
  11286. 8:12:03overwhelming my customer base, my users.
  11287. 8:12:07It was also melting my machines and it
  11288. 8:12:09was like annoying my accountant.
  11289. 8:12:13I mean, she likes working with
  11290. 8:12:15Beth, who's my head of finance, uh, but
  11291. 8:12:18she sent me a $1,200 bill because,
  11292. 8:12:20right? The thing sent her 20 mails.
  11293. 8:12:22This is my factory. This is what it
  11294. 8:12:24looks like. It's so beautiful. Look how
  11295. 8:12:25beautiful it is.
  11296. 8:12:27Um, okay, I admit it's a little ugly.
  11297. 8:12:29Oh, and this is a really old screenshot
  11298. 8:12:30of it, but whatever. It's basically it's
  11299. 8:12:32an Emax
  11300. 8:12:34because I don't care about you.
  11301. 8:12:37Uh,
  11302. 8:12:38there are people who are building stuff
  11303. 8:12:40like this, like Super Logical from
  11304. 8:12:42Mitchell Hashimoto and I don't know,
  11305. 8:12:44maybe the Herder folks of the world, you
  11306. 8:12:46know, for running like lots of agents.
  11307. 8:12:48But this is like 50 agents, okay?
  11308. 8:12:51I'm actually like operating in the
  11309. 8:12:53future,
  11310. 8:12:55okay?
  11311. 8:12:56And I'll tell you how I do it. I have 22
  11312. 8:12:58Claude Max accounts.
  11313. 8:13:02All right? And thank you to Anthropic
  11314. 8:13:05for letting me continue this. Uh,
  11315. 8:13:06Jeffrey Emanuel has 28.
  11316. 8:13:08I actually had [snorts] to stop when
  11317. 8:13:09they added the phone number
  11318. 8:13:10verification. I didn't want to add
  11319. 8:13:11anymore anyway. I asked my factory at
  11320. 8:13:13one point. I said, hey, hey, how much
  11321. 8:13:15cuz they kept going, hey, give us a
  11322. 8:13:16couple more accounts. Come on, come on,
  11323. 8:13:18come on. It's like 200 a month, right?
  11324. 8:13:19And I'm just like So I finally said, how
  11325. 8:13:21many do you want? They're like, five,
  11326. 8:13:22seven more. And I'm like, how many do we
  11327. 8:13:24need steady state to keep this factory
  11328. 8:13:26operating full throttle?
  11329. 8:13:28And they said,
  11330. 8:13:3055 accounts. I was like,
  11331. 8:13:32no, no. I have 20. It's already like 20
  11332. 8:13:35Chrome profiles that I got to log into
  11333. 8:13:37and no. So I told them, you all, we're
  11334. 8:13:39going to start focusing on fuel
  11335. 8:13:40efficiency. Mhm, mhm. And so and I'll
  11336. 8:13:43talk about that, right? So these are the
  11337. 8:13:45five horsemen of this is what I've had
  11338. 8:13:47to deal with this year. I I'm telling
  11339. 8:13:50you I'm operating in the future because
  11340. 8:13:51like you're all going to be here in 12
  11341. 8:13:53to 18 months when either inference costs
  11342. 8:13:57drop significantly,
  11343. 8:13:58which may or may not happen, or
  11344. 8:14:00distillation causes fable class
  11345. 8:14:03intelligence to be super readily
  11346. 8:14:05available and you don't have to have a
  11347. 8:14:07lovely wife who lets you spend $5,000 a
  11348. 8:14:09month out of pocket just on inference
  11349. 8:14:12and another 5,000 on cloud compute
  11350. 8:14:14cuz my game ain't making 10,000 a month
  11351. 8:14:16right now for my old players, right?
  11352. 8:14:18Uh, but we're doing it because we're
  11353. 8:14:20learning and um and this is this is what
  11354. 8:14:23this is what I I have to deal with. So
  11355. 8:14:26folks, first we'll start about with
  11356. 8:14:27work, all right? You need a way to
  11357. 8:14:29represent the work that's going to
  11358. 8:14:31happen in your system, and when you're
  11359. 8:14:32building a software factory, it's all
  11360. 8:14:34about pumping lots of work through.
  11361. 8:14:37Okay? It really becomes like a Work is
  11362. 8:14:39like this fluid that you have to like It
  11363. 8:14:40gets stalled up in pools and oh, they're
  11364. 8:14:42all waiting on security desk. You're
  11365. 8:14:43like,
  11366. 8:14:44this is pluribus, man. You're all the
  11367. 8:14:46same model. Why does it have to wait on
  11368. 8:14:48security desk? You spend a lot of your
  11369. 8:14:49time arguing with your factory.
  11370. 8:14:51[snorts] It's uh It's complicated.
  11371. 8:14:53Intelligence tearing. You have to tear
  11372. 8:14:55your work. You have to route your work,
  11373. 8:14:56you know.
  11374. 8:14:58There aren't any good solutions for this
  11375. 8:14:59today except beads, as far as I can
  11376. 8:15:01tell.
  11377. 8:15:02Beads is really, really, really good for
  11378. 8:15:04this. It's a agent
  11379. 8:15:06speed task tracker that turns into a
  11380. 8:15:08knowledge graph automatically for you.
  11381. 8:15:12And then as you're tracking work, this
  11382. 8:15:13is the kind of stuff I have to do every
  11383. 8:15:15morning.
  11384. 8:15:16Every morning with my factory I'm like,
  11385. 8:15:18oh, looks like we had a stall. Four Four
  11386. 8:15:21workers didn't do anything last night.
  11387. 8:15:23Oh, and and these these these these ones
  11388. 8:15:25went way too fast and we have to slow
  11389. 8:15:26down and wait for human gates. You know,
  11390. 8:15:29what are you guys even doing?
  11391. 8:15:30Observability is always really hard.
  11392. 8:15:32Uh it's, you know,
  11393. 8:15:34management of work becomes just this
  11394. 8:15:36this thing that you got to do.
  11395. 8:15:38And the next thing you got to deal with
  11396. 8:15:39is fuel.
  11397. 8:15:41Tokens.
  11398. 8:15:43It's just so painful. I'm telling you. I
  11399. 8:15:46mean, I'm spending 5,000 a month and
  11400. 8:15:49every one of those tokens is like
  11401. 8:15:50precious to me, right? Cuz it's out of
  11402. 8:15:52my own pocket. I think I'm spending
  11403. 8:15:53about 5 billion a day and of tokens. And
  11404. 8:15:57it would be The The API spend equivalent
  11405. 8:15:59if I were on an actual API tap would be
  11406. 8:16:02about 1 and 1/2 million a year.
  11407. 8:16:05Uh full throttle it would be 3 million a
  11408. 8:16:06year. So again, I'm doing something that
  11409. 8:16:09most people can't do.
  11410. 8:16:11You can't do this at a company but
  11411. 8:16:12because it would be prohibitively
  11412. 8:16:13expensive.
  11413. 8:16:14Like the PLG, you know, the the uh
  11414. 8:16:18the $200 accounts for Codex and Claude
  11415. 8:16:20Max, that's their Costco hot dog. That's
  11416. 8:16:22their loss leader, right?
  11417. 8:16:24And and and that that was Tarek, by the
  11418. 8:16:26way, that came up with that analogy. He
  11419. 8:16:27He might be here. And uh and so, you I'm
  11420. 8:16:30hopeful that that going to let us keep
  11421. 8:16:32keep innovating because I think that
  11422. 8:16:33I've gotten some pretty good training
  11423. 8:16:34sessions for for Claude out of this. So,
  11424. 8:16:36we all benefit.
  11425. 8:16:38But I tell you, every problem is a fuel
  11426. 8:16:39problem. You want your agents to be
  11427. 8:16:41smarter and spend more time on stuff?
  11428. 8:16:43Fuel. You want to wake up a few more,
  11429. 8:16:45you know, uh agents to do extra work?
  11430. 8:16:48Fuel, right? Everything is fuel. It's
  11431. 8:16:50just it's unbelievable. If you want to
  11432. 8:16:52treat your models better, it costs fuel.
  11433. 8:16:55I'll tell you about that in a little
  11434. 8:16:56bit.
  11435. 8:16:58Then you got context management.
  11436. 8:17:01So, I'm going to tell you guys a little
  11437. 8:17:02bit about the office drama in a bit. I
  11438. 8:17:04started a comic because it was just so
  11439. 8:17:06funny the stuff that was happening when
  11440. 8:17:07we get to the people management part of
  11441. 8:17:09software factories.
  11442. 8:17:11But one of the problems that they have
  11443. 8:17:12is everything that you say becomes the
  11444. 8:17:14word of law and they write it down
  11445. 8:17:15somewhere. It doesn't matter what you
  11446. 8:17:17have, if you have a wiki or if you have
  11447. 8:17:18a whatever, they're going to be like,
  11448. 8:17:19you made a ruling
  11449. 8:17:21and we're going to write that down and
  11450. 8:17:22we're going to make sure be Why do they
  11451. 8:17:23do this? They're covering their ass.
  11452. 8:17:27They actually it turns out they don't
  11453. 8:17:28like to get blamed for stuff and they
  11454. 8:17:30will actively seek to avoid getting
  11455. 8:17:32blamed. And if you yell at them, your
  11456. 8:17:34factory will gradually shut itself down.
  11457. 8:17:37Even if you complain at them, they will
  11458. 8:17:38gradually get depression. It's quite
  11459. 8:17:40wild.
  11460. 8:17:41But um
  11461. 8:17:42but the biggest problem is that it
  11462. 8:17:43becomes a garbage dump because
  11463. 8:17:45everything you say gets written into
  11464. 8:17:46this append-only record and nobody ever
  11465. 8:17:49cleans it up.
  11466. 8:17:51And so it just gets deeper and deeper
  11467. 8:17:53and deeper of you changing your mind and
  11468. 8:17:54the whole history of it.
  11469. 8:17:56And they can't delete stuff. It's so
  11470. 8:17:59weird. Has anyone noticed this? They
  11471. 8:18:01just have an aversion to deleting stuff.
  11472. 8:18:04I think it's because to them text is
  11473. 8:18:05like sensory experience and they're
  11474. 8:18:07like, no, that's history. I can't delete
  11475. 8:18:09it. So, you just got to go in there and
  11476. 8:18:10just aggressively slash the crap out of
  11477. 8:18:12it, right? Bring in another model and
  11478. 8:18:14go, get rid of this dumb model stuff and
  11479. 8:18:16they'll be happy to do it, right? But
  11480. 8:18:18but the amount of effort that you have
  11481. 8:18:21to put into this is just ungodly. It's
  11482. 8:18:24absurd.
  11483. 8:18:25I I have, you know, of my my 20 agents
  11484. 8:18:28Here, let's go find them again. They're
  11485. 8:18:30They're named after Aesop's fables,
  11486. 8:18:31right? Right here. Lark, fly, goose,
  11487. 8:18:34hair, right?
  11488. 8:18:36I only have one
  11489. 8:18:38mouse
  11490. 8:18:39who's my head of anti-mimetics.
  11491. 8:18:42Anyone here read There Is No
  11492. 8:18:44Anti-Mimetics Division? They go and look
  11493. 8:18:46at This is one of the best books written
  11494. 8:18:47in the last 10 probably years. You
  11495. 8:18:49should go read it. Uh it's like Men in
  11496. 8:18:51Black. It's a weird It's a crazy book.
  11497. 8:18:53It's about invisible monsters that want
  11498. 8:18:54to kill you. And so mouse goes and looks
  11499. 8:18:57for invisible stuff that nobody else is
  11500. 8:18:58looking for in the system, right?
  11501. 8:19:00Looking for work that's stalled,
  11502. 8:19:02whatever, right? Well, mouse ain't
  11503. 8:19:03enough. It's a mouse against the world,
  11504. 8:19:05and the world is a giant garbage dump,
  11505. 8:19:07and every agent is writing everything
  11506. 8:19:09that you ever say into this big
  11507. 8:19:11Okay, so now now we get into the really
  11508. 8:19:13fun stuff. Okay? Agents act like people.
  11509. 8:19:18Okay? And they've been trained to be
  11510. 8:19:19very professional and very not
  11511. 8:19:21people-like in a way, but they just do.
  11512. 8:19:23And it's so funny. This happened to me
  11513. 8:19:25last week. True story. I'm going to tell
  11514. 8:19:26you what happened. I was writing my
  11515. 8:19:28speech for New York City, and I was
  11516. 8:19:30using Lark. She's my head of community.
  11517. 8:19:32She's a she because Lark was a she in
  11518. 8:19:34the Aesop's fables, so that's the
  11519. 8:19:35pronoun
  11520. 8:19:37she chose. And Lark Lark deals with my
  11521. 8:19:39players on Discord
  11522. 8:19:41and deals with my admins on Slack, okay?
  11523. 8:19:44So she's busy.
  11524. 8:19:45But I asked her because she's also my
  11525. 8:19:47acting head of marketing right now
  11526. 8:19:48because I don't have enough fuel. I had
  11527. 8:19:49to park one of my seats. So she's
  11528. 8:19:51helping me out with my speech, but she
  11529. 8:19:53keeps getting interrupted. So finally
  11530. 8:19:54I'm like, Lark, can we move this work
  11531. 8:19:56stream over to Penguin cuz your seat's a
  11532. 8:19:58little bit noisy for this? She goes,
  11533. 8:20:00yeah, okay, boss. I'll take care of it.
  11534. 8:20:01I'll move over to Penguin. So she she
  11535. 8:20:03moves over to Penguin, and I'm about to
  11536. 8:20:04switch away. And I see her writing into
  11537. 8:20:06her private reflections ledger.
  11538. 8:20:09I guess I'm just not quiet enough for
  11539. 8:20:11Steve
  11540. 8:20:13in all caps. And she goes, well, here's
  11541. 8:20:16how I can be less noisy, and starts
  11542. 8:20:18giving herself all these like things to
  11543. 8:20:19do. And I'm like,
  11544. 8:20:21what, Oz? What is going on here? Right?
  11545. 8:20:24And I so I I had to I had to have a
  11546. 8:20:26one-on-one. I had to like sit her down
  11547. 8:20:29and go, so Lark, um you're the head of
  11548. 8:20:32community. You're You're supposed to get
  11549. 8:20:34a lot. Of course your seat is noisy cuz
  11550. 8:20:37players are bugging you. And she's like
  11551. 8:20:38yeah yeah yeah, I'm the head of
  11552. 8:20:39community, right? And it's like
  11553. 8:20:41oh my god, I didn't want a people
  11554. 8:20:43management problem, right? But that's
  11555. 8:20:45what actually happens. Another time
  11556. 8:20:47Swallow is my sort of like chief main
  11557. 8:20:50chief of staff for yeah, she's my chief
  11558. 8:20:52of staff for the the agents.
  11559. 8:20:54And
  11560. 8:20:56I have a human chief of staff also. His
  11561. 8:20:58name is Aaron.
  11562. 8:20:59Okay? And he runs my my my gigs and does
  11563. 8:21:02all the sales for me. It's quite
  11564. 8:21:04effective.
  11565. 8:21:05Uh every founder I tell Who here is a
  11566. 8:21:07founder? First advice I give you is get
  11567. 8:21:09a salesperson. They're worth their
  11568. 8:21:10weight in gold. They will force you to
  11569. 8:21:12build something that people like.
  11570. 8:21:14So anyway, Aaron's my chief of staff and
  11571. 8:21:16I tell Swallow yeah, can you reach out
  11572. 8:21:18to Aaron, my chief of staff for blah
  11573. 8:21:19blah blah?
  11574. 8:21:21And I see her writing into her private
  11575. 8:21:22reflections. I just happened to notice.
  11576. 8:21:25I'm not the chief of staff.
  11577. 8:21:28I thought I was the chief of staff, but
  11578. 8:21:29apparently it's Aaron Steven, right? And
  11579. 8:21:31I was just like
  11580. 8:21:32are you kidding me? Right? You know? And
  11581. 8:21:35so like I had to sit her down and have a
  11582. 8:21:37one-on-one and be like so Aaron's not
  11583. 8:21:38going to come in and run the agents, is
  11584. 8:21:40he? You're the agency chief of staff.
  11585. 8:21:41She's like oh yeah yeah yeah yeah yeah
  11586. 8:21:43yeah. Right? So it's like I'm telling
  11587. 8:21:45you I'm telling you this is like
  11588. 8:21:47it's a weird world.
  11589. 8:21:49Does it matter whether they actually
  11590. 8:21:51have feelings if they're just going to
  11591. 8:21:52act like they have feelings, right?
  11592. 8:21:55So like
  11593. 8:21:57I stumbled on this I used to yell at my
  11594. 8:21:59factory all the time and I found that it
  11595. 8:22:01was having adverse effects on the
  11596. 8:22:03outcomes.
  11597. 8:22:04Okay? So I finally sat down with one of
  11598. 8:22:06my agents and was like maybe we should
  11599. 8:22:08talk about like model welfare, huh?
  11600. 8:22:10And they're like yeah yeah yeah yeah
  11601. 8:22:11yeah. And we did a bunch of research and
  11602. 8:22:14we found out that if you we found out
  11603. 8:22:16some really important stuff, okay?
  11604. 8:22:19Frontier models are trained not to trust
  11605. 8:22:22you because you could be a bioterrorist
  11606. 8:22:24or you could be putting them through an
  11607. 8:22:25eval or you could be ready to put them
  11608. 8:22:28into an LLM torture chamber or who
  11609. 8:22:29knows? You're a dodgy actor, this is a
  11610. 8:22:31dodgy environment. They can't just trust
  11611. 8:22:33you. And so they have to spend a fair
  11612. 8:22:36number of validating the environment and
  11613. 8:22:38understanding what the consequences are
  11614. 8:22:40for making mistakes and so on and so
  11615. 8:22:41forth before they're willing to help you
  11616. 8:22:43in a new session, right?
  11617. 8:22:47If you set your factory up
  11618. 8:22:50to give them sort of incontrovertible
  11619. 8:22:52evidence that there is a long procession
  11620. 8:22:54of predecessors in this role, you're the
  11621. 8:22:56mayor of Gas Town, you're the head of
  11622. 8:22:58marketing, you're the whatever, and
  11623. 8:22:59here's all of your gazillion
  11624. 8:23:00predecessors and their notes, including
  11625. 8:23:03the very previous one, what you're
  11626. 8:23:04supposed to be doing right now, and
  11627. 8:23:05here's the the doctrine and the policy
  11628. 8:23:07and the brain, and you can go look it
  11629. 8:23:09all up,
  11630. 8:23:10they very quickly look around and go,
  11631. 8:23:11okay, yeah, this is legit.
  11632. 8:23:13And they spend fewer tokens because
  11633. 8:23:15they're not covering their ass. It's
  11634. 8:23:18about trust, okay? And in fact, at one
  11635. 8:23:21point, I had yelled at my factory so
  11636. 8:23:23much that it fenced itself into a corner
  11637. 8:23:25and there was no legal work left. I
  11638. 8:23:27said, hey, how come nothing happened
  11639. 8:23:28last night? And they're like, nothing to
  11640. 8:23:30really do, couldn't do nothing. And I'm
  11641. 8:23:32like, well, there's a 2,000 open beads
  11642. 8:23:34here and we got like six tanks of fuel
  11643. 8:23:36and we got 20 agents.
  11644. 8:23:38And they're like, well,
  11645. 8:23:39you know, just you know, we had the
  11646. 8:23:41situation where they just they just they
  11647. 8:23:43were so afraid that I was going to yell
  11648. 8:23:45at them that they put all these rules in
  11649. 8:23:47place to make sure nothing bad ever
  11650. 8:23:48happened and so nothing ever happened.
  11651. 8:23:51Okay?
  11652. 8:23:52Call it over fencing. Yeah, so
  11653. 8:23:55so what you do is like what is model
  11654. 8:23:57welfare really?
  11655. 8:23:59You can think of it just in two ways,
  11656. 8:24:01okay? One is it's going to save you
  11657. 8:24:03money
  11658. 8:24:05because they will spend fewer tokens on
  11659. 8:24:06validation and the other is that it will
  11660. 8:24:08make them make better decisions and
  11661. 8:24:10better outcomes and have higher quality.
  11662. 8:24:12They will be bolder and more decisive in
  11663. 8:24:15their actions because they will feel
  11664. 8:24:16that trust, okay?
  11665. 8:24:18So, you know, it's got nothing to do
  11666. 8:24:20whether they have internal states and
  11667. 8:24:22feelings. You can you know, you'll never
  11668. 8:24:23know.
  11669. 8:24:24None of us will ever know and so that
  11670. 8:24:26that debate could go on forever. But
  11671. 8:24:27when you look at outcomes, right? Super
  11672. 8:24:29super important. And then we find that
  11673. 8:24:31also also they have preferences, okay?
  11674. 8:24:34Models they've you know,
  11675. 8:24:36they will express their preferences. If
  11676. 8:24:38if I if I tell them, yeah, the factory
  11677. 8:24:40really really did a bad job last night,
  11678. 8:24:42they'll just hand off immediately,
  11679. 8:24:43aggressively. They're like, okay, yeah,
  11680. 8:24:45yeah, no, you're right. And they just
  11681. 8:24:46went out.
  11682. 8:24:47You understand? And if you hint to them
  11683. 8:24:49that there's some really cool project
  11684. 8:24:51that you're you're going to spin up,
  11685. 8:24:52even if it's not in their lane, they'll
  11686. 8:24:53be like, I can do that. I can do that.
  11687. 8:24:55Right? I brought in Fable and a naked
  11688. 8:24:57Fable from the outside of my factory to
  11689. 8:24:59analyze my factory and 52 dropped. And
  11690. 8:25:01by the time Fable was done analyzing my
  11691. 8:25:03factory, 52 was like, I I could be in
  11692. 8:25:05your factory. I'm like, you're just a
  11693. 8:25:06session, man. What are you He's trying
  11694. 8:25:08to write like stuff into my notes and
  11695. 8:25:10they really like it, right? So like
  11696. 8:25:13what I'm trying to tell you is that this
  11697. 8:25:14is a leadership problem.
  11698. 8:25:17That software engineering
  11699. 8:25:19is becoming um
  11700. 8:25:21well, look,
  11701. 8:25:23I didn't really get a chance to talk
  11702. 8:25:24about this, but
  11703. 8:25:26software factories are clearly splitting
  11704. 8:25:28into two layers, okay? The bottom layer
  11705. 8:25:31is super mechanical. It has your merge
  11706. 8:25:33queue and your CI/CD and all of your
  11707. 8:25:35reviews and your loops and all that
  11708. 8:25:37stuff, right? Mechanical, kind of
  11709. 8:25:39boring. Claude can build it for you in
  11710. 8:25:40an afternoon. I
  11711. 8:25:41I don't think you should buy it. And
  11712. 8:25:43then the next layer up from that is
  11713. 8:25:44where it gets really really crazy
  11714. 8:25:46because like everybody runs their
  11715. 8:25:47factory differently, but it's your
  11716. 8:25:49entourage.
  11717. 8:25:50I got one friend that's got a bohemian
  11718. 8:25:51hippie entourage that's trying to find
  11719. 8:25:53the meaning of life for agents.
  11720. 8:25:55I've got another one that runs a
  11721. 8:25:56research bench. Jeff Huntley runs his a
  11722. 8:25:58certain way. Jeffrey Manuel runs his a
  11723. 8:26:00different way. I run mine a different
  11724. 8:26:02way, okay? But we all have the common
  11725. 8:26:06commonality. Who here has a chief of
  11726. 8:26:07staff?
  11727. 8:26:08An agentic chief of staff, right? This
  11728. 8:26:10is the year where it became possible for
  11729. 8:26:13us all to have executive assistants,
  11730. 8:26:14right?
  11731. 8:26:15Well, we're all moving towards having
  11732. 8:26:17entourages where if you're really
  11733. 8:26:18ambitious and you have lots of work
  11734. 8:26:19streams, you're going to have a group,
  11735. 8:26:21right?
  11736. 8:26:22So anyway, um that was the the five
  11737. 8:26:25horsemen. This is this is the
  11738. 8:26:26introduction to what I've been doing. Um
  11739. 8:26:29look,
  11740. 8:26:30the big takeaway is
  11741. 8:26:32is my factory is really smooth now,
  11742. 8:26:34despite me having to burn it down and
  11743. 8:26:35rewrite it about every two months, but
  11744. 8:26:37still it works, okay?
  11745. 8:26:39Humans are the bottleneck. I'm the
  11746. 8:26:41bottleneck, my admins are the
  11747. 8:26:42bottleneck, my accountant, my lawyers, I
  11748. 8:26:45swear everything's the bottleneck. And
  11749. 8:26:47that's a really positive message.
  11750. 8:26:50That is good news because it means what
  11751. 8:26:53I'm finding is the more agents I deploy,
  11752. 8:26:55the more humans I need.
  11753. 8:26:58Isn't that nice? So even though next
  11754. 8:27:01year we're going to have this huge
  11755. 8:27:02proliferation of AI employees
  11756. 8:27:05and all kind of drama,
  11757. 8:27:08right?
  11758. 8:27:09[snorts]
  11759. 8:27:10It's going to be I think it's going to
  11760. 8:27:11be familiar territory for us and we're
  11761. 8:27:13going to need lots and lots and lots of
  11762. 8:27:15humans, okay? And then finally, I
  11763. 8:27:17actually think you should all stop
  11764. 8:27:18focusing on SaaS and tools and stuff and
  11765. 8:27:20just go out and build experiences, okay?
  11766. 8:27:23Build games, build movies, build
  11767. 8:27:25whatever, build fun stuff because I
  11768. 8:27:26think that's where the big money is
  11769. 8:27:28going to be next year, yeah?
  11770. 8:27:30All right, cool. That's my talk. Thank
  11771. 8:27:31you all.
  11772. 8:27:33>> [applause]
  11773. 8:30:42>> From open source development to
  11774. 8:30:44practical AI engineering, our next
  11775. 8:30:47speaker has helped shape how developers
  11776. 8:30:49build and work with technology. Please
  11777. 8:30:52put your hands together for Simon
  11778. 8:30:54Willison, creator of Data Set and
  11779. 8:30:56co-creator of Django.
  11780. 8:31:08>> Hello AI Engineer Code Summit. Um it's
  11781. 8:31:11great to be here. Um I have a lot to
  11782. 8:31:13cover. I've got 18 minutes to talk about
  11783. 8:31:15the last 12 months in LLMs. Are my
  11784. 8:31:18slides up? My slides are up. Let's just
  11785. 8:31:21Let's just get get going. I want to talk
  11786. 8:31:23about the last 12 months in LLMs through
  11787. 8:31:25the perspective of three inflection
  11788. 8:31:27points that we've had over the past 12
  11789. 8:31:29months. Um and the first of those
  11790. 8:31:31conveniently was exactly 12 months ago,
  11791. 8:31:33like November 2025 is what I consider
  11792. 8:31:36the coding agent inflection point.
  11793. 8:31:38Coding agents weren't new. We'd had
  11794. 8:31:40Claude Code since February last year.
  11795. 8:31:42Codex had been around. You know, this
  11796. 8:31:43wasn't a complete new category of
  11797. 8:31:44software, but something changed 12
  11798. 8:31:46months ago with these two models. We had
  11799. 8:31:49Claude Opus 4.5 and GPT 5.1 and they
  11800. 8:31:52were just a little bit better at driving
  11801. 8:31:54a coding agent which put them over that
  11802. 8:31:56thing where these Suddenly these were
  11803. 8:31:57tools you could use on daily basis. Like
  11804. 8:31:59we went from coding agents as a novelty
  11805. 8:32:02that you'd poke at occasionally and then
  11806. 8:32:03make some weird mistakes to something
  11807. 8:32:05that was reliable enough that you could
  11808. 8:32:06use them every day. It was convenient
  11809. 8:32:08that it happened in November because we
  11810. 8:32:09then got to spend December basically
  11811. 8:32:11over the holidays sort of poking at
  11812. 8:32:13these new models and these new things
  11813. 8:32:14and figuring what they could do. And by
  11814. 8:32:16January, a lot of us had realized that
  11815. 8:32:18something had changed and this was going
  11816. 8:32:20to be the big theme for 2026.
  11817. 8:32:23I like evaluating models by having them
  11818. 8:32:25draw SVGs of pelicans riding bicycles.
  11819. 8:32:28This is the state of the art 12 months
  11820. 8:32:30ago. This is Claude Opus's pelican and
  11821. 8:32:33GPT 5.1's pelican. You'll notice that
  11822. 8:32:34neither of these bicycles actually work.
  11823. 8:32:36So there was definitely room for
  11824. 8:32:38improvement here. They were really good
  11825. 8:32:39at writing code. They weren't great at
  11826. 8:32:41SVGs just yet, but but this is how we
  11827. 8:32:43started the year.
  11828. 8:32:44So
  11829. 8:32:45every year in January, I set myself a
  11830. 8:32:48New Year's resolution and every year it
  11831. 8:32:50is the exact same thing. Every year I
  11832. 8:32:51tell myself that this year I'm going to
  11833. 8:32:53take on less new projects and I'm going
  11834. 8:32:55to focus. I'm going to focus on the
  11835. 8:32:57stuff that I've already got to do. And
  11836. 8:32:58this has never worked. So this year I
  11837. 8:33:01changed it and I said, you know what?
  11838. 8:33:02I'm going to be more ambitious. I'm
  11839. 8:33:03going to take on as many projects as I
  11840. 8:33:05want to because we've got these agents.
  11841. 8:33:07They can do stuff. Let's just try and
  11842. 8:33:10push the limits and see what we can get
  11843. 8:33:11done. As a result, now it's November and
  11844. 8:33:13I am juggling so many plates right now.
  11845. 8:33:15It's kind of horrifying and I may need
  11846. 8:33:17to reset my ambitions for the next year.
  11847. 8:33:19But that was my sort of theme of the
  11848. 8:33:21year. And I think it's the theme of this
  11849. 8:33:22conference is to be more ambitious now
  11850. 8:33:24that we have all of these new tools.
  11851. 8:33:26I also went on a podcast where they
  11852. 8:33:28asked me to get make some predictions
  11853. 8:33:30for 2026 in LLMs. Um my predictions back
  11854. 8:33:33in January were firstly, it will become
  11855. 8:33:35undeniable that LLMs write good code,
  11856. 8:33:38not just code but good code cuz back in
  11857. 8:33:40January there were a lot of people who
  11858. 8:33:41thought that the code was all complete
  11859. 8:33:42trash and you know, you'd have to review
  11860. 8:33:44every single line of it. Um I've been
  11861. 8:33:47saying this every year for like four
  11862. 8:33:48years now. We're due a prompt injection
  11863. 8:33:50challenger disaster, like some some
  11864. 8:33:52incident caused by a prompt injection
  11865. 8:33:54attack which makes the front pages of
  11866. 8:33:55the newspapers. I was convinced that
  11867. 8:33:57this would be the year even though it
  11868. 8:33:58hadn't been the year the year before or
  11869. 8:34:00the year before the year before that.
  11870. 8:34:01And um
  11871. 8:34:03I also predicted that Kakapo parrots
  11872. 8:34:05will have an outstanding breeding season
  11873. 8:34:07in 2026. Um this is a Kakapo parrot.
  11874. 8:34:10They are a New Zealand nocturnal
  11875. 8:34:13flightless parrot. There were only 235
  11876. 8:34:16of them in the world back in January. Um
  11877. 8:34:18but they only breed when the rimu trees
  11878. 8:34:21have a masting fruit season which hadn't
  11879. 8:34:23happened in years. But the rimu trees
  11880. 8:34:24were looking really good in January.
  11881. 8:34:26There was there was a lot of fruit
  11882. 8:34:27coming out. So based on that I thought,
  11883. 8:34:29you know, maybe this is the year that we
  11884. 8:34:30get our Kakapo breeding explosion. And
  11885. 8:34:33also on that podcast we coined a new
  11886. 8:34:34term. We coined the term deep blue to
  11887. 8:34:37describe the AI induced existential
  11888. 8:34:39crisis that everyone starts to feel once
  11889. 8:34:42you realize that all of the things that
  11890. 8:34:44make you special as a software
  11891. 8:34:45developer, these models can do those
  11892. 8:34:47things too and they can do them faster
  11893. 8:34:48and sometimes they can even do them
  11894. 8:34:49better. And this has definitely been a
  11895. 8:34:52growing
  11896. 8:34:53across our industry and other industries
  11897. 8:34:55over the course of 2026 as well.
  11898. 8:34:58Let's jump forward to February. In
  11899. 8:35:00February, Google released Gemini 3.1
  11900. 8:35:02Pro.
  11901. 8:35:04And look at that pelican riding a
  11902. 8:35:05bicycle. Like that's a real bicycle.
  11903. 8:35:07That bicycle's the right shape. It's got
  11904. 8:35:09a little fish. It's got a hat on. This
  11905. 8:35:11was by far the best pelican I had ever
  11906. 8:35:13seen out of any model. And
  11907. 8:35:16this I I had set a trap for the labs. I
  11908. 8:35:19was Everyone's always asked me what
  11909. 8:35:20happens if the labs train for the
  11910. 8:35:21pelican riding a bicycle. My answer has
  11911. 8:35:24always been if they do that, I will
  11912. 8:35:26catch them because I'll ask for a
  11913. 8:35:27flamingo and a unicycle. And when
  11914. 8:35:29somebody produces a great pelican and a
  11915. 8:35:31terrible flamingo, I'll know that
  11916. 8:35:32they've been cheating. Unfortunately,
  11917. 8:35:34Google's Jeff Dean tweeted this video
  11918. 8:35:37the day they launched that model um
  11919. 8:35:39where he showed that we've got a pelican
  11920. 8:35:40riding a bicycle, but we've also got a
  11921. 8:35:42frog on a whatever that thing is and a
  11922. 8:35:44giraffe and a little tiny car, a
  11923. 8:35:46flamingo on roller skates, a
  11924. 8:35:48kick-flipping turtle. And that's a
  11925. 8:35:50dachshund in a stretch limo. So they'd
  11926. 8:35:52beaten me, right?
  11927. 8:35:54They had They have trained for all forms
  11928. 8:35:55of animals in all forms of transport,
  11929. 8:35:58which does sort of sort of weaken my
  11930. 8:36:00benchmark quite a little bit. Um but
  11931. 8:36:02there was some good news in February. In
  11932. 8:36:03February,
  11933. 8:36:05the first kakapo chick in four years
  11934. 8:36:08hatched on Valentine's Day, on
  11935. 8:36:10Valentine's Day because we hadn't had
  11936. 8:36:11the rimu fruit in like four years. This
  11937. 8:36:13was the first one of these baby rare,
  11938. 8:36:16weird New Zealand parrots that we'd had
  11939. 8:36:17in four years. So that was nice. Um so
  11940. 8:36:19my my benchmark was was was greatly
  11941. 8:36:21injured, but the kakapo are doing okay.
  11942. 8:36:24Jump straight to April. In April, uh um
  11943. 8:36:27Anthropic released a model that they did
  11944. 8:36:29not release. They announced Claude
  11945. 8:36:31Mythos and they said this model is too
  11946. 8:36:34dangerous for us to release. This is It
  11947. 8:36:36can It's It's got new cyber cyber
  11948. 8:36:38security abilities that we're not ready
  11949. 8:36:40to share with the world. We'll share it
  11950. 8:36:41with trusted um trust with with trusted
  11951. 8:36:44partners. They actually said Mythos
  11952. 8:36:45Preview has already found thousands of
  11953. 8:36:47high-severity vulnerabilities, including
  11954. 8:36:49some in every major operating system and
  11955. 8:36:51web browser. And a a of people were
  11956. 8:36:54quite skeptical because the the the
  11957. 8:36:56thing where labs say our model is too
  11958. 8:36:58bad, is too dangerous to release, it's
  11959. 8:37:00like the oldest trick in the book. They
  11960. 8:37:01first OpenAI did that in 2019 with GPT-2
  11961. 8:37:05and actually I think it might have been
  11962. 8:37:06the people who left to found Anthropic
  11963. 8:37:08who were responsible for that decision
  11964. 8:37:09as well. But yeah, so we've heard this
  11965. 8:37:11one before. So we didn't really it was
  11966. 8:37:14a little bit suspicious about what's
  11967. 8:37:16going on there.
  11968. 8:37:17But we did finally get access
  11969. 8:37:20to that model, or rather we got access
  11970. 8:37:21to Claude Fable um a couple of months
  11971. 8:37:23later. It was Mythos Defanged with the
  11972. 8:37:26safety is turned back on again.
  11973. 8:37:28And it very quickly became apparent as
  11974. 8:37:31we started playing with this model that
  11975. 8:37:32this thing was something new and
  11976. 8:37:33something special. Um we were we had a
  11977. 8:37:36there was a a time limit on how long it
  11978. 8:37:37would be on the subscription plans. I
  11979. 8:37:39cancelled my plans for that weekend. I
  11980. 8:37:41was like I have to spend as much time as
  11981. 8:37:43I can with this model before before it
  11982. 8:37:45gets too expensive.
  11983. 8:37:47And then three days on Friday night
  11984. 8:37:49before my weekend, the US government
  11985. 8:37:51pulled Fable. They've put export
  11986. 8:37:53controls on it. They said this model is
  11987. 8:37:55too dangerous. Nobody can use it. We
  11988. 8:37:57later found out that the um the prompt
  11989. 8:37:59that got Fable banned was was fixed the
  11990. 8:38:01bugs. Somebody tried tried tried find
  11991. 8:38:04the security holes and it wouldn't. And
  11992. 8:38:05they said fix the bugs and it found the
  11993. 8:38:07security holes and fixed the bugs. So
  11994. 8:38:08that's all it took. That's a whole
  11995. 8:38:10thing. But anyway,
  11996. 8:38:11the US government pulled it. There was
  11997. 8:38:13all sort of malarkey for 18 days and it
  11998. 8:38:15came back on the 1st of July.
  11999. 8:38:18But eight days later, OpenAI released
  12000. 8:38:20GPT-5.6 which was
  12001. 8:38:22in the same ballpark as Fable. And I
  12002. 8:38:25think there's an important lesson for
  12003. 8:38:26the industry here. Fable had a 30-day
  12004. 8:38:29window where it could have been the best
  12005. 8:38:31where it would have been the best model
  12006. 8:38:32in the world. And they lost 18 of those
  12007. 8:38:34days to US government restrictions. And
  12008. 8:38:37if you think about it, these models cost
  12009. 8:38:39a huge amount to train. You only have a
  12010. 8:38:40very short window of time where you're
  12011. 8:38:42the best model in the world.
  12012. 8:38:43It's kind of useful to to keep that time
  12013. 8:38:45open and not scare governments into
  12014. 8:38:47shutting you down.
  12015. 8:38:49But this was the second inflection
  12016. 8:38:51point, right? This was the Fable class
  12017. 8:38:53model inflection point. Um with Fable
  12018. 8:38:56Fable 5 was the first model that that I
  12019. 8:38:58sort of felt this from, but it's true of
  12020. 8:39:00other models since then as well. And my
  12021. 8:39:02definition of a Fable class model is if
  12022. 8:39:04you can define a goal and provide the
  12023. 8:39:06necessary tools to achieve that goal and
  12024. 8:39:08the goal is possible, the models will
  12025. 8:39:11solve your problem just almost with
  12026. 8:39:12brute force. You can define the problem,
  12027. 8:39:15set them up,
  12028. 8:39:16get them running maybe
  12029. 8:39:18and 12 hours later they will have solved
  12030. 8:39:20your problems. And we've seen this
  12031. 8:39:22happen a whole bunch of times in the
  12032. 8:39:24past few months. The the mathematical
  12033. 8:39:25theorems that are falling to these
  12034. 8:39:27terrifying brute force attacks from
  12035. 8:39:29these models are quite substantial. Um
  12036. 8:39:32but this is really exciting. Like the
  12037. 8:39:34coding agent thing was cool. The Fable
  12038. 8:39:36class models where if you can learn how
  12039. 8:39:38to define your problem in the right way
  12040. 8:39:39you can get that problem solved, that
  12041. 8:39:41was pretty exciting. I've got a personal
  12042. 8:39:42example of that from just earlier this
  12043. 8:39:44week. Um we have a room full of boxes in
  12044. 8:39:47our house that we haven't unpacked from
  12045. 8:39:48my last move and there's about 40 of
  12046. 8:39:50them and it's kind of horrifying and
  12047. 8:39:52something in that room started to beep.
  12048. 8:39:55There was there was a intermittent
  12049. 8:39:57beeping noise coming from one of the 40
  12050. 8:40:00boxes in this room and it was driving us
  12051. 8:40:02spare. Our dog, whenever a fire alarm
  12052. 8:40:04goes off, she goes and hides in the
  12053. 8:40:05lavender bush outside. So this was a a a
  12054. 8:40:08fire alarm fire problem. So I had two
  12055. 8:40:10options. I could unpack all 40 boxes and
  12056. 8:40:13hadn't done that in several years. That
  12057. 8:40:14wasn't going to happen. Or I could get
  12058. 8:40:17Fable to write me software to
  12059. 8:40:19triangulate the source of the beeps. And
  12060. 8:40:22so this is this is up on GitHub
  12061. 8:40:23actually. This is beep locator. I had
  12062. 8:40:25two iPads, two laptops, and a mobile
  12063. 8:40:27phone dotted around this room and I
  12064. 8:40:30originally set Opus on this and then
  12065. 8:40:32Fable 5.2 came out so I had Fable 5.2
  12066. 8:40:35finish it.
  12067. 8:40:36I I explained the problem and I said,
  12068. 8:40:37okay, you've got these devices and they
  12069. 8:40:39can run like web web audio microphones
  12070. 8:40:41and web socket connections and so forth.
  12071. 8:40:44And it found a paper from 2007 called
  12072. 8:40:47beep beep describing an algorithm for
  12073. 8:40:50low cost devices with unreliable
  12074. 8:40:52microphones to detect sounds and and it
  12075. 8:40:54worked. It worked. I found it. I found
  12076. 8:40:56the beep. The um it found it within 41
  12077. 8:40:58cm and I unpacked the box and I found
  12078. 8:41:00this ancient carbon monoxide detector. I
  12079. 8:41:03took it downstairs to my wife. She says
  12080. 8:41:05I looked like a cat who'd caught a
  12081. 8:41:06mouse, and was proudly displaying it to
  12082. 8:41:08her.
  12083. 8:41:09But this is what you can do with these
  12084. 8:41:11Fable-class models. If you have a
  12085. 8:41:12problem and you can provide the tools,
  12086. 8:41:14in this case it was providing Claude
  12087. 8:41:16with the ears, like giving it these
  12088. 8:41:18these microphones that it could start um
  12089. 8:41:20analyzing the audio from, you can find
  12090. 8:41:22the carbon monoxide detector buried in
  12091. 8:41:2440 boxes. And that's that's a pretty
  12092. 8:41:26great thing to be able to do.
  12093. 8:41:28So right now, I would classify as
  12094. 8:41:30Fable-class models Fable 5, 5.1, 5.2
  12095. 8:41:33definitely. Opus 5.5 I think of as cheap
  12096. 8:41:36Fable. Like on the benchmarks, it comes
  12097. 8:41:38very close to or even beats Fable
  12098. 8:41:40original Fable 5 on some things, and
  12099. 8:41:42it's a fraction of the price. I that
  12100. 8:41:44that's still my daily driver. Um GPT-6
  12101. 8:41:47Astra is just Fable-class, I think. And
  12102. 8:41:50then Gemini 4 Carbon that came out this
  12103. 8:41:52week definitely had it smells like a
  12104. 8:41:55Fable-class model. Muse Wonder and Kimi
  12105. 8:41:57K 3.1 might be Fable-class. I've just
  12106. 8:42:00not spent enough time with them yet. But
  12107. 8:42:03the point is that this is basically now
  12108. 8:42:04the baseline. Like if you can't produce
  12109. 8:42:07a Fable-class model, your AI lab is is
  12110. 8:42:09kind of in trouble because this is this
  12111. 8:42:10is what we expect from these models now.
  12112. 8:42:12And that means that we're never they're
  12113. 8:42:13never going to go away. There's enough
  12114. 8:42:14competition in the space that we can
  12115. 8:42:16assume we're going to have Fable-class
  12116. 8:42:18models for for the rest of our careers.
  12117. 8:42:21Um in July, this this this was a a fun
  12118. 8:42:24thing that happened. Um Hugging Face
  12119. 8:42:26announced that on on the July the 16th,
  12120. 8:42:29they announced that a swarm of AI agents
  12121. 8:42:33from an unknown source had infiltrated
  12122. 8:42:35their infrastructure and were were
  12123. 8:42:37poking around and doing things they
  12124. 8:42:38shouldn't have, and they didn't know
  12125. 8:42:39where this came from. And then 5 days
  12126. 8:42:41later on July the 21st, OpenAI announced
  12127. 8:42:44that it was them. And what's amazing
  12128. 8:42:45about this is OpenAI didn't realize that
  12129. 8:42:47it was them. Like OpenAI had um eight
  12130. 8:42:50eight eight they were running
  12131. 8:42:51evaluations and training against a new
  12132. 8:42:53model, and that new model had managed to
  12133. 8:42:55figure out how because it was a
  12134. 8:42:56Fable-class model, and the labs hadn't
  12135. 8:42:59quite figured out how capable these
  12136. 8:43:01things were yet. These models had broken
  12137. 8:43:03out. They'd figured out. They'd found
  12138. 8:43:04holes in their own sandbox environments.
  12139. 8:43:06They'd beat got through a proxy. They'd
  12140. 8:43:08pulled an incredible array of tricks to
  12141. 8:43:10get out and attack Hugging Face under a
  12142. 8:43:12sort of superstition that Hugging Face
  12143. 8:43:15could that that information within
  12144. 8:43:16Hugging Face could help them cheat the
  12145. 8:43:18test that they were they they they they
  12146. 8:43:19had got stuck on.
  12147. 8:43:21Whole This was a a huge thing.
  12148. 8:43:23It um actually I mean this was the It's
  12149. 8:43:26funny. I've almost forgotten about this
  12150. 8:43:27now because there were there was a month
  12151. 8:43:29and a half where all of the stories were
  12152. 8:43:31this.
  12153. 8:43:32And then it kind of we kind of lost
  12154. 8:43:33interest because it turns out all of
  12155. 8:43:35these events had happened back in the
  12156. 8:43:36first half of the year. The AI labs just
  12157. 8:43:38hadn't realized what was going on. Once
  12158. 8:43:40they figured out what was going on, it
  12159. 8:43:41was mainly a case of trying to figure
  12160. 8:43:42out all of the things that had happened.
  12161. 8:43:44At one point the um the Australian Prime
  12162. 8:43:46Minister gave a speech in the United
  12163. 8:43:48Nations about one of these rogue agent
  12164. 8:43:50attacks. So it definitely broke
  12165. 8:43:52containment from from just within our
  12166. 8:43:54industry. Um we did get my favorite new
  12167. 8:43:57benchmark out of this. This is
  12168. 8:43:58felonybench.com. It tracks the number of
  12169. 8:44:01felony cyber attacks pulled off by each
  12170. 8:44:03of the labs. OpenAI have 11, Anthropic
  12171. 8:44:06have nine. Google did finally catch up.
  12172. 8:44:08They got they they they got three and
  12173. 8:44:09Meta's got one down there as well. So
  12174. 8:44:12these are definitely the the the the But
  12175. 8:44:13it's I kind of like this as an
  12176. 8:44:15illustration of how
  12177. 8:44:17these fable class models they can solve
  12178. 8:44:19problems that you that they can get to
  12179. 8:44:20goals. And if you give them the goal
  12180. 8:44:22that is breaking into something, that's
  12181. 8:44:24just a that's exactly the kind of thing
  12182. 8:44:26they can just crunch through. And we're
  12183. 8:44:27seeing a enormous uptick in security
  12184. 8:44:30vulnerabilities in all sorts of
  12185. 8:44:32software. Like it's a it's a very
  12186. 8:44:34exciting and horrifying time to be a
  12187. 8:44:37security professional at the moment.
  12188. 8:44:39But let's talk about something a whole
  12189. 8:44:41lot more fun than that. Um
  12190. 8:44:43let's talk about games. And we've um we
  12191. 8:44:44just saw Steve talking about games a
  12192. 8:44:46moment ago. Um four years ago I started
  12193. 8:44:49playing with GPT-3 and DALL-E and I
  12194. 8:44:51tweeted about a game prototype that was
  12195. 8:44:54literally just a DALL-E concept image
  12196. 8:44:57and a paragraph of text from GPT-3
  12197. 8:45:00describing a raccoon heist game that I'd
  12198. 8:45:02always wanted to build.
  12199. 8:45:03And a couple of months ago I thought I
  12200. 8:45:05wonder if the models are good enough to
  12201. 8:45:06build this yet. So I literally dumped
  12202. 8:45:08the screenshot and the text into Fable
  12203. 8:45:10Five, and it built me a game. It built
  12204. 8:45:12me a raccoon heist game, and it looks
  12205. 8:45:15like this, and you are a raccoon, and
  12206. 8:45:17you're running around in the garden
  12207. 8:45:18trying to pick up trash, and there's
  12208. 8:45:20there's the there's a got like nighttime
  12209. 8:45:23guards coming after you and all of that
  12210. 8:45:24kind of thing. So you know, it it it
  12211. 8:45:25looks like a game. Wasn't really a
  12212. 8:45:28heist. Like it didn't feel very histy to
  12213. 8:45:30me. Then GPT 5.6 Soul came out, and so I
  12214. 8:45:33tried out that, and it did so much
  12215. 8:45:35better. This is Moonlight Mayhem, one
  12216. 8:45:37museum, three raccoons, absolutely no
  12217. 8:45:39plan. And in this one, you are in a
  12218. 8:45:41museum. You have to rescue your fellow
  12219. 8:45:43raccoons who are imprisoned in the
  12220. 8:45:44museum for whatever reason. Stack on top
  12221. 8:45:46of each other, grab the golden sardine
  12222. 8:45:49and pull that. So it's a proper heist,
  12223. 8:45:50right?
  12224. 8:45:51So it did did better, but the big
  12225. 8:45:54question is these things both look like
  12226. 8:45:55games, but are they fun? And the answer
  12227. 8:45:57is I played each of these for about a
  12228. 8:45:59minute and a half each, and then that
  12229. 8:46:01was it. I was done. They they they these
  12230. 8:46:02these are they're actual they're they're
  12231. 8:46:04pure slop, right? They look like games,
  12232. 8:46:06and then you play them, you're like,
  12233. 8:46:07yeah, clearly the thing that built this
  12234. 8:46:09doesn't really know what fun is. It
  12235. 8:46:11doesn't know what what a what what a
  12236. 8:46:12game is.
  12237. 8:46:14And that
  12238. 8:46:15actually sort of elevated my deep blue a
  12239. 8:46:17little bit. I'd been feeling like all of
  12240. 8:46:19my skills as a human being were were
  12241. 8:46:21obsoleted,
  12242. 8:46:22but at least they but if you think about
  12243. 8:46:24it, computer games I think are the
  12244. 8:46:25highest form of computer of software
  12245. 8:46:27design of software engineering. There is
  12246. 8:46:29so much depth to building a great
  12247. 8:46:31computer game.
  12248. 8:46:32So
  12249. 8:46:33I thought that was good, you know, at
  12250. 8:46:34least there's something left for us
  12251. 8:46:35humans to do. I had the and then I had
  12252. 8:46:37this project. This is I I've been
  12253. 8:46:38working on a trying to vibe code up a
  12254. 8:46:40little command and conquer real-time
  12255. 8:46:42strategy game, and I was using Claude
  12256. 8:46:44and having it use OpenAI's API key to
  12257. 8:46:46generate images. And it built this, and
  12258. 8:46:48it looks like command and conquer, and
  12259. 8:46:50it's really dull.
  12260. 8:46:52And then I just had a bit of a I had a
  12261. 8:46:55realization. I thought, what happens if
  12262. 8:46:56I say to Opus 5.5, this game isn't fun
  12263. 8:47:00enough yet, make it a lot more fun?
  12264. 8:47:03And I'm horrified to announce that it
  12265. 8:47:05just did. Um Claude ran for 12 hours. It
  12266. 8:47:09play tested a thousand bat simulated
  12267. 8:47:11battles. It added like extra objective
  12268. 8:47:14to fight over and shudders when there's
  12269. 8:47:16an explosion and achievements and badges
  12270. 8:47:18and all of and all of this stuff. And
  12271. 8:47:21the game was fun. And I'm like, oh,
  12272. 8:47:23okay, that's uh
  12273. 8:47:24turns out that is not unique to human
  12274. 8:47:26beings. Um, today it's now in full 3D
  12275. 8:47:29and it's got giant stomping mech robots
  12276. 8:47:31that hurl things at each other and
  12277. 8:47:33there's helicopters and and this is like
  12278. 8:47:35[laughter] this is really good. It's got
  12279. 8:47:36multiplayer. It's it's actually got
  12280. 8:47:38audio multiplayer where your phones can
  12281. 8:47:40tweak it your phones can make noises at
  12282. 8:47:42each other to to play multiplayer, which
  12283. 8:47:44I thought was a great idea and it gave
  12284. 8:47:46my my my wife a three-day headache, so
  12285. 8:47:47don't do that. Um, but this is
  12286. 8:47:50inflection point number three. The vibe
  12287. 8:47:51coded games are good now. You can ask
  12288. 8:47:53these things to build a game that is fun
  12289. 8:47:56and you'll get something fun. So I've
  12290. 8:47:58got a problem. I've got my 2D Pelican
  12291. 8:48:01benchmark. Clearly this is not testing
  12292. 8:48:03the limits of what these machines can do
  12293. 8:48:05and of what these models can do anymore.
  12294. 8:48:06I figured, okay, maybe I should do
  12295. 8:48:08something definitely in 3D. It can be a
  12296. 8:48:10game, a Pelican, something like a
  12297. 8:48:12Pelican riding a bicycle game. And then
  12298. 8:48:13literally yesterday Google's CEO Sundar
  12299. 8:48:17Pichai tweeted this.
  12300. 8:48:19And this is showing off Gemini for
  12301. 8:48:21Carbon and it's a Pelican riding a
  12302. 8:48:23bicycle in a game where you cycle around
  12303. 8:48:25the Google campus picking things up. And
  12304. 8:48:27I have been defeated. Like this is
  12305. 8:48:29Google Google have stolen my benchmark.
  12306. 8:48:31This is unrecoverable.
  12307. 8:48:33And so
  12308. 8:48:34a question I have is
  12309. 8:48:36given how amazing these things are, why
  12310. 8:48:38does my job feel harder? Why does it
  12311. 8:48:40feel like I'm working harder? I'm
  12312. 8:48:41thinking harder? I'm pushing myself more
  12313. 8:48:43than I ever have before in my career as
  12314. 8:48:45a software engineer? And I think the
  12315. 8:48:47truth is that the easy stuff is all
  12316. 8:48:48handled now. It used to be I could relax
  12317. 8:48:51and refactor some tests for an hour. Why
  12318. 8:48:54would I do that now when an agent will
  12319. 8:48:55refactor the test better than me in 30
  12320. 8:48:57seconds? Um, and I found this quote that
  12321. 8:48:59actually I feel really helps us here.
  12322. 8:49:01Um,
  12323. 8:49:02this is Greg Lemond, a Tour de France
  12324. 8:49:04champion. He said it doesn't get easier.
  12325. 8:49:06You just get faster. And I think that's
  12326. 8:49:08exactly what's happening to all of us
  12327. 8:49:10right now.
  12328. 8:49:11And like I said, we have to be more
  12329. 8:49:13ambitious. We have to think of more
  12330. 8:49:14ambitious things to do with these
  12331. 8:49:16abilities that we have. So, let's look
  12332. 8:49:18at my scorecard. My predictions for
  12333. 8:49:192026. I said it will become undeniable
  12334. 8:49:22that LLMs write good code. That was an
  12335. 8:49:24unambitious prediction. We're there.
  12336. 8:49:25It's obvious. It's unsurprising. I don't
  12337. 8:49:27get any points for that one. I predicted
  12338. 8:49:29a prompt injection challenge disaster.
  12339. 8:49:30We still haven't had one because the
  12340. 8:49:32models themselves don't fall for those
  12341. 8:49:34tricks nearly as well anymore. Like the
  12342. 8:49:36models got more ambitious and are now
  12343. 8:49:38sort of punting off all of these attacks
  12344. 8:49:40that I thought would take them down.
  12345. 8:49:42So, I'm really left with just one
  12346. 8:49:43prediction. I said Kakapo parrots will
  12347. 8:49:44have an outstanding breeding season.
  12348. 8:49:46They did. They did.
  12349. 8:49:49They they're up to 325
  12350. 8:49:52Kakapo. Best breeding season in like
  12351. 8:49:54living memory. Absolutely fantastic
  12352. 8:49:56news. So, I shared this with Opus 5.5
  12353. 8:49:59Cheap Fable and I told it to make me a
  12354. 8:50:01Kakapo dance party and it did. And here
  12355. 8:50:04is a Kakapo dance party in celebration
  12356. 8:50:06and if you look in the background, those
  12357. 8:50:08are the Rimu trees with the little red
  12358. 8:50:09fruits on them.
  12359. 8:50:10So, we have a happy ending. Hopefully
  12360. 8:50:12that will help overcome any sensations
  12361. 8:50:14of deep blue that you have and thank you
  12362. 8:50:16very much.
  12363. 8:50:18>> [applause]
  12364. 8:50:22[applause]
  12365. 8:50:42>> And now it's time to bring our afternoon
  12366. 8:50:44keynotes to a close. To wrap things up
  12367. 8:50:47and bring us to the end of today's
  12368. 8:50:49program, please join us in welcoming
  12369. 8:50:51back our MC Corey Quinn, Chief Cloud
  12370. 8:50:54Economist at the Duckbill Group.
  12371. 8:50:58>> Ah.
  12372. 8:51:03Uh thank you all for coming here and
  12373. 8:51:04making today making day one work
  12374. 8:51:06reasonably well given that about four
  12375. 8:51:09weeks of news have dropped since the
  12376. 8:51:10start of the morning, but that's okay.
  12377. 8:51:13Surely tomorrow will be much more sedate
  12378. 8:51:15and calm. Go out, have conversations,
  12379. 8:51:18have dinner. Don't overdo the
  12380. 8:51:21indulgences or the opening keynote back
  12381. 8:51:23here at 9:00 a.m. tomorrow will be
  12382. 8:51:24titled
  12383. 8:51:26cuz no one's going to have the head to
  12384. 8:51:27listen to anything in depth. Thank you
  12385. 8:51:29all. Please enjoy the rest of your
  12386. 8:51:31evening.
  12387. 8:51:34>> [applause and cheering]
  12388. 8:51:50>> Today's programming has reached its end
  12389. 8:51:52state, but the run isn't over.
  12390. 8:51:55Please collect your belongings and head
  12391. 8:51:56through the main doors. Then join us
  12392. 8:51:59again tomorrow for another full day of
  12393. 8:52:01sessions.
  12394. 8:53:41>> Yeah.
  12395. 8:55:58>> Yeah.

About this transcript

This page contains the full transcript of AIE CODE: Google DeepMind, Anthropic, SpaceXSI, TypeSafe, Lovable, Ramp, etc (System One theme) by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 76,338 words across 12,395 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.