YouTube2Text

"Where is all the automation?" — Diogo Almeida, TypeSafe AI — Transcript

by AI Engineer · 4,767 words · 692 segments · language en · Watch on YouTube

Full transcript

  1. 0:01[music]
  2. 0:12>> I'm Diego Almeida, and this talk is I
  3. 0:15Love You So Jev Much. Um,
  4. 0:18Glass Animals reference. We're not just
  5. 0:20weird. Um,
  6. 0:22like I said, I'm Diego Almeida. I'm
  7. 0:24known as the Jev guy, sometimes Jev pink
  8. 0:27man. Um, also co-created ChatGPT, RLHF,
  9. 0:31and InstructGPT, but I assume that for
  10. 0:33the crowd today, y'all know who I am, at
  11. 0:36least mostly.
  12. 0:37[snorts] Um,
  13. 0:38I have like a bunch of AI-specific stuff
  14. 0:41I want to get to, but I like going into
  15. 0:42my favorite topics in AI. First one is
  16. 0:46what the [ __ ] is actually going on and
  17. 0:48where is all the automation? If you've
  18. 0:50heard me say this before, I'm sorry, but
  19. 0:52it's such an important thing to beat
  20. 0:54into the field cuz people still do not
  21. 0:56get it, and I need to yell this until
  22. 0:58everyone really understands what the
  23. 1:00[ __ ] is going on. And if you're in AI,
  24. 1:02you should have a really good answer
  25. 1:03about why AI can do so many awesome
  26. 1:06things and yet still be so useless for a
  27. 1:08bunch of other ones.
  28. 1:10My relationship to ChatGPT is somewhat
  29. 1:12unique, one of the few people who hates
  30. 1:14on ChatGPT, who made it. I don't hate
  31. 1:18ChatGPT, to be clear. I love using
  32. 1:20agents. I love using coding agents. I
  33. 1:22love chatbots. They're just quite
  34. 1:24limited. And in particular, I think it's
  35. 1:26a little tragic how much they've
  36. 1:28bifurcated the field into overpromise,
  37. 1:30underdeliver.
  38. 1:31Really, really terrible stuff for the
  39. 1:32world in my opinion.
  40. 1:34And I like to talk about like the two
  41. 1:36cults in AI and how we can resolve the
  42. 1:39cognitive dissonance that everyone I
  43. 1:41think should be having unless they have
  44. 1:43a very good answer.
  45. 1:45There's cult one, which is you know, the
  46. 1:46cult of like EAs. They look at all the
  47. 1:49benchmarks and they say like not only
  48. 1:50are LMs surpassing human level on
  49. 1:53basically every benchmark, they have
  50. 1:55been for two years. We've been
  51. 1:56announcing AGI three months away for two
  52. 1:59years now.
  53. 2:00Even since like the '03 days. Do people
  54. 2:01remember that? No one even care-cares
  55. 2:03about that model anymore and it it was
  56. 2:05that that was like peak internal to
  57. 2:07OpenAI AGI hype.
  58. 2:08And on the other hand, you look at the
  59. 2:10non-benchmarks and you look at the
  60. 2:12reality and you see the rest of the
  61. 2:14world, especially if you guys go outside
  62. 2:16and leave San Francisco, a lot of the
  63. 2:17world is like really unchanged despite
  64. 2:20AI being so [ __ ] cool. And I think AI
  65. 2:23is transformative. It's it's a rare
  66. 2:24point of view to actually think AI is
  67. 2:26cool and transformative and also like
  68. 2:29underwhelming the world right now.
  69. 2:32And the tricky part is this is the same
  70. 2:33field and we are all seeing the same
  71. 2:35facts. So what gives? You know, why is
  72. 2:38AI jagged, so to speak? Like why can it
  73. 2:40be so good at some things and so not
  74. 2:43good at other things? And what is the
  75. 2:45Occam's razor explanation for that? Um I
  76. 2:48will be blasting through my slides fast.
  77. 2:50I'm sorry. Maybe I could slow down for
  78. 2:52people taking pictures. Um but uh I I'm
  79. 2:55noticing that uh
  80. 2:57this is going to go fast is all I'm
  81. 2:58saying. So I think that we as a field
  82. 3:01should have an Occam's razor explanation
  83. 3:03for why are there some things that are
  84. 3:05just
  85. 3:06not just good, they are unbelievably
  86. 3:08good, superhuman, like millennium prize
  87. 3:10problems in math. But on the other hand,
  88. 3:12they're not just bad, they're so
  89. 3:14useless, they can't even be used to
  90. 3:15touch accounting. That sounds way easier
  91. 3:17than million prize problems in math,
  92. 3:18right? I could do the latter. I
  93. 3:20definitely could not do the former and I
  94. 3:21was a math athlete. So what gives here?
  95. 3:24What is the reason for this? And I
  96. 3:26normally love this as an interactive
  97. 3:28thing, but I will instead just tell you
  98. 3:29my answer. Sorry for the power dynamic.
  99. 3:31Um everything on the left side is
  100. 3:34intrinsically a human in the loop task.
  101. 3:36Like deep research, I could not do
  102. 3:38better than deep research. The models
  103. 3:39are so [ __ ] good at that.
  104. 3:41But if gun to my head or gun to your
  105. 3:44head, you had to pick between me and
  106. 3:45deep research and the answer had to be
  107. 3:47correct, not well written, you'd
  108. 3:49probably pick me every time. And that's
  109. 3:51the distinction between like optimizing
  110. 3:54for automation, which tends to be a lot
  111. 3:56easier and a lot simpler, yet
  112. 3:58AI keeps failing at this. Every year
  113. 4:00there's like an infamous failure of AI
  114. 4:02failing to even automate drive-throughs.
  115. 4:04And that should be kind of shocking to
  116. 4:06people. And my claim is actually for
  117. 4:09these tasks pleasing a human is nothing
  118. 4:11to do with doing the task. What you want
  119. 4:13to do is basically be an abstraction
  120. 4:16away. You want the decision to be made
  121. 4:17and then you want to get back into doing
  122. 4:19automation and code is the language of
  123. 4:21automation. Therefore you need to
  124. 4:22integrate with code. This is a lot but I
  125. 4:24will summarize it again. I get the irony
  126. 4:27that uh you know I co-created RLHF and I
  127. 4:30do believe RLHF is part of the problem
  128. 4:32and RLHF in particular is why all AI
  129. 4:35today does require a human in the loop
  130. 4:38with an asterisk unless the task is like
  131. 4:40very trivially verifiable and can run on
  132. 4:42its own but still requires a human in
  133. 4:44the loop to verify it afterwards. It is
  134. 4:46because we literally put it in. That is
  135. 4:48what the HF is in RLHF and my claim is
  136. 4:51that there's a better way. You know my
  137. 4:53summary of the state is 100% of LMs
  138. 4:56today are optimized for assistance. You
  139. 4:58might talk about RLVR but the RLVR
  140. 5:00models are very bad to interact with if
  141. 5:02they've not been RLHFed.
  142. 5:03The result is LMs become too good to be
  143. 5:06true at assistance. They are just so
  144. 5:08[ __ ] good like you should never pick
  145. 5:09me over ChatGPT for your ChatGPT
  146. 5:11queries. But for actually doing like
  147. 5:15you know run in the background cognitive
  148. 5:17work you should not trust LMs at all
  149. 5:19because they've not been optimized for
  150. 5:20that which is why they're so bad at the
  151. 5:22tasks of automation and I would also
  152. 5:24guess that automation is a lot more
  153. 5:26valuable than assistance because you can
  154. 5:27do way cooler deeper I'm going to get
  155. 5:29into this later but I think there's
  156. 5:30going to be so much stuff that we can do
  157. 5:32and we are just scratching the surface
  158. 5:34of that right now and that's the tragic
  159. 5:35thing about AI uh the state of AI for
  160. 5:38me.
  161. 5:39Let's talk about the Jevolution. Um
  162. 5:43I I can't tell if this is appropriate
  163. 5:44but it's too late.
  164. 5:45Um
  165. 5:46oh god don't take pictures of this one.
  166. 5:48Um
  167. 5:50cool.
  168. 5:51Oh I'm getting a thumbs up. Yeah
  169. 5:53excellent. So we're we instead of doing
  170. 5:55RLHF we just have to do something else.
  171. 5:57We need to think from first principles.
  172. 5:58How do we optimize for machine native
  173. 6:00AI? You guys hopefully know the
  174. 6:01interface of Jev and that is what we are
  175. 6:03doing. More important than optimizing
  176. 6:05for its interface, we are
  177. 6:07uh rethinking how AI should be done
  178. 6:10because this is what we did again for
  179. 6:11RLHF.
  180. 6:13For Sutton's bitter lesson,
  181. 6:15I'm going to loosely paraphrase to make
  182. 6:16my point easier to say, is that
  183. 6:19scaling um and compute matters more than
  184. 6:22algorithms. This is why people become
  185. 6:23like scaling pill geronimous and I would
  186. 6:26argue that there's an even bitterer
  187. 6:28lesson than that, which is data always
  188. 6:30matters more than compute. We are
  189. 6:31learning that quite a bit in the field
  190. 6:33now, but it has been obvious since the
  191. 6:35beginning of time. I'll tell you when
  192. 6:37the this is the one you should be
  193. 6:38photographing. I would argue that the
  194. 6:40the bitterest lesson is doing the right
  195. 6:43task. And doing the right task is like a
  196. 6:45weird thing to think about, especially
  197. 6:46for ML people, cuz they like algorithms.
  198. 6:49They want number goes down or up or
  199. 6:51whichever is good direction. They don't
  200. 6:52want to think about the real world. But
  201. 6:54this is always how you make gigantic
  202. 6:56gains in performance. This is how the
  203. 6:58big breakthroughs happen. 100% of the
  204. 7:00time it comes from this. And this is
  205. 7:02true for classical RL games. This is why
  206. 7:04Sutton had his bitter lesson, it which
  207. 7:06is data is derived by compute when
  208. 7:09you're doing rollouts. Like there's
  209. 7:10there's why would you need off policy
  210. 7:12data when playing chess or something
  211. 7:13like that? And doing the right task is
  212. 7:15defined by the rules. So actually when
  213. 7:17you're doing RL, you don't even need to
  214. 7:19think about the right task.
  215. 7:20For pre-training, we actually as a field
  216. 7:23had to do a lot of thinking about the
  217. 7:24right task. I don't know if people
  218. 7:26remember the Bert days with um masked
  219. 7:28language modeling, but it took a while
  220. 7:31and it was very unclear to figure out
  221. 7:32that next pro token prediction was super
  222. 7:35[ __ ] cool, not because it was better
  223. 7:37at masked language modeling. You know,
  224. 7:38the objectives on their own aren't
  225. 7:40special. It's about what you can do with
  226. 7:42them afterwards. And it turns out next
  227. 7:44token prediction is incredible at
  228. 7:45generating strings, which is what these
  229. 7:47pre-trained models were so good at. So
  230. 7:49no level of masked language modeling,
  231. 7:50even if you had today's compute and
  232. 7:52algorithms, would make it as interesting
  233. 7:54as GPT-3 was because it was just doing a
  234. 7:56different task. And data, you know, the
  235. 7:59internet is obviously a giant
  236. 8:00bottleneck. Now we're like, you know,
  237. 8:01burning books or whatever in order to
  238. 8:03get more data, but data obviously
  239. 8:05matters a lot more than compute and
  240. 8:06people keep like rediscovering this and
  241. 8:08figuring this out.
  242. 8:09It's extra true for making LLMs useful.
  243. 8:12Um, before RLHF even existed, the data
  244. 8:15for RLHF didn't exist and you could
  245. 8:18uh like it was night and day. I'll just
  246. 8:19show you the next slide. This is the
  247. 8:20probably the relevant part. This is what
  248. 8:22this was the work in the RLHF paper.
  249. 8:25And the high level was that even with
  250. 8:27the dumbest algorithms in the world,
  251. 8:28even with a GPT-2 sized model, you got
  252. 8:31something like GPT-9 level performance,
  253. 8:34like real GPT-9, not today's like fake
  254. 8:36GPT numbers. Like the numbers used to
  255. 8:38mean like a 100x scaling in effective
  256. 8:40compute for pre-training and they've
  257. 8:42given up on that because it just can't
  258. 8:43scale anymore. And still even with that
  259. 8:45it would take something like GPT-7 to 9
  260. 8:48to match GPT-2, the model from like 2018
  261. 8:51or whatever it was, at the task of
  262. 8:53instruction following. That is kind of
  263. 8:55wild because you think big LLMs are so
  264. 8:58important for this and they are, but
  265. 9:00that is not the most important thing
  266. 9:01compared to even the gigantic jump of
  267. 9:04doing the right task. And spoilers, this
  268. 9:06is what we did for Jev, right? Like we
  269. 9:08made this gigantic leap in the field
  270. 9:09that no one could realized could be
  271. 9:11coming because we were optimizing for a
  272. 9:13task and very few people think about the
  273. 9:15right task.
  274. 9:16Um,
  275. 9:17this is why Jev has the je ne sais quoi.
  276. 9:21Um, that's a really smart pun I think.
  277. 9:24Um,
  278. 9:25that that little bit of a pity laugh,
  279. 9:27but thank you. Um, you get what you
  280. 9:29optimize for. I'm looking for a better
  281. 9:32word than reliable, so if you guys have
  282. 9:33suggestions let me know. But there's if
  283. 9:35there's this reliability of intelligence
  284. 9:37that's really missing in LLMs. I often
  285. 9:39call it you get Einstein 95% of the
  286. 9:42time, but Mr. Bean 5% of the time.
  287. 9:45And if you can't tell which is which,
  288. 9:46how the [ __ ] are you going to automate
  289. 9:47anything? And instead instead of like
  290. 9:49focusing on like sometimes occasional
  291. 9:51smarts that you can then verify, what if
  292. 9:53you focused on extreme extreme narrow
  293. 9:56focused reliability so you can get that
  294. 9:58smarts every time? And this doesn't show
  295. 10:00up on benchmarks. It is the reason why
  296. 10:02people love Jev. And this is also I
  297. 10:04realize in hindsight we obviously didn't
  298. 10:05prepare for a chat GPT moment for
  299. 10:07developers, but it happened because
  300. 10:09people in the AI space just overpromise,
  301. 10:12underdeliver so [ __ ] much that having
  302. 10:15a model that actually didn't do that was
  303. 10:16a surprise. And it is not the speed,
  304. 10:18cost, or even interface. This is
  305. 10:20something that, you know, my team tells
  306. 10:22me I shouldn't be telling people so that
  307. 10:24we can have a long-term advantage, and I
  308. 10:25just care too much. Like it's a reliable
  309. 10:27intelligence. Make sick models, please,
  310. 10:30even if we, you know, if it doesn't
  311. 10:31involve our models.
  312. 10:33Um we see that Jev is like a foundation
  313. 10:36for smart software. I'm going to skip
  314. 10:37all over this, but I think software is
  315. 10:38going to be so [ __ ] back. I think the
  316. 10:41narrative of SaaS apocalypse is
  317. 10:42obviously not going to happen, but
  318. 10:45I think that
  319. 10:46even calling it a SaaS apocalpyse is
  320. 10:48like an understatement, and I'll get
  321. 10:49into that a bit, and you guys ideally
  322. 10:51are the perfect crowd for this. I'm I'm
  323. 10:52so excited for this part because that's
  324. 10:53the part that's new to me. Um
  325. 10:56I want to make a distinction between
  326. 10:58smart software and just-in-time
  327. 11:00software. The the crazy part is AI is so
  328. 11:03transformative, and yet somehow despite
  329. 11:06AI being so awesome, software has
  330. 11:08essentially unchanged in the last decade
  331. 11:10except being cheaper to write. Isn't
  332. 11:13that what weird? Isn't the it weird that
  333. 11:15the best we can do in software is having
  334. 11:16a chatbot to the side that sometimes
  335. 11:19interacts with the software but cannot
  336. 11:20take expensive decisions for you? That
  337. 11:22is kind of like a weird thing if AI was
  338. 11:24really that transformative, and I think
  339. 11:26the problem is we're focusing too much
  340. 11:28on coding agents and not enough about
  341. 11:30putting the intelligence into the inner
  342. 11:32loop, which I'm going to get more into a
  343. 11:33bit. And people are jeving. There's been
  344. 11:36tons of successes. We don't need to flex
  345. 11:38this. There's so many cool things, but
  346. 11:40my AI E secret is going to be that the
  347. 11:43field isn't jeving hard enough. And this
  348. 11:45is what Swix said that I can make a call
  349. 11:47to action for you guys that's unique,
  350. 11:48and hopefully I have enough time for
  351. 11:49this.
  352. 11:50I think the world still has LLM
  353. 11:52Stockholm syndrome. We are still
  354. 11:54thinking from a really LLM-filled place,
  355. 11:56which is
  356. 11:57I think tragic, you know? Like I I I
  357. 11:59love the software saying the tools you
  358. 12:01use change the way you think, and I
  359. 12:03think that it's changed the way we think
  360. 12:05way too much,
  361. 12:06and we're not really rethinking software
  362. 12:08like I think the field needs. I think
  363. 12:10that we could just just doing so much
  364. 12:11more. Hopefully I can try to do my best
  365. 12:13for my remaining time to convince you
  366. 12:15guys of that.
  367. 12:16Um, so let's jump to the future.
  368. 12:19Um,
  369. 12:20I an analogy I love, bear with me, is
  370. 12:23about LEDs. Before there was a blue LED,
  371. 12:26there was red and green LEDs. These were
  372. 12:28really limited in capability, and they
  373. 12:30were used for like simple indicators and
  374. 12:32you know like clocks and stuff like
  375. 12:34that. And when the blue LED was made
  376. 12:37discovered, I actually don't really know
  377. 12:38how things outside of computer science
  378. 12:40work. Um, but when it existed, I guess,
  379. 12:43um, it enabled not just like 50% more
  380. 12:46use cases, which what you'd expect about
  381. 12:48like changing colors from two to three,
  382. 12:50it enabled so many more. Like now LEDs
  383. 12:52could be used for white light. It could
  384. 12:53be used for like the displays that we
  385. 12:55all use right now, and it created a
  386. 12:57revolution in lighting. And it's it's
  387. 12:59it's unfathomable how much bigger
  388. 13:02the capabilities of LEDs were just you
  389. 13:04know adding a new primitive that
  390. 13:06composes together with the old
  391. 13:08primitives. You can see where I'm going
  392. 13:09here maybe. And I think it's a similar
  393. 13:12story for software. Software is so
  394. 13:15[ __ ] incredible. Ignore AI for the
  395. 13:16last you know half decade, but
  396. 13:18software's done so many insanely cool
  397. 13:21things. We've done a lot of stuff you
  398. 13:23know like do I
  399. 13:25love using SAS? Not really, but does it
  400. 13:27solve problems? Abso [ __ ] lutely. And
  401. 13:30it's incredible how much we've done with
  402. 13:32three [clears throat] fundamental logic
  403. 13:33gates. You know like we are doing this
  404. 13:34with like simple binary operations that
  405. 13:37can do things like greater than, less
  406. 13:38than, equal. It's kind of nuts how much
  407. 13:41software has been built from such simple
  408. 13:43primitives. And my claim is that what we
  409. 13:47are doing with AI now and we're making
  410. 13:49it our intelligence machine native, this
  411. 13:51is kind of conceptually adding a fourth
  412. 13:53semantic logic gate.
  413. 13:55What if we can program with common
  414. 13:57sense, with intention, with human
  415. 13:59understanding? Like do we still need
  416. 14:01regexes with like crazy things or can we
  417. 14:03just say like we want a Gmail email
  418. 14:04address or a real looking one? There's
  419. 14:06so much that we can do that we are going
  420. 14:08to need to push the you know push the
  421. 14:09frontiers of cheap, smart, fast in order
  422. 14:12to get done. But I think there's going
  423. 14:14to be like so many things here, And I
  424. 14:16think just like the blue LED, it's going
  425. 14:17to be way bigger than just increasing
  426. 14:20the representational ability of software
  427. 14:22by 33 % cuz it's adding effectively
  428. 14:25another kind of sort of logic gate. And
  429. 14:27this sci-fi is the stuff that I think
  430. 14:29will be impossible to predict. Just like
  431. 14:31who could predict how powerful the
  432. 14:33internet would have been because it
  433. 14:34requires it's like much more grassroots
  434. 14:36and decentralized and it requires people
  435. 14:38to be empowered to do insane stuff.
  436. 14:40Um I call this software research. Um
  437. 14:44that's not a joke. Um but uh I I think
  438. 14:47that there's going to be like an
  439. 14:49explosion of what software can do and we
  440. 14:52will want people to be exploring those
  441. 14:54boundaries of software, not necessarily
  442. 14:56because it's valuable, but because they
  443. 14:58can, because we have like new frontiers
  444. 15:00to explore. And this is what the early
  445. 15:02internet was like. Should all of those
  446. 15:04websites existed?
  447. 15:05Probably not, but like it was very cool
  448. 15:08and we found a lot of awesome stuff from
  449. 15:09that and I think we we you know we have
  450. 15:11another opportunity for that. So we
  451. 15:13should not just keep living in this
  452. 15:15little you know fish tank that we are
  453. 15:17the victims of open AI and Anthropic on
  454. 15:18whatever they ship and we just have to
  455. 15:20be like LLM rappers. So I'm going to
  456. 15:22pitch software research as a direction
  457. 15:24here. The theme is going to be real
  458. 15:27software engineering, not just vibe
  459. 15:29coding. There's it's going to be like
  460. 15:31the kind of stuff that everyone has
  461. 15:32known if you've been a software
  462. 15:33engineer. Decomposition, breaking tasks
  463. 15:36down. We need to think about stages,
  464. 15:37ideally ones that can be verified so we
  465. 15:39have real encapsulation. And we all need
  466. 15:42to actually do state management instead
  467. 15:44of assuming everything is global state
  468. 15:46within a KV cache. And the theme is just
  469. 15:47going to be like real [ __ ]
  470. 15:49engineering by people who care a lot
  471. 15:51about the end result. It doesn't have to
  472. 15:53be valuable. Ideally it's sick as [ __ ]
  473. 15:55Um and you know we should be pushing the
  474. 15:57limits.
  475. 15:58I'm going to go into a couple of like my
  476. 16:00favorite ideas and I want to figure out
  477. 16:02how to encourage these in the world. So
  478. 16:04this is my first step at this. I'm just
  479. 16:06going to talk about it. We have raised a
  480. 16:08bunch of money so I would like to figure
  481. 16:10out how to fund this, but we don't have
  482. 16:11the operations help to do that. But here
  483. 16:13are just some ideas out there. Um I
  484. 16:15think agents are
  485. 16:18they're cool, but like they're
  486. 16:20they're so inflexible in their design
  487. 16:22because of the tyranny of the KV cache.
  488. 16:24I don't have the time for this if this
  489. 16:25timer is to be believed, but believe me
  490. 16:28that there's so much that people have
  491. 16:29tried to do but failed at because it
  492. 16:31ends up becoming inefficient because it
  493. 16:33fights against the KV cache. But I think
  494. 16:35there's a lot more to do. Why is memory
  495. 16:37O of N, right? Like why can't we have
  496. 16:39all of our history of coding agents
  497. 16:41always included in every new call? That
  498. 16:44is how continual learning works, right?
  499. 16:47And we have the memory available. It's
  500. 16:48just that we don't have the smarts to
  501. 16:50map reduce over the memory to find out
  502. 16:52like what you should what you should
  503. 16:54remember, what should you shouldn't. Um
  504. 16:56I think there's many ways to do login or
  505. 16:57even better lookups. Why when you're
  506. 16:59making many agents at once, why can't
  507. 17:01they not share state? Why do they need
  508. 17:03to rediscover the same patterns of the
  509. 17:05codebase over and over and over again?
  510. 17:07And why can't non-naive routing in sub
  511. 17:09agents work? Like even naive routing
  512. 17:10doesn't work. But like why can't we do
  513. 17:12these things? And why just like the you
  514. 17:14know the the big labs keep going back
  515. 17:15and forth on whether or not sub agents
  516. 17:16work? Um
  517. 17:18I think that like from an engineering
  518. 17:20perspective, there's elegance to it, but
  519. 17:22they fight against the KV cache. And I
  520. 17:23think that if we design agents around
  521. 17:25that, we could do so much more.
  522. 17:28This one's the probably the most vague
  523. 17:30of my directions, but I feel very
  524. 17:32strongly about it. Um the world is full
  525. 17:34of principal agent problems. This is
  526. 17:36when you have like you know ulterior
  527. 17:37motives between this is not a coding
  528. 17:39agent, but like any agent. The the
  529. 17:40classic example is citizens versus
  530. 17:42elected officials. And I don't want to
  531. 17:44make a too political claim here, but
  532. 17:46like there's misalignment in all levels
  533. 17:49of society. And this results in like
  534. 17:51waste, fraud, bad things, and it it
  535. 17:54results in for a society that wants
  536. 17:56fairness, you need to over emphasize the
  537. 17:58letter over the spirit, which then
  538. 18:00results in loopholes and pay pay to win
  539. 18:03dynamics in the world. And I think the
  540. 18:05root of that problem is we can't encode
  541. 18:06the spirit. Just like with software,
  542. 18:08we're constrained to encoding bits
  543. 18:10instead of what we actually meant in
  544. 18:12that situation. And I think so many
  545. 18:14organizations, society could be
  546. 18:16transformed by really encoding the
  547. 18:17spirit all over the place.
  548. 18:19Obviously real-time intelligence
  549. 18:21security,
  550. 18:22can we prevent the OpenAI hugging face
  551. 18:24sandbox, blah blah blah? Um, why did why
  552. 18:27couldn't they prevent it? It is not
  553. 18:29because they didn't have the
  554. 18:30intelligence. I would argue it's a lot
  555. 18:31easier to figure out they're doing a
  556. 18:32naughty thing than to actually do the
  557. 18:34naughty thing. It is because it was not
  558. 18:36fast, cheap, and reliable enough to
  559. 18:38actually run real-time. And what happens
  560. 18:41when every corporation has their own
  561. 18:44real-time security that's actually
  562. 18:45smart? How cool would that [ __ ] be?
  563. 18:48I can't talk about generative UIs
  564. 18:50because people are going to be talking
  565. 18:51about it afterwards. I don't want to
  566. 18:52steal their thunder, but it's cool as
  567. 18:54[ __ ] trust me. Um,
  568. 18:56like semantic dependent typing. Why
  569. 18:59can't
  570. 19:00the we have like a type system that
  571. 19:02actually is semantic, right? Like why
  572. 19:04can't we
  573. 19:05like declare what local properties we
  574. 19:08want so we don't have to read a chain of
  575. 19:10dependencies deep down?
  576. 19:11These all should be possible. And
  577. 19:13imagine if you could like run a JEP-like
  578. 19:15model in your lint checker to check if
  579. 19:17your it's all type-safe and it could you
  580. 19:20know uphold semantic properties. It
  581. 19:21would be so [ __ ] cool.
  582. 19:24There's there was a great Twitter video
  583. 19:26on um like a voice interface. In
  584. 19:29general, I think new interfaces are so
  585. 19:31exciting. This is to me the return of
  586. 19:33like this is going to show by agent
  587. 19:34software like do what I mean. This used
  588. 19:36to be like a popular thing, but it never
  589. 19:38really worked because you it was it was
  590. 19:40too many hard-coded rules to really
  591. 19:41figure out what I mean. Um, the stopping
  592. 19:45slop obviously a big theme of here, but
  593. 19:47like we definitely believe this. This is
  594. 19:49my props I have to
  595. 19:50We have a stop Gen AI baseball bat for
  596. 19:53type-safe. It is very very awesome. I
  597. 19:56want to talk about the root of slop. Um,
  598. 19:58the the root of slop is that AI is
  599. 20:00unreliable. Therefore, you as a company
  600. 20:02or you as you know like someone who's
  601. 20:04doing stuff, it can't be used for
  602. 20:05decisions that costs you stuff.
  603. 20:08But it's actually totally fine to push
  604. 20:09the costs into other people, which is
  605. 20:11horrible, you know externalized costs.
  606. 20:13And then this is what ends up becoming
  607. 20:14slop.
  608. 20:15You know if people actually read this
  609. 20:16stuff and filter it and like care about
  610. 20:18the quality, it becomes not slop. But
  611. 20:20now the world is full of this because
  612. 20:22there's an asymmetric advantage for
  613. 20:24people who are like bad actors and don't
  614. 20:25give a [ __ ] So, I think that we can
  615. 20:27save the world from at least inbound
  616. 20:29hell among many other things and that
  617. 20:31would be very cool. Um, and even
  618. 20:34something I think is super sick is
  619. 20:36cursed software. Like why? If you guys
  620. 20:38ever seen the subreddit bad UI battles,
  621. 20:41it is so [ __ ] cool even though they
  622. 20:43make the horrible interfaces like this
  623. 20:45whole game of like would you like to
  624. 20:46delete your account becomes, you know, a
  625. 20:48a random thing. And I actually think
  626. 20:51doing things because we can is what
  627. 20:53software engineering is about cuz who
  628. 20:55knows maybe 1% of these ideas actually
  629. 20:57lead to something super awesome. And I
  630. 21:00think like really cultivating even bad
  631. 21:02ideas is kind of what being like this,
  632. 21:04you know, playful hacker is all about.
  633. 21:07And generally just automating
  634. 21:09everything, this is what the follow-up
  635. 21:11this is this is a flop image but uh so
  636. 21:13our logo's a little bit wrong but don't
  637. 21:15stare at that that's a little weird. Um,
  638. 21:18this is what a lot of the follow-up
  639. 21:19talks are going to be about. Thank you.
  640. 21:21Um,
  641. 21:22and and if you're a software engineer,
  642. 21:24you almost certainly think in the world
  643. 21:26of bits. If you know anything about
  644. 21:28software uh and you've interacted with
  645. 21:30software, you know that there are things
  646. 21:33software can do and software cannot do
  647. 21:35because that is just only obvious,
  648. 21:37right? Like there's some things that you
  649. 21:39know you need to do because that is not
  650. 21:41compatible with software. But what if
  651. 21:43those constraints were no longer true?
  652. 21:45Maybe not today but maybe in like 6
  653. 21:47months or a year, what couldn't we
  654. 21:49automate? And that's kind of where I
  655. 21:50want to leave this whole thing because I
  656. 21:52think there is very likely to be a
  657. 21:54future where it's not a question of
  658. 21:56whether or not we can automate
  659. 21:57something, it's going to be a question
  660. 21:58of like how much work is it which is how
  661. 22:00it is right now for the stuff that can
  662. 22:02be automated and that would be cool as
  663. 22:04[ __ ]
  664. 22:05So, hell yeah. Thank you.
  665. 22:07Um,
  666. 22:09I'm
  667. 22:10super overtime but thank you to the
  668. 22:12early believers. You know, we had no
  669. 22:14idea so many people would resonate with
  670. 22:16this. We thought we were the insane ones
  671. 22:18but it turns out like we were actually
  672. 22:20the majority of the world who realized
  673. 22:21AI could be so much more. Thank you to
  674. 22:24like the people who are like realistic
  675. 22:25about reliability. Everyone knows I hate
  676. 22:28benchmarks and a lot of other people are
  677. 22:29really screaming out like benchmarks
  678. 22:31[ __ ] suck. They're actually
  679. 22:33they hurt the people who do not have
  680. 22:35time for anything other than benchmarks
  681. 22:36most. It's tragic. And for the people
  682. 22:38who are pushing the limits of software,
  683. 22:40like [ __ ] yes. And we want to be there
  684. 22:42with you helping you all do that. The
  685. 22:44the dreamers and the people who don't
  686. 22:46want to be limited by everything that
  687. 22:48has happened so far, we want to really
  688. 22:51fight for you and we want this to be as
  689. 22:54reliable as a fourth logic gate. And
  690. 22:56that is a very high bar and we have like
  691. 22:58a lot of work to do. So thank you, thank
  692. 23:01you.

About this transcript

This page contains the full transcript of "Where is all the automation?" — Diogo Almeida, TypeSafe AI by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 4,767 words across 692 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.