YouTube2Text

The Sweet Side of the Bitter Lesson — Thariq Shihipar, Independent — Transcript

by AI Engineer · 3,643 words · 534 segments · language en · Watch on YouTube

Full transcript

  1. 0:12Hey guys,
  2. 0:13I'm Tharak. Nice to meet you.
  3. 0:16Yeah,
  4. 0:17I wanted to talk to you about a couple
  5. 0:20things I've been thinking about since I
  6. 0:21left Anthropic.
  7. 0:23It's been a wild year and
  8. 0:26there a bunch of reflections I've had.
  9. 0:30And my conclusion is
  10. 0:32about, you know, what I'm calling the
  11. 0:33sweet side of the bitter lesson
  12. 0:36and the ways I've been wrong. So I I
  13. 0:37think I got a lot of things right this
  14. 0:39year.
  15. 0:41I think
  16. 0:43I was right about models being able to
  17. 0:45edit and generate videos. I was talking
  18. 0:47about this like
  19. 0:49sometime in like June last year.
  20. 0:51I was right about models being able to
  21. 0:53use HTML to understand their work.
  22. 0:56And I was right about models being able
  23. 0:59to help you understand and figure out
  24. 1:02your unknown to help like plan and learn
  25. 1:05about things. So these were all things I
  26. 1:06was like right about and I was happy and
  27. 1:09I I think that like
  28. 1:11a lot of you might be incorporating this
  29. 1:12sort of stuff in your work.
  30. 1:14But I've been wrong a lot too and I I
  31. 1:16think this is one of those things in AI
  32. 1:18where you sort of just like memory hole
  33. 1:20the things that you're wrong about and
  34. 1:21you're like, oh like I was just always
  35. 1:23expecting, you know, AI to be great.
  36. 1:26But here are some of the things I've
  37. 1:27been wrong about. So first like Bash is
  38. 1:30all you need. I I think there was this
  39. 1:31like moment where the agents were really
  40. 1:33good at using Bash and Claude Code was
  41. 1:36built around the Bash tool. And so, you
  42. 1:39know, people are like, well, we'll just
  43. 1:40turn everything into a CLI. Like, you
  44. 1:42know, like there is just going to be
  45. 1:44Bash and CLIs and that's going to like
  46. 1:46take over the world.
  47. 1:48That's actually kind of not how it
  48. 1:49turned out. Like I actually prefer MCP
  49. 1:51to CLIs now, you know, but now there's
  50. 1:53also code mode and skills and
  51. 1:55progressive disclosure. Like there are
  52. 1:56so many other like ways that agents can,
  53. 2:00you know, act on the world, right? And
  54. 2:03the fact that they were using Bash was
  55. 2:05just like this weird remnant of, you
  56. 2:07know, like them being trained in a
  57. 2:09certain way at the start.
  58. 2:11Then there's like React is all you need.
  59. 2:14I think for a moment it seemed like the
  60. 2:16models were really really good at
  61. 2:17writing React. And so, you know, if you
  62. 2:20wanted to create an AI enabled app you
  63. 2:23should figure out how to use React. Like
  64. 2:24you should use React Native, you should
  65. 2:26use, you know, like you should just make
  66. 2:28everything on the web.
  67. 2:30And
  68. 2:32that also has not been true. I I think
  69. 2:33like now, you know, models can write it
  70. 2:35in HTML, they can write it in React,
  71. 2:37they can write it in Swift, they can
  72. 2:39write it in the Figma proprietary JSON
  73. 2:41format.
  74. 2:43And they'll all look roughly the same.
  75. 2:45Right? And so
  76. 2:47I was afraid that like you know the
  77. 2:48models have been trained on React and so
  78. 2:49we'd just be writing React forever and
  79. 2:51React would be like the last language
  80. 2:53almost, right? And that's almost
  81. 2:54certainly not true, right? I think a lot
  82. 2:56of people are writing their own
  83. 2:57languages now.
  84. 3:01And I think one of the other things I
  85. 3:02was really concerned about was like
  86. 3:04diffusion models will kind of like eat
  87. 3:05art. Like it just be like, you know, we
  88. 3:07wouldn't have any control over these
  89. 3:09models and or the kind of art that these
  90. 3:11models are creating, but it would just
  91. 3:13like create the most beautiful stuff.
  92. 3:14And that hasn't been true at all. Like I
  93. 3:17think now LLMs can make art through
  94. 3:19code, through controlling your computer
  95. 3:21and using, you know, Blender or
  96. 3:24Photoshop or yeah or Figma or whatever
  97. 3:27you'd like, right? And so
  98. 3:30these were all sort of like fears I had
  99. 3:33about like I think
  100. 3:34the bitter lesson, you know, our alien
  101. 3:37intelligence, the Shoggoth, right? Like
  102. 3:39like these are all words we use I think
  103. 3:41to describe this kind of like fear that
  104. 3:45the models will just have this like way
  105. 3:47of doing things and it will be weird and
  106. 3:49alien to us and we just need to accept
  107. 3:51it. You know, like you just have to like
  108. 3:53submit to this like foreign
  109. 3:55intelligence.
  110. 3:56But that's just like I don't think
  111. 3:58that's what happened, you know?
  112. 4:01And so I think that like the question
  113. 4:03come back to a lot is like okay, like
  114. 4:05how can we be AGI piled and human
  115. 4:06centered and what can we learn about you
  116. 4:09know, this progression what we like you
  117. 4:11know, what can I learn about what I was
  118. 4:13wrong about, right?
  119. 4:14So I think this is really really
  120. 4:15important because software engineers are
  121. 4:17like the leading edge of like AI
  122. 4:19adoption, right? Like any thing that is
  123. 4:21happening to software will happen to
  124. 4:23knowledge work and other parts of the
  125. 4:24economy and so I think how we deal and
  126. 4:28engage with uh with AI is um
  127. 4:31yeah, how other people will as well and
  128. 4:33you know, I really enjoyed the different
  129. 4:34talks here talking about things like
  130. 4:36yeah, fighting slop. I think you know,
  131. 4:38we're the first defense uh for that,
  132. 4:40right? Or like being more ambitious.
  133. 4:42Um, what I like to think about it as AI
  134. 4:45is both at the same time an industrial
  135. 4:48revolution and a Renaissance, right?
  136. 4:51And I think this year was really about
  137. 4:54the industrial revolution, right? Like
  138. 4:56we use the word software factories very
  139. 4:58often
  140. 4:59um
  141. 5:00and uh like just sort of like turning
  142. 5:03software from this like
  143. 5:05craft that was very unreliable and hard
  144. 5:08to execute on into something that like
  145. 5:12is not the limiting factor, right? Um,
  146. 5:15and this sort of like industrialization
  147. 5:17is absolutely happening. This is
  148. 5:19definitely something that is happening
  149. 5:21with with software, but at the same time
  150. 5:23there's also kind of a Renaissance
  151. 5:25happening, right? Where people want and
  152. 5:28uh higher and higher quality software
  153. 5:30and every marginal bit of like
  154. 5:34creativity, utility, like spirit, like
  155. 5:37soul that you put into it is like is
  156. 5:39useful and like economically valuable,
  157. 5:41right? And so simultaneously we've got
  158. 5:44both of these things that are very hard
  159. 5:45to juggle and and sort of like
  160. 5:47hold in your head at the same time, uh
  161. 5:49but I think it is very true and uh
  162. 5:53yeah, we're doing both of them. So
  163. 5:55often times when we talk about AI
  164. 5:56capabilities, we talk about like the
  165. 5:58jagged frontier, right? And so uh
  166. 6:01you know, you maybe like you go back two
  167. 6:03years and you're like AI asking AI to
  168. 6:04like this function for you,
  169. 6:06um or, you know, like like a year and
  170. 6:09half ago, a year ago, it's like build
  171. 6:10this feature, then you're like, okay,
  172. 6:11build this like to do app, build like a
  173. 6:13kind of simple to do app, and then clone
  174. 6:15Slack for me now, right? And And And now
  175. 6:18it's like, okay, build Mario Kart from
  176. 6:19scratch, you know, and then then like
  177. 6:21port it to like the Wii or something.
  178. 6:23There's like the capability increases
  179. 6:26are the things that you see, right?
  180. 6:28Where, you know, whenever you look at
  181. 6:30the evals and benchmarks and stuff, you
  182. 6:33see this like just
  183. 6:34um
  184. 6:35this jagged frontier of like
  185. 6:36capabilities. But I also like to talk
  186. 6:39about like the smooth interior
  187. 6:43where the the way Whoops.
  188. 6:46the way the models
  189. 6:48Is there an animation playing here? Um
  190. 6:51the smooth interior is like the idea is
  191. 6:54that the models are um
  192. 6:57as they get better, the the animation
  193. 6:59here is like this jagged frontier and
  194. 7:01this circle in between. And so uh the
  195. 7:04circle sort of represents all of the
  196. 7:06things that the models are too are smart
  197. 7:09enough to do. And so like examples being
  198. 7:11like uh writing React code, right? Or
  199. 7:14like using Bash. And what you see is
  200. 7:17that like it's not jagged below the
  201. 7:20frontier, right? Like if it can write
  202. 7:22React, it can write Figma, it can write
  203. 7:24JSON, it can do sort of like all of
  204. 7:27these capabilities all together. And
  205. 7:29there's like a breadth of tasks that the
  206. 7:30models are really really good at. And
  207. 7:32this is something that's also I think
  208. 7:34really like important and um for me even
  209. 7:38more impactful because I don't really
  210. 7:39care that they can build Mario Kart from
  211. 7:42scratch anymore, right? What I really
  212. 7:44care about is like, hey, is the software
  213. 7:46that we're building, is it high quality?
  214. 7:48Is it worthy of like a Renaissance? You
  215. 7:51know what I mean? So
  216. 7:53um
  217. 7:55yeah, like I think that like, you know,
  218. 7:56you can do things the way you want. So
  219. 7:58like if you're building mockup, you
  220. 7:59could be like, hey, can I get an HTML?
  221. 8:01Can I get a React? Can I get a Figma
  222. 8:03file? Can I computer use my computer and
  223. 8:05use After Effects and make a video?
  224. 8:07There are all these different ways of
  225. 8:09getting the like final result. And I've
  226. 8:12been really happy with this. I really
  227. 8:14was afraid that there would just be like
  228. 8:16one thing that the models love doing and
  229. 8:17we'd only be doing that.
  230. 8:19Um
  231. 8:20and you know like taken to the limit
  232. 8:23kind of like, you know, we talk about
  233. 8:24this a lot but like, you know,
  234. 8:26everything we've talked about so far
  235. 8:28about like, you know, how you build like
  236. 8:29these harnesses and software factories
  237. 8:31and um these like the scaffolding for
  238. 8:33the models.
  239. 8:35Even things like, you know, for a while
  240. 8:37we were also like, hey um the model
  241. 8:40you'll like
  242. 8:42your job is to QA the model, you know,
  243. 8:44or review the code. And that's clearly
  244. 8:46not the case, right? I think as the
  245. 8:47models get smarter and smarter um it
  246. 8:50will become more of this like smooth
  247. 8:52interior, right? Where they're just
  248. 8:54doing things well, right? And like the
  249. 8:57perfect model you can imagine won't need
  250. 8:59to like really verify its work or scan
  251. 9:01for security vulnerabilities. We'll just
  252. 9:03like write the code and be like, hey I'm
  253. 9:05I'm done now. Um and you will check and
  254. 9:08you'll like, oh you're right. Like, you
  255. 9:09know, according to the spec and the
  256. 9:11goals that you that we did you wrote you
  257. 9:13did it correctly. Um
  258. 9:16and uh I think that's actually very
  259. 9:17exciting.
  260. 9:18Um one of one of the ways I like to sort
  261. 9:20of think about visualizing how how much
  262. 9:23the model knows of this eval. I think
  263. 9:25Aparthy um posted it.
  264. 9:28But basically you ask any I think these
  265. 9:30are
  266. 9:31these are Claude models. You ask a model
  267. 9:34uh you give it a longitude latitude and
  268. 9:35you say like, hey is this point
  269. 9:38uh earth or water, right? Uh ground
  270. 9:41sorry land or water. And it from that
  271. 9:43you plot a map of the earth,
  272. 9:45right? And yeah this is Opus uh 4.6
  273. 9:50uh is somewhere over there, right? With
  274. 9:52quite a good software engineering model,
  275. 9:54kind of like a blurry view. And now you
  276. 9:56look at Opus 5.5 and it's like almost
  277. 9:58perfect, right? Like it knows
  278. 10:00every piece of, you know, these like
  279. 10:04these coordinates. It knows like the
  280. 10:06entire world basically. And so this I I
  281. 10:08just gives you an idea besides like, you
  282. 10:10know, the benchmarks about like the the
  283. 10:13the jagged frontier. I feel like
  284. 10:15that's less and less important to us,
  285. 10:17and more important is just like the
  286. 10:18general knowledge that the models have.
  287. 10:21And what's exciting, I think about this,
  288. 10:23is that we can use intelligence to
  289. 10:27understand and shape how we use
  290. 10:28intelligence, right? And I think this
  291. 10:30that will be like the theme of the next
  292. 10:32year, right? Where this year I think was
  293. 10:34really about the industrialization of
  294. 10:36software and and creating this like
  295. 10:38reliable way to make software. Um,
  296. 10:41but what does that look like? So I have
  297. 10:43been building a game.
  298. 10:45Um, it's this like thing I've been
  299. 10:47posting on Twitter about. Um, it's a 2D
  300. 10:50like isometric game with like it's kind
  301. 10:52of like a fighting game inspired by
  302. 10:54things like Avatar and Smash Brothers.
  303. 10:57Uh, and
  304. 10:59I have kind of like it's fun, but it's
  305. 11:01also quite frustrating to do. Like I'm
  306. 11:03really trying to like, you know, like be
  307. 11:05like a great vibe coder here. Um, I
  308. 11:07don't really know what each character
  309. 11:08does sometimes because like I will say
  310. 11:11like, hey, I want this character to
  311. 11:12throw a rock. And then it'll be like, oh
  312. 11:14yeah, great, but like there are all
  313. 11:16these like different things that can
  314. 11:17happen when you're throwing a rock. Like
  315. 11:19this character can also summon a wall.
  316. 11:21So does the rock go through the wall or
  317. 11:24not? And like how what speed or like
  318. 11:26what what about size and charge and all
  319. 11:28these questions? Um,
  320. 11:31and I I find that like the model just
  321. 11:33makes a bunch of choices for me and then
  322. 11:34I try it out and I'm like, wow, I have
  323. 11:36no idea what this character actually
  324. 11:37does.
  325. 11:38I get distracted pretty easily. I don't
  326. 11:40feel in the zone very much. You know, I
  327. 11:42I sort of send off a prompt and I start
  328. 11:44doing something else and I come back.
  329. 11:46Um,
  330. 11:47and I honestly make less progress
  331. 11:49because of these things, right? And so
  332. 11:51uh, I think this is frustrating and I
  333. 11:53think for a while I was like, man, I
  334. 11:55guess this is just like how you work
  335. 11:56with AI. This is just like what it's
  336. 11:57going to be like, you know? But I think
  337. 11:59reflecting I'm like, man, I I don't
  338. 12:01think that's right. I think that like,
  339. 12:02you know,
  340. 12:03we have this like the AI can work in the
  341. 12:06way that we want, right? And so yeah, I
  342. 12:09don't think it's acceptable. I don't
  343. 12:10think that this is what leads to the
  344. 12:12Renaissance, right? I don't think this
  345. 12:13is what leads to like incredible
  346. 12:14software.
  347. 12:16And part of it, you know, going back to
  348. 12:18what Dex said is like like great work is
  349. 12:21just about like this this messy process,
  350. 12:23right? Of like don't know like exactly
  351. 12:25what you want. It's more about the
  352. 12:27choices you made and what wasn't
  353. 12:29involved or wasn't what wasn't included
  354. 12:32versus like what was included.
  355. 12:34And so
  356. 12:37yeah, I think there's like
  357. 12:40you you like kind of just have to be an
  358. 12:41expert. You have to like just really
  359. 12:42know your stuff in order to make
  360. 12:44something great, right? In order to like
  361. 12:46make something worthy of a Renaissance,
  362. 12:48right? And I just like when I look at
  363. 12:49something I'm vibe coding, I'm like,
  364. 12:50dude, I have no idea. Like I just don't
  365. 12:53really get it, you know?
  366. 12:55And so
  367. 12:58what what am I doing? How am I fixing
  368. 12:59this?
  369. 13:01One of the ways I'm doing it, and I'm
  370. 13:02still quite early on. I'm not saying
  371. 13:04that I've like figured it all out, but I
  372. 13:06think one of the ways I'm doing it is
  373. 13:07that I have two parts of my codebase.
  374. 13:09I've got like this like engine part of
  375. 13:10the codebase, and then I have a folder
  376. 13:12like I think I call it spec, but I think
  377. 13:14I'm just calling it human or something.
  378. 13:16And that folder is the code that I read,
  379. 13:19you know? And it's like this code that's
  380. 13:22linted and like there's all these rules
  381. 13:25for Claude about like what that code
  382. 13:27looks like. It's supposed to feel like
  383. 13:28English.
  384. 13:30It's kind of like a spec, and I'll I'll
  385. 13:32show you what it looks like.
  386. 13:34But it's at the density where if I just
  387. 13:36read this code, I know what's happening
  388. 13:39and I can edit it, I can tweak it.
  389. 13:42But more importantly, I just like, you
  390. 13:44know, can have a bit of a ground truth
  391. 13:47of like what the what my game looks
  392. 13:48like. And
  393. 13:51so let's do let's do like a quick demo
  394. 13:53here.
  395. 13:55By demo I guess I mean I'm just going to
  396. 13:57whoops. This is my eTalk. Okay.
  397. 14:00Here we go. Okay. So
  398. 14:02this is my game. I've got in here this
  399. 14:04folder game, and then I've got here this
  400. 14:06code. This is the like agent code,
  401. 14:08right? Like this is the code where I'm
  402. 14:09like, dude, I I don't know how you know
  403. 14:11cameras work or whatever. I you you like
  404. 14:13there's a bunch of code here that I
  405. 14:14don't
  406. 14:15But then in this code, I see have all my
  407. 14:18characters and I have their abilities.
  408. 14:20And so I can go in here
  409. 14:22and I can say like, okay, this is the
  410. 14:25rock move, right? Um and inside of this
  411. 14:28move, there's a bunch of ways to tune
  412. 14:30it. Um it's well commented. It uses
  413. 14:33things like meters, you know what I
  414. 14:34mean? Instead of like
  415. 14:36X like 0.5 or whatever. Um
  416. 14:40it uh tries to like create a bunch of
  417. 14:43different abstractions along the way.
  418. 14:45And so I can see that like one of the
  419. 14:46things that, you know, is useful is that
  420. 14:49like there are three ways to throw for
  421. 14:51this character to throw a rock, right?
  422. 14:52Like it can uh grab it from the ground,
  423. 14:54it can grab it from the wall or
  424. 14:56something. Um
  425. 14:58and I can see sort of like the
  426. 15:01um
  427. 15:02the true logic of the character, right?
  428. 15:05I I don't really know how the engine
  429. 15:07works or how, you know, like this is
  430. 15:10implemented underneath. But I've
  431. 15:12essentially created this like almost a
  432. 15:13custom scripting language for my game.
  433. 15:16Um and so that means like, you know,
  434. 15:18when I'm playing this game,
  435. 15:19um I can tweak little things. You know,
  436. 15:21I'm still I'm still like working on it.
  437. 15:23So like I can say I want the rock to be
  438. 15:25a lot bigger. So I can say like um
  439. 15:28you know, 1.5, but I'm using an actual
  440. 15:30IDE. Uh and so if I go here to my game,
  441. 15:33uh you know, I can see that like I have
  442. 15:36a rock, I can charge it, gets really big
  443. 15:38and I throw it, right? Um but I'm like
  444. 15:41kind of it it it's really interesting
  445. 15:43cuz I'm also starting to use cursor more
  446. 15:46and I'm using the tab model more, you
  447. 15:47know what I mean? Because now now I can
  448. 15:49be like, oh hey, what if I just want to
  449. 15:51say like,
  450. 15:52you know, uh if you summon a wall in the
  451. 15:56middle, like queue it up as an action,
  452. 15:59and then cursor will sort of like, you
  453. 16:00know, do the the tab complete here. Um
  454. 16:05and this is kind of like it's kind of
  455. 16:06like spec driven development, you know,
  456. 16:08but the spec is this like this this
  457. 16:11scripting language basically.
  458. 16:14And um
  459. 16:15yeah, it's like
  460. 16:16is this the final way I'm going to write
  461. 16:18game? I'm not sure. It's like, you know,
  462. 16:20in some ways,
  463. 16:21uh, when I need to do something
  464. 16:22complicated, I still need to ask Claude
  465. 16:24to do it, right? And I need to ask to
  466. 16:26Claude to like make the primitives, but
  467. 16:28it's actually really easy for me to
  468. 16:29understand what's happening and I feel
  469. 16:31like much more in control and while like
  470. 16:34Claude is working, I can also sort of
  471. 16:35review it and work with it.
  472. 16:37Um,
  473. 16:38and that's been really exciting. And so
  474. 16:40I think the
  475. 16:42takeaway for me has been that like
  476. 16:45um
  477. 16:47the takeaway for me has been that like I
  478. 16:49now have to think about the way
  479. 16:52the best way for me to deliver high
  480. 16:54quality software, right? And I'm not as
  481. 16:58constrained and limited by like the
  482. 17:00model's capabilities and like what it
  483. 17:01likes to write code in or something. I'm
  484. 17:03just not thinking about like I need to
  485. 17:05think about like how can I deliver great
  486. 17:07software?
  487. 17:08And so I think if we like really
  488. 17:10internalize that, I think then this the
  489. 17:13next year will be about a renaissance of
  490. 17:15software. And you know, I think many
  491. 17:17people have talked
  492. 17:19today and you know, this conference
  493. 17:21about how to be more ambitious and I
  494. 17:22don't think anything any of this is
  495. 17:23quite new, but I just think there's like
  496. 17:25many things to do like new languages,
  497. 17:27right? Like obviously like you know, uh,
  498. 17:30we want things like Rust that are very
  499. 17:32efficient, but we also want things to be
  500. 17:35uh potentially modular and like hot
  501. 17:37reloadable. Um, we want it to be
  502. 17:40potentially readable, right? Uh, new
  503. 17:42primitives like there are probably new
  504. 17:44types of databases, new types of servers
  505. 17:46like um new operating system
  506. 17:49interactions. I've been sort of like
  507. 17:50vibe coding a bunch of personal
  508. 17:51software, um, that sort of messes with
  509. 17:53my operating system.
  510. 17:55Uh, there are new interaction paradigms
  511. 17:57possible, I think. Um, a lot of like old
  512. 18:00HCI research is now like very
  513. 18:01interesting to revisit. Um, new
  514. 18:04aesthetics, I think the way that we like
  515. 18:06use and design software uh
  516. 18:09can change quite a lot.
  517. 18:11And um yeah, I I think that like there's
  518. 18:13so much to do here that I think I I
  519. 18:15don't think that any of this is
  520. 18:17everyone every speaker I think has
  521. 18:19talked about this and
  522. 18:21um I I don't have too much more to add,
  523. 18:24but I think that like what I really want
  524. 18:26us to say is that what we need to do is
  525. 18:29to decide what the future of work looks
  526. 18:31like, right? And decide what our ideal
  527. 18:33experience is because the models are
  528. 18:35not, you know, this alien intelligence
  529. 18:38that's like trying to force us into a
  530. 18:39specific way of working. Uh they can
  531. 18:42kind of do everything and and they will.
  532. 18:44Um but our job is to like bring about
  533. 18:46the Renaissance. So
  534. 18:48thank you guys. That's my talk.

About this transcript

This page contains the full transcript of The Sweet Side of the Bitter Lesson — Thariq Shihipar, Independent by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 3,643 words across 534 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.