YouTube2Text

LIVE: Uncle Bob on Software Fundamentals in the Age of AI — Transcript

by Matt Pocock · 10,121 words · 1,454 segments · language en · Watch on YouTube

Full transcript

  1. 0:00I've got a treat for you today. I've got
  2. 0:02someone who I've been wanting to speak
  3. 0:03to for a while. Someone who has been
  4. 0:05really into um I mean software for as
  5. 0:08long as I've been a developer, much
  6. 0:09longer than I've been a developer. And
  7. 0:11someone who's now making his mark on the
  8. 0:13agent space. We have uh Uncle Bob Martin
  9. 0:17live and in his bathrobe ready to throw
  10. 0:21down. [laughter]
  11. 0:23Uncle Bob, hello. Welcome.
  12. 0:26>> Hello and thank you. [laughter]
  13. 0:28Good to be here.
  14. 0:30It's great. It's great to have you. It's
  15. 0:32great to have you. Um, for folks who
  16. 0:35don't know the bathrobe thing
  17. 0:37specifically, we should probably caveat
  18. 0:38that. What's What's going on there? And
  19. 0:40why is the bathrobe become a important
  20. 0:42part of your character?
  21. 0:43>> I I it happened I think two years ago. I
  22. 0:46was on my front porch in my bathrobe. It
  23. 0:48was 6:00 in the morning and and I
  24. 0:50started thinking about how awful SQL was
  25. 0:53because of all [laughter] the SQL
  26. 0:55injections and what you know makes no
  27. 0:58sense to have a textual language as your
  28. 1:00database access um for security reasons.
  29. 1:03And I I was in my bathroom at the time
  30. 1:05and I just pulled out my phone and I
  31. 1:07ranted and that became the morning
  32. 1:09bathrobe rant. [laughter]
  33. 1:12Apparently it was popular so I did it a
  34. 1:15few more times.
  35. 1:16>> So is this the mood that you've come
  36. 1:17here today in? Is this is this the uncle
  37. 1:19[laughter]
  38. 1:21>> Bob? It's early in the morning. I
  39. 1:23haven't had my coffee and just don't
  40. 1:25bother me.
  41. 1:27>> 6:00 in the morning thinking about
  42. 1:28sequel, man. [laughter] 6 in the
  43. 1:30morning. Come on.
  44. 1:30>> I'm done with this now. It's actually 10
  45. 1:32in the morning. [laughter]
  46. 1:35>> The bathroom off. I have me. I'm on my
  47. 1:38first Diet Coke, so I'm fine.
  48. 1:40>> Very good. Okay. Well, now the polo
  49. 1:43shirt is out. Um what's what's the Uncle
  50. 1:46Bob's story like for there are probably
  51. 1:48some folks most of my folks are
  52. 1:50developers but some of my folks are not
  53. 1:52developers as well in this audience. How
  54. 1:54do we introduce Uncle Bob the phenomenon
  55. 1:56that is Uncle Bob to especially
  56. 1:57non-developers.
  57. 1:59>> Phenomenon. I don't know about that. Um
  58. 2:01I'm a programmer. I I've been a
  59. 2:03programmer for a very long time. Uh over
  60. 2:07half a century at this point. uh started
  61. 2:10it. My first program was in 1964
  62. 2:15and I was 12. [laughter]
  63. 2:20The program was was uh a little model
  64. 2:23computer that my mother had bought me
  65. 2:25for my 12th birthday and programmed it
  66. 2:28by putting little white tubes on pegs.
  67. 2:31It was essentially a three-bit finite
  68. 2:33state machine, but it fascinated the
  69. 2:35hell out of me at the age of 12.
  70. 2:38Okay. So, you're 12 years old. How do we
  71. 2:40get to uh 50 years later, Uncle Bob
  72. 2:43doing his thing now?
  73. 2:44>> Yeah, a little more than 50. Um, let's
  74. 2:46see. Well, I I you know, I just started
  75. 2:50learning as much as I could about
  76. 2:52programming. My father bought me a book
  77. 2:54about Forran, a book about Cobalt, book
  78. 2:56about PL1. I read them all. Had no
  79. 2:59machines to execute anything on, so I
  80. 3:01wrote programs on paper and executed
  81. 3:04them in my head. got a job uh um writing
  82. 3:08a little bit of code at the age of 16.
  83. 3:11Um but that was a temporary thing. Then
  84. 3:13I got a real job at the age of 18 and
  85. 3:15I've been a programmer ever since.
  86. 3:17>> So you've been in the trenches for a
  87. 3:19long time and you wrote rather an
  88. 3:21important book I think.
  89. 3:22>> A few I one of them I think was
  90. 3:25important. I've written a few more but
  91. 3:27uh yeah one of them seemed to take off.
  92. 3:29That was a nice one.
  93. 3:30>> Yeah. and um uh the cleanliness of code
  94. 3:33if I remember correctly or
  95. 3:35>> yeah clean code. Yeah. Yeah. Yeah.
  96. 3:38>> Here's the second edition. This is this
  97. 3:40is the second edition. But you know
  98. 3:44>> I mean it's it's maybe the most um
  99. 3:46certainly when I have conversations
  100. 3:47about good books. It's it's uh for
  101. 3:50software engineering. It's the one that
  102. 3:52most often gets quoted back at me. Like
  103. 3:54it's incredibly popular, incredibly
  104. 3:56influential. So, and that's why I'm
  105. 3:58interested to talk to you today because
  106. 4:01you've had this big influence in like
  107. 4:03the preAI era and you've been working in
  108. 4:07the pre-AI era for almost as long as
  109. 4:09it's existed virtually now.
  110. 4:13How have things changed for you, Uncle
  111. 4:15Bob, now that AI is out there and is a
  112. 4:18thing? Um, so this took me a little by
  113. 4:22surprise
  114. 4:24around
  115. 4:25December of of last year, Christmas
  116. 4:29time, and I'm just fiddling around and I
  117. 4:32I'd already been playing with, you know,
  118. 4:34chaty PT and a little bit of Grock and a
  119. 4:37little bit of this and a little bit of
  120. 4:38that. And I hadn't been real impressed.
  121. 4:41Uh, and then I went through a period
  122. 4:43where I thought, well, maybe these
  123. 4:44things are a little more interesting
  124. 4:46than I thought. And I I got an agent.
  125. 4:50First agent I got I think was um Grock
  126. 4:53early Grock. It was an early Grock one
  127. 4:56and I just asked it to write me some
  128. 4:58code and it did a kind of a poor job but
  129. 5:02it wrote the code and I was in the
  130. 5:04middle of a project at the time. So I
  131. 5:06said well maybe this thing can help me
  132. 5:07with this project and I started having
  133. 5:10it work with me on the project and I was
  134. 5:13cleaning up after it all the time just
  135. 5:14you know it was always making a mess.
  136. 5:16was always leaving little dog dude
  137. 5:18behind. And I thought, you know, it's
  138. 5:21it's interesting because it's fast, but
  139. 5:25it's frustrating because it makes me
  140. 5:28slow.
  141. 5:30>> Yep.
  142. 5:31>> And then I started thinking, well, wait
  143. 5:33a minute,
  144. 5:35because it's fast, it can do things that
  145. 5:38I cannot. So in the uh very early 2000s
  146. 5:43there were a couple of innovations that
  147. 5:45that caught me. I thought oo these are
  148. 5:47good ideas but they were completely
  149. 5:50impractical. One of them was called um
  150. 5:53uh crap. Um it was uh it's an acronym.
  151. 5:58What can I say? But it's it was a way to
  152. 6:01take uh code coverage. So you would you
  153. 6:04would run code coverage, test coverage
  154. 6:07over your code and you'd also measure
  155. 6:09the cyclatic complexity of every
  156. 6:11function and you would mix those two in
  157. 6:14a complicated formula and out would come
  158. 6:16a score and the score was a measure of
  159. 6:21how crappy your function was. And uh way
  160. 6:24back in the early 2000s, I thought this
  161. 6:27is a great idea. And I ran it over a big
  162. 6:29project I was working on. And yeah,
  163. 6:31there was a bunch of crappy functions,
  164. 6:32but then it took me forever to go
  165. 6:35through every one of those functions and
  166. 6:37try and fix them and rewrite the tests
  167. 6:39and but and it was all working. So I
  168. 6:41didn't think I really could afford to do
  169. 6:43that. So although it was interesting, I
  170. 6:46kind of set it aside. And another one of
  171. 6:49those um innovations was called mutation
  172. 6:52testing. And this this one really caught
  173. 6:54my attention, too, because in mutation
  174. 6:56testing, you you have a little a little
  175. 6:59program that runs through your source
  176. 7:01code and flips negative signs to
  177. 7:04positive signs and less than signs to
  178. 7:06greater signs and equal signs to not
  179. 7:08equal signs and other things as well.
  180. 7:10And for each of those flips, it runs
  181. 7:13your entire test suite and expects the
  182. 7:16test suite to fail because obviously
  183. 7:19they've changed something significant.
  184. 7:21And if it doesn't fail, well, that's a
  185. 7:23surviving mutant and it must be killed.
  186. 7:25Now, I ran this again over that same
  187. 7:28project right around the year 2000. And
  188. 7:31I had to run it overnight because the
  189. 7:33test suite ran for four minutes and I
  190. 7:35was running it maybe several hundred
  191. 7:37times. Uh, but it came up with a bunch
  192. 7:40of surviving mutants which I was able to
  193. 7:42fix. But once again, it was impractical.
  194. 7:45I could not put that as part of a a
  195. 7:47normal build scenario. And so I'm
  196. 7:50sitting there last December or maybe it
  197. 7:52was January by this time and I'm
  198. 7:54thinking,"Well, wait a minute. These
  199. 7:56guys are fast and they don't care how
  200. 7:58boring the work is
  201. 8:02and they will do what I tell them to do.
  202. 8:04So why don't you run crap over
  203. 8:06everything you've just done and it would
  204. 8:09run crap and then it would it would
  205. 8:10clean up the code." And I was watching
  206. 8:12it do this. I think, well, that's pretty
  207. 8:14cool. And why don't you run mutation
  208. 8:16testing, too? and it would run the
  209. 8:18mutation testing. Maybe it took it 30
  210. 8:20minutes instead of an overnight run and
  211. 8:23then it would plug all the holes and
  212. 8:25make sure there were tests covering
  213. 8:26everything. I thought, you know,
  214. 8:30this might be a way to clean up the dog
  215. 8:32dew. These things leave a lot of deus
  216. 8:35and fluff behind, but maybe this is a
  217. 8:37good way to clean up all the dog. So I I
  218. 8:40continued on this path of adding more
  219. 8:42tools and continuing to work with these
  220. 8:45agents and I got them to a place where
  221. 8:47they were doing a pretty good job. So
  222. 8:49nowadays my nowadays my principle is,
  223. 8:53you know, I will set the agents working.
  224. 8:56I will have them run these tools and I'm
  225. 8:58going to work very hard to get it into a
  226. 9:02situation where I don't have to look at
  227. 9:04the code at all.
  228. 9:06>> I can trust what they do. And now I do
  229. 9:08other things to verify that the code is
  230. 9:11still decent. You know, I will look at
  231. 9:13the crap scores and make sure that
  232. 9:15they're low. And I will I will do spot
  233. 9:17checks on the code from time to time.
  234. 9:19And I have a whole bunch of other tests
  235. 9:21that I run. But overall, that's my goal.
  236. 9:23If these things are fast, and they are,
  237. 9:28and if I can constrain them to do a good
  238. 9:30job, then I am not going to impose my
  239. 9:33slowness
  240. 9:35upon them. And my bonus is reviewing the
  241. 9:39code or inspecting the code at some kind
  242. 9:42of level of detail. They are fast with
  243. 9:45code. I am slow with code. So I'm going
  244. 9:47to let them have the code and I'm going
  245. 9:49to deal with the stuff around that to
  246. 9:51make sure it's all okay. So far so good.
  247. 9:55So your goal is to
  248. 9:58start pulling yourself away from the
  249. 10:00code, away from human review to
  250. 10:04construct this scaffolding around the
  251. 10:06code so that you don't have to interact
  252. 10:08with it much yourself and so that the
  253. 10:11agent is constrained as much as possible
  254. 10:14in as tight a straight jacket as
  255. 10:16possible so that it can't make a
  256. 10:18mistake. I think there's so there's an
  257. 10:20assumption there that I want to touch on
  258. 10:22first which is that
  259. 10:25the word dog do right doesn't sound good
  260. 10:28in in a British accent dog do you know
  261. 10:30so so what why is why is dog do bad why
  262. 10:37is bad code bad like why why do we care
  263. 10:41about that because these things are so
  264. 10:42fast and they can move so quickly why do
  265. 10:45we care why are we bothering with all of
  266. 10:47this harness can't we just push through
  267. 10:49until the bugs just disappear. One of
  268. 10:52the things I noticed really early on,
  269. 10:55like Decemberish time frame, was that I
  270. 10:57would be I'd be working with the the uh
  271. 10:59like this little Grock agent and I'd
  272. 11:02have it do something and it would leave
  273. 11:04all this mess around and instead of
  274. 11:06cleaning the mess, I would have it do
  275. 11:07the next thing and it would leave even
  276. 11:09more mess around and then I would have
  277. 11:10it do the next thing. And I noticed it
  278. 11:13slowing down and I noticed it having
  279. 11:16having uh difficulty. It would it would
  280. 11:20get into a mode where it would change
  281. 11:22one thing but inadvertently break
  282. 11:24another and then it would have to fix
  283. 11:25that but inadvertently break another.
  284. 11:27Started going around in circles and I
  285. 11:30thought okay these agents they may be
  286. 11:33they may be fast and they be maybe
  287. 11:35relatively smart but they are as subject
  288. 11:38as humans are to messy code. Now maybe
  289. 11:42not as subject. Maybe there's a
  290. 11:45difference in threshold, but the
  291. 11:47threshold is still there. The code can
  292. 11:49get messy enough that the agents cannot
  293. 11:52deal with it any any longer and then
  294. 11:54they'll just start to spin and make a
  295. 11:57mess even worse and and be unable. I've
  296. 11:59actually had them just give up. One
  297. 12:02agent one time said, "I just can't deal
  298. 12:04with this anymore."
  299. 12:07Parap I'm paraphrasing it. didn't use
  300. 12:08those words, but that was obviously what
  301. 12:10was going on.
  302. 12:11>> Yeah. So, your response to that then is,
  303. 12:15"How can I clean up the crap?" And I
  304. 12:18think when most people encounter that,
  305. 12:20they start loading the agent with
  306. 12:22instructions, right? They start saying,
  307. 12:24"Okay, I'm going to pour in information
  308. 12:27into the agent that's doing the
  309. 12:28implementation, right? I'm going to pile
  310. 12:30in on claw.md or agents.mmd and every
  311. 12:33time I see something bad I'm going to
  312. 12:36use an instruction instead of what
  313. 12:38you're doing where it's a deterministic
  314. 12:40mechanism. So you're using automated
  315. 12:43checks whereas a lot of other people are
  316. 12:45using in my terminology steering right
  317. 12:47they're trying to steer it. So why
  318. 12:49didn't you are you doing any steering or
  319. 12:52like how does that work in your mindset?
  320. 12:55So, initially I started with that. I
  321. 12:57start, you know, I my early prompts were
  322. 13:00here's how you do test-driven
  323. 13:01development. Here's how you do clean
  324. 13:03code. Here's what your code should look
  325. 13:05like. You should follow all these rules.
  326. 13:07And, you know, you come up with
  327. 13:08eventually a document that's five or 10
  328. 13:10pages long describing all the good
  329. 13:13things about code. I I suppose I could
  330. 13:15have fed the whole dog on book into it.
  331. 13:18Mhm.
  332. 13:18>> But what I noticed was that the agents,
  333. 13:22the the models, whatever you want to
  334. 13:24call them, um they treat those rules
  335. 13:28in the uh Pirates of the Caribbean
  336. 13:31sense. They're more like guidelines,
  337. 13:34you know, might follow.
  338. 13:36>> Can I just Can I just credit you? That's
  339. 13:38exactly the metaphor that came into my
  340. 13:39head as well. So, I'm glad we're aligned
  341. 13:40there. [laughter]
  342. 13:42>> Yeah. Well, so they they definitely will
  343. 13:44soften and there's technical reasons
  344. 13:46behind this. I did a not not a lot of
  345. 13:50research but some research into why the
  346. 13:52models behave this way and it turns out
  347. 13:55that there's this phenomenon known as
  348. 13:58lost in the middle. So as the context
  349. 14:02window builds up inside the model the
  350. 14:06stuff at the very beginning and the
  351. 14:08stuff at the very end have more
  352. 14:10prominence than the stuff in the middle
  353. 14:13uh for technical reasons. But once
  354. 14:16again, the the agent will ignore the
  355. 14:18stuff in the middle. And anything you
  356. 14:20say at the very beginning is going to
  357. 14:23get shoved into the middle if it's long,
  358. 14:26right? So maybe the first three
  359. 14:27sentences you put at the beginning will
  360. 14:29remain as priority, but the 50th and the
  361. 14:3380th sentence in there, they're gone.
  362. 14:35They're just in the middle somewhere.
  363. 14:37And the poor agent is trying to deal
  364. 14:40with this massive context that's ever
  365. 14:43growing, ever growing. and it's trying
  366. 14:45to pull the important bits out and the
  367. 14:47stuff in the middle is just gone. So
  368. 14:50deterministic tools don't disappear that
  369. 14:54way.
  370. 14:56The the key that I think with agents,
  371. 14:58and this is really hard to do, the key
  372. 15:00with agents is to trim that initial
  373. 15:02prompt down to its absolute minimum so
  374. 15:06that you can get as much of it as
  375. 15:08possible into its priority.
  376. 15:12>> Yeah.
  377. 15:12>> Right. and then do deterministic tools
  378. 15:14after the fact.
  379. 15:16>> Totally. I refer to this as like the
  380. 15:18smart zone and the dumb zone of the uh
  381. 15:21context window. It's not my term. That's
  382. 15:23Dex Hardy's term. I stole it and it's
  383. 15:24very very good. Which is especially at
  384. 15:26the early part of the context window,
  385. 15:28like the first 150k tokens, it's pretty
  386. 15:30smart. But then as you go along, then
  387. 15:32the attention relationships in the
  388. 15:34transformer get really strained. It's
  389. 15:36really diluted. It's like everyone's
  390. 15:38shouting. Each token is shouting in a
  391. 15:40crowded room and the room's getting more
  392. 15:41crowded. Right? you can't hear the
  393. 15:43signal for the noise. That that's that
  394. 15:45totally rings true for me. So you've got
  395. 15:48then you sort of rejected steering and
  396. 15:50you sort of started going back to some
  397. 15:53old techniques in automated checks. Now
  398. 15:58it sounds like then because those checks
  399. 16:00don't go into the context window in the
  400. 16:02same way that steering instructions do,
  401. 16:03you can kind of just layer those on,
  402. 16:05right? And layer them up and up and up.
  403. 16:07So if you're using a language with
  404. 16:08strong types and like tests and like how
  405. 16:13does that picture look to you? Is there
  406. 16:14ever too much when it comes to automated
  407. 16:17checks?
  408. 16:18>> That's one of the things that I'm
  409. 16:19working very hard to figure out.
  410. 16:21Obviously there has to be a case where
  411. 16:23there's too much, right? You eventually
  412. 16:25you will slow the agents down to the
  413. 16:27point where they're slower than humans.
  414. 16:28And at that point you've lost the game.
  415. 16:31>> Why do it? Um, but as long as you can
  416. 16:34keep the margin of productivity higher
  417. 16:37than a human, you're still you're still
  418. 16:39ahead of the game. Uh, and and from what
  419. 16:42I've seen so far, I you know, I can get
  420. 16:45that margin, you know, like a factor of
  421. 16:47two or three or four, right? They'll
  422. 16:50they'll still go pretty fast. Now, I'm
  423. 16:52I'm slowing them down a lot. When when
  424. 16:55uh when you use these deterministic
  425. 16:57tools, what you are really doing is
  426. 16:58you're putting them into a loop. Lots of
  427. 17:00people like to talk about loops
  428. 17:01nowadays. You're putting them into a
  429. 17:03loop and you're saying, "Okay, you must
  430. 17:05you must change the code until this tool
  431. 17:08says that it's okay." And now the agent
  432. 17:11is going around and around and around.
  433. 17:13Okay, I've got to do this. I've got to
  434. 17:14do that. I've got to I've got to add
  435. 17:15more tests over here. I've got to cut
  436. 17:17the cyclatic complexity. I've got to
  437. 17:19split these functions apart. Got to do
  438. 17:21all this work. And it takes it a while
  439. 17:23to do all that until it gets it into
  440. 17:25conformance. So you're you're you're you
  441. 17:28are sacrificing productivity for higher
  442. 17:32quality
  443. 17:33and at some point that's got to give
  444. 17:35way. But I haven't found the end point
  445. 17:37of that yet.
  446. 17:39>> So now I'm in the midst of trying to get
  447. 17:41multiple agents talking to each other
  448. 17:44and handing off to each other so that
  449. 17:46one guy does one thing and the next one
  450. 17:48reviews it and the next one tests it and
  451. 17:50the next one hardens it and so on. And
  452. 17:52there's communication overhead like
  453. 17:54crazy in that. and yet it's still faster
  454. 17:57by a large token than a human.
  455. 18:01>> Let's talk about that. Let's talk about
  456. 18:02multi- aent systems because that's I
  457. 18:04find that super fascinating. I have
  458. 18:06always had a bit of a and I think this
  459. 18:08might be something something where we
  460. 18:09differ
  461. 18:10>> which is I've always had a bit of a
  462. 18:11suspicion of like people who say uh you
  463. 18:15know I've I've got rid of all my
  464. 18:18workforce. I now have a hundred agents.
  465. 18:21Every one of them has a different role.
  466. 18:22They all talk to each other. they've got
  467. 18:24their own email accounts, all that all
  468. 18:26that rubbish, you know. Um, and so for
  469. 18:29me, what's always worked, but okay, I
  470. 18:31sort of make an exception when it comes
  471. 18:33to implementation and then review,
  472. 18:36right? And I think the benefit there is
  473. 18:38not having two very specialized agents.
  474. 18:40It's having okay, you do the
  475. 18:41implementation and it's kind of like a
  476. 18:43red green refactor approach, which is
  477. 18:46the implement all it has to do is just
  478. 18:48like write the bad test and then make it
  479. 18:51work. It doesn't have to make it
  480. 18:52beautiful. And then the reviewer comes
  481. 18:55in. It doesn't have to do the
  482. 18:56exploration because it's already got the
  483. 18:58diff from the implementer. It knows
  484. 19:00exactly what it's reviewing. And then
  485. 19:02you can pile in quite a lot more
  486. 19:03steering instructions in there because
  487. 19:04the task is much less constrained. So
  488. 19:08I'm really interested in your take on
  489. 19:09that on like what the implementer does,
  490. 19:12what the reviewer does, and what these
  491. 19:14other hardener agents. I've not
  492. 19:15experimented with that. Give me more on
  493. 19:17that. So there's two advantages to
  494. 19:19having multiple agents like this. The
  495. 19:21one is that you can run them in
  496. 19:23parallel. So you could have, you know,
  497. 19:26three coders running at the same time.
  498. 19:28And my little laptop can support a lot
  499. 19:31more than three. Um the other advantage
  500. 19:34is that when you focus the agents down
  501. 19:37to a single task, you're keeping the
  502. 19:39context window under control. The lost
  503. 19:41in the middle problem becomes much less
  504. 19:43of a problem. So you can pile a few
  505. 19:46more, not a lot more, but a few more
  506. 19:48rules up at the top and they'll tend to
  507. 19:50follow them better. You can also set up
  508. 19:53a a system where the agents um are born,
  509. 19:57do the task, and die so that the next
  510. 20:00one comes in with a clean context.
  511. 20:03So [clears throat] those are the
  512. 20:04advantages. The disadvantages are that
  513. 20:06the startup times are high, right? So an
  514. 20:08agent takes, you know, 10 15 seconds to
  515. 20:11even start up. uh and then it's got to
  516. 20:14figure out its whole context all over
  517. 20:16again. So there's that that particular
  518. 20:18startup time as well. I like to focus
  519. 20:21the task as much as I can. So I will run
  520. 20:24a specifier. The job of the specifier is
  521. 20:27to take a a human written document
  522. 20:32and turn it into a uh a girkin
  523. 20:37and a QA a QA uh procedure. Girkin is
  524. 20:41you know given when then stuff. It's a
  525. 20:43high level acceptance test. Uh and a QA
  526. 20:46procedure is a essentially a system
  527. 20:49test. You know you run the system
  528. 20:51through the UI with a with a QA
  529. 20:53procedure. And I have them write it from
  530. 20:56a human's point of view. You are a
  531. 20:58human. You are operating this system at
  532. 21:00the UI. You must prove that the system
  533. 21:02works.
  534. 21:04And I will produce those two documents.
  535. 21:06Girkin documents QA documents. And then
  536. 21:09those feed into a coder. The coder's job
  537. 21:12is to write unit tests and the code that
  538. 21:15implements the described uh story and uh
  539. 21:21also get the girkin working. So it has
  540. 21:23to do that and then that gets fed over.
  541. 21:25Once that's working, the coder will work
  542. 21:28on that for a little while that gets fed
  543. 21:30into a cleaner and the cleaner's job is
  544. 21:33to run crap analysis and just general
  545. 21:36code review. clean it clean up whatever
  546. 21:39mess the implement made because the
  547. 21:41implement will have made a horrible mess
  548. 21:42by that point. And then I have it go
  549. 21:45from there to a hardener. The hardener
  550. 21:48is the guy who runs the mutation testing
  551. 21:50and he's absolutely merciless,
  552. 21:52[laughter] right?
  553. 21:54It's going to mutate it and it's going
  554. 21:57to have 100% coverage and every equal
  555. 22:01sign and every less it's going to do
  556. 22:03that work which takes a good long time.
  557. 22:06uh and it pops out the end and then it
  558. 22:08goes into a QA agent and the QA agent
  559. 22:10takes the the written QA document, turns
  560. 22:14it into an executable script that
  561. 22:17manipulates the system and comes up with
  562. 22:20a deterministic result. And and if you
  563. 22:23can get through all of that,
  564. 22:25you've got a pretty pretty working
  565. 22:27program. I' I've had a lot of success
  566. 22:29with this. it uh if I if I give it a
  567. 22:33task that takes a single agent five
  568. 22:38minutes to complete with questionable
  569. 22:40results. Uh this will take it about an
  570. 22:43hour. It'll take about an hour to go
  571. 22:45through all that which is still a
  572. 22:47benefit because you know a person would
  573. 22:49take about a half a day. So maybe I've
  574. 22:52got a you know a factor of four factor
  575. 22:54of five improvement in productivity and
  576. 22:56very high quality much more quality than
  577. 22:58the human would ever put into it. And
  578. 23:00it's not only that, but you're you're
  579. 23:02saving productivity or or you're
  580. 23:04spending productivity early to gain it
  581. 23:06later, right? This is an investment in
  582. 23:08your own codebase. Yeah. What I found
  583. 23:11really interesting about that is this is
  584. 23:13something I've been thinking about
  585. 23:14recently is it's not only the context
  586. 23:16window that you're manipulating. You're
  587. 23:18also there's this idea of a trajectory
  588. 23:22of a context window, right? Of a session
  589. 23:24where if you get the agent to do one
  590. 23:27thing and you steer it in a certain way,
  591. 23:29then everything that follows in that
  592. 23:31same session, that same context window,
  593. 23:33will continue following that trajectory.
  594. 23:35So if you get it to say, "Okay, maybe we
  595. 23:37should test the UI here," then every
  596. 23:38single time it will test the UI again,
  597. 23:40no matter how many changes you get it to
  598. 23:42make. And the only way to clear the
  599. 23:44trajectory is to clear the context
  600. 23:45window, right? So if you're an implement
  601. 23:48agent and you're just trying to get it
  602. 23:49working, then your trajectory is kind of
  603. 23:52it's not quite as harsh as the one who's
  604. 23:55trying to make sure you have 100%
  605. 23:57coverage. And I found that really
  606. 24:00interesting. Is that something you does
  607. 24:01that click with your mental model as
  608. 24:02well?
  609. 24:03>> Yeah, it certainly does. There's a a
  610. 24:05pretty well-known effect with these
  611. 24:06models and it happens with people who
  612. 24:09aren't programmers, right? So, you're
  613. 24:10talking to a model about um oh, I don't
  614. 24:14know what what's the best coffee to have
  615. 24:17and how do you how do you brew it
  616. 24:19nicely? And you're having this nice
  617. 24:20little conversation with the agent about
  618. 24:23this or with the model about this. And
  619. 24:25then somebody walks by and they happen
  620. 24:27to be talking about the latest soap
  621. 24:29opera that they saw saw on television
  622. 24:31and that gets into the context window.
  623. 24:34You didn't put it there. This guy
  624. 24:35walking by put it in there. But then
  625. 24:37from that point on all the coffee
  626. 24:39references have to do with the soap
  627. 24:41opera, right? The the the model doesn't
  628. 24:44know. It can't differentiate. So your
  629. 24:46idea of trajectory is pretty good,
  630. 24:48right? As long as you can keep the
  631. 24:50direction of the model unconfused.
  632. 24:54It knows the direction it's going in.
  633. 24:56Everything in the context window is
  634. 24:58consistent.
  635. 25:00then it's not going to have these crazy
  636. 25:02hallucinations that that people often
  637. 25:06deal with
  638. 25:07>> or even just mis even just misalignment
  639. 25:09right like halluc I think hallucination
  640. 25:11is always a danger right like whether
  641. 25:13it's you know hallucination context you
  642. 25:14gave it whatever like it's an sort of an
  643. 25:16accepted danger and your approach of
  644. 25:18this kind of gauntlet of different
  645. 25:20agents slamming horrendous stuff is sort
  646. 25:23of a is a great way of taking out that
  647. 25:26so okay you've talked a lot about the
  648. 25:29implementation phase and we can get back
  649. 25:31to that later too. What I'm also
  650. 25:33interested in is what planning do you do
  651. 25:37up front and especially
  652. 25:39how are you thinking about the internal
  653. 25:42structure of your code bases? Because
  654. 25:46having a good test suite, right? I mean,
  655. 25:48I'm interested in what you think here.
  656. 25:49Having a good test suite is good, right?
  657. 25:51Having mutation testing, all that stuff
  658. 25:53is good. But if you've got badly
  659. 25:54designed APIs, if you've got badly
  660. 25:57shaped modules, how does that interact
  661. 26:01with all of this automated checks and
  662. 26:03stuff like how much are you thinking
  663. 26:05about that?
  664. 26:06>> So up to the last month or so, what I
  665. 26:10was doing there was doing that part
  666. 26:12manually. So I would I would have the
  667. 26:15agents build me up a nice thing and then
  668. 26:18I would interrogate the agents. And this
  669. 26:20was manual but but still using the
  670. 26:22agents, right? I'd interrogate the
  671. 26:24agents. What's the structure here? How
  672. 26:27how does this module interrelate with
  673. 26:29that module? What are the modules after
  674. 26:32all? And how do they talk to each other?
  675. 26:34I would ask those questions and then I
  676. 26:37would get scared to death because the
  677. 26:39answers were horribly frightening. And
  678. 26:41then I would design a module structure
  679. 26:44and I would tell the agent, okay, here's
  680. 26:46how the modules should really be
  681. 26:47partitioned and here's how you should
  682. 26:49communicate with them. And I would give
  683. 26:51them an implementation plan that they
  684. 26:53would then implement.
  685. 26:55Now, that's a that's a tough one. So I
  686. 26:57also had my agents build me an
  687. 27:00architecture viewer so I can pop up on
  688. 27:03the screen a nice little UML diagram
  689. 27:06essentially nice little UML diagram that
  690. 27:08shows me the modular structure of the
  691. 27:10system and where the dependencies run
  692. 27:12and I can click on a module and I can
  693. 27:14see inside it to the subm modules and I
  694. 27:16can click on the subm modules and it'll
  695. 27:18actually pop the code up on on the
  696. 27:20screen for me. So I can I can drill down
  697. 27:22as much as I want and view the system
  698. 27:25architecture at any level. That was
  699. 27:28really useful to me and I I've made good
  700. 27:31use of that. I've also put together a
  701. 27:34another deterministic tool where I can
  702. 27:37define which module should depend on
  703. 27:39which, which one should not depend on
  704. 27:41which, how the dependency should flow.
  705. 27:44That goes into a nice tight little
  706. 27:45specification file that the agents
  707. 27:48cannot violate. There's another little
  708. 27:50checker that runs at the end and if they
  709. 27:53violate it, they've got to fix it
  710. 27:54somehow. Usually by inverting a
  711. 27:56dependency or inserting an interface or
  712. 27:59splitting a module in half or something
  713. 28:00like that and it will it will um keep my
  714. 28:04rules
  715. 28:06um from being violated.
  716. 28:08I'm working now to see if I can automate
  717. 28:11that and I'm having not a lot of luck
  718. 28:14[laughter]
  719. 28:15so far.
  720. 28:17I'm basically in exactly the same boat
  721. 28:20as you, which is you just get so much
  722. 28:23leverage by having well-designed
  723. 28:24modules, right? Could you explain what
  724. 28:27that leverage is? Why is why is having a
  725. 28:30good structure for these modules
  726. 28:32important?
  727. 28:33>> Well, it's the same argument that we had
  728. 28:35for the dirty code versus clean code.
  729. 28:38Anything that is well partitioned with
  730. 28:40welld disciplined interfaces between it
  731. 28:43is something a human can grasp because
  732. 28:45we compartmentalize in our minds well so
  733. 28:48do the models so do the agents maybe at
  734. 28:51a slightly different threshold I don't
  735. 28:53know about that yet but they work far
  736. 28:55better if they can focus on a module and
  737. 28:58if that module has a trajectory right
  738. 29:01using your term so that the the model
  739. 29:04does not get confused [clears throat]
  740. 29:07by the topics inside that module. Okay,
  741. 29:11I'm using the word model and module. I
  742. 29:13want [laughter]
  743. 29:14clear.
  744. 29:15>> We get it. We get it.
  745. 29:16>> But that that's important. And it's the
  746. 29:18same argument. It's the coffee and soap
  747. 29:20op opera argument. If you load up a
  748. 29:23module with every bit of stuff under the
  749. 29:27of under the sun, the the poor agent is
  750. 29:30going to wonder, "What the heck am I
  751. 29:32doing in here? How do I do anything in
  752. 29:34here?" if you compartmentalize nicely
  753. 29:38works pretty well just like a human.
  754. 29:40>> So you're getting and you're probably
  755. 29:42getting better value out of your test
  756. 29:43suite as well because that's what I
  757. 29:45always think is like okay I suppose
  758. 29:47here's here's another question for you
  759. 29:48that's related to this I think of and I
  760. 29:52have to say I'm a huge fan of um huge
  761. 29:54fan of your work but I'm also a huge fan
  762. 29:55of John Aster's work. Yeah
  763. 29:57>> where um I mean who isn't a great guy
  764. 30:00you you probably got his book around
  765. 30:01somewhere. Um and his concept of deep
  766. 30:04modules is something I find really
  767. 30:06fascinating which is you have an you can
  768. 30:08have bad modules which is kind of
  769. 30:10shallow modules right that have a wide
  770. 30:12interface and not much hidden inside
  771. 30:14them and you can also have deep modules
  772. 30:16which have a small interface and then a
  773. 30:17deep um lots of hidden information
  774. 30:20inside them and it occurs to me that
  775. 30:22that is really good with models because
  776. 30:24they can read the interface without
  777. 30:27having to understand the implementation.
  778. 30:29Does that sort of chime with you? And
  779. 30:31it's not how you're approaching things
  780. 30:32as well.
  781. 30:32>> Yeah,
  782. 30:32>> absolutely. Does
  783. 30:34>> the models pay attention to interface
  784. 30:35names? They they pay attention to the
  785. 30:37structure. It can allow them to not read
  786. 30:40the code beneath them, which is both a
  787. 30:42danger and and an advantage.
  788. 30:45>> Y
  789. 30:45>> as long as the code is consistent,
  790. 30:46you're okay. Um they also pay attention
  791. 30:49to the tests. They read tests to
  792. 30:52understand what the system does. Um so
  793. 30:55yeah all anything you can do that helps
  794. 30:57the structure of the code will help the
  795. 31:01models understand that code. By the way
  796. 31:04in this book there is a long debate
  797. 31:06between me and John Ora in the appendix
  798. 31:09and it it was a lot of fun. He and I had
  799. 31:11a blast. Well I don't know how much fun
  800. 31:12he had but I had a lot of fun.
  801. 31:15>> I watched I watched the entire thing. I
  802. 31:16saw you guys interviewed um on a really
  803. 31:19great uh uh YouTube discussion where you
  804. 31:21talked about it and I I I I just loved
  805. 31:24it and I that's kind of the reason I
  806. 31:25wanted you to come on actually because I
  807. 31:26I just enjoyed that so much. Um okay,
  808. 31:30there's a few different ways I could go
  809. 31:32now. Um
  810. 31:35okay. Is there in your book anything
  811. 31:40that you would change or update now?
  812. 31:43because specifically I'm thinking about
  813. 31:45some advice like um create small
  814. 31:47functions let's say and keep functions
  815. 31:50small um is there is there anything that
  816. 31:54you're sort of because we've been
  817. 31:56talking a lot about how things have not
  818. 31:58changed right how gauntlets like this
  819. 32:00have always been good we've just never
  820. 32:02had the labor available to actually push
  821. 32:05through them right these ideas have been
  822. 32:07good for a long time we're just sort of
  823. 32:08modifying them in a different way but is
  824. 32:10there anything that we just need to and
  825. 32:12this is I suppose a wide question. Is
  826. 32:14there anything that we need to throw out
  827. 32:16that's just like done or is there
  828. 32:18anything that's I don't like asking this
  829. 32:21question because it puts a lot of
  830. 32:22pressure on you, but do you have an
  831. 32:23answer for that one?
  832. 32:24>> So, um thresholds for one thing. Uh it
  833. 32:28seems to me that the the agents can deal
  834. 32:30with different levels of complexity than
  835. 32:33humans. They have a a much better
  836. 32:36short-term memory. I mean, a huge
  837. 32:38short-term memory and a perfectly
  838. 32:40accurate short-term memory. So one of
  839. 32:43the things that I do is I widen the um
  840. 32:46the allowed size of a function and I do
  841. 32:49that by adjusting the crap score. So for
  842. 32:52a human I would keep crap numbers below
  843. 32:54four, right? But for the agents I've set
  844. 32:58this at six and I'm thinking and maybe
  845. 32:59I'll push it to eight. Um I'm trying to
  846. 33:02find where the threshold is and it's not
  847. 33:04an easy threshold to find. But
  848. 33:06>> what what does that look like in terms
  849. 33:07of four to eight? Like what's the or 4
  850. 33:09to 12? What's the difference there? like
  851. 33:10is it like a 20 line 100line function?
  852. 33:13>> Um so it really boils down to the
  853. 33:15cyclatic complexity which is the number
  854. 33:16of pathways through the function. And uh
  855. 33:19if you've got 100% coverage then a crap
  856. 33:23score of six means that there are six
  857. 33:25pathways through the through the
  858. 33:27function. They're all covered with
  859. 33:28tests. So that's really the goal of
  860. 33:31crap, right? Get it all covered with
  861. 33:33tests and then limit the cyclomatic
  862. 33:34complexity. Um, and you know, I've had a
  863. 33:37number of debates with the agents, and
  864. 33:38by the way, you can't trust any debate
  865. 33:40you have with an agent, but I still have
  866. 33:41them anyway. Uh, and they they seem to
  867. 33:44think that, oh yeah, you know, six is
  868. 33:46probably pretty good. Well, you know, I
  869. 33:48don't you don't really trust them, but
  870. 33:50okay. Um, but I, you know, I think
  871. 33:53there's a threshold difference there.
  872. 33:56There's another another factor. In my
  873. 33:58books, I talk about disciplines. One of
  874. 34:01them was was test-driven development.
  875. 34:03I'm a big advocate of test-driven
  876. 34:05development. But that's a human
  877. 34:07discipline that's done because humans
  878. 34:09are wired a certain way. I cannot and
  879. 34:13will not enforce that on the agents. I
  880. 34:16don't think it makes any sense to make
  881. 34:18an agent write a single line of a test
  882. 34:23and then write a single line of the
  883. 34:25production code and then the next line
  884. 34:27of the test. I don't think that makes
  885. 34:29any sense. For a human, it does. At
  886. 34:31least for me. I mean there's a a huge
  887. 34:33benefit for my work but for the agents I
  888. 34:37don't think so. So I allow the agents to
  889. 34:39behave more like John Asterhow would
  890. 34:41which is to write a function and then
  891. 34:43write the test for that function and
  892. 34:45then write the next function and write
  893. 34:46the test for that function. I allow them
  894. 34:48to do that
  895. 34:50even when I have told them to do
  896. 34:52test-driven development at high
  897. 34:54discipline. They always fall back on
  898. 34:56doing that. They always end up doing
  899. 34:58that. So I figure that's probably okay.
  900. 35:01So the bottom line there is it's
  901. 35:04probably a mistake to impose a human
  902. 35:07discipline on an agent. It is not a
  903. 35:10mistake to impose human values
  904. 35:13on the agent, but there may be
  905. 35:15thresholds that we need to change. But
  906. 35:18the disciplines themselves, the
  907. 35:19behaviors, I don't think it's wise to
  908. 35:21impose those.
  909. 35:23>> That's a lovely way of phrasing it. I
  910. 35:24really like that. So it's it's a
  911. 35:26short-term memory thing, right? Because
  912. 35:27TDD is great when you have very low
  913. 35:30short-term memory, like humans, right?
  914. 35:32You have enough short-term memory to
  915. 35:33write the test and then enough
  916. 35:35short-term memory, just enough to make
  917. 35:37the test pass, right? You can go out and
  918. 35:39get a coffee or something.
  919. 35:42>> So, okay, that was great. Thank you.
  920. 35:45[laughter] You're welcome.
  921. 35:46>> In the
  922. 35:48Okay, we've talked a lot about building
  923. 35:51the thing, right? Right. [clears throat]
  924. 35:53and building the implementation, right,
  925. 35:55and doing module architecture and that
  926. 35:57sort of thing. What do you do before you
  927. 36:00kick things off to your specifier? How
  928. 36:02much planning are you doing before you
  929. 36:04actually go into this gauntlet loop?
  930. 36:06Because like putting the wrong work in a
  931. 36:10gauntlet is super wasteful, right? How
  932. 36:14do you think about that? Well, so the
  933. 36:16temptation
  934. 36:18is to specify,
  935. 36:20you know, get the human to specify,
  936. 36:22specify, specify, specify and then give
  937. 36:25it to the agent. This is a very old
  938. 36:27temptation. It was a temptation we
  939. 36:29underwent in the 70s. It led us to the
  940. 36:32waterfall uh process and all of that.
  941. 36:36And and the agile revolution was the
  942. 36:39answer to that or the answer back to
  943. 36:42that. I don't know how how well we're
  944. 36:44doing with the agile revolution either,
  945. 36:46but but it was the it was a way to say,
  946. 36:49"Wait a minute. All this heavy upfront
  947. 36:51planning makes a mess of everything cuz
  948. 36:55what comes out at the end doesn't look
  949. 36:57anything like the plan and never has."
  950. 36:59Well, the temptation with agents is to
  951. 37:01do the same thing, right? Okay, we're
  952. 37:03going to plan plan. Then we'll give it
  953. 37:06to the agent. And I have tried this. In
  954. 37:09fact, I've been in the middle of trying
  955. 37:10this just this week and it's it's always
  956. 37:13a disaster. It and it's always the same
  957. 37:16outcome, right? The you make all these
  958. 37:19plans and then as the agents are
  959. 37:21running, you the human realize that they
  960. 37:24can't follow that plan because you
  961. 37:27didn't think of everything and they're
  962. 37:28not as wise as you are. So they're
  963. 37:31running halfcocked off on some nonsense
  964. 37:33that you have to stop, back up, rewrite
  965. 37:37the plan, and then start them over
  966. 37:39again. And so I've given up on that. I
  967. 37:42said, "Okay, wait a minute. Let's try
  968. 37:44the agile approach now." And I don't
  969. 37:46know how this is going to work out. It
  970. 37:48might not work out all that well, but
  971. 37:50but let's try the agile approach. Let's
  972. 37:52just let them do a story or two, and
  973. 37:54then we'll look at the architecture at
  974. 37:56the end, and we maybe I'll have to
  975. 37:58manually get involved. and you know sort
  976. 38:01a few things out and then a few more
  977. 38:03stories and so on. That might be a
  978. 38:06better approach. We we may never uh
  979. 38:10escape that manual organizing step at
  980. 38:13the end. Although I'm trying to figure
  981. 38:15out a way to do it, but I don't know I
  982. 38:17don't know if that's possible.
  983. 38:19>> I think of it like you've got this huge
  984. 38:22chunk of labor that needs to happen.
  985. 38:23Right? In the old days, this was
  986. 38:25building the thing, right? and building
  987. 38:27the thing would take days, weeks, months
  988. 38:29maybe. Um, now that has shrunk, right?
  989. 38:32But the planning up front and then the
  990. 38:33review is still the same, right? We're
  991. 38:36expected to do these faster iteration
  992. 38:38cycles as devs, but the stuff that was
  993. 38:41actually quite hard is still kind of the
  994. 38:45same and takes the same amount of time.
  995. 38:47Building the thing wrong or sorry,
  996. 38:49building the wrong thing is something
  997. 38:50that you can still do. And I totally
  998. 38:53agree that there's like this sort of a
  999. 38:55lot of folks are doing this kind of plan
  1000. 38:56maxing thing where they're just, you
  1001. 38:59know, they take their spec and they they
  1002. 39:01think about their spec, they run their
  1003. 39:02spec through seven different agents or
  1004. 39:04something and then they um, you know,
  1005. 39:07they get back a better plan that they
  1006. 39:08then go and implement or something. How
  1007. 39:11like that sound not sound good?
  1008. 39:13>> The agents love to write plans. Oh my
  1009. 39:15goodness, they love it. And they will
  1010. 39:17embellish the plans and the plans will
  1011. 39:20be gorgeous and beautiful and spell out
  1012. 39:22all kinds of details. [laughter]
  1013. 39:25>> Yeah. And then they fall apart at the
  1014. 39:26end.
  1015. 39:27>> Um so yeah, I I think you can see it in
  1016. 39:31the industry right now. There's this
  1017. 39:33movement towards spectriven development.
  1018. 39:36Uh and my my uh impression there is that
  1019. 39:40that's probably not going to work. My
  1020. 39:41experiments did not work particularly
  1021. 39:43well. Um, and I'm thinking now maybe we
  1022. 39:46should once again look at the agile
  1023. 39:49ideas and maybe the idea of do a little
  1024. 39:52bit, get some feedback, do a little bit
  1025. 39:55more, get some feedback, reorganize, do
  1026. 39:57a little bit more, feedback, reorganize.
  1027. 39:59I used to tell this story back when I
  1028. 40:01was doing agile lectures.
  1029. 40:04If if it cost you
  1030. 40:07$1
  1031. 40:09to make a change to a house, including
  1032. 40:13the initial laying of the foundation,
  1033. 40:15the initial roof, everything, every
  1034. 40:17change you gave to the contractor would
  1035. 40:19cost you a dollar. How would you build
  1036. 40:22that house? Would you hire an architect
  1037. 40:25and pay thousands of dollars to the
  1038. 40:27architect to come up with the perfect
  1039. 40:29plan that they then paid a dollar to the
  1040. 40:31contractor so that he could build in one
  1041. 40:33shot? Or would you walk up to the
  1042. 40:36contractor and say,
  1043. 40:38"I want the foundation here. Make it
  1044. 40:40that shape." Oh, no. That's
  1045. 40:41[clears throat] bad. Okay, let's change
  1046. 40:43that foundation. Let's do a little Put
  1047. 40:45the kitchen over here. Put the living
  1048. 40:47room there. THERE'S $2. OH, HECK NO.
  1049. 40:50Let's change those around. That's let's
  1050. 40:52let the kids walk through the oh the
  1051. 40:54traffic pattern is crappy. Move the
  1052. 40:56stairs [laughter] or whatever. Um
  1053. 40:59obviously the latter is probably better
  1054. 41:02and that's what we are looking at right
  1055. 41:04now. It costs a dollar. Well maybe two,
  1056. 41:06maybe five. But the the cost of change
  1057. 41:11has plummeted
  1058. 41:13to as close to zero as I think we're
  1059. 41:16ever going to get it. and well that's
  1060. 41:17that's a prediction I'll probably lose
  1061. 41:19but still right the cost of change has
  1062. 41:22has gone so far down that why would you
  1063. 41:26do this upfront planning because that's
  1064. 41:29expensive
  1065. 41:31why wouldn't you just fiddle fiddle
  1066. 41:33fiddle fiddle until it looks right
  1067. 41:36[laughter]
  1068. 41:36>> yes I I 100% agree I couldn't agree more
  1069. 41:40the the thing I the thing I have I think
  1070. 41:43I have a massive issue with the
  1071. 41:45specdriven development label
  1072. 41:47Because what is spectriven development,
  1073. 41:48right? Like you can have like 10
  1074. 41:50different interpretations of it and
  1075. 41:52every time you pass information to an
  1076. 41:54agent, right? Is prompt engineering
  1077. 41:56specri development, right? Like it kind
  1078. 41:58of is. You know, you don't you know, you
  1079. 42:01would say to your, you know, if in the
  1080. 42:02old days you would have like your mate
  1081. 42:04over the other side, your colleague, and
  1082. 42:05you would say, you just fix that loading
  1083. 42:07issue in the in the header or something.
  1084. 42:09Is that spec driven development? Right?
  1085. 42:11Like is that a specification that I've
  1086. 42:12given him? you know and it seems like
  1087. 42:14every time I say okay do a little bit of
  1088. 42:16upfront alignment first that approach
  1089. 42:19then gets called spectriven development
  1090. 42:21but for me I think the difference is are
  1091. 42:24you persisting your specifications right
  1092. 42:27are you returning to those
  1093. 42:28specifications so what's your attitude
  1094. 42:31there like do you do you keep like a
  1095. 42:33list of all of your specs in the repo
  1096. 42:36>> or do you like what's going on there
  1097. 42:38>> no I do not um the the specifications
  1098. 42:41are ephemeris they go away. Uh they
  1099. 42:44change a lot. I fiddleled with them and
  1100. 42:46you know that that goes away. There is
  1101. 42:49no equivalent to source code. You know
  1102. 42:53we humans wrote the source code. So that
  1103. 42:56was that was the final specification.
  1104. 42:58Well that doesn't exist anymore. There
  1105. 43:00is still source code but we humans
  1106. 43:02aren't the ones writing it. And a lot of
  1107. 43:04people feel that lack. There has to be a
  1108. 43:07human thing that defines everything up
  1109. 43:11front. Well,
  1110. 43:13I mean, in the end, even what the agents
  1111. 43:15produce was produced by humans.
  1112. 43:18The the thing that I've be been doing
  1113. 43:21lately is this. Instead of creating a
  1114. 43:25specification that defines what I want
  1115. 43:29or defines what I have even, I look at
  1116. 43:32the end result and say, well, that is
  1117. 43:34the specification. So, I've got a bunch
  1118. 43:36of tools out there. I've got like the
  1119. 43:38crap tool runs for closure. It runs for
  1120. 43:41Java. It runs for Go. I wrote a few of
  1121. 43:44them, right? I had my agents write them.
  1122. 43:45I've got the mutation tester. I've got
  1123. 43:48my my agent harness. You know, I've got
  1124. 43:50all these things up there. What I tell
  1125. 43:52people is don't download those. I wrote
  1126. 43:55them for me. What you should do is point
  1127. 43:57your agents at them, have
  1128. 43:59[clears throat] the agents look at them,
  1129. 44:00and then build one for you. [laughter]
  1130. 44:02>> Yeah. I think that's a far better way of
  1131. 44:05specifying the essence of something and
  1132. 44:09then customizing it to your to your
  1133. 44:11particular need.
  1134. 44:13>> Yep. It's what I always find weird about
  1135. 44:16agents is that if you send them
  1136. 44:18something, they read it, right?
  1137. 44:20[laughter] Which is very different to
  1138. 44:22humans, right? Very, you know, you can
  1139. 44:25maybe get a 20% hit rate if you send
  1140. 44:27someone a massive specification and 20%
  1141. 44:30is maybe a bit generous. 5% maybe if you
  1142. 44:32pass a you know a spec to an agent
  1143. 44:34they're probably going to read it right
  1144. 44:36>> the opposite side of that is the things
  1145. 44:38that the agents write the humans don't
  1146. 44:40read.
  1147. 44:41>> Yes. Exactly. Yeah. Exactly. So they're
  1148. 44:44expecting
  1149. 44:46I had thought of it like that. They're
  1150. 44:48expecting us to read everything they
  1151. 44:49write and we just don't. God.
  1152. 44:53It's so one-sided. You know, long term,
  1153. 44:55I'm interested in how human
  1154. 44:56relationships differ or get different
  1155. 44:59because we're so used to communicating
  1156. 45:01with agents, right? I dictate to my
  1157. 45:02agents. So, it's like I'm just sort of
  1158. 45:04talking to a friend or something. I'm
  1159. 45:05really interested in sort of how life
  1160. 45:07imitates art over time and stuff. I
  1161. 45:09don't know. That's fascinating to me.
  1162. 45:11So, okay. And I think I'm going to ask
  1163. 45:14you maybe the final question, and it's a
  1164. 45:16sort of fairly beefy one.
  1165. 45:18>> Um, if that's all right, finalish. Um,
  1166. 45:22I'm gonna go John Alart again
  1167. 45:24>> because John has a great definition for
  1168. 45:27>> different types of programming, right?
  1169. 45:29You've got the tactical programming. Are
  1170. 45:32you going to get it?
  1171. 45:33>> I I just have to You've mentioned I've
  1172. 45:35got to pull this book down.
  1173. 45:37>> Wait.
  1174. 45:38>> Uh,
  1175. 45:39it's it's here somewhere.
  1176. 45:41>> Where's mine? I've got mine.
  1177. 45:42>> Oh, good. Okay. All right.
  1178. 45:43>> There you go. I've got mine.
  1179. 45:44>> There it is. Yep.
  1180. 45:45>> I got it.
  1181. 45:46>> Okay.
  1182. 45:46>> Done. um tactical versus strategic
  1183. 45:50programming. Yeah.
  1184. 45:51>> So tactical is the sergeant on the
  1185. 45:53ground, the person kind of fighting the
  1186. 45:55battle. The strategic stuff is the
  1187. 45:58general kind of the person directing the
  1188. 46:00course of the war.
  1189. 46:01>> Now I think we both agree that's a good
  1190. 46:04framing, right, for different types of
  1191. 46:05programming. And agents are really good
  1192. 46:07at tactical, really bad at strategic.
  1193. 46:10>> Yeah. Now, for people who are just
  1194. 46:13starting out and AI has now eaten all of
  1195. 46:16the tactical stuff, how do they learn to
  1196. 46:20do strategic programming? Because I
  1197. 46:23think a lot of people here, that's kind
  1198. 46:25of what they want from you, Bob. They
  1199. 46:27want, please just give me your brain so
  1200. 46:30that I can inject into it, understand
  1201. 46:32why mutation testing is so good, tell me
  1202. 46:34how to explain these modules. Like,
  1203. 46:36that's what they want, I think. So
  1204. 46:38that's your job now is to give people
  1205. 46:40your brain. How how how did they learn
  1206. 46:42that stuff?
  1207. 46:43>> Okay. So, um
  1208. 46:45I get this question a lot. I don't have
  1209. 46:47perfect answers to it because I really
  1210. 46:49don't know. But but here's how I would
  1211. 46:51think about this. Here's here's how I do
  1212. 46:53think about this. First of all, the um
  1213. 46:58the way a programmer should learn
  1214. 46:59whether it's in university or somewhere
  1215. 47:01else, right? It should be to write code.
  1216. 47:04You should you should be writing code
  1217. 47:06right for a year. I don't know how long
  1218. 47:09but you should be writing code so that
  1219. 47:10you know what the agents are dealing
  1220. 47:13with. The next thing that I think should
  1221. 47:16happen is that when you get hired into a
  1222. 47:19company where that company is making
  1223. 47:21heavy use of agents,
  1224. 47:24you the young young person just coming
  1225. 47:26out of training
  1226. 47:29should be treated like an agent.
  1227. 47:31the uh the the guy who's running the you
  1228. 47:34know maybe the lead engineer or whoever
  1229. 47:37the guy who's got bunch of agents
  1230. 47:39running and he's being strategic. He
  1231. 47:41should look at you as an agent and he
  1232. 47:43should give you the same kind of tasks
  1233. 47:45that the agents have and subject you to
  1234. 47:47the same kind of deterministic tools
  1235. 47:49that the agent have agents have to use.
  1236. 47:52And you should spend several months in
  1237. 47:55that state being horribly unproductive
  1238. 47:59but learning a hell of a lot. And by the
  1239. 48:02time you've gone through that gauntlet,
  1240. 48:05maybe you can be trusted to run an agent
  1241. 48:07of your own.
  1242. 48:09I don't know. It's going to be something
  1243. 48:11like that. You cannot lose the code
  1244. 48:15entirely.
  1245. 48:17I used to tell people
  1246. 48:19um 10 years ago used to tell people um
  1247. 48:23if you've never written assembly
  1248. 48:24language, you should spend the weekend
  1249. 48:26writing assembly language just so that
  1250. 48:27you know what's really going on behind
  1251. 48:30the scenes because if if all you're
  1252. 48:32doing is writing Java all day long, you
  1253. 48:34live in a fantasy world. There is still
  1254. 48:36magic yet that you don't understand.
  1255. 48:40Spend a weekend doing assembly language
  1256. 48:42and you will finally understand what's
  1257. 48:43really going on. And I I think that
  1258. 48:46remains true
  1259. 48:48somehow during this educational pathway.
  1260. 48:52You've got to go from the basics binary
  1261. 48:55all the way through assembly language
  1262. 48:57some basic code like C some higher level
  1263. 49:00code like Python or something and then
  1264. 49:04deal with uh agent kind of work and
  1265. 49:07deterministic tools and finally be able
  1266. 49:09to strategically run an agent under
  1267. 49:12supervision.
  1268. 49:14>> Yeah, [snorts]
  1269. 49:15>> I don't know how better to say that.
  1270. 49:17>> It's so hard though, right? because
  1271. 49:19there's two there's two things going on
  1272. 49:21there, right? Like there's the agent is
  1273. 49:24a kind of abstraction layer, right, over
  1274. 49:26the code. And so you got the code and
  1275. 49:28then it's interesting you were talking
  1276. 49:29about an abstraction layer like a sort
  1277. 49:31of um module viewer, right? Like I think
  1278. 49:34that's super interesting. That's a
  1279. 49:36really good way to learn getting these
  1280. 49:38kind of abstractions over the code so
  1281. 49:40when you dive in you understand it more
  1282. 49:42deeply. But if you have someone who's
  1283. 49:44just like sitting on your books just
  1284. 49:46doing tactical work, surely as a company
  1285. 49:49you're going to look at that and go, why
  1286. 49:51are we hiring this person when we've
  1287. 49:53got, you know, we've got Uncle Bob's
  1288. 49:56swarm of five hardeners and stuff that
  1289. 49:58can beat it, you know, a fraction of the
  1290. 50:01cost, right? I'm I I just I I'm so Okay.
  1291. 50:06What What like resources or books can or
  1292. 50:08approaches or Sorry, there's one more
  1293. 50:11question here which is [laughter]
  1294. 50:15I'm I'm I'm trying to like work out how
  1295. 50:17people get this information because the
  1296. 50:20feedback loop on strategic programming
  1297. 50:22is so long, right? Or traditionally it
  1298. 50:24has been very long. So you you can often
  1299. 50:27>> like people who just like quit jobs
  1300. 50:29after six months or something, they
  1301. 50:32might never learn strategic programming,
  1302. 50:33right? Because their mistakes are maybe
  1303. 50:35nine months away, you know, so they just
  1304. 50:37never see the their own mistakes. But I
  1305. 50:40think with agents, you can, right?
  1306. 50:42Because it's sped up so much, you can
  1307. 50:45actually get more feedback on your
  1308. 50:47mistakes sooner. And I suppose what I'm
  1309. 50:49interested in is how did you know that
  1310. 50:52your agents were making mistakes back in
  1311. 50:54December when you were looking at them
  1312. 50:55and going this is dog? How did you
  1313. 50:57identify that?
  1314. 50:59>> Um early on I was just looking at the
  1315. 51:01code and seeing the dog do. That wasn't
  1316. 51:03the important part. The important part
  1317. 51:05was the next step where I watched them
  1318. 51:07thrash. I could see the agent struggle
  1319. 51:11[snorts] and I recognized the struggle
  1320. 51:14since I have been through that struggle.
  1321. 51:16Right? And that's one of the issues is
  1322. 51:18the novice would come in and not
  1323. 51:19recognize the struggle.
  1324. 51:21>> Now, how do you learn that? How do you
  1325. 51:24learn? You know, I learned it the hard
  1326. 51:25way. School of hard knocks. How do you
  1327. 51:28learn that as a young person coming in?
  1328. 51:30How do you recognize that? There is a
  1329. 51:33wealth of
  1330. 51:35information about this. Um the the old
  1331. 51:39books, the ones that nobody reads
  1332. 51:40because they're old. Uh the old books on
  1333. 51:43this topic are terrific, right? you go
  1334. 51:45to the works by Tom DeMarco or or the
  1335. 51:48works by Ed Yordan or you know go go
  1336. 51:52read you know um gez I can't think of
  1337. 51:56the names right now but
  1338. 51:58>> pragmatic programmer you know the
  1339. 52:01pragmatic programmer um the the
  1340. 52:05there's a lot
  1341. 52:06>> there's a lot of these older books
  1342. 52:08they're very good
  1343. 52:10>> um that if you if you study them when
  1344. 52:13you're young you will get may feel for
  1345. 52:17what this higher level strategic play
  1346. 52:19is. You'll have to filter out some of
  1347. 52:21the archaic stuff because a lot of these
  1348. 52:23books were written in the 70s or the
  1349. 52:2580s, right? But that's when these
  1350. 52:28lessons were learned.
  1351. 52:30And so I, you know, that's that's where
  1352. 52:32I would go initially. I would go to
  1353. 52:35those books and learn that stuff that
  1354. 52:37way. And then, of course, you're going
  1355. 52:38to have to learn it by feeling it.
  1356. 52:40>> Yeah. That's why I think they ought to
  1357. 52:42play play uh agents for a few months so
  1358. 52:45they can learn what that's really like.
  1359. 52:47>> Become the agent. Have the agent
  1360. 52:49delegate to you. Yeah. You become a sub
  1361. 52:50agent of the agent. I like that.
  1362. 52:52[laughter]
  1363. 52:54So, okay. I do have one more question.
  1364. 52:56It's I suppose a short one. Sure.
  1365. 52:58>> Um which is
  1366. 53:01>> sounds like then software fundamentals
  1367. 53:04still matter, right?
  1368. 53:06>> Yeah.
  1369. 53:06>> And why is that?
  1370. 53:09And what do you say to the people who
  1371. 53:11say that they don't matter?
  1372. 53:15>> Why is that? Um
  1373. 53:18software fundamentals matter for the
  1374. 53:21reason they have always mattered. The
  1375. 53:24let's see who said this. It was um
  1376. 53:29uh
  1377. 53:30Dystra I think who said it. I'm I'm
  1378. 53:33going to get this wrong, but
  1379. 53:36software is the most complicated thing
  1380. 53:38that humans have ever attempted to do.
  1381. 53:41More complicated than you know any other
  1382. 53:43task that we've tried. Software is the
  1383. 53:46most complicated thing. And therefore,
  1384. 53:48the fundamentals are the are way of
  1385. 53:51organizing that complexity into a form
  1386. 53:54that can be conceived not just by
  1387. 53:56humans, but by our models as well. Since
  1388. 53:59our models are modeled after humans
  1389. 54:01after all.
  1390. 54:04So the fundamentals still apply because
  1391. 54:07that's the way we organize complexity to
  1392. 54:10be conceived or be to be conceived of.
  1393. 54:14There are folks right now who think that
  1394. 54:16the fundamentals don't matter. They will
  1395. 54:17learn and they will learn that the hard
  1396. 54:19way and and it won't take very long.
  1397. 54:22Might take longer than I think it'll
  1398. 54:24take because the agents are pretty good.
  1399. 54:26But I've watched them hit the hit the uh
  1400. 54:29wall. So I know that wall is there and I
  1401. 54:32don't want to hit that again.
  1402. 54:36>> It's a very interesting interesting set
  1403. 54:39of parallels that we're in at the
  1404. 54:41moment. We you had mentioned the the
  1405. 54:43abstraction layer, right? And you know
  1406. 54:46we're we're now at a at a level where
  1407. 54:48we're above the compiler. We used to our
  1408. 54:51our our abstraction layer used to be the
  1409. 54:53compiler. Before that it was assembly
  1410. 54:55language. Before that it was binary. And
  1411. 54:58now it's up here at this level of the
  1412. 55:00model. At every one of those steps
  1413. 55:04up up the abstraction layer. The people
  1414. 55:07at the at the lower step complaint said,
  1415. 55:10"Oh, this is going to ruin everything.
  1416. 55:12We're not even going to have jobs
  1417. 55:13anymore. It's become so easy that
  1418. 55:17five-year-olds will be able to write the
  1419. 55:18code." And you know, back then they were
  1420. 55:20still talking about like binary.
  1421. 55:22[laughter]
  1422. 55:23Yes.
  1423. 55:24At every step it's the same. And and so
  1424. 55:27we're at this next step and the people
  1425. 55:29down here are saying, "Oh, it's going to
  1426. 55:31ruin everything." No, it's not. The same
  1427. 55:34rules apply. All the same fundamentals
  1428. 55:37exist for all the same reasons. The
  1429. 55:39rules you throw away are the ones you're
  1430. 55:41going to pick up off the floor in a year
  1431. 55:44and dust off and remember why you need
  1432. 55:46them.
  1433. 55:48>> Yeah. Thank you so much, Bob. There's a
  1434. 55:52there's a great quote from I think Plato
  1435. 55:54where he says that writing is going to
  1436. 55:57make people stupider. Right? So people
  1437. 55:59have had that same abstraction argument
  1438. 56:01since you know the Greeks, right?
  1439. 56:03Ridiculous.
  1440. 56:06I think everyone uh watching, how many
  1441. 56:07folks have we got on this stream? Uh
  1442. 56:09we've got 1,500 people watching.
  1443. 56:12Incredible. Uh I think we can all agree
  1444. 56:14that was a fantastic conversation. Um
  1445. 56:16Bob, thank you so much. Um, hold up your
  1446. 56:20book one more time so folks can uh can
  1447. 56:22get it. Let's see. [laughter]
  1448. 56:25Clean code. And I think what is 11:00
  1449. 56:28a.m. now your time? Back to the
  1450. 56:29bathrobe. Uh,
  1451. 56:30>> yeah. Yeah, it is. [laughter]
  1452. 56:33>> Perfect. So, folks, thank you so much.
  1453. 56:35I'm going to end the stream here. Me and
  1454. 56:37Bob are gonna stay on for a

About this transcript

This page contains the full transcript of LIVE: Uncle Bob on Software Fundamentals in the Age of AI by Matt Pocock, generated from the public captions YouTube serves with the video. The transcript has 10,121 words across 1,454 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.