LIVE: Uncle Bob on Software Fundamentals in the Age of AI — Transcript
Full transcript
- 0:00I've got a treat for you today. I've got
- 0:02someone who I've been wanting to speak
- 0:03to for a while. Someone who has been
- 0:05really into um I mean software for as
- 0:08long as I've been a developer, much
- 0:09longer than I've been a developer. And
- 0:11someone who's now making his mark on the
- 0:13agent space. We have uh Uncle Bob Martin
- 0:17live and in his bathrobe ready to throw
- 0:21down. [laughter]
- 0:23Uncle Bob, hello. Welcome.
- 0:26>> Hello and thank you. [laughter]
- 0:28Good to be here.
- 0:30It's great. It's great to have you. It's
- 0:32great to have you. Um, for folks who
- 0:35don't know the bathrobe thing
- 0:37specifically, we should probably caveat
- 0:38that. What's What's going on there? And
- 0:40why is the bathrobe become a important
- 0:42part of your character?
- 0:43>> I I it happened I think two years ago. I
- 0:46was on my front porch in my bathrobe. It
- 0:48was 6:00 in the morning and and I
- 0:50started thinking about how awful SQL was
- 0:53because of all [laughter] the SQL
- 0:55injections and what you know makes no
- 0:58sense to have a textual language as your
- 1:00database access um for security reasons.
- 1:03And I I was in my bathroom at the time
- 1:05and I just pulled out my phone and I
- 1:07ranted and that became the morning
- 1:09bathrobe rant. [laughter]
- 1:12Apparently it was popular so I did it a
- 1:15few more times.
- 1:16>> So is this the mood that you've come
- 1:17here today in? Is this is this the uncle
- 1:19[laughter]
- 1:21>> Bob? It's early in the morning. I
- 1:23haven't had my coffee and just don't
- 1:25bother me.
- 1:27>> 6:00 in the morning thinking about
- 1:28sequel, man. [laughter] 6 in the
- 1:30morning. Come on.
- 1:30>> I'm done with this now. It's actually 10
- 1:32in the morning. [laughter]
- 1:35>> The bathroom off. I have me. I'm on my
- 1:38first Diet Coke, so I'm fine.
- 1:40>> Very good. Okay. Well, now the polo
- 1:43shirt is out. Um what's what's the Uncle
- 1:46Bob's story like for there are probably
- 1:48some folks most of my folks are
- 1:50developers but some of my folks are not
- 1:52developers as well in this audience. How
- 1:54do we introduce Uncle Bob the phenomenon
- 1:56that is Uncle Bob to especially
- 1:57non-developers.
- 1:59>> Phenomenon. I don't know about that. Um
- 2:01I'm a programmer. I I've been a
- 2:03programmer for a very long time. Uh over
- 2:07half a century at this point. uh started
- 2:10it. My first program was in 1964
- 2:15and I was 12. [laughter]
- 2:20The program was was uh a little model
- 2:23computer that my mother had bought me
- 2:25for my 12th birthday and programmed it
- 2:28by putting little white tubes on pegs.
- 2:31It was essentially a three-bit finite
- 2:33state machine, but it fascinated the
- 2:35hell out of me at the age of 12.
- 2:38Okay. So, you're 12 years old. How do we
- 2:40get to uh 50 years later, Uncle Bob
- 2:43doing his thing now?
- 2:44>> Yeah, a little more than 50. Um, let's
- 2:46see. Well, I I you know, I just started
- 2:50learning as much as I could about
- 2:52programming. My father bought me a book
- 2:54about Forran, a book about Cobalt, book
- 2:56about PL1. I read them all. Had no
- 2:59machines to execute anything on, so I
- 3:01wrote programs on paper and executed
- 3:04them in my head. got a job uh um writing
- 3:08a little bit of code at the age of 16.
- 3:11Um but that was a temporary thing. Then
- 3:13I got a real job at the age of 18 and
- 3:15I've been a programmer ever since.
- 3:17>> So you've been in the trenches for a
- 3:19long time and you wrote rather an
- 3:21important book I think.
- 3:22>> A few I one of them I think was
- 3:25important. I've written a few more but
- 3:27uh yeah one of them seemed to take off.
- 3:29That was a nice one.
- 3:30>> Yeah. and um uh the cleanliness of code
- 3:33if I remember correctly or
- 3:35>> yeah clean code. Yeah. Yeah. Yeah.
- 3:38>> Here's the second edition. This is this
- 3:40is the second edition. But you know
- 3:44>> I mean it's it's maybe the most um
- 3:46certainly when I have conversations
- 3:47about good books. It's it's uh for
- 3:50software engineering. It's the one that
- 3:52most often gets quoted back at me. Like
- 3:54it's incredibly popular, incredibly
- 3:56influential. So, and that's why I'm
- 3:58interested to talk to you today because
- 4:01you've had this big influence in like
- 4:03the preAI era and you've been working in
- 4:07the pre-AI era for almost as long as
- 4:09it's existed virtually now.
- 4:13How have things changed for you, Uncle
- 4:15Bob, now that AI is out there and is a
- 4:18thing? Um, so this took me a little by
- 4:22surprise
- 4:24around
- 4:25December of of last year, Christmas
- 4:29time, and I'm just fiddling around and I
- 4:32I'd already been playing with, you know,
- 4:34chaty PT and a little bit of Grock and a
- 4:37little bit of this and a little bit of
- 4:38that. And I hadn't been real impressed.
- 4:41Uh, and then I went through a period
- 4:43where I thought, well, maybe these
- 4:44things are a little more interesting
- 4:46than I thought. And I I got an agent.
- 4:50First agent I got I think was um Grock
- 4:53early Grock. It was an early Grock one
- 4:56and I just asked it to write me some
- 4:58code and it did a kind of a poor job but
- 5:02it wrote the code and I was in the
- 5:04middle of a project at the time. So I
- 5:06said well maybe this thing can help me
- 5:07with this project and I started having
- 5:10it work with me on the project and I was
- 5:13cleaning up after it all the time just
- 5:14you know it was always making a mess.
- 5:16was always leaving little dog dude
- 5:18behind. And I thought, you know, it's
- 5:21it's interesting because it's fast, but
- 5:25it's frustrating because it makes me
- 5:28slow.
- 5:30>> Yep.
- 5:31>> And then I started thinking, well, wait
- 5:33a minute,
- 5:35because it's fast, it can do things that
- 5:38I cannot. So in the uh very early 2000s
- 5:43there were a couple of innovations that
- 5:45that caught me. I thought oo these are
- 5:47good ideas but they were completely
- 5:50impractical. One of them was called um
- 5:53uh crap. Um it was uh it's an acronym.
- 5:58What can I say? But it's it was a way to
- 6:01take uh code coverage. So you would you
- 6:04would run code coverage, test coverage
- 6:07over your code and you'd also measure
- 6:09the cyclatic complexity of every
- 6:11function and you would mix those two in
- 6:14a complicated formula and out would come
- 6:16a score and the score was a measure of
- 6:21how crappy your function was. And uh way
- 6:24back in the early 2000s, I thought this
- 6:27is a great idea. And I ran it over a big
- 6:29project I was working on. And yeah,
- 6:31there was a bunch of crappy functions,
- 6:32but then it took me forever to go
- 6:35through every one of those functions and
- 6:37try and fix them and rewrite the tests
- 6:39and but and it was all working. So I
- 6:41didn't think I really could afford to do
- 6:43that. So although it was interesting, I
- 6:46kind of set it aside. And another one of
- 6:49those um innovations was called mutation
- 6:52testing. And this this one really caught
- 6:54my attention, too, because in mutation
- 6:56testing, you you have a little a little
- 6:59program that runs through your source
- 7:01code and flips negative signs to
- 7:04positive signs and less than signs to
- 7:06greater signs and equal signs to not
- 7:08equal signs and other things as well.
- 7:10And for each of those flips, it runs
- 7:13your entire test suite and expects the
- 7:16test suite to fail because obviously
- 7:19they've changed something significant.
- 7:21And if it doesn't fail, well, that's a
- 7:23surviving mutant and it must be killed.
- 7:25Now, I ran this again over that same
- 7:28project right around the year 2000. And
- 7:31I had to run it overnight because the
- 7:33test suite ran for four minutes and I
- 7:35was running it maybe several hundred
- 7:37times. Uh, but it came up with a bunch
- 7:40of surviving mutants which I was able to
- 7:42fix. But once again, it was impractical.
- 7:45I could not put that as part of a a
- 7:47normal build scenario. And so I'm
- 7:50sitting there last December or maybe it
- 7:52was January by this time and I'm
- 7:54thinking,"Well, wait a minute. These
- 7:56guys are fast and they don't care how
- 7:58boring the work is
- 8:02and they will do what I tell them to do.
- 8:04So why don't you run crap over
- 8:06everything you've just done and it would
- 8:09run crap and then it would it would
- 8:10clean up the code." And I was watching
- 8:12it do this. I think, well, that's pretty
- 8:14cool. And why don't you run mutation
- 8:16testing, too? and it would run the
- 8:18mutation testing. Maybe it took it 30
- 8:20minutes instead of an overnight run and
- 8:23then it would plug all the holes and
- 8:25make sure there were tests covering
- 8:26everything. I thought, you know,
- 8:30this might be a way to clean up the dog
- 8:32dew. These things leave a lot of deus
- 8:35and fluff behind, but maybe this is a
- 8:37good way to clean up all the dog. So I I
- 8:40continued on this path of adding more
- 8:42tools and continuing to work with these
- 8:45agents and I got them to a place where
- 8:47they were doing a pretty good job. So
- 8:49nowadays my nowadays my principle is,
- 8:53you know, I will set the agents working.
- 8:56I will have them run these tools and I'm
- 8:58going to work very hard to get it into a
- 9:02situation where I don't have to look at
- 9:04the code at all.
- 9:06>> I can trust what they do. And now I do
- 9:08other things to verify that the code is
- 9:11still decent. You know, I will look at
- 9:13the crap scores and make sure that
- 9:15they're low. And I will I will do spot
- 9:17checks on the code from time to time.
- 9:19And I have a whole bunch of other tests
- 9:21that I run. But overall, that's my goal.
- 9:23If these things are fast, and they are,
- 9:28and if I can constrain them to do a good
- 9:30job, then I am not going to impose my
- 9:33slowness
- 9:35upon them. And my bonus is reviewing the
- 9:39code or inspecting the code at some kind
- 9:42of level of detail. They are fast with
- 9:45code. I am slow with code. So I'm going
- 9:47to let them have the code and I'm going
- 9:49to deal with the stuff around that to
- 9:51make sure it's all okay. So far so good.
- 9:55So your goal is to
- 9:58start pulling yourself away from the
- 10:00code, away from human review to
- 10:04construct this scaffolding around the
- 10:06code so that you don't have to interact
- 10:08with it much yourself and so that the
- 10:11agent is constrained as much as possible
- 10:14in as tight a straight jacket as
- 10:16possible so that it can't make a
- 10:18mistake. I think there's so there's an
- 10:20assumption there that I want to touch on
- 10:22first which is that
- 10:25the word dog do right doesn't sound good
- 10:28in in a British accent dog do you know
- 10:30so so what why is why is dog do bad why
- 10:37is bad code bad like why why do we care
- 10:41about that because these things are so
- 10:42fast and they can move so quickly why do
- 10:45we care why are we bothering with all of
- 10:47this harness can't we just push through
- 10:49until the bugs just disappear. One of
- 10:52the things I noticed really early on,
- 10:55like Decemberish time frame, was that I
- 10:57would be I'd be working with the the uh
- 10:59like this little Grock agent and I'd
- 11:02have it do something and it would leave
- 11:04all this mess around and instead of
- 11:06cleaning the mess, I would have it do
- 11:07the next thing and it would leave even
- 11:09more mess around and then I would have
- 11:10it do the next thing. And I noticed it
- 11:13slowing down and I noticed it having
- 11:16having uh difficulty. It would it would
- 11:20get into a mode where it would change
- 11:22one thing but inadvertently break
- 11:24another and then it would have to fix
- 11:25that but inadvertently break another.
- 11:27Started going around in circles and I
- 11:30thought okay these agents they may be
- 11:33they may be fast and they be maybe
- 11:35relatively smart but they are as subject
- 11:38as humans are to messy code. Now maybe
- 11:42not as subject. Maybe there's a
- 11:45difference in threshold, but the
- 11:47threshold is still there. The code can
- 11:49get messy enough that the agents cannot
- 11:52deal with it any any longer and then
- 11:54they'll just start to spin and make a
- 11:57mess even worse and and be unable. I've
- 11:59actually had them just give up. One
- 12:02agent one time said, "I just can't deal
- 12:04with this anymore."
- 12:07Parap I'm paraphrasing it. didn't use
- 12:08those words, but that was obviously what
- 12:10was going on.
- 12:11>> Yeah. So, your response to that then is,
- 12:15"How can I clean up the crap?" And I
- 12:18think when most people encounter that,
- 12:20they start loading the agent with
- 12:22instructions, right? They start saying,
- 12:24"Okay, I'm going to pour in information
- 12:27into the agent that's doing the
- 12:28implementation, right? I'm going to pile
- 12:30in on claw.md or agents.mmd and every
- 12:33time I see something bad I'm going to
- 12:36use an instruction instead of what
- 12:38you're doing where it's a deterministic
- 12:40mechanism. So you're using automated
- 12:43checks whereas a lot of other people are
- 12:45using in my terminology steering right
- 12:47they're trying to steer it. So why
- 12:49didn't you are you doing any steering or
- 12:52like how does that work in your mindset?
- 12:55So, initially I started with that. I
- 12:57start, you know, I my early prompts were
- 13:00here's how you do test-driven
- 13:01development. Here's how you do clean
- 13:03code. Here's what your code should look
- 13:05like. You should follow all these rules.
- 13:07And, you know, you come up with
- 13:08eventually a document that's five or 10
- 13:10pages long describing all the good
- 13:13things about code. I I suppose I could
- 13:15have fed the whole dog on book into it.
- 13:18Mhm.
- 13:18>> But what I noticed was that the agents,
- 13:22the the models, whatever you want to
- 13:24call them, um they treat those rules
- 13:28in the uh Pirates of the Caribbean
- 13:31sense. They're more like guidelines,
- 13:34you know, might follow.
- 13:36>> Can I just Can I just credit you? That's
- 13:38exactly the metaphor that came into my
- 13:39head as well. So, I'm glad we're aligned
- 13:40there. [laughter]
- 13:42>> Yeah. Well, so they they definitely will
- 13:44soften and there's technical reasons
- 13:46behind this. I did a not not a lot of
- 13:50research but some research into why the
- 13:52models behave this way and it turns out
- 13:55that there's this phenomenon known as
- 13:58lost in the middle. So as the context
- 14:02window builds up inside the model the
- 14:06stuff at the very beginning and the
- 14:08stuff at the very end have more
- 14:10prominence than the stuff in the middle
- 14:13uh for technical reasons. But once
- 14:16again, the the agent will ignore the
- 14:18stuff in the middle. And anything you
- 14:20say at the very beginning is going to
- 14:23get shoved into the middle if it's long,
- 14:26right? So maybe the first three
- 14:27sentences you put at the beginning will
- 14:29remain as priority, but the 50th and the
- 14:3380th sentence in there, they're gone.
- 14:35They're just in the middle somewhere.
- 14:37And the poor agent is trying to deal
- 14:40with this massive context that's ever
- 14:43growing, ever growing. and it's trying
- 14:45to pull the important bits out and the
- 14:47stuff in the middle is just gone. So
- 14:50deterministic tools don't disappear that
- 14:54way.
- 14:56The the key that I think with agents,
- 14:58and this is really hard to do, the key
- 15:00with agents is to trim that initial
- 15:02prompt down to its absolute minimum so
- 15:06that you can get as much of it as
- 15:08possible into its priority.
- 15:12>> Yeah.
- 15:12>> Right. and then do deterministic tools
- 15:14after the fact.
- 15:16>> Totally. I refer to this as like the
- 15:18smart zone and the dumb zone of the uh
- 15:21context window. It's not my term. That's
- 15:23Dex Hardy's term. I stole it and it's
- 15:24very very good. Which is especially at
- 15:26the early part of the context window,
- 15:28like the first 150k tokens, it's pretty
- 15:30smart. But then as you go along, then
- 15:32the attention relationships in the
- 15:34transformer get really strained. It's
- 15:36really diluted. It's like everyone's
- 15:38shouting. Each token is shouting in a
- 15:40crowded room and the room's getting more
- 15:41crowded. Right? you can't hear the
- 15:43signal for the noise. That that's that
- 15:45totally rings true for me. So you've got
- 15:48then you sort of rejected steering and
- 15:50you sort of started going back to some
- 15:53old techniques in automated checks. Now
- 15:58it sounds like then because those checks
- 16:00don't go into the context window in the
- 16:02same way that steering instructions do,
- 16:03you can kind of just layer those on,
- 16:05right? And layer them up and up and up.
- 16:07So if you're using a language with
- 16:08strong types and like tests and like how
- 16:13does that picture look to you? Is there
- 16:14ever too much when it comes to automated
- 16:17checks?
- 16:18>> That's one of the things that I'm
- 16:19working very hard to figure out.
- 16:21Obviously there has to be a case where
- 16:23there's too much, right? You eventually
- 16:25you will slow the agents down to the
- 16:27point where they're slower than humans.
- 16:28And at that point you've lost the game.
- 16:31>> Why do it? Um, but as long as you can
- 16:34keep the margin of productivity higher
- 16:37than a human, you're still you're still
- 16:39ahead of the game. Uh, and and from what
- 16:42I've seen so far, I you know, I can get
- 16:45that margin, you know, like a factor of
- 16:47two or three or four, right? They'll
- 16:50they'll still go pretty fast. Now, I'm
- 16:52I'm slowing them down a lot. When when
- 16:55uh when you use these deterministic
- 16:57tools, what you are really doing is
- 16:58you're putting them into a loop. Lots of
- 17:00people like to talk about loops
- 17:01nowadays. You're putting them into a
- 17:03loop and you're saying, "Okay, you must
- 17:05you must change the code until this tool
- 17:08says that it's okay." And now the agent
- 17:11is going around and around and around.
- 17:13Okay, I've got to do this. I've got to
- 17:14do that. I've got to I've got to add
- 17:15more tests over here. I've got to cut
- 17:17the cyclatic complexity. I've got to
- 17:19split these functions apart. Got to do
- 17:21all this work. And it takes it a while
- 17:23to do all that until it gets it into
- 17:25conformance. So you're you're you're you
- 17:28are sacrificing productivity for higher
- 17:32quality
- 17:33and at some point that's got to give
- 17:35way. But I haven't found the end point
- 17:37of that yet.
- 17:39>> So now I'm in the midst of trying to get
- 17:41multiple agents talking to each other
- 17:44and handing off to each other so that
- 17:46one guy does one thing and the next one
- 17:48reviews it and the next one tests it and
- 17:50the next one hardens it and so on. And
- 17:52there's communication overhead like
- 17:54crazy in that. and yet it's still faster
- 17:57by a large token than a human.
- 18:01>> Let's talk about that. Let's talk about
- 18:02multi- aent systems because that's I
- 18:04find that super fascinating. I have
- 18:06always had a bit of a and I think this
- 18:08might be something something where we
- 18:09differ
- 18:10>> which is I've always had a bit of a
- 18:11suspicion of like people who say uh you
- 18:15know I've I've got rid of all my
- 18:18workforce. I now have a hundred agents.
- 18:21Every one of them has a different role.
- 18:22They all talk to each other. they've got
- 18:24their own email accounts, all that all
- 18:26that rubbish, you know. Um, and so for
- 18:29me, what's always worked, but okay, I
- 18:31sort of make an exception when it comes
- 18:33to implementation and then review,
- 18:36right? And I think the benefit there is
- 18:38not having two very specialized agents.
- 18:40It's having okay, you do the
- 18:41implementation and it's kind of like a
- 18:43red green refactor approach, which is
- 18:46the implement all it has to do is just
- 18:48like write the bad test and then make it
- 18:51work. It doesn't have to make it
- 18:52beautiful. And then the reviewer comes
- 18:55in. It doesn't have to do the
- 18:56exploration because it's already got the
- 18:58diff from the implementer. It knows
- 19:00exactly what it's reviewing. And then
- 19:02you can pile in quite a lot more
- 19:03steering instructions in there because
- 19:04the task is much less constrained. So
- 19:08I'm really interested in your take on
- 19:09that on like what the implementer does,
- 19:12what the reviewer does, and what these
- 19:14other hardener agents. I've not
- 19:15experimented with that. Give me more on
- 19:17that. So there's two advantages to
- 19:19having multiple agents like this. The
- 19:21one is that you can run them in
- 19:23parallel. So you could have, you know,
- 19:26three coders running at the same time.
- 19:28And my little laptop can support a lot
- 19:31more than three. Um the other advantage
- 19:34is that when you focus the agents down
- 19:37to a single task, you're keeping the
- 19:39context window under control. The lost
- 19:41in the middle problem becomes much less
- 19:43of a problem. So you can pile a few
- 19:46more, not a lot more, but a few more
- 19:48rules up at the top and they'll tend to
- 19:50follow them better. You can also set up
- 19:53a a system where the agents um are born,
- 19:57do the task, and die so that the next
- 20:00one comes in with a clean context.
- 20:03So [clears throat] those are the
- 20:04advantages. The disadvantages are that
- 20:06the startup times are high, right? So an
- 20:08agent takes, you know, 10 15 seconds to
- 20:11even start up. uh and then it's got to
- 20:14figure out its whole context all over
- 20:16again. So there's that that particular
- 20:18startup time as well. I like to focus
- 20:21the task as much as I can. So I will run
- 20:24a specifier. The job of the specifier is
- 20:27to take a a human written document
- 20:32and turn it into a uh a girkin
- 20:37and a QA a QA uh procedure. Girkin is
- 20:41you know given when then stuff. It's a
- 20:43high level acceptance test. Uh and a QA
- 20:46procedure is a essentially a system
- 20:49test. You know you run the system
- 20:51through the UI with a with a QA
- 20:53procedure. And I have them write it from
- 20:56a human's point of view. You are a
- 20:58human. You are operating this system at
- 21:00the UI. You must prove that the system
- 21:02works.
- 21:04And I will produce those two documents.
- 21:06Girkin documents QA documents. And then
- 21:09those feed into a coder. The coder's job
- 21:12is to write unit tests and the code that
- 21:15implements the described uh story and uh
- 21:21also get the girkin working. So it has
- 21:23to do that and then that gets fed over.
- 21:25Once that's working, the coder will work
- 21:28on that for a little while that gets fed
- 21:30into a cleaner and the cleaner's job is
- 21:33to run crap analysis and just general
- 21:36code review. clean it clean up whatever
- 21:39mess the implement made because the
- 21:41implement will have made a horrible mess
- 21:42by that point. And then I have it go
- 21:45from there to a hardener. The hardener
- 21:48is the guy who runs the mutation testing
- 21:50and he's absolutely merciless,
- 21:52[laughter] right?
- 21:54It's going to mutate it and it's going
- 21:57to have 100% coverage and every equal
- 22:01sign and every less it's going to do
- 22:03that work which takes a good long time.
- 22:06uh and it pops out the end and then it
- 22:08goes into a QA agent and the QA agent
- 22:10takes the the written QA document, turns
- 22:14it into an executable script that
- 22:17manipulates the system and comes up with
- 22:20a deterministic result. And and if you
- 22:23can get through all of that,
- 22:25you've got a pretty pretty working
- 22:27program. I' I've had a lot of success
- 22:29with this. it uh if I if I give it a
- 22:33task that takes a single agent five
- 22:38minutes to complete with questionable
- 22:40results. Uh this will take it about an
- 22:43hour. It'll take about an hour to go
- 22:45through all that which is still a
- 22:47benefit because you know a person would
- 22:49take about a half a day. So maybe I've
- 22:52got a you know a factor of four factor
- 22:54of five improvement in productivity and
- 22:56very high quality much more quality than
- 22:58the human would ever put into it. And
- 23:00it's not only that, but you're you're
- 23:02saving productivity or or you're
- 23:04spending productivity early to gain it
- 23:06later, right? This is an investment in
- 23:08your own codebase. Yeah. What I found
- 23:11really interesting about that is this is
- 23:13something I've been thinking about
- 23:14recently is it's not only the context
- 23:16window that you're manipulating. You're
- 23:18also there's this idea of a trajectory
- 23:22of a context window, right? Of a session
- 23:24where if you get the agent to do one
- 23:27thing and you steer it in a certain way,
- 23:29then everything that follows in that
- 23:31same session, that same context window,
- 23:33will continue following that trajectory.
- 23:35So if you get it to say, "Okay, maybe we
- 23:37should test the UI here," then every
- 23:38single time it will test the UI again,
- 23:40no matter how many changes you get it to
- 23:42make. And the only way to clear the
- 23:44trajectory is to clear the context
- 23:45window, right? So if you're an implement
- 23:48agent and you're just trying to get it
- 23:49working, then your trajectory is kind of
- 23:52it's not quite as harsh as the one who's
- 23:55trying to make sure you have 100%
- 23:57coverage. And I found that really
- 24:00interesting. Is that something you does
- 24:01that click with your mental model as
- 24:02well?
- 24:03>> Yeah, it certainly does. There's a a
- 24:05pretty well-known effect with these
- 24:06models and it happens with people who
- 24:09aren't programmers, right? So, you're
- 24:10talking to a model about um oh, I don't
- 24:14know what what's the best coffee to have
- 24:17and how do you how do you brew it
- 24:19nicely? And you're having this nice
- 24:20little conversation with the agent about
- 24:23this or with the model about this. And
- 24:25then somebody walks by and they happen
- 24:27to be talking about the latest soap
- 24:29opera that they saw saw on television
- 24:31and that gets into the context window.
- 24:34You didn't put it there. This guy
- 24:35walking by put it in there. But then
- 24:37from that point on all the coffee
- 24:39references have to do with the soap
- 24:41opera, right? The the the model doesn't
- 24:44know. It can't differentiate. So your
- 24:46idea of trajectory is pretty good,
- 24:48right? As long as you can keep the
- 24:50direction of the model unconfused.
- 24:54It knows the direction it's going in.
- 24:56Everything in the context window is
- 24:58consistent.
- 25:00then it's not going to have these crazy
- 25:02hallucinations that that people often
- 25:06deal with
- 25:07>> or even just mis even just misalignment
- 25:09right like halluc I think hallucination
- 25:11is always a danger right like whether
- 25:13it's you know hallucination context you
- 25:14gave it whatever like it's an sort of an
- 25:16accepted danger and your approach of
- 25:18this kind of gauntlet of different
- 25:20agents slamming horrendous stuff is sort
- 25:23of a is a great way of taking out that
- 25:26so okay you've talked a lot about the
- 25:29implementation phase and we can get back
- 25:31to that later too. What I'm also
- 25:33interested in is what planning do you do
- 25:37up front and especially
- 25:39how are you thinking about the internal
- 25:42structure of your code bases? Because
- 25:46having a good test suite, right? I mean,
- 25:48I'm interested in what you think here.
- 25:49Having a good test suite is good, right?
- 25:51Having mutation testing, all that stuff
- 25:53is good. But if you've got badly
- 25:54designed APIs, if you've got badly
- 25:57shaped modules, how does that interact
- 26:01with all of this automated checks and
- 26:03stuff like how much are you thinking
- 26:05about that?
- 26:06>> So up to the last month or so, what I
- 26:10was doing there was doing that part
- 26:12manually. So I would I would have the
- 26:15agents build me up a nice thing and then
- 26:18I would interrogate the agents. And this
- 26:20was manual but but still using the
- 26:22agents, right? I'd interrogate the
- 26:24agents. What's the structure here? How
- 26:27how does this module interrelate with
- 26:29that module? What are the modules after
- 26:32all? And how do they talk to each other?
- 26:34I would ask those questions and then I
- 26:37would get scared to death because the
- 26:39answers were horribly frightening. And
- 26:41then I would design a module structure
- 26:44and I would tell the agent, okay, here's
- 26:46how the modules should really be
- 26:47partitioned and here's how you should
- 26:49communicate with them. And I would give
- 26:51them an implementation plan that they
- 26:53would then implement.
- 26:55Now, that's a that's a tough one. So I
- 26:57also had my agents build me an
- 27:00architecture viewer so I can pop up on
- 27:03the screen a nice little UML diagram
- 27:06essentially nice little UML diagram that
- 27:08shows me the modular structure of the
- 27:10system and where the dependencies run
- 27:12and I can click on a module and I can
- 27:14see inside it to the subm modules and I
- 27:16can click on the subm modules and it'll
- 27:18actually pop the code up on on the
- 27:20screen for me. So I can I can drill down
- 27:22as much as I want and view the system
- 27:25architecture at any level. That was
- 27:28really useful to me and I I've made good
- 27:31use of that. I've also put together a
- 27:34another deterministic tool where I can
- 27:37define which module should depend on
- 27:39which, which one should not depend on
- 27:41which, how the dependency should flow.
- 27:44That goes into a nice tight little
- 27:45specification file that the agents
- 27:48cannot violate. There's another little
- 27:50checker that runs at the end and if they
- 27:53violate it, they've got to fix it
- 27:54somehow. Usually by inverting a
- 27:56dependency or inserting an interface or
- 27:59splitting a module in half or something
- 28:00like that and it will it will um keep my
- 28:04rules
- 28:06um from being violated.
- 28:08I'm working now to see if I can automate
- 28:11that and I'm having not a lot of luck
- 28:14[laughter]
- 28:15so far.
- 28:17I'm basically in exactly the same boat
- 28:20as you, which is you just get so much
- 28:23leverage by having well-designed
- 28:24modules, right? Could you explain what
- 28:27that leverage is? Why is why is having a
- 28:30good structure for these modules
- 28:32important?
- 28:33>> Well, it's the same argument that we had
- 28:35for the dirty code versus clean code.
- 28:38Anything that is well partitioned with
- 28:40welld disciplined interfaces between it
- 28:43is something a human can grasp because
- 28:45we compartmentalize in our minds well so
- 28:48do the models so do the agents maybe at
- 28:51a slightly different threshold I don't
- 28:53know about that yet but they work far
- 28:55better if they can focus on a module and
- 28:58if that module has a trajectory right
- 29:01using your term so that the the model
- 29:04does not get confused [clears throat]
- 29:07by the topics inside that module. Okay,
- 29:11I'm using the word model and module. I
- 29:13want [laughter]
- 29:14clear.
- 29:15>> We get it. We get it.
- 29:16>> But that that's important. And it's the
- 29:18same argument. It's the coffee and soap
- 29:20op opera argument. If you load up a
- 29:23module with every bit of stuff under the
- 29:27of under the sun, the the poor agent is
- 29:30going to wonder, "What the heck am I
- 29:32doing in here? How do I do anything in
- 29:34here?" if you compartmentalize nicely
- 29:38works pretty well just like a human.
- 29:40>> So you're getting and you're probably
- 29:42getting better value out of your test
- 29:43suite as well because that's what I
- 29:45always think is like okay I suppose
- 29:47here's here's another question for you
- 29:48that's related to this I think of and I
- 29:52have to say I'm a huge fan of um huge
- 29:54fan of your work but I'm also a huge fan
- 29:55of John Aster's work. Yeah
- 29:57>> where um I mean who isn't a great guy
- 30:00you you probably got his book around
- 30:01somewhere. Um and his concept of deep
- 30:04modules is something I find really
- 30:06fascinating which is you have an you can
- 30:08have bad modules which is kind of
- 30:10shallow modules right that have a wide
- 30:12interface and not much hidden inside
- 30:14them and you can also have deep modules
- 30:16which have a small interface and then a
- 30:17deep um lots of hidden information
- 30:20inside them and it occurs to me that
- 30:22that is really good with models because
- 30:24they can read the interface without
- 30:27having to understand the implementation.
- 30:29Does that sort of chime with you? And
- 30:31it's not how you're approaching things
- 30:32as well.
- 30:32>> Yeah,
- 30:32>> absolutely. Does
- 30:34>> the models pay attention to interface
- 30:35names? They they pay attention to the
- 30:37structure. It can allow them to not read
- 30:40the code beneath them, which is both a
- 30:42danger and and an advantage.
- 30:45>> Y
- 30:45>> as long as the code is consistent,
- 30:46you're okay. Um they also pay attention
- 30:49to the tests. They read tests to
- 30:52understand what the system does. Um so
- 30:55yeah all anything you can do that helps
- 30:57the structure of the code will help the
- 31:01models understand that code. By the way
- 31:04in this book there is a long debate
- 31:06between me and John Ora in the appendix
- 31:09and it it was a lot of fun. He and I had
- 31:11a blast. Well I don't know how much fun
- 31:12he had but I had a lot of fun.
- 31:15>> I watched I watched the entire thing. I
- 31:16saw you guys interviewed um on a really
- 31:19great uh uh YouTube discussion where you
- 31:21talked about it and I I I I just loved
- 31:24it and I that's kind of the reason I
- 31:25wanted you to come on actually because I
- 31:26I just enjoyed that so much. Um okay,
- 31:30there's a few different ways I could go
- 31:32now. Um
- 31:35okay. Is there in your book anything
- 31:40that you would change or update now?
- 31:43because specifically I'm thinking about
- 31:45some advice like um create small
- 31:47functions let's say and keep functions
- 31:50small um is there is there anything that
- 31:54you're sort of because we've been
- 31:56talking a lot about how things have not
- 31:58changed right how gauntlets like this
- 32:00have always been good we've just never
- 32:02had the labor available to actually push
- 32:05through them right these ideas have been
- 32:07good for a long time we're just sort of
- 32:08modifying them in a different way but is
- 32:10there anything that we just need to and
- 32:12this is I suppose a wide question. Is
- 32:14there anything that we need to throw out
- 32:16that's just like done or is there
- 32:18anything that's I don't like asking this
- 32:21question because it puts a lot of
- 32:22pressure on you, but do you have an
- 32:23answer for that one?
- 32:24>> So, um thresholds for one thing. Uh it
- 32:28seems to me that the the agents can deal
- 32:30with different levels of complexity than
- 32:33humans. They have a a much better
- 32:36short-term memory. I mean, a huge
- 32:38short-term memory and a perfectly
- 32:40accurate short-term memory. So one of
- 32:43the things that I do is I widen the um
- 32:46the allowed size of a function and I do
- 32:49that by adjusting the crap score. So for
- 32:52a human I would keep crap numbers below
- 32:54four, right? But for the agents I've set
- 32:58this at six and I'm thinking and maybe
- 32:59I'll push it to eight. Um I'm trying to
- 33:02find where the threshold is and it's not
- 33:04an easy threshold to find. But
- 33:06>> what what does that look like in terms
- 33:07of four to eight? Like what's the or 4
- 33:09to 12? What's the difference there? like
- 33:10is it like a 20 line 100line function?
- 33:13>> Um so it really boils down to the
- 33:15cyclatic complexity which is the number
- 33:16of pathways through the function. And uh
- 33:19if you've got 100% coverage then a crap
- 33:23score of six means that there are six
- 33:25pathways through the through the
- 33:27function. They're all covered with
- 33:28tests. So that's really the goal of
- 33:31crap, right? Get it all covered with
- 33:33tests and then limit the cyclomatic
- 33:34complexity. Um, and you know, I've had a
- 33:37number of debates with the agents, and
- 33:38by the way, you can't trust any debate
- 33:40you have with an agent, but I still have
- 33:41them anyway. Uh, and they they seem to
- 33:44think that, oh yeah, you know, six is
- 33:46probably pretty good. Well, you know, I
- 33:48don't you don't really trust them, but
- 33:50okay. Um, but I, you know, I think
- 33:53there's a threshold difference there.
- 33:56There's another another factor. In my
- 33:58books, I talk about disciplines. One of
- 34:01them was was test-driven development.
- 34:03I'm a big advocate of test-driven
- 34:05development. But that's a human
- 34:07discipline that's done because humans
- 34:09are wired a certain way. I cannot and
- 34:13will not enforce that on the agents. I
- 34:16don't think it makes any sense to make
- 34:18an agent write a single line of a test
- 34:23and then write a single line of the
- 34:25production code and then the next line
- 34:27of the test. I don't think that makes
- 34:29any sense. For a human, it does. At
- 34:31least for me. I mean there's a a huge
- 34:33benefit for my work but for the agents I
- 34:37don't think so. So I allow the agents to
- 34:39behave more like John Asterhow would
- 34:41which is to write a function and then
- 34:43write the test for that function and
- 34:45then write the next function and write
- 34:46the test for that function. I allow them
- 34:48to do that
- 34:50even when I have told them to do
- 34:52test-driven development at high
- 34:54discipline. They always fall back on
- 34:56doing that. They always end up doing
- 34:58that. So I figure that's probably okay.
- 35:01So the bottom line there is it's
- 35:04probably a mistake to impose a human
- 35:07discipline on an agent. It is not a
- 35:10mistake to impose human values
- 35:13on the agent, but there may be
- 35:15thresholds that we need to change. But
- 35:18the disciplines themselves, the
- 35:19behaviors, I don't think it's wise to
- 35:21impose those.
- 35:23>> That's a lovely way of phrasing it. I
- 35:24really like that. So it's it's a
- 35:26short-term memory thing, right? Because
- 35:27TDD is great when you have very low
- 35:30short-term memory, like humans, right?
- 35:32You have enough short-term memory to
- 35:33write the test and then enough
- 35:35short-term memory, just enough to make
- 35:37the test pass, right? You can go out and
- 35:39get a coffee or something.
- 35:42>> So, okay, that was great. Thank you.
- 35:45[laughter] You're welcome.
- 35:46>> In the
- 35:48Okay, we've talked a lot about building
- 35:51the thing, right? Right. [clears throat]
- 35:53and building the implementation, right,
- 35:55and doing module architecture and that
- 35:57sort of thing. What do you do before you
- 36:00kick things off to your specifier? How
- 36:02much planning are you doing before you
- 36:04actually go into this gauntlet loop?
- 36:06Because like putting the wrong work in a
- 36:10gauntlet is super wasteful, right? How
- 36:14do you think about that? Well, so the
- 36:16temptation
- 36:18is to specify,
- 36:20you know, get the human to specify,
- 36:22specify, specify, specify and then give
- 36:25it to the agent. This is a very old
- 36:27temptation. It was a temptation we
- 36:29underwent in the 70s. It led us to the
- 36:32waterfall uh process and all of that.
- 36:36And and the agile revolution was the
- 36:39answer to that or the answer back to
- 36:42that. I don't know how how well we're
- 36:44doing with the agile revolution either,
- 36:46but but it was the it was a way to say,
- 36:49"Wait a minute. All this heavy upfront
- 36:51planning makes a mess of everything cuz
- 36:55what comes out at the end doesn't look
- 36:57anything like the plan and never has."
- 36:59Well, the temptation with agents is to
- 37:01do the same thing, right? Okay, we're
- 37:03going to plan plan. Then we'll give it
- 37:06to the agent. And I have tried this. In
- 37:09fact, I've been in the middle of trying
- 37:10this just this week and it's it's always
- 37:13a disaster. It and it's always the same
- 37:16outcome, right? The you make all these
- 37:19plans and then as the agents are
- 37:21running, you the human realize that they
- 37:24can't follow that plan because you
- 37:27didn't think of everything and they're
- 37:28not as wise as you are. So they're
- 37:31running halfcocked off on some nonsense
- 37:33that you have to stop, back up, rewrite
- 37:37the plan, and then start them over
- 37:39again. And so I've given up on that. I
- 37:42said, "Okay, wait a minute. Let's try
- 37:44the agile approach now." And I don't
- 37:46know how this is going to work out. It
- 37:48might not work out all that well, but
- 37:50but let's try the agile approach. Let's
- 37:52just let them do a story or two, and
- 37:54then we'll look at the architecture at
- 37:56the end, and we maybe I'll have to
- 37:58manually get involved. and you know sort
- 38:01a few things out and then a few more
- 38:03stories and so on. That might be a
- 38:06better approach. We we may never uh
- 38:10escape that manual organizing step at
- 38:13the end. Although I'm trying to figure
- 38:15out a way to do it, but I don't know I
- 38:17don't know if that's possible.
- 38:19>> I think of it like you've got this huge
- 38:22chunk of labor that needs to happen.
- 38:23Right? In the old days, this was
- 38:25building the thing, right? and building
- 38:27the thing would take days, weeks, months
- 38:29maybe. Um, now that has shrunk, right?
- 38:32But the planning up front and then the
- 38:33review is still the same, right? We're
- 38:36expected to do these faster iteration
- 38:38cycles as devs, but the stuff that was
- 38:41actually quite hard is still kind of the
- 38:45same and takes the same amount of time.
- 38:47Building the thing wrong or sorry,
- 38:49building the wrong thing is something
- 38:50that you can still do. And I totally
- 38:53agree that there's like this sort of a
- 38:55lot of folks are doing this kind of plan
- 38:56maxing thing where they're just, you
- 38:59know, they take their spec and they they
- 39:01think about their spec, they run their
- 39:02spec through seven different agents or
- 39:04something and then they um, you know,
- 39:07they get back a better plan that they
- 39:08then go and implement or something. How
- 39:11like that sound not sound good?
- 39:13>> The agents love to write plans. Oh my
- 39:15goodness, they love it. And they will
- 39:17embellish the plans and the plans will
- 39:20be gorgeous and beautiful and spell out
- 39:22all kinds of details. [laughter]
- 39:25>> Yeah. And then they fall apart at the
- 39:26end.
- 39:27>> Um so yeah, I I think you can see it in
- 39:31the industry right now. There's this
- 39:33movement towards spectriven development.
- 39:36Uh and my my uh impression there is that
- 39:40that's probably not going to work. My
- 39:41experiments did not work particularly
- 39:43well. Um, and I'm thinking now maybe we
- 39:46should once again look at the agile
- 39:49ideas and maybe the idea of do a little
- 39:52bit, get some feedback, do a little bit
- 39:55more, get some feedback, reorganize, do
- 39:57a little bit more, feedback, reorganize.
- 39:59I used to tell this story back when I
- 40:01was doing agile lectures.
- 40:04If if it cost you
- 40:07$1
- 40:09to make a change to a house, including
- 40:13the initial laying of the foundation,
- 40:15the initial roof, everything, every
- 40:17change you gave to the contractor would
- 40:19cost you a dollar. How would you build
- 40:22that house? Would you hire an architect
- 40:25and pay thousands of dollars to the
- 40:27architect to come up with the perfect
- 40:29plan that they then paid a dollar to the
- 40:31contractor so that he could build in one
- 40:33shot? Or would you walk up to the
- 40:36contractor and say,
- 40:38"I want the foundation here. Make it
- 40:40that shape." Oh, no. That's
- 40:41[clears throat] bad. Okay, let's change
- 40:43that foundation. Let's do a little Put
- 40:45the kitchen over here. Put the living
- 40:47room there. THERE'S $2. OH, HECK NO.
- 40:50Let's change those around. That's let's
- 40:52let the kids walk through the oh the
- 40:54traffic pattern is crappy. Move the
- 40:56stairs [laughter] or whatever. Um
- 40:59obviously the latter is probably better
- 41:02and that's what we are looking at right
- 41:04now. It costs a dollar. Well maybe two,
- 41:06maybe five. But the the cost of change
- 41:11has plummeted
- 41:13to as close to zero as I think we're
- 41:16ever going to get it. and well that's
- 41:17that's a prediction I'll probably lose
- 41:19but still right the cost of change has
- 41:22has gone so far down that why would you
- 41:26do this upfront planning because that's
- 41:29expensive
- 41:31why wouldn't you just fiddle fiddle
- 41:33fiddle fiddle until it looks right
- 41:36[laughter]
- 41:36>> yes I I 100% agree I couldn't agree more
- 41:40the the thing I the thing I have I think
- 41:43I have a massive issue with the
- 41:45specdriven development label
- 41:47Because what is spectriven development,
- 41:48right? Like you can have like 10
- 41:50different interpretations of it and
- 41:52every time you pass information to an
- 41:54agent, right? Is prompt engineering
- 41:56specri development, right? Like it kind
- 41:58of is. You know, you don't you know, you
- 42:01would say to your, you know, if in the
- 42:02old days you would have like your mate
- 42:04over the other side, your colleague, and
- 42:05you would say, you just fix that loading
- 42:07issue in the in the header or something.
- 42:09Is that spec driven development? Right?
- 42:11Like is that a specification that I've
- 42:12given him? you know and it seems like
- 42:14every time I say okay do a little bit of
- 42:16upfront alignment first that approach
- 42:19then gets called spectriven development
- 42:21but for me I think the difference is are
- 42:24you persisting your specifications right
- 42:27are you returning to those
- 42:28specifications so what's your attitude
- 42:31there like do you do you keep like a
- 42:33list of all of your specs in the repo
- 42:36>> or do you like what's going on there
- 42:38>> no I do not um the the specifications
- 42:41are ephemeris they go away. Uh they
- 42:44change a lot. I fiddleled with them and
- 42:46you know that that goes away. There is
- 42:49no equivalent to source code. You know
- 42:53we humans wrote the source code. So that
- 42:56was that was the final specification.
- 42:58Well that doesn't exist anymore. There
- 43:00is still source code but we humans
- 43:02aren't the ones writing it. And a lot of
- 43:04people feel that lack. There has to be a
- 43:07human thing that defines everything up
- 43:11front. Well,
- 43:13I mean, in the end, even what the agents
- 43:15produce was produced by humans.
- 43:18The the thing that I've be been doing
- 43:21lately is this. Instead of creating a
- 43:25specification that defines what I want
- 43:29or defines what I have even, I look at
- 43:32the end result and say, well, that is
- 43:34the specification. So, I've got a bunch
- 43:36of tools out there. I've got like the
- 43:38crap tool runs for closure. It runs for
- 43:41Java. It runs for Go. I wrote a few of
- 43:44them, right? I had my agents write them.
- 43:45I've got the mutation tester. I've got
- 43:48my my agent harness. You know, I've got
- 43:50all these things up there. What I tell
- 43:52people is don't download those. I wrote
- 43:55them for me. What you should do is point
- 43:57your agents at them, have
- 43:59[clears throat] the agents look at them,
- 44:00and then build one for you. [laughter]
- 44:02>> Yeah. I think that's a far better way of
- 44:05specifying the essence of something and
- 44:09then customizing it to your to your
- 44:11particular need.
- 44:13>> Yep. It's what I always find weird about
- 44:16agents is that if you send them
- 44:18something, they read it, right?
- 44:20[laughter] Which is very different to
- 44:22humans, right? Very, you know, you can
- 44:25maybe get a 20% hit rate if you send
- 44:27someone a massive specification and 20%
- 44:30is maybe a bit generous. 5% maybe if you
- 44:32pass a you know a spec to an agent
- 44:34they're probably going to read it right
- 44:36>> the opposite side of that is the things
- 44:38that the agents write the humans don't
- 44:40read.
- 44:41>> Yes. Exactly. Yeah. Exactly. So they're
- 44:44expecting
- 44:46I had thought of it like that. They're
- 44:48expecting us to read everything they
- 44:49write and we just don't. God.
- 44:53It's so one-sided. You know, long term,
- 44:55I'm interested in how human
- 44:56relationships differ or get different
- 44:59because we're so used to communicating
- 45:01with agents, right? I dictate to my
- 45:02agents. So, it's like I'm just sort of
- 45:04talking to a friend or something. I'm
- 45:05really interested in sort of how life
- 45:07imitates art over time and stuff. I
- 45:09don't know. That's fascinating to me.
- 45:11So, okay. And I think I'm going to ask
- 45:14you maybe the final question, and it's a
- 45:16sort of fairly beefy one.
- 45:18>> Um, if that's all right, finalish. Um,
- 45:22I'm gonna go John Alart again
- 45:24>> because John has a great definition for
- 45:27>> different types of programming, right?
- 45:29You've got the tactical programming. Are
- 45:32you going to get it?
- 45:33>> I I just have to You've mentioned I've
- 45:35got to pull this book down.
- 45:37>> Wait.
- 45:38>> Uh,
- 45:39it's it's here somewhere.
- 45:41>> Where's mine? I've got mine.
- 45:42>> Oh, good. Okay. All right.
- 45:43>> There you go. I've got mine.
- 45:44>> There it is. Yep.
- 45:45>> I got it.
- 45:46>> Okay.
- 45:46>> Done. um tactical versus strategic
- 45:50programming. Yeah.
- 45:51>> So tactical is the sergeant on the
- 45:53ground, the person kind of fighting the
- 45:55battle. The strategic stuff is the
- 45:58general kind of the person directing the
- 46:00course of the war.
- 46:01>> Now I think we both agree that's a good
- 46:04framing, right, for different types of
- 46:05programming. And agents are really good
- 46:07at tactical, really bad at strategic.
- 46:10>> Yeah. Now, for people who are just
- 46:13starting out and AI has now eaten all of
- 46:16the tactical stuff, how do they learn to
- 46:20do strategic programming? Because I
- 46:23think a lot of people here, that's kind
- 46:25of what they want from you, Bob. They
- 46:27want, please just give me your brain so
- 46:30that I can inject into it, understand
- 46:32why mutation testing is so good, tell me
- 46:34how to explain these modules. Like,
- 46:36that's what they want, I think. So
- 46:38that's your job now is to give people
- 46:40your brain. How how how did they learn
- 46:42that stuff?
- 46:43>> Okay. So, um
- 46:45I get this question a lot. I don't have
- 46:47perfect answers to it because I really
- 46:49don't know. But but here's how I would
- 46:51think about this. Here's here's how I do
- 46:53think about this. First of all, the um
- 46:58the way a programmer should learn
- 46:59whether it's in university or somewhere
- 47:01else, right? It should be to write code.
- 47:04You should you should be writing code
- 47:06right for a year. I don't know how long
- 47:09but you should be writing code so that
- 47:10you know what the agents are dealing
- 47:13with. The next thing that I think should
- 47:16happen is that when you get hired into a
- 47:19company where that company is making
- 47:21heavy use of agents,
- 47:24you the young young person just coming
- 47:26out of training
- 47:29should be treated like an agent.
- 47:31the uh the the guy who's running the you
- 47:34know maybe the lead engineer or whoever
- 47:37the guy who's got bunch of agents
- 47:39running and he's being strategic. He
- 47:41should look at you as an agent and he
- 47:43should give you the same kind of tasks
- 47:45that the agents have and subject you to
- 47:47the same kind of deterministic tools
- 47:49that the agent have agents have to use.
- 47:52And you should spend several months in
- 47:55that state being horribly unproductive
- 47:59but learning a hell of a lot. And by the
- 48:02time you've gone through that gauntlet,
- 48:05maybe you can be trusted to run an agent
- 48:07of your own.
- 48:09I don't know. It's going to be something
- 48:11like that. You cannot lose the code
- 48:15entirely.
- 48:17I used to tell people
- 48:19um 10 years ago used to tell people um
- 48:23if you've never written assembly
- 48:24language, you should spend the weekend
- 48:26writing assembly language just so that
- 48:27you know what's really going on behind
- 48:30the scenes because if if all you're
- 48:32doing is writing Java all day long, you
- 48:34live in a fantasy world. There is still
- 48:36magic yet that you don't understand.
- 48:40Spend a weekend doing assembly language
- 48:42and you will finally understand what's
- 48:43really going on. And I I think that
- 48:46remains true
- 48:48somehow during this educational pathway.
- 48:52You've got to go from the basics binary
- 48:55all the way through assembly language
- 48:57some basic code like C some higher level
- 49:00code like Python or something and then
- 49:04deal with uh agent kind of work and
- 49:07deterministic tools and finally be able
- 49:09to strategically run an agent under
- 49:12supervision.
- 49:14>> Yeah, [snorts]
- 49:15>> I don't know how better to say that.
- 49:17>> It's so hard though, right? because
- 49:19there's two there's two things going on
- 49:21there, right? Like there's the agent is
- 49:24a kind of abstraction layer, right, over
- 49:26the code. And so you got the code and
- 49:28then it's interesting you were talking
- 49:29about an abstraction layer like a sort
- 49:31of um module viewer, right? Like I think
- 49:34that's super interesting. That's a
- 49:36really good way to learn getting these
- 49:38kind of abstractions over the code so
- 49:40when you dive in you understand it more
- 49:42deeply. But if you have someone who's
- 49:44just like sitting on your books just
- 49:46doing tactical work, surely as a company
- 49:49you're going to look at that and go, why
- 49:51are we hiring this person when we've
- 49:53got, you know, we've got Uncle Bob's
- 49:56swarm of five hardeners and stuff that
- 49:58can beat it, you know, a fraction of the
- 50:01cost, right? I'm I I just I I'm so Okay.
- 50:06What What like resources or books can or
- 50:08approaches or Sorry, there's one more
- 50:11question here which is [laughter]
- 50:15I'm I'm I'm trying to like work out how
- 50:17people get this information because the
- 50:20feedback loop on strategic programming
- 50:22is so long, right? Or traditionally it
- 50:24has been very long. So you you can often
- 50:27>> like people who just like quit jobs
- 50:29after six months or something, they
- 50:32might never learn strategic programming,
- 50:33right? Because their mistakes are maybe
- 50:35nine months away, you know, so they just
- 50:37never see the their own mistakes. But I
- 50:40think with agents, you can, right?
- 50:42Because it's sped up so much, you can
- 50:45actually get more feedback on your
- 50:47mistakes sooner. And I suppose what I'm
- 50:49interested in is how did you know that
- 50:52your agents were making mistakes back in
- 50:54December when you were looking at them
- 50:55and going this is dog? How did you
- 50:57identify that?
- 50:59>> Um early on I was just looking at the
- 51:01code and seeing the dog do. That wasn't
- 51:03the important part. The important part
- 51:05was the next step where I watched them
- 51:07thrash. I could see the agent struggle
- 51:11[snorts] and I recognized the struggle
- 51:14since I have been through that struggle.
- 51:16Right? And that's one of the issues is
- 51:18the novice would come in and not
- 51:19recognize the struggle.
- 51:21>> Now, how do you learn that? How do you
- 51:24learn? You know, I learned it the hard
- 51:25way. School of hard knocks. How do you
- 51:28learn that as a young person coming in?
- 51:30How do you recognize that? There is a
- 51:33wealth of
- 51:35information about this. Um the the old
- 51:39books, the ones that nobody reads
- 51:40because they're old. Uh the old books on
- 51:43this topic are terrific, right? you go
- 51:45to the works by Tom DeMarco or or the
- 51:48works by Ed Yordan or you know go go
- 51:52read you know um gez I can't think of
- 51:56the names right now but
- 51:58>> pragmatic programmer you know the
- 52:01pragmatic programmer um the the
- 52:05there's a lot
- 52:06>> there's a lot of these older books
- 52:08they're very good
- 52:10>> um that if you if you study them when
- 52:13you're young you will get may feel for
- 52:17what this higher level strategic play
- 52:19is. You'll have to filter out some of
- 52:21the archaic stuff because a lot of these
- 52:23books were written in the 70s or the
- 52:2580s, right? But that's when these
- 52:28lessons were learned.
- 52:30And so I, you know, that's that's where
- 52:32I would go initially. I would go to
- 52:35those books and learn that stuff that
- 52:37way. And then, of course, you're going
- 52:38to have to learn it by feeling it.
- 52:40>> Yeah. That's why I think they ought to
- 52:42play play uh agents for a few months so
- 52:45they can learn what that's really like.
- 52:47>> Become the agent. Have the agent
- 52:49delegate to you. Yeah. You become a sub
- 52:50agent of the agent. I like that.
- 52:52[laughter]
- 52:54So, okay. I do have one more question.
- 52:56It's I suppose a short one. Sure.
- 52:58>> Um which is
- 53:01>> sounds like then software fundamentals
- 53:04still matter, right?
- 53:06>> Yeah.
- 53:06>> And why is that?
- 53:09And what do you say to the people who
- 53:11say that they don't matter?
- 53:15>> Why is that? Um
- 53:18software fundamentals matter for the
- 53:21reason they have always mattered. The
- 53:24let's see who said this. It was um
- 53:29uh
- 53:30Dystra I think who said it. I'm I'm
- 53:33going to get this wrong, but
- 53:36software is the most complicated thing
- 53:38that humans have ever attempted to do.
- 53:41More complicated than you know any other
- 53:43task that we've tried. Software is the
- 53:46most complicated thing. And therefore,
- 53:48the fundamentals are the are way of
- 53:51organizing that complexity into a form
- 53:54that can be conceived not just by
- 53:56humans, but by our models as well. Since
- 53:59our models are modeled after humans
- 54:01after all.
- 54:04So the fundamentals still apply because
- 54:07that's the way we organize complexity to
- 54:10be conceived or be to be conceived of.
- 54:14There are folks right now who think that
- 54:16the fundamentals don't matter. They will
- 54:17learn and they will learn that the hard
- 54:19way and and it won't take very long.
- 54:22Might take longer than I think it'll
- 54:24take because the agents are pretty good.
- 54:26But I've watched them hit the hit the uh
- 54:29wall. So I know that wall is there and I
- 54:32don't want to hit that again.
- 54:36>> It's a very interesting interesting set
- 54:39of parallels that we're in at the
- 54:41moment. We you had mentioned the the
- 54:43abstraction layer, right? And you know
- 54:46we're we're now at a at a level where
- 54:48we're above the compiler. We used to our
- 54:51our our abstraction layer used to be the
- 54:53compiler. Before that it was assembly
- 54:55language. Before that it was binary. And
- 54:58now it's up here at this level of the
- 55:00model. At every one of those steps
- 55:04up up the abstraction layer. The people
- 55:07at the at the lower step complaint said,
- 55:10"Oh, this is going to ruin everything.
- 55:12We're not even going to have jobs
- 55:13anymore. It's become so easy that
- 55:17five-year-olds will be able to write the
- 55:18code." And you know, back then they were
- 55:20still talking about like binary.
- 55:22[laughter]
- 55:23Yes.
- 55:24At every step it's the same. And and so
- 55:27we're at this next step and the people
- 55:29down here are saying, "Oh, it's going to
- 55:31ruin everything." No, it's not. The same
- 55:34rules apply. All the same fundamentals
- 55:37exist for all the same reasons. The
- 55:39rules you throw away are the ones you're
- 55:41going to pick up off the floor in a year
- 55:44and dust off and remember why you need
- 55:46them.
- 55:48>> Yeah. Thank you so much, Bob. There's a
- 55:52there's a great quote from I think Plato
- 55:54where he says that writing is going to
- 55:57make people stupider. Right? So people
- 55:59have had that same abstraction argument
- 56:01since you know the Greeks, right?
- 56:03Ridiculous.
- 56:06I think everyone uh watching, how many
- 56:07folks have we got on this stream? Uh
- 56:09we've got 1,500 people watching.
- 56:12Incredible. Uh I think we can all agree
- 56:14that was a fantastic conversation. Um
- 56:16Bob, thank you so much. Um, hold up your
- 56:20book one more time so folks can uh can
- 56:22get it. Let's see. [laughter]
- 56:25Clean code. And I think what is 11:00
- 56:28a.m. now your time? Back to the
- 56:29bathrobe. Uh,
- 56:30>> yeah. Yeah, it is. [laughter]
- 56:33>> Perfect. So, folks, thank you so much.
- 56:35I'm going to end the stream here. Me and
- 56:37Bob are gonna stay on for a
About this transcript
This page contains the full transcript of LIVE: Uncle Bob on Software Fundamentals in the Age of AI by Matt Pocock, generated from the public captions YouTube serves with the video. The transcript has 10,121 words across 1,454 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.