The third golden age of software engineering – thanks to AI, with Grady Booch — Transcript
Full transcript
- 0:00Some people worry that AI writing
- 0:01surprisingly good code could mean the
- 0:03end of software engineering. But Grady
- 0:05Buch disagrees and says that we are
- 0:06entering the third golden age of
- 0:08software engineering. Grady Buch is one
- 0:10of the founding figures of software
- 0:11engineering as we know it. He co-created
- 0:13UML, pioneered object-oriented [music]
- 0:16design, spent decades as an IBM fellow,
- 0:18and has witnessed every major
- 0:19transformation this industry has
- 0:21undergone since [music] the 1970s. In
- 0:23today's conversation, we discuss the
- 0:24three golden ages of software
- 0:26engineering and what history [music]
- 0:27teaches us about surviving and thriving
- 0:29through major technology shifts. Why
- 0:31coding has always been just one part of
- 0:33software engineering and why the human
- 0:35skills of balancing technical, economic,
- 0:37and [music] ethical forces are not going
- 0:38anywhere. Grady's direct response to
- 0:40Dario's prediction that software [music]
- 0:42engineering will be automated in 12
- 0:44months. Spoiler, he does not hold back.
- 0:47And many more. If you want to understand
- 0:48that the massive change that AI is
- 0:50bringing has in fact happened before and
- 0:53not just once, this episode is for you.
- 0:55This episode is presented by Statsig,
- 0:56[music] the Unifi platform for flags,
- 0:58analytics, experiments, and more. Check
- 1:00out the show notes to learn more about
- 1:02them and our other season sponsors. So,
- 1:04Grady, it's great to have you back on
- 1:06the podcast again. Thanks for having me.
- 1:08Aloha. So touching a little bit on the
- 1:11the history of of software engineering,
- 1:14you've said many times before that the
- 1:16entire history of software engineering
- 1:17is one of rising levels of abstraction.
- 1:20Can you walk us through the key
- 1:21inflection points that help us
- 1:23understand this and then of course tie
- 1:25it into how AI is is all tying into
- 1:28this?
- 1:29>> Well, the very term software engineering
- 1:31did not come to be until Margaret
- 1:33Hamilton was probably the first to uh
- 1:36anoint it. uh she at the time had just
- 1:39left the man orbiting laboratory
- 1:41project. She was working on the Apollo
- 1:43program and she was one of the very few
- 1:45people who were software developers in a
- 1:48sea of mostly men who were the hardware
- 1:50structural engineers and she wanted to
- 1:52come up with a phrase that distinguished
- 1:54herself from the others. So she began
- 1:57using the term software engineer and I
- 1:59think we can rightfully give her the
- 2:01claim to the first one that coined that.
- 2:03There were others that followed most
- 2:05notably people talk about the NATO
- 2:07conference uh on software engineering
- 2:10and when the organizers established that
- 2:13which was actually a few years after
- 2:14Margaret's work they did so as kind of a
- 2:17controversial name not unlike how the
- 2:20term artificial intelligence was named
- 2:22controversially for its first conference
- 2:25on the west coast. Um, so there were
- 2:28others that followed and after a period
- 2:30of time it kind of stuck and I think
- 2:32what it meant the essence of what
- 2:34Margaret and others were doing is to say
- 2:36there's something engineeringish about
- 2:39it in the sense that ours is a field
- 2:42that tries to build reasonably optimal
- 2:45solutions. You can't have perfect
- 2:47solutions that balances the static and
- 2:50dynamic forces around them much like
- 2:52what structural, electrical, chemical
- 2:54engineers do. In the software world, of
- 2:56course, we deal with the medium that is
- 2:59extraordinarily funible and elastic and
- 3:02fluid and yet we still have the same
- 3:04kinds of forces upon us. Uh here we've
- 3:07got the forces of the laws of physics.
- 3:09You can't pass information faster than
- 3:11the speed of light, which is kind of
- 3:13annoying in some cases, but hey, we'll
- 3:15have to live with it. There are issues
- 3:17about how large we could build things,
- 3:19largely constrained by our hardware
- 3:21below us. There are constraints we have
- 3:23on the algorithmic side of things. We
- 3:25may know theoretically how to do
- 3:27something such as the vertalry
- 3:29algorithm, which was essential to the
- 3:32creation of cellular phones. For the
- 3:34longest time, we didn't know how to
- 3:35implement it, but there was indeed a
- 3:37calculable solution. similar stories
- 3:39with regards to fast forier transform.
- 3:41We knew the theory but until Forier
- 3:44transforms could be turned into
- 3:46something computational we couldn't pro
- 3:49progress. And there are also other
- 3:51constraints upon us not just these
- 3:53scientific ones and and the computer
- 3:55sciency ones but constraints such as the
- 3:57human ones. Uh can I get enough people
- 4:00to do what I need to do? Can I organize
- 4:03teams doing what I want to do? Ideally
- 4:06the largest team size you want for
- 4:08software is zero. Well, that's not very
- 4:11practical. The next best one is one and
- 4:14then it kind of grows from there. And
- 4:16there are projects that simply are of a
- 4:18certain scale that you cannot conceive
- 4:21of them being done by a small group of
- 4:23people. I mean, why do any of the large
- 4:25projects we have have a cadre of folks
- 4:27in them? It's because the footprint of
- 4:29these systems and their enduring
- 4:31economic and social importance is so
- 4:34great. You can't rely upon just an
- 4:36individual. That software must endure
- 4:39beyond them. And increasingly as
- 4:41software moves into the interstitial
- 4:43spaces of the world, we have the legal
- 4:46issues uh such as we see with you know
- 4:48digital rights management but I think
- 4:50more importantly and overarching the
- 4:52ethical issues. We know how to build
- 4:54certain things but should we build them?
- 4:57Is it the right thing for us to do in
- 4:59our humanity? So these are the
- 5:01collection of things that are in a way
- 5:03well not in a way but absolutely are the
- 5:05static and dynamic forces that weigh
- 5:07upon a software engineer and that's why
- 5:10I can say we are engineers because much
- 5:12like the other kinds of engineers we
- 5:14build systems that balance those forces
- 5:17and we do so in a medium that is
- 5:19absolutely wonderful. So that's software
- 5:22engineering. Now I mentioned in our last
- 5:25call there are certain ages of software
- 5:27engineering and I think as we look from
- 5:30the from the lens of looking backward
- 5:33there are at least two identifiable
- 5:36major epics in software engineering. In
- 5:39the earliest days there was no software
- 5:41because what we did was simply managing
- 5:44our machines and the difference between
- 5:46the hardware and the software was
- 5:48completely indistinguishable. you know,
- 5:50putting plugs in a plugboard as was
- 5:52happened with the ENIAC. Is that
- 5:54programming? Well, yes, but there's not
- 5:56really software there. It's something
- 5:58else. And it wasn't until our machines
- 6:01came to the point in late 40s, early 50s
- 6:04that we began to find a difference for
- 6:06them. Most of this software written at
- 6:09that time was bespoke. Well, really all
- 6:12of it was. And virtually all that
- 6:14software was tied to a particular
- 6:16machine. But the economics of software
- 6:18were sh such that we love these
- 6:21machines. We'd like them to be faster,
- 6:22but gosh, we put a lot of investment in
- 6:25the software itself. Is there a way to
- 6:27decouple these kinds of things? We talk
- 6:30about the recent history of our of our
- 6:32world. The term digital was not coined
- 6:35until the late 40s. The term software
- 6:37was not done until the 50s. And so even
- 6:40the acknowledgement that software was an
- 6:42entity unto itself was just about in my
- 6:45lifetime which is frightening to think
- 6:47about.
- 6:48>> Yeah. Like 70 80 years ago. Wow.
- 6:50>> Yeah. Yeah. Exactly. So this is this
- 6:52were an astonishingly young young
- 6:54industry. If you were to take Carl
- 6:56Sean's cosmic calendar and uh and put
- 7:00software in it, we would be in the last
- 7:02few nanoconds of that cosmic calendar.
- 7:05It would be less than a blink of an eye.
- 7:07But anyway, as software began began to
- 7:11be decoupled from hardware itself, then
- 7:14folks such as Grace Hopper and others
- 7:17were beginning to realize that this is a
- 7:20thing that we could treat as a business
- 7:22and an industry as an institution unto
- 7:25itself. So the earliest software of
- 7:28course was as it was software itself was
- 7:30assembly language which was very much
- 7:32tied to the machine. And jumping ahead a
- 7:35little bit, as IBM came along in the
- 7:37'60s recognizing that there was a way to
- 7:40establish a whole architecture of
- 7:42machines with a common instruction
- 7:44language, then it was possible to
- 7:46preserve software investments and yet
- 7:50decouple it from hardware in a way that
- 7:51I could improve my hardware without
- 7:54throwing away the software. Once that
- 7:56realization happened which was both an
- 7:58engineering decision, a business
- 8:00decision and overall an economic
- 8:03decision then the floodgates opened up
- 8:05and all of a sudden we had a lot more
- 8:07software that could be and needed to be
- 8:10written. This was the first golden age
- 8:12of software engineering in which we had
- 8:15software was an industry unto itself.
- 8:18And so the essential problems that world
- 8:20faced were problems of complexity. uh
- 8:24complexity in that we were building
- 8:25things that were, you know, difficult to
- 8:28understand, that were trying to
- 8:29manipulate our machines in some cunning
- 8:32ways, but it was complexity that by
- 8:34today's standards was, you know,
- 8:35laughably simple. We could, you know,
- 8:37this is the equivalent of hello world,
- 8:39but they were problems that were hard
- 8:40unto themselves. And so because we were
- 8:44so coupled to the machines, the primary
- 8:46abstraction used in the first golden age
- 8:49of software engineering was that of
- 8:51algorithmic abstraction because that's
- 8:52what our machines did. Most of our
- 8:54machines were meant for mathematical
- 8:57kinds of operations and so as as was
- 9:00done in forran it was a matter of
- 9:02building our software that could do
- 9:04formula translation.
- 9:06So that was the realm and the problems
- 9:09faced by the first generation
- 9:11>> and and this first generation like in
- 9:13timeline where would you put it roughly
- 9:16>> timewise I'd put it in the late 40s to
- 9:19the late7s or thereabouts
- 9:22>> and that's what dominated that time
- 9:23frame. So the figures you would see
- 9:25would be uh Ed Jordan, Tom DeMarco,
- 9:28Larry Constantine. This is when uh ERP
- 9:32uh sorry not entity relationship ideas
- 9:34came about. And so these ideas of that
- 9:37kind of abstraction poured over not just
- 9:40into software but also into the data
- 9:42side of things as well. This was an
- 9:44extraordinarily vibrant period of time
- 9:47in software engineering in which we had
- 9:49the invention of flowcharts for example
- 9:51which were an aid to thinking about how
- 9:54to construct these kinds of systems. You
- 9:56saw a division of labor where you had
- 9:59people who would analyze the system. You
- 10:01people who would then program it, people
- 10:03who would key punch the solutions,
- 10:04people would operate the computers. And
- 10:07again this was largely driven driven by
- 10:09economic reason because the cost of
- 10:12machines were far greater than the cost
- 10:15of the humans involved in them. So a lot
- 10:18of what was happening was done to
- 10:20optimize the use of the machines which
- 10:22were very very rare resources.
- 10:25Um the lesson in this as we'll see
- 10:27coming back in the next generations is
- 10:29that these forces much like with
- 10:31software engineering itself have shaped
- 10:33the very industry of software and
- 10:36economics and the whole social context
- 10:39also influences them. So in the first
- 10:43generation it was largely focused upon
- 10:45mathematical needs and the automation of
- 10:49existing business processes. So what you
- 10:52had happen is that you would have
- 10:54businesses that have literal, you know,
- 10:56floors of offices with people doing
- 10:59accounting and payroll and like that.
- 11:01And this was the lowhanging fruit
- 11:03because now all of a sudden we could
- 11:05accelerate those processes and actually
- 11:08improve their precision by pulling the
- 11:11human out of it and automating it. So
- 11:14the vast amount of software written
- 11:16during that time was business and and
- 11:19mathematical and and numerical kinds of
- 11:21things. Now this is an important thing
- 11:23because while this was the focus, this
- 11:26was not the only kind of thing because
- 11:28you saw in the periphery or shall I say
- 11:31from the point of view of a person who
- 11:33was a programmer in that time it looked
- 11:35to them as the dominant places was in
- 11:37the IBMs, the insurance companies, the
- 11:39banks and the like. There's a lot of
- 11:41work going on outside that world in the
- 11:43defense industry as well. We saw people
- 11:45moving software and hardware into our
- 11:48machines of destruction into our
- 11:49aircraft into our missiles. We saw it
- 11:52moving into weather forecasting. We saw
- 11:54it moving into medical devices itself.
- 11:58So while the concentration was the
- 12:00things that the general public would see
- 12:02a lot of stuff happening around the
- 12:04edges as well. I would say in the first
- 12:07golden age of software engineering there
- 12:09was this central push of algorithmic
- 12:12abstractions into business and numerical
- 12:15things but the real innovation was
- 12:18happening in that fringe in particular
- 12:21it wasn't in business cases but it was
- 12:23in defense cases because Russia was the
- 12:26clear and present threat for us at the
- 12:27time in which there was a need to build
- 12:30distributed systems of real time nature
- 12:33most of the systems I've talked about
- 12:35this were were not real time. And so we
- 12:37saw the rise of of experimental machines
- 12:40such as whirlwind. We saw the work in
- 12:43the mother of all demos which was
- 12:45experimentation of various human
- 12:47interface kinds of things which was not
- 12:50the center of gravity of of software
- 12:52development at the time with the things
- 12:54on the fringes. We saw we saw
- 12:56researchers such as David Parnes who
- 12:59were coming on the scene CAR Dyster and
- 13:02others were forbidding to look at the
- 13:04formalisms of these systems and looking
- 13:05at treating software development is
- 13:07actually a formal mathematical activity.
- 13:09Grady just mentioned formal methods and
- 13:11formal mathematics and software
- 13:12engineering. Being able to verify that
- 13:15software does what it should has been a
- 13:16problem since the early days of software
- 13:18engineering. And this leads us nicely to
- 13:20our seasonal sponsor, Sonar. As we're
- 13:22living through what Grady might call the
- 13:24third golden age of software
- 13:25engineering, AI coding assistants
- 13:27generate code faster than we ever
- 13:28thought was possible. This rapid code
- 13:30generation has already created a massive
- 13:32new bottleneck at code review. We're all
- 13:34feeling it. All that new AI generated
- 13:36code must be checked for security,
- 13:38reliability, and maintainability. A
- 13:40question that is tricky to answer
- 13:41though. How do we get the speed of AI
- 13:44without inheriting a mountain of risk?
- 13:47Sonar, the makers of Sonar Cube, has a
- 13:49really clear way of framing this. Vibe
- 13:52then verify. The vibe part is about
- 13:54giving your teams the freedom to use
- 13:56these AI tools to innovate and build
- 13:57quickly. The verify part is the
- 13:59essential automated guardrail. It's the
- 14:02independent verification that checks all
- 14:03code human and AI generated against your
- 14:06quality and security standards. Helping
- 14:08developers and organizational leaders
- 14:10get the most out of AI while still
- 14:11keeping quality, security, and
- 14:13maintainability is high on the main
- 14:15themes of the upcoming Sonar Summit.
- 14:17It's not just a user conference. It's
- 14:18where devs, platform engineers, and
- 14:20engineering leaders are coming together
- 14:22to share practical strategies for this
- 14:23new era. I'm excited to share that I'll
- 14:25be speaking there as well. If you're
- 14:27trying to figure out how to adopt AI
- 14:28without sacrificing code quality, come
- 14:30join us at the Sonar Summit. To see the
- 14:32agenda and register for the event on
- 14:34March 3rd, head to
- 14:35sonarsource.com/pragmatic/sonarssummit.
- 14:39With this, let's get back to Grady and
- 14:41treating software development as a form
- 14:42of mathematical activity. And you saw
- 14:45the rise of I said distributed and
- 14:47real-time systems primarily in the
- 14:49defense world. So from whirlwind it
- 14:51begat a system called sage the
- 14:54semi-automatic ground environment which
- 14:56came about during the six during the 50s
- 14:58and60s
- 15:00and indeed the last one was
- 15:01decommissioned I think in the 1990s.
- 15:04This was based upon the threat of
- 15:06Russia. This is you know pre premissiles
- 15:08Russia would send a fleet of bombers
- 15:10over the Arctic and invade the United
- 15:12States. So thus was born the D line, the
- 15:15distance early warning system across
- 15:17Canada. And all that data was then fed
- 15:19into a series of systems called SAGE,
- 15:22the semi-automatic ground environment.
- 15:24This system was so large it consumed
- 15:26according to some reports easily 20 to
- 15:2930% of every number of software
- 15:31developers in the United States at the
- 15:33time. Wow,
- 15:34>> that's a lot of folks. But remember back
- 15:36in the time there were maybe only a few
- 15:38tens of thousands of software developers
- 15:40but this was the biggest project
- 15:42>> basically the military was the biggest
- 15:43spender uh in soft in research and
- 15:46moving the industry forward right
- 15:48because they had
- 15:48>> absolutely absolutely correct they had
- 15:50to because it was a clear and present
- 15:51threat and so a lot of the innovation
- 15:53was happening to the defense world as I
- 15:56think I passed this phrase on to you in
- 15:57the documentary I'm working on in
- 15:59computing I use the phrase that there
- 16:01are two major influences in the history
- 16:03of computing one is commerce. We've
- 16:05talked about the economics already. And
- 16:07the second is warfare. And thus I claim
- 16:12and I think there's much defense for it.
- 16:14Much of modern computing is really woven
- 16:16upon the loom of sorrow. Referring back
- 16:19to Jakard's loom. So yeah, a lot of the
- 16:21things we take for granted today like
- 16:23the internet uh like uh micro
- 16:25miniaturaturization, this all came from
- 16:28government funding in these cases. So we
- 16:30owe a lot to the cold war. This phase
- 16:32was this still the first golden age? We
- 16:34passed the first golden age. These are
- 16:36the things happening in the first golden
- 16:37age. But what I'm pointing out is there
- 16:39was sort of a center of mass to it, but
- 16:42lots of things happening on the edge
- 16:44that were driving software out from its
- 16:47primary roots. So let's recap here. In
- 16:49the first golden age, you had the focus
- 16:51primarily upon mathematical and business
- 16:53kinds of applications. And the primary
- 16:56means of decomposition was an
- 16:59algorithmic abstraction. We looked at
- 17:01the world through processes and
- 17:02functions, not so much through data. But
- 17:05on the fringe, we had organizations, use
- 17:08cases that were pushing us beyond that
- 17:11simple place. Use cases that demanded
- 17:14distribution, use cases that demanded
- 17:16the coupling of multiple machines and
- 17:18also use cases that demanded real-time
- 17:21processing and use cases that demanded
- 17:24human user interfaces. Yeah,
- 17:26>> the interfaces we deal with today, they
- 17:28had their roots in whirlwind and the
- 17:30roots in Sage. This is the first UI
- 17:32interface that was graphic tube, a CRT.
- 17:36And so these kinds of things were born
- 17:37from that. So that was the point and I
- 17:40think the lesson from this is that
- 17:42software is a wonderfully dynamic,
- 17:45fluid, fungeible domain. But it's also
- 17:48one that tends to grow because once we
- 17:51built something and we know how to build
- 17:53it and we have patterns for doing so,
- 17:55all of a sudden we discover there
- 17:57economically interesting ways we can
- 18:00apply it elsewhere. So this was the
- 18:02first generation, the first golden age
- 18:05of software engineering. But you could
- 18:07begin to see cracks in the facade in the
- 18:10late 70s early 80s. The NATO conference
- 18:14on uh software engineering uh was one of
- 18:17the first to do this in a big public
- 18:19way. And for them NATO was realizing we
- 18:23NATO have a software problem. We have an
- 18:26insatiable demand for software and yet
- 18:29our ability to produce it of quality at
- 18:33speed, we just don't know how to do it.
- 18:35And so this was the so-called software
- 18:38crisis and you know people didn't know
- 18:39what to do about it. Can you help us
- 18:41understand or take us back what what was
- 18:43the crisis about? What were people like
- 18:45kind of like saying oh my gosh this is
- 18:46the problem?
- 18:47>> Yeah the problem was to recap was
- 18:50software was clearly useful. There were
- 18:53economic incentives to use it and yet
- 18:56the industry could not generate quality
- 18:58software of scale fast enough.
- 19:02>> I see. I see. So it it was both
- 19:04expensive, slow and and not good.
- 19:06>> There's a fourth one which was the
- 19:08demand was so great that I guess you
- 19:10could call it the slow the demand was so
- 19:11great. It's like wow this is we want
- 19:13more of this stuff. Give us more
- 19:15software. So those four things together
- 19:18put us in the sense of crisis. Notice
- 19:20subtly it's not the same kind of crisis
- 19:22we have today where we worry about
- 19:24surveillance, we worry about you know
- 19:26crashes, that kind of thing. So the
- 19:28nature of the problems have changed and
- 19:30they do in every every golden age.
- 19:33>> It's fascinating that this thing
- 19:34existed, you know, living in our our
- 19:36current reality.
- 19:38>> Yes. Yes. It's a very different world
- 19:39itself. But it was a the clear and
- 19:41present danger at the time was that and
- 19:44it was an exciting vibrant time because
- 19:46there was so much that could be done and
- 19:49software being such a funible elastic
- 19:51fluid medium meant that we were
- 19:53primarily limited just by our
- 19:55imagination. You add to this then micro
- 19:58miniaturaturization. Why did integrated
- 20:00circuits come about? Why did Fairchild
- 20:03uh come about and and establish Silicon
- 20:05Valley be the basis for it? It's because
- 20:07of the transistor. Who was the first
- 20:10customer of the Fairchild? It was the
- 20:13Air Force primarily for their men
- 20:17missile. In fact, most of the
- 20:19transistors being made in Silicon Valley
- 20:22in the earliest days went to our cold
- 20:24war programs. But that was great because
- 20:26that established then the the economic
- 20:29basis for the whole infrastructure for
- 20:31doing it where it was possible to start
- 20:33doing these things at scale and of of
- 20:35course we knew that begat integrated
- 20:37circuits that begat personal computers
- 20:39and so on. So here we are now in the
- 20:42late '7s and the software crisis was
- 20:45quite clear. The US government in
- 20:47particular, to focus on one story,
- 20:50recognized that they had the problem of
- 20:52Babel and that there were so many
- 20:55programming languages in place. By their
- 20:57count, there were at least 14,000
- 21:00different programming languages used
- 21:02through military systems. Oh wow. Back
- 21:05then when software was so much smaller
- 21:07than today. Wow.
- 21:08>> Absolutely. It's incredible. And
- 21:10languages like languages like Jovial was
- 21:13a very popular one. a jovial kind of a
- 21:15play on words for cobalt and and the
- 21:18like. We had the rise of algal which was
- 21:20not a military language but the formal
- 21:23forces of caor and dystra and verth led
- 21:26to this discipline of applying
- 21:27mathematical rigor to our languages and
- 21:30so the idea of you know formal language
- 21:32research was born you had this wonderful
- 21:35confluence of resources it said by the
- 21:37late '7s the government recognizing that
- 21:41we have a problem that's when they
- 21:42funded the ADA project which at the time
- 21:45was called the joint program working
- 21:48group something something like that
- 21:50which was an attempt to remove the
- 21:53number of language that exist and try to
- 21:55reduce it to one language that ruled
- 21:58them all. Now what was interesting is
- 22:01that you saw at this time there was a
- 22:03lot of interesting research that was
- 22:05feeding into it. the work of uh abstract
- 22:08data types uh from Galan and the ideas
- 22:11of information hiding from Dave Parnes
- 22:14uh separation of concerns uh the ideas
- 22:18today we would call it clean programming
- 22:20clean coding but it's the ideas of
- 22:22literate programming from canuth so
- 22:24these kinds of things were bubbling away
- 22:27in the late 70s and early 80s and ADA
- 22:30was a little bit of a a push to make
- 22:33that happen on a big scale no other
- 22:36industry or company could really do it
- 22:38because they didn't have the exposure or
- 22:40weight or gravitas or economic
- 22:43powerhouse as the US military at the
- 22:45time did. At the same time, you had some
- 22:48interesting work going on in
- 22:49laboratories like at Bell which had
- 22:51begat C and Unix and the like which was
- 22:54becoming incredibly important. But there
- 22:56was this crazy researcher at the time by
- 22:59the name of Bjarn Struestrip who was
- 23:01saying wow you know this is kind of cool
- 23:03but hey let's take some of these ideas
- 23:05from simula I should mention simula
- 23:07which was the first object-oriented
- 23:09language and let's see if we can apply
- 23:11them to C because you know C's got
- 23:13problems with it let's see if we can
- 23:14move about so what was happening in the
- 23:16background in academia and in in these
- 23:19fringes was the realization that we
- 23:22needed new kinds of abstractions and it
- 23:24wasn't just algorithmic abstractions
- 23:26But it was object abstractions. Turns
- 23:29out there's an interesting history
- 23:31behind that dichotomy.
- 23:33There is a discourse in Plato about that
- 23:36very kind of split in which he has he
- 23:40has a dialogue between two people who
- 23:42are you know talking about how I look at
- 23:44the world and one of them says we should
- 23:47look at the world in terms of its
- 23:49processes. This is the ancient Greek
- 23:51philosopher from like before Christ.
- 23:53that guy that Plato he he he brought up
- 23:56some parallel ideas.
- 23:58>> He brought up the ideas of the dichotomy
- 24:00of looking at the world through two
- 24:02lenses. The very Plato whose work has
- 24:04now been banned in certain US
- 24:07universities because he was so radical.
- 24:09Right? But but in one of these dialogues
- 24:11he observed that one of the writers said
- 24:14oh we have to look at the world through
- 24:16through the processes how things flow.
- 24:18And the other one said no no no we have
- 24:20to look at them through things. And this
- 24:22is where the idea of atoms came about.
- 24:24The very term atom came from Greek terms
- 24:27and and that terminology. So the idea of
- 24:31looking at the world and looking at and
- 24:34looking at the world are basically
- 24:35abstractions is not a new one. But
- 24:38people like Parnis and and others and
- 24:41the the designers of Simula said, "Wait
- 24:44a minute, we can apply these ideas to
- 24:47software itself and we can look at the
- 24:49world not just through algorithmic
- 24:51abstractions, but we can look at them
- 24:53through object abstractions. Now there's
- 24:56another factor that came into the place
- 24:58and this is where uh the inventor of
- 25:00Forran came into be. After forran he
- 25:03went off and he did this at IBM of
- 25:05course he he was made a fellow and he
- 25:07went off and said this was fun but I
- 25:08want to do something else and he said
- 25:10let's let's look at a different way of
- 25:12programming and it was the idea of
- 25:14functional programming which was looking
- 25:16at the world through mathematical
- 25:17functions stateless kinds of things so
- 25:19there was work here we are talking what
- 25:21in the the 70s now in which uh the ideas
- 25:25of functional programming came to be I
- 25:27had a chance to interview him a few a
- 25:29few months before he passed away and I
- 25:31asked asked him, you know, why did
- 25:33functional programming never make the
- 25:35big time? And his answer was because
- 25:37functional programming makes it easy to
- 25:39do hard things, but it makes it
- 25:42astonishingly impossible to do easy
- 25:44things.
- 25:44>> Easy things.
- 25:45>> Yeah. So, so functional programming has
- 25:48a role. There's no doubt. And I think
- 25:50its foundations were laid at the time by
- 25:53John. But even today, it has a role. It
- 25:57has a niche but it hasn't become
- 25:58dominant because of that very same
- 26:01edict. So any rate here we are at the
- 26:05sort of end of the first golden age of
- 26:07software engineering and moving into the
- 26:10second. What were the forces that led us
- 26:12into that? First off it was growing
- 26:15complexity. Grady just mentioned how
- 26:17growing complexity was a force pushing
- 26:18the industry into a new golden age of
- 26:20software engineering. Fast forward to
- 26:22today and software complexity keeps
- 26:24growing, growing and growing in part
- 26:26thanks AI generating a lot more code a
- 26:28lot faster. And this brings us nicely to
- 26:30our season sponsor work OS. Work
- 26:32provides the primitives that make it
- 26:34easy to make your app enterprise ready.
- 26:36But under the hood, there's so much
- 26:38complexity that happens. I know this
- 26:40because I recently took part in an
- 26:41engineering planning meeting at work
- 26:43called the Hilltop review. An engineer
- 26:45walking through their proposed
- 26:46implementation. In this review, we
- 26:48discuss how to implement authentication
- 26:49for customers when their users
- 26:51authenticate across several platforms
- 26:53using work OS. For example, what should
- 26:56happen if a user logs out on the mobile
- 26:58version? Should they stay logged in in
- 27:00the web version? What about the other
- 27:02way around? We covered 10 plus similar
- 27:04questions. The answer, as I learned,
- 27:06goes down to it depends what the
- 27:07customer using work OS wants. The work
- 27:10OS team walks through edge cases I had
- 27:12no idea existed and then turns those
- 27:14decisions into configurable behavior in
- 27:16the admin panel so customers choose the
- 27:18right trade-offs for their product and
- 27:20their users without having to build and
- 27:22maintain all of this logic themselves.
- 27:24But this is not always enough. And when
- 27:26customers have unique needs, the work
- 27:28engineering team often works with them
- 27:29directly to figure out how to solve
- 27:31their very specific problem. They then
- 27:33generalize these solutions so they
- 27:35become part of the platform for
- 27:36everyone. After this planning session, I
- 27:38have a newfound appreciation for just
- 27:40how much complexity works absorbs so
- 27:43product and engineering teams don't have
- 27:44to. The same planning goes into all work
- 27:46products and customers get all the
- 27:48benefit. Learn more at workowwise.com.
- 27:51And with this, let's get back to Grady
- 27:53and how the second golden age of
- 27:54software engineering came about. As I
- 27:56mentioned, growing complexity,
- 27:58difficulty of building software fast
- 27:59enough and building building big enough
- 28:01software and I would add to this the
- 28:03things that came about in in the defense
- 28:05world which were the desire and an
- 28:08obvious value in building systems from a
- 28:11distributed kind of way. Now come on to
- 28:13the scene because what was happening
- 28:15around that same time is the fruits of
- 28:17micro miniaturaturization came to be and
- 28:19it led us to the personal computer. This
- 28:22was because transistors, right? And and
- 28:24the breakthroughs in in like electronics
- 28:26and and
- 28:27>> precisely and you know this too was a
- 28:30vibrant time because you had you know
- 28:33you had hobbyists who could put these
- 28:35things together and and build them from
- 28:37scratch and there were no personal
- 28:38computers at the time. Was this the
- 28:40first time that hobbyists could actually
- 28:42like meaningfully get their hands on it
- 28:44in in the history of computing? Really?
- 28:46I think at scale, yes, you you had you
- 28:49had hobbyists such as Pascal back in his
- 28:52day who decided that his father was so
- 28:55tediously working over his accounting
- 28:58that Pascal built a little machine for
- 29:00him. So there was hobbyist work at that
- 29:02time, no doubt about it. But in terms of
- 29:04scale and also remember post World War
- 29:06II, you had the addition of especially
- 29:09in the United States, you had more
- 29:11disposable income which made it possible
- 29:14for hobbyists to actually do these kinds
- 29:16of things. And then lastly, you had the
- 29:18military who was producing integrated
- 29:21circuits and transistors. And all of a
- 29:23sudden, especially in Silicon Valley,
- 29:25you could go down to Fry or the Fry
- 29:28equivalent. This is before Fries and buy
- 29:30these things. they were just they were
- 29:32there and so it enabled people to play
- 29:35and play is an important part in the
- 29:37history of software. So you had this
- 29:39wonderful thing happening and I'd say
- 29:41the late 70s and early 80s which was a
- 29:44vibrant time of experimentation. There's
- 29:47a delightful book called what the
- 29:48doormouse said which posits that the
- 29:51rise of the personal computer was also
- 29:54tied together with the rise of the
- 29:57hippie counterculture. And so this this
- 30:00drive toward you know power to the
- 30:02people and you know let's you know love
- 30:05make love not war these kinds of things.
- 30:07This is the era of Steuart Brand the era
- 30:10of of the Murray pranksters and the like
- 30:12and that led to things like the well
- 30:15which was the very first social network
- 30:18which was today we call them bulletin
- 30:20boards which grew up in in Silicon
- 30:22Valley. Quick aside, Stuart just a
- 30:25lovely fellow. He was actually mentioned
- 30:27as one of the merry pranksters in uh in
- 30:30the book about uh about them. He's still
- 30:33on the scene and he's just released a
- 30:34wonderful book called maintenance part
- 30:36one which looks at the problems of
- 30:39systems. Software is one of them and the
- 30:41problems of maint associated with them.
- 30:43Anyway, here we are um late 70s early
- 30:4680s uh also a very vibrant time because
- 30:49there's a lot of cool stuff that could
- 30:51be done.
- 30:52>> Yeah. And and it's Strike Press is
- 30:53publishing this actually. So, uh, I'll
- 30:55I'll leave a link in the show notes
- 30:56below. It looks like a really nice book
- 30:58and Stride Press is known to produce
- 31:00excellent quality. So, I'm actually
- 31:02excited to look into this.
- 31:03>> Yeah, it's a great great book. So, the
- 31:06realization was that we now had the
- 31:10beginnings of theories of looking at the
- 31:12world not through processes, but through
- 31:14objects and classes. We had the the
- 31:18demand pull of distributed systems, the
- 31:21demand pull from trying to build more
- 31:22and more complex systems. And so there
- 31:25was also this perfect storm that really
- 31:27launched that second golden age. And
- 31:30that's frankly where I came onto the
- 31:32scene. I was just in a lucky place at a
- 31:33lucky time. Um I was at the time working
- 31:36at Vandenberg Air Force Base on uh
- 31:39missile systems and space systems. Uh
- 31:42there was envisioned military space
- 31:45shuttle and I was part of that program
- 31:46as well. It was great. It was a fun
- 31:48place to be because we'd have launches
- 31:50like twice a week. It was pretty cool.
- 31:51You'd run up and say, "Wow, look at
- 31:53that." It was it was pretty wild. At the
- 31:56building in which I work, I had to
- 31:57evacuate whenever there was a building,
- 31:59ever a launch because if it was a Titan
- 32:01launch, the Titan launch pad was really
- 32:03close to us and if it had blown up on
- 32:05the launch pad, it would have it would
- 32:07have blown up our building, which would
- 32:08have been really annoying. So, yeah.
- 32:11Good stuff.
- 32:12>> And one other one other quick story, you
- 32:13could always tell when it was the secret
- 32:15launches going off, the secret spy
- 32:17satellites, because there were two main
- 32:20clear indications. The first is all the
- 32:23hotels would fill up because you'd have
- 32:25the contractors come in. And second, the
- 32:27day of the launch, the highway nearby
- 32:30where you could see the launch would
- 32:31fill up with people to watch it. So
- 32:33there were no secrets in that world. So
- 32:35here we are, late 80s. uh the the world
- 32:38was poised for a new way of looking at
- 32:40the world and that was object-oriented
- 32:42programming and object-oriented design.
- 32:44So how does that differ from the first
- 32:47generation? It differs in the sense that
- 32:50we approach the world at a different
- 32:52layer of abstraction. Rather than just
- 32:55looking at the data which was this raw
- 32:57lake out here and the algorithms we have
- 33:00to manipulate them, we bring them
- 33:02together into one place. We combined the
- 33:04the objects and the and the uh processes
- 33:07together and it worked. My gosh, it'll
- 33:10enable us to do things we could not do
- 33:12before. It was the foundation for a lot
- 33:15of systems. Uh go out to the computer
- 33:17history museum and go look at the
- 33:19software for for Mac Write and Mech
- 33:22Paint. It was written in object Pascal,
- 33:24one of the early object-oriented
- 33:26programming languages. One of the most
- 33:27beautiful pieces of software I've seen.
- 33:29It's it's well structured. It's well
- 33:31organized. And in fact, much of the
- 33:34design decisions made in it, you still
- 33:36see persist in systems such as Photoshop
- 33:39today. Uh they still exist, which is an
- 33:42interesting story unto itself about the
- 33:44lifetime of software. So looking at
- 33:46software through the lens of object
- 33:48proved to be very effective because it
- 33:50allowed us to attack software, the
- 33:52software complexity problem in a new and
- 33:55new and novel way. And so much like the
- 33:58first golden age, this was also a very
- 34:00vibrant time. in I would say the the 80s
- 34:03and 90s where you had people such as the
- 34:06three amigos, me, Ivar Gakasen, uh and
- 34:10Jim Rumba, you had Pete Code, you had
- 34:13Larry Constantine was back on the scene,
- 34:15uh Ed Jordan was back on the scene, uh a
- 34:18lot of folks who were saying, "Let's
- 34:21look at software not from processes but
- 34:24from objects and think about it." Now,
- 34:26this was great. We made some mistakes.
- 34:28there was an overemphasis upon the ideas
- 34:30of inheritance. We thought this would,
- 34:32you know, be the greatest thing. Uh that
- 34:35was kind of wrong. But the idea of
- 34:37looking at the world from classes and
- 34:38objects, it was kind of built in. And so
- 34:41what began to happen, this was also an
- 34:44economic thing. As it's people started
- 34:46building these things, all of a sudden
- 34:48we saw the rise of platforms. Now there
- 34:51was precedence for this because in the
- 34:53first golden age of software people
- 34:55started you know building the same kinds
- 34:57of things over and over again. The idea
- 34:59of collecting processes collecting
- 35:02algorithms that were commonly used like
- 35:05you know how do I manipulate a hard
- 35:07drive or a drum? How do I write things
- 35:10to a teletype? How do I you know put
- 35:12things on a screen? uh these kind how do
- 35:15I sort these kinds of algorithms could
- 35:19be codified and so the first ideas of if
- 35:23you will packaging them up into reusable
- 35:25things came into be. This is when at
- 35:28least in the the the world of of
- 35:30business systems IBM share came to be.
- 35:33Share was a customer uh organized group
- 35:38that literally shared software among one
- 35:41anothers. Totally.
- 35:42>> And this was in the first golden age,
- 35:43right?
- 35:44>> This is the first golden age, right?
- 35:45>> So So this was kind of like a primitive
- 35:46or like I mean looking back a more
- 35:48primitive way of just like packaging
- 35:50stuff into like yeah related may that be
- 35:52sorting algorithms or or as you said IBM
- 35:54IBM was distributing just like functions
- 35:57and things like that.
- 35:58>> IBM wasn't doing it. It was perfect. It
- 36:00was completely public driven. IBM
- 36:02supported it but was done for it.
- 36:05>> Yeah. So the point is this was the
- 36:07earliest open- source software. So the
- 36:09ideas of open source existed and
- 36:11remember too in the economics of
- 36:13software and hardware back in the time
- 36:15software was pretty much given away free
- 36:18by the main manufacturers. IBM did not
- 36:21charge for software until later in the
- 36:24later 60s7s they realized my gosh we can
- 36:26make money and they decoupled software
- 36:28and hardware and started charging you
- 36:30for it. But in the earliest days, there
- 36:32was this vibrant community of people who
- 36:35could say, you know, gosh, I've written
- 36:36this thing. Go ahead and use it. That's
- 36:38fine. No problem. So, open source was
- 36:40was late at that time. And the same
- 36:43thing began to happen in the second
- 36:46golden age in which we saw much like the
- 36:48rise of operating systems, the rise of
- 36:51open-source software, the same phenomena
- 36:54applied in the second golden age, but
- 36:56now it was a new layer of abstraction.
- 36:59Oh, I want to have now a new uh library
- 37:03for, you know, writing to these new
- 37:06fangled CRTs. Here it is. No competitive
- 37:10value in me having it, but by gosh, it
- 37:12enables me to build some really cool
- 37:14things. You can have it, too. So, open
- 37:16source laid its roots, took its ideas
- 37:19from the first golden age, applied
- 37:22itself in the second golden age, but in
- 37:23a different kind of abstraction. Lurking
- 37:25in the background. Speaking of
- 37:26economics, was the rise of platforms
- 37:29because now all of a sudden these
- 37:30libraries are becoming bigger and
- 37:32bigger. And as we moved to distributed
- 37:35systems, there was the rise of back then
- 37:38we called it serviceoriented
- 37:39architectures. There was this need of,
- 37:41you know, we had HTML and the like. We
- 37:43could, you know, pass links back and
- 37:44forth, but there was some crazy folks
- 37:46that said, wouldn't it be cool if we
- 37:48could do things like, you know, share
- 37:49images? And that was one of the things
- 37:51that uh Netscape allowed which was they
- 37:54they produced this addition to HTML that
- 37:57allow you to put images. Wouldn't it be
- 37:59cool if we could pass messages back and
- 38:01forth via HTML? So all of a sudden uh
- 38:04the internet became via HTML protocols,
- 38:06HTTP protocols became a medium at a
- 38:09higher level abstraction for passing
- 38:12information and and processes around.
- 38:15But there was a need to package it up.
- 38:17So thus was born serviceoriented
- 38:19architectures, SOAP, the serviceoriented
- 38:22architecture, serviceoriented protocols,
- 38:24all that the predecessors to what we
- 38:26have today. And this was laying the
- 38:29foundations in the second golden age for
- 38:32the the beginnings of the platform era
- 38:35which is you know what Bezos and and
- 38:37others have really brought us to where
- 38:39jumping ahead in our current age where
- 38:41you have these islands which are sort of
- 38:45formed by all sort of APIs around them.
- 38:47But it was in the second golden age is
- 38:49they were being born. And when you say
- 38:51platforms what do you mean when you say
- 38:52the rise of platforms? What how do you
- 38:55think of a platform? AWS would be a good
- 38:57one. Uh Salesforce would be another one
- 39:00in which I have these economically
- 39:03interesting castles defended by the moat
- 39:06around them and those organizations like
- 39:09Salesforce give you access across the
- 39:11moat for you know a slight fee. Well,
- 39:14not even a slight fee.
- 39:15>> Yes. Not a slight fee.
- 39:17>> Yeah. under the assumption that we as
- 39:20like a salesforce uh the cost of you
- 39:23doing it yourself is so high it makes
- 39:26sense for you to buy from us. So during
- 39:29the second golden age we saw the rise of
- 39:32those kinds of businesses because the
- 39:34cost of certain kinds of software was
- 39:36sufficiently high and the complexity was
- 39:37certainly high it allowed the business
- 39:40and the industry of these kinds of SAS
- 39:43companies. So, let's look at the the
- 39:47late '9s, early 2000s. Also a vibrant
- 39:50time, much like the first golden age. We
- 39:52had the growth of the internet. Uh, when
- 39:55did you get your first email address?
- 39:56>> My first email address I got sometime in
- 39:59maybe 2005 six. It was still very fresh
- 40:02when Gmail launched. But when did you
- 40:04get your first email address? 1987
- 40:08when it was the ARPANET. And in fact, at
- 40:11that time, yes, we had a little book. It
- 40:13was probably a hundred pages long that
- 40:15listed the email address of everybody in
- 40:17the world. It was pretty cool. You can
- 40:18find them online and you can see my
- 40:21email there. Doesn't work anymore
- 40:22because it doesn't have the same, you
- 40:24know, top level domain kind of things.
- 40:25So, I've been on email before email was
- 40:28cool. And so as you saw these kinds of
- 40:31structures like email becoming a
- 40:33commodity thing in the second golden age
- 40:36of software, this is when software began
- 40:38to filter into the interstitial spaces
- 40:41of civilization and it became not just
- 40:44this one thing fueling businesses or
- 40:46certain domains. It became something
- 40:48that became part of the very fabric of
- 40:51civilization. This was important. And so
- 40:54now the things we worried about in the
- 40:56first golden age, we'd solved them for
- 40:59the most part. They were part of the
- 41:00very atmosphere. We didn't think about
- 41:02algorithms much because, you know, gosh,
- 41:05everybody kind of knows about them. And
- 41:07this is as technology should be. The
- 41:09best technology evaporates and
- 41:11disappears and becomes part of the the
- 41:14air that we breathe. And that's what's
- 41:16happening now. But it was in the second
- 41:19golden age. The foundations of where we
- 41:21are today are here. So what happened
- 41:24around 2000 or so? Well, we had by that
- 41:27time internet was big, lots of
- 41:29businesses being built, but there was
- 41:31the crash around that time because
- 41:34economically it just didn't make sense.
- 41:36So there was this great pullback. Also
- 41:38happening was the whole Y2K situation
- 41:41where a lot of effort was put into, you
- 41:43know, solving that problem. You know,
- 41:45people in retrospect say, well gosh, we
- 41:47didn't need to worry about that. But
- 41:49being in the middle of it, you realize,
- 41:50oh no, there was a lot of heroic work.
- 41:52And if that hadn't been done, then lots
- 41:55of problems would have happened. So this
- 41:57is a good example of how the best
- 41:59technology you simply don't see. A lot
- 42:01of effort and a lot of money was spent
- 42:03to subvert a problem that simply did not
- 42:06manifest itself. That's a great thing.
- 42:08Grady just mentioned how the best
- 42:09technology is one that you simply do not
- 42:11see. This is an underrated observation
- 42:14and it's true for most mission critical
- 42:15software. When it works, it's invisible.
- 42:18It's only when it breaks when users
- 42:20notice that it's there. There is however
- 42:21a problem with building reliable
- 42:23invisible software. There's often a
- 42:25tension between moving fast with few
- 42:27guard rails that can make things break
- 42:29or putting in more guard rails for
- 42:31stability but then slowing down in
- 42:33shipping speed. Well, there's a third
- 42:34way which leads us nicely to our
- 42:36presenting sponsor stats. Static built a
- 42:39unified platform that enables the best
- 42:40of both cultures continuous shipping and
- 42:43experimentation. Feature flags let you
- 42:45ship continuously with confidence. Roll
- 42:47out to 10% of users. Catch issues early.
- 42:49Roll back instantly if needed. Built-in
- 42:51experimentation means every roll out
- 42:53automatically becomes a learning
- 42:54opportunity with proper statistical
- 42:56analysis showing you exactly how
- 42:58features impact your metrics. And
- 43:00because it's all in one platform with
- 43:02the same product data, analytics that
- 43:04should replace everything. Teams across
- 43:06your organization can collaborate and
- 43:08make datadriven decisions. Companies
- 43:09like Notion went from single digit
- 43:11experiments per quarter to over 300
- 43:13experiments with stats. They ship over
- 43:15600 features behind feature flags,
- 43:17moving fast while protecting against
- 43:19metric regression. Microsoft, Atlashian,
- 43:21and Brex use static for the same reason.
- 43:24It's the infrastructure that enables
- 43:25both speed and reliability at scale.
- 43:28They have a generous free tier to get
- 43:29started, and pro pricricing for teams
- 43:31starts at $150 per month. To learn more
- 43:33and get a 30-day enterprise trial, go to
- 43:35stats.com/pragmatic.
- 43:37And with this, let's get back to the Y2K
- 43:39event that Grady was talking about.
- 43:41Yeah, I I I I remember how stressful
- 43:43that time was leading up to year 2000. I
- 43:46think some movies even came out uh
- 43:47predicting, you know, h how the world
- 43:49would collapse, but there was this fear
- 43:51of like will all these systems crash and
- 43:54it it it started to become pretty
- 43:55intense in in the few months leading up.
- 43:57So I I I was, you know, like a a kid at
- 44:00that time. But when the year 2000, like
- 44:03that was probably the most stressful new
- 44:05year because you weren't kind of sure.
- 44:06You were hoping, you know, and then
- 44:08nothing happened and you're like, okay,
- 44:10it was just a hoax. So anyone who who
- 44:12went through there uh like kind of
- 44:14learned to like not trust these
- 44:15predictions. But you're right like
- 44:17knowing what know there was so much work
- 44:18right to make to make sure that that
- 44:20overflow did not like hit at the wrong
- 44:22place. Yeah. So here we are mentally put
- 44:25yourself in the the first first decade
- 44:28of the 2000s is a fun place because well
- 44:31yeah the there was the crash but still
- 44:34so much fun stuff to do, so much great
- 44:37software to be written. We were still
- 44:38only limited largely by our imagination.
- 44:41Now I'm going to pause for a moment and
- 44:43backfill with some history that I hadn't
- 44:44mentioned. We've been talking about
- 44:46software in general. There was a
- 44:48parallel history going on in AI in which
- 44:50we saw also some generations. The first
- 44:53golden age of AI was in the 40s and 50s
- 44:56where you had people such as Herbert
- 45:00Simon and Newell and Minsky in
- 45:02particular. The focus there was upon
- 45:04gosh we could build intelligence
- 45:06artificially using symbolic methods. So
- 45:09this was the first golden age first
- 45:12great age of AI and the ideas of neural
- 45:15networks were tried. The the thing they
- 45:17built was the snark which was the first
- 45:20vacuum tube artificial neuron. It took
- 45:23like five vacuum tubes to make a single
- 45:25neuron. And there was a report coming
- 45:27out of the UK at the time that said
- 45:29we're spending a lot of money here but
- 45:30by gosh it doesn't work. And so the
- 45:32first golden age ended when they
- 45:34realized you can't really build anything
- 45:37interesting. And furthermore, neural
- 45:39networks are a dead end. Largely a dead
- 45:41end because we didn't have the
- 45:42computational power to do them. We
- 45:44didn't have the algorithmic concepts,
- 45:47the abstractions to to know what to do
- 45:50with them once we had them at scale. The
- 45:52second golden age of of AI was really in
- 45:56the 80s when you had people like Falcon
- 45:59come along and say hey there's another
- 46:00way of looking at it and it's looking at
- 46:02it through rules. Thus was born the idea
- 46:04of machine learning uh things like m and
- 46:07the like came upon the scene but there
- 46:10too we saw the AI winter come about. By
- 46:14the way there was an interesting rise in
- 46:15hardware at the time. The list machine
- 46:17the thinking machine were all built
- 46:19during this time. vibrant periods of
- 46:21time of a of computer architectures. So
- 46:24you see these kind of feeding into one
- 46:25another, but ultimately it failed
- 46:28because they didn't scale up once you
- 46:30got beyond a few hundred if then
- 46:32statements. We simply didn't have a
- 46:35means of building inference engines that
- 46:36could do anything with them. So here we
- 46:38are in exciting time again two first
- 46:40decade of the 2000s. AI was kind of you
- 46:43know back in in the back rooms. we still
- 46:46had a lot of cool things to do and uh
- 46:49more and more distributed kind of
- 46:50systems plus fueling that also was the
- 46:53fact that software was now in the hands
- 46:55of individuals through personal
- 46:57computers. So the demand for software
- 46:59was even greater. I would claim and this
- 47:02may be a little controversial. We are in
- 47:04the third golden age of software
- 47:06engineering but it actually started
- 47:09around the turn of the millennium. It's
- 47:11not it's not now but it's then. And the
- 47:14first indication of the rise of it is we
- 47:17saw a new rise in levels of abstraction
- 47:20from individual
- 47:22components of our software programs to
- 47:25whole libraries and packages that were
- 47:28part of our platform. Oh, I need to do
- 47:31messaging. Well, I'm not going to do
- 47:32that on my own machine. I can go out to
- 47:34this library which does messaging. I
- 47:37need to manage this whole chunk of data.
- 47:40Let's, you know, use Hadoop or something
- 47:42like that. it wasn't around the time but
- 47:44the seeds where it was growing. So we
- 47:46again saw a growth in levels of
- 47:48abstraction from just simple programs to
- 47:52now subcomponents of systems and that
- 47:55was the next great shift that happened
- 47:57and our methodologies and our languages
- 48:00and all that began to follow. So the
- 48:03third golden age we've been in for
- 48:06several years already. And not to get
- 48:09ahead of ourselves, what's happening
- 48:11with AI assistance and the like in the
- 48:13coding space is in many ways a reaction
- 48:18to the growth of those kinds of things
- 48:20because we want to accelerate their use.
- 48:23We want to we have so many of those
- 48:25kinds of libraries out there and not
- 48:27enough people know about them. We want
- 48:29to accelerate the use of them by having
- 48:31aids that help us do so. So that's the
- 48:34context in which I put AI agents such as
- 48:36cursor and chat tpt in and that they are
- 48:39in a way a follow on to the forces that
- 48:42have already led us to this third golden
- 48:44age. So we are now in a very vibrant
- 48:47time but the problems are different from
- 48:50the first and second generations. What
- 48:52are the problems now? First, it's
- 48:53problems of we have so much software.
- 48:56How do we manage it? And we have to deal
- 48:58with issues of safety and security. Can
- 49:01somebody sneak in something that I can't
- 49:03trust? How do I defend myself against
- 49:06that? It is so easy to inject something
- 49:08in the software supply chain. How do I
- 49:10prevent the bad guys from putting stuff
- 49:12inside there? How do I defend against
- 49:15it? the whole history behind stuck necks
- 49:18and the like is a good one uh to show
- 49:21you know espionage and software. And so
- 49:24all of a sudden the human issues that we
- 49:26had for much of the history of software
- 49:28we were insulated about because it was
- 49:31so much part of civilization these human
- 49:33issues became front and center clear and
- 49:36present for our world. And the other
- 49:38element is to the economic issues of it.
- 49:40We had now companies that were too big
- 49:43to fail. What would happen if a
- 49:46Microsoft were to go under? What would
- 49:47happen if a Google were to go under?
- 49:50They're so economically important to the
- 49:52world that the things they do, they
- 49:55sneeze in some part of the world catches
- 49:57a cold. And so the problems we have now
- 50:00in this third golden age of software are
- 50:03different than they were than the first
- 50:05and second generations, but equally as
- 50:07exciting. And then last, we have the the
- 50:09ethical issues. because I can do this
- 50:11kind of software, it is possible for me
- 50:13to track where you are in every moment
- 50:16of the day. I can do that. Should I do
- 50:19that? Some will say yes, I should
- 50:21because it, you know, it's a good thing
- 50:22for humanity. Others will say not so
- 50:25sure about that.
- 50:26>> So, I like how you laid it on. It's very
- 50:29interesting, especially through both
- 50:31your experience and also sharing the
- 50:33history that I think a lot of us don't
- 50:35really reflect on, which is how it all
- 50:36started and just honestly how young it
- 50:38is. If if I mean you know like 70 or 80
- 50:40years can be long depending on how old
- 50:42you are but it is it's it's not even a
- 50:44generation or barely generation.
- 50:46>> It's a couple of generations. Yeah.
- 50:48>> But one thing that I'm seeing across the
- 50:50industry right now which feels very like
- 50:52this setup makes sense but one thing
- 50:54that kind of feels it contradicts it for
- 50:56a lot of software engineers today
- 50:58>> is there seems to be an ex existential
- 51:00dread that is especially accelerating
- 51:03especially over the winter break. What
- 51:04happened over the winter break is before
- 51:06the winter break, these AI uh LLMs were
- 51:08were pretty good for autocomplete.
- 51:10Sometimes they could generate this or
- 51:12that. And over the winter break, I'm not
- 51:13sure if you played with some of I have
- 51:15with the new Yeah, with the new models,
- 51:17they actually generate really good code
- 51:20to the point that I'm starting to trust
- 51:21them. And
- 51:23>> yes,
- 51:23>> as far as the history of software has
- 51:26been, my understanding is that software
- 51:29developers have written code and it's a
- 51:30hard thing to do. And a lot of us, you
- 51:32know, it takes years for us to learn and
- 51:34to be excellent at it even longer. And
- 51:36so a lot of us are starting to have this
- 51:38really existential crisis of okay, well
- 51:41the machine can write really really good
- 51:43software code first of all like WTF and
- 51:45how did this happen over the last few
- 51:47months and then the question is what
- 51:49next? this it feels that it could shake
- 51:51the profession because I feel coding has
- 51:54been so tightly coupled to software
- 51:57engineering and and now it might not be
- 51:59you know looking at I guess you know
- 52:01like taking a breathe out first and
- 52:03looking through the both the history and
- 52:05and your your what is your take on
- 52:07what's happening right now well let me
- 52:09say that this is not the first
- 52:11existential crisis the developers have
- 52:13faced tell us more they have faced the
- 52:16same kind of existential crisis in the
- 52:18first and the second generation. So
- 52:21that's why I look at this and say, you
- 52:23know, this too will pass when I talk to
- 52:25people who are concerned about it. Don't
- 52:27worry, focus upon the fundamentals
- 52:30because those skills are never going to
- 52:31go away. I had a chance to meet Grace
- 52:33Hopper. She was just delightful, you
- 52:35know, fireplug of a woman. Just amazing,
- 52:37amazing thing. For for your readers, go
- 52:40Google Grace Hopper and David Letterman
- 52:42and there's this she appeared on the
- 52:43David Letterman show and you'll get a
- 52:45sense of her personality.
- 52:46>> Well, we're going to link in the show
- 52:47notes below. She of course is the one
- 52:49who recognized that it was possible here
- 52:52we are in the 50s that it was possible
- 52:54to separate our software from our
- 52:56hardware. This was threatening to those
- 53:00who were building the early machines
- 53:01because they said you know gosh you
- 53:03could never build anything efficient
- 53:05because you have to be a tied so closely
- 53:07to the machines and many in that field
- 53:10and they wrote about it expressed
- 53:11concerns that you know this is going to
- 53:14destroy what we do and it should have.
- 53:16So we had here the beginnings of the
- 53:18first compilers. The same thing happened
- 53:21with the invention of forran where
- 53:23people were saying gosh you know we can
- 53:25write tight assembly language better
- 53:27than anybody else better than any
- 53:28machine can kind of do but that was
- 53:31proved wrong when we moved up a level of
- 53:33abstraction from the assembly language
- 53:35to the higher order programming
- 53:36languages. And so you had a set of
- 53:39people who were similarly concerned and
- 53:42distressed by the changes in levels of
- 53:44abstraction because they recognized that
- 53:46the skills they had in that time were
- 53:48going to go away and they were going to
- 53:50be replaced by the very thing themselves
- 53:53created. Now you didn't see as much of a
- 53:56crisis because there weren't that many
- 53:59of us back in that time frame. We're
- 54:02talking, you know, a few thousands of
- 54:03people now. We're talking millions of
- 54:06people who ask quite legitimately the
- 54:08question, what does it mean for me? So,
- 54:11I've had, as I'm sure you have had, a
- 54:13number of, you know, especially young
- 54:15developers come up to me and say, Grady,
- 54:17what should I do? Am I choosing the
- 54:19wrong field? Should I, you know, do
- 54:22something different? And I assure them
- 54:24that this is actually an exciting time
- 54:27to be in software because of the
- 54:30following reasons. We are moving up a
- 54:32level of abstraction much like what
- 54:34happened in the rise from machine
- 54:36language to assembly language from
- 54:38assembly language to to higher order
- 54:41programming languages from higher order
- 54:44programming languages to libraries the
- 54:47same kind of thing happened and we're
- 54:50seeing the same change in levels of
- 54:52abstraction and now I as a software
- 54:55developer I don't have to worry about
- 54:57those details so I view it as something
- 54:59that is extraordinarily ly freeing from
- 55:02the tedium of which I had to do, but the
- 55:05fundamentals still remain. As long as I
- 55:08am choosing to build software that
- 55:10endures,
- 55:12meaning that I'm not going to build it
- 55:13and I throw it away. If you're going to
- 55:14throw it away, do what you want. That's
- 55:16great. And I see a lot of people using
- 55:19these agents for that very purpose.
- 55:21That's wonderful. You're going to go off
- 55:23and automate things you could not have
- 55:24afforded to do today. And if you're a
- 55:26single user for it, then more power to
- 55:29you. This is the hobbyist rarer and the
- 55:31hobbyist side of software if you will
- 55:34much like we saw in the earliest days of
- 55:36personal and computers where people will
- 55:38build these things. Great stuff. Great
- 55:39ideas will come from it.
- 55:41>> I like the comparison. Yes.
- 55:42>> Yeah. Great ideas will come from it. You
- 55:45know, people will build skills. We'll do
- 55:47things we could not have done before.
- 55:48We'll automate things that were
- 55:50economically not possible, but they're
- 55:52not going to endure necessarily, but
- 55:54still we will have made a valuable
- 55:56impact. And I guess just like in the
- 55:58first era where personal people could
- 55:59buy it, you will have people come into
- 56:01the industry who have honestly nothing
- 56:02to do with it and they might bring
- 56:03amazing ideas, right? Like back then,
- 56:05you know, school school teacher might
- 56:07have bought a personal computer. Today I
- 56:08I just talked to my neighbor upstairs,
- 56:10an accountant. She has instructed Chad
- 56:12GBT to build some appcript to uh help
- 56:16their accounting teams process a bit
- 56:18better because she knows how that thing
- 56:20works. Nothing to do with software, but
- 56:22now creating their own personal
- 56:24throwaway software. by the way.
- 56:26>> Yes, absolutely. The same parallels and
- 56:29I celebrate that. I encourage it. I
- 56:31think it's the most wonderful thing
- 56:33which is why we are in this vibrant
- 56:35period. In the early days of of the
- 56:36personal computer, the very same thing
- 56:39happened. You found artists drawn to
- 56:41especially the PC and the amigga at the
- 56:43time. You found gamers who realized I've
- 56:46got a new medium for expression that I
- 56:48did not have before and that's why it
- 56:50was a very vibrant time. the same thing
- 56:52is happening. And so much of the
- 56:54lamenting of oh gosh, we have an
- 56:56existential crisis are those who are
- 56:59narrowly focused upon their industry not
- 57:02realizing that what's happening here is
- 57:04actually expanding in the industry.
- 57:07We're going to see more software written
- 57:09by people who are not professionals. And
- 57:11I think that's the greatest thing around
- 57:14because now we have software much like
- 57:16in the in the counterculture era of of
- 57:20the personal computer. The same thing is
- 57:22happening today as well. I like what
- 57:24you're saying. However, one however
- 57:27[laughter] however one one thing that I
- 57:28also pay attention to uh one person I
- 57:30pay attention to is is Dario Amod the
- 57:32CEO of Antropic. And the reason I pay
- 57:34attention to him is I I try I tend not
- 57:36to pay attention to CEOs but he actually
- 57:38said about a year ago he said something
- 57:40interesting. He says he thinks most code
- 57:42will be generated by AI about 90% of it
- 57:44maybe in a year and then more and we
- 57:47thought that's silly and then he was
- 57:49right and code was generated and now he
- 57:51said some another thing interesting that
- 57:53sounded interesting but the next one
- 57:55sounds scary he said I quote software
- 57:57engineering will be automatable in 12
- 57:59months now this sounds a lot more
- 58:03scarier for reasons we know coding is a
- 58:04subset of software engineering but he
- 58:06said this what is your take on on this
- 58:08and you've had you've had a strong
- 58:10response already. So,
- 58:13>> u I have one or two things to say about
- 58:15it. So, first off, I use Claude. I use
- 58:18Anthropics work. I think it's it's my
- 58:20it's my go-to system. I've been using it
- 58:23for problems with JavaScript, with
- 58:25Swift, uh with PHP of all things and
- 58:28Python. So, I use it and it's it's been
- 58:31a great thing for me primarily because,
- 58:34you know, there are certain libraries I
- 58:35want to use. Google search sucks.
- 58:38documentation for these things suck and
- 58:40so I can use these agents to accelerate
- 58:43my understanding of them. But remember
- 58:45also I have a foundation of at least one
- 58:48or two years of experience in these
- 58:50spaces okay a few decades where I sort
- 58:53of understand the fundamentals and
- 58:55that's why I said earlier that the
- 58:57fundamentals are not going to go away
- 58:59and this is true in every engineering
- 59:00discipline the fundamentals are not
- 59:02going to disappear the tools we apply
- 59:06will change so Daario man I I respect
- 59:10what you're saying but recognize also
- 59:12that Daario has a different point of
- 59:16view than I do. He's leading a company
- 59:18who needs to make money and it's a
- 59:22company who he needs to speak to his
- 59:24stakeholders. So outrageous statements
- 59:26will be said like that. I think he said
- 59:28these kind of things at Davos if I'm not
- 59:30mistaken.
- 59:30>> It it was very Yes.
- 59:32>> And and I'd say politely well I'll use a
- 59:35scientific term in terms of how I would
- 59:37characterize what Daario said and put it
- 59:40in context. It's utter uh
- 59:43that's the technical term because I
- 59:46think he's profoundly wrong and and he I
- 59:48think he's wrong for a number of
- 59:49reasons. First, I accept his point of
- 59:51view that it's going to accelerate some
- 59:53things. Is it going to eliminate
- 59:55software engineering? No. I think he has
- 59:57a fundamental misunderstanding as to
- 59:59what software engineering is. Go back to
- 1:00:01what I said at the beginning. Software
- 1:00:03engineers are the engineers who balance
- 1:00:06these forces. So we use code as one of
- 1:00:10our mechanisms, but it's not the only
- 1:00:12thing that drives us. None of the things
- 1:00:14that he or any of his colleagues are
- 1:00:16talking about attend to any of those
- 1:00:19decision problems that a software
- 1:00:21engineer has to deal with. None of those
- 1:00:23we see within the within the realm of
- 1:00:26automation. His work is primarily
- 1:00:28focused upon the automation at the
- 1:00:30lowest levels which is I would put akin
- 1:00:33to what was happening with compilers in
- 1:00:35these days. That's why I say it's
- 1:00:37another level abstraction. Fear not, O
- 1:00:40developers. Your tools are changing, but
- 1:00:43your problems are not. There's another
- 1:00:45reason why I I push back on what he's
- 1:00:47saying. And that is if you look at
- 1:00:49things like cursor and the like, they
- 1:00:52have mostly been trained upon a set of
- 1:00:54problems that we have seen served over
- 1:00:57and over again. And that's okay. Much
- 1:01:00like I said in the first generation,
- 1:01:02first golden age, we had a certain set
- 1:01:04of problems. And so libraries are built
- 1:01:06around them. The same thing is happening
- 1:01:08here. If I need to build a UI on top of
- 1:01:11CRUD, it's sub winter or some web
- 1:01:14ccentric kind of thing. I can do it. And
- 1:01:16much like your friend, more power to
- 1:01:19them. They can do it themselves because
- 1:01:20the power is there to do so. They're
- 1:01:23going to, you know, probably not build a
- 1:01:24business around it. Some small percent
- 1:01:26of them might do so. But it's enabled
- 1:01:28them to do things they could not do
- 1:01:30before because they're now at a higher
- 1:01:32level abstraction. what Daario neglects
- 1:01:34and I used a a bit of a paraphrase from
- 1:01:38from Shakespeare. There are more things
- 1:01:40in computing Daario that are dreamt of
- 1:01:43in your philosophy. The world of
- 1:01:45computing is far larger than web centric
- 1:01:48systems of scale. So we see many of the
- 1:01:51things applied today on these webric
- 1:01:53systems and I think that's great and
- 1:01:55wonderful but it means that there's
- 1:01:58still a lot of stuff out there that
- 1:01:59hasn't yet been automated. So we have we
- 1:02:02keep pushing these fringes away. So I
- 1:02:04told you those stories at the beginning
- 1:02:05because history is repeating itself
- 1:02:07where some will say history is rhyming
- 1:02:10again. The same kinds of phenomena are
- 1:02:12applying today just at a different level
- 1:02:14of abstraction. So that's the first one.
- 1:02:16Software is bigger than this world of
- 1:02:18software is bigger than what he's
- 1:02:19looking at. It's bigger than just
- 1:02:21software intensive systems. And then
- 1:02:23second, you know, if you look at the
- 1:02:26kinds of systems that most of these
- 1:02:28agents deal with, they are in effect
- 1:02:30automating patterns that we see over and
- 1:02:33over again for which they have been
- 1:02:34trained upon. Patterns themselves are
- 1:02:37new abstractions that are in effect not
- 1:02:40just single algorithms or single
- 1:02:42objects, but they represent societies of
- 1:02:45objects and algorithms that work
- 1:02:47together. These agents are great at
- 1:02:49automating generations of patterns. I
- 1:02:51want to do, you know, this kind of thing
- 1:02:53and I can tell you in English because
- 1:02:55that's how I describe the pattern. So
- 1:02:57anyway, that's why I think he's wrong.
- 1:02:59More power to him. But, you know, I
- 1:03:01think this is an exciting time more than
- 1:03:03things to worry about exist
- 1:03:05existentially. Let me offer another
- 1:03:07story with regards to how we see a shift
- 1:03:10in levels of abstraction. English is a
- 1:03:12very imprecise language full of
- 1:03:15ambiguity and nuance and the like.
- 1:03:17Though one would wonder how could I ever
- 1:03:20make that you know as a useful language
- 1:03:22and the answer is we already do this as
- 1:03:24software engineers. I go to somebody and
- 1:03:26say hey I want my system to do this. It
- 1:03:29kind of looks like this and I give them
- 1:03:31some examples. I do that already. And
- 1:03:34then somebody goes and turns that into
- 1:03:35code. We've moved up a level of
- 1:03:37abstraction to say I'd like it to do
- 1:03:39this. I'll give you a concrete example.
- 1:03:41I'm working with a library I'd never
- 1:03:43touched before. It's the JavaScript D3
- 1:03:45library which allows me to do some
- 1:03:47really fascinating visualizations. I go
- 1:03:50off and search for a site called
- 1:03:53Victorian Engineering Connections. It's
- 1:03:55just this lovely little site where the
- 1:03:57gentleman did this for a museum Andrew
- 1:04:01and you can, you know, put in a name
- 1:04:02like George Bool and you see his name,
- 1:04:05you find things about him and you find
- 1:04:07his social network around him and you
- 1:04:09can go touch it and explore. It's very,
- 1:04:11very cool. And I said,"I want that kind
- 1:04:13of thing, but my gosh, I don't know how
- 1:04:14to do that. So, what can I do?" He gave
- 1:04:17me his code. I realized it uses the D3
- 1:04:20library. I knew nothing about the D3
- 1:04:22library. So, I said to Cursor, "Go build
- 1:04:24me the simplest one possible. Go do it
- 1:04:27out of, you know, five nodes and show
- 1:04:29me." So, I could then study the code.
- 1:04:31And then I could say, "Well, what they
- 1:04:32wanted would really wanted to do is
- 1:04:34this. Go make the nodes look like this,
- 1:04:37depending upon their kind." So, just
- 1:04:39like I would do with a human, I was
- 1:04:41expressing my needs in an English
- 1:04:44language that now all of a sudden I
- 1:04:47didn't need to labor to turn that into
- 1:04:49reality. I could simply have a
- 1:04:51conversation with my tool to help me do
- 1:04:54that. So, it it reduced the distance
- 1:04:57between what I wanted and what it could
- 1:05:01do. And I think that's great. That's a
- 1:05:02breakthrough. But remember, as I said to
- 1:05:04Daario, this only works in those
- 1:05:06circumstances where I'm doing something
- 1:05:08that people have done hundreds and
- 1:05:10hundreds of times before. I could have
- 1:05:12learned it on my own. As Fineman would
- 1:05:14have said, you know, go do it yourself
- 1:05:16because then that's the only way you're
- 1:05:17going to understand. And I my reaction
- 1:05:19is that's great, but there's so much in
- 1:05:21the world I'm curious about. I can't
- 1:05:23understand it all. Let's go, you know,
- 1:05:25let's decide what I want to do. So go do
- 1:05:27it for me. So that's why I say these
- 1:05:29kinds of tools are another shift in the
- 1:05:31levels of abstraction because they're
- 1:05:33reducing the distance from what I'm
- 1:05:35saying my English language to the the
- 1:05:37programming language. Last thing I'll
- 1:05:39say is that you know what do we call a
- 1:05:41language that is precise and expressive
- 1:05:44enough to be able to build executable
- 1:05:46artifacts? We call them programming
- 1:05:49languages. And it just so happens that
- 1:05:51English is a good enough programming
- 1:05:53language much like Cobalt was in that if
- 1:05:57I give it those phrases in a domain that
- 1:06:00is well enough structured, it allows me
- 1:06:03to have good enough solutions that I who
- 1:06:05know those fundamentals can begin
- 1:06:07nudging and cleaning out the pieces.
- 1:06:09That's why the fundamentals are so
- 1:06:11important. And speaking of history
- 1:06:13rhyming, one thing that happened in both
- 1:06:15the first age and the the sec second
- 1:06:17golden age or as we jumped abstractions
- 1:06:19or every time we had an abstraction is
- 1:06:22some skills became obsolete and then
- 1:06:24there was a demand for for new skills.
- 1:06:26For example, when we from assembly level
- 1:06:28the the skill of like knowing how the
- 1:06:31instruction set of a certain board and
- 1:06:33knowing how to optimize it, that became
- 1:06:34obsolete in favor of thinking at a
- 1:06:36higher level. In this jump right now
- 1:06:39where I think it's safe to say we're
- 1:06:41going from we do not need to write any
- 1:06:42more code and the computer will do it
- 1:06:44pretty good and we'll check it and tweak
- 1:06:46it. What do you think will become
- 1:06:48obsolete and what will become more
- 1:06:50important as software professionals?
- 1:06:51>> Great question. The software delivery
- 1:06:55pipeline is far more complex than it
- 1:06:58should be. Uh that my gosh just getting
- 1:07:01something running is hard if you have no
- 1:07:03pipeline. If you're within a company
- 1:07:05such as a Google or a Stripe or
- 1:07:07whatever, you have
- 1:07:09>> you have a huge infrastructure about
- 1:07:11around them.
- 1:07:12>> A custom one.
- 1:07:13>> Yes.
- 1:07:14>> Yeah. A custom one. Yes. And so there is
- 1:07:16lowhanging fruit for the automation of
- 1:07:18those. I mean I don't need a human that
- 1:07:20fills in the edges of those kind of
- 1:07:22things. By the way, I'm talking about in
- 1:07:24effect infrastructure is software.
- 1:07:27[clears throat]
- 1:07:27>> It's not just, you know, not just raw
- 1:07:28lines of code. So, this is lowhanging
- 1:07:31fruit where we could begin seeing these
- 1:07:33agents that say, "Hey, you know, I want
- 1:07:36you to go, you know, gosh, I don't know,
- 1:07:38you know, spin up something for this
- 1:07:40part of the world. I don't want to write
- 1:07:42the code for that stuff because it's
- 1:07:43complex and messy. I'd rather use an
- 1:07:46agent that helps me do it." So there's a
- 1:07:48case where I think you're going to have
- 1:07:50the loss of jobs in those places where
- 1:07:53it's messy and complex because the
- 1:07:55automation has clear economic and you
- 1:07:59know frankly value in terms of security.
- 1:08:01That's a place where people are going to
- 1:08:03need to reskill in the building of
- 1:08:05simple applications and the like. Well,
- 1:08:08I think you know people who had uh who
- 1:08:10had skills in saying I want to build
- 1:08:12this you know thing for iOS or whatever
- 1:08:15they're going to lose you know they're
- 1:08:16going to lose some jobs cuz frankly
- 1:08:18people could do it just by you know
- 1:08:20prompting it that's great that's fine
- 1:08:23because we've enabled a whole another
- 1:08:24generation of folks to do things that
- 1:08:26professionals did in the past exactly
- 1:08:28what happened in the era of PCs
- 1:08:30themselves what should these people do
- 1:08:32move up a level of abstraction start
- 1:08:34worrying about systems so the shift now
- 1:08:37I think is less so from dealing with
- 1:08:40programs and apps to dealing with
- 1:08:42systems themselves and that's where the
- 1:08:44new skill set should come in. If you
- 1:08:46have the skills of knowing how to manage
- 1:08:48complexity at scale if you know as a
- 1:08:51software engineer how to deal with all
- 1:08:54of these multiple forces which are human
- 1:08:56as well as technical your job's not
- 1:08:59going to go away. If anything, there
- 1:09:00will be even greater demand for what
- 1:09:02you're doing because those human skills
- 1:09:05are so rare and delicate.
- 1:09:06>> So, you mentioned the importance of of
- 1:09:08having strong foundations and and you've
- 1:09:10previously said, I'm actually quoting
- 1:09:13you, the field is moving at an incomp
- 1:09:15incomprehensible pace for people without
- 1:09:17deep foundations and a strong model of
- 1:09:18understanding. What foundations would
- 1:09:21you recommend people to look at? both
- 1:09:23students, people who are at university
- 1:09:25studying or looking for their first job
- 1:09:27and also software professionals who you
- 1:09:30know now actually want to go back and
- 1:09:31strengthen those foundations that that
- 1:09:33will be helpful. I find
- 1:09:36my my uh my happy place if you will, my
- 1:09:40sweet space that I retreat back to when
- 1:09:42I'm faced with a difficult problem back
- 1:09:44into systems theory. go read the work of
- 1:09:47of Simon and Newell in the the sciences
- 1:09:51of the artificial. Uh there's a whole
- 1:09:53set of work that's come out on
- 1:09:55complexity and systems from the Santa Fe
- 1:09:58Institute. It's those kinds of
- 1:10:00fundamentals of system theory that
- 1:10:03ground me in the next set of things in
- 1:10:05which I want to build. I think I
- 1:10:07mentioned to you in in one of our our
- 1:10:09previous discussions, I was doing some
- 1:10:11really interesting work on NASA's
- 1:10:13mission to Mars. we were faced with an
- 1:10:15issue of saying, "Hey, you know, we we
- 1:10:18want to, you know, have people go off on
- 1:10:20these long missions. We want to put
- 1:10:21robots on the surface of Mars." And so I
- 1:10:23was commissioned to go off and think
- 1:10:25about that for a while. And in effect, I
- 1:10:27realized NASA wanted to build a howl.
- 1:10:30And you'll notice I've got a how above
- 1:10:32me here.
- 1:10:32>> Yes.
- 1:10:32>> Uh this is I I'm a great one for
- 1:10:34history. This is my sword of Dicles that
- 1:10:37passes behind me. If you know the
- 1:10:39history behind the sword of Dacles, the
- 1:10:41king Damacles, he was always kept humble
- 1:10:44because at his throne there was a sword
- 1:10:47right above him on a thread. So he felt,
- 1:10:50you know, constantly, you know, unease.
- 1:10:53And this is why I have Hal behind me as
- 1:10:55well. For for some reason, NASA didn't
- 1:10:57want the kill all the astronauts use
- 1:10:59case. Don't understand why, but we we
- 1:11:02threw that one kind of out. But if you
- 1:11:04look at the problems there, this is a
- 1:11:05systems engineering problem because you
- 1:11:08needed something that was embodied in
- 1:11:10the spacecraft.
- 1:11:12Much of the kind of software we have
- 1:11:13today in AI is disembodied. Uh the
- 1:11:16cursor, the copilot and like they have
- 1:11:19no connection to the physical world. So
- 1:11:21our work was primarily in embodied
- 1:11:23cognition. Around the same time, I was
- 1:11:26studying under a number of
- 1:11:27neuroscientists trying to better
- 1:11:29understand the architecture of the
- 1:11:30brain. And here's where the fundamentals
- 1:11:34of that came together for me because I
- 1:11:36began to realize there are some certain
- 1:11:38structures we see in systems engineering
- 1:11:42that I can apply to the structure of
- 1:11:44these really large systems. Taking ideas
- 1:11:47of Marvin Minsky society of mind which
- 1:11:50is a way of of systems architecting
- 1:11:52multiple agents. We're in agent
- 1:11:54programming now which I think people are
- 1:11:55just beginning to tap upon how those
- 1:11:58things apply. they need to go look at
- 1:12:00systems theory because that problem has
- 1:12:02been looked at with multiple agents
- 1:12:03already. Go read Minsky society of mine.
- 1:12:06You'll see some ideas that will guide
- 1:12:08you there in dealing with multiple
- 1:12:10agents. The ideas from bears of uh which
- 1:12:13was manifest in early AI systems such as
- 1:12:16hearsay. The ideas of of global
- 1:12:19workspaces, blackboards and the like.
- 1:12:21Another architectural element. the ideas
- 1:12:24of subumption architectures from uh from
- 1:12:27Rodney Brooks. Uh his was influenced by
- 1:12:30by biological things. If you look at a
- 1:12:32cockroach, a cockroach is not a very
- 1:12:35intelligent thing. But we know there's
- 1:12:38there's there's not a central brain in
- 1:12:39it and yet it does some magnificent
- 1:12:41things. We have been able to map the
- 1:12:44entire neural network of the common
- 1:12:46worm. We're not flush with, you know,
- 1:12:48evil worms running around the world.
- 1:12:50There's something else going on there.
- 1:12:52But biological systems have an
- 1:12:54architecture to them. So to go back to
- 1:12:56your question by looking at architecture
- 1:12:59from a systems point of view from
- 1:13:01biology from uh neurology from systems
- 1:13:05in in the real world as Herbert Herbert
- 1:13:07Simon and New did this is what's guiding
- 1:13:10me to the next generation of systems and
- 1:13:12so I would urge you know people looking
- 1:13:14at systems now go back to those
- 1:13:17fundamentals. There is nothing new under
- 1:13:19the sun in many ways. We've just, you
- 1:13:22know, applied them in different ways.
- 1:13:24Those fundamentals in engineering,
- 1:13:27they're still there. And then as
- 1:13:28closing, uh, you gave some really good
- 1:13:30recommendations to read, to ponder, to
- 1:13:33educate yourself, and and get ideas that
- 1:13:35will probably useful in this new world,
- 1:13:37especially as as we're going to have a
- 1:13:38lot more agents. For example, like I now
- 1:13:40just heard that agents will be part of
- 1:13:42Windows 11 and operating system. So,
- 1:13:44they will be everywhere. But looking
- 1:13:46back at the the previous rises of
- 1:13:49abstractions and also the previous
- 1:13:51golden ages, the people who who did
- 1:13:53great at the start of a new golden age
- 1:13:56or at the start of a new abstraction
- 1:13:58even if they were not amazing at the
- 1:13:59previous one, what have you seen those
- 1:14:02people do? Like what and and based on
- 1:14:04this historical lesson, what would you
- 1:14:06recommend if if we were just kind to
- 1:14:07kind of copy successful, you know,
- 1:14:10things that that that people did because
- 1:14:12I feel this is an opportunity as well,
- 1:14:13right? we have this rise of abstraction.
- 1:14:15A lot of people will be paralyzed. But
- 1:14:16there will be new superstars being born
- 1:14:18who will be basically riding the wave
- 1:14:20and they will be the experts of uh
- 1:14:22agents of of AI of building these new
- 1:14:24and complex a lot more complex systems
- 1:14:26that we could have done before.
- 1:14:28>> So I as I alluded to earlier the main
- 1:14:32thing that constrains us in software is
- 1:14:34our imagination. Well actually that's
- 1:14:36where we begin. We're actually not
- 1:14:37constrained by imagination. We can dream
- 1:14:39up amazing things and yet we are
- 1:14:41constrained by the laws of physics by
- 1:14:43how we build algorithms and the like
- 1:14:44ethical issues and the like. So what's
- 1:14:47happening now is that you are actually
- 1:14:50being freed because some of the
- 1:14:52friction, some of the constraints, some
- 1:14:55of the costs of development are actually
- 1:14:58disappearing for you. Which means now I
- 1:15:01could put my attention upon my
- 1:15:03imagination to build things that simply
- 1:15:06were not possible before. I could not
- 1:15:09have done them because I couldn't have
- 1:15:10raised a teen to do them. I couldn't
- 1:15:12have afforded that. I could not have uh
- 1:15:14done it because I couldn't have had the
- 1:15:16reach in the world as I did before. So
- 1:15:19think of it as an opportunity. So it's
- 1:15:22not a loss. It'll be a loss for some who
- 1:15:24have a vested interest in the economics
- 1:15:26of this, but it's an a net gain because
- 1:15:29now all of a sudden these things unleash
- 1:15:32my imagination to allow me to do things
- 1:15:35that were simply not possible before in
- 1:15:37the real world. This is an exciting time
- 1:15:40to be in the industry. It's frightening
- 1:15:42at the same time, but that's as it
- 1:15:44should be. When there's an opportunity
- 1:15:46where you're on the cusp of something
- 1:15:47wonderful, you should look at the abyss
- 1:15:50and say, you can either take a look and
- 1:15:52say, "Crap, I'm gonna fall into it." Or
- 1:15:54you can say, "No, I'm going to leap and
- 1:15:57I'm going to soar. This is the time to
- 1:15:59soar."
- 1:16:00>> Grady, thank you so much for giving us
- 1:16:02the the overview, the outlook, and and
- 1:16:04for and for a little bit of perspective.
- 1:16:06I I personally really appreciate this,
- 1:16:08>> and I hope I offered some hope as well.
- 1:16:11>> I think you definitely did. This was a
- 1:16:13really inspiring episode. Thank you,
- 1:16:15Grady. One thing that really struck with
- 1:16:17me was when Grady pointed out that
- 1:16:18developers [music] have faced this exact
- 1:16:20existential crisis before, multiple
- 1:16:23times, in fact. When compilers came
- 1:16:25along, assembly programmers thought
- 1:16:26their careers were over. When highle
- 1:16:28languages emerged, [music] the same fear
- 1:16:30ripped through the industry. And each
- 1:16:32time the people who understood what
- 1:16:34actually was happening, that [music] it
- 1:16:35was just a new level of traction, they
- 1:16:37came out ahead. This historical lens is
- 1:16:39something that I think we often miss
- 1:16:41when some of us are caught up in the
- 1:16:42[music] day-to-day anxiety of new AI
- 1:16:44capabilities. I don't think we're at the
- 1:16:46end of software engineering and neither
- 1:16:48does a Grady. We're at the beginning of
- 1:16:50another chapter and if history has any
- 1:16:52guide, it's going to [music] be a pretty
- 1:16:53exciting one. If you found this episode
- 1:16:55interesting, please do subscribe in your
- 1:16:57favorite podcast platform and [music] on
- 1:16:58YouTube. A special thank you if you also
- 1:17:00leave a rating on the show. Thanks and
- 1:17:03see you in the next one.
About this transcript
This page contains the full transcript of The third golden age of software engineering – thanks to AI, with Grady Booch by The Pragmatic Engineer, generated from the public captions YouTube serves with the video. The transcript has 14,247 words across 2,061 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.