YouTube2Text

Dave Farley - Vibe Coding - Is this really the best we can do? - AI Native DevCon June 2026 — Transcript

by AI Native Dev · 4,836 words · 798 segments · language en · Watch on YouTube

Full transcript

  1. 0:00I'm not going to say [music] we saved
  2. 0:01the best for last cuz that's probably
  3. 0:03not doing justice for our other speakers
  4. 0:04as well. But I'm really interested in
  5. 0:06this session and Dave Farley is a great
  6. 0:08a great speaker.
  7. 0:09So vibe coding is this the best we can
  8. 0:11do? I think this is going to be an
  9. 0:12interesting look at the processes and
  10. 0:15the practices we do today with vibe
  11. 0:16coding. What needs to go forward into
  12. 0:19our new practices?
  13. 0:21This is the last session of the day
  14. 0:22before the party. So afterwards I'll let
  15. 0:24you know a little bit more about what
  16. 0:26the party is doing. But in the meantime,
  17. 0:28please make sure you're still doing your
  18. 0:30ratings and and things like that on the
  19. 0:31app.
  20. 0:32But without further ado, please give it
  21. 0:34up for Dave Farley.
  22. 0:37>> [applause]
  23. 0:40>> Thank you.
  24. 0:43Thank you. I'm the last speaker between
  25. 0:45you and beer.
  26. 0:47So I'll try not to go on too long. Um
  27. 0:51I'm interested in the the things that
  28. 0:54seem durable. And that's may seem odd
  29. 0:57when we're talking about such a
  30. 0:58disruptive change as we are talking
  31. 1:00about when we're talking about
  32. 1:03the adoption of AI for programming.
  33. 1:06Um but I think there's quite a lot of
  34. 1:08things that are durable. There's a lot
  35. 1:10of behaviors
  36. 1:12that sta- have stood the test of time
  37. 1:14and seem to continue to be important.
  38. 1:17And I'm not the only person that's been
  39. 1:19saying that today. If you've been
  40. 1:21listening carefully, there's been people
  41. 1:22saying very similar things as we go
  42. 1:25through. I'd like to look at maybe some
  43. 1:28of the reasons why why this matters and
  44. 1:32question some of the the things that
  45. 1:34that we might otherwise think about the
  46. 1:36new world of agentic programming.
  47. 1:40I think one of the things that we can
  48. 1:42certainly say though is that whatever
  49. 1:43does come next, it's going to be
  50. 1:45different. This is a huge change. This
  51. 1:49is a sea change in the way that our
  52. 1:52industry and probably our industry is
  53. 1:55just a forerunner, how our world is
  54. 1:57going to work.
  55. 1:59So so so what does this really mean?
  56. 2:01What what does this mean in terms of
  57. 2:03the impact on people like us who are
  58. 2:06involved in the production of software
  59. 2:08at the moment and
  60. 2:09I suppose to some degree the pathfinders
  61. 2:13for everybody else in the world as they
  62. 2:15start to adapt to some of these things.
  63. 2:18I think a starting point, if you'll
  64. 2:20forgive me for being slightly
  65. 2:21provocative, some of the differences
  66. 2:23that we talk about kind of just dumb.
  67. 2:26So I would argue that vibe coding,
  68. 2:29programming with natural languages,
  69. 2:32while having a place,
  70. 2:35are also kind of bad ideas. And I want
  71. 2:38to talk a little bit about what I mean
  72. 2:39by that and we'll get into it.
  73. 2:42I think
  74. 2:44sacking all junior developers, AIs will
  75. 2:46write all the code, we don't need no
  76. 2:48programmers. Some of these are kind of
  77. 2:51right, some of them are probably true,
  78. 2:53and some of these are a terrible idea.
  79. 2:57I think AI generated tests
  80. 3:01also a terrible idea. So I'd like to
  81. 3:03explore all of these things in a bit
  82. 3:05more detail and there's quite a lot of
  83. 3:06nuance in this because there are aspects
  84. 3:09of these where they're all true.
  85. 3:11But let's talk about it.
  86. 3:14So first of all, I'd like to I'm a I'm a
  87. 3:17old school card carrying test-driven
  88. 3:20development developer. So let's start
  89. 3:22with tests cuz that's where you're
  90. 3:24supposed to start.
  91. 3:25What's a test for? Is it to prove
  92. 3:27success?
  93. 3:29Not really.
  94. 3:32Is it to find our mistakes?
  95. 3:34Well, kind of but not really. Is it to
  96. 3:36challenge my genius as a programmer?
  97. 3:38Certainly not.
  98. 3:40It's not those are not really what tests
  99. 3:42are for.
  100. 3:44They're not what make tests useful. I
  101. 3:46would argue that what tests are for us,
  102. 3:50they're a form of measurement. They are
  103. 3:53equivalent of a carpenter having a tape
  104. 3:55measure in his pocket
  105. 3:57that he can measure things with.
  106. 4:00We use tests to figure out that we're
  107. 4:03doing the right things. We use tests to
  108. 4:06figure out that when we've done them,
  109. 4:08they they continue to be the right
  110. 4:09things.
  111. 4:10They're [snorts] measurements to see if
  112. 4:12we've achieved our goals.
  113. 4:15But for that, we've got to define what
  114. 4:16our goals are.
  115. 4:19And we can't infer the the goals from
  116. 4:22the solution cuz the solution can always
  117. 4:24be wrong. So, anybody that's selling you
  118. 4:28free AI testing that will come in and
  119. 4:30look at your existing system and test
  120. 4:32it,
  121. 4:33that might have some value, but it's not
  122. 4:36the same value as proper testing that we
  123. 4:39need as part of a development process.
  124. 4:41It's a different thing. And the reason
  125. 4:44for that is fairly obviously. If I If I
  126. 4:46write a function something like this,
  127. 4:49calculate tax,
  128. 4:52and
  129. 4:53I'm going to return 50 times the amount
  130. 4:55that I put in. That's a very bad place
  131. 4:57to live if that's what your tax looks
  132. 4:58like.
  133. 5:00But if that's my starting point, the
  134. 5:02only thing that an agentic AI or any
  135. 5:04other kind of AI can do
  136. 5:06is look at that and say, "Okay,
  137. 5:09here's my test."
  138. 5:11He's just going to reinforce the
  139. 5:12wrongness that I wrote into that
  140. 5:14function. All that a test can do at that
  141. 5:17point because the only input it's got
  142. 5:20is the test
  143. 5:22is the working code. He's verified that
  144. 5:24the working code does what the working
  145. 5:25code does.
  146. 5:27That has its use uses. That's very good
  147. 5:30if you want to refactor the code, do
  148. 5:32behavior-preserving change, and not
  149. 5:34break anything.
  150. 5:36But it's rubbish if you want to develop
  151. 5:37a new system and verify that this new
  152. 5:40system continues to do or does the right
  153. 5:43things because now it's proving that it
  154. 5:46does the wrong thing
  155. 5:48because that's where we started from.
  156. 5:50So, that that's not giving us our
  157. 5:52ability an ability
  158. 5:54to
  159. 5:55define what we really wanted.
  160. 5:57AI-generated tests, if the code is the
  161. 5:59only input, we can only verify that the
  162. 6:01code remains the same.
  163. 6:03We can't infer the goals from the
  164. 6:05solution because the solution [laughter]
  165. 6:07can always be wrong.
  166. 6:13>> [clears throat and snorts]
  167. 6:13>> So, they're mostly a dumb idea.
  168. 6:16They have a They have a place, but
  169. 6:18mostly a dumb idea. They tend to be a
  170. 6:20cop-out for people who don't can't be
  171. 6:22bothered to state their goals. And I
  172. 6:24think that's a problem because
  173. 6:26specifying the goals is kind of what our
  174. 6:29job is. It's kind of what we're here to
  175. 6:31do as software developers.
  176. 6:38My next question, what's a program for?
  177. 6:41To define a sequence of instructions.
  178. 6:45Well, not really. That's not the goal.
  179. 6:47It might be what it is, but to encode
  180. 6:50algorithms. That's not the goal again of
  181. 6:53what of a program. That's not what we do
  182. 6:55it for. That's not the value that it
  183. 6:57brings.
  184. 7:00To implement our brilliant design.
  185. 7:02Again, that's some of the
  186. 7:04self-fulfillment that I as a programmer
  187. 7:07might get from it, but that's not the
  188. 7:09goal. That's That's not the goal. That's
  189. 7:11not the reason that people pay me to do
  190. 7:13it.
  191. 7:16So, the commercial pressure is not none
  192. 7:18of those things. That's not really what
  193. 7:21it's for as much as we might might like
  194. 7:23to think of it in that way as
  195. 7:24programmers.
  196. 7:25Programming languages I would argue have
  197. 7:27three goals as tools.
  198. 7:31So, first, they are tools that have been
  199. 7:34designed to help us to organize our
  200. 7:36thinking about a problem. They allow us
  201. 7:39to explore the surface area of of a
  202. 7:41problem and kind of understand it in
  203. 7:43more detail, in more depth. And they're
  204. 7:46designed to work that way. Programming
  205. 7:48languages aren't difficult because we
  206. 7:51want it to be obscure and abstruse. We
  207. 7:54wanted tools that would help us to
  208. 7:56explore the problems in in these kind of
  209. 7:58way. They're designed to help us to do
  210. 8:01that.
  211. 8:03They're also a means of communicating
  212. 8:05our understanding with other
  213. 8:06programmers, other humans.
  214. 8:09So, they are communication tool between
  215. 8:10us so we can work as part of a team and
  216. 8:13understand what we are in some technical
  217. 8:15detail exploring the depth and the
  218. 8:18breadth and the complexity of a
  219. 8:20of a the problem that we're trying to
  220. 8:22solve and the solution that we've come
  221. 8:24up with
  222. 8:25encoded as a program in in a programming
  223. 8:27language.
  224. 8:29And ultimately, they're there to tell a
  225. 8:32computer what to do. But, that's kind of
  226. 8:34the last part because assembly language
  227. 8:36instructions in in form a computer what
  228. 8:39to do perfectly well. And we don't not
  229. 8:41many of us write spend our time writing
  230. 8:43programs in assembly language.
  231. 8:52There are also three techniques that are
  232. 8:54embodied in programming languages that
  233. 8:56are I think are important
  234. 8:58for us to think about and to remember
  235. 9:00when we're talking about programming as
  236. 9:02a discipline.
  237. 9:04The first is we have a simple consistent
  238. 9:07grammar. It allows us to
  239. 9:10express our ideas in in a concrete,
  240. 9:14specific, precise way,
  241. 9:17precisely enough to be executable by a
  242. 9:18computer.
  243. 9:21They are an unambiguous expression of
  244. 9:24our intent in that respect. And that
  245. 9:26this this matters.
  246. 9:28And they're repeatable and deterministic
  247. 9:30in terms of execution. And this matters
  248. 9:32a lot, too. If we write something in the
  249. 9:35programming language of our choice and
  250. 9:38run it twice
  251. 9:40given the same inputs, we're going to
  252. 9:41get the same result every time.
  253. 9:45That's important. It means that we can
  254. 9:47reason about it. It means that we can
  255. 9:48test it. It means that we can understand
  256. 9:51what it means and we can design systems
  257. 9:53that are more or less repeatable in that
  258. 9:55sense, but the tools themselves are
  259. 9:57repeatable.
  260. 10:01How does natural language measure up to
  261. 10:03that? Because natural language Somebody
  262. 10:05said today I I think they quoted Andre
  263. 10:08Karpathy saying that the programming
  264. 10:11language of the future is English.
  265. 10:14I don't like that idea very much for the
  266. 10:16reasons that I'm talking about. It's not
  267. 10:18precise enough. It's
  268. 10:20too vague.
  269. 10:22Natural language helps us
  270. 10:24How does that measure up? Helps us to
  271. 10:25organize our thinking about a problem.
  272. 10:28Well, it's too vague, really.
  273. 10:31Natural language is open to
  274. 10:33misinterpretation.
  275. 10:35If anybody's been around, I can't really
  276. 10:36see you very well, but if you've got a
  277. 10:39gray beard like mine or you've been
  278. 10:40around enough,
  279. 10:42I bet that you've been in a situation
  280. 10:45where somebody's coming down from on
  281. 10:47high and given a
  282. 10:49a very vague instruction to a bunch of
  283. 10:51programmers, gone away, and then been
  284. 10:53really disappointed in the result cuz
  285. 10:55they didn't tell us in enough detail.
  286. 10:57That's really common. It's really easy
  287. 10:59to misinterpret what what the goal
  288. 11:01really is.
  289. 11:07Natural language is always open to
  290. 11:09interpretation.
  291. 11:11Time flies like an arrow. These are some
  292. 11:13time flies like an arrows.
  293. 11:19Vibe coding alone is simply not good
  294. 11:22enough. If we're just chatting with the
  295. 11:24computer to express our needs, that's
  296. 11:26not enough. We need tools that allow us
  297. 11:30to be more precise than that, to be more
  298. 11:32specific than that, to be better prompts
  299. 11:35of what it is that we want to achieve.
  300. 11:37That doesn't mean
  301. 11:39that we need to look at all the code and
  302. 11:42are only allowed to do this in a
  303. 11:44programming language.
  304. 11:45I mean I'm in the category of it's been
  305. 11:48a long time now since I've So I've
  306. 11:50written any code by hand cuz my AI agent
  307. 11:54writes all the code. But I've been more
  308. 11:56precise than just natural language in
  309. 11:59some respects in specifying what I want
  310. 12:02from it.
  311. 12:03I'm able to get what I want from it by
  312. 12:05using a precise prescriptive version of
  313. 12:08natural language in order to be able to
  314. 12:10prompt it. And that's what I'm really
  315. 12:12talking about here.
  316. 12:14So,
  317. 12:15does it communicate our understanding
  318. 12:17with other humans?
  319. 12:20It's a bit too ambiguous to do that very
  320. 12:22well because we all as we've as we all
  321. 12:25know.
  322. 12:26Does it tell a computer what we want it
  323. 12:28to do? It's too imprecise for that. So
  324. 12:30it's very easy unless what we're telling
  325. 12:33it is something very obvious like asking
  326. 12:36our AI assistant to implement FizzBuzz
  327. 12:38for us.
  328. 12:40It might do that in a repeated way, but
  329. 12:42otherwise it's probably not. It's going
  330. 12:44to be a little bit different and give us
  331. 12:45something a little bit different every
  332. 12:47time.
  333. 12:49Certainly in the earlier versions of AI
  334. 12:52coding assistants, we've all seen that.
  335. 12:55Where we've asked our assistant to build
  336. 12:57something, you ask it the same thing
  337. 12:59again, and you get something very
  338. 13:00different.
  339. 13:03That certainly happened to me a few
  340. 13:05times.
  341. 13:12So, we've got these What about these
  342. 13:14three techniques that we have? Simple
  343. 13:16consistent formal grammar.
  344. 13:19Does natural language give us that?
  345. 13:22No, it's complex and inconsistent.
  346. 13:26Unambiguous expression of intent. Now,
  347. 13:28it's ambiguous and vague. Repeatable
  348. 13:30deterministic
  349. 13:31execution. Not repeatable, not
  350. 13:34deterministic, not version controllable
  351. 13:35as a result. We can if we version
  352. 13:38control the language that we prompted
  353. 13:40our AI with, that's not enough to get
  354. 13:42back to a the same result again.
  355. 13:46On its own.
  356. 13:48So, programming with AI presents three
  357. 13:52important problems for us to replace
  358. 13:56traditional programming.
  359. 13:59So, how do we specify what we want with
  360. 14:02precision? That's problem one.
  361. 14:06Problem two is how do we confirm that we
  362. 14:09got what we wanted? This is the
  363. 14:11verification problem.
  364. 14:14And problem three is that we don't work
  365. 14:17in the same way as the machines. They
  366. 14:21don't work in the same way as us. Either
  367. 14:23way, which which you
  368. 14:25look at that. The way that we do things
  369. 14:28is that we tend to deal with problems
  370. 14:31with with the limits of our own context
  371. 14:33window. So, what will fit inside our
  372. 14:35heads at a time. We might have a picture
  373. 14:37of what it is that we want to build, but
  374. 14:39then
  375. 14:40small change by small change, the way
  376. 14:43that we deal with it is that we do it
  377. 14:45incrementally. We do a small piece of
  378. 14:46work, we evaluate that piece of work,
  379. 14:49and then we then we move on to the next
  380. 14:51small piece of work. I would argue that
  381. 14:55high-quality system development
  382. 14:59uh in software is always an incremental
  383. 15:03process of learning and discovery.
  384. 15:05We are always exploring the problem
  385. 15:08space and trying to understand it over
  386. 15:11time, bit by bit by bit. I'd go further
  387. 15:13than that. I would say that's all
  388. 15:15engineering. All engineering is a is an
  389. 15:18act of
  390. 15:21incrementally improving on what we know
  391. 15:24about the problem and in dealing with
  392. 15:27the with the new things that we learn as
  393. 15:29we do that over time.
  394. 15:32And that's a hard part of software
  395. 15:34development, a hard part of engineering
  396. 15:36in general. And then we've got to verify
  397. 15:40that we got what we wanted and it does
  398. 15:42all of the things that we need in a
  399. 15:44repeatable reliable way. And that's a
  400. 15:47hard part of software development as
  401. 15:49well.
  402. 15:51And what have we done? We sped up the
  403. 15:53coding bit. That was the easy part of
  404. 15:55software development, which is good.
  405. 15:58It's great. It does accelerate us,
  406. 16:01but it also moves the bottleneck. If
  407. 16:04you've ever read The Goal, the theory of
  408. 16:06constraints, that's what we've done.
  409. 16:08We've just moved the bottleneck to
  410. 16:10somewhere else. And actually, we
  411. 16:12probably haven't moved the bottleneck
  412. 16:13cuz the bottleneck was nearly always
  413. 16:15about verification and release rather
  414. 16:17and figuring out what that we really
  415. 16:19wanted rather than the encoding of it if
  416. 16:21we were any good
  417. 16:23as delivery agents.
  418. 16:27So, an AI works differently. An AI works
  419. 16:30It kind of builds this context, this
  420. 16:32picture,
  421. 16:34and then it goes bang and it makes a
  422. 16:36huge change.
  423. 16:38Probably
  424. 16:40until recently you you kind of rewrite
  425. 16:42the whole system every time.
  426. 16:44And there's nothing really to stop it
  427. 16:45while it's building your stock control
  428. 16:47system to write Space Invaders instead
  429. 16:50if we haven't told it not to, if we
  430. 16:52haven't defined what our goals are and
  431. 16:55are in a in a position where we're able
  432. 16:57to evaluate it. Now, part of the problem
  433. 16:59here
  434. 17:01is that a lot of people that talk about
  435. 17:04using AI AI's and
  436. 17:07agents to help with coding
  437. 17:10are often talking from a perspective of
  438. 17:12building small simple systems. I don't
  439. 17:14know about all of you, but those aren't
  440. 17:16the sorts of systems that I generally
  441. 17:19build professionally. They're certainly
  442. 17:20how I've played with the AIs. I've I've
  443. 17:23written Space Invaders with an AI agent,
  444. 17:26and I've done simple things like that.
  445. 17:28All those nice straightforward things.
  446. 17:30But my professional building of software
  447. 17:33has been for bigger, more complicated
  448. 17:36systems than that.
  449. 17:37And that's not good enough. Just Just
  450. 17:39being able to check them manually to see
  451. 17:41if they're okay is not even close to
  452. 17:44being good enough to verify that we get
  453. 17:47what we wanted.
  454. 17:50So,
  455. 17:52if the AI is making huge steps and
  456. 17:55changing things in one big
  457. 17:57big big um
  458. 17:59change at a time, and we're not able to
  459. 18:02verify that we got what we wanted
  460. 18:03effectively, how do we learn? How do we
  461. 18:06find our understanding? How do we
  462. 18:08reliably update our solution to improve
  463. 18:10it? And how do we regenerate if if it's
  464. 18:13regenerating it from scratch every time?
  465. 18:16We can't. It's not really helping us to
  466. 18:18learn and adapt and figure out where we
  467. 18:21are and figure out what we need to do
  468. 18:22next.
  469. 18:25It's It's It's a problem.
  470. 18:31The last two problems, problems two and
  471. 18:33three, mean that we're losing our
  472. 18:35ability to work incrementally. And that
  473. 18:38is
  474. 18:39fundamental to
  475. 18:41building complex systems of any kind,
  476. 18:44but certainly complex software systems.
  477. 18:47It's a this process of learning and
  478. 18:49discovery. So, how do we keep our
  479. 18:51ability to learn, discover, and
  480. 18:53incrementally add to our systems as we
  481. 18:56go?
  482. 18:57We have to have We have to have a means
  483. 19:00that can keep up with the rate at which
  484. 19:02we can produce software.
  485. 19:04I spoke to somebody not very long ago
  486. 19:06who who's who's engaged wholeheartedly
  487. 19:10in the adoption of agentic programming
  488. 19:12using multiple agents to build systems.
  489. 19:15He's writing a game. He reckons he make
  490. 19:17he writes 12,000 lines of code per day.
  491. 19:21No human being can review 12,000 lines
  492. 19:25of code per day. No human being can test
  493. 19:29manually test the output of 12,000 lines
  494. 19:32a day behavior to figure out whether
  495. 19:34it's doing the right things. We've got
  496. 19:36to find ways of speeding up the rate of
  497. 19:39verification and assurance to tell what
  498. 19:41the whether we got what we wanted to
  499. 19:43keep up with that speed of generation of
  500. 19:46code. It's the only thing that makes
  501. 19:48sense, it seems to me, if we're not to
  502. 19:50suffer from
  503. 19:51um the theory of constraints problem.
  504. 19:58How do we
  505. 19:59confirm then
  506. 20:01that we always get what we want? So,
  507. 20:04first we've got to specify what we want
  508. 20:07clearly, reproduce in a reproduceable
  509. 20:10manner, and then we've got to be able to
  510. 20:12verify repeatably that we still have it
  511. 20:15after every change.
  512. 20:19If you're familiar with my work with
  513. 20:20continuous delivery and so on, that
  514. 20:23might sound familiar.
  515. 20:25It's cuz it is. It It It It's the same
  516. 20:28that we've been doing when we're talking
  517. 20:29about practicing continuous delivery.
  518. 20:31It's table stakes. We have to have this
  519. 20:34fast cycle of verifying that we got what
  520. 20:37we want.
  521. 20:41So, how should programming change to
  522. 20:43keep up with all of this?
  523. 20:47So, problem one,
  524. 20:48specifying what we want with precision.
  525. 20:51In the past, the way that we did this
  526. 20:54was that a program was a precise
  527. 20:56solution encoded as algorithms.
  528. 21:00Weirdly, we kind of lost what the
  529. 21:02problem was. It's kind of implicit in
  530. 21:04our solution, but we'd keep that in our
  531. 21:06heads we'd build a solution and that
  532. 21:09would
  533. 21:10be our definition
  534. 21:12that we'd work to.
  535. 21:16But the the solution itself was
  536. 21:17implicit.
  537. 21:20So in this in my example, this is a
  538. 21:22routing algorithm. So I'd like to find
  539. 21:25my way home is the outcome that I'm
  540. 21:27trying to achieve here.
  541. 21:34In the future,
  542. 21:36a program as is a will be a precise
  543. 21:39description of what it is that we want,
  544. 21:41I think. Encoded as specifications,
  545. 21:44translated into executable instructions
  546. 21:47by the AI that will verify that we got
  547. 21:50what we wanted and what we want now is
  548. 21:53explicitly part of, if you like, the
  549. 21:56program. So the program moves away from
  550. 22:00being
  551. 22:01solution focused to being a more
  552. 22:04accurate description of the problem that
  553. 22:06we're trying to solve.
  554. 22:09I'm describing a form of spec spec
  555. 22:11driven development, but the
  556. 22:12specification takes the form of
  557. 22:14executable specifications, tests, if you
  558. 22:17like, that both specify what we want and
  559. 22:21also verify that we got it.
  560. 22:26>> [clears throat]
  561. 22:28>> What it takes to achieve that. Program
  562. 22:30uh I've just This is I'm just repeating
  563. 22:32what I said. Designing a domain specific
  564. 22:34language, if you like, for specifying
  565. 22:36what it is that we want from our
  566. 22:37software with a deal of precision.
  567. 22:41And then specifying what we want aim to
  568. 22:43achieve in detail all of the time. And
  569. 22:46those are those are all of the behaviors
  570. 22:48of the system.
  571. 22:50Not just the conventional user behaviors
  572. 22:53that we might think about. We're not
  573. 22:54talking about sketchy testing of the
  574. 22:56system, but all of the behaviors of the
  575. 22:57system. So how fast would we like the
  576. 23:00results back if performance matters? How
  577. 23:02secure do we want it to be? If we want
  578. 23:04secure that system to be secure, because
  579. 23:07all of those are contextual. You can't
  580. 23:08have the AI to infer that, to make it
  581. 23:11up, cuz it depends.
  582. 23:13The real you know, how secure do you
  583. 23:15want your system to be? You'd be stupid
  584. 23:18to apply the level of security to a
  585. 23:22single user game than you would if you
  586. 23:24were building a system for a bank, cuz
  587. 23:26you'd be over-engineering the game and
  588. 23:28probably under-engineering the bank.
  589. 23:31So, you need to be specific about what
  590. 23:34it is that you're trying to achieve
  591. 23:36in all of the dimensions of architecture
  592. 23:38and design. So, some of those are
  593. 23:39behaviors of the system that we need to
  594. 23:40specify, too.
  595. 23:42The second problem, how does it how
  596. 23:44about that? So, how do we confirm that
  597. 23:46we got what we wanted? In the past, we
  598. 23:49hoped that our solution solved the
  599. 23:50problem.
  600. 23:52Language verified some aspects of this
  601. 23:54in terms of the syntax that we use,
  602. 23:56particularly if we use types languages
  603. 23:58and stuff like that.
  604. 24:01But, they didn't it it didn't work to
  605. 24:03verify functional correctness. If you
  606. 24:06were a well-behaved test-driven
  607. 24:07development developer,
  608. 24:09then testing would would fill that gap
  609. 24:12and would verify that our code did what
  610. 24:15we wanted it to do from that set from
  611. 24:17that in that sense, but otherwise maybe
  612. 24:19not.
  613. 24:22In the future,
  614. 24:23uh executable specifications
  615. 24:26will work as the verification of our
  616. 24:29system as well as the specification of
  617. 24:32what it is that we aim to achieve.
  618. 24:35I think.
  619. 24:37This makes the AI we can do this in in
  620. 24:41human language. This doesn't have to be
  621. 24:43difficult or hard. The way that I do it
  622. 24:46is that I teach my AI how to do the kind
  623. 24:50of
  624. 24:51BDD style uh tests that I like to use,
  625. 24:55and I specify what I want, and my AI
  626. 24:58fills in the gaps from relatively simple
  627. 25:00English descriptions of what it is what
  628. 25:02the specification that I that I of what
  629. 25:05I want.
  630. 25:08AI generates solutions that match the
  631. 25:10specifications.
  632. 25:12And now we can run those tests against
  633. 25:15those solutions. We can
  634. 25:17verify that the AI is doing the right
  635. 25:19thing by giving test test values that it
  636. 25:22hasn't seen before, so it can't cheat
  637. 25:24the tests.
  638. 25:27And get feedback that we got we actually
  639. 25:29did get what we wanted. What it takes to
  640. 25:32achieve that,
  641. 25:33the programmer
  642. 25:37invents problem-specific DSL,
  643. 25:40defines all of the desired behavior of
  644. 25:42the system as prompts to the AI in the
  645. 25:44form of BDD-style specifications,
  646. 25:47and the AI
  647. 25:49takes the specifications, builds test
  648. 25:51infrastructure to make them executable,
  649. 25:53generates code to fulfill all of the
  650. 25:55executable specifications,
  651. 26:00and we've got what we wanted. We've got
  652. 26:01the the def a precise definition of the
  653. 26:04behaviors that we want, and a
  654. 26:06verification that we got those
  655. 26:08behaviors.
  656. 26:11Regaining the last of the three,
  657. 26:13regaining incrementalism.
  658. 26:15So, in the past, we would work
  659. 26:17iteratively in small steps, adding new
  660. 26:19tests and code.
  661. 26:21We would gather feedback validated by
  662. 26:24our deployment pipelines.
  663. 26:26Build systems incrementally, treat each
  664. 26:28change as an experiment, and value
  665. 26:30empirical learning.
  666. 26:31In the future,
  667. 26:36we're going to work iteratively in small
  668. 26:38steps, adding new tests and code. The AI
  669. 26:41will generate the code.
  670. 26:43Gather feedback to validate every
  671. 26:44change.
  672. 26:48Build systems incrementally, many small
  673. 26:50changes, treat each change as an
  674. 26:51experiment, and value empirical
  675. 26:53learning. So, it's exactly the same. So,
  676. 26:55the stuff if you've been practicing
  677. 26:56continuous delivery
  678. 26:58as
  679. 27:00two of the speakers that I heard
  680. 27:01speaking today
  681. 27:04said
  682. 27:05at least implicitly
  683. 27:09then
  684. 27:10things become easier
  685. 27:12because that's what we're doing the same
  686. 27:14things. We're evaluating stuff as we go,
  687. 27:17working in small steps, and we're
  688. 27:18understanding where we are at all times.
  689. 27:21And so are our AI agents.
  690. 27:24What does this take to regain the
  691. 27:26incrementalism in this sense? Well, we
  692. 27:27version control of the specifications
  693. 27:29cuz those are now in effect a program.
  694. 27:34We run the tests ourselves in a
  695. 27:36deployment pipeline.
  696. 27:38Otherwise, the AI is going to cheat.
  697. 27:42I'm going to stop the AI gaming the test
  698. 27:44by using the sorts of techniques that I
  699. 27:45mentioned about maybe keeping some tests
  700. 27:48back and not showing the AI. Uh for
  701. 27:51going to what's
  702. 27:52so it can't cheat them.
  703. 27:55Most of this that I've described is how
  704. 27:57acceptance testing with behavior-driven
  705. 27:59development already works. So, this
  706. 28:01isn't anything necessarily new. If
  707. 28:04you've been working this way before,
  708. 28:06this is a really simple step.
  709. 28:11Here's an example that I've used.
  710. 28:15I've done this now several times, many
  711. 28:18times for building real systems. This is
  712. 28:21not a real system. This is a This is an
  713. 28:23example from a training course that I
  714. 28:25that I did an exercise with.
  715. 28:27So, here are some These kinds of
  716. 28:29specifications that I'm talking about.
  717. 28:31If you think about what I'm trying to
  718. 28:33describe
  719. 28:34we start off with a vague wish of what
  720. 28:36we want. We get to a slightly more
  721. 28:38formal version of describing that, which
  722. 28:40is a user story. And then we come up
  723. 28:43with some examples that would
  724. 28:44demonstrate that the feature that the
  725. 28:45story describes exists. Those are our
  726. 28:48examples, those are our executable
  727. 28:50specifications. That's what these things
  728. 28:52are on the screen. These are the ones
  729. 28:54that prompted building a system that
  730. 28:56actually did this. It actually did
  731. 28:58flight planning in this way.
  732. 29:05My conclusions are that natural language
  733. 29:08alone is not good enough.
  734. 29:11BDD style domain-specific languages are
  735. 29:14the best alternative for prompting
  736. 29:16effectively and giving us the
  737. 29:18combination of precise specification of
  738. 29:21what we want and built-in verification
  739. 29:24that we got it.
  740. 29:26Automating testing from a solution
  741. 29:30is a bit of a joke. It's it's a niche
  742. 29:33it has some utility, but it's a corner
  743. 29:36case. It doesn't solve the real problem.
  744. 29:40We should treat skills to analyze and
  745. 29:42decompose problems as the real core of
  746. 29:44our discipline. That's the real
  747. 29:46discipline that we as engineers,
  748. 29:49technologists bring. It's that
  749. 29:52way of reasoning about a problem, of
  750. 29:54decomposing it and thinking about the
  751. 29:56corner cases.
  752. 29:58Thinking about the kind of the
  753. 30:00architectural level. One of the ways of
  754. 30:02thinking about all of this is that
  755. 30:04effectively what we're doing is we're
  756. 30:06involved in invoking a fifth-generation
  757. 30:10programming language, a fifth-generation
  758. 30:12programming system. And I think that's
  759. 30:14where we're at.
  760. 30:17So, we need a different model for the
  761. 30:18role of AI. AI assistance is rather like
  762. 30:21the compiler. We're not going to
  763. 30:23care for very much longer at all if we
  764. 30:26do now about the code that it generates
  765. 30:28cuz we'll verify that we got the
  766. 30:30results. And as long as we get the
  767. 30:31results, who cares about the
  768. 30:32implementation? It's rather like the
  769. 30:35move from assembly language programming
  770. 30:38to high-level language programming. I
  771. 30:40don't know whether anybody in the room
  772. 30:42is old enough to remember that. I
  773. 30:44I don't quite, but I I was an assembly
  774. 30:46language programmer for a while. And for
  775. 30:48a while
  776. 30:50I used to look at what the assembly
  777. 30:52language was that was generated by my
  778. 30:54compilers, but I haven't done that for
  779. 30:56years. And most people have never done
  780. 30:58that.
  781. 30:59So, why should we care about what the
  782. 31:01code that was generated was if we can
  783. 31:03verify that the behaviors that we wanted
  784. 31:05are the ones that we got.
  785. 31:06>> [snorts]
  786. 31:09>> If you'd like to see a demonstration of
  787. 31:11this in use, there's an open source
  788. 31:13project called Nwave that's worth a
  789. 31:15look.
  790. 31:16Um that imply
  791. 31:18that applies this thinking and promotes
  792. 31:21this way of development.
  793. 31:28Skip.
  794. 31:38So, that's
  795. 31:42I'm going to skip over here cuz I got uh
  796. 31:45Where am I? There.
  797. 31:49That's the end of my talk. Thank you
  798. 31:51very much.

About this transcript

This page contains the full transcript of Dave Farley - Vibe Coding - Is this really the best we can do? - AI Native DevCon June 2026 by AI Native Dev, generated from the public captions YouTube serves with the video. The transcript has 4,836 words across 798 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.