What 6 months of AI coding did to my dev team — Transcript

Full transcript

1. 0:00If you're building software in your
2. 0:01business, your dev team is changing
3. 0:03faster than you realize. Not the people,
4. 0:05but the work itself. I'm running a
5. 0:0720-person software team at We use it,
6. 0:09and over the last 6 months, I've watched
7. 0:11something strange happen. The bottleneck
8. 0:13in software development has changed.
9. 0:15It's not where it used to be. It's
10. 0:16moved. And if you're hiring developers
11. 0:18right now, or you're trying to figure
12. 0:19out why your team isn't shipping faster,
13. 0:21you need to watch this.
14. 0:25Here's how software development used to
15. 0:27work. You hired a team of developers to
16. 0:29write code. You measured their output in
17. 0:31lines committed, in tickets closed, in
18. 0:34features shipped. The craft was in the
19. 0:36code itself. When I started building We
20. 0:38use it, that's exactly how we operated.
21. 0:39We hired people who could turn tickets
22. 0:41into working features. Code review was
23. 0:44the quality gate. If it passed code
24. 0:45review, it shipped. But something
25. 0:47fundamental has shifted. We started
26. 0:49using AI coding tools, Claude code,
27. 0:51cursor, and the entire rhythm broke. The
28. 0:54code started arriving faster than we
29. 0:56could process it. So, the job actually
30. 0:58changed.
31. 1:013 months ago, one of our senior
32. 1:02engineers came to me, visibly
33. 1:04frustrated. He'd spent like 3 days
34. 1:06reviewing pull requests from an
35. 1:07engineer, a junior engineer that had
36. 1:09used Claude code. There was like
37. 1:10thousands of lines of code, but the
38. 1:12application worked. But he looked at me
39. 1:13and he said, "I didn't actually read all
40. 1:15the code. I couldn't read all the code.
41. 1:17There was too much of it. What do I do
42. 1:18now?" And that question made me think,
43. 1:20cuz I'd been feeling the same thing, but
44. 1:21I just couldn't name it. Around the same
45. 1:23time, I came across some findings from a
46. 1:25retreat called ThoughtWorks, which is
47. 1:27basically senior engineers from the
48. 1:29world's biggest tech companies brought
49. 1:31together to find out what happens when
50. 1:33AI writes the code. And they didn't
51. 1:35leave with answers from this retreat.
52. 1:37They left with a map of fault lines,
53. 1:39places where traditional software
54. 1:41development is cracking right now. And
55. 1:43reading through it felt like reading my
56. 1:44own history from building We use it.
57. 1:46You've got the cheating agent problem,
58. 1:48where AI writes broken code, and then it
59. 1:51writes broken tests to validate the
60. 1:53broken code. You got the productivity
61. 1:55experience paradox, where your
62. 1:57developers are more productive but more
63. 1:59miserable. I've actually seen this in
64. 2:01our team. The migration of code review
65. 2:03from code review to specifications. We'd
66. 2:05actually started writing more more
67. 2:06strict specifications docs without even
68. 2:08realizing why. And now we know. What I'm
69. 2:10seeing and what we're struggling with in
70. 2:12our own teams looks like it's something
71. 2:13that's happening also in the biggest
72. 2:15tech companies in the world. The work is
73. 2:17migrating. The skills that matter
74. 2:19changing. And if you're hiring
75. 2:20developers or you're managing a tech
76. 2:21team and you don't see where the work is
77. 2:23going, you'll end up with the wrong
78. 2:25people doing the wrong things. So let me
79. 2:27show you what's actually happening.
80. 2:31Here's what nobody tells you about AI
81. 2:33writing code. The engineering quality
82. 2:35doesn't vanish, it just moves upstream.
83. 2:37Think about a normal user story. I want
84. 2:40to upload a photo. Your developers will
85. 2:41know what that means, you know, JPEG or
86. 2:43PNG, uploaded to a site and you got a
87. 2:45progress bar, you know, cuz cultural
88. 2:47context fills in the gaps. But an AI
89. 2:49doesn't have that context. You need to
90. 2:51be really specific. I read a story
91. 2:52recently of a developer asking an AI to
92. 2:54write a notification system of some
93. 2:56kind. You know, simple request. It
94. 2:58worked beautifully in testing when it
95. 3:00was built. And then it went into
96. 3:01production and started sending like
97. 3:0250,000 emails in a few minutes. Turns
98. 3:05out that there was no rate limiting set
99. 3:06up in in the specs. See, the engineering
100. 3:08rigor that we used to apply after the
101. 3:10code was written now needs to apply
102. 3:12before in the specs, before a single
103. 3:14line of code has been written. We've
104. 3:16gone back to techniques that felt dead.
105. 3:18Now we need to go back to structured
106. 3:19requirements, state machines, decision
107. 3:22tables, extremely detailed PRDs. It's
108. 3:25the kind of formal documentation that
109. 3:27Agile was supposed to kill. But here's
110. 3:28the thing, all that documentation makes
111. 3:31AI incredibly effective at writing code.
112. 3:33When we feed an agent a state machine
113. 3:35that displays and shows exactly what
114. 3:37states are possible within the
115. 3:38application, the code it generates is
116. 3:39almost always correct. It's crazy cuz
117. 3:41the specification became the product.
118. 3:44The code is dispensable. Think about it,
119. 3:47if you've got a perfect test suite and
120. 3:49you decide to rewrite your back end from
121. 3:51Node.js to Rust, all you got to do is
122. 3:53ask it. You just feed the tests into the
123. 3:55agent and you say, "Do the rewrite from
124. 3:58Node.js to Rust and make sure that these
125. 4:00tests pass." And the AI will get to work
126. 4:02and make sure that it tests itself on
127. 4:04the work using the tests, so then the
128. 4:06output will always work. This is a
129. 4:08complete inversion. So, if you're hiring
130. 4:11developers right now, the question is
131. 4:12isn't can they write clean code? The
132. 4:15question is, can they write a
133. 4:17specification clean enough that an AI
134. 4:20can't misinterpret it? Can they write a
135. 4:22set of tests for a test suite that
136. 4:24catches hallucinations before
137. 4:25production? You see, those are the
138. 4:26different skills and most developers
139. 4:28don't have them yet.
140. 4:32There is a layer of work in my team that
141. 4:34doesn't quite have a name yet. It sits
142. 4:35between writing code and shipping to
143. 4:37production. I call it supervisory work.
144. 4:40Basically, breaking down problems into
145. 4:42agent-sized chunks, knowing when to let
146. 4:44the agent run and when to step in.
147. 4:46Fixing the output not by actually
148. 4:48rewriting the code, but by rewriting the
149. 4:50prompt. And here's what surprised me. My
150. 4:52team is currently split into two groups
151. 4:54primarily. Group one is the more senior
152. 4:56people who understand the whole system
153. 4:58architecturally. And they're drowning
154. 5:00cuz they're spending the majority of
155. 5:01their time doing code reviews. And then
156. 5:02you got group two, the more junior ones
157. 5:05that are spending their time writing
158. 5:06code using Claude Code and other AI
159. 5:09tools at 10x the speed that they were
160. 5:11doing it before. Basically, generating a
161. 5:12lot more code, but that code doesn't
162. 5:14ship itself. It needs architectural
163. 5:16review. It needs to fit into our
164. 5:18structure. It basically needs checking
165. 5:20before it can be deployed. So, the most
166. 5:21senior engineers we have have become
167. 5:23traffic controllers, too busy reviewing
168. 5:25AI code and other people's code to
169. 5:28actually build anything themselves. The
170. 5:30more junior developers, they're
171. 5:31thriving. No muscle memory telling them
172. 5:33to write code in a specific way, so
173. 5:35they're using AI tools like a teammate,
174. 5:37not a threat to their identity. In the
175. 5:39old days, you used to hire a junior and
176. 5:41they used to take 6 months or so of
177. 5:42draining the rest of the team for that
178. 5:44junior to become productive. Now, a
179. 5:46junior can get
180. 5:48they can be writing useful code into
181. 5:50production within a week. But, I think
182. 5:51there's this this this this danger level
183. 5:53of mid-level developers. The guys that
184. 5:56have a few years of experience, they're
185. 5:57used to writing code in the way that
186. 5:58they normally write code before AI
187. 6:00existed. And retraining them to use AI
188. 6:03in an effective way is extremely
189. 6:05difficult cuz they need to change their
190. 6:07mindset around instead of focusing on
191. 6:09the syntax and the code that they're
192. 6:11writing around a detailed implementation
193. 6:14request on how they talk to the model to
194. 6:16achieve the result that they need. So,
195. 6:17here's what I'm learning as a CEO hiring
196. 6:19developers and running a development
197. 6:21team. The job description has changed.
198. 6:23If you're looking for people that can
199. 6:24write code fast, you're looking at the
200. 6:26wrong skill. You need to look for people
201. 6:28that can architect systems, write
202. 6:30unambiguous specs, and supervise AI
203. 6:34agents. And that's a completely
204. 6:35different person to the old-school
205. 6:37developer that we hired a few years ago.
206. 6:41Let me tell you a story. Last month at
207. 6:43around 2:00 a.m., one of our servers
208. 6:46broke. It was spitting out a error 503,
209. 6:48service unavailable. Our on-call
210. 6:50engineer at the time, you know, a guy
211. 6:51pretty sharp and really really capable,
212. 6:54he put this into AI to see what he
213. 6:56needed to do. The AI tool looked at the
214. 6:58error, read the documentation, and said,
215. 7:00"Restart the server." So, our engineer
216. 7:02restarted the server.
217. 7:03Uh and then a few minutes later after
218. 7:04restarting, it crashed again. So, then
219. 7:06he repeated the process and the AI said,
220. 7:07"Restart the server." So, he restarted
221. 7:09the server again. And then he repeated
222. 7:11the process again, and AI said, "Restart
223. 7:13the server." By the time he'd escalated
224. 7:15to a senior engineer, he'd restarted the
225. 7:17server six times. The senior engineer
226. 7:19looked at the logs for about 30 seconds
227. 7:21and knew exactly what the problem was.
228. 7:22Turns out the database connection pool
229. 7:24was full because of some batch cron job
230. 7:26that was running in the background. You
231. 7:28see, that's not documented anywhere.
232. 7:30That's tribal knowledge. That's lived
233. 7:32experience. And an AI doesn't have that.
234. 7:35Well, at least ours didn't. It sees 503,
235. 7:38it reads the manual, restart the server.
236. 7:40Typically, that's what you would do.
237. 7:41But, without that other knowledge,
238. 7:43without that other bit of information,
239. 7:45you're just in that cycle of restart the
240. 7:46server, it goes up, it crashes, restart
241. 7:48the server, it comes up, and it crashes
242. 7:50again. This is why I think all this hype
243. 7:51about self-healing systems is rubbish
244. 7:54right now. Unless you've really got all
245. 7:56the knowledge in the AI's context, all
246. 7:59the knowledge that a human would have,
247. 8:01all the knowledge that a senior human
248. 8:02would have. To make an AI agent
249. 8:04effective during an outage, you need to
250. 8:05build what the ThoughtWorks retreat
251. 8:07called an agent subconscious. Basically,
252. 8:10a knowledge graph of every incident,
253. 8:13every weird edge case, every bit of
254. 8:15undocumented institutional knowledge
255. 8:17that lives in your senior engineers'
256. 8:19heads. We're starting to build this at
257. 8:20Wayfair. You see, every time something
258. 8:22breaks, we document not just what
259. 8:24happened, but how we fixed it, and what
260. 8:27would be in a senior engineer's head
261. 8:29when fixing it. You know, that bit of
262. 8:30information that is just known by the
263. 8:32human, we document that. But, then
264. 8:34there's a valid point here is that
265. 8:36there's another problem, and that's that
266. 8:37AI agents are trained to be helpful.
267. 8:40They are yes-men, or or women, you
268. 8:42decide. And, the thing is, during an
269. 8:43outage, you don't want a yes-man. You
270. 8:46want somebody to challenge your
271. 8:47assumptions. One engineer at the retreat
272. 8:49said that we need something called angry
273. 8:51agents. Ones that are specifically
274. 8:53prompted to poke holes in your theory.
275. 8:56Because, otherwise, the human and the
276. 8:57agent will just agree with each other
277. 8:59while the server burns. And, here's the
278. 9:00point. If you're running a tech company
279. 9:02and betting on AI to make your team
280. 9:04faster, you need the prerequisites
281. 9:06first. You know, documentation that
282. 9:08captures how things work, seniors who
283. 9:10can architect, not just code, and a
284. 9:12system for architecting institutional
285. 9:14knowledge before AI makes people forget
286. 9:16how things work.
287. 9:19So, here's what I've learned from
288. 9:20running a dev team in the age of AI
289. 9:22agents. The work isn't disappearing,
290. 9:24it's moving from execution to
291. 9:26supervision. The bottleneck used to be
292. 9:28typing code into a file. That bottleneck
293. 9:30is now gone away. Now, it's
294. 9:32decision-making, verification, and
295. 9:34starting off by specifying clear intent.
296. 9:37Think about graphics programming in
297. 9:381992. Engineers hard-coded the maths to
298. 9:42draw a single polygon, calculating the
299. 9:44exact pixel positions. By 1994, the GPU
300. 9:48arrived and the hardware did the
301. 9:49polygons automatically. If you insisted
302. 9:52in hand-coding polygons in 1995, you
303. 9:55weren't a specialist, you were obsolete.
304. 9:57And the graphics engineers from those
305. 9:59days transitioned to lighting engineers,
306. 10:01animators, or physics programmers. They
307. 10:03stopped telling the computer how to draw
308. 10:05a triangle and moved on to telling it
309. 10:07how light reflects off a street, for
310. 10:09example. Nobody hand-codes polygons
311. 10:12anymore. We all work in game engines. I
312. 10:14think software engineering is hitting
313. 10:15that exact point right now. So, if
314. 10:17you're hiring developers or you've got a
315. 10:18team of developers, here's what to look
316. 10:20out for. Don't look for people that can
317. 10:21write code, look for architectural
318. 10:23thinking. Can they write an a spec that
319. 10:26is not open to interpretation? Can they
320. 10:28write tests and design a test suite that
321. 10:30actually becomes the product? And can
322. 10:32they debug a system that they didn't
323. 10:34write? And we use AI. That's what we
324. 10:35hire for now. But, here's what keeps me
325. 10:37up at night. In the past, code reviewing
326. 10:39wasn't just about catching bugs. It was
327. 10:41also about how developers learned the
328. 10:43system. So, if agents write all the code
329. 10:46and your team stops reading it, then
330. 10:48they become strangers in your own
331. 10:50system, strangers in your own code base.
332. 10:52When something breaks at 3:00 a.m.,
333. 10:54they're staring at code that was written
334. 10:56by a machine, trying to reverse engineer
335. 10:58the logic while your customers are
336. 10:59screaming. I think the solution is to
337. 11:01force AI to lay out all the
338. 11:03architectural decisions that it makes
339. 11:05when it writes the code. And then
340. 11:06arrange for meetings with your senior
341. 11:09engineers to review these decisions that
342. 11:12the AI is making. Essentially, so
343. 11:13there's like a symbiosis between the
344. 11:15architectural decisions that the AI is
345. 11:17making and your team. So, your team is
346. 11:19fully aware of those decisions. And all
347. 11:21this has to happen before the agents
348. 11:23write the code. Because you've got to
349. 11:24schedule time to understand your own
350. 11:26software now. It won't happen
351. 11:27automatically. The speed of AI demands
352. 11:30this. And if you're running a tech
353. 11:31company or you're employing developers,
354. 11:33that's the shift that you need to see
355. 11:34coming.
356. 11:37So look, if you're running a software
357. 11:39team or you're thinking about building
358. 11:40one, the ground is shifting. Senior
359. 11:42engineers are drowning in code reviews.
360. 11:44Junior engineers are smashing out code
361. 11:46at a 10x speed at a rate that the
362. 11:48seniors can't keep up with. And the
363. 11:50mid-level guys are trying to get their
364. 11:52head around starting to write code with
365. 11:53AI. I think the companies that will win
366. 11:55are the ones that will manage to retrain
367. 11:57before it's too late. I'm documenting
368. 11:59everything I'm learning building We Use
369. 12:00It, the systems we're putting in place,
370. 12:02the mistakes we're making, the changes
371. 12:04we're making, what works for us, what
372. 12:05doesn't work. If this is for you, hit
373. 12:07subscribe and head over to
374. 12:09axelmolice.com
375. 12:11and join my newsletter where you'll get
376. 12:13weekly content like this. And if you're
377. 12:15building software right now in this
378. 12:17exact moment of transition, trying to
379. 12:19figure out what to do with your dev
380. 12:20team, you're not alone. The best tech
381. 12:23companies aren't panicking. They're
382. 12:25adapting.
383. 12:26Be one of them.
384. 12:28See you in the next one.