I was building loops wrong... — Transcript

Full transcript

1. 0:00It was 12:00 a.m. yesterday and a bunch
2. 0:02of PRs still come into our codebase and
3. 0:04it was not because of us working
4. 0:05particularly hard, but we have agent
5. 0:07wake up every 30 minutes, scan our
6. 0:09codebase for any improvements and
7. 0:11errors, check our server logs, and ship
8. 0:13a PR. And every PR in Verify agent will
9. 0:15test fully and attach your evidence for
10. 0:16us to review. For some low-risk fix, it
11. 0:18can even merge directly. And also, a CRM
12. 0:20lifecycle agent will scan through our
13. 0:22user base and find useful segments to
14. 0:24outreach and nurture. And also, poor
15. 0:26ticket is automatically handled, no
16. 0:28matter which language users are
17. 0:29speaking. And none of this is a demo.
18. 0:31All this loop has been running our
19. 0:32company Super Divine for the past
20. 0:34months. And today I'm going to show you
21. 0:35exactly what we learned from building
22. 0:37this loops, what worked and what didn't
23. 0:38work. So, hopefully by the end of this
24. 0:39video, you understand loops than most
25. 0:41people in your timeline. So, last week I
26. 0:43wrote about Loop Engineer, the shift
27. 0:44from you prompting the agent to do a
28. 0:46task towards designing system where the
29. 0:48agent can decide what to work on,
30. 0:50execute, verifies, and improve over
31. 0:52time. And a lot of you said, "Okay, I'm
32. 0:53sold, but how do I actually
33. 0:55[clears throat] build one that actually
34. 0:56works?" Because honestly, everyone can
35. 0:58just put together a loop using Copilot
36. 1:00or Codex and code it down. But that's
37. 1:02easy 5%. The real work is how do you
38. 1:04design guardrails that let you walk away
39. 1:06from it? The loop that can consistently
40. 1:08delivering the work safely and improve
41. 1:10themselves over time. So, every loop we
42. 1:11build internally has this exact
43. 1:13structure. We normally have one markdown
44. 1:15file that is holding both what we call
45. 1:16loop contract as well as state and lock
46. 1:19of this loop. And this serving as a
47. 1:20living documentations about the loop
48. 1:22itself. Then there will be trigger layer
49. 1:24that actually wake up agent to do the
50. 1:25work. And there are different type of
51. 1:27trigger, which I will talk through a bit
52. 1:28more. Then the actual agent that's going
53. 1:30to find a work, does the work, improves
54. 1:32the work has been done properly, and
55. 1:34keep updating the docs. So, the next
56. 1:35round starts smarter. So, first part of
57. 1:37the loop contract, this is crux of the
58. 1:39MD file. It's like the constitution of
59. 1:41loop. There's three things really matter
60. 1:42and lives inside. One is the goal. So,
61. 1:44what does the winning look like? Whether
62. 1:46there's even a finish line. And also,
63. 1:48the boundaries. You want to make it very
64. 1:49clear what agent can just do by itself
65. 1:52versus the ones that actually need to
66. 1:53escalate human. And third is SOP. If you
67. 1:55have a specific workflow or principles
68. 1:57that you want agent follow every time.
69. 1:59And after that, there's a state and
70. 2:00logs. So, this is what it keep agent
71. 2:02remember what are the things that they
72. 2:03have tried before, what are the
73. 2:05learnings. And I'm splitting into two
74. 2:06parts. One is the state. So, state is
75. 2:08like a durable picture. What's the
76. 2:10current hypothesis agent has? What are
77. 2:12the old backlogs? What are the things
78. 2:13that agent has shipped but needs follow
79. 2:15up? And this part generally you want to
80. 2:17keep it very small and on purpose.
81. 2:19Versus log is the append-only record of
82. 2:22whatever happened run by run. And
83. 2:24without this block, every morning in
84. 2:25loop just rediscover the same noise
85. 2:27arrow, waste token on chasing the scene
86. 2:28that already tried. And for most of
87. 2:30small loops, you can basically just put
88. 2:32most loop contract and state and block
89. 2:34into just one file. And let's take
90. 2:35through one example. This is one loop we
91. 2:37are running in our code base called
92. 2:38react doctor checks. What it does is
93. 2:40that we're utilizing this open source
94. 2:42CRI tool called react doctor, which is
95. 2:44CRI you can run in your phone and code
96. 2:46base and it will automatically identify
97. 2:48some of the critical issues, listing
98. 2:50them and give you score. So, this loop
99. 2:51is basically uh running this react
100. 2:53doctor scan on our code base every day
101. 2:56and pick up the one of the most critical
102. 2:57issue to fix automatically. So, in this
103. 2:59loop's contract doc, we have a scope
104. 3:00folder that is talking about the purpose
105. 3:02of this loop and also a boundary
106. 3:04section. And we're talking about how it
107. 3:06should spawn a sub-agent to do the fix
108. 3:08in isolated work trees format and how it
109. 3:10needs to run through the verification
110. 3:12process as well, as well as what kind of
111. 3:14change it can just merge by itself
112. 3:16versus one that it needs human reviews.
113. 3:18Below that, it lists out a SOP about the
114. 3:21common workflow it should follow, as
115. 3:22well as a state section that is
116. 3:24mentioning its current understanding
117. 3:25about this loop, what are the tickets
118. 3:27and PR already open and shipped, what
119. 3:29are the things that needs follow up, and
120. 3:30then append-only logs after each round
121. 3:33to log what are the things has been
122. 3:34happening. So, every day we agent will
123. 3:36be wake up with this documentation as a
124. 3:38prompt to trigger the agent start the
125. 3:39work. And we actually build an internal
126. 3:41tool so that we have a dashboard to
127. 3:42track what are the PR has been opened
128. 3:44and merged, and how is the react doctor
129. 3:47health score improve over time. And this
130. 3:48loop contract documentation is not just
131. 3:50applicable to engineering tasks. As I
132. 3:52mentioned, we also have this CRM uh uh
133. 3:54life cycle loop, where every day I would
134. 3:56have the our agent to monitor all the
135. 3:58daily active users, group them into
136. 4:00different segmentations, like ones are
137. 4:02small influencers that we should reach
138. 4:03out for affiliates and distribution,
139. 4:05users who are clearly frustrated based
140. 4:07on the large language model logs we
141. 4:08have, as well as users who are pretty
142. 4:10engaged but haven't upgraded yet. And
143. 4:11for each segment, depending on the
144. 4:13priorities and risk level, agent can
145. 4:15just either auto outreach or draft a
146. 4:17message to waiting for me for approval.
147. 4:19And this CRM daily run also has very
148. 4:20similar structure. So, we have this
149. 4:22documentation that is talking about the
150. 4:23goal and SOP for this loop. Then we have
151. 4:26a list of boundaries. What are personal
152. 4:28pitfalls agent should be aware of when
153. 4:30they were doing the research, and what
154. 4:31kind of message agent can just send by
155. 4:33themselves versus where need approval.
156. 4:35Then it has state section that listing
157. 4:37out its current understanding and
158. 4:38hypothesis, as well as the timeline.
159. 4:41Since So, since we have been running
160. 4:42this loop for like almost a month, this
161. 4:44documentation just keep getting updated
162. 4:46after every single run. And the
163. 4:47structure we have is basically have loop
164. 4:49folder and under each loop, we can
165. 4:51contain one readme file. And sometimes
166. 4:53there might be other artifacts, which
167. 4:55can also be saved in the folder and
168. 4:56referenced in that readme file. So,
169. 4:58these are first two parts, the loop
170. 4:59contract as well as the state log. The
171. 5:01next part is a trigger. So, every loop
172. 5:03will have some sort of a trigger, but
173. 5:05depending on type of loop, you might
174. 5:06define trigger differently. And part of
175. 5:08reason why current loop were feel a bit
176. 5:10confusing is because of different
177. 5:11trigger types. So, we all know Codex and
178. 5:13Cloud Code has this go command, where
179. 5:15you can define condition and give it go.
180. 5:17Then you'll keep running for loops until
181. 5:19task has completed. And this is the
182. 5:20first type of trigger, is like a
183. 5:22continuous for loop. You can imagine
184. 5:24behind things is just a while loop.
185. 5:26While the goal is not satisfied, or you
186. 5:28sometimes can define max back and forth
187. 5:30turns, or a token budget. And before
188. 5:32those things are hit, it will just
189. 5:33continuously doing the same, look at
190. 5:35this contract, do next step, until the
191. 5:37goal is satisfied. And this as I
192. 5:39mentioned before is basically the goal
193. 5:40command. It is particularly good for
194. 5:42scenario where you can get immediate
195. 5:43feedback loops, like bug fixing or
196. 5:46implementing a piece of complex software
197. 5:48that the specs is well defined. And
198. 5:49meanwhile, second type of trigger is
199. 5:50also type of cron job. This is like
200. 5:52Codex automation or cloud codes loop or
201. 5:54schedule comment. Fundamentally, it will
202. 5:56just trigger and wake up the agent in
203. 5:58certain interval. And the only
204. 5:59difference between loop and schedule is
205. 6:01that one is running in the cloud versus
206. 6:03one is running in the same session. And
207. 6:04these two type of trigger already handle
208. 6:06a good amount of scenarios. But often I
209. 6:08found also useful to have event-based
210. 6:10loop as well, which means it's kind of
211. 6:11reactive to certain events, like
212. 6:13whenever there's a new email, you might
213. 6:15want to wake up agent and let it handle
214. 6:17the email. Or whenever your server get
215. 6:19incident, then you want to wake up the
216. 6:20agent and get it to fix the incident
217. 6:22immediately. And those type of
218. 6:23event-based trigger is particularly good
219. 6:26for things that needs immediate
220. 6:27handling. However, both cloud code and
221. 6:29Codex don't natively support those type
222. 6:31of triggers. So, what you want to do is
223. 6:33actually set up a local daemon process
224. 6:35that expose a URL for any webhook to
225. 6:37send notifications. So, for example, you
226. 6:39can create a webhook on Render and uh
227. 6:41point to your local daemon URL to
228. 6:43receive certain failed events. And
229. 6:45fourth and probably the most useful type
230. 6:46of trigger I found is what I call uh
231. 6:49combo or workflow. So, we can have a
232. 6:51ticker that's still running interval,
233. 6:53but instead of trigger the agent
234. 6:54immediately, ideally we just run a
235. 6:56script to fetch from data source to
236. 6:58check programmatically the where there
237. 7:00are new works to handle. For example,
238. 7:01for our support inbox triage loop, we
239. 7:04actually define a JavaScript here. So,
240. 7:06you can fetch from Intercom what are the
241. 7:07recent updates from the past 30 minutes.
242. 7:09If there are any real updates, then
243. 7:12trigger the agent to handle. If not,
244. 7:13just skip this this the run. And this
245. 7:15one we found is particularly effective
246. 7:17and useful because we can run the loop
247. 7:19in much more cost and token efficient
248. 7:21way by batching a good amount of work
249. 7:23together for the agent to handle and
250. 7:25only wake up agent when there's real
251. 7:26work. And both first two triggers
252. 7:28support out of box from cloud code and
253. 7:29Codex, but the other two type of
254. 7:31triggers you will need to set up your
255. 7:33own local script and daemon service. And
256. 7:35for us, we have this internal tool that
257. 7:36we built to run these type of triggers,
258. 7:39which we just open source and you can
259. 7:40run locally. It talks directly to your
260. 7:42local agent, but your team will have
261. 7:43this kind of centralized place to manage
262. 7:45all the loops, contracts, state, login,
263. 7:47triggers. And I'll talk about this
264. 7:48project a bit later. But this third part
265. 7:50of the trigger, I think choosing the
266. 7:52right trigger for the specific type of
267. 7:53loop that you're running will really
268. 7:55drive down the cost a lot. Third is the
269. 7:56actual agent that's is going to be wake
270. 7:58up and execute the tasks. And normally
271. 8:00agent will go through the three stages
272. 8:02of First, it gathers the signals and
273. 8:04find and prioritize the work. Then start
274. 8:06executing the task. And if it's a
275. 8:08complex, high-stakes task like engineer
276. 8:10tickets, you typically will have a
277. 8:11verifier at the quality before it claim
278. 8:13finish. For simpler tasks, you might
279. 8:15have just one agent doing both three of
280. 8:17these jobs. But for more complex type of
281. 8:19tasks, you typically will want to break
282. 8:21down into three different roles. And
283. 8:23most common structure is this
284. 8:24orchestrator agent which will receive
285. 8:26the prompt doing the research and
286. 8:28planning of the task. Then sponsor
287. 8:30agents, each one working isolated work
288. 8:32tree, so tasks can be run in parallel.
289. 8:34And each executor should hand off task
290. 8:36to a verifier who can test the result
291. 8:39and also attach evidence to the PR, so
292. 8:41it can become much easier for human to
293. 8:43review the tasks. And all the updates
294. 8:45will be sent back to the loop contract
295. 8:47doc. So this kind of the core piece of
296. 8:49every loop that we run. And I have
297. 8:51talked about verifier quite a bit
298. 8:52before, especially for engineer task.
299. 8:54This is like the prerequisite to any
300. 8:56loop that delivering high-stake work
301. 8:58like real production code change or
302. 9:00messaging real customers. Basically, you
303. 9:02want to making sure the process is quite
304. 9:04easy and produce evidence that human can
305. 9:06easily review. Generally, that means
306. 9:07that you want to give agent an
307. 9:09environment that they can verify their
308. 9:12work token efficiently. If you've been
309. 9:13following my content, I've made a few
310. 9:15articles around that already regarding
311. 9:17how do you use things like Playwright
312. 9:19CRI to let agent test and record video
313. 9:21or image evidence about the work and
314. 9:23utilizing things like Crabbox to set up
315. 9:26remote sandbox environment to do the
316. 9:28test, so you won't be restricted about
317. 9:30how many dev server you can run in
318. 9:32parallel on your local machine. And I
319. 9:34have packaged this links into a skew
320. 9:36called verifier setup, so you can just
321. 9:38give to cloud code or codex in your
322. 9:40actual code base. Then set up the proper
323. 9:42verification systems. And I put the link
324. 9:44in the description below for this GitHub
325. 9:47repo with all the skills that we talk
326. 9:48about here. So you can use it for free.
327. 9:50So this is overall system and anatomy of
328. 9:52what does a good loop look like in our
329. 9:53case. And meanwhile, another really
330. 9:55useful layer we found is this evolve
331. 9:57loop. So the concept here is that most
332. 9:58of loop when you just getting started
333. 10:00might not be perfect. There could be a
334. 10:02lot of room for improvements like how do
335. 10:03you design the trigger? So it's more
336. 10:05cost effective. How can you turn some of
337. 10:07repetitive SOP into script that agent
338. 10:09can run? And a lot of those improvement
339. 10:11can actually be optimized by the large
340. 10:12language model itself. If you give agent
341. 10:14all the existing configuration of the
342. 10:16agent, the past run state and logs, as
343. 10:19well as the raw conversation history so
344. 10:21that it can inspect further. And all we
345. 10:22need to do is just after every five or
346. 10:2510 loop runs, we trigger a dedicated
347. 10:27evolve session where the agent will be
348. 10:29given the existing configuration, past
349. 10:31log histories, then prioritize some
350. 10:33change. And change can be the the loop
351. 10:36contract itself or some outdated state,
352. 10:38as well as the trigger script that you
353. 10:39can create for repetitive actions. And
354. 10:41support loop here is actually a perfect
355. 10:43example. It was actually during the
356. 10:45evolve run, it starts setting up this
357. 10:47programmatic trigger to only wake up
358. 10:49agent when it's necessary. So this is
359. 10:50overall system and anatomy of what does
360. 10:53a good loop look like for us. And we can
361. 10:55go through a few examples together to
362. 10:57set this up. And one really good one to
363. 10:59start with is this documentation
364. 11:00maintainer loop. So I know most of you
365. 11:02will have either documentations or some
366. 11:05sort of cloud or MB and code base
367. 11:06documentation file to help agent gain
368. 11:08the full context. But that context can
369. 11:10often get outdated. We can solve this by
370. 11:12setting up a simple loop where every day
371. 11:15it can wake up the agent to check what
372. 11:17has been shipped past 24 hours and what
373. 11:19are diffs. And compare those change with
374. 11:21the read me, the setup guide, the
375. 11:23examples and run books. And for each
376. 11:25abnormal part, it can actually verify
377. 11:27which one is actually true. And if
378. 11:28there's nothing stored, then just finish
379. 11:30the session. But if there is, then you
380. 11:32can just make some small quick fix to
381. 11:34making sure the documentation is up to
382. 11:35date and open a PR for that. This loop
383. 11:38is very small, but quite a useful to
384. 11:40use. And the way we will set up is loop
385. 11:42contract look like this. They clearly
386. 11:44list out the goal and the boundary about
387. 11:46what it can ship by itself and what are
388. 11:48the things it is allowed to do. And you
389. 11:50will notice this one rules here called
390. 11:52never rewrite accurate doc to look busy
391. 11:54cuz one default behavior we saw agent
392. 11:57has is that it will tend to do something
393. 11:59even though it's not necessary. So this
394. 12:01kind of the rules is actually quite
395. 12:02useful to making sure it produce useful
396. 12:05results. And then in SOP and state and
397. 12:07logs. So you can simply give this
398. 12:09markdown file to your cloud code or code
399. 12:10acts and ask it to set up loop based on
400. 12:12this. But as I mentioned before, we
401. 12:14actually build this internal tool that
402. 12:16we have been using to coordinate and set
403. 12:18up loops for us. It allow you to define
404. 12:19those kind of programmatic triggers that
405. 12:21we shared before. Also set up a contract
406. 12:23based on the best practice that we
407. 12:25learned. And also past raw wrong logs
408. 12:27will also be stored so the agent can
409. 12:29remember right itself. And it has this
410. 12:31evolve loop behavior building
411. 12:32automatically. So you can see after
412. 12:33every few sessions there will be a blue
413. 12:35dot popping up. And this blue dot is a
414. 12:37dedicated evolve run. They will look at
415. 12:39past run and then figure out
416. 12:40improvements that can be done. Like in
417. 12:42this specific evolve, it sharpen the
418. 12:44specs and SOP, clean up some outdated
419. 12:46state, and also update dashboard for us
420. 12:48to easier track the performance, which
421. 12:49we found is actually pretty useful.
422. 12:51Because quite often you don't really
423. 12:52know the output from the loop. But with
424. 12:54this one we can easily track what are
425. 12:55the open things that needs our attention
426. 12:58and how the performance change over
427. 13:00time. And if needed, can just click a
428. 13:02button which will copy all the contacts
429. 13:04over about this loop to my local
430. 13:05computer so I can chat with agent to
431. 13:07evolve the loop. And we have included a
432. 13:09few templates here already. And doc
433. 13:11maintainer is one of them. So I can just
434. 13:13copy this prompt and go to my repo and
435. 13:15paste it in. It will automatically
436. 13:17include a list of instruction about how
437. 13:20this loop should be built alongside some
438. 13:22best practice that we encode here. Then
439. 13:24it will automatically create this loop
440. 13:25knee folder in your code base as well as
441. 13:28this doc drift sweep loop. So this is a
442. 13:30loop where it will talk about scope of
443. 13:32things it should review which is readme,
444. 13:34cloud MD, everything in the docs folder,
445. 13:36the skill file, the scripts, and then it
446. 13:38will talk through the SOP here and the
447. 13:40state of current understanding and
448. 13:41timeline items. And then we'll have this
449. 13:43loop created on Loopery which will
450. 13:44automatically trigger the agent on time
451. 13:46and keep log of things ever shipped, all
452. 13:48the artifacts and all the runs. And
453. 13:49there are few other templates that we've
454. 13:51been using and you can just copy paste
455. 13:53like this react doctor loop. Other loops
456. 13:55that periodically clean up the tech
457. 13:57debt. And this is totally open source
458. 13:58and free to use. So feel free to just go
459. 14:00to GitHub and start using it. I've put a
460. 14:02GitHub link in the description below so
461. 14:04you can try. Thank you and I'll see you
462. 14:05next time.