How One Founder Manages 20 AI Coding Agents at Once - Ryan Carson, Untangle — Transcript
Full transcript
- 0:01[music]
- 0:12So, my name is Ryan Carson. I'm the
- 0:14founder and CEO of Untangle. So, we're
- 0:18building an AI parallegal specifically
- 0:20for family law, uh, focusing on divorce
- 0:23in particular. Um, so, quick audience
- 0:25poll. Who has been divorced? Raise your
- 0:27hand. Raise it. Be don't be shy. Raise
- 0:29your hand. Who's been divorced? Okay,
- 0:30keep your hands up. Who knows somebody
- 0:32that's been divorced? Raise your hand.
- 0:34Everybody. Yeah. So, for you watching
- 0:36the video, everyone raised their hand by
- 0:37the end. Divorce touches a lot of
- 0:39people. Um, two of my sisters got
- 0:41divorced. My parents got divorced. Um,
- 0:43and it's one of the most painful things
- 0:45in your whole life. And I wanted to work
- 0:47on that problem and just make it a
- 0:49little less painful. Um, started off as
- 0:51a consumer app. Uh, quickly pivoted to
- 0:53B2B and we sell to family law firms now.
- 0:56And thankfully product market fit is
- 0:58there and we're growing really fast. So
- 1:00that's really fun. Um so I've built and
- 1:04sold four companies. Um my last startup
- 1:06we scaled to about 110 people and I
- 1:09learned a lot about managing people and
- 1:11it turns out managing people is like
- 1:13managing agents. And so I'm going to
- 1:14share with you how we manage agents at
- 1:17Untangle and how everything I've learned
- 1:19uh managing people kind of fits there.
- 1:22So I'm hoping to teach you some of those
- 1:23things. Um, I'm also going to show you
- 1:26how we hire. I I think hiring is very
- 1:28different in the age of agents. Um, I've
- 1:30hired a lot of people over my career.
- 1:32Um, I've made a lot of mistakes and I'm
- 1:34going to share how we're hiring now at
- 1:35Untangle and I think how it's really
- 1:37working. Um, and then I'm going to show
- 1:38you a live demo.
- 1:41So, let's get started. This is me on X
- 1:45and this is Untangle. If you want to
- 1:46know more about us,
- 1:49if my little clicker works, you can do
- 1:51it.
- 1:55Okay, so what does the org chart look
- 1:58like now in the age of agents? How do
- 2:01you run your agents? Um, so the way we
- 2:04do it is pretty straightforward. So we
- 2:05have humans at the top. Um, and
- 2:07obviously we set priorities, we talk to
- 2:09our customers, we're doing all the human
- 2:10stuff, but what we use are manager
- 2:13agents and these are literally just
- 2:14sessions. So we use Devon. Love Devon.
- 2:17We do all our engine the cloud. Um, I've
- 2:20done [clears throat] everything, you
- 2:21know, from coding by hand to, you know,
- 2:24using cursor to here and we're in Devon.
- 2:26But we have a manager session. I'm going
- 2:28to show you this. It's literally just a
- 2:29session. I I call it engineer manager.
- 2:32Um, and it never writes code. And I
- 2:34mostly talk to this agent. Um, and it
- 2:37manages a bunch of children. So, I'll
- 2:38talk to you how about how we do that. We
- 2:40also I mostly use audio to manage this
- 2:43agent. Um, I don't know about you, but
- 2:46I'm not a fast reader. I'm definitely
- 2:48not an agent and I prefer audio. I can
- 2:50ingest audio a lot better for some
- 2:53reason than reading. So, I talk mostly
- 2:55to my manager agent over audio. I'll
- 2:58show you that. Um, I just find it I I
- 3:01can absorb it. I can understand it. I
- 3:03can see what's happening. It's much more
- 3:04natural. So, I'm going to show you that
- 3:06as well. Now, the manager agent uh
- 3:09primarily works from a ticket board. We
- 3:11use linear. Doesn't really matter what
- 3:13you use, but this is kind of the source
- 3:14of truth about what needs to be built.
- 3:16every ticket should be one PR um which
- 3:19is how we we work and that the uh
- 3:21manager agent knows which tickets to
- 3:23pull based off dependencies. It's like
- 3:25pretty basic engineering manager stuff
- 3:27here, but this is where it gets
- 3:29interesting. So each manager agent uh
- 3:32concurrently manages up to about 20
- 3:35builder agents. Um this number varies
- 3:38like it can go anywhere from five to
- 3:40probably 25. Um, this is somewhat
- 3:43bottlenecked by your ability to talk to
- 3:45the manager agent and understand what's
- 3:47happening. Um, uh, I think if you push
- 3:51too hard on this, you you quickly spin
- 3:53out of control. Um, I find we can
- 3:56concurrently manage about 25 builders.
- 3:58Um, now the key here is that the manager
- 4:01is talking to this these builders. You
- 4:02never go into these sessions. You never
- 4:04talk to them. You never manage them. You
- 4:06never do anything. Um, it's all done
- 4:08through the manager agent. And usually
- 4:10what it's doing is
- 4:12answering technical questions that don't
- 4:14I don't need to answer. Um or unblocking
- 4:17small things. Um I also have an
- 4:19automation that forces the manager agent
- 4:22to read all the children sessions every
- 4:24it's like every 15 minutes. This says
- 4:2610, but I've decided 15 is about the
- 4:28right call. You have to you have to
- 4:30build an automation. You can't trust
- 4:32your Androme your manager agent manager
- 4:35agent to do this right. It's not
- 4:38deterministic. So this is basically a
- 4:40script again in Devon. It has
- 4:42automations. You just tell it, hey,
- 4:44every 15 minutes go read every single
- 4:46child, push things along, keep things
- 4:49moving, right? Um, and then any
- 4:51questions go back to the manager agent.
- 4:53So, who's tracking so far? Do you
- 4:55understand what I'm saying? Okay, good.
- 4:57And probably a lot of you are doing
- 4:59this. I'm not claiming this is rocket
- 5:00science, but this is what we're doing
- 5:02right now in prod and it's working,
- 5:04right? Um, each of these builder agents
- 5:07uh has a a a review agent that it calls
- 5:10upon. Um, Devon has built-in reviews.
- 5:12Probably every code harness you have has
- 5:14built-in reviewers. Um, we typically
- 5:17stick to a three uh review max where uh
- 5:21we allow the builders to loop about
- 5:23three times with reviews. If they still
- 5:26have any important bugs that have been
- 5:27fixed, after three, then they bubble
- 5:29that back up to the manager agent. And
- 5:31if the manager agent needs to, it
- 5:32bubbles it back up to me, right? Um,
- 5:35that seems to work pretty well.
- 5:38Okay, so what's happening under the code
- 5:40under under the hood here? Like all the
- 5:41normal stuff, right? You've got code
- 5:42plus tests. You've got a QA agent. This
- 5:45is actually an additional uh process
- 5:48that we run. Um, it actually spins up a
- 5:50child of the builder to run QA. Um,
- 5:54y'all, this is cheap. like you should be
- 5:57spinning up as many agents as you can as
- 5:59long as you're not holding up your merge
- 6:01queue, right? So I this works well for
- 6:04us now. Um a a wonderful engineer named
- 6:07Dale recently joined. He helped install
- 6:09this QA process. I think it's working
- 6:11pretty well so far. The thing that
- 6:12you're all going to encounter is just
- 6:14merge Q slowdown. Um there's this fine
- 6:16balance of just, you know, pecking your
- 6:19PRs to death um versus just getting them
- 6:22merged, right? Um, the other thing I
- 6:25love, love, love, love is videos. So,
- 6:28Devon has this built in. Um, if not, you
- 6:31should build this. Um, it's hard. Um,
- 6:34every builder records a video of the
- 6:36feature being used and passing tests.
- 6:39So, if it's a UI facing feature, you've
- 6:42got to look at it, right? Um, this is
- 6:45how we know that what we've built looks
- 6:47good, how it works, right? So, in Devon,
- 6:50it's a built-in feature. It's called
- 6:51test. And basically it it builds test
- 6:53cases and then it records a video and it
- 6:55gives you green or or red on those on
- 6:57those tests. And if you get a video
- 6:59that's all green, you can watch it and
- 7:01know what we think is happening is
- 7:03happening and visually it looks good.
- 7:05Right? So this is the human in the loop
- 7:07where a human has to say ship it. So the
- 7:11manager session will will let me know
- 7:13there's a video ready for you. Watch it.
- 7:15Right? So I watched the video and this
- 7:17is the hard part. you've got to zoom in
- 7:19and understand like what is this PR
- 7:20doing? Um what am I watching? Um so I
- 7:24usually read a summary of like what is
- 7:26this thing? What is it supposed to do? I
- 7:28watch it and I ga engage. And I would
- 7:30encourage you this is the part where you
- 7:31want to cut corners. You got to watch
- 7:33the video. Like there is no this is the
- 7:36part of the bottleneck you have to
- 7:38engage in as a product owner. Um got to
- 7:40watch the video, got to pay attention.
- 7:42Usually it has to be on your laptop. Um
- 7:45I I I this literally before this talk I
- 7:47was on my phone you know talking to my
- 7:49manager my manager agent and it said
- 7:51there's a video for you. I watch it and
- 7:53I just couldn't see it well enough and I
- 7:55was so [snorts] tempted to be like it's
- 7:56okay ship [laughter] it like no get back
- 8:00to my machine watch the watch the video.
- 8:02Um this is working like you know we we
- 8:06ship a lot of PRs uh at high quality
- 8:09this way. We've iterated, you know,
- 8:11hundreds of times on this process and I
- 8:13I feel pretty good about this. Now, this
- 8:14will change tomorrow [laughter] and
- 8:17probably will be useless a week from
- 8:18now. Uh, but it works well today. Um,
- 8:21uh, the thing I will say is like if
- 8:23you're coding locally, stop. Like, you
- 8:25really need to be coding with a a cloud
- 8:27agent. I don't care if it's Devon or
- 8:29whatever it is, but if you're running
- 8:31local dev, you are screwing up. You're
- 8:34holding yourself back. You're holding
- 8:35your team back. You're holding
- 8:36everything back because you can't do
- 8:38this. Um so that's where we're going and
- 8:41I feel strongly about that.
- 8:43So how does this work at scale? Well,
- 8:45you know, say you have 10 folks um uh in
- 8:49the who are working on things. Typically
- 8:51they'll have an engineering manager uh
- 8:54session each. Sometimes they'll have two
- 8:56like right now I have two running. Um
- 8:59and what's beautiful is that these
- 9:01things talk to each other. And so uh
- 9:04again in Devon this is built in. It's
- 9:06probably built into whatever harness you
- 9:08use. the sessions all need to be able to
- 9:10talk to each other and they all need to
- 9:11know who each other is, right? And
- 9:14they're really good at this. Like this
- 9:16is such a delight. So what you'll see if
- 9:19if you if you zoom into any sessions,
- 9:21they're messaging each other. It's
- 9:23clear, it's straightforward, it's fast,
- 9:26it's good. Like you should be allowing
- 9:29your sessions to talk to each other. Um
- 9:31and again, if you're not, you are
- 9:33holding yourself back and you're
- 9:34probably shipping worse software. Um so
- 9:36they all talk a lot. Um, and it's not
- 9:38chitchat or stupid stuff like this is
- 9:41important stuff that they need to know
- 9:42to unblock each other. Um, which is very
- 9:45useful. However, you still need what we
- 9:48call a merge Q manager. Um,
- 9:51so this is how we coordinate this just
- 9:55wave of PRs that come in. You know, you
- 9:59have things like database migration
- 10:01numbers. These need to be coordinated.
- 10:03Like if all the agents are trying to
- 10:04grab these things, you constantly
- 10:06collide, right? So the merge manager
- 10:08just helps all that go away. It also is
- 10:11a little more intelligent about how the
- 10:13merge Q is happening. So this is really
- 10:15nice. Um it's working well for us. And
- 10:18again, these are just sessions. This is
- 10:20not like something special. These are
- 10:22just sessions in Devon and I and I name
- 10:25them so I understand what's happening.
- 10:27Um and it's pretty straightforward. You
- 10:30know, the talk from the gentleman from
- 10:31OpenAI, you know, said basically you got
- 10:33to unhobble. Just unhobble the thing,
- 10:35right? So set up the roles and then
- 10:39unhobble the model, right? Um, and I
- 10:41think just having a merge manager say,
- 10:43"Hey, make all this better." Uh, really
- 10:45works well. All right. Who's tracking so
- 10:47far? Good. Who thinks this is
- 10:52Raise your hand. Oh, one guy. Two. Wait.
- 10:54One. Come on. Be be more brave. We have
- 10:58to talk afterwards. Where Where are you?
- 11:00Um, okay. And then we use GitHub merge
- 11:02cues. It's fine. It's not rocket
- 11:04science. It works pretty well. So,
- 11:07all right. Next up, how we hire. Um, so
- 11:10who's hiring engineers or employees
- 11:12right now? Raise your hand. Yeah, decent
- 11:15number of you. Um, I think everything
- 11:17has changed. Every everything you used
- 11:19to do about hiring employees is over and
- 11:22none of it makes sense. Um, it's all
- 11:24outdated and I would encourage you to do
- 11:26something like this. So, we don't do
- 11:30number one any sort of pre-screens. We
- 11:32don't do any video chats. We don't do
- 11:34anything with people um it for most of
- 11:37this process because it will bias you.
- 11:40You'll like their hobbies. You'll like
- 11:42the way they talk to you. You'll think
- 11:44they're interesting. And it's all noise.
- 11:46None of it is signal, right? All that
- 11:48matters is can they operate an agent,
- 11:51right? Can they do that? So, the way we
- 11:54do that is we ask candidates to submit a
- 11:5745ish minute video, whole screen
- 12:00recording. Um, and we ask them to ship a
- 12:03feature PR typically on a app that they
- 12:06already maintain and we just we just ask
- 12:09them to record that. No instruction, no
- 12:12guidance, no nothing. And we just want
- 12:14to see like how good of an agent
- 12:16operator is this person, right? So, um,
- 12:20that is free. We don't pay for that.
- 12:21That's basically how you get in the
- 12:23door. Um, and what we then do is we feed
- 12:26these videos into a ranking agent and
- 12:28basically say watch the video, watch the
- 12:30transcript, stack rank these people. Um,
- 12:33now this what's interesting is I did
- 12:35this and I didn't trust it the first
- 12:37time. I didn't like what it said. Um,
- 12:41and what's ironic is it actually was
- 12:43correct. Like the person that was number
- 12:45one, I I should have hired them. Um, uh,
- 12:48but I didn't trust it because I was I
- 12:50was biased. So you stack rank them. Now,
- 12:53this is just a lot of video to watch.
- 12:54You could watch all these videos if you
- 12:56want. If you have time, do it. But I
- 12:58think you'll find actually a very
- 13:00intelligent model is pretty good at
- 13:01understanding like how how good is this
- 13:03person. Um, what's interesting though is
- 13:06uh candidates will mumble um or uh
- 13:10sometimes or they won't think out loud,
- 13:11which is kind of fascinating. So, you
- 13:13can't really tell what they're thinking
- 13:14or doing. Um, which is interesting. So,
- 13:16that's uh something to sort of note. Um,
- 13:19and some talk a lot. [laughter] So, you
- 13:23never know. Uh, then we basically pick
- 13:25the top five and we invite them to a
- 13:27paid project. So, we pay them a fixed
- 13:28fee for 16 hours of their time. We
- 13:31actually give them a seat on Devon and
- 13:33we ask them to uh to get a real PR ready
- 13:37for merge. Um, we have them sign an NDA.
- 13:40Um, and we record all the video. This is
- 13:4416 hours of video. Um, we basically use
- 13:46Loom and then uh we feed all the videos
- 13:49into uh our our agent basically Devon
- 13:52and we say hey who you know which one
- 13:54should we pick? I actually watched all
- 13:5716 hours of these videos for the last
- 13:59round. Uh it was brutal but it did give
- 14:01me a good read on the people. Um and
- 14:04then you basically pick one. You pick
- 14:06your finalists and then what's important
- 14:09is that you don't do calls before this.
- 14:12Um you will get biased. you'll get
- 14:14confused. You'll try to pick somebody
- 14:16you shouldn't. And I would just call the
- 14:19finalists and say, "Hey, you know, we
- 14:22think we'd like to hire you. Let's talk
- 14:23a little bit." Um, and that gives you
- 14:26the final sort of human read, but you're
- 14:27going to watch videos of these people
- 14:28for hours. Like, you're not really going
- 14:30to need that call. Um, but I think
- 14:32there's a little bit of of of human
- 14:33connection there that's important.
- 14:36I think this is how everyone's going to
- 14:37hire. Like, this works. Um, you know,
- 14:40I've hired hundreds and hundreds and
- 14:42hundreds of people in my career.
- 14:44This is just way easier and way better.
- 14:46Um, and I think it's where we're all
- 14:48moving to. So,
- 14:52I'm actually going to finish 3 minutes
- 14:53early. Um, if you want to know more
- 14:56about Untangle, just go to Untangle. Um,
- 14:58I'm on X like just all the time. I
- 15:00literally been on X for 20 years. Um,
- 15:03and so, uh, that's where I live and
- 15:05breathe. So, if you want to know more
- 15:06about uh leading companies, managing
- 15:09agents, um that's where I am. If you're
- 15:12interested in family law and you're
- 15:14passionate about making divorce less
- 15:16painful, um get in touch. Um and that is
- 15:19all I have to say. Thanks everybody.
- 15:21Appreciate it. [applause]
About this transcript
This page contains the full transcript of How One Founder Manages 20 AI Coding Agents at Once - Ryan Carson, Untangle by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 2,902 words across 397 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.