YT YouTube2Text

YouTube2Text → Transcripts → How to Build a Software Factory for AI Coding Agents

How to Build a Software Factory for AI Coding Agents — Transcript

by Boundary · 15,403 words · 2,214 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00You start with the normal software
0:01factory where a human builds the thing
0:03and then you swap it out with agent
0:04builds the thing. There's a couple
0:06different options, right? You could like
0:07build everything inhouse and then
0:10there's like the other side of the
0:11spectrum which is like buy everything.
0:13>> In terms of buy versus build, where do
0:15you land?
0:15>> The real answer is like you should have
0:17as an engineer choice and the ability to
0:19like work in open systems and plug these
0:22things together. So this was a really
0:24fun episode where we talked about
0:25software factory design patterns. We've
0:27talked a lot about the broad software
0:29factory. And today we zoomed in on this
0:31part where it's like agents building the
0:33thing and testing the thing and getting
0:34feedback. We talked a lot about how the
0:37boundary in-house uh software factory
0:39works and the architecture of it and
0:41contrast it to like sort of full stack
0:43cloud agents where you just buy
0:45everything and everything is managed for
0:46you. Um, so the architecture here was
0:48really interesting and then we kind of
0:50dug into like the four layers of the
0:52enterprise software factory, what the
0:54stacks are and some actual like
0:56implementation examples of like how you
0:58can mix and match parts of this stack
1:00and how different like open- source and
1:03closed source products like let you
1:05bring your own compute, your own dev
1:07environment, build or buy the harness
1:09and build or buy the uh control plane
1:11and orchestration. So a lot more detail
1:13there. We talked a lot in detail about
1:14the boundary software factory. Super
1:16fun. And then uh as a little bonus at
1:18the start you can see our take on the
1:20Financial Times article of uh hey is
1:22OpenAI taking market share from
1:24Anthropic and why and why are we also
1:26codeex build. So super fun episode uh
1:28excited to get into it. Uh let's go. Hey
1:31folks this is AI that works. We talk
1:33about AI that actually works uh beyond
1:35the demo for real production use cases
1:37enterprise whatever it is you want
1:38something that actually works well and
1:40is reliable. This is uh all we do. We do
1:42a lot of systems design. We do a lot of
1:44context engineering. Um, the very first
1:47uh conversations that led to the term
1:49context engineering existing happened on
1:50this show. Uh, I'm going to stop
1:53yapping. I'm Dex. I'm the CEO and
1:55co-founder of Human Layer. We build a
1:56multiplayer coding agent workspace. And
1:58Vibb here is
2:00>> I'm Vibb. I'm one of the co-founders at
2:02Boundary and we build a programming
2:04language designed for agentic coding.
2:06>> Amazing. Check it out. boundaryml.com.
2:09Is that right? Did you guys not get
2:10baml.com? Uh sadly, Bank of America has
2:14a slightly bigger war chest than us.
2:16>> We'll get it. That's your That's your
2:17goal, right? It's like by the end of the
2:19uh
2:21>> We are slightly bigger than them on SEO
2:23for Google, though. So, we do beat them
2:24on in almost every region. Yeah. So, if
2:26you Google Bam, we're now rank one or
2:28rank two and rank three and they are on
2:30like page two, I think.
2:32>> Um
2:32>> amazing. We have go Google BAML uh and
2:35help their SEO even more. Click the
2:37Boundary ML website. Go check it out.
2:39Programming language for agents. I've
2:40been building a bunch of internal tools
2:41in it. It's a lot of fun. Uh you can try
2:44human layer at humanllayer.com. Um
2:47someone asked me today about this
2:48article. Did you read the uh Financial
2:51Times article about how uh Fable and
2:54Opus are actually like losing market
2:56share and Codex is taking over.
2:59>> So [clears throat]
3:01like if you think about it, this is kind
3:03of interesting. I mean, I'll tell you
3:06honestly, like my mom knows nothing
3:08about Claude or Anthropic as a company,
3:11she only knows Chad GBT.
3:14>> Yep.
3:14>> If she ever wants to go ahead and build
3:16an agentic stuff, she would only ever
3:18use Shad GBT cuz it's the closest she
3:20can get. So, I suspect it's just a
3:23relationship of building a consumer
3:24brand versus an enterprise brand. like
3:26no matter what you do at some point if
3:29more people get exposed to your topic
3:31the consumer brand will grow bigger
3:34>> and that's I suspect that might be it
3:36>> I completely disagree I think
3:38>> really
3:39>> I think I saw the change in like
3:43December and January of this year where
3:47all of the best agent decoders the ones
3:50that are like
3:51>> 6 to 12 months ahead of the like SF meta
3:54they all switch to codeex So, okay,
3:56that's a totally separate thing. Uh,
3:58like Codeex is definitely I mean I
4:00switched to Codeex, too.
4:01>> Like, and I think
4:02>> No, yeah, because it's better.
4:04>> Well, I was actually hesitant for a long
4:06time cuz I could only get Claude to
4:07work,
4:08>> but for some reason I I kept on seeing I
4:10was like I saw Theo switch to Codeex,
4:13Dylan switch to Codeex. A lot of these
4:14people on Twitter that I was watching
4:16only use codec switch. Peter, well,
4:19>> that's a Peter was early. I had a
4:22conversation with Peter Steinberger in
4:23October of last year.
4:25>> Peter have totally different incentives.
4:28>> This was way before openclaw or
4:30anything. He was like when he was I mean
4:32he was working on like 40 side 50 side
4:34projects. This is before openclaw launch
4:36but I was like yeah we built this whole
4:37research plan implement system for
4:38claude and like it makes Claude so much
4:40better and he was basically his take was
4:42just like this was codeex
4:44five maybe uh maybe
4:47>> I don't know if codex is better than a
4:48fable back then or back then. No, it was
4:51Codex back then is not better than Faber
4:52now, of course. But he's basically like
4:54all that research upfront [ __ ] that you
4:56have to tell Claude to do, you're just
4:57compensating for the fact that you're
4:59using a worse model and if you like you
5:01ask Codeex to do something, it will just
5:02read stuff for longer, which is exactly
5:04what you want.
5:06>> Yeah. Um, so have you guys switched
5:08fully to Codex?
5:10>> We I so I I will use Opus models for
5:13like generic knowledge work type stuff
5:16like running the company and like like
5:18managing pipeline and things like that.
5:20Um, it's just like a little bit better
5:22at like tool use and things like that.
5:24Like barely noticeable. It's just like
5:25what I'm used to. Um, but 90 or 95% of
5:30the features I write are written with uh
5:32Terra or Soul.
5:33>> You know what's so funny?
5:35>> Yeah.
5:35>> Um, you know what I want to see being
5:37done is you know how people are like
5:39talking about startups like there's all
5:41these people on Twitter's like uh vibes
5:42slopping about how much revenue they
5:44make and people are like just post the
5:45revenue or it doesn't count. What I kind
5:47of want to see with people talking about
5:48AI models is I kind of want to see
5:51people posting their token usage or it
5:53doesn't count. It's like you can't say
5:54you're a Codeex man unless you like
5:57actually like for example you you can't
5:59say you're a CEX man until you post your
6:01token usage and prove that you're a CEX
6:03man cuz so many people just follow the
6:05hype without doing it. So I can never
6:07tell like for you I know you actually do
6:09it like in your case, right? That's Oh
6:12yeah, show us.
6:13>> I'll show you the token usage. There you
6:16go.
6:16>> Let's see.
6:17>> This is cool.
6:19>> Um,
6:22let's find Yeah. cost cost by model. I
6:24mean, so this is by spend.
6:26>> Um, but yeah, we're using a [ __ ] ton of
6:29codecs.
6:29>> There you go. Um, yeah. It's also
6:32interesting. It's not like you have
6:33stopped using the Opus models. It sounds
6:35like you use it more targeted though.
6:37>> It's a more like things that are UI
6:39focused will still use Fable or Opus.
6:42Like, okay,
6:42>> if I have Fable credits, I will let I
6:44will let Soul drive and then if I find a
6:46shitty UI, I will I will I will kind of
6:48like, okay, throw this out and then
6:50we'll we'll have it either invoke a
6:51Fable sub agent or I'll just start a new
6:53session with Fable.
6:55>> Yeah. I mean, what I find is actually
6:57like so I've been doing a lot of UI work
6:58myself as well, and for that work, I
7:00also end up using like the Opus models
7:02because like I feel like for whatever
7:04reason, Codex sounds a lot more like AI
7:06than Claw does when it's writing content
7:09>> when I'm writing things with it. And
7:10like or at least maybe I just know how
7:12to tweak codeex better to sound more
7:13like me or claude to sound more like me
7:15than I know how to tweak codeex better.
7:18Codex is kind of like but for
7:19engineering task it's not even a
7:20question like codeex is
7:23execution codeex is better.
7:26>> Yeah. Whether it's codeex or claude like
7:28I don't use I don't let the models write
7:30much. I basically like here's a
7:31transcript of a thing. I'll feed it like
7:33a transcript of an episode of this show
7:36and I will um give it a
7:41um I'll be like write the headers and
7:44write a little snippet like a a markdown
7:47comment about what goes in this section
7:49and the reference to where in the
7:51transcript it's from. And I'll have it
7:53write one little literally like two or
7:54three sentences at a time. And I'll
7:56polish them. And in one session I will
7:59have it write the thing. I will change
8:01it. I will be like, "Read what I changed
8:03it to. Do the next section. Don't [ __ ]
8:05it." And then like after four or five
8:08iterations, you start it starts in one
8:10session, you can start to give it a
8:11little bit of a sense of a tone in the
8:13patterns to follow.
8:14>> And then sometimes I'll flush that out
8:16to a doc that is like here's all the
8:18anti-atterns that you do by default. But
8:20it's like my writing process is like I
8:21cannot stand like three pages of Claude
8:24or or codec slop. Like I just I won't
8:27read it. I will. So, like I have to do
8:29it really small increments at a time,
8:30otherwise it's overwhelming.
8:31>> Because otherwise, you just don't read.
8:33>> Otherwise, I'm just like, "Oh, I [ __ ]
8:35hate this. I might as well write this
8:36from scratch." And then I'm like, "Well,
8:37I don't have time to do that." Like, how
8:38do I get how do I get leverage from AI?
8:40How do I get it to like kind of like
8:42unblock my writer's block without me
8:44actually having to like without actually
8:46like having to fix every single issue
8:48that it that it introduces.
8:49>> I found that I can't really have my
8:51whole team switch. I still don't have a
8:53good way to convert any humans to like
8:56force them to go do the
8:57>> like to either model like to codeex or
9:00to claude. And the reason is like
9:02>> I think a large part of how these AI
9:04models work today and these AI workloads
9:06work is like they're kind of personal to
9:08every human
9:10and like for example like obviously like
9:11we land on two different spectrums of
9:13how we like to engineer. I I like to sit
9:15and think really hard about the problem
9:17for a while and like try and like remove
9:18as many roadblocks up front and you do a
9:20lot more iterative approach. Neither
9:22approach is necessarily bad or good, but
9:24like clearly like if I and like it's
9:26also not like both of us live in one
9:27world at all times. We live on the
9:29spectrum. We just have a tendency on one
9:31end or the other. But like if we for if
9:34you were forced to always plan upfront
9:37and do every single amount of planning
9:39up front and I was forced to always be
9:41iterative,
9:42we would kind of not enjoy the work as
9:44much. And I think that's the problem
9:45with cloud and codeex is you as an
9:47engineer got used to a workflow. It's
9:49like [ __ ] now I have to change again.
9:51[ __ ] I have to change again. I have to
9:52like get the tweaks and the nuance of
9:54the model right every single time.
9:56>> Yeah, there's intuition that you have to
9:58build up and you need it's almost like a
10:00separate mode of being like you're like,
10:02"Okay, I'm going to be less productive
10:04and the the the benefit is not what I
10:06actually produce. It is going to be the
10:09the intuition I build."
10:11>> Have you ever switched keyboards? Like
10:13every time you switch keyboards, like
10:14you're just like slightly less
10:15productive for a while. Like when I
10:17first used the Kinesis, I was like
10:19>> I literally just lost three months of my
10:20life.
10:21>> Like because I could not type like I
10:22used to type. I just feel so dumb for
10:24that three months until I finally got
10:26used to it. And I think that's kind of
10:27what switching models and providers are
10:29like.
10:30>> And yet I can't convince you to try Vim.
10:33>> Uh Vim is so hard. Vim is I cannot for
10:36the life of me understand HJKL. Like
10:40what the like this was designed for an
10:43era with no mice.
10:45>> Can I show you? Can I show you
10:46something?
10:47>> Okay, while you do that, let me read
10:48some of the questions that are popping
10:50up. Um, Matas was asking, "Do you guys
10:54use Soul with sub agents or just soul
10:55alone?" I use Fable for code writing,
10:58research, and web search sub agents and
10:59it beats Soul by a mile, but within the
11:01same harness
11:03um uh without using uh sub agents like I
11:07don't know how you make anything happen.
11:08So, here is my most recent workflow for
11:11big stuff is like
11:13>> basically like I do some research and
11:16create a product outline back and forth
11:17and build a canvas that looks uh
11:19something like let me see if I can pull
11:22something up. Uh oh, I updated
11:26um let me find a good PRD. Uh show me a
11:31workflow.
11:32>> V3 task launcher.
11:37No, not this one. Actually, this
11:39probably has it in there. The Yeah, the
11:41PRD.
11:43So, I will work back and forth and build
11:45like a bunch of like interaction mocks
11:46with the model. Like, okay, here's how
11:48it works. Here's the different parts of
11:50this. Here's the visual stuff that we
11:51need to add.
11:53>> Not fully in like again like these are
11:55HTML mockups. They don't have to be
11:56faithful. It's just like get the
11:58direction
11:59>> and then I will have Soul and Fable
12:00implemented overnight. I'll literally
12:02queue up a fable session and I'll be
12:03like implement and then I'll cue another
12:06prompt that is like okay review it.
12:07Okay, another prompt and it's like use
12:09the codeex CLI to have C codeex review
12:11it and then compact and then do it again
12:13and then do it again and basically like
12:15queuing up like 15 prompts to let them
12:17just argue it out overnight
12:19>> and then I look at what they built in
12:20the morning and I look at at a human in
12:22the loop like polish okay make this look
12:23like this this thing is broken and then
12:25once that's ready I'll go make the PR
12:27and if it's 20,000 lines of code I will
12:29actually work back and forth with the
12:31models are really good at this like how
12:32do I break up this work into separate
12:34PRs where do the migrations go and then
12:36we merge like a one to 3,000 line PR at
12:40a time.
12:41>> Okay. But I've Okay, so I've got a
12:42question for you. When you do this
12:44workload, um
12:46>> what's your wall time of landing it?
12:49>> Like all the work?
12:54>> It's It's hard to say because I am I
12:56will have like bursts of like two days
12:58of a lot of coding and then I'll have
12:59like four days of like doing like CEO
13:02[ __ ]
13:03>> Um
13:04>> that's fair.
13:05>> The feature that this is about I think
13:07This was finished. This part took
13:10probably like a day or two on and off
13:12with doing other stuff. And then this
13:15took like a day. And then I ship the
13:19first five PRs, maybe like one a day in
13:21addition to doing other things. And then
13:24Kyle is picking up the last three PRs
13:26and like polishing it and getting it
13:27ready to like turn on for customers.
13:28>> Oh, you're literally the manifestation
13:30of a CEO that says, "I vibe coded
13:32something. Can my engineer go implement
13:34it?"
13:35That's what you're doing.
13:37>> I implemented the first five pull
13:38requests. I laid the I laid all the GR
13:40and then we got to PR number five and
13:42he's like I don't like how this is
13:43architected. I'm like great. You can fix
13:45it.
13:45>> But it was digestible, right? That PR
13:47where we had to like
13:49>> I'm just No, this is the thing though is
13:50like prototyping is great and like it's
13:52a great way to figure out what you want.
13:54It's a great way to see what's possible.
13:56But like I'm not asking anyone to review
13:58a 20k line PR because it's it's like I'm
14:01not I'm not gonna like have the same bar
14:03of quality. The person who's reviewing
14:04it is not going to have the same bar of
14:06quality and they're going to they're
14:07going to resent it. So you got to make
14:08these like models are really good at
14:10making these things digestible. We don't
14:11do this for everything. I'd say 90% of
14:13what we ship is small enough that you
14:15could just do it in one pass. But if
14:17it's huge and it's prototype and it's
14:18experimental and it's just like I need
14:20to create the vision for what this is
14:22beyond just those HTML mockups, then
14:24it's super valuable.
14:26>> Yeah, I actually agree. I find it's very
14:28like kind of what I'm hearing you say
14:30and I'm going to repeat it back to you
14:31and see if it this is what this is what
14:33I heard
14:34>> which is you basically go through you
14:36get an idea you spend a lot of time on
14:38like model auto looping to produce a
14:41pretty good spec. Then you implement the
14:44whole spec end to end and it produces
14:46basically a slot PR.
14:47>> But the slot PR allows you to decide
14:49whether or not the features are exactly
14:51what you want them to be and you keep
14:53iterating on slot PR.
14:55>> Then you basically
14:56>> this is pretty cheap.
14:58>> This this this part is very cheap in
15:00terms of human time and we have subs and
15:01it's like we use those tokens anyways.
15:04>> Um but the engineering happens here.
15:06[clears throat]
15:07>> Yeah. Yeah, but once you have the slot
15:08PR, then you say, "Okay, this is a slot
15:10PR. Use this as the spec and say that
15:12you were going to implement this from
15:13scratch. How would you redo it?"
15:15>> And like what are the faces down into it
15:19pieces?
15:20>> Yep, that's right. It's like how well
15:21it's like how do you break this down
15:22into digestible chunks and like what are
15:24things that we can incrementally ship to
15:26users that maybe add incremental value
15:28or uh or at least like easy to digest
15:31and review.
15:32>> I want to show you something really cool
15:34actually. Uh this is kind of something
15:37uh I know I know we were talking about a
15:39bunch of stuff
15:40>> but I actually um so we've talked about
15:42this a little bit
15:43>> which is on the idea of like um
15:48okay I'm going to screen share. We've
15:49talked about like uh our one of the
15:51things that we're running into now uh
15:54for context is like we're kind of
15:56running into this issue where we get a
15:57bunch of like tiny PRs and like the
15:59amount of tiny work is like
16:00overwhelmingly large. Uh and each of
16:03those tiny work items will improve the
16:05system. It just takes a while to go do
16:06all of them. So like we've been uh we've
16:08been trying to make our software factor
16:10a little bit better. And I wonder if
16:11you've seen this kind of thing before
16:12and if you guys have considered this
16:14where basically like users report
16:16feedback.
16:17>> Yep. And every single feedback that
16:19comes in, we basically like the idea is
16:22you can't trust any feedback ever
16:23because all feedback is basically
16:25untrustworthy because users are
16:27reporting stuff who knows how. So we
16:29like immediately check if the feedback
16:30is already fixed on the latest version.
16:33>> And if it is, we can just notify that
16:34user really quickly. If it's not, we
16:36basically try and search if it's like an
16:38existing repro or new one.
16:40>> And these are like basically like we put
16:42as like percentages of like how correct
16:43is every single subsystem meant to be
16:45here. like this this will likely be
16:47wrong at some cadence because like you
16:49know like you might like maybe this
16:50feedback report in like you think it's
16:52fixed but it's a flaky issue
16:54>> dduping is going to be buggy like I've
16:56never seen a good dduping agent for this
16:58and the key ones are bad agents are
17:00>> bad if it ddupes half of the things then
17:02you've saved yourself a lot of time
17:04>> exactly so then you do this then you
17:06create like a really really good repro
17:08and I think we can spend a lot of
17:09engineering to make this repro really
17:10really good uh just by like producing
17:13like n others then you actually create
17:15an issue So the idea is like an issue
17:17isn't created from feedback like issues
17:19are created post feedback and then you
17:21just constantly check basically on every
17:23single PR every single one is run and we
17:26just check if every issue is fixed
17:27always.
17:28>> It's very cheap. It's like CPU time uh
17:30to measure it. We don't have like web
17:32browsers or anything to go check. Uh for
17:35some stuff we do and that isn't
17:36>> and you just like keep basically like
17:38prioritizing the work. You assign a
17:40human to it. You basically communicate
17:42with the system and you either like put
17:43a plan together or you like just fix it
17:45and then you just like do the whole
17:46merging system. But like we tried to I'm
17:49have you I mean you guys have built
17:51tried building something like this
17:52before, right?
17:53>> Where was the bottom?
17:54>> This is I mean like
17:56>> you have at the very top an important
17:58piece of text which is for issues with
18:00clear code repros.
18:03>> Uh where do I where do I have that?
18:04Yeah. Yeah. I mean
18:05>> at the very top above feedback reported
18:06like if it's easy to reproduce then this
18:08whole thing works. If it's hard to
18:10reproduce, then uh then you need
18:12something else
18:13>> or you need you need to send an agent to
18:15see if it can create one.
18:18>> Yeah. Yeah. And that's kind of what the
18:19whole like creator repro is cuz like I
18:22don't trust any feedback files by users.
18:24So we kind of just need to spend a burn
18:25our own tokens to try and produce a
18:27clear repro and you can't produce for
18:29clear repro then you assign a human to
18:31it. So like this is the thing we've been
18:33working on internally and like it's like
18:36the next version of agent tries baml but
18:37for like human file feedback and I was
18:39like
18:39>> I was going to say this this feels
18:41shaped very similar to the
18:42infrastructure you wrote for agent tries
18:44BAML and now it's just like cool let's
18:46add another like input stream which is
18:48human tries BAML
18:49>> or humans agent tries BAML.
18:52>> Yeah, it's that one more so than
18:53[laughter] anything else. But it's like
18:56it's fascinating because like this PR
18:57review part is actually the thing that
18:59we're really trying to bottleneck and
19:00like I actually learned this concept
19:01from you when we were talking about this
19:02in the team. What we were talking about
19:04a lot is like how do we push the human
19:06time as low as possible.
19:09So you'll notice like a large part of
19:10this is like assign a human to this like
19:13a shepherd is like down here
19:16and like really this is assignment but
19:18then we actually ping the shepherd when
19:20this happened in Slack. We're like hey
19:21here's a bug fix. Here's the repro.
19:23Here's the bug fix. go read the PR or
19:26this is really hard. Here's a plan. Can
19:28you work can you either like download
19:29the plan into your local agent or like
19:31tell me this plan is good and I'll go
19:33implement it.
19:35>> What's the um what's the uh accuracy
19:38rate of your classify issue difficulty?
19:41Cuz that seems like an is that a human
19:42task?
19:43>> No, we're going to do an agent task and
19:45I think we'll actually what we're going
19:47to do with this is actually a twofold
19:49idea and like we're still playing around
19:51with this. So, we're building like evals
19:52for this actually internally.
19:54>> Yeah.
19:55>> Um, and now we have enough issues as a
19:58database that we actually like have eval
20:00which is kind of nice. Like our first PR
20:01just went out and I think where they go
20:05like what's cool about this approach
20:07that we're doing.
20:08>> By the way, I only see your Excal tab. I
20:10don't know if you're trying to show us
20:11something else.
20:11>> Uh, let me pull this over here.
20:14What's cool about this PR for example
20:16that we're making now is all these PRs.
20:18This is the one rule that we have which
20:20is like you have to push repros onto
20:22here.
20:23>> Yep.
20:23>> So you actually have to give us metrics
20:25for every single version of this
20:26pipeline. So then we can decide whether
20:27or not it's good or not. And like this
20:29is the dduping PR for example. Like
20:32obviously the dduping PR we know is low.
20:34It's lower than we expected. So like we
20:36want to do it even better.
20:38>> Yep.
20:39>> Right. But like
20:41>> Yeah. So we don't actually know what the
20:43XT rate here is yet. So, we're still
20:45going to be playing with this um to
20:48figure out what the accuracy rate is,
20:50but I'm hoping we can get to at least
20:52like 60% correct and that is enough. And
20:56we can measure it always by lines of
20:58code. If the fix is more than 60, if the
21:00fix ends up being mclassified as a small
21:02issue and ends up being 500 lines of
21:04code, we just we can just force classify
21:06it as the other one. So, it's not that
21:08hard.
21:08>> Yeah, that makes sense.
21:10>> Um
21:11>> I'm going to show you one. None of this
21:12was what today's ep Oh, one more. Let's
21:15go.
21:15>> So, uh, if you come,
21:16>> we got a little distracted as as you
21:19guys could see
21:19>> from the chat. If you type Vimtutor,
21:23>> it is a interactive guide that teaches
21:25you Vim. And if you do this for 10
21:27minutes a day, uh, you're welcome.
21:31>> No, no, no. Amigos, we're we're we're in
21:34the voice era. You don't need Vim. You
21:36just need voice.
21:38>> Voice tutor. Um,
21:39>> voice t. That's what [clears throat] I
21:41need. I need a voice that tells me how I
21:43should go talk to talk to other agents.
21:46>> Make you more clear. Uh yeah, me and
21:48Kyle were joking about like giving the
21:49agents a/ bro skill that they can give
21:52back to humans of like that was super
21:54unclear. I need you to like uh speak
21:57more like articulate yourself more
21:59clearly.
22:00>> Honestly, that would probably fix
22:02context problems more than anything else
22:04would. It's probably a good idea.
22:07>> Yep. Cool. Okay. I want to talk about
22:10today our topic which is software
22:12factory design patterns. So we've talked
22:14a lot about the software factory in
22:16various different versions and like cool
22:19you start with the normal software
22:20factory where a human builds the thing
22:22and then you swap it out with agent
22:23builds the thing and you have all these
22:25pieces that like make up your your your
22:28software factory and then you make the
22:29agent test the thing and then like this
22:32part is now fast but the review part is
22:34still slow and like we've talked about
22:35this a bunch of times. Um, I want to
22:38come and talk actually a thing that we
22:39haven't really done is like zoom in on
22:41this part of the factory. Um, which is
22:44like agent builds the thing. And the way
22:48that I think about this is basically
22:50like there's a couple different layers.
22:51We've talked to a lot of different teams
22:53and we'll talk about kind of like
22:54there's there's a couple different
22:55options, right? You could like build
22:58everything inhouse,
23:00uh, which I think is probably what your
23:02team has done. Um and then there's like
23:04the other side of the spectrum which is
23:06like buy everything.
23:08So build everything in house might make
23:10like what do you guys use for your like
23:12um sandbox execution
23:14like where where do your cloud sessions
23:16run
23:17>> for like agent tries?
23:18>> Uh we we buy MacBooks and Mac minis.
23:22>> Okay. So your compute is like agent
23:25sessions run on MacBooks
23:30>> and like basically when an agent session
23:32runs on demand like it I imagine it sets
23:35up some kind of developer environment.
23:38>> Uh yeah and stuff like this.
23:41>> Yeah. But it's all kind of already
23:42pre-installed on the MacBook. So like
23:44part of this is like speed is why we
23:46have this
23:47>> cuz like this core stuff is preset up
23:49and there's no boot up time in that
23:51case.
23:52>> Yep.
23:52>> Right. Um, and that's kind of nice. And
23:55then the other reason that we use
23:56MacBooks is because like we just had
23:57some extra MacBooks lying around.
23:59>> Yep.
24:00>> And then for the harness, you use cloud
24:03code primarily.
24:04>> Cloud code uh cloud code and codeex.
24:07>> I don't know. I don't know what the
24:09latest one is, but it's agnost. It's
24:10like we it's just like CLI you swap out.
24:14>> Yeah. So you use the CLI with some sort
24:16of some sort of headless mode that like
24:18streams out JSON or something, right?
24:20>> Yeah. Exactly. Uh we use headless mode
24:22that stream.json via the file. It dumps
24:25all the chat logs like the chat
24:27transcript, you know, like the JSONL
24:30file. So we just read that
24:32>> and then Yeah. So then you just tail
24:34tail the JSONL file or you like you
24:36scoop it up once the session is done.
24:38>> Yeah, exactly.
24:40>> Um cool. And then like do you have
24:43customizations that you add on there? I
24:45call it like the outer harness, right?
24:46So you have like skills you're
24:48injecting. Do you inject any MCPS or
24:50anything like that?
24:51>> We actually have like actual while loops
24:52that we have that we've built around
24:54this. So like for example to drive
24:56things to completion.
24:58>> Um
24:58>> Okay, cool.
24:59>> So like like for like we have a pull
25:01request bot like code rabbit that we
25:03use.
25:04>> Uh we want to merge after code rabbit.
25:05That's a while loop. That's like a
25:07hard-coded Y loop that actually says
25:08this. And like if the Y loop hits like
25:10max iterations like three,
25:11>> then what we do is like but then we boot
25:13it out and we ask a human to take a
25:14look.
25:19Cool.
25:20>> That makes sense. Um
25:23>> uh
25:23>> we don't merge automatically. Uh we do
25:26have a human hit merge.
25:28>> Yeah. Yeah. Okay. But it's like fix code
25:30rabbit until mergeable.
25:31>> Yeah. Like basically do not notify a
25:33human until code rabbit is happy.
25:37>> Uh and then we have
25:38>> or until three iterations get hit.
25:40>> Yeah. And then uh we have another thing
25:42that we're actually about to add now in
25:44this new loop that I built which is like
25:46our CI/CD takes a long time. So we want
25:49a human to hit that they're happy and
25:51after a human hits that they're happy
25:53this agent is supposed to just
25:54constantly merge against our main branch
25:57and keep it up to date so it can just
25:58mer drive it to merge.
26:01>> I see. So you're merging from main into
26:02the other thing and like basically
26:04looping this part.
26:06>> Yeah. So after the human is happy it's
26:07basically like I I want this to merge.
26:09just babysit this thing until it merges
26:11in cuz we have a merge queue and like
26:13sometimes conflicts happen etc.
26:16>> So this is the last part that we're
26:17adding now and we think this is just
26:18going to help. It's basically just a
26:20better a it's an agentic merge queue.
26:22That's how you can think about it. This
26:23>> Yep. Okay. And where does this run?
26:26>> Um and actually all this is uh going to
26:28be
26:30it's the same machine. It's the same
26:31thing, right?
26:32>> Um so does this run like so this is this
26:35is separate from from agent tries BAML,
26:37right? This is separate from the factory
26:38stuff. This is like running on someone's
26:40workstation.
26:41>> No, this actually we don't want it to
26:42run on your workstation. We want it to
26:44run on a non on a on a work on a like a
26:46sandbox type machine.
26:49>> Okay. So, do you run it in GitHub
26:50actions or does it run on the same
26:52MacBook where the agent
26:54>> runs? Everything is running on the same
26:55MacBook.
26:56>> Yeah.
26:56>> Okay. So, you just have a pool of
26:58MacBooks that run any generic agent
27:00task.
27:02>> Yeah. the agent loop for example. I bet
27:04that one could run as like a pure web
27:06layer because like GitHub is automerge.
27:08So we could just do like automerge and
27:10then go do it and then but right now
27:12like that's kind of how we're thinking
27:14about it.
27:15>> Okay. Um and then like my last question
27:17is like okay so you have this MacBook
27:21and then you have you know your you know
27:23Rust tool chain etc set up.
27:26>> Yeah. Uh,
27:31and then you have your CC/codeex
27:35like your like inner harness. You have
27:37your like
27:38>> outer harness. And then you basically
27:41have like I guess like how does how does
27:43this system know that there's a new
27:45thing to work on? Is it pulling some
27:47queue or like
27:49>> there's some there's some way to turn
27:51this it runs a web server on it locally
27:54>> and then we use some like herder or
27:56something. So, we're running like a web
27:57server.
27:59>> Okay. So, there's a there's a web server
28:01that you can on each MacBook.
28:04>> Yeah. Yeah. And then there's like
28:06another way that we I forgot the app we
28:08use. We use some app that basically
28:09makes that app that makes that thing
28:11like
28:12>> uh accessible
28:14>> like a tail scale or enrock or
28:16something.
28:17>> It's not either of those, but it's
28:18something else. I don't remember it off
28:19the top of my head. I would um but we
28:21use something that makes it basically
28:22like behave like a remote server that's
28:24like deployed somewhere. You have some
28:26sort of like privately accessible like
28:28thing you can open in a browser.
28:31>> Yeah. Well, it's it's a REST API. So
28:33like there's no we don't build a UI for
28:36it. I mean we could but like we could
28:37sort of react off of it but like yeah
28:39it's a rest API.
28:40>> Okay. So what sends requests to this?
28:43>> Uh
28:46now we have yeah well now we have
28:49another web server that we actually host
28:51that's a pure rest endpoint. So now
28:53there's a web app that we build.
28:56Um yeah, and this web app is the thing
28:59that like listens to linear, listens to
29:01Slack, listens to kind of everything and
29:03that communicates with these.
29:05>> Okay, cool. So this is actually like on
29:08on the host.
29:09>> Yeah. Yeah, exactly.
29:10>> And then you have basically a cloud web
29:12server that receives like web hooks from
29:15Slack and Linear.
29:16>> Yeah. And GitHub and all the other
29:18stuff.
29:19>> Yep. Okay.
29:22>> Yeah. And it also has its like own
29:23endpoint that you can communicate with
29:25if you want.
29:27>> Yeah. So you have this thing as kind of
29:28like what I would call your like
29:30dispatcher.
29:32>> Yeah. Yeah. It's also the thing that has
29:34a database and everything on there.
29:36>> Yeah.
29:36>> Um so like
29:38>> so yeah this is the this is basically
29:39what we call like the orchestration
29:41layer. Um, and basically the the the
29:44kind of thing I was like want to get
29:46across is like you can buy all of this,
29:48right? If you use like cursor cloud
29:50agents or cognition or whatever, you get
29:53you get all of that, right?
29:54>> Mhm.
29:56>> Uh,
29:57>> yeah. Every single one of these is
29:59viable. 100% viable.
30:01>> And so like basically what I what I get
30:03into is like what is the what I'm trying
30:04to like elaborate here is like what is
30:06the um what is the what are the building
30:09blocks? What are like the core
30:11interfaces inside the software factory
30:13so that if you decide there's some part
30:16of it that's hard and you want to build
30:17certain things and buy other things like
30:19what does that all look like? And I
30:20think I've sent you this picture before
30:21but I'm going to I'm going to draw it
30:22out here for the for the fans.
30:24>> Yeah. Well, people
30:27>> here I'll let you draw it out and then
30:28I'll just chat.
30:30>> Yeah. People are asking them questions.
30:31I'll just answer them along the way.
30:33What steps you use to run evals?
30:34Honestly, the eval system is pretty
30:36simple and you're seeing that we're
30:37making more rigorous in the PR that I
30:39showed earlier where now we actually
30:40have like benchmarks that we're trying
30:41to set of like we download past issues.
30:44We see if they reproduce and what the
30:45rate is. So like now we actually just
30:47build tests around them and we have like
30:48benchmarks for like what percentage of
30:50tests pass and we have like a pass rate
30:52and stuff that we're building out. Um
30:56all the code is actually open source.
30:57You'll be able to go see it if you want.
30:59Uh do you use containers with G Visor?
31:02Uh we do not use containers. I mean
31:03these are all trusted systems. So the
31:05reason for example that you saw earlier
31:06in my in my Excala draw that I was
31:09showing is like raw feedback coming from
31:11a user is completely untrustworthy. We
31:14run our own prompts to create repros and
31:16those repros are things that we trust.
31:18So we never execute raw code off the
31:20prompt up there. We don't want to. So
31:22this is why it's like slightly more
31:23trustable. So we can just run it
31:24directly on the machine.
31:26Uh, I guess someone could try and
31:28Bitcoin mine it, but I think Claude Code
31:29and stuff are like smart enough to like
31:31not try and do that and we don't really
31:32accept remote code.
31:34>> Uh, and then do we have a sandbox or
31:35Dro? No, we don't use any sandboxes or
31:38anything else. We just run it directly
31:39on the MacBook because again, like I
31:40said, it's trustworthy. So, we don't
31:41have any issues on this.
31:44>> Yep. And so, like again, you can buy or
31:46build that. You could throw it in
31:47Daytona. You could throw it in your
31:48Kubernetes cluster. You could throw it
31:49ETP, Freestyle, whatever it is. Uh, but
31:51you need a place for the aven isolates,
31:54right? Like some people have done this
31:55thing with jetpash where like the agent
31:57runs it doesn't even have a real Linux
31:58file system. It's literally just tools
32:00that look like a file system.
32:02>> Yeah. And then Simon asked the
32:03interesting question like how do you
32:04treat issues that are dependent on
32:06external sources, data providers and
32:08such. The thing is like we don't um
32:10lucky for us we don't have that much of
32:12a problem because we're building a
32:14programming language. So a lot of stuff
32:16is like either self-reroducable or not
32:18and it's very clear when it's not and if
32:20it's not we'll just bump it up to a
32:22human.
32:23>> Like external data source will just go
32:24to a human and a human can make some
32:26educated decision on that kind of
32:27problem.
32:29>> Yep.
32:29>> I think I think the folly that a lot of
32:31people make is they try and make a
32:33system that is fully automatic. That is
32:35much harder than getting a system that's
32:3795% automatic.
32:40So like that's how we're we're aiming
32:42for that latter bit. We're not trying to
32:44be fully automatic. We sadly, not sadly,
32:46I guess luckily will actually keep
32:48employing humans for a while because I
32:50like people and I like making friends
32:53>> if for no other reason.
32:54>> Um, yeah. So, anyways, yeah, the the the
32:57comput stuff is basically like you can
32:59build it, you can manage your own
33:00compute, whether it's EC2 or your own
33:03like Kubernetes cluster or literally you
33:05have a stack of MacBooks in the office
33:06like you can buy it and run it in your
33:09infrastructure. So I know like people
33:11like Daytona will like basically give
33:12you like kind of a black box and you get
33:14an API and just be like hey I need a
33:15sandbox and they'll just spit it up for
33:16you but it runs in your AWS cloud or in
33:19your data center or on your like rack
33:22and office
33:23>> or you can buy where it's just like I
33:26don't want to run any infrastructure. I
33:28have like here's my stuff and here's
33:29like the vendor stuff and I just send a
33:31request and all the execution happens
33:33like in their thing.
33:35>> Yeah. Like basically it's like you can
33:37do as much BOC as you want.
33:40>> Yep.
33:41>> Uh and this is like a very
33:42well-established domain. And
33:45>> I'm gonna ask you a question.
33:47>> How do you decide how much BYOC you
33:49want?
33:50>> So this is this is kind of what I want
33:52to get at is like if you understand
33:53these like four layers or five layers of
33:56the stack, you can make the decisions
33:59based on your needs. Um, and I think the
34:02most interesting part of this whole
34:03thing that I think is the most
34:05controversial is the idea of the dev
34:07environment. Um, because the dev
34:09environment has many layers, right? It's
34:11like, okay, like what like language run
34:14times
34:15do I need to actually be able to uh like
34:19compile and test my software? There's
34:21like how do I like test it from the
34:25outside? So for BAML is very verifiable,
34:27but if I need to run a web app, I need a
34:29way not just to like send work to this
34:32workstation, but I need to be able to go
34:34like look at what the agent is building
34:35and see a preview of it, right?
34:38>> You have like a lot of a lot of larger
34:40teams have like internal services where
34:43like you might run like four layers of
34:45your stack when you're developing, but
34:47actually the app has like 50, you know,
34:5050 plus services that it needs to run
34:52and these are all shared.
34:55>> Yep. Uh, you know what's
34:56>> like shared dev? We call it like a dev
34:59cloud or something like this.
35:02>> You know what's um Okay, I'll I'll let
35:05you keep going. Actually, I want to keep
35:06I want to see where this goes.
35:07>> Yeah. Yeah. Yeah. Yeah. Uh just in the
35:10interest of time. Um, basically my take
35:14is like once you're going into the
35:15vendor's cloud, if if it's running here
35:17and you need to test it, like telling
35:20telling Daytona or whatever, hey, I need
35:22to install like I need to use this base
35:24Docker image cuz it has Rust
35:25pre-installed at a specific version that
35:27we like is easy, but when you get to the
35:29point where like I need to like poke
35:30holes in my in my uh in my like cloud to
35:34allow this running thing in someone
35:36else's compute cloud to access uh the
35:40shared things that my stack pack needs
35:42cuz we have a 100 repos and I don't want
35:43to run all of them in the sandbox. Uh
35:46this becomes like quite a bit of
35:48friction and the the need to basically
35:51like debug oh the rust thing is not
35:53working on this thing because they're
35:55using like a fake Linux kernel that
35:56doesn't support this sys call or
35:58something. Uh basically like uh it it
36:03can cause a lot of friction and my one
36:04of my thesis is like you are going to
36:06want to own this uh unless you're
36:08building like tiny little toy nex.js JS
36:10apps and you're like, you know, in the
36:12like majority like in distribution set
36:15of stuff, you're going to start to hit
36:17friction unless you can own this in a
36:19really clean way. Does that make sense?
36:21>> I agree.
36:23>> Cool. Um, the next layer is the harness.
36:27So, this is like basically and I I think
36:29we probably split this into like in
36:30yours we kind of split it into like
36:31inner hardness versus outer harness. you
36:33have like the core cloud code or codeex
36:36or um this could be like or you can like
36:40or amp or devon or factory. You can like
36:43buy this from a vendor who builds a
36:44harness or you could like build your own
36:47on top of like open code PI custom
36:49things. And basically like you can you
36:52can you can kind of say like okay cool
36:54there's the
36:56inner harness is really thin and we
36:58build a bunch of like outer harness
37:00stuff or you can basically buy an inner
37:03harness that comes with a ton of stuff
37:05like a browser and testing and stuff. Uh
37:10oh my god I can't type.
37:12And then you can have your outer outer
37:14harness be like very thin where it's
37:16like just your skills and stuff.
37:19Does that make sense?
37:21>> Yep.
37:22>> Yeah. You have options here how you do
37:24[clears throat] compaction, how you do
37:26testing. Yeah.
37:27>> Uh what's fascinating here is like I
37:29don't know if anyone in the audience has
37:31ever worked at a like Google or
37:33Facebook, but if you have then you might
37:36see a lot of this stack is literally
37:37something that you have probably worked
37:38on before. And Google, for example, they
37:40buy you a really nice MacBook, but you
37:42never code on your MacBook. You always
37:44get what's called a cloud top. And every
37:46single time you get a new environment,
37:47you actually set up a whole new cloudtop
37:49and you just set that up repeatedly. And
37:51Facebook's the same way. And like that's
37:53just how your default coding. And like
37:56same thing here, like when
37:57>> no one ever has
38:00no one ever has code checked out on
38:02their workstation because like they
38:03basically don't want you to take the
38:05code outside the building. Well, that's
38:07one part of it, but also it's just like
38:08too big. Like you can't have that. And
38:10then the other thing that's really
38:12fascinating to me is like
38:13>> whenever you make like web services and
38:15you go do them, you get default URLs
38:17that are attached to you. So like my uh
38:20uh my I don't I don't even remember what
38:22my LDAP was. I'm pretty sure it's like
38:24what my user ID was. Like I got a
38:26subdomain that's like myeldap.c.google
38:30or some internal Google domain.com. I
38:32could send that around to people. So if
38:33I ever build like an internal UI, by
38:36default it's sharable in my whole
38:37company if I want it to be. And you get
38:40>> you can just share that URL and a PM
38:42could go look at the web app or whatever
38:43it was.
38:44>> Exact. And what that means is like any
38:46service I want to point to, I can have
38:47my own version of the service or I can
38:49have a hosted version of the service
38:50that I point to by default and it's
38:51always kind of just works
38:54>> um in that way. And what's funny to me
38:56is it feels like we're all trying to
38:58reinvent that kind of golden stack where
39:00like you go in, you press a button, you
39:01get a cloud top issue to you, you can
39:03start coding on it, it does the work,
39:05you can access it from anywhere. You can
39:07see all the code on it from the same
39:08editor that you're the same editor can
39:10just switch to like different machines
39:12really really fast.
39:14>> And then um on top of that you get all
39:16these like nice little web systems where
39:18like it just points the right database,
39:20right URLs and like sharing and
39:21everything is provisioned. You can't
39:23share URLs publicly unless you
39:25explicitly escalate them to a public URL
39:27that's sharable. So, it's internet only.
39:30>> Anyway, I just find it so interesting
39:32that like we're doing all this stuff and
39:34like there are a couple companies that
39:36have done this like perfectly already by
39:38necessity of their work. But it's also
39:41it's like there's something to be said
39:42for like you might not be Google and you
39:44might be able to get away with basically
39:46like a full stack cloud agent that does
39:49all of this kind of like for you because
39:51you don't need that flexibility.
39:52>> Yeah. Yeah. Yeah. Exactly.
39:55>> Um cool. The last piece here is I think
39:57the most interesting and underserved
39:59piece. I'll take the pitch part of this
40:00out but is basically the control plan,
40:03the orchestration layer. Uh, and so like
40:07basically the idea here is like you want
40:09to be able to dispatch new work. You
40:11want to be able to see what's you want
40:12to be able to look at the session
40:13traces. You want to be able to look at
40:15the plans and architecture docs that are
40:18being made. You want to be able to like
40:20schedule things to run every night or on
40:22a cron or in response to web hooks. You
40:25want to be able to like review and
40:26iterate on the code itself. uh sort of
40:29like a PR shaped thing but like doesn't
40:32necessarily need to happen in GitHub.
40:34There's like permissions and audit and
40:37who can talk to what and who can see
40:38which services and like managing spend
40:40and budgeting. And then there's like
40:43this idea of compounding engineering. I
40:44we'll start high level. We'll dig into
40:46like whichever one of these you think is
40:47the most interesting. Compounding
40:48engineering is we talked about this on a
40:50an episode like a month ago of like how
40:52do we how do we build a memory system
40:54where it's like hey if all your
40:56engineers across your whole team are
40:57yelling at Claude all day and they're
40:59all saying the same thing or codecs or
41:01whatever it is how do you incorporate
41:04that into your uh your outer harness
41:07basically.
41:08>> Yeah. Um this is I I I mean if I were to
41:11go and write the layers of stack I could
41:13not come up with a better representation
41:14of the stack. Um
41:16>> there is there is one argument to say
41:18that like the dev environment is
41:21actually part of the outer harness and
41:23it doesn't it doesn't sit underneath the
41:24harness. It's actually part of the
41:25harness. But uh
41:27>> I disagree.
41:27>> We'll we'll we'll avoid that that
41:29semantic right now.
41:30>> I I I disagree. I think it's I think
41:32it's better to have the separate layer
41:33because it's like
41:36when I get a company laptop for example,
41:38there's a bunch of crap I have to
41:39install. I just want that installed.
41:42>> Yeah. There. And like to me like what
41:43else goes in the dev environment is like
41:45identity provisioning like that all goes
41:47into the dev environment not the harness
41:50and that's why I think um
41:52>> you mean like authenticating to internal
41:54services and integrations and stuff.
41:56>> Yeah. Just like identity attached to
41:57that environment. There's some sort and
41:59that implicitly gives you internal
42:01services. It gives you access to certain
42:02API keys, gives you access to certain
42:04scopes like all the stuff you might
42:06expect.
42:07>> Um but that belongs in the dev
42:09environment layer. It's not in the
42:10infrastructure layer. it's not in the
42:11harness layer. Um because like I want my
42:14harness to not have to think about
42:16certain provisioning.
42:18>> Yep.
42:18>> Um and
42:19>> one other thing I think is interesting
42:20on the dev environment side, then I want
42:21to flip it to the audience and hear like
42:23which parts of this stack are the most
42:24interesting to drill into. Um are you
42:27familiar with the idea of like pets
42:28versus cattle?
42:30>> Okay. While while you describe a new uh
42:33concept to me, um I will read the
42:35comments. Describe the concept to me.
42:38>> So yeah. So
42:38>> I have no idea what this is. basically
42:40this idea of like you kind of like have
42:43some number of server you like prod
42:45web01
42:47you have prod web 02 you have like a set
42:49of servers that have like roles in your
42:51infrastructure
42:52>> okay
42:53>> uh and like if one of them goes down
42:55it's like cool I got to go rebuild prod
42:57web 03 cuz the disc filled up or
42:59whatever it is
43:00>> okay what's cattle
43:02>> and cattle is basically this idea of
43:03like you don't care if it goes away
43:06you've designed your
43:07>> oh yeah it's like web previews for
43:09example web previews on PRS or cattle.
43:12>> Exactly. So it's basically just like web
43:13and then you have some like you know
43:16>> random UID or something. Yeah. Yeah.
43:18>> Yeah. And then you just kind of have
43:20infinite of these and these are usually
43:21cattle is more like okay we have a
43:23script that can provision these. No one
43:24has to go SSH into the server and set it
43:26up.
43:27>> Uh and like basically just on demand. So
43:30like I would say your MacBook setup
43:31today is actually more of a pet setup.
43:33And I actually think this is a great
43:35place to start. I think you can get very
43:37far with like basically like if if you
43:40bought a new MacBook and you wanted to
43:42add it to the stack, you would go
43:44someone would go like pull it up and
43:46manually set it up. You don't have a
43:47button you can push that makes it ready
43:49to receive like workloads, right?
43:51>> Well, there's a repo that you download,
43:53you start running uh some start some
43:56script and it just provisions it
43:57automatically.
43:59>> Yeah. Um
44:00>> but you do have to download the repo and
44:01go do that. We made it. It's not like
44:03someone has to go basically chef or
44:06>> you need the hardware.
44:07>> You need the hardware to go run it. Uh
44:09which means and we own the hardware.
44:12>> Yeah. So when I worked it, we basically
44:14like we built a system to turn
44:16everybody's workstation in like
44:18undergrad in college. We we worked it
44:20for the library and we basically built a
44:22system to turn all of the workstation
44:24from pets into cattle
44:25>> cattle. And we had this like literal
44:27station in the basement where like
44:29whenever someone's machine got corrupted
44:31or got a virus or whatever it was,
44:33instead of going in and trying to fix
44:34what was going on, we would literally
44:37pop two drives in the in the bay, clone
44:39from one drive to the other, bring them
44:41a new drive, put it in their machine,
44:43and be like, "Cool, go download your
44:44[ __ ] from the from the file share."
44:47>> Yeah. Yeah. No, I I I agree that dev
44:50environment should be more cattle than
44:51pro uh than bats. 100%. I actually like
44:54when we build our dev environments and
44:56like we have a similar stack to you. I
44:57run some stuff in EC2. I run some stuff
44:59in XC.dev.
45:00>> We actually treat it like pets for now
45:02because we don't have [ __ ] thousands
45:04of work workflows starting up and down.
45:06I just leave one server running all day
45:09>> and when I need to run stuff on it, I
45:10run stuff on it.
45:11>> When I need to authenticate to GitHub
45:13again, I just log in and reauth to
45:15GitHub. It's
45:16>> another great analogy for everyone else
45:18is like I would say like AWS Lambda
45:19functions or cattle. like an EC2
45:22container you own is more pet than
45:24cattle.
45:25>> It's it's kind of in the middle. Uh but
45:27it's it's kind of in that same kind of
45:29workflow.
45:30>> Um we've got a couple questions uh in
45:32here that I want to go over.
45:33>> Uh it's like um it's a weird network
45:36cluster. My factory ended up as a tail
45:38scale sprawl. Tunnels to extend. Uh
45:40tunnels are Yeah, I mean yeah, we don't
45:42use tail scale for that reason. Like
45:43it's kind of annoying. Um Sedarth has
45:46some things. It's like uh I sort of
45:48understand how software factories can
45:50behave in a single repro. How do you
45:52deal with multiple repros when you have
45:54like multiple things to be completely
45:56honest sadly my recommendation is uh
45:59you're going to be sad because you're
46:01not using a monor repo. So the best
46:03thing you can do and I know Dexter has
46:05helped teams that have tons of like
46:07micro repos
46:10teams with 80 repos and or 100 repos or
46:13whatever it is. So like my
46:14recommendation would be like simulate a
46:16monor repo as much as you can and you
46:18can do that in many ways. You can get
46:20like a checkout of you can go to a
46:22folder where you sim link all the monor
46:23repos in there. You sim link it and
46:25treat it like a monor repo. You can have
46:27an actual monor repo that actually like
46:29just contains a list of sim links to
46:31every single repo that you have and it
46:32just like
46:33>> with sim links
46:35>> like cloud can traverse your workspace
46:37pretty easily. So if I come here and
46:39look at the code here we literally this
46:41is our template for this. We don't use
46:42this anymore cuz this is built into
46:43human layer now. But uh yeah, we
46:45basically say like if you're using
46:47especially if you're doing research and
46:48you want to go find the stuff, it's like
46:50cool, you're in a coordination repo for
46:52multiple repos. They're all like one
46:54level up basically. And so the way the
46:56way that you lay things out on disk is
46:58basically like source repo one, repo 2
47:01coordination. They don't need to be in
47:02the same git tree. You don't need subm
47:03modules. You don't need sub trees. You
47:05don't need sim links. As long as the
47:07model knows all of the things that might
47:08be in scope and you give it a like part
47:11of your workflow that is like go do
47:12research and understand what matters to
47:14this uh this is like it's really not
47:17like a product thing. It's literally
47:19just like a cool just tell your model
47:21where the other [ __ ] is and like
47:22>> you still uh while you say that actually
47:24you still don't need one
47:27like control file that tells you the
47:29organization structure and the layout
47:30system that you're going to have.
47:32>> Yeah. Right. So this is like but like it
47:34doesn't have to be sim linked into every
47:35repo. You don't need every repo to have
47:37like a doubly linked like two-way
47:40reference to all the other ones. We just
47:42run all of our session you just run all
47:43of your sessions from one coordinate and
47:45this can be a separate repo with just
47:47this cloud MD in it
47:48>> or it can be or it can be one you know
47:52the repo that everyone uses the most and
47:54then sometimes you touch other repos
47:55well you put all the config there and
47:57even if you're not touching that main
47:58repo you just you have one canonical
48:00place where you start agent work from
48:02and then you give it the ability to find
48:04those other repos. The sandbox thing
48:06gets a little trickier if you're doing
48:07on demand sandboxes because like do you
48:09really want to clone 200 repos onto the
48:11box every single time? Well, that's
48:13going to make your thing slower and you
48:15need more disk and things like this. The
48:17alternative is like make the human pick
48:19which repos are relevant, which works to
48:20a point, but as soon as I want to like
48:22take advantage of the fact that agents
48:24can help me be productive in code bases
48:26that I'm not an expert in. Now we're
48:28back to like, oh, if I missed a thing
48:30that was important, then I'm back to
48:31like, oh, I need to go talk to the
48:33person or know, I need to have that
48:34intuition about the codebase to know
48:36that. And like we really want to
48:37>> enable you to leverage agents to do
48:40this. So, this is another time where
48:41it's like
48:42>> you could do cattle, but then you're
48:44bulk cloning all the repos all the time
48:45and have to keep them all up to date. Or
48:47you could do pets and just have them all
48:48checked out.
48:49>> Yeah. Um, what I would the other thing I
48:50would recommend is please don't use subm
48:52modules.
48:53>> Uh, when you use subm modules, subm
48:54modules are horrible because they pin
48:56the specific commits in every repro.
48:58You're going to be a sad sad cookie.
49:00Unless you're
49:00>> sub trees are good though.
49:02>> I know a lot of people who are who are
49:03skilled with this stuff that do a lot of
49:05work with get sub trees and I've
49:06actually played with them and like I
49:07think I think it's an option that makes
49:09>> I haven't actually used sub trees. So,
49:10I'd be I I'll have to go check it out
49:12and see how they're different. How are
49:13they different? Do you know off the top
49:14of your head?
49:15>> I I did the tutorial like a month or two
49:18ago. I I I can't I I can't do it justice
49:20today, but it's it's somewhere it's a
49:22it's kind of like a sub a work tree and
49:24a subm module had a baby.
49:27>> I see. Okay. Yeah. Cuz the biggest
49:29problem I have with subm modules, it's
49:30just like you you have to like commit in
49:32two places every time you update the
49:34every time you update the module, the
49:36subm module and their actual repo. And
49:37it's just like not fun. Yeah, sub trees
49:40have a little bit of that. But I think I
49:42think also I've heard from people who do
49:43a lot of this and like who especially I
49:45think Mos Mike Hostetler was on an
49:47episode like six months ago where he was
49:49showing his like Ralph loop with git sub
49:50trees set up to help people basically do
49:52agent work across all their reps.
49:54>> You want to pull up your thing really
49:55fast? I have a couple other interesting
49:56you scaladraw that you had. I had a
49:58couple of things I was going to do.
50:00>> People have got some other interesting
50:01things of like uh where do you see the
50:03software factory design pattern evolving
50:06in the near in the near term. Honestly,
50:08I think the biggest thing that's going
50:09to happen is more people like actually
50:12what's funny is I should actually just
50:13paste our our workflow into here because
50:15it's basically the same. It's like a
50:17more micro version of a part of this
50:19stack.
50:20>> Yeah.
50:20>> So, I'll put that in here really fast.
50:22But like when I think about this,
50:25when I think about the evolution, I'm
50:26curious about your thoughts on this
50:28evolution.
50:29>> Yep.
50:30>> Uh let me Sorry, I'm coming to the
50:32Excala draw and I'll post it down here.
50:33There you go. It's at the bottom part.
50:36>> Nice.
50:37Um when I when I go think about this
50:42um
50:44uh when I the way I see this evolving is
50:47one it's going to become a lot more
50:48frequent.
50:49Like every company is likely going to
50:51build some sort of internal agent that
50:53does something like this. And like I
50:54personally don't think the whole thing
50:56is viable. Uh, and the reason I don't
50:58think the whole stack is viable is
51:00because every company has some weird
51:03quirk about their um internal stuff.
51:06That is how their setup works and
51:08everything. So, it's like
51:11you just can't know everything. Like for
51:13us, like someone asked earlier, how do
51:14you deal with issues with external
51:16resources? Well, like
51:17>> we're lucky we don't have that problem
51:19or it's so rare for us. We don't have to
51:20design around that system.
51:22On the other hand,
51:24>> this is
51:24>> this is why I'm so obsessed with like
51:27defining these building blocks. Yeah.
51:29And like defining really good interfaces
51:31between them. So like the interface here
51:34is is an interesting one. Like ACP tries
51:37to do this uh but ACP is kind of
51:40>> on hit X on Riverside again please.
51:43>> I can't see it full screen and I want to
51:44see Dexter's diagramming and max
51:46potential.
51:47>> Yeah. So like between the control plane
51:49and the harness you have something
51:50called ACP as an option. It's quite
51:52narrow. Um I think we need like a
51:54thicker version like a wider version of
51:56this interface
51:57>> for a like coding harness to talk to a
52:01UI or an editor or a web app. Basically
52:04it's like a protocol for things to talk
52:06along. You also have things like AGUI
52:08which again is great for broadcasting
52:10the UI and the events but actually like
52:12the
52:13>> the challenge I have is neither of these
52:14support hooks which means like something
52:16that is like job it is to life cycle a
52:19harness and like its events and do
52:21special things in response to like I
52:23don't know in human layer we have a
52:24bunch of hooks of like
52:26>> yeah go ahead
52:26>> do you know why though I think it's
52:28because every single harness is
52:30literally basically its own bespoke UI.
52:33>> Yep. and like converting every like
52:35claude code and codeex do not have the
52:37same hooks and pi has totally different
52:39hooks.
52:40>> Open code has a plug-in system that's
52:42like the hooks are like a completely
52:44separate concept.
52:45>> Yeah. And I think this is kind of where
52:47it like all these things die which is
52:49that it's not a
52:51>> there's no standard for this there and
52:53and rightfully so like every harness has
52:56different trade-offs. Like clawed code
52:58is much much more like you bring it and
53:00it's good like it's guaranteed to be
53:03good.
53:04>> Yep.
53:04>> Whereas like uh pi is more like you have
53:07to configure every little thing and that
53:09allows you to have
53:11>> like it comes with control but you have
53:13to you have to build more stuff or but
53:16also you have to build more stuff.
53:18>> Yeah. Exactly. Right. And I think that's
53:20why like you can't really do this cuz
53:22the abstraction layer in which you
53:24represent everything there is just not
53:27that good because it and and it by
53:30design can't be that good at least in my
53:32eyes uh not right now but also like you
53:34know I used to think about web and then
53:35I finally learned about react I was like
53:36oh [ __ ] they actually did solve this
53:38hard problem by inventing this concept
53:40of state turns out state is all you need
53:42and everything else kind of just
53:43devolves from that in a really nice way
53:46and like whether you like react or
53:47angular doesn't really matter but like
53:48web web systems generally are state
53:50driven engines and that was a novel
53:53concept.
53:54>> So like you just got to think about
53:55what's the boundary here to think about
53:57and I don't know what it is and I think
53:58that's why it feels like you guys I know
54:00do a lot of work of wrapping every
54:02single one of these harnesses and making
54:03it good to plug into.
54:06>> Yep.
54:08Yes sir. So um I think that's most of
54:12what I had. I mean are there more are
54:13there more comments in the chat? I mean
54:15like yeah the the interface between the
54:16harness and the dev environment is
54:17something like the Linux kernel and like
54:19I don't know the dev environment the
54:21infrastructure
54:23>> infrastructure is mostly like how do you
54:25go set it up it's like what APIs do you
54:27have what control do you have how do you
54:28go build this
54:30>> there is something called compute SDK
54:32that tries to be this which is basically
54:35like one library for oh it's become a
54:38they made decided to be a benchmark
54:40instead of a instead of an SDK but it
54:43used to be
54:44>> like it doesn't really matter for that
54:46layer. Like it's the same reason that
54:47you don't like yes it's nice to be able
54:49to abstract over like GCP and AWS but
54:52thing is once you're a big enough
54:53company you're basically either a GCP
54:55shop or AWS up no one really needs
54:58>> to swap stuff over that often and when
55:00you do use
55:01>> the AWS interface is already pretty
55:03good.
55:04>> Yeah, that's kind of what I mean. Like
55:06if you're using Daytona then Daytona
55:07interface is good. If you're using like
55:09GCP, the GCP SDK is good and the GCP
55:12CLIs.
55:13>> So like all these things ship CLI. Go
55:16ahead.
55:16>> We used to have this rule of like at
55:18Sprout, we said don't wrap interfaces
55:20with other interfaces or don't wrap APIs
55:22with other APIs. It was like okay cool.
55:24like we have the official Twitter SDK
55:26and then I'm going to make our version
55:28of the Twitter client which just wraps
55:30some of the methods that we need and all
55:32the data is just passed through with a
55:34little bit of like transformation and
55:35it's way better to just like
55:37>> just call the official SDK at every call
55:40site. You don't need to like put an
55:41abstraction in front of it if it all it
55:43does is like actually create
55:45>> like okay now I have to like do extra
55:48work every time I want to add a new call
55:50to a new endpoint on Twitter. That's
55:52kind anyway that was my spiel about
55:53where I think the interface stuff was
55:55going. In terms of buy versus build,
55:57where do you land? Obviously, I know
55:59you're biased, but that's okay.
56:00>> I mean, I think the idea is like I think
56:03when you use a f fully verticalized
56:06cloud agent, some of the challenges you
56:08will hit is there's friction in defining
56:10your dev environment. There's friction
56:11in having limited like choices on
56:13compute. I think what Devon and
56:15Cognition are doing is really
56:16interesting in terms of like letting you
56:18run what they call like outposts or
56:20workers where it's like okay you can run
56:22the compute and the outpost will bring
56:24the harness and the tooling into your
56:28level of compute and you and the harness
56:30can kind of co-own the dev environment.
56:33Uh, and so like things like that I think
56:35we're going to see more and more of and
56:37like the the conversations we're having
56:38with a lot of people and like trying to
56:40figure out is like what are the right
56:41interfaces and I think what a lot of
56:43people are starting to realize one more
56:46time or I can sure
56:48>> most people are either like buy the
56:50whole thing or build the whole thing and
56:53I think the answer is like you should be
56:55able to instead of having like it's like
56:57it's like composition over inheritance
56:59right it's like rather than having to be
57:00like okay whatever layer I buy at I have
57:03to buy everything below it. And so it's
57:05like, oh, if I want orchestration and I
57:07want to be able to ping agents from
57:09Slack, I either have to build the whole
57:11freaking thing or I have to buy the
57:13whole freaking thing. And like the the
57:15real answer is like you should have as
57:16an engineer choice and the ability to
57:18like work in open systems and plug these
57:21things together.
57:23>> Uh so like what's interesting for us is
57:25like uh our a you're Devon is kind of at
57:28these three layers, right?
57:30>> Yeah.
57:31>> And they do the infrastructure too. No,
57:32they do the cloud sandboxes. Yeah, you
57:33can move that.
57:34>> Yes, but I think they kind of own they
57:36don't want to own all of it. It's like
57:37optional for them.
57:39>> Uh I I think in most cases they they
57:43want to own it. Uh just because it's a
57:45better experience. Like it's easier if I
57:46can just sign up and like send a Slack
57:48message and see work start getting done
57:50versus like the price.
57:53>> Huh.
57:54>> For enterprise customers, I think they
57:55just run the whole thing on prem. Um but
57:56yeah, enterprise customers. Yeah.
57:58>> So enterprise wants to bring the compute
57:59and probably bring parts of the dev
58:01environment. Yep.
58:02>> Yeah. Like for us, what's interesting is
58:04like when we build our agent drives BAML
58:06stack, what we're really building is
58:07this this kind of we're building like up
58:09here and we're kind of like we don't we
58:11explicitly do not want to own this
58:13layer. We don't want to own
58:14>> because you want to use the harness that
58:15everyone else is using
58:16>> and like really
58:19well I want to make it swappable cuz
58:21yeah cuz the harness is going to change
58:22like all the time. And really I don't
58:23really want to own the compute sandbox.
58:25I just do this because of like
58:26convenience. Like whether we run on a
58:28box or not, I don't really care.
58:30>> But these are the parts that I I find
58:31valuable. It sounds like
58:33>> the layer that you're trying to help is
58:34like, hey, you for human layer
58:38>> uh human layer,
58:40>> you just wanted to buy the
58:42orchestration,
58:43>> but you wanted to bring your own
58:44harness.
58:46>> Yeah.
58:47>> This is kind of what you're trying to
58:48help plug into.
58:49>> Yeah. and Devon is there and like cloud
58:52cloud automations like cloud background
58:54stuff is like cool you can do like cloud
58:56code as a building block
58:59>> or you can bring the entire cloud code
59:01st Oh oh that's not good or you can like
59:04>> I got it
59:04>> basically I got I got stop I don't want
59:07that one you can have like basically
59:10like clawed uh what do they call it like
59:13automations or I think they call them
59:15routines or something but it's like
59:17basically like where they'll bring the
59:19compute environment and stuff too.
59:22>> Yeah. And I'm going to do one more thing
59:24really fast. Um, put this up here. And
59:27like I would say like there Claude has
59:28cron jobs and stuff that you can do and
59:30they're trying to like plug into this
59:31part over here, right?
59:33>> Yeah, exactly.
59:34>> Craw where they're trying to be like
59:36>> so you can if you want to be the full
59:39Cloud stack and like uh I know Pearson
59:40Marks does this the the Jelly Pod guy.
59:43He was demoing this at the last thing.
59:44like he uses the full cloud stack and
59:46like he knows it's separable but he
59:47likes cloud code as a harness. He thinks
59:49the dev environments there are good
59:51enough for him and the compute on demand
59:52is great and he can schedule automations
59:54on it and so he his you know composed
59:57stack happens to be all in the cloud
59:59ecosystem.
1:00:00>> Exactly. [clears throat] Um I'm going to
1:00:02go read some comments because we got a
1:00:03bunch more really fast.
1:00:05>> Some people are building both. They're
1:00:06like fully multiplayer. It sounds like
1:00:08there's a product there that's kind of
1:00:09cool. if you can drop in the chat if
1:00:11it's interesting. Uh given that we have
1:00:13a choice that it stack would a lampstack
1:00:14equivalent emerge I think so I actually
1:00:17think we will get an equivalent to a
1:00:18lamp stack here over time. Um
1:00:20>> you mean nextJS
1:00:22next.js superbase and uh work.
1:00:25>> No no no no I actually I don't know
1:00:27about that because that's not really
1:00:29what this is about. This is like the
1:00:31harness the dev environment the
1:00:32infrastructure deployments like dev
1:00:34environment is much more closer to
1:00:35docker for me
1:00:36>> like the infrastructure
1:00:37>> application stack but the yeah
1:00:40>> the yeah like for this like
1:00:41>> software factory stack
1:00:42>> software factory stack yeah exactly uh
1:00:44ATB is like a thing that we call like
1:00:46agent tries BAML which is like our own
1:00:48software factory that we built to
1:00:49automatically improve the BAML
1:00:51programming language
1:00:53and then yes these videos are recorded
1:00:55if you want to check them out they're on
1:00:56the YouTube channel I'll drop them off
1:00:58in a bit
1:00:59>> we post them on Twitter as well.
1:01:00>> Yeah. And then is there an open-source
1:01:03project for control plane orchestration?
1:01:05I think the reason that there isn't an
1:01:07open source project for control plane
1:01:08orchestration right now
1:01:11um is because like one like no one you
1:01:15don't want to use someone else's control
1:01:16plane because it's so easy to write code
1:01:18now unless you actually get all the
1:01:21integrations built with it. And right
1:01:23now everyone is everyone that's doing
1:01:25that to some degree wants to make a
1:01:27product and they have some value to
1:01:28serve. So like they want to sell it to
1:01:30you and that's not an unreasonable
1:01:31request. There's a lot of good design
1:01:33and intuition that goes into it and like
1:01:36>> there is um we met the open inspect guy
1:01:39last last week but he's that's open
1:01:42source but it's another full stack. His
1:01:44his is like okay cool we bring I think
1:01:46it's like basically like it does all of
1:01:48this and you bring the compute to run it
1:01:50on basically.
1:01:51>> Yeah. Exactly. And like I that's what I
1:01:53mean like there is stuff but it's so
1:01:55easy to build now that I would just
1:01:57build like for us for example like we
1:01:59just decided the two layers of the stack
1:02:00we care about and we just plug and play
1:02:02and like you ask me like why would like
1:02:04I actually have thought like should we
1:02:06use human layer stuff for part of our
1:02:07control plane and like the only reason
1:02:09we don't is because like
1:02:10>> they don't have an API surface that I
1:02:12can't like plug into my slack into our
1:02:13GitHub issues into our BAML CLI. So now
1:02:16we have to build this whole layer around
1:02:17this and I think the input control plane
1:02:22Oh yeah, but the point is like we still
1:02:24have to build an input control plane to
1:02:26define what we consider an issue. Like
1:02:28maybe we could use human layer for like
1:02:29the good once an issue is good, but for
1:02:32that untrustworthy feedback that we get
1:02:34from a another user's agent.
1:02:37>> Where do we run that? Like that's a and
1:02:38this is the same reason that we don't
1:02:40just use linear pure linear for this
1:02:41anymore because like we have two
1:02:44different versions of issues and when
1:02:46you ask like is there an open source
1:02:47project the problem is every company's
1:02:49working style has different definitions.
1:02:53Yeah, this is David Cross's thing of
1:02:55like software should be extensible now
1:02:57that agents can write code. Like why are
1:02:59you going to make me plumb some like
1:03:01wissy wig like integration manager of
1:03:04like cool when a linear web hook comes
1:03:05in transform it in this way and then do
1:03:08a workflow. It's like just let me write
1:03:09the code for that.
1:03:10>> Yeah, exactly. And Matas is asking like
1:03:13do I see an open source I do personally
1:03:14see an open source solution happening
1:03:16and like I suspect companies like you
1:03:18will probably eventually support
1:03:19something like this because like it's in
1:03:20everyone's best interest to go do that
1:03:22and if you have a good prod stack you
1:03:24would build a good open lure solution
1:03:26but I think it's really hard to upfront
1:03:28build an open source solution like part
1:03:29of what I really admire about the work
1:03:31that Dra has done is that like they're
1:03:33working with real companies and real
1:03:35companies have real products and instead
1:03:37of optimizing for one company what they
1:03:39can do is they can take the learnings
1:03:40across like 10, 20, 30, 40, 50, 100, a
1:03:43thousand companies and build the best
1:03:45interface that really serves across all
1:03:47of these. And then you can imagine like
1:03:50an open source thing being much more u
1:03:54um able to being built like you can't
1:03:57build React the first time the web was
1:03:59invented. You [laughter] really have to
1:04:00watch a lot of stuff happen right and
1:04:04wrong before you see that happening.
1:04:06>> Well, and I think step one is the
1:04:08interface layer. like MCP did this
1:04:10really interesting thing where it
1:04:11created this really explosion of
1:04:14ecosystems that let both like agent
1:04:16client and agent harness builders and
1:04:19integration builders like mix and match
1:04:22with each other really easily. And I
1:04:24think basically like there's a lot of
1:04:26things that we're working on is like
1:04:27less like what is the what does the
1:04:29product look like at any one of these
1:04:31layers and more like what are the open
1:04:33interfaces that enable people to have
1:04:36choice and be able to buy versus build
1:04:39any part of this stack for the parts
1:04:41they want control over without having to
1:04:43uh
1:04:44>> without having to uh to to go all in on
1:04:47like let's say the cloud stack or the
1:04:50the codec stack or the open inspect or
1:04:52build the whole thing themselves.
1:04:54And it still is to show like how nice do
1:04:56Claude and Codex want to play with each
1:04:58other.
1:04:59>> Yeah.
1:04:59>> Like if they don't want to play nice
1:05:00with each other, they will keep breaking
1:05:02APIs and open.
1:05:04>> This is why Claude doesn't support
1:05:05agents MD cuz they're like we want
1:05:06Claude MD to be the Cloud thing and
1:05:08everyone else can use the other file,
1:05:10but we're special.
1:05:12>> Yeah. And like they've just decided
1:05:13that's the case and it's annoying,
1:05:15>> but like we'll just have to kind of see.
1:05:16And yeah, Eric has this great thing.
1:05:18like the browser wars because the thing
1:05:20is like in the past the companies
1:05:22believe owning the browser was worth it
1:05:23and it turns out like it was right. If
1:05:25you own the browser you can make a lot
1:05:26of money
1:05:27>> by doing that. Um and owning mobile was
1:05:30worth a lot of money. I think a lot of
1:05:33people are betting that owning the
1:05:34harness is going to be worth a lot of
1:05:36money.
1:05:37You want to see something one last thing
1:05:39that's kind of interesting before uh we
1:05:41close it out for the day.
1:05:42>> You talked about a question from Eric
1:05:44real quick and then yes which is is
1:05:46human layer composable decomposible? So,
1:05:48so human layer owns the uh orchestration
1:05:50layer and it can also we have a harness
1:05:53uh but it's swappable. You can use cloud
1:05:54code, you can use our harness, you can
1:05:56use codecs. Um we kind of have noped out
1:05:59of the like we'll eventually have
1:06:00managed cloud sandboxes and stuff but
1:06:02our kind of take is like if you're
1:06:03serious and like the customers that we
1:06:05work with all of them would prefer a
1:06:08good interface on top of compute and dev
1:06:10environment that they own basically.
1:06:12>> Yeah. Exactly. Yeah. It's just way
1:06:14easier. like I don't want to deal with
1:06:15setting it up on someone else's
1:06:16infrastructure.
1:06:17>> You got one more thing.
1:06:18>> I I'll show you extendable software. I
1:06:20was going to make this a whole podcast.
1:06:21So if it's interesting, let us know and
1:06:22we can make this a whole thing. Um I've
1:06:25been playing around with this concept a
1:06:26lot.
1:06:27>> Um so when I think about extendable
1:06:29software, I think the best way to extend
1:06:31software is let agents write code.
1:06:33>> It's called code mode. You've probably
1:06:34seen it. So in this case, here's like a
1:06:36string that an agent wrote. It does some
1:06:38code. It does some stuff. And I'm
1:06:40basically just going to start
1:06:42mcking with this string in really
1:06:44interesting ways. So I'm going to pass
1:06:46in some code that I'm additional code
1:06:48that I'm going to dynamically generate.
1:06:49So like if I do like name two uppercase
1:06:54just run this really fast
1:06:57and I'll run uh dexter. So when I run
1:07:00this I have compilation errors. U
1:07:07let me just do one grap instead.
1:07:12H we changed our API to be only reflect.
1:07:15So I had to replace all these
1:07:20when I run this code.
1:07:23Oh, never mind. I got nothing good to
1:07:25show. I apologize.
1:07:26>> Uh this is broken for whatever reason.
1:07:28Uh but anyway, we'll do an episode on
1:07:31this
1:07:31>> about how to run like dynamic code
1:07:33generation, how to actually have your
1:07:34app be extendable from a really extend
1:07:36from a really interesting perspective.
1:07:37We I just upgraded BAML version so I
1:07:39can't show this yet I guess. uh I should
1:07:41stop running local dev is what I've
1:07:43really learned. But um I think
1:07:45extendable code is definitely going to
1:07:47be one of the most interesting features
1:07:48out uh out ever because like like for
1:07:51example you in human layer how great
1:07:53would it be if people could define their
1:07:54own columns and their own like you could
1:07:57be like hey here's a small little code
1:07:59workflow that I want to run
1:08:01>> for doing this and like they can just go
1:08:02say like oh here's an agent loop that I
1:08:04want to add on because that's kind of
1:08:05what cloud automations is effectively
1:08:10but just done kind of shittily like a
1:08:13cron job a chron Crown job is basically
1:08:14like code that says while true at this
1:08:17time run this piece of code. That's like
1:08:19the ultimate crown job.
1:08:21>> Yeah. In a scalable, you know, fault
1:08:23tolerant way where like if the server
1:08:25goes down at that time, it still runs a
1:08:27minute later on another server.
1:08:28>> Yeah. Yeah. So that's the hard part
1:08:30about the execution. But the extendable
1:08:32part is what I really want because if I
1:08:34can only write the workflows that you
1:08:35let me write,
1:08:36>> I'm kind of sad. But if I can write the
1:08:38workflows I can write on your stack and
1:08:41I can buy the reliability from you, then
1:08:43buying the reliability is totally worth
1:08:45paying for,
1:08:46>> right?
1:08:47>> I mean, I think it comes into like this
1:08:49idea. I mean, in in our own code bases,
1:08:51I know many people have talked about
1:08:53they have like viable zones and they
1:08:56have no vibe zones, right? They have the
1:08:58the functional core and then they have
1:09:00an interface and things like pragmatic
1:09:02code can sit on top of it. And there's
1:09:04no reason not to basically create space
1:09:07for users to deploy their cut like
1:09:10instead of user files a bug and if it's
1:09:12in the no slop zone our factory builds
1:09:13it chips a feature goes in the next
1:09:15release. It could literally be like in
1:09:17real time it's just like cool these
1:09:19parts of this have you know sandbox
1:09:21execution adapters where your agent
1:09:23writes code and that modifies your app
1:09:25in real time. But getting the interfaces
1:09:27right and figuring out okay what is the
1:09:29core versus what is the code and what
1:09:30are the interfaces that are available in
1:09:33that code is is uh it's an interesting
1:09:35space that yeah I think I think we've
1:09:37both been exploring a little bit uh and
1:09:38excited to I don't know I I don't have
1:09:40any like cool like demos of that yet but
1:09:42I think Ree from Exeutor actually built
1:09:44something like this where like the way
1:09:45you customize and configure exeutor is
1:09:47just by dropping TypeScript files in
1:09:49your directory. Yeah. I I think there's
1:09:51like no other way around this is like
1:09:52long term what you want to do is you
1:09:54just want to be like here's my code and
1:09:56every piece of software effectively
1:09:58turns into like a harness.
1:10:00>> Yeah.
1:10:01>> And open code like the whole point of
1:10:03those things is like hey you want to
1:10:04customize the behavior. Cool. Have your
1:10:06have your agent write code and it live
1:10:08reloads into the agent and now you have
1:10:09new extensions.
1:10:11>> Yeah. Exactly. Because like nothing else
1:10:12I think is like that's like the most
1:10:15interesting stuff in my in my eyes about
1:10:17like how people build apps. It's like
1:10:19the like React built like the uh
1:10:21interactive web and I think we're about
1:10:23to have like just like extendable
1:10:24software as like a common place like VS
1:10:27Code amazing because you can add
1:10:28extensions. Tails Tailwind comes out you
1:10:31can have it work in VS Code without VS
1:10:33Code updating
1:10:34>> and that's like magic.
1:10:36>> Um cool we're way over time.
1:10:38>> Oh [ __ ] we are. It was a really fun
1:10:40conversation today. I love today's
1:10:41conversation. It's been so long since
1:10:42we've had one of these. Yeah, had a good
1:10:44time.

About this transcript

This page contains the full transcript of How to Build a Software Factory for AI Coding Agents by Boundary, generated from the public captions YouTube serves with the video. The transcript has 15,403 words across 2,214 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.