YT YouTube2Text

YouTube2Text → Transcripts → Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev)

Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev) — Transcript

by IndyDevDan · 8,086 words · 1,167 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00What's up engineers? Indy Devdan here.
0:02It's time to focus. This is going to be
0:04a big video and a lot of engineers are
0:06going to miss the leverage I'm going to
0:07share in this one. I don't want you to
0:09be one of them. Your software factory
0:10can unlock an unprecedented level of
0:13engineering results. But there's a
0:15massive roadblock every agentic engineer
0:17runs into at some point. A roadblock you
0:20might have. Where should your software
0:22factory run? Where do your agents plus
0:25code actually run? Most engineers
0:27allocate a tiny corner of their own
0:30computer for their agents, or they lean
0:32too heavily on CI/CD or some container.
0:35The best engineers aren't doing this.
0:37Engineers at big AI labs running the
0:39most insane workloads you can imagine do
0:42something else because they've realized
0:44one simple fact. The more autonomous and
0:47secure your agents are, the more they
0:49can do for you riskfree. On this
0:52channel, across hundreds of agentic
0:54engineering videos, you've heard me say
0:56scale your compute to scale your impact.
1:00Compute applies equally to GPU compute
1:02and CPU compute. Old school bare metal.
1:06An agent sandbox is the best space to
1:09place your agents. [music] Why is that?
1:11Why can't we just use a container and
1:13call it solved? It's because agent
1:15sandboxes give you three key advantages:
1:18true isolation, insane scale, and
1:20agency. Here's the raw reality of the
1:23state of agentic engineering. If you are
1:26inside the loop, you are the bottleneck.
1:28In this video, I'm going to break down
1:30exactly how you can step outside the
1:32loop by putting your software factory
1:35inside an agent sandbox. Let's put your
1:38[music] factory in a box.
1:45Here's the tech stack we're going to be
1:46using to put our factory inside a box.
1:49Don't focus on the tool. Focus on the
1:51purpose and choose the best tool you
1:53need. Here's the architecture of the
1:55system as well. We'll throw this up on
1:56the screen. And why did I put the second
1:58orchestrator agent inside the sandbox?
2:01It'll all make sense as we progress.
2:03Let's start with a powerful workflow you
2:05can't do on containers on your device.
2:08Best of N.
2:13We'll boot up Herder. This is my
2:15terminal multiplexer of choice for
2:17watching agents, kicking off jobs,
2:19running arbitrary commands, application
2:21directory. And we're going to use claw
2:23code fable as our tople orchestrator
2:27inside of our software factory. We're
2:28going to use pi as our agent SDK. Don't
2:30just limit yourself to one agentic
2:32coding tool. The tools you use directly
2:34limit what you believe is possible.
2:36Let's go ahead and kick off the skill
2:37that activates our agents understanding.
2:40This is going to be SSF, agent, sandbox,
2:42orchestrator. I'll break this all down
2:43in a second. It understands the skill.
2:45It's got all the context primed, ready
2:47to go. And it can see that we have one
2:49VM running. We're not going to touch
2:51this one. We're going to open up brand
2:52new VMs. We're going to run the best of
2:54end pattern. We're putting our software
2:56factories into an agent sandbox. Let me
2:58show you exactly what that looks like.
3:00I'm going to open up VS Code. I'll grab
3:01my demo prompt here. Copy this. Feel
3:04free to pause. We're going to break this
3:05down in a second. I'm going to paste
3:06this in and fire it off. This top level
3:09orchestrator agent that operates outside
3:11of all of our sandboxes is going to
3:13start orchestrating this workflow. If
3:15you were here for last week's video
3:16where we talked about the super simple
3:18software factory, you're going to love
3:20what you're going to see next. So, we're
3:21running five different agent
3:22configurations. Default, Frontier,
3:24Deepest, Seek, Open Weights, and Top
3:27Speed, each with its own sandbox. And
3:28they're all going to solve the same
3:31problem. But we're not just launching
3:32agents. We're running an AI developer
3:34workflow. We're running the full
3:36software developer life cycle against
3:38our simple application which you're
3:40going to see here in a second. But you
3:42can see here our orchestrator is kicking
3:43off not just agents inside sandboxes.
3:46Our agent is building our software
3:47factory and placing the entire factory
3:50inside our agent sandbox. And this
3:52unlocks a lot of really really powerful
3:54things. When you use agent sandboxes,
3:56you're getting isolation, you're getting
3:57scale, and you're getting autonomy.
3:59Every single agent owns the computer.
4:02They're not just running on a little
4:03corner of yours interfering with your
4:05work, you interfering with their work.
4:07They have everything they need to own
4:09the outcome. We are using exe.dev as my
4:12primary sandbox tool. One of the best,
4:15simplest ways to get started with agent
4:17sandboxes. I'll of course link all the
4:19resources we're going to talk about
4:20throughout this video in the description
4:21for you, but their tagline keeps it nice
4:23and simple for you. Computers for
4:25developers and agents. Durable
4:27sandboxes, fast, secure, and sharable.
4:29The agent sandbox is a developer device
4:31for your agents. Let's just like really
4:33atomize everything. Let's say exactly
4:35what things are. There's a lot of hype
4:36and noise in the AI industry. I try to
4:38dehype things as much as possible as
4:41complex as something like this is. We're
4:43not just throwing a expensive claw opus
4:46cloud fable Asian at this. We are
4:47understanding what model needs to run
4:50where with what code surrounding it.
4:52Right? That's the software factory.
4:54That's the next step. So this is a lot
4:56more powerful than just a skill and an
4:59agent and spin up this web app inside of
5:01this service. Orchestrating other
5:03agents, orchestrating sandboxes does
5:05require state-of-the-art intelligence. I
5:08always track these models inside of my
5:10model stack here. You've probably seen
5:11this if you've been with the channel. I
5:13delineate between three tiers of models.
5:15We have state-of-the-art, we have
5:17workhorse, and we have our lightweight
5:19models, which are basically the floor.
5:20These lightweight models are defined by
5:22their ability to be run by previous
5:24generation GPU node or directly on your
5:26Mac device if you have the unified
5:28memory for it. And then we have our
5:29workh horses. These are all of the
5:31powerful models that aren't quite
5:33state-of-the-art. The newest entry which
5:34you're going to see here is the brand
5:36new Deep Seek V4 Flash0731.
5:39This model is absurd for its price.
5:41You're going to see a model
5:42configuration that we're going to kick
5:44off that I call Deepest Seek, which is
5:46just V4 Flash models. We're going to see
5:48how they compare to a bunch of other
5:50model plus harness configurations inside
5:52of our software factory. If you're still
5:54fixated on models, you are behind.
5:55That's not the name of the game anymore.
5:57We're entering an age of abundance of
5:59compute. And now it's about focusing on
6:01what model plus what code do you need to
6:03do the job. And the software factory is
6:06that system. So our agent is finished.
6:08You can see we have the application
6:09layer and the agentic layer. Let's go
6:11ahead and jump into them. Open these all
6:13in Chrome. Let's see what's going on in
6:15our system. We're going to open them all
6:17up in Chrome. And as you can see, we're
6:19working on Inkwell. This is a simple
6:21mock application. This is not the
6:23important piece. We're using this as a
6:24demo. What's important here is our
6:26software factories. So, what you're
6:28going to see here is 1 2 3 4 five unique
6:33software factories running in their own
6:35agent sandboxes. So, inside of that
6:37prompt, this user interface is like not
6:39clear. There's too many buttons. Things
6:40are too out in the open. And we want a
6:42more unified experience for our Inkwell
6:45writing application. Instead of just
6:46running this once on one agent or two
6:49agents, I'm running them in software
6:51factories. We're running the simple
6:52software developer life cycle. Again, we
6:54covered this last week. The super simple
6:55software factory is a key piece of this,
6:57but it's not our key focus. We're using
6:59this as our observability system into
7:01our sandbox. We have discrete URLs for
7:04every sandbox. This is our default Asian
7:06config. This is our front tier Asian
7:08config. This is our deepest seek Asian
7:11config. We then have our open weights
7:13configuration and then our top speed.
7:15And as you can see here, we've opened up
7:17two applications for each set. The
7:20software factory and the actual
7:21application. So you can see we have five
7:23unique versions. These are each inside
7:25an agent sandbox. If we hop back over to
7:27exe.dev, do a refresh, you can actually
7:29see all of our sandboxes up plus the one
7:32I had up earlier. Okay, so we have five
7:34new redesigned agent sandboxes. So
7:37again, like really important concept.
7:38I'm not giving my agents a corner of my
7:40computer. I'm giving them an entire
7:42computer to do all the work they need to
7:44with isolation, with scale, and
7:46autonomy. Let's click into our software
7:48factories. You can kind of see our model
7:50breakdown here on the left. And we can
7:51do this for every one of our versions,
7:53right? Let's go ahead and click into
7:54everything. Now, we can get an
7:55understanding of the models that are
7:57running, the agent harnesses that each
7:59model is running, the tools available. I
8:01want to keep uping the discussion here.
8:03It's not just about a single agent
8:05anymore. One agent is not enough. just
8:07like one prompt was not enough. We then
8:09started running sub agents. We then
8:11started chaining agents. Maybe you put
8:13together your own custom agent harness
8:15thanks to great tools like the PI agent
8:16harness. Fantastic. At some level you
8:18realize that agentic engineering is
8:21software engineering. We have a new
8:23primitive which is the agent autonomous
8:25software and we need to put them inside
8:28of our existing developer workflows that
8:30we ourselves would run. Hence the term
8:32AI developer workflow. We're adding AI
8:34to the work you and I used to do as
8:36engineers. every single day. So, we have
8:38this new tool. We need a new frame of
8:40mind to understand how we can really use
8:42this in the best way. And spoilers, you
8:44know, the whole point of me creating
8:45this video and sharing it with you here
8:46is that when your agents have their own
8:48developer device just like you do, guess
8:51what they can do? They can perform and
8:52act and even outperform you. That's the
8:55whole goal here. A lot of engineers are
8:56afraid, you know, of these great models
8:58progressing further and further and
8:59further. I am not. This is a new tool in
9:01your toolbox. Don't run away from the
9:03fire. Run into the fire. These agents
9:05can do incredible work on your behalf
9:07for you, for your business, for your
9:08career, for your users, but you have to
9:10leverage it. So, inside of every single
9:12sandbox, we have placed a set of agent
9:14configurations and we built an AI
9:16developer workflow that walks through
9:18the software developer life cycle and
9:19our agents are shipping work on our
9:21behalf. Here's a key word without us.
9:23The key idea that I really want to
9:25communicate here is that if you are in
9:26the loop, you are always the bottleneck.
9:29Now, there are exceptions to this. If
9:30you're doing hands-on agentic coding
9:32work that requires you, that requires
9:34your expertise and you know a great
9:35exception to this is like if you are
9:37building the system that builds the
9:39system, then of course get in the loop,
9:40do the work, go hands-on, prompt back
9:42and forth, babysit your agent, you need
9:44to do that at some level. But there's a
9:45lot of work you might be doing. I know
9:47for a fact teams are doing and god
9:49forbid we talk about enterprises like
9:51there's a ton of work all these sets of
9:53engineers are doing that they don't need
9:55to be. Why not? because they can have a
9:57team of agents do it for them. And not
10:00just a team of agents, a team of agents
10:02plus code plus engineers. This is our
10:05default agent set. This is our frontier
10:07agent set. This is our deepest seek
10:09agent set. Spoiler, everything's
10:11deepseek v4 flash.
10:14We're saving an incredible amount of
10:16cache of compute. Then we have our open
10:18weight. So pure openweight
10:20implementation, no closed labs here. At
10:22the end, by the way, we're going to
10:23compare all the changes made to Inkwell.
10:25in each one of these versions. Back to
10:27the source. That's why this is called
10:28best of end. Say you have a great idea.
10:30Say you have multiple directions you
10:32want to go or even one single concrete
10:35direction. In our example here, we just
10:36pass in this one singular prompt. And
10:38this is actually running our redesign
10:40quiet room. We want a quiet room as our
10:43design principle for this application.
10:45The current version is cluttered. We
10:46want it to be simple and clear. So this
10:49is the prompt we pass into every agent
10:50sandbox and then into the software
10:52factory inside each agent sandbox. As
10:55you can see, I'm not just running one
10:56agent. I'm running a pipeline of agents
10:58plus code. And of course, inside the
11:00incoming task, we can dial into all the
11:03results here. You know, this is a
11:04classic agent observability system.
11:06We've talked about these on the channel
11:07in the past. You know, you can see the
11:08request here. Redesign Inkwell. But
11:10then, you know, we can look at the plan
11:12workflow. Of course, we can see all the
11:13thinking, all the agent calls, the gate
11:15passes. We have deterministic gate
11:17checks on our nondeterministic systems.
11:19JSON error, it fixes JSON error, so on
11:22and so forth. Again, we talked about
11:23this last week, so check out that video
11:25if you want to dig in depth into our
11:27super simple software factory that I
11:28have fully open sourced. Oh, nice. GLM
11:305.2 review has kicked off here. Love to
11:32see that. Pure open weights. In our
11:34super simple software factory, we are
11:36not just doing lightweight prompts. We
11:38are fully in control of the system. What
11:40do I mean by that? We are selecting a
11:42specific coding agent. Of course, the
11:43model, of course, the thinking level, of
11:45course, the tools. Oh, what are these
11:47tools? These tools came from our custom
11:49agent harness. Yes, we are harness
11:52engineering inside of our software
11:53factory. If you don't have custom
11:54harnesses, you can customize and
11:56control. You don't have a software
11:58factory. The software factory is your
12:00full control over the agents and your
12:02code. We have a nice simple purpose
12:04breaking down what this phase is, which
12:06is just the plan phase of this workflow.
12:09And then we have a nice description. We
12:10have our prompt engineering. Okay, so
12:12what is prompt engineering? It's both
12:14your system prompt and it's your user
12:16prompt. Okay, so on and so forth. I'm
12:17not going to download this again. Check
12:18out last week's video where we really
12:20dive in to the super simple software
12:22factory. If you've been with the
12:23channel, you're going to notice we're
12:24going to start compounding some really
12:26heavy-hitting ideas. We're going to lose
12:27a lot of engineers. A lot of engineers
12:29are going to look at this and just
12:30think, I don't need this. I don't need
12:31to build this. Insert whatever excuse or
12:33lack of focus or whatever. I don't
12:35really know. I don't want to judge
12:36anyone, but engineers that can focus
12:38right now. This is the time to focus
12:39because the next leap is available. It's
12:41your software factory. And then you can
12:43really leverage and scale your software
12:45factory by putting your factory in a
12:47box. So that's all we're doing here. At
12:49a high level architecturally, it's
12:51relatively simple. The implementation
12:52details do make things a little more
12:54complex, but you can see here our
12:55default run is running, you know, gem
12:57flash, deep flash, GLM52, and GBT Luna.
13:02Just a lot of great compute. If you're
13:04constantly going back and forth on what
13:05model you're using, you're missing the
13:07point. You need a model stack, not a
13:09single model. Combine compute. Don't
13:11select compute. Let's go and look. Of
13:12course, our Opus 5 going crazy. 13
13:15minutes on the plan phase to be
13:17expected. I'm sure this is in high
13:19thinking. If we select this, look at our
13:20Asian config. Yeah, of course we're in
13:22high thinking. Maybe should have dropped
13:23that down for this demo. Who cares? Our
13:25deepest config. [laughter]
13:27All pure DeepSk implementation, saving a
13:31ton of cash, using a ton of
13:33tokens. Love to see this. Again, we're
13:35entering an age of abundant compute. And
13:37you know, by the way, we're going to
13:38dial into the system a little bit more
13:40here. You can see here our top level
13:42orchestrator agent has stopped. It's not
13:44doing anything. All it did was kick off
13:46the orchestrators inside our sandbox. So
13:49again, we have this three- tier
13:50architecture and there's a reason for
13:51that. I'll explain that in a moment. We
13:52have this three-tier architecture where
13:54we have an outloop orchestrator. We have
13:56an in sandbox orchestrator and then we
13:58have our software factory. Looks like we
14:00have a full fail here. One of our jobs,
14:02open weights, completely failed. ChemK3
14:04bombed. You can see here we have a nice
14:06detail of why this happened. That's
14:08okay. This is another reason why best
14:09event is so important. It looks like it
14:11couldn't get the JSON format out. We
14:12could dial into the results. I'm not
14:14going to waste your time with this. This
14:15is why we use best event. This is why we
14:17scaler compute to scaler impact. And
14:19what do we have here? This is our top
14:21speed. Um, of course, top speed is
14:23already done. 14 minutes for this full
14:24run. 2 million toes. And look at all of
14:26our fast models. If you just care about
14:28speed, you want to be looking at Gemini
14:293.6 Flash, DS4 Vlash, 0731, and of
14:33course the incredible GBT 5.6 Luna
14:35model. We can start looking at our
14:37Inkwell applications. Then we'll dive
14:38into more of the like heavy technical
14:40side of it. Okay, I just want to like
14:41really connect with you on the core idea
14:44here. Five instances. Looks like one of
14:46them failed. Let's go ahead and just
14:47like start refreshing. So here's the v1.
14:49Let's refresh. There is our updated
14:51version. So this is just one sandbox. We
14:53have our entire app and our Asian
14:56factory and all of our Asian configs and
14:58an orchestrator inside this one box. Our
15:01agent has a computer. Our agent can do
15:03more. It can solve more problems. It is
15:04fully autonomous. And guess what? The
15:06blast radius is zero. The blast radius
15:08is the box. Very, very important stuff.
15:10And API keys are completely solved here.
15:13You know, for all of our PI coding
15:14agents running on the software factory
15:16level, they're using an open router
15:18provisioned key. Really important
15:20concept. It's all going to be here
15:21linked in the description for you. But
15:22let's keep looking at our versions. So,
15:23our default agents put this together.
15:25Looks really, really good. We have the
15:26title. We can start writing. We feel a
15:28lot more focused than our V1. So, here's
15:31that V1. We have hotkeys all over the
15:33place. It's not clear. I don't feel
15:35focused. We have the sidebar open. I'm
15:36distracted. But now in this new clear
15:38version, I am focused at the same time.
15:41We didn't lose our sidebar. We can still
15:43do all of our editing and we can do our
15:44delete publish. So our actions are
15:46there. Total words. Let's go over to our
15:48frontier version and let's actually
15:50check on this because it might not even
15:52be done. Okay. Yeah. So Cammy K3 and our
15:54Frontier version is getting to work.
15:56This is not complete yet. So refreshing
15:58this won't do anything. So let's move
15:59on. Deepest seek. [laughter]
16:02Let's see how our deepest seek is doing.
16:03Very very nice. Uh let's see if we have
16:05our sidebar. Good. We don't want to lose
16:07anything. Menu at the top right. Very
16:09very good. Love the way that looks. We
16:10can scale up, scale down. Three words
16:12saved. We are just writing we feel
16:14focused. Shout out to the Deep Seek
16:16engineers. What an incredible model and
16:18what insane prices. When we look at
16:20artificial analysis, we go to flash.
16:22Guess who's at the very bottom of the
16:25cost per task benchmark. And guess who's
16:28at a decent amount of speed, right? 110.
16:31Okay, very good. And guess who's in that
16:32workhorse tier? It's Deepseek V4 Flash.
16:35Now, I'm super stoked. I'm waiting for
16:37two new sets of models to come out. I'm
16:39waiting for aside from all the
16:40state-of-the-art stuff. You know, I
16:41always want Anthropic OpenAI to really
16:43push out the state of the art, but aside
16:45from that, I'm really paying attention
16:46to two models. And comment down below if
16:48you are as well. I'm looking for that
16:50Deep Seek V4 Pro. Super curious where
16:52that's going to land, if we're going to
16:54get into that state-of-the-art territory
16:55or if that model is just going to be a
16:56super economical uh Pro that has
16:59improved benchmarks. And I'm also
17:01waiting for Quinn 3.830 billion or 50
17:05billion or 12 billion. That like 30
17:07billion parameter model that they've
17:09been putting out has been incredible.
17:10Really really excited for that to run on
17:12device but also put that on a B300 and
17:15that thing is going to run ultra ultra
17:17fast or some other sets of nodes. My
17:19point is there is that I'm super super
17:20excited for that compute to be available
17:23because again I'm looking at all these
17:25levels. We're constantly trading off
17:27three things. Performance, cost, speed.
17:30Keep your eye on the ball, but also
17:32always move where the ball is going,
17:34move where the models are going, and
17:35have a plethora of options you can
17:37deploy into your software factory. So,
17:39back to our factory. This looks good.
17:40Deepest seek, incredible model,
17:42incredible economics. Like, frankly,
17:44stock market crashing economics. If you
17:46really start using and really start
17:47understanding this model, it's insane
17:49what you can do with this. This is a
17:50great builder. It's a great worker
17:52model. Again, that's why I named these
17:54workhorse. It's for your daily 90% of
17:56your work. Previous Deep Seek Flash was
17:58in that B tier. This is clearly in the A
18:00tier. And A tier is like roughly just to
18:02have something to benchmark against. The
18:04A tier kind of is in this zone here,
18:06right? It's in that 50 plus level in the
18:09artificial analysis intelligence index.
18:11And then of course, your
18:11state-of-the-arts are like up here. It's
18:14more like this, right? This cut off here
18:15is what I consider state-of-the-art.
18:16It's like that 56 plus level. But again,
18:19that's just rough benchmarking. This is
18:21an index and this thing changes. So, you
18:23have to be dynamic with comparing your
18:25model stack with public benchmarks.
18:27Anyway, let's see what else we have
18:28here. deepest seek has fully completed.
18:30And by the way, once again, this is not
18:31just an agent. This is not sub agent
18:33delegation. This is not random cloud
18:35code/workflows
18:36execution where you have really no idea
18:38what workflow it shows or why. We have
18:40built a software developer life cycle
18:42that we have fine-tuned. We've templated
18:44our engineering into the software
18:46development life cycle. We've added AI
18:48to it and we wrapped it in a software
18:49factory. And this gives us full control
18:52over our engineering, over our products.
18:54Be careful how much of your engineering
18:57work you're handing off to these AI
18:58labs. You're handing off to third-party
19:00resources. You're handing off to these
19:02all auxiliary tools. Don't outsource
19:04your thinking. Stay close to the
19:06engineering. This is aic engineering,
19:08not vibe coding. Okay? And I say it all
19:10the time. I'm going to keep saying it.
19:12Vibe coding is not knowing what your
19:13system does and not looking. Aentic
19:16engineering is knowing what your system
19:18does so well you don't have to look. Big
19:22difference. like that. It's like a
19:24really fine line. If you look at a vibe
19:26coder and an agentic engineer, it might
19:27look like they're doing the same thing.
19:29If you spend a little more time, if you
19:30really think about what's happening when
19:32an agentic engineer sits down to do
19:34work, they're thinking in systems.
19:36They're thinking about the hundth and
19:38thousandth run of the system. They're
19:40thinking in observability, reusability,
19:42isolation, scale. So, I hope that makes
19:44sense. Again, a lot of engineers, I'm
19:46going to lose a lot of my audience here.
19:48I know it for a fact. We're at the point
19:49in the age of agents where you have to
19:51start working for it. So, here we go.
19:52Let's see what top speed did. Open
19:54weights bombed. Okay, we just go ahead
19:56and close that. And then we had our top
19:58speed. How does top speed look? Pretty
20:00good. Still a little bit noise, although
20:01you know, nice hot keys in the footer.
20:03We have shortcuts at question mark.
20:05Okay, we like to see that. And then we
20:06have our nice writing view. And that
20:09looks good. We do like that confirmation
20:11of saving. And we can scale this up and
20:13down. We have some controls here.
20:15Overall, really, really good. These
20:17workhorse models, Gemini 3.6 Flash, V4,
20:20Luna. These fast, relatively cheap
20:22models. They can do a lot of the work
20:23you're probably doing right now. That is
20:26our changes to our live application
20:28here. Of course, this is just a mock
20:29application. I can't share production
20:31code with you, but um you know, I can
20:33share open source stuff. I can create uh
20:35demos, mocks for you. And you know, the
20:37the super simple software factory is the
20:39same thing. It is a template for how I'm
20:42pulling it in to my products, to my
20:44client work, to my production level
20:46workloads, then doing a lot of
20:47fine-tuning and tweaking. Remember,
20:48specialization is your product. The
20:51advantage you're going to have is not an
20:52out-of-the-box agent. It's a specialized
20:54system that does something really,
20:55really well at the best economics,
20:57right? The best tokconomics. And again,
20:59your software factory inside your agent
21:01sandbox lets you scale that. It lets you
21:03move faster, lets you get out the loop.
21:06Best event, fuse the results, move on,
21:08hand the work off to your factory. We
21:10really dive into the details of the
21:11super simple software factory in last
21:13week's video. Check that one out. I
21:14haven't looked to see how much engineers
21:15like that one yet, but that's there. A
21:17lot of value in that one and a lot more
21:19value in this one. It's hard to follow,
21:21but let me break down some of the
21:23implementation details for you so that
21:24it's easier to follow. I'm still
21:26observing this stuff. At any point in
21:27time, I can enter these sandboxes and I
21:30can enter the orchestration. So, let me
21:31show you what I mean. In my demo prompt
21:33here, we can get shell access. So, let's
21:35get shell access. I'm going to fire this
21:36off. And I've got a couple really,
21:38really important skills that my
21:40orchestrator agent is working off. If
21:41you open up cloud skills, you can see
21:43here I have four essential skills.
21:45Herder sandbox exe.dev super simple
21:47software factory and then our super
21:48simple software factory orchestrator
21:50agent. So this is a composite skill. I
21:52don't like to compose my skills. It
21:54makes it really hard to change things.
21:55You create these nasty dependency
21:57graphs. But when things are unified in a
21:59single monor repo, you can make an
22:01exception to the rule. Our agent is
22:03going to use herder, my terminal
22:05multiplexer here to kick off this
22:07sandbox redesign fleet. Check this out.
22:08We have five panes. And now my agent is
22:10going to boot up SSH access into every
22:13single one of them. There we go. A key
22:15idea I've been talking about on the
22:16channel over and over is a gentic
22:18access. If you want to move at light
22:20speed, if you want to move the speed of
22:21agents, you need an agent layer wrapping
22:24whatever you're trying to do, right?
22:25It's that meta layer. It's that building
22:27the system that builds the system. You
22:29can see here we're in all of our
22:30sandboxes, right? Here's our redesign
22:32default. Here's the frontier, open
22:33weights, top speed, deepest seek, right?
22:36It's all here. And then we can Oh, yeah.
22:38I don't have any of my hot keys. We can
22:40get status. We can get log to see the
22:42exact change. Document the equal quiet
22:44room redesign. Redesign document blah
22:46blah blah. And we can go from there. So
22:48this is useful. It's always that same
22:50idea in here. I have to do everything
22:52manually. So we can go one step further.
22:54If we open up code and we go to our
22:57third agent access. Let me just show you
22:59what this does. I want the exact same
23:01thing except I want to talk to the
23:03orchestrator inside each running
23:06sandbox. So what's that? It's an
23:07interactive cloud code on top of the
23:09boxes. This is workflow B. Again, I've
23:11taught my agent exactly how to do this.
23:13So, let's see how it does it. We have a
23:15new workspace inside of Herder. Let's go
23:17ahead and crack it open. We have a nice
23:18grid here. There's the boot up command.
23:21We have to trust the system all the
23:23time. I'm sure there's some way to
23:25bypass this. This is one of the annoying
23:27things about cloud code. Yes, I booted
23:29it. Just open it for me. Our agent will
23:30likely take care of this for us though
23:32in each one of the sandboxes. Let's go
23:33ahead and see how it does here. And
23:35there we go. Great. We have a policy and
23:38we're getting that dangerous mode
23:40warning. We are operating inside of a
23:42sandbox. So, we're doing exactly what
23:44the anthropic engineers uh want us to
23:46do. And there we go. So, notice here
23:48we've connected to an agent that's
23:50already ran. This is the three- tier
23:52orchestration system. We can run
23:53whatever we want to, right? Summarize
23:55changes made bullets. Copy that. Fire.
23:58Move. Fire. Move. Fire. Move. Fire. And
24:01again, I'm doing this by hand. You can
24:02always do this with an agent. My whole
24:04point here is if you don't measure it,
24:05you can't improve. But if you can't
24:06touch your boxes, if you can't jump in,
24:08you can't actually get anything done.
24:10Our agents are kicking off. They're
24:11reading the ADW state, the actual agent
24:15work that was completed. Again, check
24:16out last week's video. There's a whole
24:18system around this. Our agents are
24:19summarizing what exactly was done in
24:21each one of these sandboxes. Let me show
24:22you what's going on underneath the hood.
24:24This is going to be fully available to
24:26you. Link in the description. Although I
24:28do a lot of agentic coding for this
24:30stuff, it still takes me a lot of time
24:31to really build this out, right? and to
24:33really make it reusable for you. If you
24:36made it this far in the video, if you're
24:37getting value out of this, you know,
24:38like, subscribe, comment, let the
24:40algorithm know you're interested and let
24:41me know you're interested so I can keep
24:42delivering this type of content. You
24:43know, if this video doesn't perform well
24:45cuz the ideas are too advanced, it's
24:47fine. Like, I'll just dial it back and
24:48go to safer, more hype, you know, based
24:51stuff. But I think you and other top
24:53engineers are interested in this. You
24:55want to be on the frontier. You want to
24:56do as much as you possibly can do with
24:58agents. And also like, you know, I might
25:01sound like a super red pill engineer
25:03like just trying to maximize
25:05productivity, but I think a lot of the
25:06value in this technology is that you can
25:08just work faster, get everything you
25:10want done, and then chill, right? Go
25:12back to your life, go back to real life,
25:13do your thing. I should probably make
25:15that clear on the channel. It's not
25:16super important to me what you think of
25:19my my philosophy around work life
25:21balance, but I just want to mention that
25:22a lot of this technology you can use to
25:24speed up your work and then chill or
25:26speed up your work and get a massive
25:28edge in front of your competition. So
25:29anyway, factory in a box. Okay, the
25:31whole idea here is that you and I are
25:33the bottleneck. We want to be building
25:34systems that prevent that now. So what
25:36we can do, we can take two great pieces
25:38of technology. These are compositions
25:41that are emerging. We can take
25:42sandboxes, throwaway sandboxes that are
25:44also permanent if you want to keep them
25:46up. This is why exe.dev is so important.
25:48A lot of other sandbox providers give
25:49you 24 hours. They give you a couple
25:51days. Is a VM you own, and you can spin
25:54them up and down as much as you want.
25:55It's nice and secure. We have SSH
25:57access, but then inside of it, we have
25:59this factory. We have our Asian factory.
26:01Again, check out last week's video, but
26:03a lot of that again is going to be super
26:04familiar for tactical agent coding
26:06members. We have this ADW's directory.
26:09It's not just about agents. It's about
26:10agents plus code. And then of course we
26:12have the actual application. So here we
26:14have what are effectively two agentic
26:17layers. Really it's just one. And then
26:18we have our application layer. The whole
26:20idea is that you want to get yourself
26:23out the loop. If you are letting your
26:25agents run on your computer, even inside
26:27of a container, that's only getting you
26:28isolation. It's not giving you scale and
26:30autonomy. In a sandbox, you get to leave
26:33the loop. You get to show up when you
26:35want to. As you can see here in Herder,
26:37I can still show up into the box. I can
26:40still jump in, get things done,
26:41understand what's been built inside of
26:43every agent box. But the whole idea here
26:45is I am out this loop. Great agentic
26:48engineering is about showing up at the
26:49beginning and the end, the planning, the
26:51reviewing, the prompting, and the
26:53validating. Uh the more you're in the
26:54loop, the more you are the bottleneck.
26:56Okay, so that's what this system allows
26:58us to do. So we have a couple
26:59interesting things happening here. The
27:00orchestrator agent, as you can see here,
27:02is inside the sandbox. I put an agent in
27:05the sandbox. So we have one operating on
27:07all the sandboxes. On the outside, we
27:09have an agent operating inside the
27:10sandbox on the inside. And this allows
27:12them to run our super simple software
27:14factory system, right? And then we have
27:16the actual agents running from our
27:20pi.dev instance inside of the individual
27:22software factories. And we have a great
27:24visual here. Jesus, you can see that our
27:26state-of-the-art still churning here.
27:28Looks like Kimmy 3 is having a hard
27:30time. But anyway, our software factory
27:32runs any agent configuration. We're
27:33controlling the core 4 on an individual
27:36agent basis. We are scaling. We are
27:37composing. We are creating compositions.
27:40This is what the architecture looks
27:41like. Again, this is the like highle
27:42architecture, right? We have our out
27:44sandbox orchestrator. We have our in
27:46sandbox orchestrator. And then we have
27:47our ADW agents. The ADWs. Once you have
27:50two or more, what you really have is a
27:52software factory. Outbox orchestrator
27:54runs on my machine. Inbox orchestrator
27:56runs on the sandbox virtual machine. And
27:58then we have the factory. Again, still
28:00in the sandbox virtual machine. Couple
28:02key commands are agent ran. I'm not
28:03going to go through all the details.
28:05We'll be here for a while. But you can
28:06imagine there's a setup flow to this. We
28:08need to mount, create the sandbox, set
28:10it up. We can tear it down whenever we
28:12need to. Inside the orchestrator runs
28:14whatever a developer workflow we need.
28:16Usually, it's going to be the full
28:17software developer life cycle or we're
28:18fixing a bug or we're doing a hot fix.
28:20Whatever it is, that's up for you to
28:22decide what's the most valuable chain of
28:24agents plus code your developer workflow
28:27must run. And then we have our actual
28:28ADW agents, which of course there are
28:31atoms, there are scouters or planners or
28:33builders or testers. This is what most
28:35engineers are hyperfixated on and really
28:37it's a key piece but a small piece of a
28:39bigger picture getting actual
28:41engineering work done and being able to
28:43step away from the loop. Here's the
28:45actual setup process our orchestrator
28:47did end to end. So we host and then
28:49inside the box we set up an execute
28:51where we kick off the agent and then of
28:53course we can observe from outside the
28:55loop thanks to our observability tool
28:57and the actual application hosting from
28:59exe.dev my sandboxing tool. And then of
29:01course at the end here which you'll see
29:03in a moment we're just going to tear it
29:04all down. These are ephemeral sandboxes.
29:06We spin them up. We do work. We spin
29:07them down. Environment variable keys is
29:09a big deal right? So in the system this
29:11example codebase system. What I'm doing
29:13here is showcasing that. What I've done
29:15here is used open router's provision key
29:17system. I've limited to 50 bucks. And
29:19then when I tear down all these I'm
29:20going to kill the key. So really
29:22powerful way to spin up intelligence.
29:24Open router is a great tool for this.
29:26And it all creates a result that looks
29:28something like this. again with the best
29:30event system. Our single prompt fired
29:32off in our orchestrator. Orchestrator
29:33spun up in sandbox orchestrators to run
29:36our ADW system. Whatever model
29:38configuration we wanted plus whatever
29:39code it needs and then we can pull the
29:42best result as you saw here. Not all of
29:44our systems finished, but you can
29:45imagine your application built in
29:48multiple different ways. And this is
29:50just one pattern, right? Best event is a
29:52great sandboxing pattern for you to
29:54explore potential futures you want in
29:56your product, in your system, in your
29:57application. And best of n gives you the
29:59best of some set of results. And of
30:01course the great part about that is you
30:03can configure your model selection.
30:05Every one of these runs is running a
30:07different set a different team of
30:09agents. A side effect here is as an
30:11agentic engineer as you're running these
30:13different variants of your software
30:15factory. You're just picking up on data.
30:17How good is this configuration? Is it
30:19getting me the results I need in a
30:21cheaper, more efficient way? Do I need a
30:23state-of-the-art system? By the way,
30:25this is still running. This total
30:26workflow has taken almost 45 minutes. I
30:29mean, I've been filming for 50 minutes
30:30now. This workflow has been running
30:31basically the entire time. There's
30:32clearly a bug here with Kimmy. So, you
30:34know, we'll dismiss this runtime, but
30:36you can see here Opus 5 churning up
30:37tokens, churning up spend, right? Do you
30:39need this? Interesting question. And
30:41then, of course, we have our deepest
30:42seek doing things the cheapest and one
30:44of the fastest. Definitely churning up
30:46some tokens though. Here's read, here's
30:48right, but getting the job done right in
30:49the deepest seek version here, you pay
30:51attention to the title, Deepest Seek,
30:53and we go over here. This version was
30:55good. like I feel a lot more focused.
30:57It's a quiet room for me to write in.
30:59That's just a nice side effect of
31:00running these multiple variants of your
31:02software factory. It's not just about
31:04understanding the difference of the
31:05model. It's about understanding the
31:06stack of models plus code and the
31:08results and the cost of the results. And
31:10ultimately the architecture looks like
31:11this. We have the application. We have
31:13the agent view outbox orchestrator inbox
31:16orchestrator and then our software
31:17factory. If you want you absolutely can
31:20take your outbox orchestrator, go right
31:22to the ADW system, your software factory
31:24system. that's fine. Uh you get a little
31:26bit less control there because the inbox
31:28orchestrator shouldn't really be doing
31:29anything but running and kicking off the
31:31workflows. The key here is just keeping
31:33it simple. Complexity is always going to
31:35come as you add another thing. But in
31:37reality, you know, it looks like this.
31:39We have our software factory inside the
31:40sandbox, public port for the
31:41application, and then a private port.
31:42You have to log into exe.dev. But
31:44basically, you get the software factory
31:45view. This factory in a box system is
31:48going to be available to you. Link in
31:49the description. It's going to build on
31:51a super simple software factory. I'm
31:52going to link that video as well. I
31:54skipped over some of those key deeper
31:56ideas, but as you can see on the
31:58channel, I'm going to be stacking up all
32:00of these big ideas so that you can scale
32:02your computer, to scale your impact. I'm
32:04building and using these systems because
32:05I need them to get to the next level.
32:07And part of my journey here, part of my
32:09goal is to share them with you. Let me
32:11be super clear about this. I come here
32:12every single week as a secondary, almost
32:14a tertiary goal. All of it comes from my
32:17goal to build living software that works
32:19for me while I sleep. And then my second
32:21and third goal is like democratize this
32:24because as we continue to become more
32:26capable agentic engineers as you can see
32:28and feel this is scary powerful
32:31technology especially when you start
32:32using it properly especially when you
32:34start embedding your engineering into it
32:36it's very very powerful there's going to
32:37be a lot of economic value created a lot
32:39of careers changed and I want that to be
32:41dispersed I want that to be democratized
32:43equally or as equal as we can technology
32:46should not be limited to the lucky few
32:47that's why I come here and I share these
32:49ideas every single week with you. Let's
32:51go full circle. Why are software
32:52factories important? Why is this
32:54valuable for your engineering? Software
32:56factories let you scale your compute to
32:58scale your impact by combining
33:00non-deterministic agents with
33:02deterministic code. You want the best of
33:04both worlds. There will be a point in
33:06time when agentic engineering is just
33:08called software engineering because
33:10agents will be part of the fabric of how
33:12we build software as they are becoming
33:14already today. But right now we need to
33:16clearly delineate we are agentic
33:18engineering. This is not really
33:20traditional software engineering yet.
33:21And we're absolutely not vibe coding. I
33:23know what the system is doing. I could
33:25find the exact file doing the exact
33:27thing. Big difference there. That's why
33:28software factories are important. They
33:30let you compose agents plus code into
33:32the workflows that you used to run
33:34yourself as an engineer to get results
33:36done. We're talking plan, build, test,
33:38review, document. We're talking hot fix.
33:41A production issue comes in. Can you
33:43solve it in record time with your ADW?
33:45We're talking about support requests.
33:46We're talking about a bunch of And
33:47your software factory is the mechanism.
33:49It's the system that lets you do that.
33:51That's why that's important. Why are
33:52agent sandboxes important? They're
33:54important for three reasons. Isolation,
33:56scale, autonomy. Isolate your agents so
33:59they don't blow up your production
34:00system. Right? Strip them of their
34:02ability to access AWS and GCP and insert
34:05the next really important tool. They
34:07give you scale. I use one orchestrator
34:09agent to spin up five sandboxes to spin
34:11up factories of agentic work. A bunch of
34:14stuff happened there. Plan build test
34:16review document. and it happened end
34:17times. Best event is just one pattern.
34:19It's going to be a really popular
34:20important pattern because it lets you
34:22see multiple futures. Scale is very
34:25important. Many of us, just as we have
34:27been for a long time, underestimate the
34:29importance of scale. What happens if you
34:32could just do this 10 times? What about
34:34a 100 times? What could you see? What
34:36could you find? What could you build?
34:37What could you learn if you just threw
34:39more scale at your problem? Sandboxes
34:42are scale for your agents plus code.
34:45Sandboxes give you scale. And of course,
34:46they give you autonomy. Don't just limit
34:49your agents to a little corner on your
34:51computer. You can and should do that
34:52when you're working with your agents on
34:54net new work, specifically building
34:56systems that build systems. I'll just
34:58say this, you know, upfront, right in
35:00your face. If you're using an agent to
35:02directly modify the application layer
35:04code, you are wasting time. There are
35:06very few exceptions to that. I hope that
35:08makes sense to you. I'm not trying to be
35:09a douchebag or talk like I'm above you
35:11or anything like that. I'm saying this
35:13so you understand where the leverage is.
35:15I don't want you to fall behind. So that
35:17is autonomy. Sandboxes give you
35:19isolation, scale, and autonomy. When you
35:21put these two things together, you get
35:23something extraordinary. You get absurd
35:25scale inside your sandboxes. When you
35:27put a factory in a box, you get absurd
35:30scale, absurd leverage. You get really
35:32great isolation because you're going to
35:34want it [laughter] as you build more and
35:36more agents on top of your system. And
35:38of course, you can just scale and solve
35:39many, many, many problems at the same
35:41time while not limiting the agency of
35:43your agents. Okay? Ultimately, you and
35:45I, what are we doing here? We're
35:47preparing for an age of abundant
35:49compute. You see how the landscape is
35:51shifting. You can see these new great
35:53cheap workhorse models. You can see the
35:54intelligence getting passed down the
35:56tier levels. It doesn't matter if you
35:58believe it's right or wrong, it's
35:59happening. And so, what do we do as
36:01engineers? We act like a function. We
36:02take the inputs, we execute, we
36:04synthesize, and we deliver outputs.
36:06That's all we're doing here. taking the
36:07information, make the best decision,
36:08act. This implementation of the factory
36:11in a box is going to be available to
36:13you. Link in the description. I also
36:14highly recommend you check out last
36:15week's video. Super simple software
36:17factory. It's going to be a banger one.
36:19You'll notice once again I didn't
36:20mention loop engineering once. I believe
36:22this is the wrong term, the wrong
36:23phrase. What we really should be
36:24focusing on is the software developer
36:26life cycle. If you haven't seen that,
36:27I'll link that in the description as
36:29well. Take the pieces of everything I'm
36:31sharing with you, merge it with your
36:32own. I don't have all the answers here,
36:34but what I do have is a great direction.
36:37Try to get the leverage of the next
36:38phase of agentic engineering. All right,
36:40these are relatively advanced concepts,
36:42but I don't want you to miss out on the
36:43opportunity to scale up. If you made it
36:45to the end, you know, hats off to you.
36:46Things are going to get more complex
36:48from here because all the low-level
36:50leverage has been sucked up. Everyone's
36:52prompting inside Cloud Code now.
36:53Everyone has a terminal agent.
36:54Everyone's using a co-work tool. The
36:56next level of leverage is going to take
36:58work. It's going to take upfront
36:59investment. But as software engineers,
37:01there's a lot you can do that you don't
37:03realize yet. And that's what I'm going
37:04to be here week after week to help you
37:06unlock. You know where to find me
37:07[music] every single week. Stay focused
37:10and keep building.

About this transcript

This page contains the full transcript of Engineers… Your Software Factory NEEDS Agent Sandboxes to SCALE (exe.dev) by IndyDevDan, generated from the public captions YouTube serves with the video. The transcript has 8,086 words across 1,167 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.