YT YouTube2Text

YouTube2Text → Transcripts → Anthropic Rebuilt Git (yes really)

Anthropic Rebuilt Git (yes really) — Transcript

by Theo - t3․gg · 9,843 words · 1,463 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00For as long as I've been writing code,
0:01I've been using Git and GitHub. Funny
0:03enough, I actually started using Git
0:04before I wrote code cuz I used it to
0:06manage plugins for my Minecraft servers
0:08way, way back when I was in like early
0:09high school. Git is core to the way that
0:12I build and work with software, but I
0:14have been starting to notice rough
0:15edges. A lot of the way that Git was
0:17built was around code being expensive
0:20and humans needing to review the code,
0:21approve the code, and control where this
0:23code goes because losing code or merging
0:25the wrong code was a very expensive and
0:27painful thing. The reality of how Git
0:29works has influenced the whole industry
0:31from how we work with our team to how
0:33products like GitHub are built to the
0:35way that we think about things that
0:36should be basic like secrets management.
0:38Shoutout to my viral clip saying that we
0:39should be able to commit our environment
0:41variables. Seriously though, Git is not
0:43the right primitive for the way that
0:45things are moving right now. I'll be
0:47real with y'all. A lot of my problems
0:48with Git are coming from GitHub because
0:51it's just a slow and miserable platform.
0:54External estimates are measuring as much
0:56as 7% downtime across different things
0:59within GitHub. I can't tell you how many
1:01times I've had releases blocked because
1:02actions were failing or I haven't been
1:04able to merge an important change
1:06because the page wasn't loading. The
1:07fact that GitHub's uptime looks more
1:08like a Christmas tree than a green bar
1:11like it's supposed to should say
1:13everything you need to know. But what if
1:15I told you GitHub wasn't the only
1:17problem here? That Git itself is not the
1:20right solution for working heavily in
1:23parallel across a lot of people and even
1:26more agents. I've been thinking about
1:27this a lot because back in December I
1:29explored JJ and I loved it. And ever
1:33since then I've been thinking about what
1:35would it look like to build something
1:37new for shipping really, really fast
1:40with agents. It turns out I'm not the
1:42only one who's been thinking about this.
1:44Anthropic just put up an article about
1:46their new in-house source control
1:49system. Back in May they were merging
1:512,500 changes per day, but now in
1:53September they're doing more than 30,000
1:56changes merged today. They are now
1:58bottlenecked by developer infrastructure
1:59that was built for humans and not for
2:02agents. And they've decided to build
2:04something of their own. It seems kind of
2:06crazy that agents are now smart enough
2:07to do things like this, but they really
2:09are. And I love this idea of challenging
2:12the fundamentals of how you're building
2:14to make the right solution for yourself.
2:16I have a feeling this one's going to be
2:17a really fun deep dive, especially now
2:20that I'm ripping things out like
2:21TypeScript for my own vibe coded
2:23alternatives. That said, I don't think
2:25everything should be ripped out for your
2:27own vibe coded alternatives. There's a
2:29lot of places where it makes sense to
2:30use a vendor product, like for example,
2:32today's sponsor. It kind of feels like
2:34AI has solved most of modern web dev,
2:36but there are some things it hasn't
2:37solved. In particular, mobile. As good
2:39as it is at starting up an app and
2:41making small changes here and there, I
2:42find myself just fighting the model
2:44constantly when I'm trying to actually
2:46get my apps ready to ship, fix real
2:48bugs, deal with Apple reviews, and all
2:49these other pieces. As I ran into more
2:51and more walls as we tried shipping the
2:53apps we were building, I hit up my
2:54friend James at Infinite Red, and he has
2:55made life so much easier. If you're not
2:57familiar with these guys, they're
2:58industry-leading experts in all things
3:00mobile, especially React Native and
3:02Expo, but also Swift and Kotlin as well.
3:04There's a reason that everyone from
3:05Domino's to Microsoft has called these
3:07guys in. They really know how to ship.
3:10I'm going to be real with you guys. The
3:11role of a consultancy has fundamentally
3:13changed as a result of AI. You're not
3:15just hiring these guys to come in and do
3:16grunt code work anymore. You have agents
3:18for that. What you bring them in for is
3:20the expertise, making sure you're not
3:22making simple mistakes, getting your
3:23pipeline set up so you can actually
3:25build and ship the apps, making it
3:26possible for your agents to actually
3:28verify their changes and know things are
3:30working, and just making sure everything
3:32start-to-finish is ready to go in a
3:33world where we're all shipping faster.
3:35These guys can come in and fix anything
3:36from your team to your app. They can
3:38build a whole new app from scratch. They
3:39can take over development, or they can
3:41just give you the pipelines you need to
3:42do it yourselves. If you want to ship
3:44better apps sooner with less bugs, look
3:46no further than swrve.link/infinitered.
3:48Man, I'm actually really excited to dive
3:50into this article. As soon as I saw it
3:51drop, I wanted to read it, but I decided
3:53to put it off for this filming. So,
3:55you're getting my organic take on this.
3:57If you're wondering why the site looks
3:58gross, it's because when I leave it on
4:00the default light mode that the blog is
4:02in, everyone yells at me. So, using the
4:04dark reader extension to make it not
4:06just bright white, and this is the poop
4:08green brown color it chose. Redesigning
4:11source control at Anthropic. As our
4:13merge rate per day grew more than
4:15tenfold, source control became our
4:16biggest development bottleneck. Here, we
4:18share how we migrated to an in-house
4:20version control system in 5 months. I
4:23won't lie, the 5 months feels like one
4:25of those Claude timeline hallucinations
4:27when you ask it how long will this take,
4:29and it's like, "Oh, 2 to 3 days." And
4:30then it finishes the work in an hour.
4:32Regardless, this is impressive.
4:34Replacing Git at a company as big as
4:36Anthropic sounds insane. Last I checked,
4:39they had around 2,000 employees, so
4:42yeah. At Anthropic, coding is no longer
4:44the bottleneck to building systems or
4:46doing research. Claude is able to
4:47autonomously iterate on research loops,
4:49micro-optimize complex programs, and
4:51build a new services from the ground up
4:53at a speed and scale we've never seen
4:54before. The bottlenecks now all of the
4:56developer infra that was built for human
4:58scale. Source control, builds, CI,
5:00things like that. In May of 2026,
5:03Anthropic averaged around 2,500 code
5:05changes merged a day. As of September,
5:07it's over 30,000 a day. We don't expect
5:10this rate of growth to slow anytime
5:12soon. The latent demand for merging code
5:13is practically infinite, and every
5:15bottleneck we remove immediately leads
5:17to further increasing the amount of code
5:19that we produce. The way I framed this
5:21in the past is de-risking the merge
5:23button. Making it so by the time a
5:24person's looking at merge, it's pretty
5:26likely the code is safe to merge. But
5:28also the other side, where if you merge
5:30something you shouldn't have, the blast
5:32radius and the damage that it can do is
5:34reduced massively by systems to make it
5:36easy to test the code once it goes out
5:38with a limited set of users, and revert
5:40the things if they do actually end up
5:42breaking. And I've seen the difference
5:43when we introduce these types of things.
5:45We just ship faster. For example, I
5:48stayed up a bit too late last night
5:49because I was annoyed with some niche
5:51performance issues when you had a really
5:53big database in T3 code. So, I decided
5:55to fix as much of it as I possibly could
5:58in one sitting, as well as clearing out
6:00some old issues and PRs that were
6:02annoying me, as well as taking over the
6:03PRs that weren't quite up to date how I
6:04wanted. And between the hours of like
6:07midnight and I think around 3:00 or 4:00
6:11a.m., I merged from here
6:14through to the top of this page
6:19to here.
6:21Another full page of just Theo merges.
6:23You'll see t3.gg on the left for I think
6:26every single one of these. Some of them
6:27are multiple people, but they are me.
6:31We're still going.
6:34All the way up. Okay, these were two I
6:37merged from others
6:39to here.
6:41That is three full pages of GitHub merge
6:44history that I did in 4 hours. I know it
6:47was 4 hours because I did it between
6:50nightly releases we do every 6 hours
6:51because I wanted to have as much in as
6:53possible for the nightly, so before I
6:55went to bed, I could download it, test
6:56it, make sure all was good, fix anything
6:58that was broken, and revoke the build if
6:59it was bad, and then go to bed smiling
7:02knowing that performance would be way
7:04better for everybody on the nightly when
7:05I woke up. The fact that I can
7:06confidently merge 50 plus PRs in a
7:09sitting, it just feels insane. Like
7:12imagine going even a year back and
7:14telling yourself that you were merging
7:1550 plus PRs a day, hard to fathom. If I
7:18had only heard people saying these
7:19things and I hadn't experienced it
7:20myself, I'd be calling [ __ ] right
7:22now. But this is reality. If you're not
7:24moving this fast yet, you are either in
7:26like a place where you absolutely can't
7:28because people will die, or you just
7:29don't realize how powerful these things
7:31are and you haven't taken the time to
7:32set up the tooling to make the stuff
7:34less risky. So, let's dive in a bit
7:35here. This is interesting. They have
7:38blue and orange. Blue is pull requests,
7:40orange is antique change requests. That
7:44must be their new thing. It lines up
7:46with the way they name [ __ ] Antique
7:48seems like their system for all of this.
7:51And if you think this is Anthropic being
7:52uniquely weird and silly here, there's a
7:54long history of companies reinventing
7:56source control for their specific needs.
7:59For example, Meta built a crazy system
8:02around Mercurial that I think is
8:04actually a hard fork of Mercurial. They
8:06built all of that to make their stacked
8:07diff system as strong as possible for
8:10when people were working on large arcs
8:12and approving pieces of it to merge it
8:14all in at once. Graphite is a whole
8:15company that was built on top of GitHub
8:17to bring the way Meta worked with source
8:20control to people who were on GitHub
8:22because it was so far ahead of how we
8:24were building if we we were shipping
8:25fast. At my company Ping Labs, we were
8:27heavy users of Graphite. Now they're
8:30part of Cursor and they're building an
8:31alternative to GitHub into Cursor with
8:33Origin. The point I'm trying to make
8:34here is that this isn't as unprecedented
8:36as it might seem. And many companies
8:38have found meaningfully better ways to
8:40work with and manage their source
8:41outside of Git. And Jay Gold here, who
8:44used to work at Meta and also works at
8:46Graphite and now Cursor, pointed out
8:47that a lot of the reason Meta built
8:49their systems in themselves instead of
8:51using off-the-shelf things is because
8:53their massive checkouts just didn't work
8:55well with Git. I vaguely remember the
8:57history here of Meta trying to convince
9:00the team working on Git to make it work
9:02better for giant mono repos instead of
9:04doing separate repos for everything. The
9:06Git maintainers didn't want to budge, so
9:08they ended up going their own way. And
9:09that's fine. You shouldn't have to force
9:11the industry standard into your workflow
9:13if it doesn't fit. And it is nice seeing
9:15more companies challenge this cuz Git is
9:17not perfect and it certainly is not
9:19perfect for agents. So let's learn more
9:21about what Anthropic has built. In order
9:22to support their insane growth, they
9:24built Scion, an in-house version control
9:26system, and they fully migrated their
9:28mono repo from Git in 5 months. Ready
9:30for a really silly fun fact? Both
9:32Anthropic and OpenAI are built in mono
9:35repos. So everything they're working on
9:36is different projects and folders inside
9:38of this one giant repo. If you've ever
9:40been curious why the sidebar in the
9:42ChatGPT and Codex app is so bad, it's
9:44cuz of this. The vast majority of
9:46employees at OpenAI have one project,
9:48OpenAI. And they're still using
9:50github.com, by the way. Pretty much all
9:52of OpenAI's code lives in openai/openai,
9:55which 404s if you're not an employee.
9:56So, for those saying I'm secretly
9:58employed, here's your proof.
10:00That makes a lot of the way they think
10:02and work and build strange to people who
10:04have more than one project. So, yeah,
10:06just a silly fun fact I wanted to
10:07include. Back to what Anthropic was
10:09saying here, not only did they build
10:11Scion, which is their in-house version
10:12control system, they also built and
10:14shipped a code review service called
10:15Antique, alongside code search, a rich
10:18source control IAM system, novel ways of
10:20managing and landing large amounts of
10:22reviewed code, and more, all in that
10:24same 5-month period. In this post,
10:27they're going to focus on how they built
10:28Scion specifically. It makes for an
10:30interesting case study. Claude's ability
10:32to write code was both what put them in
10:33the situation in the first place, and
10:35how they're able to build a high-quality
10:37scalable system to deal with it. Growing
10:39our monorepo.
10:41Before we explore what we built, we
10:43should acknowledge how unusual our
10:44workload is. Unsurprisingly, Anthropic
10:47uses Claude to write software far more
10:49than almost any other company. You might
10:51be surprised, guys. Okay, seriously
10:53though, they're
10:54they're definitely doing it more than we
10:56are, although we are doing quite a lot.
10:58But since they're using it so heavily,
10:59they're often the first to see where
11:01tools built for human scale start to
11:02strain. Source control is just one
11:04example of this. While we don't expect
11:06many organizations to have these
11:07problems in the very near term, we
11:08suspect it won't be too long before
11:10others who use AI code tools experience
11:12these same pressure points, which is why
11:14we want to share what we've built and
11:16what we learned. Git is a decentralized
11:18system. Every clone is a complete
11:20self-contained repository that can
11:21compute diffs, read history, and blame,
11:23and check out any revision while fully
11:25offline. For the vast majority of repos,
11:27that is the right design, and it is a
11:29large part part why Git became the
11:30standard. But as their monorepo's grown,
11:32it increasingly doesn't fit that design
11:34space. The history of their main branch
11:36alone is almost 100 gigs, and a
11:38compressed working copy is over 25 gigs.
11:41They perform on the order of 25 million
11:43code checkouts per day for CI, before
11:46counting developer environments, agent
11:48sandboxes, and research jobs. Fetched
11:50natively, that would be 600 petabytes of
11:54Git traffic a day. Git offers tools for
11:56working with a slice of our repository,
11:58like shallow clones, sparse checkouts,
12:00and promiser remotes. Beyond these,
12:02there are standard workarounds for large
12:04monorepos, like internal Git mirrors and
12:05repository snapshots, like seeds, that
12:07are kept in blob storage. They've tried
12:09and used all of them. At their peak,
12:11they operated several hundred internal
12:13Git mirrors to deal with the load. This
12:15bought them time, but it became harder
12:17to operate as merge rates were
12:18increasing. Seeds were falling further
12:20from the tip of main between refreshes,
12:22and it took more and more care to keep
12:23operations that depend on history, like
12:26diffs and blame, working on these
12:28partial clones. Taking a step back, we
12:30noticed that, like most companies, we
12:32were using Git as a centralized system
12:34with one authoritative copy. But, nearly
12:36all of our clients were agents in jobs
12:38that are online by definition. This is a
12:40very interesting and worthwhile callout.
12:43You can't use Cloud if you're offline,
12:45because Cloud is being served from a
12:47giant data center that has GPUs you
12:50don't. And unless you are plugged into
12:52those directly, and somehow the internet
12:54for that data center is offline, you're
12:56online. Building everything around this
12:58idea that everyone needs everything
13:00synced just doesn't make that much
13:02sense. The properties that make Git
13:04excellent for distributed offline work
13:06were actively undermining their ability
13:08to scale as the repo grew. So, they
13:10asked themselves, if they designed
13:11source control with the fact that agents
13:13are always online in mind from the
13:15start, what would it look like? How
13:17would it be different? With agentic
13:18coding, many common assumptions about
13:20how software is developed are evolving.
13:23They embraced these changes when they
13:24built Scion, which which them to take a
13:26fundamentally different approach than
13:28Git on several core principles. Source
13:30control for huge mono repos does have
13:32prior art, like Google's Piper and
13:33Meta's Sapling. I'm touching on Meta's
13:35stuff before. So, they were able to
13:36build Scion with their hard-won lessons
13:38in mind. It is worth noting just how
13:40many people at Anthropic are ex-Meta and
13:43ex-Google, and they were often working
13:45on these types of things at those
13:47companies. Like, I know Boris was deep
13:48in developer tooling at Meta, and now he
13:51is making Claude good.
13:53The first core change they made is
13:55again, this idea that clients are always
13:56online. Since nearly all of the clients
13:58are agents in jobs, they designed Scion
14:00client and protocols around a server
14:02that's always reachable, and they have
14:03clients fetch data when they need it.
14:05Also, clearly not using GitHub, because
14:08even when you're online, you can't
14:09assume GitHub's online. The next point
14:11is this idea of client-server asymmetry.
14:13In Git, every client is capable of being
14:15a full-fledged server. This is one of
14:17those things that makes Git Git. It is
14:19truly decentralized. Anything can be a
14:22Git server.
14:23>> [snorts]
14:23>> But, in Scion, the server answers
14:25questions about file history and blame,
14:26creates commits, performs merges and
14:28rebases, and so on. The client maintains
14:30a bounded local object cache. It lays
14:33out the working copy files on disk. It
14:35monitors for file system changes, and it
14:37gives users ergonomic ways to work with
14:39that server. It can compute very little
14:41on its own, but in exchange, it requires
14:44way less data locally. Next, they talk
14:46about how the source control service can
14:47power many novel application types. With
14:50Git, a CI job or tool that needs to ask
14:52a question about the repo, like who last
14:54changed the line, or how two commits are
14:57related, it would have to maintain the
14:59whole local copy of that repo, or hope
15:01that some vendor-specific API is able to
15:03provide the answer. With Scion, all the
15:05source control queries that the client
15:07uses are also exposed via API. So, in
15:09many cases, those jobs don't need the
15:11local repository at all. Many internal
15:13tools, like their automated bisection
15:15revert pipelines, wouldn't be practical
15:17at the scale they're operating at
15:19without this cheap programmatic access
15:21to the source. They also call out that
15:23everything must be horizontally
15:24scalable. At their peak, they ran
15:26several hundred Git mirrors, which
15:27significantly strained capacity and no
15:29longer scaled once the repo outgrew the
15:31mirrors' disks or memory limits. You
15:33know it's bad when your Git mirrors are
15:36running out of disk space. And to be
15:38fair, disks did get a lot more
15:40expensive. I still can't believe that
15:41hard drives increase in price as much as
15:43they did. Thanks, Anthropic. In Scion,
15:45the repo's data is distributed and every
15:47component is independently scalable.
15:49There is no material copy of the repo
15:51anywhere, even on the server. There is
15:53no equivalent of pack files, just Git's
15:55compressed bundles of objects, so
15:57there's no repo maintenance needed,
15:58either. And then, a very important
16:00piece, commits must be cheap,
16:01incremental, and lock-free. Lock-free is
16:04particularly important. One person
16:06committing shouldn't block somebody
16:07else, or more realistically, 500 agents
16:10committing shouldn't block another 500
16:12agents. In Git, you commit locally and
16:14then you push in order to share your
16:15work. In Scion, every commit, rebase, or
16:17merge happens on the server, so there's
16:19nothing to push. But, each of these
16:21needs to be fast. The cost of creating a
16:23commit and updating its downstream data
16:25structures has to scale with the size of
16:27the change, not the overall size of the
16:29repo, or the server won't be able to
16:30keep up. The development life cycle in
16:32Scion discourages refs, pointers to
16:34commits such as branches and tags,
16:35except for the main ref or release
16:37branches. The reason for that is because
16:39global state that every server has to
16:41agree on is always going to cause scale
16:43issues. Globals are bad when you're
16:45operating this fast, always. And then,
16:47the most important piece, the working
16:48copy is optional and it is sparse by
16:51design. Because the repo is so large and
16:53they're checking out code tens of
16:55millions of times a day for CI, they
16:56want to avoid checkouts wherever they
16:58can. With Scion, a CI job that needs the
17:01diff between a commit and its merge
17:03base, for example, can ask the server
17:05just for that. The same goes for writes.
17:07Many write workloads can skip the
17:09checkout and function entirely over API.
17:11Scion's protocol is centered around
17:13sparse checkouts, which only contain the
17:15directories the client asked for. Then
17:17there's also virtualized checkouts,
17:18which materialize files on demand. So
17:20when the job needs the file on the disk,
17:22the client only pays for what it's
17:23actually reading. I was playing with
17:25some of these ideas really late last
17:27year and early this year trying to make
17:28it easier to manage state of like a code
17:32base in a way where you don't have to
17:33have it all on your computer, but your
17:34agents can still access it. It's dope to
17:36see them doing the real version of this
17:38cuz mine was a chaotic experiment. So
17:40how does it actually work? Now that
17:41we've seen the principles, we can
17:43hopefully understand how it all comes
17:45together. These insights can apply to
17:47other source control systems or more
17:49broadly to other stages of the software
17:51development life cycle. Oh boy, we're
17:52now in the data structure section. This
17:54is where we're going to get into the fun
17:55details. The unit of work is a change
17:57set. I I see the meta inspiration here.
17:59>> [snorts]
17:59>> And change sets are entries in
18:00write-ahead logs. A change set stores
18:03most of what you'd expect from a commit
18:04in Git. Where a Git commit stores a
18:06pointer to the tree, a change set stores
18:08a list of the files that changed
18:10relative to its first parent. Where a
18:12Git commit stores a pointer to the tree,
18:14a change set stores a list of the files
18:16that changed relative to the first
18:17parent along with content address
18:19pointers to the compressed file contents
18:21in blob storage. Change sets are created
18:24server-side and are available everywhere
18:25instantly. Fascinating. I knew they were
18:27doing blob storage for a lot of this,
18:28but kind of crazy to see the like the
18:31direct callout that the source code does
18:33not exist in the system and when you
18:35request the source, it has to go fetch
18:37that from blob store and it's
18:38effectively just a database pointing at
18:40those blobs. Seems like all the metadata
18:41is in Spanner. Makes sense. Again, very
18:43Google inspired. Very interesting they
18:45have separate author and committer
18:47fields. Makes sense cuz the author is
18:49the person who could who triggered it
18:51and the committer is whatever agent did
18:54it or where it happened. If you have
18:55like different systems that are doing
18:56these things, very interesting. And
18:58other usual fields like author date,
19:00committer date, message, the parent
19:02which we talked about before. That is
19:03the thing that this is in reference to.
19:06And then the file changes which
19:07specifies the file that changed and then
19:10has pointers to the blobs in their blob
19:13storage. The structure lets them accept
19:15change sets with very high throughput
19:17cuz there's no pack files they have to
19:18deal with, no locking or crossover
19:20coordination, and no refs to move
19:21around. What this is trying to say is
19:23you just have to append. You only are
19:26adding to the data set. You don't have
19:28to go through the whole history before
19:30appending it. You don't have to move
19:32pointers from here to there. You don't
19:33have to restructure the tree. You don't
19:35have to sync all of the data back and
19:37forth to make sure it's all in the right
19:38place. You just put up the change set
19:41and store the blobs in blob storage, and
19:43you're done. Oh, is it not even storing
19:45the diffs for the file change? It's just
19:47storing the new state of the file? That
19:49would make sense. So, it's saying like
19:50these files changed. Here is where they
19:53point. But this thing in the storage is
19:56the full content of that file, not a
19:58diff. Because if it was a diff,
19:59recalculating would be hell. Change sets
20:02are recording only what changed. So, on
20:03their own, they can't quickly answer
20:05questions like who edited a line or what
20:07commit affected a file. To answer those,
20:09Scien builds additional data structures
20:11from the change sets, including
20:13dedicated structures for file blame and
20:15path history, as well as skip lists for
20:17the commit graph. They call everything
20:19that isn't directly part of the change
20:20set derived data, and they compute it
20:23all through one generic mechanism. Every
20:25type of derived data is defined
20:26recursively, so its value for a change
20:29set is computed from its values for the
20:31change sets' parents. The mechanism has
20:33three benefits. First, they can easily
20:35add new types of derived data when new
20:36data structures would make some
20:38operations faster. Second, for each
20:40type, they can choose whether to compute
20:42it eagerly or defer the work, so they
20:43can avoid materializing data they don't
20:45need. And third, because the client
20:47never sees the internals of the
20:49structures, they can optimize them
20:50without changing the client. They've
20:52added new types of derived data many
20:54times as the system has grown, such as
20:56new indices for performing copy
20:58detection or path-based file search.
21:00Makes a ton of sense. To try and dumb
21:03this down, cuz we're deep in the weeds,
21:04which I I love. A big part of what makes
21:07Git so powerful is that you have whole
21:10of the data when you clone the repo. So,
21:12if you want to find something like when
21:14this file changed, all of the things
21:16this person did, stuff like that, you
21:18can go through all the data yourself.
21:20The problem with this is if you want to
21:22add something new to get or make some
21:24type of action faster, you're kind of
21:27screwed because you'd have to upend
21:29everybody's Git. But, if people don't
21:31have to clone the whole thing and
21:32they're just hitting an API for this,
21:34you can choose what you do and don't
21:35expose. And since the server here is
21:38full of cache layers, over-provisioned
21:40CPU, and so many other things, they look
21:43for the things clients want to request
21:45to get more information. They built some
21:47way to compute that that's efficient
21:49without having to change the data
21:50structure. And now the clients can do
21:52the thing. And the clients aren't going
21:53to cheat and do something weird, which
21:54I've seen a lot. When you give client
21:57access to the internals of something, it
21:59will invent some crazy novel and also
22:02crazy slow and CPU-intensive solutions
22:04to get what it's looking for. This hard
22:06API boundary prevents that, which is
22:08almost certainly a huge benefit. So, how
22:11did they scale this? Starts with a small
22:13fleet of Scion servers providing source
22:15control APIs for any operation a client
22:17needs, like getting change sets,
22:18computing diffs, finding merge bases,
22:21and fetching blobs. Computing diffs is a
22:23fun thing to realize here because again,
22:25it's storing the file content at the end
22:28of a change. So, when you are asking
22:30what changed between these two like
22:33change sets, you don't just look at the
22:35diff because you don't have a diff. The
22:37diff is computed when you say, "Here are
22:40two change sets, what changed between
22:41them?" And it does that by grabbing the
22:43blobs for all the files that are
22:45affected and seeing how they differ
22:47between those states. All source control
22:49data is content-addressed, so a piece of
22:51data never changes once it's stored.
22:53This is also important. The ability to
22:55override history in Git isn't too bad
22:58because whenever you force push, you're
23:00changing all the commit hashes. But, if
23:01you had the ability to edit the contents
23:04of a given hash, that would
23:05fundamentally break Git at its core.
23:08Immutability is always more scalable
23:10than mutable stuff pretty much 100% of
23:12the time. So, building the whole system
23:14around these change sets being fixed and
23:16permanent, and all of the flexible stuff
23:19being calculations done on top, this is
23:21touching my functional programming heart
23:24deep at its core. And since they have
23:25these immutable content addressed change
23:28sets, all servers can cache them without
23:30checking with the others. As soon as
23:32it's sent up to one server, it knows
23:34that change set's unique, it can just
23:36blindly trust that. As a result of this,
23:38the servers are horizontally scalable
23:40and they require very little
23:41coordination. They're essentially large
23:43caches that can also compute derived
23:45data on the fly and write the results
23:47back to Spanner. Scion's many consumers
23:49no longer need to maintain local repos
23:51of their own. Fetching and maintaining
23:53the repo used to be a significant
23:54maintenance burden for the CI jobs and
23:55tools that needed to ask source control
23:57questions, especially around commit
23:59ancestry and blame. Early in the
24:00migration, before they even had their
24:02own client repo format and protocols,
24:04Scion was only used to answer these
24:06types of metadata queries. Fetching file
24:07contents and materializing working
24:09copies was far more data intensive
24:11operations, so they needed to design
24:13something of their own there. Scion's
24:14SCN client deviates from the Git fetch
24:17protocols in several important ways, and
24:20as a result, it lends itself to highly
24:21scalable architectures that require
24:23significantly less CPU than Git servers
24:25do. Scion's negotiation doesn't tell the
24:27server what the client already has,
24:29instead, it operates exclusively on
24:31trees and blobs that the client wants.
24:33Starting from a root tree, the client
24:35checks its local store for missing trees
24:38and requests them recursively. Then it
24:40repeats the process for the blobs. So,
24:42it starts by finding the change sets
24:44that it needs to do the thing it wants
24:46to do, and once it has those, it goes
24:48and collects all of the blobs to have
24:50the files it needs, and then it can
24:52actually start working. Every request is
24:54for a specific object that never
24:55changes, so any cache can answer it. And
24:58the architecture closely resembles
24:59traditional tiered caches. The process
25:01is also efficient and resumable because
25:04there's no single multi-gigabyte pack
25:06file response. Sparse and even
25:08virtualized checkouts don't need special
25:10support because the client simply
25:11doesn't ask for trees it doesn't want.
25:13This paragraph is using a bunch of terms
25:15that they did not define and they are
25:17using in unique ways, so I will do my
25:19best to translate from Claude-isms to
25:22English here. The clients don't connect
25:24directly to the Scion servers cuz those
25:27are managing all of the change sets.
25:29They're talking to a daemon over a
25:31socket, and that daemon stores objects
25:34for the whole node. So, it's effectively
25:35keeping a cache of like what was the
25:37most recent version of a given file at a
25:40given file path. This allows it to
25:42respond to requests for files without
25:44having to go down the whole tree to
25:46compute the current state of things. I
25:49really hate how they use node here. Each
25:51of these daemons that does this caching
25:53gets periodically reset on the latest
25:55snapshot so that it isn't like
25:58recalculating all the things that have
25:59changed in the last 3 days. They update
26:01it constantly, and let's be real, with
26:04the way people are spinning up these
26:05agents, spinning up CI, all of these
26:06things, they're probably starting that
26:08work from latest. So, it makes a lot of
26:10sense to get the cache up to date that
26:13way and then just resolve things that
26:15have changed since. The cache servers
26:17stream objects directly from memory
26:18instead of like storing it in the hard
26:20drive, which makes it way faster to get
26:22that data when you're on a client on the
26:24other side. And it stores this in the
26:25same format that it would have been on
26:27disk so that it doesn't have to change
26:30the shape or format of the data when it
26:31comes out of memory and gets onto a
26:33drive after going over the network. When
26:35caches are missing objects, they ask the
26:37authoritative Scion servers, which is
26:38the layer we're talking about before,
26:40which will read from Spanner or cloud
26:41storage as needed in order to get the
26:44updated data and the updated files in
26:45that cache layer. The Scion daemon,
26:47Scion cache, and Scion server all
26:49respond to identical RPCs for this
26:51protocol. So, the client doesn't need to
26:53know the cache topology. Any tier could
26:56theoretically talk to any other, but
26:57they strictly enforce the tearing to
26:59isolate failure domains. They want to
27:00avoid a situation where a failure in a
27:02regional Scion cache overwhelms the
27:04upstream servers and brings down the
27:06entire system.
27:07So, again, they have that cluster of
27:08Scion servers that are the like proper
27:11management of the real-world
27:14state of things, the authoritative
27:15layer. That is the layer that stores in
27:17Spanner for the change sets in the cloud
27:19storage for the actual contents There is
27:22then a cache layer that is what gets hit
27:25often for your actual work that is
27:27regional, so you don't have to like
27:29fetch across the world to hit the
27:30server. This is what will hit the Scion
27:34server to get a reasonable state in the
27:36cache. Developer machines directly hit
27:38the Scion cache when you're like pulling
27:40down the code to do work. And if the
27:42Scion cache is missing things that you
27:44need, it will go pull it from the
27:45server. But, if you're working on CI,
27:47even this isn't the right abstraction,
27:48and that's where this daemon comes in
27:50that we were just talking about that
27:51gets periodic snapshots so that it's
27:53ready to spin up CI jobs as quickly as
27:55possible. Each tier of this system
27:57maintains a cache hit rate of over 99%
28:00and it doesn't require long-lived state.
28:02So, it's easily scaled up if needed.
28:05Their infrastructure footprint is 200x
28:07smaller than their previous Git
28:08mirroring infra in terms of CPU and
28:10memory requirements. That's insane.
28:13Again, like this is like the crazy idea
28:16that has been like hurting me a bit. The
28:19realization that that like all software
28:21was built to be somewhat generalizable.
28:24Now, we can build software for our
28:26specific needs. And like it's a silly
28:28thing to pivot to, but hear me out. One
28:31of my weird YOLO projects right now is
28:33called Lust. Lust lang to be specific.
28:36I'm trying to take a subset of Rust that
28:39gets you most of the benefits of Rust,
28:42but make it so that it could be compiled
28:43and built 100 times faster because the
28:46build times for TS Rust, which is my
28:48rewrite of the TypeScript compiler in
28:49Rust, are too slow. This is an insane
28:51project that I didn't think would make
28:53any progress, but it's already able to
28:55compile over 40% of the files from TS
28:57Rust, and I am expecting by the end of
29:00the week it might actually work. Would
29:02this work for most Rust projects?
29:05[ __ ] no, absolutely not. That is not
29:07what this is for. This is a bespoke
29:09experiment to see if I take a given code
29:11base that I'm not happy with compile
29:13times on, can I invent a new language to
29:15make it compile faster?
29:17It's like
29:18insane that this is even
29:20a thing you can think, much less do. But
29:23I'm using a model that's a lot dumber
29:25than what they have access to. I'm using
29:26the distilled version of two model
29:29generations away from what they have
29:30internally. They're rocking Fable Ultra
29:33Max 7 internally, and I'm here with Opus
29:365.5, more than happy to do these types
29:37of things. The fact that we can build
29:39this way now is just crazy. And while
29:42this system might not be 200 times
29:44smaller and faster for some cases, it is
29:47for theirs. And I would expect more and
29:49more companies that have these types of
29:51big, painful scaling things to go
29:53reinvent the painful pieces that are
29:55slowing them down. And a fun fact, as
29:57John, the borrow checker is not why
30:00build times suck in Rust. It's a bunch
30:02of weird LLVM [ __ ] Anyways,
30:05CI and research jobs are able to check
30:07out code two orders of magnitude faster,
30:09seconds rather than minutes, which frees
30:11up a sizable amount of capacity across
30:13the fleet. CI alone had a 25% CPU
30:17capacity win. Neither clients nor caches
30:19need to perform repo maintenance, which
30:21simplifies the life cycle management of
30:22our jobs. Out of curiosity, I'm trying
30:25to get the hard numbers for how much
30:26time T3 code builds in CI have spent
30:30dealing with GitHub and get like clones.
30:33It's going to be a number that hurts me
30:35a lot. We'll come back and check on that
30:36in a bit, but I have a feeling I'm going
30:39to feel a lot of pain as soon as I see
30:41that. So, what did they learn building
30:43Scion? They started building it around
30:45the same time as Mythos preview and
30:47Claude tag becoming available
30:48internally. Both of these releases
30:50simultaneously amplified the need for
30:52Scion and meaningfully accelerated how
30:53quickly they'd be able to build it.
30:55Here's a particularly striking example.
30:56Earlier in the year, they had largely
30:58handwritten the first version of the Git
31:00to Scion ingestion pipeline as a
31:02prototype. They thought it was good
31:03until they got their hands on early
31:04internal snapshots of Mythos preview.
31:06They asked it to rewrite the whole thing
31:08and with no further instructions, it
31:09produced a version that consumed 20x
31:11less memory and was four times faster
31:13while preserving all of the implicit
31:15invariations around ingestion, ordering,
31:17and data structures. This is the history
31:20of changes to Scion and Antique. I will
31:22say, worth noting here, they are
31:24bragging about 30k changes a day, but
31:26they've only made 12,250
31:29changes to this architecture total. So,
31:31their daily velocity is more than double
31:35the amount of like requests, commits,
31:39changesets, whatever you want to call
31:40them, that have landed for the whole of
31:43their source control. Kind of crazy.
31:45What they learned building and operating
31:47Scion offers some interesting lessons
31:49for what building agent-scale software
31:51will look like. Rapid development with
31:52small teams is going to become more
31:54common. Can confirm. A team of seven
31:57people built both Scion and Antique,
31:59their code review tool, which they could
32:01have easily written a whole separate
32:03blog post about. Considering how awful
32:05code review is, I feel it. I'm thankful
32:07that we have so many wonderful sponsors
32:09working on that now. They also follow up
32:10with Not to mention, half a dozen
32:12related systems that they fully migrated
32:14all of in 5 months. They did this all
32:16while the volume of code merged per day
32:18was increasing 10x. In the past,
32:20developing and deploying source control
32:21systems like this would have taken years
32:23of iterative migrations with many more
32:25engineers. We think this sort of fast,
32:27nimble migration will become more common
32:29in the future.
32:31Yeah.
32:32It just will. Switching costs has gone
32:34down more than almost anything.
32:36Historically, people adopt a technology,
32:38a solution, a vendor, and then they kind
32:40of are stuck there cuz switching off is
32:41more expensive than whatever cost
32:43they're incurring staying. Now, it's
32:45easier to write your own alternative and
32:46move to it.
32:48It's insane. I personally threw together
32:50a solution to clone and mirror and
32:53backfill all of our analytics data that
32:56I have on one particular analytics
32:58provider so that I have it directly on
33:00my own managed ClickHouse on top of R2.
33:02Ended up being more than 10x cheaper and
33:04way easier for my agents with. And I did
33:07this part-time with a handful of threads
33:09on the side while I was working on my
33:11real day-to-day work. And these types of
33:12things are miserable to do without AI.
33:15Like this sucked long ago. And now it's
33:18relatively trivial to set the stuff up
33:20yourself. Like truly crazy. As Chad is
33:22pointing out, I literally wrote my own
33:24TypeScript compiler and it's now running
33:26thousands of times a day in T3 code.
33:28It's over 40,000 weekly downloads and I
33:30threw this together kind of as like a
33:32joke just to see if it was possible.
33:34Truly unbelievable. Oh, I got my
33:36numbers. Of all of my CI, almost 1,000
33:40hours of the CI on T3 code was spent
33:43waiting for Git checkouts. That's 42
33:46days of waiting for your Git pull. And
33:48that's just CI. That's not counting
33:50people doing it locally. Yeah, the
33:52estimates just for T3 code time spent
33:55waiting for Git pull is between $350 and
33:59$600 just for the renting of a CPU
34:02pulling down the changes. Insane. And
34:05that's for a small project by a startup
34:07with five-ish employees depending on how
34:09you cut.
34:10Imagine how much crazier that could get
34:13at this level of scale. I do like this
34:15section. This is a thing I've never
34:16heard from Anthropic. Engineers are no
34:18less valuable, but their work is
34:19changing. Thank you, Anthropic, for not
34:21saying engineers don't matter anymore.
34:23Nice change. All the engineers who
34:24worked on the system would now rate
34:26Claude as better at writing code than
34:28them. But the engineer still supply the
34:30invaluable understanding of the
34:31high-level inter-system components in
34:33the design level trade-offs, making
34:35informed choices around the problems
34:37they wanted to solve and the scale they
34:39knew they needed to achieve. Very often,
34:41they found that the role of engineers
34:43was to tell Claude no. Agents will
34:45sometimes violate core system
34:46constraints to satisfy a pattern that
34:48should never have been allowed in the
34:49first place. Okay, maybe they are using
34:51the same Claude that we are.
34:53This was particularly common in their
34:54migrations. For example, there were
34:56multiple systems that tried to brute
34:57force looking up when a change merged to
34:59main by linearly reading the commit log
35:02instead of asking the code review tool.
35:03Anthropic also makes extensive use of
35:05adversarial review, and while this is
35:08incredibly effective in many ways, can
35:10also be myopic. So, skilled engineers
35:12need to steer Claude away from
35:14over-engineering parts of the system in
35:16response to hypothetical concerns.
35:18Yep, this is even worse when you have
35:20feedback from other model families cuz
35:22they're worse at convincing each other.
35:23Making sure that your AI review systems
35:26don't accidentally cause huge
35:28over-engineering and scope creep is a
35:29genuine real challenge. For example, one
35:31such review was overly defensive about
35:33trusting other internal services,
35:35resulted in Claude adding excessive
35:37validation and complexity. Our team
35:39members still review every change
35:40request for the system, but they now
35:42examine the intent of the change as
35:44opposed to reading every line of code.
35:47Oh, how the world has changed. We were
35:49able to substantially improve
35:51optimization and quality by virtue of
35:53Claude's ability to write immense
35:54amounts of code much faster than they
35:56could themselves. Scion is among the
35:58most well-tested and well-defended code
36:00bases they've ever worked on. Even on
36:02their tight timelines, they were able to
36:03do hermetic simulations of every
36:05non-trivial infrastructure and source
36:07control interaction that they've ever
36:08encountered, as well as fuzzing and
36:10sophisticated rate limiting. It still
36:12blows me away that I can be like, "Hey
36:14Claude, build a fuzzer for this system,
36:15find bugs." And it just does.
36:18Surreal. Claude also found significant
36:21optimizations. For example, algorithmic
36:23optimizations to ancestry queries that
36:25missed one of their indexes, which would
36:27have taken an expert days or weeks of
36:29analyzing and tuning, and then did it
36:31all again the next day. This one's
36:33particularly funny cuz I just filmed my
36:35video about Gemini 4 Argon. Gemini 4 was
36:39hallucinating columns in my SQLite
36:42tables. After reading the file that is
36:46my whole schema, it would then fire a
36:48query for columns that don't exist.
36:51Meanwhile, Claude is doing algorithmic
36:55optimizations to get around missing
36:58indexes.
36:59Anybody who says Gemini is in a good
37:01state right now is not trustable as an
37:03engineer.
37:04It's pretty rough.
37:06Yeah, Gemini's imagination is
37:07incredible. It hallucinates so well.
37:09It's truly a creative model. You can see
37:12here the difference in everyday source
37:14control commands when they run their SCN
37:16system instead of Git. Pulls are 25x
37:19faster, checkouts are 14x faster, rebase
37:21is 5x faster, status is 207x faster,
37:25commits are only 30% faster,
37:27disappointing, but diffs are 6x faster.
37:30New work trees, oh god. This is one of
37:32the biggest issues and like we used to
37:34deal with this at T3 Code. Making work
37:36trees when you have to clone everything
37:38is really expensive. 84x faster is a
37:42huge win there.
37:43Pull latest is 20x faster, commits are
37:4624x faster, and status for agent
37:48environments is 20x faster. Huge. Oh,
37:51this first section was developer
37:52environments versus agent environments,
37:53that makes sense.
37:54Yeah, all of these wins are massive.
37:56Alongside all the performance wins, they
37:58also laid the groundwork for new forms
37:59of mono repo security. Managing safety,
38:01security, and compliance in massive mono
38:03repos is as important as managing the
38:05size and scale. They built Scien with
38:07these in mind from the start. It
38:08supports rich access controls on
38:11repositories, as well as the ability to
38:13apply permissions to individual
38:15directories in order to protect the most
38:17sensitive code without losing the
38:18benefits of the mono repo. Look who was
38:21right about committing your environment
38:22variables, [ __ ] The the Theo
38:24wins never end.
38:27Can't wait for another anime profile
38:29picture on Twitter to say I'm wrong cuz
38:31they don't have a job. Furthermore, the
38:32code review tools that they built can
38:34express complex semantic constraints on
38:35approvals, which allows them to
38:37differentiate simple things like comment
38:38only changes from sensitive logical
38:40changes, so that human reviewers can
38:41focus on the most consequential parts.
38:43You [ __ ] you don't need to do
38:45this. You don't have to do this
38:46programmatically. You don't have to
38:48build code to change the categorization
38:53of the change sets. I don't know if you
38:55guys have heard about this. There's this
38:56thing called Claude. You can give it
38:58code and ask it a question and you get
39:01an answer. This actually might be the
39:03thing I'm most bearish on in this whole
39:04thing. If they are compute constrained
39:06enough that they're not running Claude
39:08to summarize and filter their diffs, and
39:11they feel like they need to make
39:12programmatic solutions for this instead,
39:14maybe they are out of compute. Maybe we
39:15are screwed. With all of this, they
39:17claim that a new level of operational
39:19excellence is possible. And you know
39:20what? To their [ __ ] credit, it is.
39:22Their uptime is significantly better
39:26than it was in the past. It was bad bad
39:29before. I can't remember the last time
39:31Claude was down. The website goes down
39:33decently often, but like Claude code and
39:35the Claude APIs, those are pretty
39:37reliable lately. Funny enough, I think
39:39I've had more problems with OpenAI and
39:40Codex outages than I have with Claude
39:42recently, which is unbelievable
39:44considering how big the gap was before.
39:46So, yeah, if they're saying that this is
39:47helping make operational excellence more
39:50possible, I'll believe them because they
39:52are they are fixing their downtime
39:53problems. Their team uses Claude tag,
39:55which if you don't know, that's their
39:56Slackbot that they're all obsessed with.
39:58They use it heavily and it helps them
40:00monitor each component of the system.
40:01For example, they have Slack channels
40:03where they tasked Claude with being
40:04obsessively focused on the health of
40:05single components. They call these super
40:07fans. Also, they they refer to them as
40:09Clauds, plural. That hurts me a bit.
40:11Guys, why why do you have to do this,
40:13Anthropic? I I was defending you and it
40:15felt good. Now you're doing cringe [ __ ]
40:17like this. We call these clouds super
40:19fans. I'm like imagining the person
40:22saying this to me and somehow their face
40:24is more punchable than mine. They have a
40:26Scion Damon super fan, a Scion cash
40:29super fan, an antique merge queue super
40:32fan, and so on. They have access to the
40:34telemetry and logs and they act like
40:36experienced engineers who have nothing
40:37better to do than refresh charts and
40:40look for anything strange. Are you
40:41talking [ __ ] That's my job. They do
40:44call out that they have traditional
40:45alerts as well, but the super fans
40:47detect subtle and novel issues that make
40:48those alerts seem one-dimensional by
40:50comparison. That does actually seem kind
40:51of cool. Everything earlier I said where
40:53they were reducing compute with the
40:54programmatic thing, this is the
40:56opposite. This is them burning compute
40:57for fun.
40:58And here is what's probably the most
40:59important lesson in all of this. We're
41:01completely rethinking our priors.
41:04Engineers, for good reasons, get pretty
41:08fixed in their way of doing things
41:10because in order to be a world-class
41:13engineer, you need to have a
41:16the thing. And generally speaking, the
41:18hot, hyped-up, cool new thing is not the
41:21best long-term solution. So a lot of the
41:24best engineers in the world have gotten
41:26used to this and are inherently
41:29skeptical by default. They trust what
41:31they know and what they know works and
41:33they're hesitant to try new things
41:34because they've been burned so many
41:35times. You got to get over that. Things
41:38are changing too fast. You have to flip
41:40the way you think about this almost like
41:42upside down where you have this
41:43knowledge of what does and doesn't work,
41:45which leads you to picking certain
41:46tools. Now you need to do the opposite.
41:48You need to know what does and doesn't
41:50work and where it breaks and where it
41:52functions and use that to reinvent the
41:55things you've trusted your whole life to
41:56be perfect for your needs. I have been
41:58writing TypeScript for almost a decade
42:00at this point and I rewrote TypeScript
42:02itself for fun. Let's see how Anthropic
42:04explained this phenomena. This project
42:06prompted them to take a very hard look
42:08at other workflows that were designed
42:10for human scale. For our merge queue for
42:12example, just doesn't make sense at
42:13agent scale. What does CI even mean when
42:15hundreds or even thousands of changes
42:17can merge while tests are running? We
42:19haven't been able to effectively
42:20replicate the industry standard of a
42:22global repo lock on the merge queue. So,
42:24instead, they've invested in post-merge
42:26and pre-deploy automated testing. Look
42:28at that. Theo was right again. I I'm
42:31sorry to
42:33freak out about these things again, but
42:34you guys have no idea how long I've been
42:38harassed for thinking it's more
42:40important to be able to undo a bad
42:41change than to add thousands of layers
42:44in front of the changes to make it
42:46slightly harder to make a bad change. So
42:48much of engineering has been focused on
42:50increasing friction more and more and
42:52more to lower the likelihood something
42:54breaks. I have been focused on make the
42:56friction as low as possible both to get
42:59code out, but more importantly, to fix
43:01something when it breaks. Because no
43:03matter how strict your processes are,
43:05you will ship something broken. I don't
43:07care who the [ __ ] you are. If you
43:09actually think you will never ship
43:10broken code or that you have built your
43:12system in a way where will never ever
43:15have bugs, I hope that your mom's
43:17basement doesn't smell too bad. Because
43:18in the real world things break. And when
43:20things break, you need to fix them. And
43:22focusing on ways to fix things and
43:24reduce the damage that broken things can
43:26do is incredibly important. They do call
43:28out that by working this way, sometimes
43:30concurrent changes conflict, but Claude
43:32can typically fix these conflicts faster
43:34than humans can even identify them. I've
43:36been thinking a lot about this. The idea
43:37I've had is what if the merge queue
43:40wasn't PRs that need to be merged one
43:42after another? What if the merge queue
43:44was a super general here are things that
43:47we're happy to have go in. Hey Claude,
43:50land all of this. And it would try to
43:53combine like once something has the the
43:55merge thumbs up, Claude tries to squash
43:58as many of those chunks into individual
44:00things as possible like in groups, and
44:02then pull them into main as effectively
44:05as it can dealing with the conflicts as
44:07it goes. Because Claude is now better at
44:09dealing with merge conflicts than I am.
44:10And now we got an interesting end here,
44:13contributing to the wider ecosystem. We
44:15see Scion is just one part of the answer
44:18to the scaling concerns that
44:19increasingly capable AI agents have
44:21raised. We're sharing these details as
44:22an example of how teams can tackle
44:24similar challenges. But they don't
44:25expect their solution or their
44:27considerations to suit every case. They
44:29specify this is in no means a suitable
44:30replacement for Git in general use. The
44:32concepts of outlined here really only
44:34make sense for truly enormous mono repos
44:36with either tens of thousands of
44:38engineers or swarms of agents
44:40prolifically writing code. Plus,
44:42Anthropic's unique in that their ability
44:44to spend tokens for internal use far
44:45exceeds what any other business would
44:47realistically want to spend. Never
44:49thought I'd see the day. To be fair, as
44:51I've talked about many times, tokens are
44:53getting cheaper, way cheaper. A given
44:56level of intelligence is achievable at
44:58lower and lower prices constantly. The
45:00Fable rewrite of Bun into Rust was like
45:04300K plus of Fable tokens. I was able to
45:07do a bolder rewrite, which was
45:09TypeScript in Rust, myself, in around 20
45:11to 25,000 dollars of Opus 5.5 tokens.
45:14The speed that these prices are going
45:15down is insane. So, it is absolutely
45:17possible by this time next year you'd be
45:19able to write your own similar thing to
45:21Scion, your own alternative to Git for
45:23your company, for just a few thousand
45:24dollars in tokens. They also call out a
45:26thing that is important to understand.
45:27They're doing this both to address their
45:29own software development needs, but also
45:30to test Claude's capabilities, to see
45:32what it can do. Practically speaking,
45:34it's difficult to offer Scion in a form
45:36that would be broadly useful to other
45:37organizations. It diverges significantly
45:39from Git, meaning companies would need
45:40to migrate every piece of the
45:42development life cycle, and it requires
45:43its own code review, code search, and
45:46CI. Their internal version of these
45:47components are highly bespoke, and they
45:49would be difficult for other
45:50organizations to adopt. And they worry
45:52that attending to build a public version
45:53that tries to please everybody wouldn't
45:55actually be very useful. Okay, I know
45:57what this paragraph is here for. There
45:59was a conversation internally about
46:01whether or not they could sell what they
46:02built. The people who built it said,
46:04"No, [ __ ] you. We're not going to do
46:06that. Here's all the reasons why it
46:07doesn't make sense." Enterprise sales
46:09finally probably gave us like, "Okay,
46:11whatever. We don't have to sell this."
46:13And then they wanted to talk about it.
46:14And coms is like, wait, wait, wait,
46:15people are going to ask for this. So,
46:17they had to take all the arguments they
46:19had internally and do this so that I can
46:21justify now as I read this to you why
46:24they are not open sourcing this for
46:25others to use. Okay, more justification.
46:27Let's see this part. However, they
46:29anticipate that this class of problems
46:30will become more commonplace over the
46:32next 6 to 12 months. And it would be an
46:34inefficient use of engineering resources
46:35and tokens. Every company using Git to
46:37face similar challenges alone. Given
46:39that, we think the most impactful thing
46:41we can do is share our thinking so that
46:43other devs can apply some of the most
46:45successful parts of Syon to get without
46:47departing from the overall model. A Git
46:49server doesn't have to work the same way
46:51as a Git client does. Novel server
46:52architectures exist that provide
46:54significant scaling improvements. Git
46:56platforms can provide comprehensive
46:58server-side RPCs like Syon, perhaps even
47:01in a standardized way across forges that
47:03the Git clients can learn to talk to.
47:05Negotiation protocol doesn't have to
47:07look exactly like it does today, either.
47:09And the various sparse and shallow
47:10features in Git can be made to compose
47:12well. Lastly, they'd like to acknowledge
47:14that the Git community has spent nearly
47:16two decades scaling the platform for
47:18millions of devs. Their contribution is
47:20far narrower. They're sharing their
47:21early views into how a genetic software
47:23development interacts with these
47:24systems. They can partner with GitHub,
47:25GitLab, and the open source community to
47:28support the changing needs of devs.
47:30We're all in it together as the software
47:31development life cycle intersects with
47:33the AI exponential. It's crazy how
47:35something like this would have felt
47:37comical a year ago. And now it's almost
47:40like, yeah, of course. Makes a lot of
47:42sense. I'm going to go fight the urge to
47:44make my own Git clone.
47:46It's going to be hard, but I'll do my
47:48best. I hope you enjoyed this deep dive.
47:50I know I did. I love these types of
47:51nerdy rants, and this was a very fun one
47:53for me getting to go deep on how source
47:55control works and how to make it scale
47:57the way that agents need. Do you still
47:58enjoy these deep dives or do you just
48:00want to see me vibe code more? Let me
48:01know in the comments and until next
48:02time,
48:03peace nerds.

About this transcript

This page contains the full transcript of Anthropic Rebuilt Git (yes really) by Theo - t3․gg, generated from the public captions YouTube serves with the video. The transcript has 9,843 words across 1,463 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.