YT YouTube2Text

YouTube2Text → Transcripts → ¿Ya No Necesitas Una GPU Cara? Strata Cambia La IA Local

¿Ya No Necesitas Una GPU Cara? Strata Cambia La IA Local — Transcript

by Jorge Marfil · 6,577 words · 991 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00A few days ago, I told everyone with a
0:0212 GB graphics card to forget about
0:04DeepSeek at home and use Qwen with
0:05Strata, which was the sensible thing to
0:07do, and I said it quite convincingly
0:09with my calculations done and the PC in
0:11front of me.
0:12A few days later, Strata put DeepSeek
0:14right on that graphics card, so this
0:17starts with me eating my words.
0:19And if you have a standard gaming PC,
0:21the kind you bought to play games and
0:23ended up using to tinker with models at
0:25night, you’ve surely been in the same
0:26spot as me: looking at the cost of a
0:28high-end graphics card, crunching
0:30numbers on your phone, and closing the
0:31tab with a sad look on your face.
0:34The news is that Strata, the local
0:35inference engine that came out at the
0:37end of September, just turned one month
0:39old, and in that month, it has released
0:41versions almost daily. It now works on
0:44AMD, Intel Arc, and Mac.
0:47It reads long prompts much faster and,
0:50above all, it now runs DeepSeek V3/R1
0:52Flash, which is a huge model, on a 12 GB
0:55gaming card.
0:58When it was released, it became famous
1:00for running a model counted at 125
1:01billion parameters on that same graphics
1:03card.
1:05And what is being said out there now is
1:07more or less what’s in the title: you no
1:09longer need to spend a fortune on a
1:11graphics card to have a large model at
1:12home, with posts boasting about DeepSeek
1:14on a single card.
1:16What I conclude after looking at it
1:19calmly is that this is only partially
1:21true. The graphics card is no longer
1:23what holds you back. Now what decides it
1:25is the rest of the PC: the RAM, the disk
1:27, and the processor.
1:29And even with all that in order,
1:31DeepSeek at home is slow for working
1:32with an agent.
1:34So let’s see what has really changed
1:36this month.
1:37How does it manage such a model with so
1:39little memory? What does it demand from
1:41each part of your PC? How long does a
1:43real work step take? And how much does
1:45it cost compared to the cloud? And with
1:46that, you’ll know if your current gaming
1:48PC is enough as is or what it’s missing
1:50. I’ll start with the good, because
1:52there is quite a bit of it.
1:54The first version of Strata, 0.1.0, was
1:57published on September 24th, and as of
1:59today, there are 65 versions. The latest
2:02one is from November 5th. In the first 5
2:05weeks, it went over 36,000 stars on
2:07GitHub; now it's around 40,900.
2:10And for a project that was born a month
2:12ago, that’s a lot of people watching.
2:15When it first launched, it did one very
2:17specific thing: running a single Qwen
2:18Flash Next model on Nvidia cards and
2:20that was it, no Mac or anything else.
2:24On September 29th, they added
2:26experimental Radeon support,
2:27specifically the RX 7900 XT and XTX on
2:30Linux.
2:32On October 7th, they added Intel Arc
2:34support on Linux, and on October 13th,
2:36Apple Silicon Macs from M1 to M5.
2:39Before I give you more numbers, I need
2:40to mention something important for
2:42everything that follows. Almost all the
2:44Strata speeds I'm about to mention come
2:46from the developer's own PC, unless I
2:48say otherwise. That PC has an RTX 5070
2:51with 12GB of VRAM, a 6-core Ryzen
2:53processor, and 64GB of RAM. It's a
2:56fairly standard gaming PC, which is
2:58exactly what we care about here, but
3:00it's just one machine, and what bothered
3:02me most about Strata at launch was the
3:04wait before the first word. There are
3:06two phases when you send something to a
3:07model. The phase of reading what you
3:10sent, called prefill, and the phase of
3:12writing the response, the decode. Strata
3:14wrote extremely fast, but it took its
3:17time reading. With Qwen Flash Next, a
3:1932,000-token prompt took nearly a minute
3:22to read.
3:23Now it reads at about 2,800 tokens per
3:26second, so that wait has been cut to
3:28about 12 seconds. And with 128,000
3:31tokens, which used to mean staring at
3:33the screen for 4 minutes, it's now under
3:36one.
3:37This is especially noticeable with a
3:39coding agent, since your first request
3:41carries your instructions, tools, and a
3:43huge chunk of code, and you used to have
3:45to go get water every time you opened a
3:47session. The writing speed, however, has
3:50barely changed.
3:52It’s still hovering around 95 tokens per
3:54second with the smaller version of Qwen
3:55, which was already very good.
3:58So, the month wasn't spent on faster
4:00writing, and almost all the effort not
4:02spent on reading prompts went into
4:04something else: adding a second, much
4:06larger model.
4:08DeepSeek V3 Flash arrived in Strata on
4:10October 24th, and I want to defend it
4:12properly before pointing out any flaws
4:15because the case for it is truly strong
4:17. According to its specs, it has about
4:19671 billion parameters—values the model
4:22learns—and it’s published under an MIT
4:24license, which is open. You can use it
4:26for whatever you want. In Artificial
4:29Analysis tests, the full September
4:31version scored a 69 in an agent
4:34automation test, compared to Qwen's 56.
4:37Although, to be fair, Qwen stayed ahead
4:39in two other tests.
4:41But for what matters to many people,
4:43which is letting some agents work on
4:45their code, DeepSeek is in a different
4:47league, and the usual idea—the one I
4:49repeated myself—had its reason.
4:51The full weights of DeepSeek take up
4:53about 510 GB.
4:56And to give you an idea, one of the
4:58servers I'll talk about later, which was
5:00used to log actual agent traffic, served
5:02the full model with 4 RTX Pro 6000s,
5:04which are professional workstation cards
5:07.
5:08For years, the rule has been simple.
5:11The model has to fit in the VRAM, the
5:13graphics memory, and if it doesn't fit,
5:15you either buy more GPU or forget about
5:16it.
5:18That's why the usual advice was to save
5:20up for a big card, and the problem is
5:21how much they cost right now.
5:24According to a GPU price tracker, an RTX
5:275090 that launched at $ 1,999
5:32was around $ 4,999 in stores in
5:34September, and even an RTX 5070, the one
5:37in the developer PC that launched at 549
5:41, was around 900.
5:44So, what Strata promises is very
5:46tempting.
5:48In a thread where someone complained
5:50that the application was confusing, the
5:51top-voted response said that Strata has
5:53been and continues to be a milestone in
5:55showing that you can run huge
5:56mixture-of-experts models without
5:58spending more than $ 5,000.
6:01And I agree with that. What you have to
6:03understand is how it does it, because
6:05that’s where everything it demands from
6:06your PC comes from. I’ll make a quick
6:08stop here for the key concept, which is
6:09short, and then everything falls into
6:11place on its own. DeepSeek is what is
6:13called a mixture of experts.
6:15In a dense model, every word it
6:17generates passes through all of its
6:19parameters. They all work, always.
6:22In a mixture of experts, each layer of
6:24the model is split into many small
6:26specialized blocks: the experts.
6:28And for each word, a small router
6:30decides which ones are used.
6:33In DeepSeek there are 40 layers, each
6:36with 384 experts, and for each word,
6:38only six from each layer work.
6:41That means the active parameters, the
6:43ones that actually perform calculations
6:46for each word, are a tiny fraction of
6:48the total.
6:50The catch is that those that aren't
6:52working on this word might be needed for
6:53the next one, so they have to be stored
6:55somewhere and kept close at hand.
6:58To make this understood, I see it as a
7:00kitchen.
7:01The GPU is the countertop right next to
7:03the fast cook; it's where work happens
7:05at full speed, but it's small.
7:07RAM is the pantry just two steps away;
7:10it holds much more, but it takes a
7:12little longer to grab things.
7:14And the SSD is the basement storage room
7:16, where everything fits, but every time
7:18you need something from down there,
7:19someone has to go get it.
7:21Also, there is a second cook, the
7:23processor, slower than the one at the
7:26countertop, who cooks what isn't on the
7:28countertop. Well, Strata distributes
7:30DeepSeek throughout that kitchen.
7:32On the countertop, it puts what every
7:34word absolutely needs, plus all the
7:36most-used experts that can fit.
7:38In the pantry, which adjusts itself
7:40based on your free RAM, it stores the
7:42next most-used ones, and everything else
7:44stays in the storage room and is read
7:46when a word asks for it. The experts not
7:49on the countertop are calculated by the
7:51CPU, reading them directly from the
7:52model files, and the GPU doesn't touch
7:54them. There are two more tricks that
7:57make the kitchen sufficient.
7:59The first is that the weights come very
8:01tightly packed.
8:02That is quantization. Saving each value
8:04with less precision so it takes up less
8:06space, in exchange for the model
8:08potentially losing some quality.
8:11The versions Strata uses come from a lab
8:13called Isthmus Lab, and they save each
8:15parameter in between 1.75 and 2.25 bits
8:18—the minimum units of information—when
8:20the original uses quite a few more.
8:24Thanks to that, the smallest download
8:26takes up 239 GB instead of 510, and the
8:292-bit version, which is used in almost
8:32all tests, takes 257.
8:35The second trick is a giant 111 GB
8:37lookup table in the 2-bit versions that
8:40stays in the storage room because only a
8:43few rows are read for each word. And
8:46here, the Strata guide warns about
8:48something that will keep coming up: that
8:50a slow disk slows down every word and
8:52the files must go on an NVMe.
8:55Now, the hardest concept of all this is
8:57why a more powerful GPU doesn't fix
8:59anything for you. And I'm going to
9:01explain it with an example I'm making up
9:02—let it be clear I'm inventing it just
9:04to illustrate.
9:05You write a line of code, and the model
9:07is going to generate the next word. In
9:09the first layer, the router asks for six
9:11experts. Five are on the countertop and
9:13one is in the pantry. In the second,
9:16four on the countertop, one in the
9:17pantry, and one in the storage room.
9:20And so on for 40 layers, one after
9:22another for every word.
9:24The fast cook finishes their tasks in a
9:26moment and sits idly by, because the
9:28word won't come out until the second
9:30cook has done their part with what came
9:32from the pantry and someone has brought
9:34up what was missing from the basement.
9:38The word travels only as fast as the
9:40slowest trip of each layer.
9:42The Strata model guide puts it even more
9:45clearly in a single line: 12GB cards
9:47spend most of the time for each token
9:49waiting on the SSD and the CPU, and the
9:51card is barely to blame.
9:54With that in mind, let’s go piece by
9:56piece. The first strange thing I found
9:59is in the Strata table itself.
10:01On the developer’s PC with an RTX 5070
10:04and 64GB of RAM, DeepSeek writes between
10:074 and 6 tokens per second.
10:11And on another PC, with a 12GB RTX 3060
10:13—which is an older, cheaper card, and
10:16with even less RAM, 46 GB—the same table
10:18shows between 9 and 11.
10:21The PC with the cheaper card runs nearly
10:24twice as fast, and the card has nothing
10:26to do with it.
10:28Others have run into the same thing,
10:30like a YouTuber who tested DeepSeek with
10:33an RTX 4070, got between two and three
10:35tokens per second and deemed it not
10:37recommended without explaining why, or a
10:40tester with an 8GB RTX 2080 and 64GB of
10:42RAM who only got 2.91 tokens per second
10:44and said it was too slow for daily
10:46coding.
10:48Using the kitchen analogy, it makes
10:50sense because, on those PCs, the fast
10:51cook is waiting almost the entire time,
10:53and swapping them for a faster one is
10:55little help if the basement and the
10:57second cook remain the same.
10:58There is another thing that lives on the
11:00countertop that is worth knowing.
11:02The context cache—what the model saves
11:04of what it has already processed to
11:06avoid repeating it, and which grows with
11:08every message in the conversation—is in
11:10the graphics memory.
11:12Every context position you reserve takes
11:14space away from the experts, so the more
11:16context you ask for, the fewer experts
11:18fit on the countertop and the more trips
11:20to the basement are needed. That is why
11:22Strata starts DeepSeek with 8,192
11:24context tokens, which is the size they
11:27have verified, even though the model’s
11:29specs talk about 1 million.
11:32You can increase it, but the guide
11:34recommends not exceeding 256,000 at
11:36first and warns that the first response
11:38with half a million or 1 million tokens
11:40can take many minutes.
11:43And yet, the countertop is the part that
11:46will cost you the least because for
11:47DeepSeek, Strata asks for a 12 GB card
11:49or more, and half the world already has
11:51that in their gaming PC.
11:54The pantry was the first thing Strata
11:56built for DeepSeek, and it shows.
11:59With version 0.2.44, the same one from
12:01October 24th, they added that expert
12:04store in RAM, and on the developer's PC
12:06, it went from between 1.8 and 2.2
12:08tokens per second, which is what it got
12:11reading raw files, to between 4.3 and
12:134.9; more or less double, just from
12:16having the pantry well organized.
12:20Where you can clearly see how much RAM
12:22weighs in is a test from the Strata
12:24guide for DeepSeek with a Radeon Pro
12:26W7900, a 32 GB card using a trimmed-down
12:29version of DeepSeek for code that I'll
12:31talk about later.
12:34They limited the RAM in each run, and
12:37with 40 GB it wrote 21 tokens per second
12:39, with 32 it dropped to 18, with 24 to
12:4215, and with 16 it sank to eight.
12:46Same card, same model, and the speed
12:48dropping to less than half just from
12:50emptying the pantry. And besides how
12:53much RAM you have, how fast it moves
12:55also counts. A user with two Radeon
12:58W7900s mentioned on r/LocalLLaMA that
13:00Strata would hang on long tasks when it
13:03sent experts to the RAM.
13:07And the top-voted answer told them that
13:09when you move things to system memory,
13:11the bandwidth of that memory—that is,
13:13how much data per second it can move—
13:15becomes very important, as does the CPU
13:17.
13:20In another thread, someone with an old
13:22Xeon PC and an RTX 3090 asked if a
13:24second-hand X99 rig bumped up to 256 GB
13:26of DDR4 would work, and they were told
13:28that with their three-module
13:30configuration, they should add a fourth
13:32to have more memory channels.
13:36Because Strata pulls a lot from system
13:37memory, and they would surely notice it
13:39.
13:40So what Strata asks of your PC for
13:42DeepSeek is primarily RAM.
13:44The model guide says 46 GB as a minimum
13:47and 64 GB as a comfortable size, which
13:49is what they have tested. Below about 40
13:52GB, it says it is not tested and that it
13:54runs slowly. And watch out, the
13:56installer warns you if you are missing
13:58RAM or disk space, but it doesn't stop
14:00you; it lets you install anyway, and
14:02then the experts that don't fit are read
14:04from the SSD, which works but much more
14:06slowly. The storage room is the part
14:08that surprised me the most because there
14:10is an improvement here that can be free
14:12. A user with a 16 GB RTX 5070 Ti, a
14:16Ryzen 9 5900 XT, and 64 GB of DDR4 had
14:20the model drive plugged into the chipset
14:24slot running on PCI 3.0.
14:28The link between the drive and the rest
14:29of the system.
14:31He moved the same drive to a PCI 4.0
14:34slot and DeepSeek went from 6.91 to 9.55
14:37tokens per second.
14:40Same drive, same PC, and nearly 40%
14:42faster just by changing its location. I
14:45bet there’s more than one person in the
14:47comments who has their drive in the
14:49chipset slot without knowing it. He
14:51measured why himself.
14:53Each token read about 260 MB from
14:56storage, and the drive on PCI 3.0 gave
14:59about 3,265 MB per second in his test,
15:02while on PCI 4.0 it reached about 7,100
15:05.
15:08If I do the math, just reading those 260
15:10MB in the slow slot is about 8
15:12hundredths of a second per word before
15:14calculating anything, and in the fast
15:16one, it’s less than half that.
15:18And on that same PC, bumping the RAM
15:22from 24 to 32 GB, it went from 9.53 to
15:2510.63.
15:26So both things add up.
15:29To give you an idea of what’s moving
15:32down there, another user who set up
15:35DeepSeek with a Radeon RX 7900 XT, a
15:38Ryzen 5800X3D, 64 GB of DDR4, and a PCI
15:413.0 NVMe, reported that a prompt of only
15:4410,000 tokens made it read 58 GB from
15:47the SSD.
15:4958 GB to read a short message, and
15:51that’s with a drive that isn't even one
15:53of the fast ones. What will cost you
15:56something is the space.
15:58The 2-bit version takes up 256 GB and
16:00asks for about 265 GB free.
16:03And the 2.5-bit one, which is
16:06recommended for code, is 274 GB and
16:09about 280 GB free. On a gaming PC with
16:12the drive full of games, which is
16:13typical, you have to make room, and it
16:16has to be on an NVMe.
16:17And there's the second cook, which is
16:19the one most often forgotten.
16:21Everything that isn't on the countertop
16:23is calculated by the CPU, and the
16:24developer's PC has a Ryzen with only six
16:26cores.
16:28My reading, and it is just an opinion,
16:30is that a good part of the difference
16:31with the 3060 PC comes from that and the
16:33drive, because the guide doesn't give
16:35more details about that other system's
16:37processor.
16:38There is a clue that supports this.
16:40Strata uses a trick called speculative
16:42generation, which works like this.
16:45A tiny model, the drafter, guesses the
16:47following words very quickly and the
16:49large model checks them all at once.
16:51So if it gets it right, you save work,
16:53and if it fails, you've spent a little
16:55effort for nothing.
16:57That effort also falls on the machine,
16:59and the Strata guide says its draft for
17:02DeepSeek, one called D-Spark, didn't
17:04provide any gains on the PC with the
17:065070 and six cores, so automatic mode
17:08leaves it off there.
17:11While on the 3060 PC, it went from
17:13between 6.1 and 6.7 to between 8.2 and
17:168.7 tokens per second.
17:19And there is one more detail that I
17:20found funny because it takes away
17:21another argument for expensive graphics
17:23cards.
17:24In the X99 thread from before, the
17:26top-voted reply from someone with an old
17:29Xeon on PCIe 3 told the 3090 owner to
17:32subtract 25%from any 3090 benchmark they
17:34found.
17:37You can have a great graphics card, but
17:39if the surrounding hardware is old, the
17:41card spends all its time waiting.
17:43So, for this model, the bottleneck has
17:46been shifting.
17:47At first, it was VRAM, then it became
17:49RAM, and right now it's on the disk and
17:52the processor. With that, we can finally
17:54get to what matters, which is how long
17:56it actually takes you to work with this
17:57.
17:58I'm going to do the math for a single
18:00coding agent step because that’s what
18:02most people would want DeepSeek for.
18:05There is a public log of real agent
18:07traffic on a server with DeepSeek V 4.1
18:10Flash from September 20th to the 25th,
18:12over 70,000 responses, and in a typical
18:15step, it sends about 2,700 new tokens
18:17and writes about 760.
18:21The rest of the prompt, which averaged
18:23over 121,000 tokens, was already in the
18:25cache. And Strata also saves the
18:27previous conversation to read only what
18:30is new. On the developer's PC, DeepSeek
18:33reads those 2,700 tokens in about 25
18:36seconds.
18:38Then it writes the 760 at about 5 per
18:40second, which is about 150 seconds more
18:42.
18:44Total, about 3 minutes per step, the
18:46length of a long song. With Qwen Flash
18:48Next on the same PC, the same step is
18:51read in a second and a half, and its
18:53default version writes at about 53
18:55tokens per second. So writing takes
18:57about 14 seconds, and the whole step
18:59finishes in about 15.
19:01And now for a normal task of the kind
19:03where people take 20 steps to fix
19:05something. With DeepSeek, it’s about 60
19:08minutes, a whole hour watching it work.
19:10And with Qwen, it’s about 5 minutes.
19:13Same PC, same task. We also have to add
19:16the startup time, which is where what
19:17bothered me most at the beginning
19:19returns.
19:20Each new agent session starts by reading
19:22its system prompt with the tools, and in
19:25OpenDevin that’s about 37,000 tokens.
19:28If I read it at the 231 tokens per
19:31second measured by the guide on the 5070
19:33PC with 8,000-token prompts, it’s about
19:36160 seconds before the first word.
19:39And on the 3060 PC, which reads at 149
19:42in that measurement, it’s about 248—
19:44more than 4 minutes.
19:46The user with the RX 9070 XT from before
19:49experienced exactly that, a 100-second
19:51wait every time he opened a session, and
19:54he summed it up as a data center wait
19:56without the data center.
19:59This same user later posted an Estrata
20:01branch with two changes so the session
20:03could reuse what was already read, and
20:05the title said sessions started 20 times
20:07faster, although he warned that it only
20:10works with AMD cards, the cut is
20:11approximate and without formal
20:13evaluation, and it changes the responses
20:15slightly.
20:18And another user with two RTX 5070 Tis
20:20and 64 GB said the first cold request
20:23took about 110 seconds, and in an agent
20:26test with several rounds, each execution
20:28took between 320 and 530 seconds with
20:31Estrata, versus between 280 and 400 with
20:34a 27-billion parameter Qwen, which he
20:36blamed on cache issues.
20:39He was told that Estrata does reuse the
20:41cache, but only within the same session
20:44. Time for an honest review of the
20:46headline. What holds up and what doesn't
20:47. What holds up is the main thing.
20:49DeepSeek V3 4.1 Flash runs on a 12 GB
20:52graphics card with Estrata, and just
20:54recently, I myself said that wouldn't
20:56happen. What falls short starts with how
20:59the tests are counted.
21:01On October 28th, a post appeared on r/
21:04LocalLLaMA boasting about DeepSeek at 41
21:07tokens per second on a single RTX 3090.
21:11The top-voted reply was to quote the
21:14title and add, "Yes, on a single 3090,
21:16and with 192 GB of RAM."
21:19Another person replied that it was fair
21:22, but that it also runs at reasonable
21:24speeds with 128 GB, and the post's own
21:26table showed 33 tokens per second with
21:28128.
21:30And it also used a third-party engine
21:32built on top of Estrata. The graphics
21:34card was one, okay, but the pantry was
21:37three times that of a well-equipped
21:39gaming PC.
21:40And that is exactly what is most debated
21:42in the comments on this topic: that
21:43perhaps an expensive graphics card is no
21:45longer needed, but RAM is now expensive
21:46.
21:48The second thing is what I’ve already
21:50told you, that almost all the speeds
21:51come from a single PC—the developer’s—
21:54and that the Estrata tables don't even
21:56match each other.
21:58The README gives the 5070 PC about 110
22:01tokens per second with a 4,000 prompt,
22:03and the model guide 231 with an 8,000
22:05one, while on the 3060 it's the other
22:08way around.
22:10One table gives 155 with 4,000 and
22:12another 103 with 8,000, and the times
22:14per step I've given you are my own
22:16calculations based on those figures, not
22:18a full end-to-end test. So take them as
22:21an idea of the order of magnitude. The
22:24third thing is a small correction, but
22:26one that changes a comparison. The model
22:28Strata launched with was counted as 125
22:31billion parameters, and that's where the
22:33claim that DeepSeek was about six times
22:36larger came from.
22:37In the Hugging Face registry, Qwen 2.5
22:40Flash Next appears with nearly 180
22:42billion, so DeepSeek is more or less
22:44four times larger, which is still a lot
22:47, but not six.
22:49And the fourth point is about Llama.cpp
22:51, which is the reference engine for
22:53almost everything local.
22:55When Strata came out, it was said to be
22:57more than four times faster than
22:59Llama.cpp when writing with Qwen, and
23:00that has become outdated.
23:03On October 7th, Llama.cpp incorporated
23:06its own expert cache on the GPU, and in
23:08a test on the same PC, Strata wrote at
23:1186 tokens per second versus 58, so the
23:13lead has dropped to one and a half times
23:15.
23:17Reading long prompts. Yes, it is still
23:19way ahead.
23:202,320 tokens per second versus 451, a
23:23bit more than five times faster.
23:27And Llama.cpp added DeepSeek V3 Flash on
23:30October 25th, but without the SSD reads
23:32from Strata.
23:35So for now, it's a path for machines
23:37with a lot of memory, and on a 12GB
23:38graphics card with 64GB of RAM, it's
23:40left out.
23:42The obvious objection to the 3 minutes
23:44per step is that DeepSeek is smarter, so
23:46it should need fewer steps and
23:48compensate.
23:50The problem is that this advantage was
23:51measured with the full model in the
23:53cloud, and at home you are running a 2-
23:55bit version.
23:57The Strata guide measures quality
23:59against the original with nearly 5,000
24:01test texts using perplexity, which is
24:02essentially how much the correct text
24:04surprises the model, and the higher it
24:06goes, the worse it is.
24:09With the 2-bit version, perplexity rises
24:128.6%in code and 5.3%in agent work.
24:16With the 1.75-bit one, the smallest, it
24:20rises 8.9%in code.
24:22But in normal English prose, it shoots
24:25up by 53%.
24:26And with the 2.25-bit one, it stays at
24:293.2%in code.
24:31That's why the guide says if it fits on
24:33your drive, take that one for
24:35programming. On top of that, on cards
24:37like ours, there's another cut that
24:39comes applied.
24:40DeepSeek chooses six experts per word,
24:43while the Strata fast mode uses five.
24:46The guide says it’s a minor quality
24:48change that’s usually unnoticeable and
24:50comes enabled by default on any PC where
24:53the graphics memory is 16 GB or less.
24:55If you want all six, you have to request
24:57it.
24:58To be honest, nobody has measured
25:00whether the 2-bit home version still
25:02needs fewer steps than Qwen to finish a
25:03task, so I wouldn’t count on it.
25:07The closest thing I’ve seen is a user
25:09test that ran a 100-question subset of
25:11the Humanity’s Last Exam on a 2.5-bit
25:14Project Maya model on a large PC against
25:16the DeepSeek API.
25:18Out of the 74 the API completed, the
25:21local model got 19 right and the API 21
25:24, and the local execution took about 12
25:27hours.
25:29He himself warns that it’s a small
25:31sample and results vary from run to run
25:33, but it seems the shrunken model holds
25:35up quite well, albeit taking its time.
25:39And the last part of the review is about
25:41Strata itself.
25:42It has been said that its developer
25:44wrote about four out of every five
25:46commits. I looked at the repository
25:48contributions and one account has 4,734
25:52and the next one has 35.
25:55So in practice, it’s one person with 458
25:58open issues and 584 pull requests
26:01waiting.
26:03That has a consequence that we’ve
26:04already seen. Version 0.2.7 introduced a
26:07speed improvement for RTX cards with 12
26:10GB or less. which on a 3060 made stories
26:1415%faster and code 37%faster.
26:18On October 29th, version 0.2.8 turned it
26:20off by default because users and the
26:23developer himself linked it to repeated
26:26text in long sessions. You can turn it
26:28back on, but only if you know what
26:30you’re risking. And there are two known
26:32bugs that you should care about if
26:34you’re going to use it with an agent.
26:36With zero temperature, which in theory
26:38is for always getting the same response
26:40, five identical requests to DeepSeek
26:42can produce up to five different answers
26:44, and they attribute it to the expert
26:46cache.
26:48And with only 13%of the model in memory
26:50, in one out of every five runs it
26:52inserted a weird character inside the
26:54arguments of a tool call, which is
26:55exactly what breaks an agent.
26:58While with the full model in memory, all
27:00eight calls went through fine. The story
27:02of where this engine came from is quite
27:04funny. On October 20th, Strata imported
27:07the DeepSeek engine as-is from another
27:09project, Project Maya, a literal copy of
27:12its version 1.1.17.
27:14And Maya, in turn, was born from the
27:16Strata engine, so much so that it still
27:18accepts the old name in one of its
27:20options. Strata brought it along with
27:22its credit and has continued to carry
27:24mesh changes on top.
27:26This moves very fast for better or worse
27:27, and it’s best to keep that in mind
27:29when using it.
27:30Now, about the money—what really makes
27:32the decision? If your graphics card is
27:34already 12 GB, the card costs you
27:36nothing, and that's true; what costs you
27:38is the pantry. When I looked at RAM
27:41after Strata came out, a 64 GB DDR5 kit
27:45was around $ 1,100 on a US price tracker
27:49, and as of November 3rd, it's at 1,360
27:53, at 21.25 per GB.
27:55If your board is DDR4, you’re luckier,
27:59because the 64 GB kit is at $ 552, at
28:028.62 per gig. And in general, the
28:05tracker lists DDR4 as 56%cheaper per gig
28:09than DDR5.
28:10And now I’ll compare that to buying
28:12nothing and using DeepSeek in the cloud
28:14.
28:15The API price list for this same model,
28:18off-peak, is 0.15 per million new input
28:21tokens, 0.000 per million for those
28:24already cached, and 0.6 per million
28:27written tokens.
28:29And during peak hours, it’s double. With
28:31the typical step from before, 2,700 new
28:35tokens, about 121,000 cached, and 760
28:37written. The step comes out to about
28:400.0012,
28:42a tenth of a cent.
28:45If a day of intense work is 200 agent
28:47steps and you do it for 30 days a month
28:49, which is a lot, it comes to about $
28:517.34 a month.
28:53With that, the $ 552 DDR4 kit is
28:56equivalent to about 6.3 years of cloud,
29:00and the $ 1,360 DDR5 one to about 15.4
29:04years. It has been said out there that
29:07the cheapest kit for about $ 870 is
29:10equivalent to 9 years at about $ 8 a
29:12month, and with my 7.34, it’s closer to
29:1510.
29:16And this is without counting the disk if
29:18you need one, which I haven't priced, or
29:20the electricity, which I haven't
29:22measured.
29:23But what impresses me most is the time.
29:26Those same 200 steps a day at 3 minutes
29:29each in local DeepSeek is 600 minutes,
29:3110 hours of waiting a day.
29:34In local Qwen, at 15 seconds per step,
29:36it’s 50 minutes, and in the cloud, every
29:39step responds in seconds. And if what
29:42you want is for it to write a lot, a
29:44million output tokens at 5 per second is
29:46about 55 and a half hours, more than 2
29:49days straight with the PC at full tilt,
29:51versus 60 cents on the API. So the
29:53numbers make it quite clear. Running
29:55DeepSeek at home is worth it for other
29:57reasons: because you don't want your
29:59code leaving your machine, because you
30:00want it to be yours, or because you
30:02don't want to depend on anyone. And
30:04those are very respectable reasons, but
30:06if you only look at the money, as of
30:07today, it doesn't pay off. All of this
30:10assumes an Nvidia with 12 GB and 64 GB
30:12of RAM, and your PC might be different.
30:16If you have 32 GB of RAM, which is
30:17standard for a gaming PC, you’ll be left
30:19without DeepSeek because Strata requires
30:21at least 46, but you won't be left with
30:23nothing.
30:25You have Qwen 2.5 32B, which is a
30:27smaller model.
30:29The file takes up about 23 GB. And in
30:31the developer's test, it wrote at about
30:3460 tokens per second on an RTX 3060 with
30:37most experts offloaded to the CPU.
30:40And there is a version of Qwen 2.5 Coder
30:42that works with about 32 GB of RAM and a
30:45decent graphics card.
30:48A user with an RTX 4070 Super with 12 GB
30:50and 32 GB of DDR5 used Qwen 2.5 Coder
30:53with 256,000 tokens of context and it
30:55wrote at 40 tokens per second, although
30:58they said reading the prompt at that
31:00size felt slow.
31:03If you have a Mac with Apple Silicon and
31:0664 GB or more, DeepSeek works with text
31:08only according to the guide, although
31:10version 0.2.10 already adds images.
31:14On an M3 Max with 128 GB, Strata's notes
31:17say it writes between 1.6 and 1.9 times
31:20faster than llama.cpp on the same files
31:23.
31:24With AMD, it depends a lot on the card.
31:27An RX 7900 XT that was around $ 750 in
31:30October runs Qwen at 80 tokens per
31:32second on Windows according to one user
31:34.
31:36For DeepSeek, the guide only lists the
31:39Radeon Pro W7900 on Linux and Ryzen AI
31:41GPUs on Windows.
31:43And version 0.2.10 clearly states that
31:45for the rest of Radeon cards on Windows
31:47, DeepSeek still doesn't have an engine
31:49.
31:50The 7900 XT user I was telling you about
31:52got it working via AMD ROCm and doesn't
31:54say which system, so I wouldn't take for
31:57granted that it will work for you on
31:59Windows.
32:00With Intel Arc, official support on
32:02Linux arrived on October 7th thanks to
32:04the work of someone in the community,
32:06and it only lists Qwen 2.5 models.
32:08No DeepSeek. The B580 with 12 GB costs
32:12about $ 250.
32:15And the only data I've found is from a
32:17user who, through the Windows Linux
32:19subsystem, got about 21 tokens per
32:21second and occasionally lost the card.
32:25And if your graphics card has less than
32:2712 GB, you already saw the test with the
32:298 GB 2080, less than three tokens per
32:31second; the guide asks for 12, and I
32:33wouldn't try it with anything less.
32:35At the other extreme, if you have a PC
32:38with 128 GB or more, things change in
32:40your favor quite a bit.
32:42That's where the 33 tokens per second
32:45with 128 GB come in, along with the
32:4741.19 from the 3090 publication, which
32:50on a PC like that are already
32:52production-ready speeds. And with a lot
32:54of RAM, Strata doesn't necessarily have
32:56to be the best.
32:57Three users who tested Strata and
32:59Project Maya on the same PC, all with
33:01significantly more RAM than a gaming PC
33:03, saw Maya write between 1.2 and 1.4
33:06times faster in all three cases.
33:08On a 12 GB card, Maya has much less to
33:12show.
33:13Its guide doesn't give figures for that
33:15size, and Strata also has a smaller
33:17download and a desktop app for Windows,
33:19Linux, and Mac.
33:21So for a normal gaming PC, go with
33:22Strata, and if you have memory to spare
33:25, it's worth trying both.
33:27There is also the trimmed version for
33:30code, the Insta Dust Coder, which keeps
33:32half the experts, 192 out of 384 per
33:35layer, and takes up 191 GB.
33:38On the 32 GB R9700, it wrote about 26
33:41tokens per second, and on an M5 Max,
33:44between 19 and 22.5.
33:47According to the lab, in SL Bench
33:49Verified, it scores a 77 compared to the
33:5281.2 of the base model.
33:54In the thread where they announced it,
33:56someone asked what the point was if a
33:58much smaller model gives better results
34:00, and they replied that the Deepsck
34:02figures were measured with half the
34:04reasoning effort and the Quen ones at
34:07maximum, so the comparison wasn't clean
34:09.
34:09I see it as an interesting option if you
34:11have card space to spare, but I don't
34:13have figures for 12 GB.
34:15With all this, I'll leave you with how I
34:17would look at it in order before
34:18spending a single euro.
34:20The first thing is RAM because it's the
34:21deciding factor. If you have 64 GB, you
34:24can already try Deepsic as is.
34:27And if you have 32, start with Quen
34:293.635 B or the code version of Quen
34:32Flash Next and think if it's worth
34:34upgrading by checking if your board is
34:37DDR4 or DDR5, because the price
34:39difference is huge.
34:41Second, the disk.
34:44It has to be an NVMe with about 265 GB
34:46free for the 2-bit version or 280 for
34:49the 2.25 one.
34:51And first of all, check which slot it's
34:53in, because if it's in a PCI 3.0 or
34:56chipset-linked slot and your motherboard
34:58has another free PCIe 4.0, moving it can
35:01give you almost 40%for free.
35:03Third, the processor. With six cores,
35:06DeepSeek runs, but it's tight and it's
35:08the part that will slow you down the
35:10most after the disk. Fourth, the
35:12graphics card, which is the last thing I
35:14would look at.
35:15With 12 GB or more of Nvidia, from the
35:1730, 40, or 50 series, you're in, as well
35:20as with AMD or Intel. Check the specific
35:22list for your card. And if you're going
35:25to buy something, a second-hand RTX 3060
35:27for about $ 320 was the graphics card of
35:29the fastest PC with DeepSeek in the
35:31Strata table, with between a minute and
35:34a half and 2 minutes per step, and it
35:36also runs all the models on the list.
35:39Fifth is, what do you want it for?
35:42To work with a Qwen agent at about 15
35:44seconds per step and DeepSeek for that
35:46big question you launch and let run
35:47while you do something else.
35:51If you're going to use it for coding,
35:53the 2.25-bit version if it fits, context
35:55under 256,000 to start.
35:59And if you miss the speed boost they
36:01turned off on your 12 GB RTX, you can
36:03turn it back on in models, in the native
36:05engine settings, under speed, knowing it
36:07might repeat text in long sessions.
36:10When Strata launched, it ran a single
36:13model, only on Nvidia and never on Mac,
36:15so it left a lot of people out.
36:18A month later, that list has almost
36:19disappeared, and the only thing
36:20remaining is the RAM, which is actually
36:22more expensive than it was then.
36:25What I said a few days ago to those with
36:2712 GB graphics cards is only half true
36:29because DeepSeek does fit, but to work
36:30with an agent, I would still use Qwen,
36:32and what has changed for me this month
36:34is where I look.
36:36If you take one sentence away, let it be
36:38this one. Before changing your graphics
36:40card, check your RAM, your disk, and
36:41your processor, because that is where
36:43the limit is now.
36:44And since almost all the figures there
36:47come from a single PC, the ones you
36:48leave in the comments with your graphics
36:51card, your RAM, and your disk will count
36:53for more than any table.
36:55If you want to try it today, install
36:56Strata, which doesn't require an account
36:58, with the application or with "strata
37:00launch" in a terminal. Start with Qwen
37:02and pay attention to what the installer
37:04tells you about your RAM and your disk.

About this transcript

This page contains the full transcript of ¿Ya No Necesitas Una GPU Cara? Strata Cambia La IA Local by Jorge Marfil, generated from the public captions YouTube serves with the video. The transcript has 6,577 words across 991 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.