Riesiges KI-Modell auf einer RTX 3090: 116 Token/s mit Strata! LLM Benchmark mit Qwen 3.8 Flash Next — Transcript
Full transcript
- 0:00If you have a consumer PC and you want
- 0:02to run a really awesome AI model on it,
- 0:04then you definitely need to watch this
- 0:06video. With that said, hello and
- 0:10welcome to today's video, because even
- 0:12if you might not believe it, what I
- 0:14have here is not a special AI server
- 0:17like you usually see in my videos, but
- 0:19rather, let's take a look, a completely
- 0:21normal consumer PC in a rack case. What
- 0:25exactly is inside this server? Well,
- 0:27hardware that you probably have in your
- 0:29own gaming PC at home to play Fortnite
- 0:31and other games. A 750-watt power
- 0:35supply, nothing special. A very simple
- 0:38AM4 CPU, no idea which series,
- 0:40definitely not a high-performance one.
- 0:43Plus 32 GB of RAM and an RTX 3090.
- 0:48There’s nothing more to it than that.
- 0:49So, in the end, everything that you
- 0:50would also have yourself. I'm checking
- 0:52right now, does it have an NVMe? Yes.
- 0:54Oh, it even has an NVMe, I have to
- 0:56mention that. So, it actually has a
- 0:58decent amount of power. The difficulty
- 1:01now with the 3090 is that it only has
- 1:0424 GB of VRAM. That's a fair amount,
- 1:07relative to the consumer average, but
- 1:09when we look at the AI sector, we all
- 1:11know that's nothing. It's ridiculous.
- 1:15But someone has brought something
- 1:17really cool to the market called Strat,
- 1:20and with Strat, it's possible to run
- 1:23really large models on such consumer
- 1:25PCs. Although, "really large" is
- 1:28actually incorrect. Currently, it only
- 1:30works with—yes, over there—Qwen 1.5
- 1:32110B. That is what is runnable with it.
- 1:35A model that normally requires around
- 1:38180 GB of memory. Yes, and you can
- 1:42comfortably run that here on a 24 GB
- 1:45graphics card, and supposedly with
- 1:47quite okay performance, too. And I’d
- 1:51say we take a look at the whole thing,
- 1:53evaluate it, and benchmark whether
- 1:54it’s actually true and if we can
- 1:56really get it to perform that well. And
- 1:59we are talking here about a 110 billion
- 2:03parameter model on an RTX 3090. This
- 2:07means we are trying to pack a model
- 2:10that is so big into such a small amount
- 2:14of VRAM, which is quite an exciting
- 2:17story, and especially the Qwen 1.5
- 2:20model is very powerful. So, anyone who
- 2:23knows my channel knows that I’ve
- 2:24already set up a complete server with
- 2:26it and built entire programs with it.
- 2:28So, as far as that goes, you can really
- 2:30get a lot done with it. Of course,
- 2:32it’s also clear that it has certain
- 2:34limits, one has to say. Especially
- 2:36regarding the commercial license,
- 2:38that's more about the licensing, but
- 2:40software-wise it is super, super
- 2:42exciting. And if you're interested in
- 2:44the whole thing, definitely check out
- 2:45my channel. That's where exactly that
- 2:47video is, a Prompt Browser Game. Check
- 2:49it out. I was super surprised at what
- 2:52this AI model built here, and being
- 2:54able to run the model on a 3090, where
- 2:57I previously needed an RTX 6000. Yeah,
- 3:00if that works properly, then it is
- 3:02super, super good. Now you always have
- 3:06to add, and for me, the whole thing is
- 3:08a bit like Hummingbird. And by
- 3:10Hummingbird, I don't mean the one you
- 3:13might know from me, Hummingbird 1 from
- 3:16Aleph Alpha, the AI model from Germany,
- 3:19but I mean the Hummingbird, uh, where
- 3:21it was said: Yes, you can run giant AI
- 3:24models—there's news about it—on a
- 3:2724 GB laptop. Ultimately, exactly the
- 3:31same news that you read about Strater,
- 3:33because for Strater it is also said
- 3:35that you need 32 GB of RAM, 12 GB of
- 3:37VRAM, and it works. The deciding factor
- 3:40is, and you always have to see this,
- 3:42the hype that is created around things
- 3:44that aren't even there. Because with
- 3:46Hummingbird, it was the case back then,
- 3:49if we take a look at the GitHub account
- 3:52, it was described exactly that you
- 3:54need, for example, 6 RTX 5090s to
- 3:57generate 9 to 9.2 tokens per second. I
- 4:00had tested it myself on an RTX 6000
- 4:02with 96 GB of VRAM and I think I got
- 4:050.1 tokens or something like that. An
- 4:08absolutely massively poor value, and I
- 4:10have to say, that made me a bit, I
- 4:12don't know, with Strater at the
- 4:14beginning, I thought, oh, this is just
- 4:16another hype story again. But I can
- 4:19tell you in advance: It is something
- 4:21that is really super exciting and
- 4:23interesting, and it is not a hype like
- 4:25Hummingbird, which is quite cool in
- 4:27theory in the end. But rather Strater,
- 4:30if you keep building on it, can be a
- 4:32foundation for really cool future AI
- 4:34projects. That's why we should first
- 4:37take a look at what Strater actually is
- 4:39and how it ultimately works. And so
- 4:41we'll dive a little bit into, um, AI
- 4:44models. How is an AI model structured
- 4:46and how do they work? We have here a
- 4:49125 billion parameter model. That means
- 4:54the model is huge and contains a lot of
- 4:56content, a lot of knowledge, and a lot
- 4:58of data, and until now, if you wanted
- 5:01to run something like this, you needed
- 5:03server-grade cards, but in this case,
- 5:05we're talking about a gaming PC. And
- 5:09now you have to imagine the model. That
- 5:12is also one of the reasons why this
- 5:13Strater. for example, currently only
- 5:15works with Qwen 38B, because it's a
- 5:18so-called MoE model, and for an MoE
- 5:20model, imagine all these dots as
- 5:23experts; in this case, it consists of
- 5:25almost 25,000 experts, namely this Qwen
- 5:2838B Flash Next. And not every one of
- 5:31these experts is always active, but
- 5:33only the golden dots you see right now,
- 5:35those are. for example, the active
- 5:38experts that you need. This means the
- 5:40entire model does not have to be loaded
- 5:43all the time; parts of the model are
- 5:45sufficient. The difficulty is just
- 5:47which parts of the AI model actually
- 5:49need to be loaded. And to make this
- 5:53whole thing performant, it's the case
- 5:55with Strater that they said: "Hey,
- 5:57we'll distribute the experts across
- 5:59four locations." On one hand, there's
- 6:03the graphics card; here are the experts
- 6:04that are constantly being queried, the
- 6:06ones you need all the time. One expert,
- 6:08for example, handles logic, another
- 6:10handles mathematics, another handles
- 6:12some knowledge topics. That's just a
- 6:14way you can imagine it a little bit.
- 6:15And if I'm writing source code all the
- 6:17time, I don't need an archaeologist who
- 6:19can tell me something about dinosaurs;
- 6:20instead, I ultimately need ones that
- 6:22are responsible for my specific field.
- 6:24This means I have the graphics card on
- 6:26one side, and the experts that are
- 6:28especially relevant to me are loaded
- 6:30there. All 25,000 experts are kept in
- 6:35the RAM. And that is also partly the
- 6:38reason why, if we go to the GitHub
- 6:41account of Strater. I will also show it
- 6:44myself in a moment. You just need at
- 6:46least 32 GB of RAM, because it's simply
- 6:49necessary for this technique to work,
- 6:51so that you can ultimately define this
- 6:54waiting room. The other thing, the
- 6:57processor, is also involved. It helps
- 6:59out, it handles some of the
- 7:00calculations. And on the hard drive,
- 7:02that is ultimately the directory of
- 7:04experts. But very little is loaded from
- 7:07there. That means the hard drive, which
- 7:10is a slow component, isn't quite as bad
- 7:11in this case, although it is still
- 7:13recommended to use an NVME. But exactly
- 7:16these four locations are what work
- 7:17together at this point. And how should
- 7:21I imagine all of this? So, whenever you
- 7:23generate a new word or a new token, the
- 7:25question is always: "Hey, which experts
- 7:27do I need for this? Then the experts
- 7:30are fetched, the calculation is
- 7:31performed, the whole thing is finished,
- 7:33and it starts all over again. And I
- 7:36just said, well, not all experts have
- 7:37to be loaded all the time. This means
- 7:39they reside in the graphics card, and
- 7:41partly come from the system memory.
- 7:43This is a technique already known from
- 7:46other models, but in this case, it's
- 7:48optimized even more precisely, and most
- 7:50importantly, only those ten experts
- 7:52that are needed at that exact moment
- 7:54are loaded. Now, it is also the case—
- 8:00and this is an additional factor—that
- 8:02normally, if you calculate four words
- 8:04or four tokens, you need four separate
- 8:06passes. That means pass one, I fetch
- 8:09the experts, I calculate, next token,
- 8:11pass two, fetch expert, calculate,
- 8:14token, and so on and so on. But this
- 8:17whole process also costs time, and that
- 8:20is......for example, one of the tricks
- 8:22that Strater uses, because Strater says
- 8:25:" Look, we don't go and reload word by
- 8:28word by word, but instead I do one pass
- 8:30, load all the relevant experts, and
- 8:33then, let's say, I finish my four items
- 8:35and then I reload again. "And this
- 8:38fetching of the experts consumes
- 8:40resources. And in that moment, it is
- 8:42said that you are 1.6 to 1.8 times
- 8:45faster. Because you ultimately fare
- 8:48better with a sort of preview of which
- 8:51experts you likely need and keep
- 8:53loading things in, even in the
- 8:54background, instead of doing it fresh
- 8:57in every single run, as just shown,
- 8:59over and over again. And that is
- 9:02ultimately the actual trick of the
- 9:03whole technique, that you manage to do
- 9:05this. Well, we distribute a lot into
- 9:08the RAM, use it in between, and then
- 9:10gain this resource advantage. Yes, and
- 9:15why is all of this so interesting? I
- 9:18mean, sure, I don't think I need to
- 9:19explain much to you guys. Uh, it’s
- 9:22generally about considering what RAM
- 9:24and VRAM cost these days. If you manage
- 9:29to run 125 billion parameter models,
- 9:32large models, with 12 GB of VRAM and 32
- 9:35GB of RAM, that’s an extremely grand
- 9:38thing—on one hand for the entire AI
- 9:40market, but also for you, because you
- 9:43can drastically reduce your ongoing
- 9:46subscription costs. I also realize that
- 9:49if you say," I want to code, I use my
- 9:51Cloud, I have my Astra, I need that
- 9:53quality, "then there’s no question
- 9:55about it. It’s the same for me; I use
- 9:58Qwen 2.5 Coder 32B for about 50%of my
- 10:01AI agents. I have over 40 agents, half
- 10:05of them are roughly Qwen 2.5 Coder 32B,
- 10:07and I am extremely happy. But I can't
- 10:10replace it completely, because it's
- 10:12clear that for large coding tasks, the
- 10:14frontier models are simply much better.
- 10:17Yes, and I mean I’ve already
- 10:18presented the requirements a few times.
- 10:20It’s just important for me to explain
- 10:23a few things, especially regarding the
- 10:25recommended hardware. So, one thing
- 10:27that is very important for you: you can
- 10:29run it with Nvidia or with AMD as well.
- 10:31However, 64 GB of RAM is recommended. I
- 10:35consciously used 32 GB in this test
- 10:37because, relative to consumer graphics
- 10:39cards, I have to say I didn't have
- 10:42anything else in the server I could use
- 10:44for it. That’s why it was the RTX
- 10:473090, but with 32 GB. I used an NVMe
- 10:50drive and ran it under Linux, but those
- 10:52are just the possibilities of how you
- 10:54can run all of this yourselves. And if
- 10:57we take a look at the results now, I
- 10:59have to say, it gets extremely exciting
- 11:01. Because it was important to me to say
- 11:04: Okay, generally, vLLM is great
- 11:06technology on a 3090 with all the
- 11:08requirements we just mentioned, like
- 11:10the Ryzen 5 5600X. I’ve now evaluated
- 11:14it again with a two-bit quantized model
- 11:17—I’ll explain that in a moment—to
- 11:20compare it with an AI server using an
- 11:23RTX 6000 Ada, with 96 GB VRAM and 128
- 11:26GB RAM available, though it only used
- 11:2960 GB and a four-bit quantized model,
- 11:33to see:" Hey, is there an advantage if
- 11:36I run this on hardware that can already
- 11:39run such a model? "So, I definitely
- 11:43included that in the benchmark. And
- 11:45very importantly, I also wanted to do
- 11:47it on the DGX Park, but here is the
- 11:49problem: it's ARM instead of X86. That
- 11:53means the architecture didn't work, and
- 11:55I don't have unified memory, meaning I
- 11:57don't have RAM and graphics card memory
- 12:00combined, but rather I'm using the
- 12:02memory that's available together, and
- 12:04the current setup isn't compatible with
- 12:06that, but we'll talk about that in a
- 12:08moment. Let's first talk about
- 12:11quantization. I chose both 2-bit and 4-
- 12:14bit quantization. And now you might be
- 12:17asking yourselves, what is the
- 12:19disadvantage? So, 2-bit quantization
- 12:22basically means the model is more
- 12:25compressed. This means it's no longer
- 12:28the original model in its large size,
- 12:30but you can think of it like drawers; a
- 12:32lot of information that was previously
- 12:34in separate small drawers has been
- 12:36packed into one, and the quality
- 12:38generally suffers a bit because of that
- 12:40. However, the 2-bit quantization was
- 12:43necessary to even be able to run the
- 12:44whole thing, because otherwise I
- 12:45wouldn't have been able to load the
- 12:47model at all. And that is also a bit of
- 12:49a disadvantage. So if you are operating
- 12:51the whole thing with the minimum
- 12:53requirements here, then you have to
- 12:54work with 2-bit quantization. But we
- 12:57will talk a bit about the quality later
- 12:59and say, okay, is the model even usable
- 13:01at all, or is it just plain stupid to
- 13:03the point where it doesn't work anymore
- 13:05? Now let's take a look at the
- 13:07benchmark in general and the results,
- 13:09and I have to say, it surprised me
- 13:11extremely. I didn't expect it. The 2-
- 13:14bit quantized model on a 3090 with 32
- 13:18GB RAM, 116 tokens per second. That's a
- 13:23terrifically good value, guys. That's a
- 13:25really good value, I can tell you. And
- 13:28I'll go to the website lmbenchmark.de
- 13:30for that, because that's where you can
- 13:32also look up all the benchmarks I do.
- 13:35If you filter by Nvidia GeForce RTX
- 13:373090 there, you will see this benchmark
- 13:39value right here. Also very important
- 13:41to note there. You can see I also did a
- 13:45benchmark value there with a 3-bit
- 13:46quantization, and you can see that the
- 13:48performance goes down significantly
- 13:50because it's not recommended by the,
- 13:52let's say, manufacturer. They would
- 13:55rather say, just take the 2-bit
- 13:56quantized one. I included it anyway so
- 13:58you have some comparison values and can
- 14:00see: yes, if you choose the models too
- 14:02large, then a lot is offloaded to RAM,
- 14:04then maybe the hard drive is partially
- 14:06used as well, and then the whole thing
- 14:08just gets slow. And you can really see
- 14:11that extremely when you look at the
- 14:14prefill here, i.e., this loading of
- 14:16data; it's already relatively slow. And
- 14:20so you can put it into perspective: I
- 14:22have another benchmark for comparison,
- 14:25which I'll show you, a Qwen 36, which
- 14:27is only a 35B model. So, mind you, with
- 14:31the Qwen 38, we're talking about a 180B
- 14:33model, and here we're talking about a
- 14:3635B model, and the 35B model is almost
- 14:39just as fast with 162 tokens compared
- 14:41to 116 tokens. So you can really see
- 14:45that where you might have only had
- 14:47small models before, you can now
- 14:49suddenly load a large model yourself
- 14:51and it just works incredibly,
- 14:53incredibly well. Therefore, the
- 14:55benchmark results for a single request
- 14:57are ultra-good, and for five parallel
- 14:59requests—meaning when multiple
- 15:00processes are running at the same time
- 15:02—they are still ultra-good; because
- 15:04that is usually the problem, that one
- 15:06request is fine, but as soon as two or
- 15:08three happen, it starts to fall apart.
- 15:11That is not the case here at all, and
- 15:13even with ten parallel requests, we get
- 15:151100 tokens per second—sure, not per
- 15:17request, you have to divide by 10 to
- 15:19get the individual value, but those are
- 15:22still ultimately good values. And this
- 15:25so-called prefill, this token
- 15:27processing—meaning you give it text
- 15:29and it processes it at 1400—is also
- 15:32an extremely good value. We can compare
- 15:35that as well. I'll switch over again.
- 15:37Let's look at a small model again. Sure
- 15:40, if I run a Qwen 36 on a graphics card
- 15:42like that, I get 3000 in prefill, but
- 15:44we have 1400, which is still a very,
- 15:46very good value, because others
- 15:48sometimes only have 400, 500, and so on
- 15:50, and for such a large model, it's
- 15:52bombastic, I definitely have to say.
- 15:56Let's take a closer look at the whole
- 15:57thing. Um, if we now look at the 3090
- 16:00at the lowest level, then it's exactly
- 16:04what I said. Um, we quantized it with 2
- 16:09bits, and that costs accuracy. Normally
- 16:13, you would use 4-bit quantization;
- 16:14that's actually the typical standard.
- 16:16That means the model isn't shrunk down
- 16:19any further; four is a typical value.
- 16:22Eight is basically the original that
- 16:24you would normally use. There is also
- 16:2716, but that’s only used if you need
- 16:29to train new models. So, in other words
- 16:32, 8 or 4-bit would be normal. And there
- 16:34is the danger that people say, well,
- 16:36the model is getting dumber, it's
- 16:38hallucinating, it's claiming weird
- 16:40things. Those are risks that you
- 16:41naturally have. I also took a look at
- 16:43what the general quality is like. But I
- 16:47didn't do an extensive long-term test.
- 16:48We'll take a look at the results in a
- 16:50moment. So you don't say, yeah cool,
- 16:53it's great that it runs, but it's just
- 16:54talking nonsense. It's definitely not
- 16:58quite that bad at the moment. Um,
- 17:01exactly. Why did I measure it this way?
- 17:02I already explained that quite clearly
- 17:03here. Ultimately, the problem is the 32
- 17:06GB, which is why I can only do it with
- 17:08that quantization. As I said, I also
- 17:11measured it with 3-bit quantization,
- 17:13but the values—especially when the
- 17:16prefill is, for example, at 106—you
- 17:18can imagine 106 tokens being processed
- 17:20per second. Especially if you want to
- 17:23process ultra-large documents, so 106
- 17:25words, let's say roughly around that
- 17:27mark. It just takes forever, and in the
- 17:29end, it's no longer usable. So,
- 17:31therefore, if you have more RAM, you
- 17:34can use the larger variants. Otherwise,
- 17:37Strater only recommends 4-bit if you
- 17:40have 96 GB of RAM. That’s just how it
- 17:43is. I also did this because I was
- 17:46interested: Does Strater offer a
- 17:49performance advantage if I have a
- 17:51graphics card with 96 GB of VRAM, where
- 17:54I could fit the 4-bit quantized version
- 17:56anyway? And so I compared it: I ran it
- 17:59with Strater and with normal Llama.cpp.
- 18:03You can see the values are almost
- 18:05identical. So, there is a minimal
- 18:08difference with one request. With ten
- 18:11requests, it is a bit faster with
- 18:13Strater. So I would say, on average,
- 18:15it's about the same. Yes, that means
- 18:19for you, if you have a graphics card
- 18:20where it already fits, then you don't
- 18:22have this deficit where you say, yeah,
- 18:23it doesn't fit, so you have to keep
- 18:25copying back and forth. Llama CPP also
- 18:28had techniques to manage expert layers
- 18:30and such itself. That means the
- 18:32bottleneck is somewhere else entirely.
- 18:34It brings you no advantage. So in that
- 18:35moment, you don't need to use Strater.
- 18:37So it only makes sense if you really
- 18:39have small graphics cards and if it
- 18:40just doesn't work any other way. That's
- 18:42where it gets exciting, too. Because as
- 18:44of today, Strater can only manage one
- 18:46graphics card. I will introduce the
- 18:48forks to you later. There are already
- 18:50Strater forks that can handle two or
- 18:52more graphics cards. You could look at
- 18:54that again, you could then check the
- 18:56whole thing to see to what extent it
- 18:58brings a performance advantage. If you
- 19:00have three graphics cards with, for
- 19:03example, 32 GB, you have 96 GB. Does it
- 19:05bring an advantage? I don't know. You
- 19:07definitely have to look at that
- 19:08separately again. Exactly, as I said,
- 19:10DGXPK unfortunately didn't work. Um,
- 19:12it's a different platform, so I will
- 19:15make a separate video about that
- 19:17because with DGXPK it's also exciting,
- 19:19is there a performance advantage, yes
- 19:21or no? But we will definitely make a
- 19:23separate video for that. I already
- 19:26touched on the forks a bit, you have to
- 19:28consider, um, it is now the 13th.
- 19:30October. The whole thing was just Must
- 19:32check GitHub briefly to see if I can
- 19:34find that out quickly. Um. I think
- 19:38there are already 48 releases now.
- 19:40Let's take a quick look. Let's look at
- 19:43one of the last releases to perhaps
- 19:45find out when Strater actually is from.
- 19:48So, you can see it, two weeks ago, no
- 19:50idea what. So, it's really not that,
- 19:53it's a relatively fresh model server.
- 19:57Exactly, that is probably the first
- 19:58launch. So the whole thing is two weeks
- 20:01old and there are already 2,300
- 20:02derivatives. So that is really quite
- 20:04something. A rating with 23,000 stars.
- 20:07So that, I think that says quite a lot.
- 20:09But what interesting forks are there? I
- 20:11can definitely recommend the one for V1
- 20:13to you. The dual GPU thing, that is
- 20:15definitely currently under development.
- 20:17There are others that work without Cuda
- 20:19so that it also works better on AMD and
- 20:22Intel. As I said, for DGX Park and so
- 20:23on, there are still others. There's
- 20:25another one for notebooks, there's one
- 20:27for older AMD graphics cards, and so on
- 20:29. So, these are just the best ones I've
- 20:31picked out for you. I really have to
- 20:34say, this is exciting, and I believe
- 20:36Strater will change a lot in the market
- 20:38because you can really use it
- 20:40effectively. Now, of course, the
- 20:45question remains, as I mentioned
- 20:46earlier, how big is the difference in
- 20:48quality? I took a rough—really rough
- 20:52—look at it and gave both, meaning
- 20:55the two-bit quantized version and the
- 20:58four-bit quantized version, identical
- 21:01tasks; in this case, 200 of them, and
- 21:04one achieved 99.5, the other achieved
- 21:07100. That means for one task, which was
- 21:11about grammar or something similar, the
- 21:13two-bit version messed up a little bit.
- 21:17The point is, it’s 0.5%. That doesn't
- 21:20sound like much. But you have to
- 21:22consider one thing: in a harness, like
- 21:24when using OpenCL or something similar,
- 21:26the thing runs in a loop the whole time
- 21:28. That means it does hundreds of tasks
- 21:33per hour, and if it gets a few tasks
- 21:36wrong here and there, then the overall
- 21:39results might end up being incorrect.
- 21:43So, it's hard to estimate. I think to
- 21:47really determine if the two-bit
- 21:48quantized version works, you'd have to
- 21:51do a long-term benchmark over several
- 21:53hours with really complex, large tasks
- 21:55that build on each other, like creating
- 21:57a website and so on. Maybe I'll make a
- 22:01separate video on that, where I
- 22:02generally compare these quantizations,
- 22:05because in a short-term test, I'd say
- 22:07it's okay. So, you can probably use it.
- 22:09You probably won't even notice it, but
- 22:11as I said, it would need to be looked
- 22:13at in detail. In general, who is
- 22:15Strater worth it for? Clearly, for
- 22:17everyone with small graphics cards,
- 22:19because the token results are wonderful
- 22:21. If you have enough VRAM, you don't
- 22:24need it. And with Unified Memory, you
- 22:26don't even need to use the normal,
- 22:27original Strater, because as I said, it
- 22:29won't add any value for you and it
- 22:31won't work. Yeah, so a question that
- 22:34definitely interests me: how much VRAM
- 22:35do you have on average? Um, maybe write
- 22:38it in the comments. It helps me in the
- 22:40future to categorize such videos and
- 22:42see, okay, does it make sense to
- 22:44perhaps install a different graphics
- 22:46card? Um, if you're interested in
- 22:48videos like these, then definitely
- 22:49leave a subscription. Please feel free
- 22:51to leave a like as well, because that
- 22:52definitely helps the videos reach more
- 22:54people. Yes, with that in mind, I'll
- 22:56say see you next time. Until then, and
- 22:57ciao.
About this transcript
This page contains the full transcript of Riesiges KI-Modell auf einer RTX 3090: 116 Token/s mit Strata! LLM Benchmark mit Qwen 3.8 Flash Next by Mario Alka, generated from the public captions YouTube serves with the video. The transcript has 3,974 words across 574 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.