YT YouTube2Text

YouTube2Text → Transcripts → The CUDA hour is Late. Also, Who Made the R9700 this fast?

The CUDA hour is Late. Also, Who Made the R9700 this fast? — Transcript

by Level1Techs · 2,893 words · 430 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Holy crap, if you have an R9700 and
0:03you've been messing around with the
0:04software,
0:06uh drop everything because the software
0:07thing has changed. Unless you already
0:08know about Radiance, in which case hello
0:10and welcome. We've got a lot to chat
0:11about. It's going to be a little bit of
0:12a ramble because Nvidia also just
0:14discontinued their 32 gig gamer GPU, the
0:185090, in favor of much more expensive
0:20GPUs. So, the other version of this
0:23video could be the CUDA hour is later
0:26than Nvidia imagines. We need to talk
0:28about this.
0:30>> [music]
0:35>> I really want to talk about how much the
0:37software has improved for the R9700.
0:41The R9700 hardware is now firing on all
0:44cylinders and it is
0:46uh hip and ROCm, but it is really
0:49Radiance. And Radiance, as far as I
0:51know, the the people behind Radiance are
0:53not really exactly AMD insiders.
0:57RDNA, CDNA
1:00AMD's always had the hardware, but the
1:01software has been a little bit of a
1:02challenge. At the same time, if we
1:04switch gears, talk about Nvidia for a
1:06second,
1:07Nvidia did this to themselves. So, when
1:09I say the CUDA hour is later than Nvidia
1:13imagines, Nvidia knows that.
1:15And it has to do with the software. A
1:18lot of people think of CUDA as a moat
1:20around what Nvidia is doing and the
1:23hardware and that it is difficult to
1:25adapt. And just because AMD has been in
1:28a little bit on the struggle bus with
1:30RDNA and AI in that context, that has
1:33sort of reinforces their opinion. Never
1:36mind, Intel is killing it with Int 4 on
1:38the B50 and there's some really
1:40interesting things coming out of Intel
1:42on that side as well. And Intel software
1:44and hardware have gone together pretty
1:46well. No, no, that the hour is late on
1:48CUDA because the tools exist now to be
1:52able to move really quickly.
1:54And the tools exist now to be able to
1:56remove really quickly also with
1:58Blackwell. See, Blackwell has also been
2:01a problem for people like me that have
2:04bought the RTX Pro 6000. It's a
2:06fantastic GPU, but that is not the same
2:09Blackwell. And the Blackwell that's in
2:11DGX Spark is not the same Blackwell
2:14that's in GB200 or GB300 or B200 or B300
2:18base systems. That is also Blackwell.
2:19That's enterprise Blackwell. It's
2:21basically unobtainium unless you're
2:23you're one of the hyperscalers or you're
2:25you're spending an an obscene amount of
2:26money. Uh the DGX Spark system that's
2:29laid out on my desk here with some PCIE
2:31upgrades that I'm attempting and farting
2:33around with. That's SM121. RTX Pro
2:366000's SM120.
2:38And that uses different math kernels.
2:40The hardware is a little different than
2:43what you get in B200 and B300. And
2:45Nvidia calls all of that Blackwell I
2:47think because there are some
2:48commonalities in terms of like NVFP4
2:50support and Nvidia rolling out NVFP4.
2:53Nvidia betting the farm that NVFP4
2:56on the DGX Spark is going to save it
2:58from the 300 GB per second of of memory
3:01bandwidth, you know, being
3:03relatively anemic. AI tools helping
3:06develop that kind of software has come
3:08through for SM120 time and time again.
3:12In my opinion, I don't think it Nvidia
3:13has put enough
3:15resources in developing
3:17SM120 because the amount of
3:20breakthroughs that the community has
3:21done for SM120 has been incredible
3:24compared to some of the releases from
3:26Nvidia. Also, local inference lab.ai
3:29just popped up on my radar and they have
3:31Flare. They're working on their own
3:32engine for SM120 to overcome some of the
3:35problems with with SM SM120. There are a
3:38couple of engineers that work at Nvidia
3:41on the VL LM side. It almost seems like
3:44the VL LM stuff is a moonlighting
3:46project for them on SM120. I mean, SM120
3:49just got it was like some kind of a
3:50fused matmul operator just recently and
3:53that really improves SM120 performance.
3:57And uh you know, some of the the labs
3:59behind DeepSeek and GLM deep seek 4.1
4:03flash and GLM 5.3. I've done a lot of
4:05Herculean things to be able to support
4:08the SM120 version of Blackwell.
4:10Um and contrast that to
4:12AMD. Like AMD has had CDNA and all of
4:15their dev work has been going into CDA.
4:17Relatively small amount of work has been
4:18going into
4:20RDNA. Although AMD has moved, you know,
4:23we did this demo with this gold system
4:25here which is currently configured with
4:26our two
4:27R9 700s. And from the initial system,
4:32like when these first came out and I
4:33first did a review with four
4:36RDNA R9 700s in one system, where we are
4:39today at the middle of October at the
4:42end of October in 2026 is absolutely
4:45bonkers insane. Check out the forum
4:46thread and like tokens per second and
4:48token generation and quality. We're
4:50doing qualitative and quantitative
4:51analysis on on what we have. And again,
4:54the AI tooling and like here's the
4:56hardware, here's the hardware
4:57documentation, here's what we have.
5:00AMD is much more open with what their
5:02hardware has on the RDNA side and on the
5:05CDNA side and what the the functionality
5:07is. And so tooling and AI models are
5:09better able to pick that up and and be
5:11able to run with it. And so I think
5:12internally Nvidia has sort of realized
5:14that and they haven't really wanted to
5:16help themselves along so they have it's
5:17like, oh you know, SM120 is good but
5:19it's too good. It's like the the DGX
5:21park aspects of it. It's like, oh we're
5:22worried about it being too good. So we
5:24won't put in a full size 2280 m.2 which
5:27is seems insane but why else would you
5:30not include a 2280 m.2? It makes makes
5:32no sense at all.
5:35Why would, you know, V100 V100s can run
5:38reasonable things thanks to the
5:40community. 6 7 8 9 year old GPU, the
5:44V100, can run relatively modern AI
5:46models 27 B dense. Nvidia doesn't really
5:48care about that. There's no money in
5:51doing that but the community is able to
5:52use tooling and sort of bring that
5:53forward. The community has used AI to
5:56bring forward this and a bleeding edge
5:59kind of a way. And I get why
6:02why AMD you be a little reluctant to
6:03come out and support that or even try to
6:05do the same approach as themselves, but
6:07then Nvidia has also experienced the
6:09same thing. Flare for SM120, not an
6:12official Nvidia project, and yet it's
6:15sort of crystallizing the shortcomings
6:17on SM120. And so if you look at Flare as
6:20evidence of the CUDA moat evaporation,
6:23then it changes the game a little bit,
6:25doesn't it? Because Nvidia created this
6:27situation. They they didn't make the
6:28software stack universal enough. Uh it
6:32is universal in the sense that things
6:33run, but it's not universal in the sense
6:35that things run as fast as they possibly
6:37can. And that was what AMD ran into as
6:39well. It's like you could get RDNA
6:42running with ROCm on R9 700, and even
6:45though they're only 600 GB per second
6:48memory bandwidth, the performance wasn't
6:50where you would expect it to be based on
6:52that. Until now. Uh you know, the
6:54Radiance engine on that side, RDNA
6:57strict Halo.
6:59Halogen. Halogen is another similar
7:01project.
7:03Totally different software stack. It's
7:04like, let's vibe code this, let's put
7:06this together. Um some of this would be
7:09uh in a official way you describe
7:11described as like hill climbing. It's
7:13like, hello algorithm. Our hill that
7:15we're going to climb today is this is
7:16the hardware that we have. This is the
7:18brass tacks of what the performance
7:19should be as we iterate over these
7:21models. What's stopping us from getting
7:24there? And it can hill climb
7:27to get there and explain it in a
7:28reasonable way. It doesn't it doesn't
7:29necessarily produce incomprehensible
7:31code if you don't want it to. And so in
7:33all of this, you know, the the Nvidia
7:36CUDA moat is basically already gone. Now
7:38to be clear, that doesn't mean that
7:39there's not a bunch more moats that
7:40Nvidia has. Nvidia actually really does
7:42have a lot more moats, but they're
7:44they're playing it kind of close to
7:46their chest and they really hope nobody
7:47notices. Things like if you have a
7:50thousand GPUs and some of them are idle,
7:52better scheduling across GPUs, better
7:55job management. Run AI was recently
7:57open-sourced, and that's so that they
7:59can help, you know,
8:01uh
8:02protect the moat. Uh hugging face, even
8:05if Nvidia doesn't have any protect the
8:07moat motivation in Hugging Face, they're
8:10going to have excellent data and
8:11telemetry from the Hugging Face
8:12acquisition as to what people are trying
8:14to do, what people are building, what
8:16what's actually going on. They can keep
8:18their finger on the pulse of the
8:19community. And so, it doesn't matter
8:21that the CUDA mote is actually already
8:22gone if you're paying attention because
8:25randos should not be able to bang out
8:27math kernels that maximize the hardware
8:30utilization of GPUs like the R9 700. And
8:33yet, they have. And that is
8:35transformational. You also see it in
8:36like vibe-coded software. And yeah, the
8:38vibe-coded software isn't as good as the
8:41state of the art, but think about word
8:43processing or photo manipulation
8:45software or spreadsheet software 10 or
8:4715 years ago. If you could have that
8:49today and it was basically free, that's
8:51where we should be. It's just that the
8:53ecosystem, not enough programmers. But
8:55now, we can. And enough people will get
8:57together and enough people will pool
8:58their tokens together and their
9:00expertise, and we will actually get
9:02reasonable, completely free, open-source
9:05software that is at least as good as
9:07software 5 or 10 years ago. And I think
9:09they're probably going to close the gap
9:10on 2 3 4 years. So, how much of a lead
9:13does Nvidia have with their software
9:15ecosystem? It's It's basically
9:16everything not at the point-blank range.
9:18It's everything at the macroscopic
9:20scale. And that's what AMD and and
9:22certainly Intel are going to struggle
9:24with over the next three or four or five
9:26years because of the head start that
9:28Nvidia has in that scale of dealing with
9:34AI and projects and everything else.
9:36Not not just like, "Oh, point-blank
9:38generate tokens and manage users." But
9:40project lifecycle and robotics and
9:42vision and all of the stuff that goes
9:44with that. And
9:46you know, self-driving cars and all this
9:48other kind of stuff. And in that
9:50macroscopic scale, there's a lot more
9:54stuff that Nvidia has, I think, for
9:56their mote. But point-blank math kernels
9:58at a hardware level,
10:00it ain't it.
10:01Uh one other data point I'll give you.
10:03This is our GMK Tech X5. This is 192 GB
10:07on the the
10:08Framework has a similar machine. There's
10:10a There's a some stuff in the level one
10:13tech review pipeline. But check out the
10:15review on this if you get a chance. It's
10:16basically the Strix Halo platform. It's
10:18Gorgon Halo. But 192 GB of memory
10:21instead of 128. It's basically the same
10:24memory bandwidth as
10:26DGX Spark. It doesn't have a QSFP or 100
10:28gigabit network interface. It has 80
10:30gigabit Thunderbolt 5. That is not as
10:33much as I want that to be a replacement
10:35for QSFP, it is insanely not. And I've
10:40done the work. I've done the kernel
10:42hackery. I've done a bunch of stuff on
10:45Thunderbolt 4 and Thunderbolt 5. And
10:47there's a lot of people that want you to
10:49believe that you can do insanely fast
10:51low latency networking over Thunderbolt.
10:53And you can do fast lower latency over
10:56Thunderbolt, but it is not anywhere near
10:58the same class as QSFP. And I do think
11:01that that was short-sided on AMD's part
11:04to not bake in some kind of high-speed
11:06networking for this for the model. That
11:08said, this is 192 GB with the same
11:11memory bandwidth. And [snorts] the
11:13software gap has closed now. This piece
11:16of hardware was not even really designed
11:18for AI. It just has absurdly fast memory
11:21and a lot of it locally. The reason it
11:24has absurdly fast memory is for gaming
11:26and like APU type use cases. For a
11:28gaming laptop, APUs in general are the
11:31future. It's It's It's It's too
11:33complicated and too annoying and too
11:35dumb to manage discrete GPUs and a
11:38portable low power device when we've
11:40got, you know, some of these new Panther
11:42Lake laptops that are like 7 W
11:44or 6 W when you're using it
11:46interactively and it's it's so insanely
11:48low power. Like you're just You're never
11:50going to get there with a dGPU. It's
11:52It's hard to do that even with an APU
11:53that has this much this much horsepower.
11:55And Intel basically has a high GPU
11:58horsepower APU that has the power
12:01problem solved basically in the other
12:05ver- the LPDDR version of Panther Lake.
12:07So this from from AMD with 192 GB
12:11has already basically leapfrogged
12:13Nvidia. I mean, at 16 cores that are x86
12:16versus 20 arm cores, there's only a
12:18couple of benchmarks where the 20 arm
12:20cores pull ahead of the x86 cores, and
12:23almost all of the benchmarks where the
12:25x86 cores fall short of the arm cores
12:28have to do with lack of benchmark
12:30optimizations or the fact that there's
12:32like 72 ways to do things on x86, and
12:36asking a human being to figure that out
12:37is too onerous. But, guess what? The AI
12:40applies there. That doesn't have
12:41anything to do with CUDA or anything
12:43like that. It's just that there's been
12:44enough software breakthroughs on the
12:47Strix Halo side of things that Strix
12:48Halo in 2026, launching at the end of
12:502026, the software gap has more or less
12:53been closed. And Halogen
12:56really gives you a run for the money on
12:59the DGX Spark platform. Also,
13:02you know, ROCm SP4, again, community
13:06efforts doing a lot of the heavy lifting
13:08here. 4-bit formats that run reasonably
13:12well on Strix Halo that was never
13:13designed for it, whereas DGX Spark
13:16really was designed with
13:18with you know, NVFP4. At the time that
13:21I'm doing this video, there's kind of
13:22been a refresh of NVFP4 that is a sort
13:24of lower loss version of NVFP4. But, if
13:27you dig into that and you look at the
13:29get commits behind that project and the
13:31stuff that you see on GitHub, that is
13:33almost all community driven. Nvidia is
13:35not focusing on that kind of stuff
13:38because this is old news. This is
13:392-year-old core. They need to get the
13:41Vera Rubin hardware out the door for
13:43enterprise customers and everybody else,
13:45and probably also a Vera Rubin version
13:47of Spark. I would say that a Vera Rubin
13:48version of Spark is overdue at this
13:50point. Remember, Jensen said, "Yearly
13:52cadence." "Oh, global macroeconomic
13:54conditions." Yearly cadence. So, again,
13:58another data point, the hour for CUDA is
14:00is later than people imagine when you
14:01can just
14:03like here, I'm going to run this thing
14:04and do this with it. And if that's what
14:05you want to do, you can do it, and you
14:07can do it competently with a very small
14:09team.
14:10So, Nvidia discontinuing the 5090,
14:13they're probably going to do a 24-gig
14:15version of the 5080, and that's that's
14:17cool. I mean, that's something.
14:19But
14:21these GPUs are available now, and
14:24they're 32 GB, and they do a reasonable
14:26job with AI now, where
14:29uh once they struggled a little bit
14:30before. Yeah, the memory bandwidth and
14:32the level of compute is not on the same
14:34level as a 5090, but if you're going to
14:36buy a GPU for AI, and you need 32 gigs
14:39of VRAM,
14:40these cost a fraction of the 5090, and
14:42it has always been a relatively small
14:44fraction, even when the 5090 was
14:46supposed to be two grand.
14:48And now they're like, you know, five or
14:49six, which seems absurd. These are on
14:51the order of like 16 to 1800 dollars.
14:55MSRP was was lower.
14:58You know, be that as it may, 32 GB here
15:03with pretty awesome gaming performance
15:04as well, but it's not about gaming.
15:07It's about
15:08being able to run local models.
15:11I want to level one if you want to chat
15:12or or hang out or show me cool stuff,
15:15come to the forum,
15:15forum.level1techs.com. I'm signing out,
15:17and I'll see you there. Be sure to check
15:19out Radiance and Flare and Halogen,
15:21because they are uh
15:24they're they're three of the four
15:25horsemen of the CUDA apocalypse for for
15:28Nvidia. If you really sit down and think
15:29about it and look at it, and it's like,
15:31yeah, I know one of those has to do with
15:32Nvidia hardware, but, you know, one of
15:34these things is not like the other, cuz
15:36you've also Well, maybe even one cat one
15:38cat in there, cuz that's the kernel for
15:41the V1. I I You you know, you just got
15:43to come hang out at the forum and get
15:44right in on this. It's a lot of fun.
15:45It's a lot of exciting. [music] Anyway,
15:47I'm signing out, and I'll see you there.
15:53>> [music]

About this transcript

This page contains the full transcript of The CUDA hour is Late. Also, Who Made the R9700 this fast? by Level1Techs, generated from the public captions YouTube serves with the video. The transcript has 2,893 words across 430 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.