YT YouTube2Text

YouTube2Text → Transcripts → Can AI Keep Growing Exponentially? Let’s Ask SemiAnalysis — With Dylan Patel and Jordan Nanos

Can AI Keep Growing Exponentially? Let’s Ask SemiAnalysis — With Dylan Patel and Jordan Nanos — Transcript

by Alex Kantrowitz · 11,795 words · 1,831 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00How big does AI have to get to justify
0:02the trillions in infrastructure
0:04spending? Let's talk about it with the
0:06leading authorities from SemiAnalysis
0:08right [music] after this. Welcome to the
0:09Big Technology podcast, a show for
0:11cool-headed and nuanced conversation of
0:13the tech world and beyond. The AI
0:16infrastructure numbers just seem to be
0:17getting bigger with no end in sight. So,
0:20we're going to talk about what the
0:21physics of this entire buildout looks
0:23like and whether it can continue to to
0:25sustain itself without not maybe not
0:28destroying the world, but destroying the
0:30economy. Uh cuz we have some very
0:31interesting guests to speak with us
0:33about it today. We're joined today by
0:35Dylan Patel, CEO and founder of
0:37SemiAnalysis, [music] and Jordan Nanos,
0:39member of the technical staff at
0:42SemiAnalysis, to talk all about
0:44everything uh going on within [music]
0:46the infrastructure buildout. Uh Dylan,
0:48Jordan, welcome to the show.
0:50>> Thanks, great to be here.
0:51>> Let's just start by talking about the
0:53scale of what we're seeing uh in this AI
0:56infrastructure buildout if we can. Uh
0:58Dylan, there's like a chart going around
1:00X right now that compares uh the AI
1:03infrastructure buildout to previous
1:05buildouts in the past like railroads, um
1:09like highways. And the chart is showing
1:12that the percent of GDP for the AI
1:14buildout today is about 3.6% or sorry,
1:17the the median is about 3.6%
1:20uh the average year compared to 2.2% for
1:23the railroads. And basically the
1:24commentary is that this is far bigger,
1:26right? That would put it basically 50%
1:28bigger than railroads even in terms of
1:30its share of GDP. And the commentary is
1:31this is bigger than anything we've ever
1:33seen before on a scale that is, you
1:36know, effectively unprecedented. Um I'd
1:38love to hear your perspective on just
1:39how big this is, whether that's the
1:41right way to look at it, um and then
1:43what it means if we're building out
1:45something this big.
1:47>> Yeah, I mean uh the first thing I when
1:48that chart was going around, I was like,
1:49"Whoa,
1:50people's numbers are too low. It's
1:51actually much higher than that already."
1:53Um
1:54>> Really? Okay, so share the real numbers.
1:56>> Yeah, yeah. So, so like I mean it's it's
1:58if you look at like next year,
2:00um CapEx across the US will be on the
2:02order of $2 trillion
2:06uh include not just data centers and
2:07chips, but also all the rest of the
2:09supply chain that people are investing
2:10in. Um so, you're close to $2 trillion
2:12of CapEx.
2:14Um
2:14and then you stack on top of that
2:17uh GEP is like close to 30 30 and some
2:21change trillion. So, you're at like
2:23you know, 5 or 6% right? Not not 3% or
2:262%.
2:27Um so, so sort of my thought process is
2:30actually like, "Oh, wow, these numbers
2:31are lower than the actual cuz people are
2:33understating the amount and size of
2:35CapEx that's happening in the US and and
2:37and North America and and the world."
2:40And I think like part of this is because
2:44you know, with railroads, your CapEx is
2:46in the country that you're deploying it
2:47in. With fiber, your CapEx is in the
2:50country you're deploying it in. But for
2:52AI infrastructure, your CapEx in America
2:54is actually a lot of it's being used to
2:56serve outside of America uses, right?
2:58So, if you look at for example, Europe,
3:01most of Europe's inference for Anthropic
3:03is in America. And most of OpenAI's
3:06inference in Europe is in America.
3:09Um and and similarly, if you look at a
3:10lot of Asian countries.
3:12Um and so, there is a bit of this like
3:14this this you know, CapEx in America is
3:17actually like much higher as a result.
3:19Um because one, you're doing all the
3:20training here, and then two, you're
3:21actually serving other countries from
3:23data centers in America. Um and so,
3:26ultimately, you're sort of at this stage
3:28where capacity is much um the economic
3:32impact of it is much larger um
3:35than any other sort of physical
3:37infrastructure investment cycle ever
3:39before.
3:40>> All right, Jordan, and you are the
3:41author of Cluster Max 3.0. This is
3:44SemiAnalysis rankings of the GPU uh
3:48cloud systems. Uh basically, putting a
3:50lot of the hyperscalers and the
3:51neo-clouds uh in tiers talking a little
3:54bit about their performance and we're
3:55going to get to that in a bit but I you
3:58know you do look across the board in
3:59your report and this is something that I
4:01like you to comment on. Basically we're
4:04looking at
4:05where the the hyperscaler quarterly
4:08year-on-year capex growth is going and
4:12in the third quarter of 2026
4:14the growth is at 116%
4:19>> year-over-year.
4:20>> Yeah, which is astonishing. The
4:23consensus estimate so what Wall Street
4:25is expecting to come after that, you
4:27know,
4:28they drop down in the first quarter well
4:30in the fourth quarter to below 100%
4:33year-over-year growth and in the first
4:35quarter next year in the 70% and it
4:37continues to drop from there. I I think
4:39this is a cycle that we're used to right
4:41where it's like well you know we're
4:43we're all in a bigger base so it can't
4:45possibly exceed expectations again and
4:48can't possibly double again. So I'm
4:51curious to hear from your perspective
4:52and your research
4:54do you think that we're actually going
4:55to see that trend like the growth is
4:56going to decelerate or are you know are
4:59we again falling into this trap or is
5:01Wall Street again falling into this trap
5:03where it's like all right you've you're
5:05basically at the biggest numbers you
5:07could possibly imagine but you still
5:08need to think bigger.
5:09>> [sighs]
5:10>> Yeah, I think we're definitely going to
5:12hit limits towards the end of next year
5:14and into the
5:14>> Wait Jordan before you answer seriously
5:16I got I got to I got you know it's it's
5:17like when you ask a kid what's the
5:19biggest number they can think of and
5:20they say like 1100 [laughter]
5:23right and and now and it's like no no no
5:25no we're at a trillion babe.
5:27>> Yeah.
5:28>> [laughter]
5:29>> Yeah, four commas club.
5:31>> So wait in in Dylan in in this example
5:33the kid is Wall Street.
5:34>> [laughter]
5:36>> Not not making any any statements that
5:38people are children just uh
5:41>> I got it.
5:41>> Yeah go ahead sorry.
5:42>> [laughter]
5:43>> Dylan may have an a habit of referring
5:45to his employees and his customers as
5:47kids sometimes, but that aside, um
5:50>> [laughter]
5:50>> I think I think I use a much more mean
5:52word.
5:55>> Okay, yeah.
5:56>> I like where this is going already.
5:57>> We're We're running out of big numbers.
5:59There are There are things to
6:02uh look at when it comes to the
6:03build-out. This could be
6:05everything from like industrial
6:07equipment to firms that are prepared to
6:11do EPC work, like engineering
6:13procurement construction, to chips, to
6:16cooling systems, to, you know, all the
6:18components that go outside of that and
6:21um and IT,
6:23you know, estate. Basically, uh
6:26there's a lot of work being done to
6:27expand the supply chain in order to be
6:29able to accommodate for all of this
6:30CapEx to to go into it.
6:32Um and it's A lot of it's working, and
6:35so we're going to see massive growth,
6:36but yeah, you you you can't just have
6:38triple-digit growth and accelerate the
6:40second derivative, you know, even
6:42further from where it is right now.
6:45Um you run out of things. And I think
6:46the biggest thing right now is uh
6:48twofold. One is like it it becomes
6:49really hard for the biggest neo clouds
6:51and hyperscalers to just organize and
6:54marshal the resources required to build
6:57another gigawatt on top of what they're
6:59already trying to build, for example.
7:01And it becomes pretty hard for
7:03them to raise the capital.
7:05Even though there's incredible returns
7:07on a lot of these investments already,
7:10you still just need to find the capital
7:12to pour into these projects and that's a
7:15you know, a difficult thing to do when
7:16you're when you're talking about numbers
7:18in the trillions of dollars.
7:20>> I would I would say the other thing is
7:21like by definition the numbers have to
7:23shrink. Um you know, this year we have
7:2510x growth in profit going from sub $10
7:27billion of revenue to north of 10
7:29north of 100 billion, so they had 10x
7:31growth. Um definitionally, they're just
7:33not going to have a trillion dollars of
7:34revenue by the end of next year. Um so
7:36yes, there's some decelerating growth
7:38there.
7:39Um
7:39you you Nvidia's revenue
7:42uh growth is still 2x year on year, but
7:45it's you know, it's not as much as it
7:46was at an earlier point just because it
7:49is definitionally harder to grow 2x off
7:52the same base, so 3x off the same base.
7:54So, it's sort of like
7:55same applies to like other firms. So,
7:57it's it is it is definitionally like
7:59definitely decelerating.
8:01>> Okay, so the growth on spending is going
8:03to need to decelerate at some point, but
8:05the growth on revenue and profit for
8:08that matter is going to have to
8:10accelerate, you know, beyond it's
8:13basically going to have to trail this
8:14big infrastructure build out and
8:16accelerate probably beyond where we've
8:18seen the the dollars being spent. I
8:21mean, you would you would imagine
8:22to facilitate all of the AI action. And
8:25there's been further further
8:27conversations that have circulated about
8:28this. And Tim, let me just throw these
8:30numbers out to you, right? So, this is a
8:31Goldman Sachs report.
8:35Sorry, so Goldman Sachs, they say meta,
8:38this is a tweet, right? Meta's committed
8:40a corporate suicide after taking on
8:42insurmountable debt that the company
8:43will struggle to pay back. There's Bain.
8:46Bain says hyperscalers will need 6
8:48trillion in revenue, not ever. This is
8:50per year by 2020 by 2031
8:54to break even. And there's a Columbia
8:56professor
8:58Sten Van Nieuwenhuizen who said that
9:00America will need to spend roughly 3.5
9:03trillion annually on AI services by
9:052032, 8.8% of GDP to justify today's
9:09massive data center investment.
9:13What do you think of these these sort of
9:15assessments of where the spending and
9:17where the revenue needs to be to pay off
9:19the investments and do they have a
9:21chance at making good on those
9:23investments?
9:24>> Yes, I mean, I think I think everyone's
9:25numbers for revenue requirements are
9:27quite low. Um given the
9:29>> Okay.
9:29>> where this build out is going to go. I
9:31think I think it's going to go like I
9:32think revenue will be much higher than
9:33that by 2031 for example. Um, mean,
9:36there's a couple points here, right? One
9:38is like
9:39you know, how much is are people
9:41spending on capex and then revenue? And
9:43the revenue always trails the capex.
9:44These infrastructure investments are six
9:47years on an accounting basis at least
9:48appreciable assets. Although, there's
9:50quite a bit of evidence that they're
9:51actually going to last longer than that,
9:53but you know, six years is let's just
9:55continue to use six years. So, if I
9:56spend a trillion dollars today on, you
9:58know, AI data center and and GPUs,
10:01um, the GPUs are lasting six years, the
10:03data centers are lasting 15. Um,
10:06but that means definitionally in terms
10:07of revenue, um, either I could assume
10:09it's a straight line, right? So, then
10:11over the six years I need call it 1.5
10:14trillion of revenue. So, $250 billion of
10:17revenue a year for that trillion dollars
10:18of capex, or it's going to be something
10:21sloped where the first year is much less
10:23and the last year is much higher, um, to
10:25get the payback.
10:26Um,
10:27as far as like, you know, like
10:29statements on meta, it's like this is
10:30this is just kind of silly because meta
10:32meta at any point can just stop buying
10:35new stuff and their cash flow can easily
10:37pay for all the stuff that they've
10:38signed. Right? They just don't have to
10:40you know, they don't have to build a
10:42hundred billion dollars of data centers
10:44next year. They could pull the plug
10:46whenever they feel like it. Um, there's
10:48obviously contracts that they've a
10:49committed to, but there's also a lot of
10:50stuff they haven't committed to yet. Um,
10:52and so as far as like the solvency of
10:53meta, like they're completely solvent,
10:55they're completely fine.
10:57Um,
10:57and then the question is like what
10:58revenue do they make on top of that? Um,
11:01and as far as like AI's impact in the in
11:03the US economy, um,
11:06yeah, I mean, if if AI revenue is only
11:08three trillion dollars in 2031, then,
11:11yeah, we'll have big problems.
11:13Um, but you know, I I'm I'm like of the
11:14belief that we can get to like
11:16600, 700 million across open AI and
11:19Anthropic alone uh, by the by December
11:21of next year.
11:23Um, I'm of the opinion that you can uh,
11:25get to, you know, one and a half
11:27trillion maybe even by 2020 end of 2027
11:30uh, with just those two companies. Um,
11:31so that's end of 27. So then the payback
11:33is clearer, right? Um,
11:35you know, where they're getting that
11:36from. Um, ultimately, um,
11:39the the capacity that people are
11:43building out is matched to what they
11:44think business is coming to.
11:46Um, and so there's always going to be a
11:47mismatch if we're in the building phase
11:50of how much infrastructure you're
11:51spending on versus what's the AI
11:53revenue. Cuz that AI revenue for today
11:55is really based on the infrastructure
11:57you spent in prior years.
11:59Um, and right now CAPEX is not going up
12:01that 10x year, but inference just
12:03revenue just went up 10x year on year
12:05um, for the AI labs. And so, um, we've
12:08seen Anthropic is now, you know,
12:09profitable in terms of, um, compute cost
12:13minus revenue or revenue minus compute
12:15cost, um, both training and inference
12:17combined. Um, and OpenAI gets there, you
12:20know, within the next couple quarters,
12:22uh, maybe two or three quarters. Um,
12:24so based on at least what like we expect
12:26in the tokenomics model. So so I think,
12:28you know, those companies are already
12:30uh, breaking even. If you look at like
12:31Amazon, for example, the biggest builder
12:33of AI infrastructure, they all of their
12:36AI infrastructure investments are
12:38profitable today.
12:39Um, now, you know, the customers that
12:41for them could go insolvent, those
12:43customers being OpenAI, Anthropic, etc.
12:45Um, but right now they're solvent on
12:47those, um, you know, making very strong
12:49gross margins. In fact, AI has made
12:51their gross margin go up in the most
12:52recent quarter,
12:54um, from Bedrock and AI revenue from
12:56Bedrock. Um,
12:57so so yes, AI revenue needs to grow a
13:00lot, um, but I would posit that like
13:04being able to automate all the stuff
13:05that we've been able to and being able
13:07to build all the stuff that the world
13:08has been able to,
13:10um, is going to generate meaningful
13:12revenue and surplus for the consumer
13:14ultimately.
13:15>> Yeah, I think we need to stretch it a
13:17little bit in our minds to understand
13:19what we are going to measure as AI
13:20revenue going to the future. If these
13:22companies really do have the best models
13:24in in world that are capable of
13:26effectively all knowledge work, you can
13:28imagine them going into a whole bunch of
13:29different industries beyond coding and
13:31having returns on their investment
13:33similar to what they've seen in coding.
13:35So, you can look at things like robotics
13:37and self-driving vehicles. You can look
13:38at drug discovery. You can look at just
13:40health and life sciences generally. We
13:42can take this across all of these
13:44industries and say
13:45if they want to bring a product to
13:46market
13:48like a cancer drug or like a
13:50self-driving vehicle, you can just
13:51imagine how much revenue they can
13:53potentially return based on the
13:54investments they have today. And if they
13:56have the
13:57AI, the system to be able to produce the
14:00knowledge work required to build that
14:02product incredibly well,
14:04I think they're the only ones that can
14:05really compete on the frontier of being
14:07able to do these things. And so,
14:09I think people lack the creativity to
14:11understand how returns can
14:13be viewed beyond the like cloud code
14:15subscription and per token pricing
14:18regime of today.
14:19>> Yeah, there's there's this chart that
14:20looks at how much we're spending as a
14:21share of GDP. This is just 2025. So, I
14:25share of GDP, eight health is 18%.
14:27That's the top category. Food is 9.1%.
14:30AI services is 8.8%. It's the third
14:34category in 2025. I imagine it's going
14:36to lap or it will pass food in 2026. So,
14:40guys, I'm just curious, where's this
14:42money coming from? Like it you know,
14:44let's say OpenAI and Anthropic combined
14:47get to $700 billion in revenue next
14:50year.
14:51Um is the money coming from these
14:54companies labor pools? Is it just
14:57additive because they can sell more?
14:59Uh where is the money that these
15:01companies are the customers of these
15:03companies? Uh where's that money coming
15:05from that they can funnel it into the AI
15:07companies?
15:09>> It It's a few factors, right? Well,
15:10first off is
15:12obviously companies are going to be able
15:14to accelerate what they're building.
15:16Um and so, they can invest more.
15:18Um increasing share of R&D dollars will
15:20go towards AI. Increasing share of cost
15:23of goods sold dollars will go towards
15:24AI.
15:25Um but also you'll have a significant
15:27amount of money sort of going coming
15:29from
15:30um
15:31you know, labor as well, right? Labor
15:33and I don't think labor shrinks per se.
15:35I think the rest of the economy grows so
15:37fast at least in the short term that
15:39labor labor sort of stays, you know,
15:41flatish.
15:43Um
15:43and and so like um you know, in the past
15:46we've had different cycles across the
15:47economy but one long-standing one has
15:50been labor share of GDP has fallen over
15:52the decades. And it's been a steady slow
15:54line since the '70s and so a lot of the
15:56like strife that people say America's
15:58economy sucks and yet oh look the
16:00economy's bigger is actually well like
16:02labor share of economy has shrunk and
16:04that's why.
16:06In reality a lot more of the labor the
16:07value is getting accruing to capital.
16:09What AI does is it's going to
16:11supercharge that trend long-term trend
16:13of labor share versus capital share
16:15because AI tokens are general
16:18basically just capital.
16:20You know, in the form of chips and data
16:22centers that are sitting there and
16:23creating labor. Um and so that labor
16:25share is going to fall even faster. Um
16:28and and and so, you know, I I would hope
16:31and and for society that that that's my
16:33belief, you know, and maybe it's like a
16:35irrational belief but I believe that we
16:36will just grow the economy so fast that
16:39labor in terms of dollars will not
16:41shrink
16:42um and it might even grow slowly. It's
16:44just that the rest of the economy grows
16:46so fast. Um and that's where that's sort
16:48of where it comes from, right? So, all
16:49these companies that do offer various
16:52services, their services are going to
16:54get better, they're going to be able to
16:55charge more, they're going to be able to
16:56sell more
16:57and they're going to be able to do it
16:58with fewer people. Um or another way to
17:01look at it is they're going to have the
17:03same number of people and their business
17:04is going to get better. Um and that's
17:06that's and and there's a number of
17:08industries where this is possible,
17:09right?
17:12You know, obviously coding is an area
17:13where people have done it quite a bit.
17:15Um obviously general white collar work
17:17is getting rapidly automated, but also
17:19companies are just able to do a shitload
17:21more than they were able to do in the
17:22past. Um, but there's a lot of
17:23industries like pharma and uh trading um
17:27and chip design and such, right? Like
17:30chip design is a great example, right?
17:31Chips the number of American R&D
17:35engineers working in chip design has
17:36been basically flat for 20 years and yet
17:39the economy for it has exploded. Now
17:41obviously manufacturing isn't We're not
17:43counting manufacturing, we're just
17:44counting the designers. Um, but the
17:46designers are like, you know, two the
17:48most valuable companies in the world,
17:49right? Nvidia, Broadcom,
17:51um as well as like Google and all these
17:52other companies that are making their
17:53own chips and many others, right? These
17:55designers
17:56have not really grown that much in terms
17:58of number of people. It's been flat as
18:00an industry for US. And yet the value
18:02that's accrued has exploded. And why is
18:04that? There've been a lot of tools. Now
18:05what is What does AI assisted chip
18:07design now do? Well, now all of the
18:09chips that are across the economy get
18:11better and better. Your phones are going
18:13to get better. Your, you know, AirPods
18:14are going to get better. The chip in
18:15them, chips for robots are going to get
18:17better, chips for obviously AI um are
18:20going to get better, chips for all sorts
18:22of things, Bluetooth and GPS and all
18:23these other things, sensors um for gas
18:26leaks or whatever that Bosch makes.
18:27These are all just going to get better
18:28because AI can help you design chips.
18:30You do it with the same number of
18:31people, revenue grows, margins grow, um
18:34everything looks fantastic, right? And
18:35that's sort of like There's many
18:37industries where this can get replicated
18:39um
18:40in my view.
18:42>> Dylan, you've said that uh I think in
18:43the past you've said something to the
18:45extent of knowledge work is effed. So do
18:47you no longer believe that?
18:49>> Um,
18:51I think most knowledge work
18:54Okay, like let's separate like my belief
18:56in like Looney Tunes 2035 versus like,
18:59>> [laughter]
18:59>> you know, 2028. Um,
19:02I think in 20 I think for the next, you
19:04know, X number of years, as long as AI
19:06is very powerful, but it is not better
19:08than humans in most every way,
19:11the most the most high agency people are
19:14going to be able to do the coolest
19:15stuff. They're going to be able to make
19:17way bigger impacts on the world. Um,
19:19now, you know, in in like the case where
19:22AI is just better than everyone,
19:24um,
19:25that that could change drastically,
19:27right? Um, you know, then then like
19:29knowledge work is screwed and actually
19:31like you know, what matters is like
19:33people being able to like play games and
19:35have fun, people being able to act,
19:37people being able to like, you know, do
19:39the things that humans do that humans
19:41only appreciate in each other. If the AI
19:43doesn't kill us all, right? But that's
19:45sort of like, you know,
19:46>> [snorts]
19:46>> We We won't go there today.
19:48Um, Jordan, I you know, this is kind of
19:51in your your area of expertise. I want
19:52to get started talking a little bit
19:53about the risks to like what might
19:55derail this story.
19:57Um, and you know, we'd already talked
19:58about the fact that uh there are some
20:00contracts that a company like Meta
20:03uh could you know, may not have to go
20:05through with.
20:06Um, you're obviously looking a lot at
20:07the businesses of the neo clouds which
20:10are dependent on uh companies like Meta
20:13actually making good uh on the on the
20:16promises that they've they've put forth
20:18to build with these companies. Um, is
20:20this sort of an under appreciated risk
20:22that like everything looks like fairly
20:24[snorts]
20:24fine right now if you speak to one of
20:26these neo clouds because they'll tell
20:28you that they have the best customers in
20:30the world. Uh, they have the Metas and
20:32Meta will make good on the on the um
20:35on the promise and they're raising debt
20:36based off of this. Uh, but in reality
20:39the risk might be that a company like
20:41Meta might just decide yeah, we're not
20:43going to do that and that the contracts
20:45might not be as binding as uh you know,
20:47the neo clouds tell us they are.
20:49Okay, so I I want to be clear about one
20:51thing Dylan was talking about there in
20:52terms of termination versus what's going
20:54on with Meta and the neo clouds which is
20:56that generally speaking
20:57these hyperscalers
21:00or you know, large frontier labs or
21:02signing up with contracts for the neo
21:04clouds do not have arbitrary
21:06cancellation rights on like a monthly
21:08basis where they can just pull the plug
21:09and the contract's done. Uh generally
21:11speaking, they're signing 5-year, 6-year
21:13term lengths and they're off-taking the
21:15full capacity and they have a set, you
21:17know, hourly rate and total contract
21:19value that's very very attractive to the
21:20new cloud providers because those guys
21:22have gone out and secured land and power
21:24and built a site and even in some cases
21:26purchased the chips prospectively before
21:28they've actually gone and done that
21:29contract. What Dylan's referring to with
21:31the the potential to slow down or pause
21:34or cancel projects is more with respect
21:37to the self-build sites, which is to say
21:39like Meta has massive campuses that are
21:41going to serve as more of a base load of
21:44capacity going into the future, which
21:47they can either choose to buy chips for
21:49or just leave as an empty shell for
21:50years if they want to slow down and not
21:52go as hard as they can. And generally
21:54speaking, construction costs, while very
21:57expensive, are a small percentage of the
21:59total cost of a new chip uh project.
22:02Like, in other words, the concrete and
22:04the steel and the walls is quite,
22:06you know, it's important and big and
22:08and, you know, very a real challenge to
22:10get it done, but on a dollar basis, like
22:13the HBM and the optics and the GPUs
22:15themselves are the the bigger uh portion
22:18of the bill.
22:19Um
22:20And so, and you know, what it's what
22:22it's looking like right now is that all
22:24of these big companies are really
22:25needing to contend with the massive
22:26growth that they're expected to pursue
22:29by their demand from their customer
22:31base. Either that's Meta or OpenAI or
22:34Anthropic have demand on their
22:36consumer-facing applications like a
22:38ChatGPT or Claude Code or Codex or even
22:42Muse from Meta, things like that.
22:45In order to serve that demand, they need
22:46GPUs and it's profitable to serve the
22:48demand, so they need to ramp up and be
22:50prepared to roll this thing out to the
22:52billions of monthly active users on the
22:55Meta uh different, you know,
22:58uh surfaces that they serve and
23:00approaching a billion of monthly active
23:01users for
23:02chat GPT and hundreds of millions for
23:06cloud code and things like that. So,
23:08anyway, um
23:09with that distinction in mind,
23:12what's the the risk with project
23:14cancellation or a downturn or things
23:16like that? I mean, in order to for for
23:19us to
23:20forecast some sort of change in the
23:22demand supply curve there, you'd really
23:25need to see demand slow.
23:28Um there are hard constraints on how
23:29many chips you can produce, how many
23:30data centers you can build, and how fast
23:32this stuff is going to come online. And
23:34as of right now, all of the modeling we
23:36can do basically shows that demand
23:37outstrips supply. And so,
23:41is demand
23:4210% more than supply right now? Is it
23:4450%? Is it infinite? I mean,
23:48we we don't get to answer that question
23:50because we are not able to catch up to
23:52demand with the supply of GPUs that you
23:54can bring online right now. And so, when
23:56that question gets answered, if it gets
23:58answered, if we find limits to the
24:00demand for knowledge work, to the demand
24:02for intelligence from the economy, then
24:05we'll we'll be able to figure out who
24:08needs to slow at what time, where, and
24:10how that plays out.
24:12>> Yeah, but so far, I guess those limits
24:14haven't been found. So,
24:16um all right. Uh before we get into
24:18cluster max, one last topic I want to
24:19cover, uh big IPO on the way, uh which
24:22is of course the Anthropic IPO.
24:24Uh Dylan, you had a very interesting
24:26tweet about it. I'm just going to read
24:28it. You said, "You can't invest in the
24:29Anthropic IPO at $2 trillion because it
24:32could be zero and I'm or it
24:34could be 20 trillion, but at 20
24:36trillion, we're all fucked." Uh can you
24:38explain that and and tell us a little
24:40bit about what you mean by that?
24:41>> Um to be clear, it's it's entirely a
24:43post. Um I was I was like listening
24:47>> know what? I I get it, but I think
24:48there's some truth to to what you're
24:50saying
24:51at the time. So, talk to us about it.
24:53>> I I'm I'm generally extremely bullish
24:56Anthropic. I think Anthropic will make
24:59you all make, you know, be a very
25:01successful company in the world.
25:02Anthropic could even be the first
25:03company to be a $10 trillion valuation
25:06company in the world. Like I'm like that
25:08bullish Anthropic, but at the
25:10same time
25:11um for them to be a 10 or 20 trillion
25:13dollar valuation company, especially 20,
25:16um their first of all, their multiples
25:18on revenue are going to have to collapse
25:19quite a bit.
25:21Um you know, so they used to you know,
25:22they used to raise at 30 X and now
25:24they're, you know, IPO is going to be,
25:26you know, let's call it they're at a
25:27$100 billion of revenue and they're
25:28going to maybe be at 20 X. Um you know,
25:31if they get to $10 trillion,
25:33I bet you they would need to be at like
25:35a trillion dollars of revenue
25:36um or something crazy right like that,
25:38right? Um which could happen in at some
25:40point in '28 um or '29.
25:44Um
25:45you know, it depends of course like the
25:46stock market can also go crazy and like
25:47make multiples go crazy, but you know,
25:49in the world where they're doing, you
25:51know, where they're worth 20 trillion,
25:52you know, I don't think their revenue, I
25:53don't think they're even at a 10 X sales
25:55multiple now. They're probably at like a
25:565 X or even less. Um in which case
25:59they're, you know, if they're at $20
26:00trillion of revenue, they're at are they
26:02at like two, three, four trillion
26:03dollars of a 20 trillion dollars
26:05valuation, two, three, four trillion
26:06dollars revenue? And at that point, you
26:09know, what is the state of AI? If on
26:12[snorts] a $150 billion global economy
26:15or trillion dollar economy, they're
26:17generating multiple trillion dollars of
26:19revenue and by far the largest company
26:21in the world, like AI's capabilities are
26:24freaking insane.
26:26Um in which case like, you know, all of
26:28this worry of like whether it's AI kill
26:30us all or AI so good that everyone loses
26:33their jobs and because distribution of
26:36that earnings is going to be heavily
26:38leveraged to the people who own
26:39Anthropic stock or um constituent supply
26:42chains of it and not distributed well,
26:44then people are going to have dis-
26:46disrest and like overthrow governments
26:49because all of the
26:50growth is accumulating to a very small
26:52number of people. So, like these things
26:55could like tear the fabric of society
26:56apart and therefore, like if you believe
26:58Anthropic will get to 20 trillion, you
27:01know, society as a whole could be
27:04>> Jordan, you have similar worries?
27:05>> I think Yeah, I think I think
27:08uh there's a lot to do to get to that
27:10point. Like um
27:12I I actually don't think I'm necessarily
27:14as worried uh
27:16as Dylan on the
27:17>> by the way. I just painted a pessimistic
27:18picture. I'll I'll go optimistic
27:20instead.
27:20>> Yeah, I mean like the the rosy
27:22optimistic view is kind of what I
27:23described earlier, which is like if
27:24they're getting to 20 trillion dollars
27:26of revenue, this is not done on a per
27:28token metered basis for people using
27:31Claude code as a product. And I don't
27:33think you know, you go and see a Claude
27:35code go 20X better from where it is
27:37right now in terms of like quality and
27:39that drives the revenue. I think that
27:41they get into other areas of business
27:43and I think I would trust Anthropic
27:45quite a bit to bring, like I said
27:47earlier, a cancer drug to market or
27:48something like that. And we've seen
27:50similar stuff happen recently with
27:52GLP-1s just be such a meaningful thing
27:54for so many people in the economy and as
27:56a result, you know, companies like Lilly
27:59have have had massive uh
28:02you know, returns on on that incredible
28:04research that led to those
28:05uh incredible drugs that help people.
28:07And so, I think um
28:09I would be more optimistic for the
28:11future where
28:13theoretically, [laughter]
28:14Anthropic sat at a 20 trillion-dollar
28:16valuation.
28:17Um but I do think that there are a lot
28:20of uh
28:21going to be a lot of bumps along the
28:23road along the way. Um
28:25which is to say,
28:28going back to your to your question
28:29previously, like how can they possibly
28:31serve all of this demand and train these
28:33models and uh you know,
28:36have intelligence diffused to all of
28:37these different parts of the economy so
28:39that people can realize the benefits.
28:40And I think the way you do that is to
28:41build compute and it so depends on,
28:44you know, the nitty-gritty details of
28:46construction and installing GPUs and
28:48running them reliably and all of that.
28:50Like to me the center of the world is
28:52the Neo clouds on that road from
28:55a $2 trillion IPO to a $20 trillion
28:58valuation in the public markets.
29:01>> Yeah, Anthropic kind of an interesting
29:03business case, right? They're spending
29:04all this money on infrastructure.
29:07Uh even in the S1, you saw big losses
29:09accompanying their revenue. Although
29:12um Dylan, as you mentioned, it looks
29:13like what they've done is they've turned
29:15the tide a bit and they're making more
29:16money uh right now than they're spending
29:18on compute.
29:20Um
29:21Talk a little bit cuz I think the the um
29:23the belief here is that a company like
29:25Anthropic is a low-margin business or at
29:28least much less than uh than software,
29:31but that doesn't actually seem to be the
29:32case and you guys have run the numbers.
29:35>> I would say like if you look at like
29:37Salesforce, which is probably like one
29:38of the greatest SaaS companies out
29:40there.
29:41Um
29:42you know, I mean I think like at least
29:43like consistently grown to become one of
29:44the biggest SaaS companies.
29:46Um their gross margins are like 75%
29:49and Anthropic on their inference is
29:52doing 75% gross margins, a little bit
29:54higher even. Um
29:56you know, especially on the frontier
29:57models. So it's sort of like well, and
29:59it took it took Salesforce quite some
30:01time to get there, right? They were not
30:03at 75% when they IPO'd.
30:06Um you know, they they they took quite a
30:08bit of time. Actually, they were at but
30:09then they like, you know, in the mid in
30:11the late 2000s like they were, you know,
30:13I think like it took they were like not
30:15quite there. It took, you know,
30:16Anthropic quite a bit less time to get
30:18to that strong levels of gross margin.
30:20Um and if you look at many other SaaS
30:21companies that are not even that high.
30:23Um
30:24and then the other thing is like their
30:25customer acquisition cost is insane.
30:27Right? If you look at like a
30:28a SaaS company, it's like, you know,
30:30find my uh
30:32my COGS is really low, but then my
30:34customer acquisition cost can be as high
30:36as like 30%. Um and Anthropic has near
30:40or customer acquisition cost. Right now,
30:42obviously, they have a humongous R&D
30:44spend,
30:45um
30:46but their R&D spend has grown slower
30:48than their revenue.
30:49Um and so now they have reached
30:50profitability. And there's no reason why
30:52that trend doesn't continue, right? And
30:53they continue to grow more profitable.
30:56Um so at least
30:57operating margin-wise. And then and then
30:59like scalability is the other thing,
31:00right? Like the the beautiful thing
31:02about a lot of SaaS companies is even if
31:03your customer acquisition cost is so
31:05high,
31:07you know, as long as your margins are
31:08high enough to pay for it initially,
31:09eventually you don't have to reacquire
31:11that customer. Now you've got this super
31:12profitable business. But with like the
31:14AI labs, their business seems somewhat
31:16sticky. Yes, there's all this like
31:18messaging of like, oh, you know, Meta's
31:20cutting spend or Microsoft cutting spend
31:22or what have you, but ultimately, the
31:24revenue has only been a straight line up
31:26on the chart and almost every customer's
31:27only grown on revenue for them. And uh
31:30switching cost, even when, you know,
31:31some other Chinese model comes out or
31:33OpenAI comes out with a better model,
31:35doesn't really shift the spend share
31:37that much um for existing customers. Um
31:41you end up with like this like pretty
31:43sticky customer potentially who is
31:45growing revenue at some pace and there's
31:47no customer acquisition cost and the
31:49margins are high. So it's arguably a
31:51better business than software is is an
31:53AI lab.
31:54If you can successfully
31:56>> But how much is the risk the fact that
31:58like they're at this pacing the frontier
32:00um move- movement movement is coming
32:02out. Um
32:03>> I think that's just signaling.
32:05I think it's completely
32:05>> Talk about that. You don't think that's
32:07going to that's true at all.
32:08>> Um uh I think pacing the frontier is a
32:11responsible thing to do if you believe
32:13that we could kill ourselves.
32:16Um
32:18I don't necessarily think that
32:22um they will pace the frontier too much,
32:24though. I think like there's there's
32:26like very little pacing they can do
32:28given the race that they're in. Um
32:30and people have made skits about this,
32:32but like, you know, you know, Dario
32:34can't slow down because he doesn't trust
32:35Sam and Sam's not going to slow down.
32:36And Sam can't slow down much because
32:38same with Darya. Obviously, they're
32:39going to slow down a little bit when
32:40they have these massive security
32:42incidents.
32:44Um
32:44and neither of them can slow down
32:45because China's not too far behind. So,
32:47it's like sort of like very difficult.
32:49>> That's reassuring.
32:50>> [laughter]
32:52>> All right. Um
32:53look, we're going to take a quick break
32:55and then when we come back, we're going
32:56to talk a little bit about
32:58um the state of the AI clouds, uh the
33:01ClusterMax of course,
33:02uh and then we'll continue on with some
33:04more questions about the technology and
33:05the business. So, we'll do that when
33:07we're back right after this. And we're
33:09back here on Big Technology Podcast with
33:12Dylan Patel and Jordan Nanos from
33:13SemiAnalysis. SemiAnalysis, I'm sure
33:16you're familiar. It's the Bible for
33:17anything AI infrastructure.
33:20Uh these guys are the best. We're very
33:21grateful to have your time, both of you
33:23and um you know, I definitely recommend
33:25you at least uh check out the blog. You
33:28can also get uh SemiAnalysis
33:30uh on Substack. It's
33:31newsletter.semianalysis.com.
33:34It's the number one technology
33:36newsletter, uh and well deserved. So,
33:38congratulations, guys.
33:40Um in the newsletter, you can uh find
33:43the ClusterMax 3.0.
33:45Uh it's it's actually even if you're not
33:47technical, I was talking with Jordan
33:48about this, it's a fun read. It goes
33:50into the state of the build out um
33:53and the subhead tells you, you know, it
33:55already kicks off in the right way in
33:56the gory detail, the reliability,
33:58performance, support, and pricing, and
34:00security uh the GPU cloud providers.
34:02There's a lot in there.
34:04Um
34:05There's a lot of areas we can go in
34:06this, uh Jordan and Dylan, but let's
34:08begin with the role of Nvidia here.
34:11Um
34:11the report, you know, though it spends
34:13most of the time on the technical side
34:15of things, does go into the fact that uh
34:17Nvidia is playing a major role in
34:21keeping these these um these neo clouds,
34:23so the companies providing
34:25uh infrastructure AI infrastructure,
34:27maybe to some of the frontier labs, but
34:29really across the board uh in making
34:31this whole thing work and Nvidia has
34:32been backstopping uh a lot of their um
34:36their businesses with uh as you write um
34:39revenue floors, landlord guarantees, and
34:42leases that it plans to transfer to
34:43third parties. Uh helping the neo clouds
34:46and AI labs in obtain ingre- investment
34:49grade financing without a hyperscaler
34:52being involved. Um Jordan, to you on
34:54this one, uh
34:56you know, there's the the criticism,
34:58sort of like the Ed Zitron criticism, of
35:00this uh of this industry is that Nvidia
35:03is basically funding uh the demand for
35:07all these neo clouds and without that uh
35:10backstop, um you know, this whole thing
35:13would basically evaporate and make
35:15Nvidia look a bit smaller and make these
35:17neo clouds
35:18um you know, non-nonviable.
35:20What's your thought on that after going
35:22so deep into this?
35:24>> Uh yeah, well, I mean, both in
35:26ClusterTrucks and another article we did
35:28recently called Nvidia's backstop
35:29universe, we go into a little bit of the
35:31dynamics of what backstops are and how
35:33they help the neo clouds raise the money
35:35they need and to deploy the GPUs, which
35:37then customers are renting pretty
35:39strongly at this point. And uh the
35:42number that we have is that currently
35:43we've got tracking of $588 billion of
35:47off-balance sheet backstops that Nvidia
35:49is supporting right now, which is one of
35:51yet another unbelievable number in the
35:53list that we've been going through on
35:55this podcast so far. Um I mean, but
35:58Nvidia's revenue is approaching
36:00400 of those across $500 so we're
36:02talking about really big numbers there.
36:04For context, that's like 2 and 1/2% of
36:07the M2 money supply in the US. So,
36:10obviously there's massive massive
36:12amounts of money going on
36:14uh in this
36:15um universe as we refer to it.
36:18And uh I mean, the only thing I can say
36:20about this is that it just seems really
36:22pragmatic from Nvidia. They really
36:25uh want a wide diversity of customers.
36:27They won't don't want to be dependent on
36:28the hyperscalers for all of their
36:30revenue. Um, they also don't want their
36:32customers to be dependent on getting
36:34their chips from a select set of three
36:36or four different hyperscalers. And so,
36:38they have every incentive to have a a
36:41nice healthy market with a diversity of
36:43options and uh, lots of selection for
36:45all of the customers to go
36:47uh, work with.
36:48And in order to deliver on that, people
36:50have more problems than just
36:53uh, raising the money required to buy
36:56the GPUs. Well, first of all, they need
36:58the guarantee that if they raise the
37:00money, those creditors are going to see
37:02returns and that requires, you know,
37:04some sort of investment grade off-taker,
37:06which Nvidia can serve in these cases in
37:08many in many of these cases.
37:10But, the way it's worked so far is that
37:12Nvidia signs up as that off-taker and
37:15then when it comes time to actually, you
37:17know, take those GPUs for themselves for
37:19their own internal research work,
37:22somebody else comes along and wants them
37:23more at a higher price. And so, then
37:25they sell them to their to those guys.
37:27They don't take them for their internal
37:28research and they roll on to the next
37:30one.
37:30And for everybody who says that Nvidia,
37:32you know, is doing this for some
37:33nefarious purpose or to to keep some
37:35bubble going, I'd encourage you to look
37:37at the quality of Nvidia's research.
37:39They publish more models on hugging face
37:40than anybody else. They have incredible
37:42CI across the suite of software that
37:45they they support on their GPUs and they
37:47train models like NeMo Triton. We're not
37:49the biggest fan of these models or this
37:51software, you know, we think there
37:53should be a diverse ecosystem. But, the
37:55idea that Nvidia does not use their own
37:57GPUs for research that serves the
37:59purposes of their company is, you know,
38:01itself wrong. And then the dynamic that
38:03we've seen is that this amount of
38:06compute is actually not a meaningful
38:07portion of Nvidia's revenue. It's it's a
38:09very small percentage of Nvidia's total
38:11revenue that is, you know, going back
38:13for their own internal research
38:15workloads. So, what we're seeing is just
38:17that they are supporting the old cloud's
38:18build up the capacity for the demand
38:20from the customers, which is so
38:22incredibly strong right now and hard to
38:24serve.
38:25Um and then maybe the the last thing to
38:27say about this is that it's not just the
38:29chips. Like we we we track a whole wide
38:32range of things that they're
38:33backstopping. This is everything from
38:35like, you know, the construction costs,
38:37the memory suppliers, the people
38:39building fabs, the people building
38:40industrial equipment. I mean, it's not
38:42just like I need help raising money for
38:44my GPUs. It's I need help raising money
38:46for all of the other things in the
38:48supply chain, which is just the long
38:51pole in the tent for that specific
38:52project.
38:54>> Yep. Let's talk about security because
38:57uh you know, the first image in this
39:00report uh is a quote tweet of an Ilya
39:02Sutskever post, which was made right
39:04after uh the Hugging Face incident. Uh
39:07Ilya says, "Neo clouds have limited
39:09cybersecurity. Next time agents
39:11successfully go rogue, uh they'll try
39:13taking over a Neo cloud to run more
39:15copies. This is bad. Thus, Neo clouds
39:18should greatly strengthen their
39:19cybersecurity and every company with
39:21strong cyber models uh should help with
39:23that." This is a rare tweet uh from
39:25Ilya. Dylan, what do you make of this?
39:28Uh is this It sounds to me like if he's
39:30out saying this, right? Remember, he's
39:31building this stuff himself. Uh I'd say
39:33if superintelligence that we might have
39:35a problem here.
39:36>> Yeah, so we we posted an article titled
39:38most Neo clouds suck at security.
39:41>> Yeah.
39:41>> documenting
39:42>> way, the art for that uh is is
39:44incredible. We'll put it on the screen
39:46uh for those who are watching.
39:47>> The it's I'll tell you what, that guy is
39:49getting
39:50>> [laughter]
39:50>> Yeah, exactly. Getting stabbed by
39:52everybody and it's like it's SOC 2
39:54compliant.
39:56>> [laughter]
39:56>> Um but yeah, so we we posted that report
39:59and then
40:00uh that, you know, like maybe eight or
40:02eight hours later or whatever, Ilya uh
40:04posted his tweet
40:06um
40:07because they are very much um
40:10you know, [snorts] using various clouds.
40:13Um they're obviously super secretive
40:15um and care a lot about the secrecy of
40:18like what they're working on, but also
40:20they're big believers in super
40:21intelligence.
40:22Um, you know, I guess safe super
40:24intelligence SSI, they were the first to
40:26and Trump actually copied Ilya. Ilya's
40:28first with another one, right? Um, first
40:30to reasoning, first to uh, LLM scaling,
40:33and now first to uh, super intelligence
40:35branding.
40:36Um, but Ilya sort of tweeted that out.
40:39Um,
40:40and and it sort of echoed what we wrote,
40:41right? Which is like, look, most of
40:43these clouds are terrible security. You
40:45can get open source models to just hack
40:47neo clouds or get access to stuff within
40:50other within neo clouds. Um, you know,
40:53GLM 5.3 especially is like really good
40:55at this. So, open source Chinese models
40:57can just hack dozens of the neo clouds
41:00that exist out there and you can go look
41:01at what other people are doing. And so,
41:03we've seen stuff like this
41:05um, at other clouds. There was a cloud
41:06that you know, we we didn't even intend
41:08to, but we just accidentally ended up
41:10like seeing all of the other clients on
41:13that cluster, seeing what stuff they had
41:14in storage, see what stuff they had in
41:15Slurm, etc. And they were like
41:18>> intelligence agencies, right? Like on
41:20this So, we immediately posted out
41:22like reported it to them, we're like,
41:24yo.
41:24>> agency?
41:26>> Which intelligence agencies?
41:27>> Uh, we we cannot say. I don't think
41:29that's a valid thing to say.
41:30>> Uh, national like like federal well,
41:34security agencies of governments.
41:35>> They were not American.
41:36>> They were not American, but we said it
41:38was for a country that was in a has a
41:39top 10 GDP. It's not like some tiny
41:43country.
41:44>> I'm I'm losing my mind here that that
41:46was possible.
41:47>> Yeah, and it's like it's like if we were
41:48malicious, and by the way, there were
41:49probably a dozen other customers, and
41:51maybe one of them was malicious, we
41:53could have just like seen what they were
41:54training.
41:56Right? And they were training models and
41:57running models for their themselves,
42:00presumably.
42:01Um, cuz this was a GPU infrastructure.
42:03So, what else would you do with a bunch
42:04of Blackwell GPUs?
42:06>> Yeah, we we throw around this this word
42:08sovereign AI, and it it gets entangled
42:11with the concept of a neo cloud many
42:12times and and people even when they're
42:14raising money they use this word a lot
42:16because they view compute as
42:17infrastructure. And I think that's
42:19exactly right. This is just like the
42:20telecommunications build-out where this
42:22is this infrastructure is viewed by the
42:25sovereign nations that are deploying it
42:27and are involved in this as a asset that
42:30the country would like to control and
42:32have sovereignty over and they believe
42:34that that similar to what you're saying
42:36at the beginning of the show
42:38uh they want to be a net token importer
42:41or exporter and that is driven by the
42:43infrastructure they build which means
42:45data centers with GPUs inside of it. And
42:46so I think it's actually no surprise
42:48that uh federal government research and
42:51intelligence agencies are on some of
42:53these clusters sharing
42:56infrastructure with others. You would
42:57like them to have an air-gapped data
42:59center but if there's exactly one
43:00facility in the entire country with
43:02GB200 NVL72
43:05I mean it's a seller's market.
43:10>> Goodness. Uh all right. Well, there goes
43:12my sleep tonight. So uh what's
43:15you know you you you do in this report
43:16also rank uh all you know a lot of these
43:19hyperscalers um into tiers right and and
43:22the neo-clouds into tiers and the at the
43:23top tier is CoreWeave and Nebius.
43:26Um I'm more interested in the second
43:27tier and that is Oracle and Google
43:30Cloud. Um mostly Jordan because
43:34you know I I I you know I have like in
43:36the document that I put together for you
43:38know our conversation today I have like
43:40a lot of like topics and sentences
43:42um that I've like pulled out of the
43:44report with questions and then with
43:46Oracle I just have Oracle and then a
43:48bunch of question marks cuz that's kind
43:50of kind of how I feel about that
43:51company. I don't know how to feel about
43:53about them. They're obviously their debt
43:55has been rated um you know not well by
43:58the rating agencies. They're making this
44:00all-in bet. Um they're they rate pretty
44:02high and they actually in your report
44:04have been credited with pioneering some
44:06techniques.
44:07Um but I would love to get your read out
44:09on on Oracle and, you know, whether
44:12whether this has been a smart bet for
44:14them and whether, you know, they're
44:15they're going to to rise in glory or or
44:18sort of fall in flames like a lot of
44:19people anticipate.
44:21>> I think the key thing about ClusterMax
44:23is it is not stock recommendations,
44:26right? Um, it is not anything to do
44:28with, "Hey, this neo cloud is going to
44:31build a gigawatt next year or 200
44:33megawatts." Um, which is far more
44:35relevant for the stock than, you know,
44:37what how good are they at operating a
44:39cluster, right? Um, has nothing to do
44:41with, "Hey, this data center that they
44:43said they're going to deliver with this
44:44much capacity next year got delayed by 3
44:46months." That's not what ClusterMax
44:48measures. ClusterMax is explicitly
44:49measuring sort of how how good is the
44:52cluster for someone to use if you rent
44:55the GPUs, um, how reliable is it, how
44:57secure is it, how's the storage, how's
44:59the network, how's the life cycle, how's
45:01the Slurm on Kubernetes, how are all
45:02these things? Um, in which case, you
45:05know,
45:06it's just sort of I we try to be more
45:07objective. Obviously, making a rating
45:09pyramid is like, you know, it is it is
45:11something you kind of have to do. It
45:12would be nice if you could just make a
45:14long report and no one cared if there
45:15were no rankings. I mean, that's what
45:17would happen. So, you kind of have to do
45:18the rankings,
45:19um, and and set certain boundaries and
45:21you can always litigate the hell out of
45:23why and what is in what tier. Um,
45:26but like, it is an objective fact that
45:28we think Oracle is one of the top neo
45:30clouds in terms of quality of service.
45:32Now, that doesn't mean on the other
45:33hand, "Hey, it's semi analysis in April
45:36we published to our clients that their
45:38New Mexico data center was going to be
45:40delayed because they weren't going to
45:41get the pipelines." And this was a big
45:43issue for Open AI because Open AI has
45:45contracted a huge data center with them
45:47in New Mexico. But if these pipelines
45:49don't get built, this data center can't
45:51go up. Um, and then later, you know, we
45:54said it again in, um, May, um, in June,
45:57and then most recently Oracle announced
46:00a force majeure on it and their stock
46:02tanked because of it. Right? So, so
46:04we're disentangling two things, right?
46:05Yes, Oracle was an amazing cloud at
46:07managed for their managed cluster, one
46:09of the top neo clouds, but they also at
46:11the same time may have execution issues,
46:13whether it's regulatory in the case of
46:15this New Mexico data center where they
46:16can't get the pipeline there, or other
46:18issues, you know, on their balance
46:20sheet, right? And so that is we're
46:21disentangling like what is good or bad
46:23for the stock versus what are we trying
46:25to test here. The goal in first
46:27ClusterMax was
46:29create something that people renting
46:30GPUs can use.
46:32Um and it turns out now like there's a
46:33lot of other uses for it that people try
46:35to ham-fist in, whether it's like, you
46:38know, um hey, we're going to write debt
46:39for this neo cloud. If their neo if they
46:42if they upgrade rankings from bronze to
46:43silver, their rate will actually go
46:45down.
46:45Um you know, things like this, or like,
46:47hey, I'm going to invest in stocks that
46:48are at the top of ClusterMax. Um so
46:50obviously these are like these are not
46:53the decision these are not what
46:54ClusterMax is designed for. Um it's a
46:57data point that you should use if you're
46:59doing anything with these companies, but
47:01um as far as Oracle
47:03as a company, um there's sort of like,
47:06you know, two things to go here, right?
47:08Like one is they signed a ton of AI data
47:11center deals um with OpenAI. Um was it
47:14last year? Yeah, it was last year. Holy
47:16crap, it was only last year. It feels
47:17like eons ago.
47:17>> Yep.
47:19>> I think it might have been exactly a
47:20year ago. Um
47:22but the interesting thing is they signed
47:23these massive deals,
47:25um and there's a number of issue risks
47:27embedded with them. One is they signed
47:30them with a fixed margin, essentially.
47:32Um so now that the price of GPUs has
47:34gone up a lot for clouds, they can't
47:36take advantage of this higher margin.
47:38So, they're kind of hemmed in there.
47:39[clears throat] Number two, um they
47:41raised a bunch of debt, and then the
47:42market started getting freaked out about
47:44how much debt companies can have, and so
47:45they spooked out, even though Oracle's
47:47completely solvent. And then number
47:49three, they have some execution issues,
47:50you know, I mentioned the regulatory
47:52one. There's some other data centers
47:53that have some delays and such there as
47:55well, but the main one being this new
47:56you know, such as in in Wisconsin, uh
47:58but the main one being this New Mexico
48:00site that is delayed potentially
48:01multiple years because of this pipeline
48:03issue. Um and so, you know, these things
48:06can have very negative impacts on the
48:08stock, but I think the company is still
48:09fine. Now, you know, they're not Nebius.
48:12Nebius gets to turn around and they had
48:14the they had the shortest average
48:15contract length of the top clouds in
48:17ClusterMax.
48:19Um and so,
48:20Nebius has been able to turn around and
48:21really jack up pricing and take
48:23advantage of this. Right? Whereas
48:25CoreWeave and Oracle, despite having
48:27also good cluster um, you know,
48:29management and cluster and reliability
48:31and security and all this stuff, were
48:33not able to take as much advantage
48:35because they locked themselves into
48:36long-term contracts primarily.
48:38>> Oh, that is fascinating. All right,
48:39Jordan, let me ask you a technology
48:41question then.
48:42Um
48:44you know,
48:45as Dylan put out uh put it out there, it
48:47is a pyramid.
48:49Um and there's like these top-tier uh
48:53groups like we talked about CoreWeave
48:55and Nebius, Oracle, Google Cloud is up
48:57there. But in the not recommended
48:59section, there's a lot of logos.
49:02Uh
49:02there's underperforming and unavailable.
49:05Um I think there's more, something like
49:07more logos in the unavailable than, you
49:10know, the top four tiers combined. What
49:12do you think that says about the fact
49:13that, I mean, what do you think that
49:15says about this technology that there's
49:17so many that are out there
49:19uh that just aren't meeting the bar for
49:20you?
49:23>> Look, as people who are trying to
49:24represent the interests of buyers, I
49:26think it just makes us sad. It's sad to
49:28see the
49:29uh
49:31Look, it's sad to see that like a year
49:33and a half, almost, you know, since um
49:36the first version of ClusterMax was
49:37published, actually approaching
49:39two years now. Um
49:42and since we coined the term neo cloud
49:44at SemiAnalysis, that a lot of people
49:46have just not made progress towards the
49:49things that we were writing about. And
49:51the things we were writing about is just
49:53exactly what we talked to the customers
49:55of these neo clouds about what they
49:56want.
49:57Um and so I I think it's just like the
49:59dynamic of it being a seller's market
50:01means that the priority for these
50:02companies is how can they raise as much
50:04money as they can, how can they build
50:05data centers as fast as they can, and
50:07how can they deploy chips as fast as
50:08they can, rather than working on the
50:10true quality of the underlying service.
50:13And that's that's a little bit of the
50:14dynamic that you see represented there
50:16where people can have these great
50:17businesses without actually worrying too
50:19much about delivering a great service
50:21that their customers really enjoy and
50:22recommend to other people.
50:24The other thing is that I think there's
50:25like this constant dynamic that we'll
50:27continue to see over time, which is that
50:29there can only be so many leaders
50:31because there's only so many people who
50:33truly innovate and then everybody else
50:34copies them.
50:36And when you have a market that for by
50:38our tracking is actually 323 providers
50:41as of this morning, uh which we track in
50:43our cloud TCO model for people who are
50:45subscribers to semi analysis, they have
50:46this full list.
50:48There's only a certain amount of them
50:49that are even relevant, like large
50:51enough to put on the list, and then a
50:53certain number of those that can pass
50:55the bar for what we believe is
50:57what we need to see to
50:59recommend them. And you know,
51:01in some ways we think that the bar for
51:03being recommended is quite low. Like the
51:07um
51:08the basic things that fail with respect
51:10to security and reliability as being the
51:12two top ones are just um
51:16in our view dead simple. And it's it's
51:18just sad to see that others aren't
51:21paying attention enough or caring enough
51:22to actually implement this stuff.
51:24>> Okay, two quick ones before we go. Um
51:26just a more broad question.
51:28Uh
51:29you know, Google obviously they're not
51:30they've announced they have this Gemini
51:33Argon coming out, Gemini for Argon. Um
51:36do is it your view that that there you
51:38know, there've been some people who've
51:39called upon Google to just be you know,
51:41an
51:43basically a neo cloud and rent out Uh,
51:45TPUs from Google Cloud and the
51:47computing. Uh, do you see them going
51:48that way, or do you think that they're
51:50still going to use like a large
51:51percentage of their compute for model
51:53training and inference?
51:55>> I mean, definitionally they have
51:57allocated less compute to uh, model
52:00training and deployment um, in the last
52:036 months than they had 2 3 years ago.
52:06Um, and that that percentage allocation
52:08has fallen over time. Um, this is sort
52:10of, you know, one of the things that we
52:11made a joke about like uh, maybe not
52:12even joke, but just a statement that
52:14Thomas Kurian won and Demis lost, right?
52:17Um, now I guess Karai. Um,
52:19Google is producing significantly more
52:22compute um, you know, year on year, but
52:26the amount that's going to DeepMind is
52:28not rising as fast on a, you know, it's
52:30not like, it's not like, oh, DeepMind
52:32had a third of Google's compute and now
52:33it still has a third of Google's
52:34compute. Um, it's actually falling as a
52:37percentage, uh, which is really
52:38interesting because now if I look at
52:40just DeepMind versus Anthropic and
52:43OpenAI, DeepMind has the least compute
52:45of the three.
52:47Um,
52:47and by the end of this year. So, like
52:50it's it's a really it's really uh, big
52:52deal. Um,
52:54and so Google, you know, whether or not
52:56there's a neo cloud, what the word neo
52:57cloud even means, um, you know, does
52:59selling tokens of other models count as
53:01neo cloud? Um, you know, does does, you
53:04know, in a sense that's what the new
53:06cloud business is is selling AI model
53:07tokens, right? So, maybe that's a neo
53:09cloud, right? Um, the definition is
53:10quite bizarre.
53:11>> Blackstone and going and doing new
53:12construction projects to build your GPUs
53:15to deploy directly for customers outside
53:16of your core platform.
53:18>> Or selling chips, right? Is that neo
53:20cloud? No, I mean, they're selling a lot
53:21of chips to Anthropic. So, it's, you
53:22know, physically selling the chips
53:24rather than like just renting them out.
53:26So, I would say like Google's business
53:27model's evolving and they want to make
53:28money at every layer of the stack, um,
53:31but by definition, you know, helping
53:33Anthropic this much with TPUs,
53:35um, and helping them this much with
53:37these other things is hurting Google
53:39DeepMind's business.
53:41>> Okay.
53:41>> I think that's
53:42they
53:43>> they need to find a niche, man. Like for
53:44Gemini, I mean, nobody wants the third
53:47or fourth or fifth best coding model.
53:49So, they need to find a niche that
53:51drives Gemini token sales if they want
53:54DeepMind to continue competing on the
53:55frontier of AI research and because it
53:58right now it's I mean, even with this
53:59Gemini 4 Argon release, it's just not
54:02there on coding compared to the top two
54:04or three.
54:05>> And and and the risk here is what
54:07happens next year when OpenAI and
54:09Anthropic both attempt to add 5 to 10
54:11gigawatts of additional compute for
54:14their labs.
54:15And you know, for OpenAI and Anthropic
54:17each, and then
54:19um,
54:19Google only adds a few gigawatts. Now,
54:22their compute deficit it it was Google
54:23was way ahead in compute and they were
54:25kind of behind {slash} equivalent. Um,
54:28mostly just a little bit behind. Um, and
54:30now they've fallen further behind, but
54:32their compute is the same. So, what
54:34happens when they have less compute? Are
54:35they Do they somehow magically catch up?
54:37It's possible, right? They have the
54:39talent, they have the people, they need
54:40to reorganize and really focus up. Um,
54:43but also like more and more of the best
54:44people just keep getting poached to
54:46Anthropic, to OpenAI, and it it's it it
54:48meta.
54:49>> Discovery loop.
54:50>> Discovery loop, yeah.
54:51>> Discovery loop, right. Okay, I I want to
54:53ask one more before we go if that's
54:55okay.
54:56You know, the I think Dylan, you said in
54:58the past that um, the the people who are
55:01getting most of the value out of the AI
55:03models that are created are the end
55:05users, right? The people that are using
55:06the Claudes, for instance, at the you
55:08know, at the base. Um, and of course,
55:10Anthropic makes that model available via
55:12API and to go into all these various
55:14uses. You know, it's it's going to have
55:16to do more licensing and and who knows
55:18if they'll be able to do products for
55:19all this. I'm curious if you think if
55:21you see a scenario where given the big
55:23numbers that we talked out about in the
55:25past,
55:27where do they keep their frontier models
55:29at home for longer, much longer, and use
55:32them to build their own products? Like
55:33we know they have the wet lab, we know
55:34they have Claude code
55:36uh, and co-work. Um, and then they don't
55:38make them available because they want to
55:39get more margin out of the
55:41infrastructure that they've they've
55:43bought and built.
55:44>> Yeah, so I mean I think I think we've
55:46already seen this, right?
55:48Anthropic had Mythos ready in February.
55:52Uh they've their model safety cards
55:54noted noted noted that they have two
55:56other models since Mythos that have
55:59released and publicly we still don't
56:01have full Mythos. We have a neutered
56:02version of it called Fable. Um and it
56:06and it took many many months before
56:07Fable even came out. They had the glass
56:09wing which was like Mythos but for only
56:11some subsectors and it was neutered in
56:13some areas, right? Not all areas. And so
56:15it's clear that Anthropic is not
56:16released new models. Open AI likewise,
56:18Astra was actually done a couple months
56:20ago and it only came out like, you know,
56:22it took a couple months before they
56:23released Astra. Um and their new model
56:26Bell, um they even showed a blog where
56:29they showed the performance of it in
56:30math um and how it, you know, solved all
56:34this stuff. Um
56:36and they have not released it. So
56:37clearly they are just using it
56:38internally, not externally. And and so,
56:42you know, the labs have already moved
56:44along this line, right? Where it's like,
56:46oh yeah, yeah, the gap between them and
56:48and Chinese open source is still 6
56:49months.
56:50Um they're kind of keeping that but in
56:52reality they've kept, you know, months
56:54of progress internal only
56:56um because the moment you make it
56:57external people can distill it um
57:00you know, and and all these other
57:01things. So I think, you know, Anthropic
57:03will likely release a new model in the
57:04coming weeks, you know, for their IPO.
57:07Um I think Open AI will likely have to
57:09release a new model in the coming weeks
57:11as well just because it's such a
57:12competitive environment. But at the same
57:14time they are like keeping a lot of
57:15their stuff internal only.
57:18>> Yeah, we were discussing internally some
57:19of the
57:20capitalistic incentives if if the
57:22incentives are actually going to
57:24motivate people to release more models
57:26and get them in the hands of consumers
57:27because of the incredible gross margins
57:29on the per token API pricing
57:32or if the incentive is to keep it
57:34internally so that you can pursue the
57:36end-user applications which are even
57:39more uh high value than selling the
57:42access to the model per token. It's
57:44going to be interesting to see how it
57:45plays out, but in one of those scenarios
57:48the capitalist approach will be to pace
57:51the frontier for yourself, right?
57:53>> And where which what's your gut on where
57:54it's going to go? Which one of those
57:56scenarios you think plays out?
57:57>> I think Anthropic has publicly stated
57:59that they are
58:01pacing the frontier by keeping
58:03themselves
58:04uh restricted, and that's the only way
58:06that their blog posts have manifest in
58:08reality. Nobody else is slowing down.
58:10They're only slowing themselves down. I
58:12think that's true. I think that's what's
58:13happening.
58:15And uh I think the incentive's to
58:18both improve the status of AI by solving
58:23things that are
58:25you know, like cancer that are just
58:27unequivocally good that everybody can
58:29agree is a good thing would be, you
58:32know, a positive incentive.
58:33And then I think there's also plenty of
58:35money to be made if you do something
58:36like that. So
58:38I my my bet is that we continue to see
58:40the minimum viable models released by
58:43both OpenAI and Anthropic to keep a lead
58:45over Chinese and the rest of the open
58:47source ecosystem.
58:48>> The website semianalysis.com,
58:51uh sign up for the blog as Dylan says,
58:53you know, sign up for the newsletter as
58:55I have. Uh it's a wealth of insight and
58:57analysis, and I think you'll love it if
58:59you're not there already. Uh and this
59:01conversation has just, you know, been
59:03the manifestation of that. So uh Dylan,
59:05great to see you again. Jordan, really
59:07nice to meet you. You guys are welcome
59:09anytime, and we appreciate your time
59:10today.
59:11>> Thanks for having us, Alex.
59:12>> Thanks, Alex.
59:13>> All right, good stuff. Thanks everybody
59:15for listening and watching, and we'll
59:16see you next time on Big Technology
59:18Podcast.
59:34>> Mhm.

About this transcript

This page contains the full transcript of Can AI Keep Growing Exponentially? Let’s Ask SemiAnalysis — With Dylan Patel and Jordan Nanos by Alex Kantrowitz, generated from the public captions YouTube serves with the video. The transcript has 11,795 words across 1,831 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.