YT YouTube2Text

YouTube2Text → Transcripts → Introducing Benchmark Breakfast with Jack & Ev

Introducing Benchmark Breakfast with Jack & Ev — Transcript

by Uncapped | Benchmark Breakfast · 12,489 words · 1,968 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00And I think I could joke about being
0:01just like a little bit bored of
0:04It's like I interviewed every
0:05>> Just on Just on all the VCs
0:07you've interviewed.
0:07>> And we could joke about that though.
0:09>> Yeah.
0:10>> But I asked them all the questions and
0:12you know
0:12>> Learned all the lessons.
0:13>> Maybe we come back, but it was time for
0:15time for an update. [music]
0:20All right. Well, this is the beginning
0:22of a new iteration of the pod where I
0:25guess um you know at some point I've
0:27been I've learned a ton, but at some
0:29point you've interviewed a lot of VCs
0:32and you say it's time to just talk to a
0:34VC that you talk that you talk to you 10
0:35times a day instead. But uh just
0:38changing up a little bit. So now we're
0:39going to do this thing Benchmark
0:40Breakfast we're calling it where we're
0:43kind of going to I guess I don't know
0:44mock you know a little bit of the time
0:46that we spend on Mondays and I guess you
0:49know what we what we do on Mondays at
0:51Benchmark has basically been no agenda.
0:53We get together as you know the five of
0:56us and we sit around talking about
0:58whatever came up that week or whatever
0:59we're thinking about. And I have and I
1:01were just saying, "Well, that would be a
1:02That would be a fun new version and if
1:04I'm losing if I'm losing steam on the
1:06current you know the current podcast
1:07like let's let's let's take a new
1:09route." So I'm excited to be doing this.
1:10>> 100% and I feel like whenever we tell
1:12anyone um about our Monday morning or
1:16just our our Monday partner meetings and
1:17the fact that they're typically 5 to 6
1:19hours um
1:20the two I mean the two responses we get
1:22is like that is absolutely insane. It's
1:24a little long. Uh it's like you know I I
1:26you know people will be like I can
1:27barely stand our hour-long partner
1:28meeting. Um and then the second question
1:30is like, "Well, what's the agenda?"
1:31>> I think that's a really important thing
1:32about our Mondays is you review the
1:33pipeline.
1:33>> Yeah, yeah, yeah. Yeah, their agenda is
1:35like you review a very strict list and
1:37um you know it feels like you're you're
1:38like a corporate vice president at
1:40Microsoft or something. Um but our
1:42answer is always there is no agenda. We
1:43we just think about you know it's a
1:45continuous conversation as I think our
1:47partner Eric says
1:48um from one week to the next. Um and I
1:51think um
1:52you know we we thought it'd be fun to
1:54like
1:55you know open source [clears throat] or
1:56or make public um uh, like the the the
1:59form factor of that conversation.
2:00>> So, one thing we talked about,
2:01obviously, is big news is Anthropic
2:03dropped an S-1, which is, you know,
2:06pretty critical to the industry. So, um,
2:08as somebody who can actually read an S-1
2:10of our of our mostly early-stage group,
2:13what was uh what was your reaction?
2:15>> Yeah, it's it's um it's it's
2:17it was amazing timing because I feel
2:19like leading up to the S-1 day,
2:22um, you know, there was a lot of market
2:23turmoil, there was a lot of carnage in
2:26the public AI trade, a lot of fud about
2:28Anthropic, like are they going to IPO or
2:30are they going to delay it? Um, and
2:32then, you know, they come out with these
2:34this S-1 and on its face, this is
2:36everything that you want from a
2:38technology business. And it's funny
2:39because most of the numbers in the S-1
2:43had been leaked at one point or another.
2:44Like there wasn't like that that much
2:46new information.
2:47>> could have stitched it together from,
2:49you know, if you believe the right
2:50things on Twitter. As you always say,
2:51Twitter's making us all dumber. But it
2:52was almost it was almost there. But it
2:53was like, you know, if you really were
2:55good at parsing through, it was all
2:56basically there, but I think
2:57>> another thing entirely to have all the
2:59numbers put together in one place.
3:02Um, and and again, it was it was like on
3:04its face, what you want out of a
3:06technology business typically is you
3:08want immense growth and you want
3:10operating leverage. Operating leverage
3:12is the sense that like you have this big
3:14upfront investment in R&D or sometimes
3:17with software businesses, you pay a ton
3:18for sales people upfront and they take
3:20some time to pay themselves back, but
3:22you invest all this money upfront, you
3:24have fixed costs, and then your revenue
3:25grows so much that and your income uh
3:28your incremental margins are so good
3:30that you become very profitable very
3:32quickly as you scale.
3:33>> Yeah.
3:33>> Um, and we don't have to go through
3:35every number and stat, obviously, it's
3:36all over X, it's all over the internet,
3:38but I think the most important visual
3:40from the S-1, in my opinion, there's a
3:42chart that I'm looking at um that you
3:43guys can't see.
3:44>> No, no, no, we'll put it up.
3:46We have the power of technology.
3:47>> That's true. We have we have the power
3:48of, you know, maybe even more.
3:49>> Unbelievable. Maybe we'll use AI.
3:50>> Um, but the the chart is um a is showing
3:54the relationship between their revenue
3:57and their total compute expenses per
3:59quarter. And so, it's both the compute
4:01expenses for serving customers, but then
4:03also training. And in Q1 of 2025,
4:06um Anthropic had 400 million-ish, you
4:08know, they only they do it in billions,
4:10and so it's only 0.4, so it's like 400
4:12million-ish of revenue against 900
4:15million of compute infra expenses. And
4:18so that's like a 0.4x ratio, um where,
4:21you know, compute is larger than
4:22revenue. Fast forward six quarters, in
4:25Q2 of '26, they had 11.6 billion of
4:28revenue against 5.6 billion of compute
4:31infra expenses. So a 2.1x ratio of
4:34revenue to compute. So like that, like
4:37the entire concern that everyone had is
4:38like, are the labs, are all these AI
4:41companies, you know, selling a dollar
4:43for 90 cents, um you know, or or 50
4:46cents? Like are they ever going to be
4:47able to turn a profit? Anthropic in six
4:50quarters did what, you know, it takes a
4:52lot of software and technology companies
4:54like a decade to prove out, which is
4:56like, no, these are money printing
4:58machines.
4:58>> printing machines.
4:59>> Um in a huge way.
5:00>> Yeah, and it's like the revenue just
5:02turned out that it's big enough to make
5:04up for all these fixed costs of
5:05training. And the other surprising
5:07thing, and you know, this has played out
5:09throughout inference and all sorts of
5:11areas of
5:11of the AI stack, but the the price per
5:15gigawatt has just surprised everybody.
5:17Like I don't think anybody
5:18>> Like the monetization?
5:20>> Yeah.
5:20>> Yeah.
5:20>> Yeah, and just like what the value of
5:22the frontier tokens are.
5:23>> That's right.
5:24>> has just worked has just made it all
5:25work.
5:26>> It's made it all absolutely work. And I,
5:27you know, I I don't think it like I
5:29think there was a lot of contention
5:32around the quality of the frontier lab
5:34business model that was somewhat that
5:36was partially answered by the numbers in
5:38the S-1. Like you cannot say that like
5:40Anthropic's business model isn't working
5:42on like a per unit basis or an aggregate
5:44basis. Like they had 8% operate like
5:47GAAP operating margins in Q2 2026. So
5:49it's clearly working. I don't
5:51necessarily think that means that it's
5:52going to be a slam dunk out of the gates
5:55in terms of how public investors in the
5:58public markets value it.
6:00>> Mhm.
6:00>> And if there's a ton of share price
6:01appreciation right out of the IPO.
6:04There's two reasons for that. The first
6:06is that there's still some battleground
6:07topics that people have uncertainty over
6:10that I think are somewhat fair. So, the
6:12first is, you know, Anthropic in Q2 grew
6:15massively. They grew 145%
6:17quarter-over-quarter. So, it's like
6:19insane sequential growth, but then they
6:21didn't grow as much in Q3. Um and
6:23there's a lot of, you know, pretty good
6:24reasons for this. Um but they grew 45%
6:27quarter-over-quarter, which is still
6:29which is still insane. Um but you know,
6:30it's like the deceleration from 145%
6:32growth quarter-over-quarter [snorts] to
6:3345% growth. People are like, "Wait a
6:35minute. Like this doesn't, you know,
6:37with SaaS with like if you go look at
6:38like the net new per quarter of like
6:40CrowdStrike or DataDog,
6:41>> Yeah.
6:42>> it's just like consistent upward
6:43trajectory. There's very little
6:45volatility to the growth rate. And then
6:4780% of the revenue is consumption-based.
6:50And so there's there's not like these,
6:51you know, if you had like, you know,
6:53Snowflake or, you know, Databricks or
6:55one of these It's like a lot of that
6:57revenue is like multi-year long-term
6:59contracts.
7:00>> mean, you have more like fixed
7:01visibility into the future growth.
7:03>> One of the things that's been surprising
7:04to me, which I wouldn't have I would not
7:06have guessed it at least at this point
7:07because like,
7:09you know, it's like the last generation
7:10of models pretty good now, mhm, but it
7:12still just seems that so much turns on
7:14who has the the very best model at any
7:17given point in time.
7:18And the leading edge, even if it's by a
7:20little bit and even if the last model
7:22could do most of the tasks, it just
7:24seems like it's worth a crazy amount of
7:25momentum. And we saw this go back and
7:27forth between the labs all year, and we
7:29saw it, you know, we saw it again sort
7:31of in the last month, and then you saw
7:33it over the summer, and
7:35that [snorts] that part is surprising to
7:36me that like people will always seem to
7:39go for the highest intelligence
7:42even with some price insensitivity.
7:43>> You know, you know what I think it it it
7:45probably is is there's just so much
7:47surplus.
7:49And what I mean by that is like
7:51whether, you know, whether you're paying
7:54$0 or $200 for like a Claude Max account
7:57or you're you're paying like next to
7:58nothing with, you know, some free
8:00service,
8:01the value that you can get as an
8:03individual is probably in like the at at
8:05in the thousands of dollars if not tens
8:06of thousands of dollars.
8:07>> a Keith Rabois-ism that there's like,
8:09you know, two task types. There's the
8:11ones where it like just needs to be
8:13completed and it needs to be right. And
8:15you don't get any more bonus points for
8:16getting any better past a certain point.
8:18And like, you know, that's often in a
8:20company context talked about, you know,
8:22with regard to like what type of
8:23executive you should hire. Are you
8:24protecting downside or are you looking
8:26for uncapped upside?
8:28And like filing taxes, for example, you
8:30kind of either did it right or you
8:31didn't. At some point, I just have to
8:33believe those types of tasks will get
8:34saturated.
8:35>> Yeah.
8:36>> But then if you think about writing code
8:38or doing sales or discovering, you know,
8:43drugs and biology or robotics,
8:46>> [clears throat]
8:46>> it's probably the majority of tasks
8:49people understand it as unbounded
8:51upside.
8:51>> Yeah.
8:51>> And in an unbounded upside world, if
8:54that's the mindset, you're probably
8:55always going to buy the better thing. I
8:57mean, I I do still think
8:59there's it's obvious that the sellers of
9:01tokens are doing really well.
9:03>> Mhm.
9:03>> I still think that
9:05I believe it'll get there.
9:07But I don't think that in all cases the
9:09buyers of tokens are proving to be that
9:12much more productive in all cases.
9:14>> That's yeah.
9:14>> I think it's totally
9:15>> There's still a lot of hope going on.
9:16>> Yeah.
9:17>> And, you know, obviously code code got
9:19from, you know, it was actually a year
9:22ago I think it was not being productive
9:24and people were buying a ton of it. Now
9:25I think it looks like it is totally
9:26productive. But I still think we have a
9:27way to go until like, you know, Perena
9:30Farms can buy AI and we're sure that
9:33it's a better business now.
9:34And I don't know like
9:35>> You don't know about Perena Farms.
9:36>> I don't know about Perena Farms. I'm
9:37just I'm just guessing.
9:38>> Chick-fil-A has an amazing engineering
9:39team.
9:40>> Yeah. And you think they're supercharged
9:41by AI? I bet they are.
9:42>> Everyone Everyone loves the Chick-fil-A
9:44I don't know why.
9:44>> I mean, well, robotics might help them
9:45to make but
9:46>> Yeah.
9:47>> But so I think there's a little
9:48diffusion that needs to happen on the
9:50consumption of token side.
9:51>> It's 100% true. And I think so then so
9:53so you have like weirdness in growth is
9:56I think one reason why people will be
9:58cautious about the stock potentially. Um
10:01there's debates. I think this is a core
10:03debate, which is how much code gen and
10:06related TAM. So like for code gen
10:08specifically, how long does the fusion
10:09take? And then how much does CodeGen TAM
10:11is left?
10:12>> Yep.
10:12>> And then what is the next thing?
10:13>> I mean, how do you think about the
10:14CodeGen TAM? Like, you know, it's a
10:19the analogy could be something like in
10:21in ZIRP, you know, time when everybody
10:24went remote. It was like Zoom got this
10:26absurd pull forward.
10:28>> Yep.
10:28>> And it was basically like everybody who
10:29wanted a Zoom account got it. And so
10:31Zoom grew ridiculously, and then it had
10:34maybe not full saturation, but like
10:36roughly it got saturated.
10:38And obviously, at least in our corner of
10:40the world, everybody who
10:42might want to use AI programming has
10:45been using AI programming for a while.
10:47And then you have market share trades,
10:48you have those kinds of other things,
10:49but you don't get 10x year-over-year
10:51from that forever.
10:52>> Yep.
10:53>> But then you probably have a lot of
10:54other places in the economy where it's
10:56not even close.
10:57>> Yeah.
10:57>> So, I don't know how you think about
10:59like the the code TAM.
11:00>> Yeah, that's it's funny cuz I like I
11:02think of back to your saturation point
11:04of like things that are saturated with
11:06like a Zoom call.
11:07Here's like a very easy saturation point
11:10that's like
11:10>> Can I get on? Do my headphones crack
11:12correctly?
11:13>> But you could always have another coding
11:14assistant. You could always have It's
11:16like you, you have [clears throat] like
11:1730 agents going at once probably, right?
11:19>> All the time. Yeah.
11:19>> You're just building stuff.
11:20>> Just building stuff.
11:21>> Yeah.
11:21>> Just just figuring out how to be mean to
11:23you in our in our partner meeting in the
11:25chat.
11:26>> And
11:26but with the
11:28um
11:29But with Zoom Yeah, it's like with Zoom
11:30it's like there there's a very clear
11:32ceiling and asymptote. And I think the
11:34really interesting thing that we've seen
11:35with code is like I think there probably
11:37is some asymptote, but the asymptote
11:39isn't like a crazy place. And what I
11:40mean by that is like some of our newer
11:42companies, like the companies that have
11:43been started kind of post Opus 45, I'd
11:45say, the way that they build software
11:48Yes. looks
11:49almost nothing.
11:51>> It's like nothing. It's crazy.
11:52>> Nothing like how you built software in
11:532019.
11:54>> I met up with one of our founders that
11:55we just seeded, and
11:58he you know, there's no other or there's
12:00like one other engineer on the team
12:01right now. But we went for a walk, and
12:04he's got like, you know, he's doing some
12:06stuff on his phone. I'm like, "What are
12:06you doing?" He's like, "Oh, I just I
12:08have like a few agents running, and it's
12:10going to be done by the time we're done
12:11and all this stuff." And I'm just like
12:13it's just a different there's just a
12:15different way to do it now.
12:16>> And just like the like the idea of a
12:17software factory um and it's like a it's
12:20like a new organizational structure for
12:22building
12:24software products and like a lot of it
12:26is not even human driven anymore.
12:28>> with in Silicon Valley, there could
12:29still be many acts to go.
12:31>> Yeah. And then I think it's like if you
12:33think that like okay, you either have to
12:35adopt that method of building software
12:37products or get left in the dust.
12:39>> Mhm.
12:39>> You assume that on some timeline
12:42the rest of the economy starts to at
12:44least look more like that than not. And
12:46that's where I think that like I'm
12:47relatively I don't know if you can go
12:50have like the 10x growth that you had
12:52again or something. But in terms of the
12:54durability of code gen, I think that you
12:55could continue to see like
12:57>> Double every year for a while. It's
12:58insane.
12:59>> 100%. And I think I think that I think
13:00there's like enough to do that.
13:01>> I mean the other thing around that is um
13:04all of these startups and markets around
13:07code the right answer has basically just
13:10been better than all of it. And it's
13:11been funny because like in a lot of
13:12these
13:13sort of let's say code related spaces
13:16>> Mhm.
13:17>> you know, you're like ah this can't
13:18possibly be that big or there's three of
13:20them or whatever. They just seem to keep
13:22going. And it seems like um
13:25the market pull on some of the code
13:27adjacent areas is ridiculous. It's just
13:29one of the things we talked about
13:30earlier this week.
13:31>> 100%. It's it's funny. So like a lot of
13:33people I think know about the sort of
13:36like AI bottleneck trade that happened
13:39in the public markets. A lot of people
13:40probably participated in in some version
13:42of that. Um but basically like after
13:44ChatGPT came out in November of 2022 um
13:47there was like you know, almost
13:49immediately AI demand first for ChatGPT
13:51and then other products started
13:53skyrocketing and people were like okay,
13:54well LLM training and inference runs on
13:56GPU, so let's buy Nvidia.
13:58It's like Nvidia goes up a ton. But then
14:00people were like what else goes
14:01into it like an AI data center? Like
14:03what else like goes on like what else
14:04will be impacted? What derivatives of
14:06GPUs are impacted? And so um I have like
14:09a list of these things that went
14:11through. So like 2023 servers and racks.
14:13Supermicro went up 10x. Dell went up a
14:15ton.
14:16Cooling equipment, so a company called
14:17Vertiv went up a ton. 23 and 24, Arista
14:20basically doubled in both 23 and 24.
14:23>> It's crazy.
14:23>> Um in 25, um uh
14:25power generation, Vistra went up 3 and
14:271/2 x. Um Oklo in nuclear, people were,
14:31you know, pulling forward the future of
14:32of nuclear. Bloom Energy went up 4x in
14:342025. Data center equipment, GE Vernova
14:38went up 2x in 25. And then 25 [snorts]
14:40and 26, obviously we've seen memory and
14:42storage, so SanDisk, Western Digital,
14:44Micron, Seagate all went up at least 3x,
14:47uh I think in 2025. And then now in 26,
14:49now that agents are now using more CPUs
14:52again, you have Intel up 120% this year.
14:54So it was like
14:55>> following an egg through a snake. It's
14:57just like
14:57>> And it's like a new derivative like
15:00every like every every every year. Or
15:02like there's like three new derivatives
15:03every year that you could have gotten
15:05like a multi-bagger return on in the
15:06public markets.
15:07>> Leopold spoke of this, you know, it's
15:09just like the prophet.
15:10>> Yeah, the prophet. But then there was
15:11like so then inventure, the version of
15:14this that happened is sort of like what
15:16what I think you could call like the
15:17code gen PMF derivative.
15:19>> Yeah.
15:20>> So like obviously like in so so ChatGPT
15:23comes out in November of 2022, but
15:24really this really kicks off in November
15:26of 25 when Opus 45 comes out and
15:29supercharges cloud code.
15:31>> That's right.
15:31>> And that creates this mega product
15:33market fit moment for code gen.
15:35And then like right after that you had
15:37like basically um and again this is sort
15:39of a parallel to all these public
15:41categories where you had vibe coding
15:42infra.
15:42>> Yeah.
15:43>> You had like back end as a service and
15:44databases, so Superbase took off.
15:46ClickHouse in our portfolios continued
15:47to accelerate like crazy. Um uh
15:49deployment hosts, so like Vercel, um
15:52Railway, Cloudflare. Um the inference
15:54clouds, obviously we're large investors
15:56in Fireworks which is growing like
15:57crazy, Base 10, Modal. Um data companies
15:59like Mercor in the Benchmark portfolio.
16:02Code sandbox companies like Modal and
16:04EDB are growing like crazy. Web search,
16:07Exa, also Benchmark portfolio company.
16:09Code review.
16:10>> I'm I'm glad you're getting all the
16:11plugs in as we go.
16:12>> I know, I'm [laughter] like I'm just
16:13hitting all the But it's crazy. It's
16:14like if if If you know, 10,000x more
16:17code in the world, all of a sudden like
16:19all of these derivatives of code
16:21generation also need to grow that much.
16:23And then um
16:24obviously you have a very exciting I
16:26know it's not public yet, but you have a
16:27very exciting investment in the CICD
16:29GitHub action space yep um because you
16:31you you know again durable execution
16:33with temporal, browser use APIs, and now
16:35supply chain security with with um you
16:38know, folks like Chain Guard and Socket.
16:40Like all of these categories, every
16:42single one I just mentioned have
16:43companies in them that are doing like
16:45the 10 to 100 to 300 [snorts]
16:47like you know, you start you know, one
16:49year 10, you go to 100, and then you go
16:51to like 300 or more.
16:52>> Which never used to happen.
16:53>> Never used to happen.
16:54>> And multiple are doing it in the same
16:55category at the same time.
16:57>> Yes.
16:57>> This is what was wild. I mean, you know,
16:59it's like lovable and Replit for
17:01example. Which have obviously done
17:03great. But it's like everyone was like
17:05isn't this you know, isn't this Club
17:07Code, isn't this Codex?
17:09Yeah, it is. And they're going to do a
17:11version of it that is form-fitted for a
17:14different purpose and the whole thing is
17:16so big that it's just all going to work.
17:18Which kind of goes to the Eric point of
17:19it's all going to work where you know,
17:21and I think you know, probably the
17:22modification on that is
17:24in some of these categories where the
17:26thing where the technology hits that
17:28hard,
17:29it's just it's so easy to underestimate
17:31it.
17:32>> Yeah. It really is.
17:33>> Yeah.
17:34>> It's like I I I think like code generate
17:36like AI code generation
17:38you can sort of abstract most of the
17:40value generation in AI today to code
17:42gen.
17:43It's like it's probably the most
17:45impactful product market fit moment in
17:48the history of technology.
17:49>> Probably right. Yeah, I'm trying to
17:50think what else would even I mean, it's
17:52in it's on it's on the order of iPhone
17:53moment or something like that.
17:54>> That's right. Except for you know,
17:56because so much of it is in the digital
17:57like because the infrastructure was
17:58already there.
17:59>> Yeah, it can just go so much wider so
18:00much faster.
18:01>> so much faster. Whereas like Uber and
18:03DoorDash and you know, these these um
18:06companies that were built on the mobile
18:07internet that were in iPhones.
18:09>> Yes, they were limited by the growth
18:10because they
18:11>> The growth of mobile that's right. Yeah,
18:12mobile phones had internet.
18:13>> Yeah.
18:14>> Yeah, it's fascinating.
18:15>> Yeah, and you know, it's funny like for
18:17like um knock-on effects of this.
18:20I think part of what's weird for
18:22founders and VCs is when you've got a
18:26thread like this that's pulling so hard,
18:28not just on code itself, but like you're
18:29saying all of the like accoutrements
18:31around it that support it and come
18:33before it and come after it and are just
18:34kind of all getting sort of yanked
18:37upwards.
18:38It's like other categories that are very
18:40good. It's like when the opportunity
18:42cost is people are like, well, I can get
18:43this ridiculous ride over here.
18:46>> Right.
18:47>> That has a big effect.
18:48>> Huge impact.
18:49>> And it's weird because at once it's
18:51making it so that venture demand in
18:55these spaces is so obscene and then LPs
18:58are obviously doing great and they're
19:01funding it and it's all part of like one
19:04rational circus.
19:06But then you have these other categories
19:08and the founders like, well, what the
19:09hell I have a good business and it's
19:10hard to raise and all that thing. So
19:11it's it's it creates a weirdness.
19:13>> I like I I had um,
19:15you know, historically I've I've spent a
19:17fair amount of time in in growth
19:19investing and you know,
19:21as a growth investor you you keep like
19:23all these historical precedents of like
19:25the growth trajectories of all these
19:26amazing previous companies.
19:28And I'll go back and look and I think
19:29like data, I mean this might be wrong,
19:30but I'm pretty sure DataDog went like 5
19:32to 18.
19:33>> Yeah.
19:34>> 5 to 18 and like 5 to 18 to like maybe
19:3640 to 50 or something.
19:37>> now 5 to 18 it's like I don't know if
19:39you can raise. I don't know if there's
19:40capital there.
19:40>> Would you I don't know, like would you
19:41look at a 5 to 18 or what would it take
19:43for you to look at a 5 to
19:44>> you would need in this in this
19:45environment you would need to believe in
19:4818 to 100 story for the next year or you
19:51would just need to think that they were
19:52doing something that was so difficult
19:54and unique that it just had insane
19:56growth durability forever after.
19:57>> Yeah.
19:58>> But on the surface that used to be
20:01enough where everybody would be super
20:02interested. If all you heard was it's a
20:04software company going 5 to 18. Yeah.
20:06Whereas now if that's all you heard,
20:07you'd be like, well, why wasn't it 5 to
20:0940?
20:10>> That's right.
20:11>> which is insane.
20:11>> It's so insane.
20:12>> But it's all because of this code pull.
20:14>> That's right. 100%.
20:15>> [snorts]
20:15>> Which you know, and like a related topic
20:18that we've been talking about is that
20:20there are sort of two archetypes or
20:22there's almost like a continuum
20:24of like how levered you are to the trend
20:29and then do you have like the old
20:31hallmarks of like durability, enterprise
20:35stickiness, um like some of the things
20:37that we loved about you know the the the
20:39scions of the last era like DataDog and
20:42CrowdStrike and you know even DataBricks
20:44even though they're not public. Um there
20:46there seems to be some form of trade-off
20:48between like how levered you are to the
20:52AI like basically the code gen pole or
20:55just like an AI trade in general um and
20:58some of the like more traditional um
21:02like signs of durability and
21:04defensibility from like an enterprise
21:06context where it's like oh this is like
21:07three-year long-term contracts and all
21:10these things. What have you like what do
21:12you think about sort of that trade-off
21:15um and um and if you're attracted to one
21:18or the other or anything that you think
21:19about that?
21:20>> Well, for I mean this might be wrong,
21:22but for me um I'm in a mindset right now
21:24where durability is really hard to call.
21:27Like I think in the last era by let's
21:30say
21:31the late 2010s,
21:33I think you had a good enough feel for
21:34cloud and enterprise at that point that
21:36you could not perfectly, but you could
21:38kind of name it when you're like that's
21:40durable or that's going to get competed,
21:43that's sticky, that's got a high
21:45switching cost. Like we got to a place
21:47where
21:48I think you could make those calls
21:50somewhat correctly because you had too
21:52many examples that looked like it and
21:54you just knew how software worked.
21:56You also had situations where
21:58enterprises would look at software for
22:00years before they were buying and
22:01they're like I know I'm going to buy
22:02Workday, but I'm not going to buy it
22:04yet, but when I do I'm going to use it
22:06forever. You just had dynamics like that
22:08where
22:09right now I don't know that durability
22:11is
22:13nearly as easy to identify. You know
22:15it's like even take like take a
22:17application software.
22:19I don't know 6 months ago everybody was
22:21kind of like it's dead. Yeah. At the
22:23moment I think people are like it's
22:24alive and it's good. And why should the
22:27labs do it? And there's all these edge
22:30cases to do. And
22:32they can actually like, you know, be
22:33value add to the intelligence tokens.
22:35And there's all these reasons to put
22:36your data in there and blah blah blah.
22:39And you need a UI. And, you know, that's
22:41great. And 6 months from now people
22:43might flip again and be like, "Never
22:44mind. Code got so good that that's all
22:47not true anymore." and all these things.
22:49So, calling the durability seems
22:51difficult to me. Yeah. I think um
22:55you know, I was thinking about like what
22:56is it about the 5D team stuff? I think
22:58if you are
23:00if you are trying to identify the next
23:02code
23:03the next thing that's going to pull like
23:05crazy, I think that's probably the
23:07reason that people get really attracted.
23:09You know, like as an example, I think we
23:11all believe robotics will work.
23:13>> Yeah.
23:14>> We would all probably say it's not
23:15exactly working yet. We can't
23:17[clears throat] have a robot in here
23:18doing, you know, a bunch of
23:20things that, you know, a human could do.
23:22You can't really have one at home just
23:23yet.
23:24But I think we all believe we'll have
23:26one. Like you and I both love our
23:27Teslas. You like see the beginnings of
23:29it. And you know, you go to these You go
23:31to cool companies like if you visit like
23:33Pi or something like that, you're like,
23:35this You know, you can see the future.
23:36>> Yeah.
23:37>> So, it's going to come. And so, you can
23:38bet ahead on things like that.
23:41Bio is another one where it seems like,
23:43you know, the smartest people at the
23:45labs seem to be saying it's going to
23:47work. There seems to be early
23:48indications that, you know, target
23:50identification and other things are
23:52possible that didn't used to be. And so,
23:54I think that to me is more the spirit of
23:57investing at the moment is either on the
23:59trend or you're calling the next one.
24:01And obviously the returns are highest if
24:02you call the next one.
24:03>> 100%.
24:04>> You're calling Yeah, I feel like that is
24:05um it's funny like as soon as you can
24:08sort of put in the chat
24:11um you know, like you know, for for
24:14obviously the segment of like what are
24:15all the derivative things. I went to
24:17chat and I'm like, "Well, we've seen
24:18these work. What are all What other
24:19trends have worked?" As soon as like
24:20chat knows
24:21>> That means it's the derivative trade.
24:23>> talking about
24:23>> Everyone's talking about it. And the the
24:25companies that you could invest in are
24:26probably already at like a billion plus
24:28valuation or something like that.
24:29>> Yes, and you can still make that call
24:30because you can believe that the market
24:32will still somehow just not appreciate
24:34how far it will go.
24:35>> Yeah, 100% 100%
24:37>> And that's fine. It's super fine.
24:39>> Yeah, I think I think on the
24:40on some of the like
24:42a final thing on the difference or the
24:45comparison between
24:47even both categories that are growing
24:50and like what is healthy growth versus
24:52you know, riskier growth and that
24:53there's like two sort of
24:56conflicting views. One is that like
24:58well, if you want to see like a really
24:59healthy
25:01like a sign of health for the AI
25:03ecosystem would be money that is not in
25:05the system
25:07coming into the system.
25:08>> And so you don't like when it just
25:09circles around?
25:10>> Yeah, so well, I I think there's two
25:12arguments, but I think the argument here
25:13is like, you know, okay, circular
25:15revenue and just like if you if the only
25:17people buying AI stuff are AI companies
25:20that is like, you know, it's just it's
25:22more and more leverage. Like that's more
25:23and more leverage on the system because
25:25if it doesn't work out then it all comes
25:26crashing down whereas like if large
25:28banks or large health care systems
25:30>> Yeah.
25:31>> in like Ohio and New York and Florida
25:33are getting immense value and buying a
25:34ton of AI, that is really healthy
25:36because that's that is like an
25:37independent non-AI oriented end market
25:41>> And it goes back to those are buyers of
25:42tokens.
25:43>> And those are buyers of tokens.
25:44>> Which is ultimately what needs to happen
25:45here.
25:45>> Yes.
25:46>> Spending a lot of money to make these
25:47tokens.
25:47>> Yes.
25:48>> They better be useful.
25:49>> They better be useful. And and like I
25:50think AI companies are much more willing
25:52to be risky in their token spending or
25:54token buying relative to like a big
25:57health care system
25:58>> Yes.
25:58>> with you know, 8% profit margins.
26:00They're not going to blow up their P&L
26:01>> Well, again, you know, if you're if you
26:03are an AI application company buying
26:05tokens
26:06>> Yeah.
26:07>> and then doing something to them and
26:08then giving them to somebody else
26:10somebody still needs to make people's
26:11lives better. Like at the end of the
26:12day, somebody needs to turn the token
26:14into a better life in a way that you
26:17couldn't have done without the token or
26:18at least not as well.
26:19>> That's right. But then the other the
26:21other side of the argument that I think
26:22also has a lot of merit is like, well,
26:24look like the AI companies that are, you
26:28know, also buying all of tokens,
26:29creating software factories they're
26:31going to become the massive companies of
26:33the future because they're going to
26:34outcompete all the companies not doing
26:35this.
26:35>> Yeah.
26:36>> And so, like this is like when you look
26:37at Stripe, like Stripe didn't get Now
26:40they work with a lot of old kind of
26:41economy companies, but they didn't get
26:43big because they worked with old economy
26:45companies. They got big because they
26:46worked with DoorDash and Shopify and
26:48Instacart
26:49>> Lift.
26:50>> and Lyft and like these and Uber, you
26:52know, a lot of the sort of like YC era
26:55>> Yeah.
26:55>> internet native companies got big. That
26:57got massive. And like that's what And
26:59then once they were massive, then they
27:00were it was much easier to attract Yeah.
27:02old economy companies because they were
27:03so like They're like, "Hey, we just
27:05helped DoorDash,
27:06>> Yeah.
27:06>> you know, kick the hell out of you. So
27:07like you should probably use us as
27:08well." And so I think that like I I have
27:10no problem I I have I don't really have
27:12any problem with AI companies buying AI
27:15like, you know, AI tokens and like the
27:17circularity because I'm like, "Look,
27:18like these are companies They're just at
27:19the frontier." And just like internet
27:21native companies were, you know, at the
27:23frontier of the internet and older
27:25economy companies didn't really buy into
27:27what they were doing until it was too
27:29late and those companies were very
27:30large. I think the same thing's going to
27:31happen in AI.
27:32>> Yeah. That's probably right.
27:33>> Yeah.
27:33>> How do you feel about the um
27:35the closed versus open source stuff?
27:37Like we've talked about this a bunch.
27:38>> Yeah.
27:38>> And obviously,
27:40you know, we are we are exposed to a lot
27:43of the open source world through
27:45Fireworks and and other things.
27:48Um
27:50what we've seen in general, I think, has
27:52been a rise in open source this year.
27:54>> Yeah.
27:54>> At the same time, obviously, the you
27:56know, the frontier labs have gotten
27:58massive.
28:00And you know, the price per token
28:01there's very different. The use cases
28:03are
28:04similar, but different. So anyway, like
28:06what's your latest on sort of open
28:08closed stuff?
28:09>> Yeah. I mean, if if you'll if you'll
28:11entertain me, I um
28:13I I feel like we probably have a similar
28:14synthesis of what we feel now, but I
28:18think what would be interesting to do is
28:19like maybe we steel man each side of the
28:22argument.
28:23>> Great.
28:23>> And then like that I think once you have
28:25the steel man for each argument cuz I
28:26think there's a sort of like a lot of
28:27like there's a lot of talk online about
28:30open source versus closed source, which
28:31I think is sort of off track Right. from
28:33like the reality on the ground. So I
28:35think if you steel man both sides, you
28:37can actually understand like, okay,
28:38well, what's the synthesis that probably
28:40blends the two?
28:41>> Great.
28:41>> Steel man. So like I'll I'll do I'll do
28:43close source, you do open source.
28:44>> you go first.
28:44>> Um so I I think so the steel man of of
28:47the close source argument and this is
28:49and what what we mean by this is like,
28:50you know, why would a company um you
28:54know, um get inference from a close
28:57source frontier API rather than from an
29:00open source API
29:01>> PI inference provider like a Fireworks
29:03using Kimiko 3 or any of these other
29:04models. Um US or Chinese open models.
29:07Um and people say, well, look,
29:09intelligence is fungible or it's like
29:11sort of a commodity. And if we take that
29:13seriously, that idea that like, you
29:15know, a unit of intelligence is this
29:16fungible thing, there's a lot of history
29:19and a lot of good literature about who
29:20wins those types of markets. And um when
29:24you're when you're optimizing for the
29:25cost of a fungible output. And it's um
29:28often times the winner is something we
29:29call a low cost producer.
29:32And so uh the defining feature of one of
29:34these low cost producers is that you are
29:36structurally as a company, you have a
29:38lower cost to deliver a comparable
29:40outcome. And so you can either charge
29:42lower margins or you can charge lower
29:44prices at the same margin as competitors
29:47or uh you can earn higher margins than
29:49your competitors at the same price. And
29:50so two famous examples in tech,
29:53SpaceX is probably the easiest one to
29:54understand. SpaceX has the lowest cost
29:57to them in order to launch a kilogram
30:00into space.
30:00>> Yep.
30:01>> in the world. No one can launch a
30:02kilogram into space as cheaply as SpaceX
30:05can. And that gives them a pseudo
30:06monopoly on launch. Uh at least at at
30:09the same level unit level of of
30:12capability. Um uh a previous one in
30:14computing is AWS. They had massive
30:17aggregated demand that created economies
30:19of scale. They're able to amortize a lot
30:21of investments over a massive customer
30:23base, which gave them lower cost
30:25relative to competitors in the CPU cloud
30:28business. So for AI, the equivalent um
30:32here would be how do you become the low
30:34cost producer of a unit of intelligence?
30:37So we could say this is like a given
30:38level of capability, latency, context
30:41length, and reliability for some task.
30:44Like, who can produce that task at a
30:46given level of capability the lowest
30:48reliably?
30:50Uh and I think proponents of the
30:52closed-source argument would say the
30:53leading frontier labs have the best
30:55models in the world, many of which are
30:57not released to the public or not
30:58released via API, and they're using
31:00those models and pointing them at their
31:02infrastructure to improve their serving
31:03efficiency.
31:05Um and and you know, co-design sort of
31:07their serving infrastructure with the
31:09actual models that they're training.
31:11Um they have the most aggregated demand,
31:13they're the largest, and so they have
31:14economies of scale. Like, the data
31:16center space that they're buying is
31:17cheaper for them on a per unit basis
31:19than anyone else because they're the
31:20largest purchasers of compute. Um and if
31:24all of these are true, if you have
31:25economies of scale, if you have like if
31:28we are in some weak form of RSI right
31:30now where the models are actually
31:32designing tre- like, you know, actual
31:34inference efficiencies um at at at any
31:37sort of rate that that is compounding,
31:40uh that means that you can have a
31:41low-cost producer of intelligence in
31:42form of in the form of an AI lab, and
31:44that means that this argument that like
31:46open-source equals cheaper is a misnomer
31:49and is [clears throat] not necessarily
31:50true because at the same level of
31:51margin, if this was true, a frontier lab
31:54at any point of the intelligence curve,
31:57like the capability curve, would be
31:59cheaper than a inference provider
32:01serving on an open-source system. So,
32:04that's sort of the economic argument for
32:05closed-source, and then I'll let you
32:06take open-source.
32:08>> Okay, so I mean, there's I'll I'll start
32:10with the economic piece. And obviously,
32:11as you know, I mostly I mostly buy, but
32:14economically, one difference is that
32:17because the you know, to
32:19there's the model cost, and then you got
32:21to do the inference on it.
32:24The model in theory has some training
32:26that's been amortized into the price.
32:30And so, to the extent that the labs want
32:31to be profitable or have, you know,
32:33enough margin that they make up for it,
32:35they do need to do something to make up
32:37for the training costs.
32:39>> [snorts]
32:39>> versus, you know, if you're using
32:41Gemini, you don't you don't the models
32:43the model was free. So, there's no
32:45amortization there. And then obviously
32:47you still need to run the inference and
32:48I probably, you know, it's probably
32:50right that you don't want to per sort of
32:52inference unit basis, the labs are still
32:54going to have the advantage over the
32:55inference companies, but you know, maybe
32:57in theory that's that's made up. The
32:58other thing is, you know, do the models
33:00care to take their margins down to that
33:03level and is that actually worth it or
33:05is that really expensive for them to
33:07And so, you know,
33:08the inference
33:09>> they don't then they don't have as much
33:10money to train. They don't have as much
33:11money to train.
33:12>> And also it's like, you know, if they're
33:14really focused on
33:15the the real frontier and they're like,
33:18you know what? Use your open source for
33:20the taxes. Use it for recommending
33:23which, you know, iPhone you should buy
33:25and should you get the duo or not. And
33:27we're going to go solve biology, we're
33:29going to go do robotics and it's just
33:30not worth it to go optimize
33:33two models ago to go do the basic stuff.
33:36And so, there's some there's basically
33:38some argument along the lines of
33:42we're going to get saturation on many
33:43tasks and as open source of the future
33:48is better than, you know, Opus 55 and
33:51Astra are today,
33:53that's going to be good enough for most
33:55tasks and the model the the frontier
33:57labs are not going to care to go reduce
34:00their margins, optimize two models ago.
34:02They just need to keep their eyes on the
34:03frontier. And so, maybe there will be
34:05some some savings for that. But what I
34:07would say probably
34:09is more than the money pieces and I I
34:11wish I actually had cuz to your point,
34:13the the the specific numbers on the
34:16margin are where it all turns on, right?
34:19You know, so like I and I don't have
34:20that on here, but I think more important
34:22is do you want to own your own
34:24intelligence? And this is like the Alex
34:25Karp stuff of just like you cannot trust
34:29as an enterprise. You cannot trust any
34:31company to own all of your data and to
34:34be fully reliant on any company and you
34:36need to own your own models. You need to
34:38own your intelligence. And I think
34:39there's an argument to be made there And
34:41I think for certain industries that
34:43trade is going to be worth it. Yeah. And
34:45particularly as the open source models
34:46get good enough,
34:48even if you're not saving any money,
34:49you're in control. And that's not
34:51nothing.
34:52>> And then even like think about the
34:53branding of like if you let's say you're
34:56all all your all your tokens are
34:58Anthropic tokens. And all of a sudden
35:00Anthropic enters your space.
35:01>> Yeah.
35:02>> And then your customers are like, well
35:04>> Right.
35:04>> Yes.
35:05>> You know, like what what model are you
35:06using?
35:07>> Yeah. [snorts]
35:07>> And you're like 100% of our models that
35:08we're using is Anthropic. And they're
35:09like, well then why don't we just go to
35:10Anthropic?
35:11>> you're Eli Lilly
35:11>> No special sauce.
35:12>> If you're Eli Lilly and all of your data
35:14lives in Anthropic.
35:15>> Yeah.
35:16>> And then Anthropic's like, hey
35:18by the way, BTW, we got some drugs
35:20coming.
35:21>> You're like, okay, well I don't feel so
35:22good now.
35:22>> Right.
35:23>> So I think there are some reasons that
35:24people won't want to do that. Yeah. And
35:25then I think, you know, there's there's
35:27situations where you can post train on
35:29open source and that can be useful. And
35:31if you've got a lot of data, that data
35:33can outperform, you know, the extra
35:35intelligence that you might get from the
35:37next model. And so you'll see some of
35:38that, too.
35:38>> I feel like I hear that pretty
35:39consistently from from founders
35:41that are using, you know, that don't
35:43necessarily aren't even selling an AI
35:44product, but just have a big product
35:46corpus and they're using AI throughout
35:47their products. Where they're like,
35:48look, you can really like if you really
35:50spend the resources to fine-tune and you
35:52have a really good trace data from the
35:54task that you're trying to do. Like yes,
35:56like maybe vanilla can be Claude 3 or
35:58whatever isn't quite as good as as as
36:00whatever the model, but like you can
36:02actually get a really small model. Like
36:04a like a really really small open source
36:06model. And if you fine-tune the hell out
36:08of it
36:08>> Yeah.
36:09>> like it it gets to like, you know, a
36:10Claude 3 or a Sonnet level performance
36:13and [snorts] it is dirt cheap. Yeah. It
36:14is dirt cheap. I think yeah, we yeah, we
36:16had a a large public CEO come in and
36:18talk to us about that. It's like there's
36:20clearly all these these these these
36:22cases where that's true.
36:23>> Yeah, but then you know, it's like
36:25the the the nice thing is we know the
36:27answer. The market's telling us the
36:29answer. And the answer is that the
36:31inference companies in combination are
36:33doing mid single digits worth of total
36:36revenue. Yeah. That's a reasonable proxy
36:38for open source cuz that's that's where
36:40it's running. And that's, you know,
36:42whatever percent of
36:44the labs companies.
36:45>> Yeah.
36:45>> And you can argue for that there'll be
36:47some mix up, but what we're basically
36:48seeing is that you know, open source
36:50might be might be five, it might be 12,
36:52it'll be some percentage like that
36:54unless there's some change
36:55>> That's right.
36:55>> of the overall spend and that's what
36:57will be there.
36:57>> Yeah, I think yeah, so that I so I think
36:58the synthesis between these two is that,
37:01you know, again, shout out Eric Vishria,
37:03um it all works it all works.
37:05>> Um
37:05>> But also like the way it all works is
37:07like that these these actually like
37:08they're probably not even rivalrous to
37:09each other. And I like I don't know if
37:11this would be the case, but like also
37:13I'm sure the frontier labs,
37:15you know, they're probably even happy
37:16about there being an open source
37:17ecosystem because it probably creates
37:20all like from an antitrust perspective.
37:21>> 100%.
37:22>> If it's like, "Oh, they're getting too
37:22powerful." It's like, "No, there's a
37:23healthy open source ecosystem where
37:25people can also use and that they're
37:27like I think that's probably also
37:28healthy. I have no [snorts] idea if
37:29that's an actual thing that they think
37:31about." But like right now, so if like
37:32this will be in excluding the
37:34>> Yeah.
37:34>> I'm not sure if this will be included,
37:36but yeah.
37:36>> Yeah, they But if you look at So like
37:39right now, if you look at the
37:41exclude the hyperscalers cuz we've no
37:43idea how much open source inference
37:45they're they're doing. Um but if you
37:46look at, you know, just the inference
37:48clouds as we've talked about and then
37:50you look at just OpenAI and Anthropic.
37:51So this is very imperfect, but it gives
37:53us some some okay proxy. By the end of
37:55this year, I think the main in the top
37:56three inference three or four inference
37:58clouds will probably do like six billion
38:00of run rate.
38:01>> Yep.
38:01>> ish.
38:01>> Yep.
38:02>> Um and then, you know, let's say OpenAI
38:04and Anthropic will do like 170 billion
38:05of run rate. Um ish. Um if they, you
38:08know, if their growth kind of
38:09extrapolates from public disclosures. So
38:11that's like three-ish percent thus far
38:13>> Right.
38:14>> between the inference clouds and the
38:15frontier and the top two frontier labs.
38:17>> Yep.
38:17>> If the growth continues
38:18>> Yeah, maybe if you add the clouds in,
38:20maybe it's maybe it's double at most or
38:22something like that.
38:22>> That's right. So maybe maybe it's like
38:23so we can say like five to 10% yeah um
38:26in terms of revenue
38:27>> Yep.
38:27>> for open source versus versus closed.
38:29But if you have the growth that the labs
38:31have had and that people I think expect
38:33again, this isn't a guarantee, so this
38:34isn't a projection, but if you do have
38:36the growth for three to four years on
38:37the frontier side and the labs combined
38:40are at a trillion
38:40>> Yeah.
38:41>> and open source takes 10% of that, that
38:43is like a hundred billion dollars of
38:44revenue and these inference like the
38:46inference market has consolidated. Like
38:48there's really only like three players.
38:49>> That's right.
38:50>> Like we have Fireworks, we have Base 10,
38:52we have Modal. And then it's like I mean
38:53there's there's other ones that are good
38:54and all that stuff but like they really
38:56are like the three champions of the
38:57market and I think we're going to see
38:59more consolidation to those three than
39:00less. So you have like probably two or
39:02three companies at that time splitting a
39:05hundred billion dollars of revenue which
39:07is incredible.
39:08>> I mean it's incredible. I mean it means
39:09one of those companies could be worth a
39:10hundred billion or more.
39:11>> Probably both.
39:12>> Probably both.
39:13>> a forty billion dollar company with like
39:15even if it's fifty percent gross margin,
39:16you know it's probably two hundred
39:17billion dollar company. You you have
39:19like hundreds of billions of enterprise
39:20value for open source and then you have
39:23the trillions in Frontier Labs and like
39:25everyone is very happy with that. Like
39:26that's that's huge like humongous
39:28companies all around and it's through
39:30the synthesis of like open source will
39:32always on revenue basis be some
39:34percentage of frontier and whatever that
39:36percentage is like the there's so much
39:39room for for the inference platforms to
39:40grow.
39:41>> By the way and then we can we can move
39:42on from this but um
39:44it
39:44>> it's like a a couple years ago I feel
39:46like it was um
39:48there was there was a worry of like man
39:50it seems like AI's either going to wake
39:52up the Terminator Skynet and
39:54everything's going to be a really bad.
39:56It's like too good.
39:57>> Yeah.
39:58>> Or it's just like not going to matter
39:59that much. Crypto all over again head
40:01fake.
40:02And at the moment and you know this is
40:05why I'm I'm I'm happy to see the market
40:07reacting positively to Anthropic. It's
40:10like at the moment like unless you know
40:13RSI takes us into a weird place like we
40:15might actually actually be splitting the
40:17middle landing where like the labs are
40:20extremely important but they are not
40:23sixty trillion dollar companies. Where
40:25it is not useless technology that
40:27doesn't help anybody. Like that that
40:30that's a nice thing.
40:31>> Seems like the happy path.
40:32>> It seems like the
40:33I feel like my my odd my my P happy path
40:36is higher than it used to be.
40:38>> And that's awesome amazing.
40:39>> Which is amazing.
40:39>> That's amazing.
40:40>> It's amazing.
40:41>> Um do you want to talk about um bio?
40:44>> Obviously you've you've got I I feel
40:45like you've been really interested in
40:46robotics
40:47>> bio?
40:47>> Yeah. I have been. I mean the
40:49I was going to go to the dark place of
40:50cyber.
40:51>> We could do cyber first.
40:52>> Let's do cyber. Because like that would
40:53be that is one of the scary things. And
40:55this this last week, you know, there was
40:57like the PHP thing, and people are very
40:59afraid. And understandably so. It does
41:02feel like we are in a point in time
41:04where
41:05one of the risks that is most imminently
41:08being expressed is cyber risk.
41:09>> Yeah.
41:10>> Where, you know, we we we've spoken to
41:12some like prominent CEOs in the last few
41:14weeks who have told us that like they
41:16pointed frontier models inward and said
41:19hack us. And the model's like, I'm going
41:21to hack the out of you. And it
41:23does.
41:24>> And
41:24>> And yeah, they they said that like day
41:25after day they were getting company
41:28ending vulnerabilities surfaced.
41:30>> Yes.
41:30>> That like if it had been actually hacked
41:32from an external force.
41:33>> are great companies with great founders.
41:36And you know, with great engineering
41:38teams.
41:38>> Like tech forward companies. These
41:39aren't like, you know,
41:40>> Yes.
41:41>> old companies.
41:41>> And, you know, obviously when you attack
41:44internally, you have crazy advantages
41:46that outside attackers don't have.
41:48You've got all the [clears throat] data,
41:49you've got the code base. It's just a
41:51different thing.
41:52But still, it seems like, you know, fear
41:55to be on the horizon. And then, you
41:56know, you see all this stuff. So, um
41:58obviously it's probably a good time to
42:00be a security company.
42:01>> Yeah.
42:02>> But, um it is also a scary It's a scary,
42:04you know, first exposure of real risk, I
42:06would say.
42:07>> 100%.
42:07>> Yeah.
42:08>> Yeah, the um and I think that this the
42:10really scary thing is that
42:14cyber attacks, at least for the public,
42:16I feel like cyber attacks historically
42:18have been sort of an abstract thing.
42:22Where like at worst though, like your
42:23your data was what like your password
42:26for this website was was hacked. And you
42:28get like some notice in the mail, and
42:29you like, you know, you change your
42:31password, [snorts] but like nothing
42:31actually happens. Yeah.
42:33And recently,
42:34um there's been a lot creepier and like
42:38more things that feel really personally
42:40intrusive that are both now possible and
42:42have actually been happening.
42:43>> Mhm.
42:43>> And so like they're like I pulled up
42:44like a few examples where like they're
42:48um there was a researcher who used
42:50Claude to disabled a webcam's recording
42:54indicator.
42:55>> Wow.
42:55>> Um so started recording without like the
42:58light coming on. Um and access the
43:01microphone in the firmware.
43:03>> That's scary.
43:03>> All those things like a French
43:04supermarket suffered an attack that
43:07manipulated refrigeration controls and
43:09actually spoiled food. So I think that
43:11the issue is that like now like the
43:13systems that are now get that we thought
43:15were secure
43:16because these are open source systems
43:18that have existed for 15 years with
43:19humans looking to patch them and hack
43:22them in all sorts of ways. Um like these
43:25these critical systems to like firmware
43:27and hardware and like the overall
43:29internet like these are now getting
43:30exposed.
43:32>> That that only have been able to be
43:33exposed via these AI models.
43:35>> Yeah, and you can patch the software but
43:36it's can't really patch the hardware.
43:37>> Yeah, and so it's like you know people
43:38are like oh like you know like update
43:41your smart TV. It's like what the
43:43like my TV?
43:44>> What's wrong with the TV?
43:45>> like you know and they're like oh
43:46disconnect it from the internet. Like
43:47well that's really scary.
43:48>> Yeah, if you have a Samsung friend smash
43:50it immediately.
43:51>> yeah smash all of your smart devices in
43:52your home or disconnect it from the
43:53internet. So I feel like it just feels
43:55so much more
43:57widespread and intrusive
43:58>> Mhm.
43:59>> Um than than before which is why it's
44:01now sort of like I I you know I
44:03definitely bought YubiKeys
44:04>> Yeah.
44:04>> Um a couple days ago like I want to go
44:06through the whole actually Andrej
44:08Karpathy has a really good blog post on
44:11digital hygiene.
44:11>> Yeah.
44:12>> And just things you should do and I feel
44:13like for the first time given the the
44:16parallelism and the scale that people
44:18can use and just like the extent to
44:20which that we're seeing as like
44:22strongholds like PHP
44:23>> Yeah.
44:24>> Um are are being hacked it's like okay
44:25this is this is for real.
44:27>> Yes.
44:27>> Um I think that the like really
44:28interesting thing so there's one other
44:30um
44:31piece of the cyber thing is this idea of
44:33obliterated models.
44:34>> Mhm.
44:35>> So an obliterated model is an open
44:37weight model that has its internal sort
44:39of safety guards removed.
44:41And
44:42um to help help with its work because
44:44you know ultimately like when the
44:45hugging face incident happened you know
44:47they they had some complaints where
44:48they're like we had to use an open
44:50source model because we couldn't turn
44:52off we couldn't get you know the
44:53frontier labs to turn off the safety
44:55guards in time to help us with the hack.
44:57Um
44:58And so, obliterated models are literally
45:00open-weight models that have no safety
45:01guards. So, it's very scary. And like, I
45:04think one of these is actually what um
45:06some some groups in China have been
45:07using. I think it's called Artex. Um uh
45:10have been using to to, you know, commit
45:12some of these hacks globally. It's also
45:14an incredible um defense tool for
45:16enterprises. Because you take an
45:18obliterated model, you do what that CEO
45:20told us, and you just say, "Hack us."
45:22Like, hack us to death. But like,
45:24obviously, like you're the one using it,
45:25and so nothing actually happens from it.
45:27And like, that is I think going to be
45:29like an incredible tool. Is like, you
45:31need these models that will Because
45:33guess what? The adversaries have
45:34obliterated models. The adversaries are
45:36not working with safety controls. You
45:38need to have that that, you know, you
45:40need to see the tools that they're using
45:42and use it on your own infrastructure.
45:43And I feel like we're going to hear a
45:44lot more about this idea of obliterated
45:45models.
45:46>> By the way, I think, you know, like the
45:48the tech world, which I think is mostly
45:50a good thing, you know, at first I would
45:52say reacted negatively to like pacing
45:56the frontier type stuff. And there's
45:57much more of just like a let it rip
45:58mindset. Which is, you know, it's like
46:00very libertarian, tech forward, good.
46:02Like, I think that's basically a good
46:03thing. But this is the the security
46:06dynamic is an instance where it not
46:08going too fast would be very helpful.
46:10Because you actually do want the
46:12defenses to penetrate before we have,
46:14you know, four generations forward of
46:16models.
46:17>> That's right. Cuz you do need things to
46:19get a little time to harden against this
46:20stuff.
46:21>> It is. But but
46:22>> And this is the crazy thing is like, how
46:24long do we think it's going to take to
46:26harden all this stuff?
46:27>> It's going to take a long time.
46:28>> Like, think about like, I don't know,
46:29think about like a hospital system in
46:31like West Virginia. Yeah. How long is it
46:33going to take for them
46:34to Cuz I think if you talk to some of
46:36these cybersecurity professionals,
46:37they're like, "We basically have to
46:38re-architect." This is like the biggest
46:40code modernization project in all of
46:42history.
46:43>> That's what we need cognition for.
46:44>> That's right. That's what we need.
46:45That's what we need cognition for.
46:47Um you know,
46:49>> But I mean, [snorts] I think like um
46:50>> I was going to try to do like an ad read
46:51there.
46:52>> Yeah. Cognition. Update your code.
46:54Cognition, update your code.
46:55>> Yeah. And security.
46:56>> That's good.
46:56>> But it's like um
46:58it it's going to take a long time, but I
47:00mean
47:00the
47:01the hope I have is that you know, like
47:04we were talking about you know, in the
47:06cloud era, it's like people bought, you
47:08know, Workday and Salesforce on these a
47:10really elongated timelines. They were
47:12slow and you know, if you were talking
47:14to a government or a hospital or a law
47:15firm, it was like 36 month cycles on
47:18some of the stuff.
47:19>> Mhm.
47:19>> That's not really how it's going right
47:20now on the purchasing of software side.
47:22>> Right.
47:22>> I think people are pretty alert.
47:24>> Yeah.
47:24>> So the hope would be that people see the
47:26security stuff for real. People are
47:28making it a top priority and everybody
47:30reacts a lot faster than you think. I
47:32still don't think that that means it
47:33happens in
47:343 months.
47:35>> Yeah.
47:36>> But maybe in 12 months people could do a
47:38lot.
47:39>> That'd be amazing. People are
47:40>> I think people see the urgency.
47:42>> Yeah.
47:43>> Hopefully.
47:44>> Yeah, maybe as a as a last topic, the um
47:47I think what's cool is the bio thing is
47:49interesting to me because
47:53I've kind of been of the view
47:54>> I still am of the view
47:56>> that all the software improvements that
47:58we're making are awesome.
47:59>> Mhm.
48:00>> But there is a ceiling to how much that
48:01can improve the world.
48:02>> Mhm.
48:02>> And if you like touch some grass and you
48:04get out of San Francisco and you're
48:06like, what will actually make the world
48:08a better place?
48:09It's not that we're shipping software
48:11way faster and way more easily. It's
48:14things like people have better housing
48:16or people live longer or people live
48:19healthier or people have food or people
48:22have transportation. Like it's physical
48:24things.
48:24>> I just think there's a
48:25>> there is a there's a limit to what you
48:28can get when it's all sort of in the in
48:31the screen.
48:32>> Yeah.
48:32>> And bio to me would be like one of the
48:34very best possible things if we're
48:37discovering drug targets and you know,
48:40finding amazing new, you know, drugs
48:42quickly or at all. That would be What
48:45are you laughing about?
48:47So anyway,
48:47>> You were on a good tirade.
48:48>> I was on a good tirade. So anyway, but I
48:50think that it is both really promising
48:52and it seems like it's on the horizon.
48:54>> Yeah.
48:54>> Same as robotics, by the way.
48:55>> Yeah.
48:56>> Those are the Those Those are the two to
48:58me where you see the you see the labs
49:01both talking about it, building teams
49:04around it, purchasing huge amount of
49:06data around it. You know, they're
49:08writing about it publicly. And so I
49:10think that that to me is likely an
49:12indicator that it's there. I mean, I
49:13haven't yet seen the labs be wrong
49:15>> Yeah.
49:15>> when they say something's coming.
49:16>> That's right.
49:17>> Getting the timeline right is really
49:18difficult. You know, with self-driving
49:19it took now it's fully here, but it like
49:22took a while.
49:22>> Yeah.
49:23>> And you know, self-driving, you know,
49:24Tesla's obviously dominant.
49:26>> Yeah.
49:26>> But a lot of that was
49:27>> because of just like infinite data.
49:29>> Right.
49:29>> And once you get enough data at these
49:30things, like maybe it just works.
49:32>> I think that's and that that's the
49:33there's this really weird sort of like
49:36inverse relationship between
49:37>> I completely agree. I feel like you you
49:40do you want like the you want the rest
49:42of America who doesn't have like an
49:44amazing, I think, view on AI, or
49:46[clears throat] at least a lot of people
49:47don't, throughout the world and in
49:48America don't have an amazing view on
49:49AI. Like, do you want to change
49:51perception of AI like solve disease,
49:54save people's lives, help people live
49:56longer,
49:56>> Yeah.
49:57>> build more housing so housing prices are
49:58lower, like create physical abundance?
50:01>> That's what makes a better world.
50:01>> That's what makes a better world.
50:02>> Yes.
50:03>> Um and and I the weird thing though is
50:05that like there's almost this inverse
50:07relationship with those things, the
50:08things that like when you actually touch
50:10the physical world could actually make
50:11the physical world a better place, and
50:13the things that have been able to get
50:15better very quickly from AI. And so if
50:17we take like you know, like AI just
50:19tends to improve fastest when it can
50:21generate large amounts of trustworthy
50:23feedback cheaply. I think it's like a
50:25good way to abstract it. So if you look
50:26at like
50:27>> code, um you know, it's like this
50:29incredible area for this because it's
50:31extremely verifiable. Like you know, you
50:33can do a unit test, you know if it works
50:34or not. You can scale up experiments
50:37extremely cheaply. They're like
50:38infinitely scalable. It's all in the
50:40digital space.
50:41And so when you think about like cloud
50:43code or like how cursor improved its
50:44models very quickly, or even ChatGPT
50:47when they started getting feedback from
50:48users, all of these were in a digital
50:51realm where there was an API and
50:54customers could come on at the click of
50:56a button
50:57>> and start, you know, contributing into
50:59the leviathan of training data for these
51:01labs.
51:02>> Yeah. And in the real world, robotics
51:04and now bio, the tough thing is that
51:05like you you have real bottlenecks to
51:08that feedback loop. Like in robotics,
51:09obviously like robots are kind of
51:11expensive. Like they're expensive to
51:12deploy, expensive to maintain. There
51:14isn't like a big customer base that can
51:15press a button. I mean, you could order
51:17a robot and you could start doing these
51:19things and Pi has an API now and you can
51:20do the Sunday beta program or any of
51:22these things. But it isn't like ChatGPT
51:24where it's like you can get 100 million
51:25people in a few months via a website.
51:27Yeah. And so like that's just very
51:29different. Bio, you also have natural
51:30rate limiters. Like the equivalent of
51:32like running a unit test in biology is a
51:34physical experiment. And like that, you
51:37know, like it could, you know, like the
51:39and even that experiment might only
51:41answer like a small part of the question
51:42you actually care about. And a lot of
51:43things do just take time. They just take
51:45time. Like you know, it's like I don't
51:46know a physical experiment. Like even if
51:47you speed it up,
51:48you know, making your own virtual
51:50>> to parallelize. That's what it is. And I
51:52think so in bio, it's like you can
51:54imagine a virtual sort of or like a real
51:56lab but where you're sort of, you know,
51:58kind of run somewhat autonomously or at
51:59massive scale. Where you're just doing a
52:02ridiculous number of thoughtful, you
52:05know, trials at once. I think that could
52:06be like, you know,
52:08the the time will hold you back but the
52:11parallelizing will still speed things up
52:12quite a lot, I think.
52:13>> Yeah. And I but I think like this and
52:15this is I I know like you you have an
52:17investment that you recently did in in
52:18this general space for robotics.
52:20We've we've been very interested in
52:22things happening in bio as well. But
52:23like the infrastructure to I mean, what
52:26we've obviously seen it with the AI labs
52:27as well. Like Recurs as a business has
52:29grown to an unbelievable amount of scale
52:31because there's basically there's just
52:32no limit
52:33>> Very unintuitively, by the way.
52:34Everybody kind of thought that like the
52:36data thing would run out and they would
52:37buy the data and then they bought the
52:38data and now they can move on.
52:40>> So like turns out it just turned out to
52:41be wrong.
52:42>> Yeah, reality has a surprising amount of
52:43detail, as they say. And there's just so
52:46many different things that these labs
52:47want to train on. There's so many
52:48different data types that emerge. And I
52:50think like how much we've seen in the
52:52digital space, in the digital data
52:53space, we're going to see
52:55I think you have to see more. Like
52:57Brendan, I don't know if he was the
52:58first one to come up with this, but
52:59Brendan and Foodie at Recurs says um the
53:01internet was a one-time subsidy on the
53:04product on the production of LLM's.
53:06>> Yep.
53:06>> And these physical spaces, they don't
53:07have as much of that one-time subsidy.
53:09They don't have the internet. Um they
53:10have some data. There's a lot of bio
53:11data. There's some robotics data, but
53:13like nowhere near
53:14>> And you also need a ridiculous diversity
53:15of data, it turns out, to build these
53:17models well.
53:18>> 100%
53:18>> So it's just it's just not going to
53:19happen overnight.
53:21>> But I think it's it's going to be like a
53:22civilization scale thing.
53:23>> Yes.
53:24>> Um maybe that's maybe economy scale
53:26civilization scale is maybe a little
53:27dramatic, but like to get these things
53:28where they need to be
53:29>> Yeah.
53:29>> it's going to take an it's I think it's
53:31going to be like, you know, Manhattan
53:32Project level scale type of thing.
53:35>> It'll be a bigger It'll be a bigger
53:36effort to make it actually happen. But
53:39if you imagine what's really possible
53:40with, you know, good AI robotics, it's
53:44probably bigger than It's probably
53:45bigger than all the digital stuff so far
53:47if it actually works.
53:48>> I mean yes. I mean even like if you
53:49think about the GDP impact of I mean
53:51maybe it could get scary because if you
53:53extend lifespans, then you have like a
53:54even more graying
53:55>> Yes. It's too hard to think
53:56>> don't need to get too much in the macro.
53:58But like I you know it's like you know,
53:59not to get too personal with it, but
54:01like again, my dad got diagnosed with a
54:04very advanced form of cancer. I look at
54:06like the survivor cohort rates. Yeah. It
54:08was like, you know,
54:098% [snorts] by year two or whatever.
54:11Yet immunotherapy drugs produced in the
54:13last 10 years have given him like a
54:1510-plus year prognosis.
54:16>> Yes.
54:17>> Which was like like, you know, if you
54:18got the disease 12 years ago, he's dead.
54:21>> Yeah.
54:21>> And now he's in remission. And so like
54:23like and we are just at the start of
54:26probably a hundred to a thousand times
54:29that amount of things happening where
54:31it's like every single disease area
54:32every single cancer
54:34like you know, we solved everything else
54:36from AI and when we solve bio through
54:39all the scale of investment we're going
54:40to need to make, it's going to make
54:42probably a much much larger um impact
54:44than any of these other things.
54:47>> Well, I think that was enough for this
54:48one.
54:48>> Yeah. This was lovely.
54:49>> This is great. What are you going to do
54:50for Halloween with the family?
54:52>> Uh we're going to we're going to we're
54:53going to go trick-or-treating in an
54:55extremely dense neighborhood.
54:57>> Wow.
54:57>> And I'm going to just try not to lose
54:58the kids.
54:59>> That's great.
54:59>> That's always my goal.
55:00>> Is it in NSF?
55:01>> In the on the peninsula.
55:02>> On the peninsula?
55:03>> Yeah. Yeah.
55:04>> Yeah, not like
55:04>> Which you're welcome to come to.
55:06>> Well, Brett and I are going to be in a
55:07in San Francisco. Why don't you guys
55:08come down?
55:09We we might we might come
55:10>> We'll talk about it.
55:11>> And then do we do we get to do a review?
55:12Or is that next episode?
55:13>> The duo the duo is unbelievable.
55:15>> Do you have the duo?
55:16>> Yeah.
55:17The duo is unbelievable. With the with
55:19the open.
55:21>> Wow.
55:21>> It's incredible.
55:22>> You liking it? So, this is day two?
55:23>> This is this is yeah, I'm 24 hours in.
55:26>> And you love it?
55:27>> I'm obsessed with it.
55:28>> Amazing. But if it next episode, let's
55:30see if
55:30>> Yeah, yeah, we'll see we'll see the two
55:31of you. See if I returned it.
55:32>> Yeah. Yeah. I'll see you in the next
55:34episode. Bye.

About this transcript

This page contains the full transcript of Introducing Benchmark Breakfast with Jack & Ev by Uncapped | Benchmark Breakfast, generated from the public captions YouTube serves with the video. The transcript has 12,489 words across 1,968 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.