YT YouTube2Text

YouTube2Text → Transcripts → AI Progress Explained: From Math Olympiads to Navier-Stokes

AI Progress Explained: From Math Olympiads to Navier-Stokes — Transcript

by The TWIML AI Podcast with Sam Charrington · 11,860 words · 1,835 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00AI's progress in mathematics has been
0:02remarkably fast. Not too long ago,
0:05frontier models struggled with grade
0:06school math. Now, they're contributing
0:09solutions to problems that have resisted
0:11mathematicians for decades, including
0:13most recently the Navier-Stokes
0:15equations. But, as AI moves from tests
0:18we already know the answers to to more
0:20open-ended research challenges, it gets
0:22harder to understand what these systems
0:24are truly capable of and what their
0:26progress tells us about where AI is
0:28headed. My guest, Greg Burnham, leads
0:30capabilities research at Epoch AI, where
0:33he and his team are developing new ways
0:35to track AI's rapidly evolving strengths
0:37and limitations. Here's Greg on why
0:40measuring and understanding AI's
0:42capabilities matter so much right now.
0:44>> AI capabilities writ large have gotten
0:48to such a point where
0:50what they're good at and bad at is
0:52starting to have real impact on the
0:54world, like on things we care about for
0:56other reasons. We're past the academic
0:59exam phase of understanding AI
1:02capabilities. And for that, we need
1:04high-quality benchmarking, high-quality
1:05evaluations. And one of my big questions
1:08here is just can AI come up with new
1:10ideas? If AI systems get sort of
1:12superhuman at this, that's a a big deal
1:15for for the world, both in terms of
1:17economic and scientific progress, but
1:19also in terms of becomes harder to
1:22predict what AI systems are capable of.
1:25We use math as a testbed for this.
1:28>> I'm Sam Charrington, and this is the
1:30Foomool AI podcast. For over a decade,
1:32I've been exploring the ideas and
1:34innovations shaping the future of AI
1:36through conversations like [music] this
1:37one that help you understand what's
1:40real, what's next, and what matters.
1:42Let's jump in.
1:52Let's talk a little bit about AI
1:53capabilities research broadly, how you
1:56pursue that research, and why you think
1:57it's important.
1:58>> We're getting to a point in the in the
2:01trajectory of AI overall where
2:05AI capabilities can have a real impact
2:08on the world. So,
2:11maybe the transition could be
2:12characterized as school to work.
2:17In maybe even up through Yeah, up
2:20through 2025 most ways that we tested
2:24what AI could do looked more like human
2:27exams, like that you might give to, you
2:30know, to kids or to maybe advanced
2:33graduate students.
2:34>> I was going to say.
2:36>> Right, right. The The kid exams. It It
2:38is remarkable.
2:39>> the
2:40medical exams.
2:42>> And these like AI was perfectly good at
2:44these already end of 2024 beginning of
2:462025. So, we really saw saw this
2:49transition happening over time to to be
2:52clear. But, this is This is certainly
2:55the year where AI capabilities are
2:57impacting
3:01work activities that humans were engaged
3:04in anyway for, you know, for their own
3:07purposes. And so,
3:09as these capabilities grow, it's just
3:12important for us to keep tabs on them.
3:15They grow very rapidly in some
3:17qualitative sense. We see no No
3:19measurement we have shows any slowdown
3:23in what and how AI is getting better and
3:26better at any tasks that we're able to
3:28measure. And so, we
3:31think that just understanding the impact
3:33AI will have on the world it's important
3:36to know what it can do, what it can't
3:37do, and keep it and keep tabs on how
3:39that how that trajectory is going.
3:41>> I want to push back on one thing you
3:43said
3:44in terms of we're not seeing any
3:47slowdowns
3:48that can be looked at broadly like in
3:52terms of maybe the number of, you know,
3:54benchmark data points that we're looking
3:56at. You know, certainly we're seeing
3:58this broad performance.
4:00But, is it also true that like on a
4:03particular
4:05I feel like on a particular benchmark we
4:07get it like the, you know, for for
4:09expected reasons when we went from, you
4:12know, GPT-2 to GPT-3 like we're taking
4:15these huge steps up these benchmarks and
4:18now like there's just
4:19a lot less room left and we're making
4:21more incremental progress. Do you see
4:24that as a a slowdown in AI performance
4:26or is would you characterize it
4:28differently?
4:29>> I would say, well, I'd say there's two
4:30issues here. One is on the benchmarks we
4:33have, what does performance look like?
4:36Two is
4:38are there things our benchmarks don't
4:40measure that maybe there has been less
4:43or could even be negative uh
4:46improvement on but because and is there
4:48some reason why
4:51it's the things that are hard to measure
4:53where there's been the least progress.
4:54But on the first point, cuz this is a
4:56very this is a much more cut-and-dried
4:58statistical point. We absolutely see
5:01continued progress on benchmarks like
5:04a benchmark that was challenging for
5:06GPT-3,
5:08uh
5:08you know, many years whatever, six years
5:10ago, uh is now completely, you know,
5:14solved, uh aced by GPT-4. Benchmarks
5:18challenging for GPT-4, same for GPT-5.
5:21Now we've got GPT-6. So, we do some
5:23statistical work to try to aggregate
5:26benchmarks over time because the
5:29benchmarks that people used to track
5:32progress years ago, now every AI model
5:36off the shelf that you've heard of that
5:37you might use is too good at them. Uh
5:40so, people make new benchmarks and then
5:41you have to sort of do this like
5:42stitching of like, okay, well, a model
5:45that was pretty darn good at this
5:47benchmark, was not so good at this next
5:49one. And you can use like a you can
5:51stitch those together and get this sort
5:53of composite score. We call that we we
5:55have a methodology for doing this we
5:57call the Epoch Capabilities Index. And
6:00that's a unified way of tracking
6:02benchmarks over time and and we really
6:04do see this thing continuing to go up uh
6:07you know, pretty smoothly, linearly. And
6:10and that march of progress is uh is is
6:13steady. That's an important part. Uh and
6:15that that's truly a if
6:17>> [clears throat]
6:18>> um
6:18we we have we've made more benchmarks
6:21and AI systems that are harder, that
6:23models start off at zero and then within
6:26a year or two years they're at 100% and
6:29we have to make a new benchmark to keep
6:30up. Uh and that's that's really been the
6:32story of the last uh certainly the last
6:35three years I'll I'll I'll say with
6:36confidence there.
6:38>> It seems like that like normalizing
6:41across different benchmarks to
6:44produce a result that is coherent. That
6:49sounds like a very challenging problem.
6:51Like you
6:52you you describe the result as like you
6:56know, kind of continued linear
6:57improvement and
6:59I imagine if
7:00that's like a constraint, you can kind
7:03of map backwards to you know,
7:06uh to factors that will show you that
7:09continuous linear improvement, but you
7:11know, starting from
7:13you know, set of disparate benchmarks
7:15that have their own kind of you know,
7:17ranges and challenges and you know,
7:19where AI you know, and where AI sits in
7:22them and and kind of mapping this across
7:24different generations of AI and
7:27coming up with a you know, something
7:29that makes sense
7:31you know, objectively at the end and
7:32then having that thing show continued
7:35linear progress, that sounds like
7:38um
7:41you know, almost like too to be true and
7:44like suspicious that it is true.
7:46>> I I
7:46Yeah, I I agree. So so let let me go
7:49into that a little cuz I think it's a
7:51it's a killer point to understand. And
7:53it is surprising. So I think you're
7:55you're very right to be surprised.
7:57In fact, if you look at AI benchmarks,
8:01they are all correlated with each other.
8:03Even if the benchmark the scores across
8:06models are correlated with each other,
8:08even if the benchmarks are in nominally
8:10different domains. So maybe you don't
8:14expect you don't find it so surprising
8:16if it's like I've got one benchmark
8:18that's, you know, graduate chemistry
8:21questions and I've got another one that
8:23is coding puzzles. And the models that
8:26are good on chemistry are also good on
8:28software engineering.
8:29>> Well, they should be correlated largely
8:32by the the models themselves. Like as
8:34the models get better, they're going to
8:35get better across these, you know,
8:37disparate benchmarks.
8:38>> But but this
8:39Exactly, but that's that's the
8:41surprising point. So all that you need
8:43to be true to see this linear trend over
8:45time is basically every time, you know,
8:49Opus 4.5, Opus 4.6, 4.7, GPT 5.2, 5.4,
8:54every time these come out, if they get
8:56better on all benchmarks at once, then
8:59you're going to see this linear trend
9:00this this big over time linear trend.
9:03Like that's it's two sides of the same
9:05coin statistically. Why does this
9:07happen? Like this is a huge question.
9:09Like you're right to find that
9:11suspicious. And I would say there are
9:13two explanations that we see. And it's
9:17an important question which of these
9:19explanations is more true. And of course
9:21the truth is some mix, but but let me
9:23let me give the the two explanations.
9:26One is
9:28the AI companies are making darn sure
9:32that their model gets better at all of
9:34these benchmarks
9:36with each new generation of model. And
9:39the main mechanism you might imagine
9:40them doing that, if they're like looking
9:42around, okay, people care about
9:44chemistry, people care about software,
9:45people care about operating, but you
9:48know, answering my email. Like they they
9:50would collect training data one form or
9:53another that would help the model learn
9:55how to do these how to do these tasks.
9:57And that's something of a
10:01um that's something of a you know, very
10:03manual human in the loop process of just
10:05saying, hey, we're doing product
10:06development. Like they want this
10:07feature, they want that feature. Our
10:09users want all these features. And the
10:10way you do that in machine learning is
10:12you collect training data and throw it
10:13in.
10:14I'm oversimplifying, but that that's
10:16that. Um
10:17it's sort of like a shallow way to make
10:19it happen. That so call that shallow.
10:21The other one is deep. The deep way to
10:23make it happen is if the model general
10:26is if the AI system can generalize very
10:28very well. So yeah, you never trained it
10:31on chemistry, but you trained it a lot
10:34on math and software engineering, and it
10:37just got super smart, and with a modicum
10:40of chemistry basics, it was able to
10:43derive the rest from first principles so
10:45it's sort of thing. This is like a deep
10:47form of generalization. Both of these
10:49happened to some extent. Like kind of
10:51the revolution with GPT even GPT-3
10:55around that era, this is like quite a
10:56while ago, was oh, if you train it to
10:58like predict the next word, then it
11:01actually gets better at a wide wide
11:03range of language oriented tasks that
11:05people had been trying to handle
11:07individually, suddenly it just got got
11:09good at everything. And that wasn't
11:11because anyone was like, oh, let's have
11:13data specifically around
11:16sentiment analysis or or syntactic
11:20structural analysis. Like it wasn't
11:21anything like that. It was just give it
11:23give it this very general thing. But now
11:25we're on to this world where you have to
11:27like train it specific Right now,
11:28there's a huge industry of collecting
11:30all this rich training data, and it's
11:32really unclear how much generalization
11:34you're getting from, you know, in this
11:36deep way.
11:38Uh and
11:39and if and anyway, that's that's sort of
11:41a big question cuz that deep
11:43generalization means you could really
11:44see if that happened in a in a rich way
11:47where you just didn't have to even be
11:48trained on something at all to gain
11:50capability in that area, then you might
11:53have a or like radically superhuman AI
11:55that would be um pretty hard to to
11:58understand like to to bound its
12:00capabilities.
12:02>> It's still a little counterintuitive
12:03that this would be
12:05that this would be linear and maybe
12:08yeah, maybe part of the question is like
12:10are we talking about
12:14you know, linear across generations of
12:18of of model releases or linear within
12:21some regime like segment, you know,
12:23segmentally linear or
12:25>> The The thing I can say is
12:27we have like a statistical methodol-
12:29from a benchmark score. Like what are we
12:31talking about here? Our unit here is a
12:33score from 0 to 100% and typically the
12:37way these scores go just historically
12:39empirically is like they follow like an
12:41S curve. Like like they start out sort
12:43of bad, then they get better, they get
12:45better pretty rapidly, and then you can
12:47only get up to 100%, so then they sort
12:49of level off, takes them a while to get
12:50that last, you know, little chunk, and
12:53then they're at 100%. So you've you've
12:55got these like sigmoids. Uh is is what
12:58that S curve is called. We just got a
12:59statistical methodology for stitching
13:02sigmoids together, and what comes out is
13:05a linear function, and this looks like
13:07like this is a artifact of the
13:08statistics to be clear, but this sure
13:11enough looks very regular in its own
13:14statistical analysis terms over across
13:17generations of of models. Like
13:19absolutely, like it's been a
13:21if anything it accelerated a little with
13:23the development of so-called um
13:25reasoning models around late 2024 where
13:28the model started to get good at math
13:30and and software engineering and things
13:33that require lots of logical reasoning.
13:35Uh maybe like with a higher linear
13:37slope, but uh but this is just like a
13:39that's a statistical artifact. You now
13:40have to ask, "Okay, well, if your score
13:42is like one model score is 150 and the
13:45next model score is 152, like what do
13:48those two points mean on your benchmark
13:51aggregate index?"
13:53Now
13:54>> part of part of what I'm what I'm asking
13:57myself as I hear this
13:59described as linear is does it say that
14:02in spite of the model the frontier lab
14:05vendor claims that, you know, Fable is
14:10like dramatically better than Opus or
14:12that GPT-6 is like the step function uh
14:16you know, improvement over GPT-5.6,
14:19you know, linear implies to me that that
14:22it's kind of incremental and there's
14:23really been no step functions.
14:26>> This is um this is a really great
14:28question and yeah, the we believe this
14:30is one of the big uses of this
14:33statsy tool we've got is you can try to
14:36find,
14:37you know,
14:38trend breaks and you can try to find
14:40accelerations.
14:42Uh and we mostly don't see that. Yeah,
14:44like one way to put it that is
14:47everything just about is on trend.
14:49Fable,
14:51uh Astra, whatever is on trend, but the
14:54trend is crazy. Like
14:56>> [laughter]
14:56>> you got to you got to hold both of these
14:58together.
15:01>> Right. Right. Right. Right. Right.
15:03>> This is like this is sort of the zoomed
15:04out view Epoch takes with with
15:06everything is a lot of this we think the
15:09engine of AI progress is heavily
15:11mediated by raw inputs like GPUs, the
15:16training compute, like training data.
15:19Maybe those are the big ones. And then
15:20there's this like factor of the labs
15:22come up with algorithmic like
15:23innovations, like research
15:25breakthroughs. But we really think like
15:27a lot of it is coming from the scale up
15:29in the in the compute and the scale up
15:31in the training data.
15:32>> So until you see the data and the GP the
15:35data and compute supply chain like ramp
15:39uh significantly or discontinuously
15:42uh to be more precise, you're not going
15:44to see
15:45you're not likely to see discontinuities
15:48on the model side.
15:49>> I think that's I think that's right. A
15:51big thing we're keeping our eye on to be
15:53clear is
15:55are there the are AI abilities emerging
15:58that could cause a discontinuity on the
16:00model side? So this is why people talk
16:03about recursive self-improvement or
16:05sometimes they call this a software-only
16:08intelligence explosion. Uh and the point
16:10of these concepts is well, sure right
16:14now there's a slow it comparatively slow
16:17process for building more micro building
16:20more GPUs and collecting more human
16:22training data and whatnot. But if an AI
16:24system got really good on the
16:27algorithmic innovation side, could it do
16:30more with less or or do more with the
16:32with a fixed uh
16:34compute budget? And that could change
16:37the dynamics here. So the feedback loop
16:39wouldn't might no longer run through the
16:41physical world. We don't really it's not
16:43clear we see evidence of this yet. I'd
16:45say on balance we don't really see
16:47evidence of this yet. But it's the sort
16:50of thing that has spooked the people
16:52inside the labs that the AI companies I
16:55mean saying like gosh, this could be
16:57right around the corner. That's like a
16:58bit of the fear. And if that happens,
17:00then the pace we're used to
17:02we might get that discontinuity.
17:05>> Have you developed a particular
17:06benchmark to identify and track this or
17:10is it more
17:12the way you look at the benchmarks that
17:15we've been talking about, the
17:16statistical model that we've been
17:17talking about for an indication
17:20that, you know, there's some kind of,
17:23you know, exponential
17:25uh in the curve and and that's, you
17:27know, probably one of the likely causes,
17:29you know, for that kind of
17:30exponentiality discontinuity.
17:32>> All of the above. Uh we we want as many
17:35as many tools in our arsenal as as we
17:37can for this. So that to this statsy
17:40like acceleration detector is definitely
17:44a good uh is definitely a good tool. But
17:46we also sort of get, you know, down in
17:48the weeds with some of our specific
17:51evaluations and benchmarks saying, I'll
17:53give you an example.
17:55Here, this is some forthcoming work we
17:56have uh that that I'm happy to to
17:58preview.
18:00We take a recent AI research
18:03breakthrough or innovation,
18:07uh
18:07from humans, and we take AI systems, we
18:12don't connect them to the internet, and
18:13we choose AI systems that uh were
18:16developed and released just before this
18:18recent cute whatever the recent human
18:20innovation is, and we basically say,
18:23look, here's the metric that that
18:24innovation, the number that it makes go
18:26up, it improves this sort of efficiency,
18:28what whatever, some some metric. And we
18:31say, AI system, your goal is to improve
18:32this metric as much as you can.
18:34Uh but we don't tell it the innovation.
18:37And so we see, is it capable of
18:39replicating that human innovation that
18:42was highly relevant for AI, you know,
18:45research and development.
18:47And so far we don't see them able to do
18:49this. There's this nebulous idea called
18:52research taste, which is like, where's
18:55the good idea? Like, what what
18:56experiment should I do next? Where
18:58should I go hunting for uh big
19:00improvements more than the mundane
19:03improvements of just more compute, more
19:05data? Like, where should I look for for
19:07a breakthrough? And AI systems don't
19:09seem great at doing this in um at least
19:12in an AI
19:14R&D context. Uh, but but they're like
19:17getting a lot better at a lot of things
19:19that are adjacent to this uh, to this
19:21area. So, we we really do think this is
19:23an important, sometimes I almost call it
19:25a tripwire, that where you know, the the
19:27future is foggy uh, is some foggy
19:29landscape. We want to put a tripwire out
19:31there so that if AI ever does cross some
19:33threshold where it's able to do rapid
19:36improvements in AI algorithms
19:39themselves, then uh, we we want that
19:41tripwire to sound so we go, "Okay, gosh,
19:43watch out. Like maybe maybe something
19:45uh, maybe we're about to see a big
19:47improvement a big big um, you know,
19:49uptick in AI AI capabilities."
19:51>> And how does this relate to
19:54AI's performance on some of the math
19:57challenges? You know, certainly
19:59Navier-Stokes has been
20:01uh,
20:01in the news quite a bit recently. Prior
20:04to that, there were some Erdos results
20:06uh, and others like
20:08how do you think about the role that
20:10those math challenges play in this
20:12broader
20:14view of trying to understand AI's
20:16trajectory?
20:18>> Yeah, the very good question. Uh,
20:20if if I may, I would
20:23go rewind history just a just a bit here
20:26to um, say some of the say the
20:28trajectory we've been on cuz once you
20:31you step back, it's um, it's it's wild.
20:34Uh,
20:35so
20:36up until
20:40Yeah, up until
20:42fall of 2024, so we're talking 2 years
20:44ago.
20:45Uh,
20:46grade school math was still a challenge
20:49for these LLM based AI systems, the the
20:52kind of AI we're talking about here.
20:55And all and the benchmark there was
20:56literally a benchmark of 8,000,
20:59you know, word problems.
21:02So, and so yeah, so has so many apples
21:04that sell for this and
21:05>> GSM8K.
21:06>> GSM8K, that's the one. And
21:10GPT-4, which is 2023, had like done
21:12pretty well on this. So, people started
21:14looking at
21:15like adding into the mix a benchmark
21:17just called math, like four capital
21:19letters,
21:20that
21:21started pulling from some high school
21:23math competitions. And there was like
21:25slow progress on this, but but not a
21:27lot. And then at the end of fall of
21:292024, OpenAI comes out with the 01
21:32preview model, full 01 by the end of
21:352024. And this showed a huge spike in in
21:38math competition capabilities, but we're
21:40still just talking like medium hard high
21:43school math competitions. Like
21:44something, you know, thousands of math
21:46nerd nerdy math 16-year-olds were were
21:49into
21:50across the across the US. And
21:54and this became, you know, the new the
21:56new territory. My my company, Epoch,
21:58around that time released a benchmark
22:00called frontier math. At the time,
22:03that's all we called it. We now call it
22:05tiers 1 through 4 to distinguish from
22:08later versions, I'll get to in a bit.
22:10But the original frontier math consisted
22:13of problems that I would say go from
22:15advanced undergraduate to advanced grad
22:18student. So, sort of early career
22:21research warm-ups almost. These are
22:23problems that are no longer thousands of
22:28thousands of high schoolers would tackle
22:31them. These required deep background
22:32knowledge in in various areas that that,
22:36you know, niche areas. Like we're
22:38talking a a problem from you know,
22:41elliptic curves that you'd give to a
22:45grad student like shortly before they're
22:47going to embark on their novel thesis
22:49research. Just to get them familiar with
22:50the details of the of the niche that
22:52they're going to be trying to work in
22:54and make an original contribution to.
22:56Over the same period, we also saw high
22:58like so over 2025, we also saw high
23:00school math contests get completely
23:03aced. We got the gold medal on the
23:05International Math Olympiad by the um
23:09even by the beginning of 2026, we saw
23:12most of these frontier math problems
23:14solved, though not all of them had had
23:16in fact been solved by an AI system uh
23:19at that point. And so just like
23:22even just here, this is a very rapid
23:24trajectory from uh over the course of
23:27just a little over a year going from you
23:30know, like
23:32hobbyist high schooler to graduate
23:35student. Like very competent kind of
23:38you know,
23:39up there graduate student in terms of
23:41the problems they could solve. Really
23:43rapid AI progress.
23:46At the same time, we started to see
23:48people test AI systems on
23:51unsolved math problems. Uh problem like
23:54the the frontier was no longer something
23:57This is what I mean with We've gone from
23:58exam to on the job uh where it wasn't
24:03like here's your qualifying exam as a
24:05grad student anymore. It became more
24:06interesting to say, "Well, here's a
24:08small problem that maybe a mathematician
24:10a professional thought about for an
24:12hour, didn't solve. AI systems, can you
24:14do it?"
24:15And at that level, we started This was
24:19all just this year, like 2026. At the
24:21beginning of the year, there were like
24:22starting to be claims of oh, maybe
24:24here's a problem that was posed by a
24:26famous mathematician. This is maybe the
24:28Erdős problems. The This is just for
24:30background, there's this very prolific
24:32mathematician Paul Erdős
24:34who proposed over the course of his life
24:36maybe like well over a thousand, maybe a
24:39couple thousand problems. Other people
24:40still like haven't fully cataloged all
24:42the questions he asked. Some of those
24:44became central to to whole fields of
24:46math. Some of them were, you know, no
24:48one really paid much attention to,
24:50including Erdős himself. And AI started
24:52to notch a few solutions to some of the
24:56easier ones of those.
24:59Fast forward to May
25:01of this year and and an internal model,
25:04probably something like the Astra model
25:07that we now have publicly available, was
25:10able to solve a big one of those. This
25:12was sort of a a historic moment. May the
25:14so-called unit distance problem
25:17from
25:18which was a problem about how many
25:20points you can fit in the plane that are
25:23just one unit distance of one apart from
25:25each other. Sort of like very cool that
25:27you could frame it such a simple
25:28problem, but there hadn't been progress
25:30much progress on this problem despite a
25:32lot of attempts
25:34since Erdos had had posed it, you know,
25:36decades and decades before. And it was a
25:38serious problem. It wasn't one of these
25:39no one had thought about it. It was
25:41definitely had received a lot of
25:42attention and an AI system solved it and
25:44it was really this this was like a
25:47aha moment of oh my goodness like AI can
25:49solve problems that humans have cared
25:52about just independently.
25:55And we've seen a lot more of that since
25:57and the the you know, with the sort of
25:59the the recent wow moment being the
26:02solution of the Millennium Prize
26:04problem. This was, you know, the
26:05Navier-Stokes problem. This was a
26:07collection of this these Millennium
26:09Prize problems, a collection of seven
26:11problems that
26:13mathematicians sort of or at least one
26:15math institute set as like good goals
26:17for the new century. Like they were they
26:19were they're much older than
26:21the year 2000, but but they were sort of
26:23canonized in the year 2000 as hey like
26:26these are worthy targets of entire
26:28subfields of mathematics and it would be
26:30a big deal. And so one of them was
26:32solved. So it's been a hell of a hell of
26:34a ride.
26:35>> The historical context is super
26:37interesting. It you know, we're all
26:39living through this trajectory, but to
26:42step back and see, you know, how much
26:44has happened in the past couple of years
26:46is pretty pretty nuts.
26:49>> Yeah, you you have to as one of my
26:51colleagues say, you have to like not get
26:52frog boiled by it. Like like the
26:55>> [laughter]
26:55>> the temperature is rising
26:57and it's easy to
26:58I don't know. Anyway,
27:00so so there is this interesting gap so
27:03far. Like you might hear the phrase
27:05jagged frontier where the AI
27:08capabilities are jagged in the sense
27:10that they'll be superhuman in some on
27:13some axis and some you know, not
27:16superhuman still
27:17inferior in capability to human in in
27:20other axis on some other axis.
27:22And
27:24that
27:25remains the case so far in math.
27:28In particular, there's a couple things.
27:30In hindsight, the analysis by
27:32mathematicians of the solutions AI has
27:35found to these like big important
27:37previously unsolved math problems has
27:41it's always
27:42in first place, it's been nothing too
27:45super duper surprising. Maybe there was
27:48like a a direction that humans
27:52had overlooked or could have invested
27:55more in and AI sort of had the
27:58persistence. May maybe this is a good a
28:00good word or
28:02or the bravery or the temerity to like
28:04wade into these very in-depth intricate
28:07computations and calculations and sure
28:10enough came out the other side with with
28:12a solution. And humans look back and are
28:14like a
28:15that's not a crazy direction to try at
28:17all and maybe if we'd like gone in that
28:21direction, we we we would have come up
28:22with this. This isn't something that
28:24like would be surprising.
28:26>> What I'm curious about is
28:30are we seeing the AI systems come up
28:33with a solution to the problem or an
28:36algorithm to create the solution to the
28:40problem? And and
28:42I think I'm trying to like ask a lot of
28:44different things here. You know, one is
28:47like creative creativity, one is maybe
28:51levels of abstraction or the way it's
28:53like you know, thinking, abstracting,
28:55you know, around these problems. Um
28:59Yeah, I'll kind of pause and let you
29:01react to that.
29:02>> Yeah, let let let me let me throw out a
29:04couple things there. Let me know if you
29:06want me to expand on
29:07any of them.
29:09When humans solve these math problems,
29:11they they write up the solutions in
29:13paper and like papers, proofs that you
29:16know, this is true or this is not true.
29:18And those proofs are are not usually
29:21like not exactly algorithms. They're
29:22written in natural language. They have
29:24sort of
29:26They leave the reader to fill in some of
29:27the gaps. And
29:30AI systems are coming up with just the
29:31same kind of thing. Like like it's it's
29:34the same product. There's a lot of
29:35complaints about the writing style, but
29:37like the arguments, the substance of the
29:39arguments really are the thing we expect
29:42to see from humans such that if a human
29:45came up with this and yeah, maybe
29:46cleaned it up a little I don't know. Uh
29:49Like would be lauded for having like
29:51made a very insightful connection
29:54between two different areas or having
29:57gotten a an intricate
30:00argument just right and balanced
30:02different considerations to make
30:04everything come together to to land the
30:06proof. 100% like this is not a this is
30:10something where we're past the point
30:11where we can say, well, AI systems are
30:13sort of like
30:15not really doing something that humans
30:17would, you know, be impressed by on
30:19human terms. Now, AI systems are doing
30:21doing things that are
30:22>> And is there a way to categorize or
30:26qualitatively
30:28are we finding that and I think you
30:30spoke to this a little bit like are you
30:32know, is there a qualitative difference
30:34between you know, the best human
30:36approach and the AI approach in a sense
30:38of like
30:40you know, the AI brute forced it. You
30:43kind of spoke to this versus like pulled
30:45in things from some other domain and
30:48like created some elegant solution or
30:50are we past that point as well?
30:51>> Here I'd I'd give more nuance. Uh we we
30:55um
30:56there's an emerging sense of at least
30:59for the moment what an AI shaped problem
31:03looks like. Maybe I could say AI has two
31:06big advantages right now. One is it
31:08knows everything. Uh like it knows all
31:10the prior work. It's got the literature
31:12memorized. I'm speaking loosely, but
31:14that this is this is basically
31:17uh valid. Um
31:19the other is it's extremely patient and
31:21persistent. So if there is some
31:24intricate
31:25uh
31:26you know, calculation you have to do. I
31:28don't mean like numerically or
31:29algorithmically. When I say calculation,
31:31I just mean lots of equations and you
31:34have to make all the pieces work out
31:36just as a very high-level conceptual
31:38logical kind of computation, but still
31:41something very um intricate. Uh
31:45they just have the patience to to go for
31:47that. And if you take the Navier-Stokes
31:49example, I think it's not a coincidence
31:51that um OpenAI's reported uh system that
31:56that solved it included a swarm of
31:59thousands of different instances of AI
32:01systems trying lots of different things
32:03and sharing ideas where where where
32:05clearly it was able to like there is an
32:08element of brute force there.
32:10So there's degrees of brute force. This
32:12is where the nuance is. It wasn't like
32:14somehow uh
32:16you know,
32:17you you you asked it to like it wasn't
32:19solving this in by any means in the
32:21dumbest, most low-level way. Like we're
32:24we're past that point as well.
32:26They used to solve some problems in like
32:27kind of surprisingly low-level like
32:30grind it out kind of ways. Like that's
32:32not really what these look like anymore,
32:34but like the
32:35the high-level ver a higher-level
32:37version of that still is maybe what what
32:40you might what you might say here. So,
32:42and I like the frame of precise and
32:44intricate computations here. It's
32:47important I'll say one more thing about
32:49the Navier-Stokes solution. It's
32:51important to recognize that humans sort
32:53of put in place a lot of the structure
32:55through previous work that AI then went
32:58and as far as we can tell and humans are
33:00still figuring out exactly what goes
33:01into the AI solution, but as far as we
33:04can tell it builds on a lot of prior
33:07human work. The AI didn't like build it
33:08up from the ground. That kind of theory
33:11building as mathematicians call it, we
33:14still don't see that very much from AI
33:16systems.
33:17Last point I'll make before pausing on
33:19this is just uh
33:21remember it's point in time. Like
33:23remember that trajectory. We've seen
33:25very rapid uh so so mathematicians were
33:28saying we have to adapt our profession
33:30to the fact that now you can push a
33:31button and get what used to have been a
33:33career-making result. Like like you
33:36know, are also I think cautioning and we
33:38can't expect the capabilities will
33:40freeze right now. We we see, you know,
33:41that that linear whether it's linear or
33:44exponential is a matter of perspective,
33:46but it's going up.
33:47Uh and it's probably like we see no
33:49reason to expect it to stop there. So,
33:51this is a big thing Epoch is looking
33:53for. Can AI move from intricate
33:56computations plus lots of background
33:58knowledge to more of a
34:01developing uh new ideas more from whole
34:05cloth. Uh and and if so, and that's a
34:08big thing we're watching for in our in
34:09our own math benchmarking activities.
34:12>> Yeah, it's interesting to think that
34:17at least the way I characterize this as
34:18brute force versus pulling in things
34:21from other domains,
34:24those are
34:26um
34:27you know, those are not mutually
34:28exclusive. Like
34:30for for AI that has access to, you know
34:33all of the research from every domain,
34:35you know, an approach that could be very
34:38fruitful is to brute force the
34:39exploration of adjacent domains and pull
34:42that into the solution to a given
34:44problem.
34:45>> That's it, I think. A lot of what we see
34:48and maybe just to say the the other
34:50thing that would really supercharge them
34:53and bring them closer to full human
34:56parity or even human you know dominating
34:58human capabilities would be something
35:00like coming up with a new idea.
35:03And maybe like in calculus the idea of
35:07the derivative or the integral of a
35:08function. Like that was something that
35:10was you know maybe mathematicians in the
35:131500s had been like grasping for this
35:15idea and then Newton or Leibniz or
35:17whomever comes up with like hey, here's
35:20like I can I can describe a new thing, a
35:23new mathematical concept and that
35:25unlocks a lot for us. That lets us solve
35:28problems we previously couldn't solve.
35:31And so that's the sort of thing we don't
35:32yet see AI doing. It's also the sort of
35:35thing that's rarer for humans to do.
35:37Like a lot of human math progress has
35:39been well, I sat down and I put in the
35:41work and I did the computation and I
35:43worked out which was the dead end and
35:45which was the good path and here's the
35:46answer. Now XYZ is true. I've proven it.
35:50Or I studied one area and I noticed
35:52something and I thought it might be
35:54applicable to this nominally
35:55superficially different area. Like
35:56humans do that sort of stuff all the
35:58time. It is 90%
36:01vaguely speaking of human mathematical
36:02work from what I hear from
36:04mathematicians. The new idea stuff is
36:06rarer. So maybe we should expect it to
36:08be not the first thing AI gets good at.
36:10But so far we don't see AI really, you
36:13know, AI really doing that yet.
36:16>> How
36:17easy or difficult is it to
36:20define
36:21a new idea in such a way that
36:26you know we can easily distinguish them.
36:27Like, it it you know, if we think about
36:30kind of the evolution of this discourse
36:32around like, is it AI creative? You
36:35know, initially it was like, "Yeah, it's
36:36very creative. It created this like poem
36:38in, you know, in pirate. Like, that's
36:41creativity. That's, you know, that was a
36:42new idea." Uh but that's certainly not
36:45what we're talking about when we're
36:46talking about like advancing science,
36:49advancing mathematics, etc.
36:52Um
36:54Yeah, so how
36:55A A A how do we like
36:57how do we define that? But also, you
36:59know,
37:01from your perspective as someone who's
37:03studying this, is it like you know,
37:05we've got this long line of uh folks
37:07that are coming that says, "Here's this
37:09AI with a new idea." And you're like,
37:10"Yeah, well, let me evaluate it against
37:12my new idea criteria. No, it's not
37:14really that." Or
37:17are we actually like, is it pretty clear
37:19to everyone that the AI is not having
37:20new ideas and like, we're waiting for
37:23the first person to come and say that AI
37:25has the new idea? Like, how do you think
37:27about that whole space?
37:29>> Very very challenging nut to crack. Uh
37:32so, we, you know, try to approach it
37:35from different angles.
37:37One thing you can try to say is
37:42I've got a problem. I don't know if a
37:44new idea will be required to crack it.
37:47And this could be a problem in math or
37:48in science or or any quantitative area.
37:52But, humans have tried to crack it.
37:54Humans have tried to make, you know,
37:55this number go up, whether that number
37:57is the efficacy of some, you know, drug
38:00in fighting some disease or something
38:01like that. Or some you you know, or it's
38:04a math result of some you conjecture
38:07that that math have stumped
38:08mathematicians for a long time.
38:11And you can say, "Okay, well, I don't
38:12know if I'm not sure how I'm going to
38:14measure a new idea, but I can at least
38:16tell if this problem has been solved or
38:18this number has been made to go up." Or
38:21whatever that the concrete metric is.
38:24And you can then then you're you can at
38:26least check if that has happened. Now,
38:30I was probably a year ago more
38:31optimistic that there would be a tighter
38:34relationship between math problem that
38:37humans have failed to solve and post hoc
38:40qualitative assessment of creativity.
38:44Uh
38:45but uh but you know, that this is so
38:47this is sort of the combination that
38:49like, you know, double punch we're we're
38:51trying here. First you set a target
38:53that's hard on human terms that humans
38:55care about. The humans have tried to be
38:57creative to solve.
38:58And then you do post hoc review with
39:01experts and say, "What do you make of
39:02the solution? Why hadn't you found it?
39:05No offense.
39:06Uh you know, what what is in the AI
39:08solution that humans had missed and so
39:10on." And this is risky like risky in
39:13terms of humans being honest about it
39:15cuz you know, your your ego might be
39:17wrapped up in it or you might once you
39:19see the solution it feels more obvious
39:21in hindsight. So would you have
39:22described it like very very very risky
39:25uh but but you still you you try your
39:27best. And you know, you might say things
39:29like, "Well,
39:31uh how like here's an interesting
39:32approach that that we haven't
39:34operationalized in like in detail but
39:36it's it's a nice heuristic I think. How
39:38big a hint would a human have needed to
39:41solve this in in hindsight? Uh
39:44So for example, that unit distance a
39:47very prominent mathematician Fields
39:49medalist Tim Gowers
39:51did this sort of exercise on this unit
39:53distance problem once AI had solved it
39:55saying, "How big a hint and how like
39:59specific of a hint would I do I think a
40:01mathematician would have needed now in
40:03hindsight like to solve this problem?"
40:06Uh one one Yeah, and he came away with
40:09like an interesting exploration of well,
40:10actually AI sort of disproved this
40:12conjecture as opposed to proving it
40:14true. It said, "Actually, it's false."
40:15Even that is a huge hint. Humans had
40:17mostly thought it was true. They were
40:19wrong, but they've been trying to prove
40:20that it was true instead of looking for
40:22a counterexample. So, even just the hint
40:24of try to find a counterexample is
40:26pretty big. And then there was like one
40:28more piece and he was like, I think
40:30though very hard to say, we can't really
40:32do the counterfactual experiment. I
40:34think with just that
40:35with the like the sort of two-part hint
40:37it would have been possible for like a
40:39human would be like, oh, oh, okay, I
40:40know what to do now, blah, blah, blah,
40:41and like probably would have at least
40:43hastened a human solution. So, you can
40:45do things like that. It may be still
40:47very qualitative, but you try to be
40:49honest with yourself and be rigorous.
40:50And you could imagine doing a more
40:52rigorous version of this experiment. Uh
40:55and anyway, things like that. But then
40:56the the the fact is math is still for
40:59the most part a um
41:02a sort of discipline removed from
41:05physical real-world impact. Uh not
41:07always, but in surprise its impact is
41:09often hard to predict. There are domains
41:11where impact is not at all hard to like
41:13I'm in the chemistry lab and I'm trying
41:14to improve the yield of my synthesis
41:17process. And if AI you know, makes
41:19incremental improvements, I expect
41:21incremental yield. And if AI suddenly
41:22comes and says, "No, no, no, you're
41:24doing it all wrong. Let me like redesign
41:26your whole thing from scratch. I've got
41:27like a you know, maybe creative idea."
41:29Well, at some point you don't care if
41:30it's creative or not in the qualitative
41:32sense. You just care that you had saw a
41:34discontinuity in the yield of your
41:36chemical substance or whatever. And
41:38that's, you know, that's uh
41:40that that's a bigger bigger impact for
41:43the world anyway.
41:44>> What I took from that is that we filter
41:46on the problem and how hard humans have
41:48kind of banged their head heads against
41:50the problem. Then we look at the results
41:54that AI has come up with and try to kind
41:58of subjectively say, you know, is this
42:01based on a new idea or is it, you know,
42:03more brute force or something else like
42:06um but it's kind of imperfect and messy
42:09and like we're
42:10not really sure, but we're we don't
42:13think that we've come up with, you know,
42:15great examples of AIs coming up with
42:18uh you know,
42:20new quote unquote new ideas.
42:23>> W- One That's exactly right. W- One
42:25reference point I'd give is way back in
42:27uh
42:28gosh, well, the
42:29the teens. I forget which year the the
42:32famous Go playing system, the game of
42:34Go, AlphaGo. Uh that played
42:36>> had that thought earlier as we were
42:38discussing this. Like,
42:40if I remember correctly, the
42:43the
42:44you know, part of the solution was like,
42:47"Oh, that was an innovative move that no
42:48human ever would have done."
42:50>> There was a specific one. Yeah, exactly.
42:52>> a very specific move in there, yeah.
42:54>> Yeah, they call it move 37 cuz it was
42:57the 37th move in this game where uh
43:01yeah, apparently this was a very
43:03surprising move. Like you said, no human
43:05would have done it um in this particular
43:08position in this in this game of Go. And
43:12I think it's like encouraging for our
43:14admittedly quite subjective program here
43:17that Go experts looked at and were like,
43:19"Oh my goodness, like that's not, you
43:20know, that's that's new." And then it
43:23also became clear that that was pivotal,
43:25that that move gave the AI system
43:28playing Go a decisive advantage. And so
43:31that's sort of what we're looking for in
43:33in math. And at least, you know, maybe
43:35we have some hope that humans will be
43:37able to spot it when it happens. Uh but
43:40but yeah, that's that's what we haven't
43:41seen quite yet. And again,
43:45the trajectory's fast. The things are
43:47always a little more qualitative than y-
43:50you know, that you know, than they are
43:52or or continuous than discrete.
43:55Uh but, you know, so anyway, this is
43:56this is where we are now, which is kind
43:58of wild in its own right.
43:59>> Earlier when we were talking about
44:01frontier math, you kind of characterized
44:03it as, you know, one to four, I think,
44:06and then advanced. and we never circle
44:08back to
44:09>> Yeah, we we've got a couple we've got a
44:11couple uh two new sets of frontier math
44:14problems. We ditched the tier system,
44:16whatever.
44:17>> Okay.
44:17>> Uh so so now um these are what
44:21I'll lump them together cuz cuz it
44:23really is the
44:24the this is our our main tool. So we
44:26have uh
44:28you know, our latest iteration on
44:30frontier math uh consists of a bunch of
44:32problems that humans have tried and
44:34failed to solve. They're curated to be
44:36of a special interest to mathematicians
44:39where and some of them we've
44:41characterized even on a bit of a scale
44:43from like this would be moderately
44:46interesting or this would be like maybe
44:49what that means is
44:51two teams of mathematicians have tried
44:54this problem. It's at least 10 years
44:55old, but it's not central to any
44:57research program. It's just something
44:59that people would be happy to get an
45:00answer to. All the way up to like a
45:02major breakthrough.
45:04The four tiers here, breakthroughs the
45:06highest. Uh none of the breakthroughs
45:07have been solved yet, but don't get me
45:09wrong. If Navier-Stokes had been in this
45:11problem set, it would surely be a
45:13breakthrough there. Um regardless of the
45:15fact that humans had like done a lot of
45:18work and gotten maybe within spitting
45:20distance of the of the solution before
45:22AI finished it off. Um but yeah, we
45:24really view this as just uh a way of
45:27finding
45:28AI um
45:30a way of finding cases where AI might
45:33have had a creative idea because we can
45:36tell easily whether it solved these
45:38problems. This is maybe the the the big
45:40piece of work on our side was to make it
45:42easy to verify automatically whether an
45:45AI system has solved this problem. It's
45:47not obvious when you think about it that
45:49no human knows the answer. So how do you
45:50tell without a human looking if AI has
45:53has found the answer? But there's a
45:54couple techniques you can use for this
45:56and and we employ different techniques
45:58for this. Um we can go into that, but
46:00it's something of a technical detail.
46:02And so [clears throat] when we when a
46:04new model comes out, Astra came out, we
46:06hit go, and it's like, oh, it solved
46:09three new three new problems from our
46:10list. Okay, let's go look at the
46:11solutions. Let's share them with
46:13mathematicians. Let's see what's going
46:14on here.
46:15And and then we we analyze that. We with
46:18the help of mathematicians analyze that
46:19for like what kind of qualitatively what
46:21kind of solution was this.
46:23>> What does it mean when the models are so
46:25good that we have to shift from, you
46:28know, these benchmarks that we
46:29understand to like this collection these
46:31collections of problems that we just
46:33don't have the answers to?
46:34>> It's definitely a phase transition in
46:35the
46:37the science of AI capabilities
46:39measurement, which which is you know,
46:41what what I work on. But but it doesn't
46:43change the it doesn't change the game
46:46all that much. It just means we have to
46:48look for harder problems. Uh
46:50I think I joke sometimes that even
46:52certain forms of
46:54optimization problems like
46:57allocating scarce resources efficiently,
46:59that's a benchmark of sorts that will
47:01survive the singularity. Like even if AI
47:04is like off going crazy running rampant
47:07in the galaxy, it'll still have to you
47:08know, it'll still have to decide how to
47:11you know, optimize. So optimization is
47:14is always going to be a benchmark even
47:16if we're superhuman. Basically, it used
47:18to just be we knew the answers ahead of
47:19time. Now we don't. So it takes this
47:22extra trick of, okay, how do we tell
47:23when AI has gotten something right? But
47:26this isn't this isn't insurmountable.
47:29Especially like I would say with um
47:31you know, numbers in a scientific
47:32context. It's intuitive that okay, the
47:34best humans were able to do was, you
47:36know, X. But we just say to AI, hey, try
47:39to do Y greater than X. And, you know,
47:42if it does, that's, you know, those
47:44numbers go on and on. So it's it's no
47:46problem. But yeah, you're right to sort
47:48of note the the phase shift and to be
47:51somewhat surprised by it. Yeah.
47:53>> And is there a methodology, a concrete
47:55methodology for assessing and awarding
47:58partial credit. Is that part of the way
48:00you think about uh benchmarks in this uh
48:04you know, post phase shift?
48:06>> Yeah, it's a good question. It depends
48:07on the benchmark. So, sometimes it's
48:09something like, well, you just want to
48:11make this number go up and the higher
48:12the better. So, partial credit is very
48:14uh continuous and very very natural in
48:17those cases. For math problems,
48:19sometimes it's just like, well, what we
48:20care about is is you know, if X does Y
48:24always hold? Like like is this is this
48:26always true? And then it's kind of you
48:29sometimes humans will find and publish
48:31partial results for the AI systems. We
48:33usually just say, you know, all or
48:35nothing, but we have a large enough
48:36collection of such conjectures that, you
48:39know, we hope to have some, you know,
48:40smooth gradient of, well, okay, this one
48:43got two more out of 100 or something.
48:45And so so it uh you know, looks like a
48:48pretty smooth signal. Um that that's
48:50typically how we how we approach these
48:52things uh
48:53yeah, across benchmarks.
48:55>> Yeah, maybe what I was thinking was, you
48:57know, once you're in the regime of very
49:00challenging unsolved problems, is it
49:03possible that AI
49:06systems are doing, you know, surprising
49:08or novel or creative things, but still
49:10not quite getting them to the full uh
49:14you know, the full solution? And do we
49:15care about that? Do we want to to track
49:18that? And is that part of the way you've
49:20constructed the benchmark?
49:21>> It's a great question. Uh it's not
49:23central to how we construct the
49:24benchmark. Like it's probably like it's
49:26certainly worth tracking. The The
49:27problem is it's just a labor intensive.
49:29Uh like you could have AI review the AI.
49:31We We do this occasionally. Say like,
49:34"Which problems did it make partial
49:35progress on? What if we let it think for
49:37longer on those problems?" And nothing
49:40yet. Like it doesn't seem good at
49:41understanding how close it is to a
49:43solution. Humans aren't necessarily very
49:45good at this either.
49:47I did want to mention that while we
49:49focused on maybe the area where AI is
49:53the very best, like has come the
49:55farthest, the fastest, math. We do see
49:58lots of areas where we also think it's
50:01important to measure AI capabilities,
50:03where it's um
50:06where the progress is somewhat fuzzier,
50:08less clear, and certainly less
50:10superhuman, or even in even in some,
50:13well, may
50:14may be still superhuman in some ways,
50:16but the ways in which it's not yet are
50:19maybe more obvious and feel less
50:21rarefied. Uh
50:22>> What are some examples?
50:23>> A couple couple examples.
50:26One area that
50:27forthcoming work we're excited about is
50:29just
50:30it's a question everyone must be
50:32wondering, can AI take my job?
50:34So, our
50:35our job at Epoch often involves many
50:38things and
50:40all sorts of research, not just AI
50:42benchmarking, but other topics about
50:44trends in AI and uh writing research
50:47reports and doing data analysis, making
50:49info graphics that are, you know, make a
50:52point in an easy-to-digest way.
50:54We've been collecting examples of these
50:57from within Epoch, where the judgment at
51:02the end of did the AI system manage to
51:04do this, you know, Epoch task, um
51:08it is too messy for traditional
51:10benchmarking.
51:11So, it wouldn't be obvious like how to
51:14say did it get the right answer or not.
51:16It's not about making a number go up.
51:18It's not about satisfying some logical
51:20chain of deductions. It's like, here's
51:22the infographic to go along with this
51:24report. Like, does it does it meet our
51:26style guide and does it look good and
51:29does it, you know, is it better or worse
51:31than the
51:32the human reference that that we, you
51:35know, one of my colleagues did.
51:37Um
51:38And here we see like an interesting mix
51:41of
51:41some things I think we we sort of all
51:44our experience is like, yeah, if you ask
51:45it for a summary of the literature, it's
51:47going to do a pretty good job. It's like
51:49very good at that." But but there's
51:50other tasks where it's um, you know,
51:52hard like creating compelling visuals
51:55that follow our style guide. Or, this is
51:57a good one, coming up with a new um,
52:01like
52:02uh,
52:03short-form data-driven insight. Uh, we
52:06call these data insights. You can find
52:08them on our website. And uh, these like
52:11often we try to boil down a single like
52:14important observation into a single
52:16chart and a single sentence. Uh, and
52:19this is like this is hard. Uh, but we
52:21don't see them doing a great job at
52:22this.
52:23Come up with a new project for Epoch to
52:26undertake and do a prototype. Very
52:28open-ended. Not so good at this. Like
52:31like these more open-ended tasks, we we
52:33do see Again, my guess is if we we'll do
52:36this, go back in time some and see how
52:38models gotten better at this. My guess
52:40is we'll see progress. But the level
52:42compared to like humans is still a
52:44little
52:46still a little meh.
52:48I want to give one other example, uh,
52:50um, which is learning on the fly. This
52:53is a case where This is another case
52:55relevant for Ken AI take my job, cuz you
52:57didn't start you you weren't great at
52:59your job on day one, but as you
53:02practice, you know, you got better and
53:04humans, you know, have this sort of it
53:06you know, they learn on the fly. Uh, we
53:08take a
53:10we like to measure this for AI. We have
53:11a a benchmark that's taking a
53:13complicated board game. We did a human
53:15study for for saying, "How many times do
53:17humans have to play this board game in a
53:19row before they sort of figure out how
53:21it works and can get a very high score
53:23on it?" And we do the same with AI. We
53:25say, "Play it once, take notes." Same AI
53:28instance, like play it again, take
53:30notes, play it again, take notes, and
53:32see how you can do. AI systems have
53:33gotten good at this board game, but not
53:36by playing it repeatedly. It's more of a
53:38intergenerational thing. Like GPT 5.6
53:42scores X out of the box and then stays
53:44flat on its play throughs. GPT 6 scores
53:47much higher. It's like much better at
53:48it, but it's still out of the box and
53:50then it stays flat in its play throughs
53:52and it's still less than the top human.
53:54So, that this kind of on the fly
53:56learning is something we also don't see
53:57AI systems as being good enough at even
54:00to just pick up a board game, let alone
54:02a messier
54:04job, you know, real world job like that.
54:06So, these are sorts of things where we
54:08don't see these capabilities and we
54:09think it's important to try to catch
54:11when they emerge.
54:12>> My gut on the the ladder, the AI
54:16learning on the fly is that it's maybe a
54:19harness solvable challenge versus a
54:22model solvable challenge and maybe you
54:25know, no one's focused on that.
54:27Like
54:28with the right set of memory structures
54:30and like some kind of side car
54:33heuristic-y thing, like
54:35you know, we can probably make a big
54:37step improvement in
54:40an AI's ability to like
54:43you know, improve game to game, but it's
54:45not
54:46uh it might not require dramatic model
54:50changes. Do you do you think about that
54:53and is is do you have, you know, in this
54:57domain or in other domains like
55:00formal or structured ways of thinking
55:02about you know, what
55:05uh you know, model versus harness. You
55:07know, granted this whole model versus
55:09harness thing in and of itself is, you
55:11know, practically brand new,
55:13um
55:14but it's proving to be important in, you
55:17know, distinguishing, you know, where
55:19the these improvements come from.
55:21>> Uh yeah, definitely. Uh
55:25it We've done some experiments on this
55:27for the board game case in particular,
55:30where we've tried uh
55:33uh very simple harness first the first
55:35party harnesses like Claude code or code
55:37acts. We've tried multi-agent setups.
55:40We've tried like we we give them you
55:43know, note-taking tools and say you
55:45know, make it clear like okay,
55:47you you're you're you're goal is to
55:48learn the game so you can do really well
55:50like in your final plays of the game. So
55:52explore, take notes, figure out like we
55:54try to prompt them well.
55:56I mean we want to find good results if
55:58we can cuz it's like that's a big deal
56:00for the world.
56:01The game is a test bed but real jobs
56:04would also you know, benefit
56:06significantly. Risks would also go up if
56:08this capability, you know, you test in
56:10the lab is it a great hacker? You find
56:12maybe it is, maybe it isn't but if you
56:14find that it isn't and yet it can learn
56:15on the fly then okay, once you deploy it
56:17like look out. Um
56:19So anyway, we we try we've tried pretty
56:21hard and have mostly not found anything
56:24that cracks this
56:26that cracks this on-the-fly learning.
56:28There's been some general research about
56:29this as well, not just us about
56:32you know, people talk like there was a
56:34people say it uses these so-called
56:36skills where you like just write some
56:38notes to an AI system of look, when you
56:41do work I ask you to do you're going to
56:43need to use this tool or produce this
56:45kind of output and like here's my notes
56:47on how you could do that well so you
56:48don't have to like learn it on the like
56:51pick it up fresh every time. So use
56:53these skills.
56:54And these show some improvement but then
56:56there's like a plateau like pretty
56:57quickly
56:58at least in the the research I'm
57:00familiar with of like you can get some
57:02improvements. My sense is that this is
57:04very loosely speaking
57:06it's like low-hanging fruit. Like if
57:08you're telling it oh you're going to
57:09have to navigate this like finicky
57:11website that my company makes me use and
57:13like just you're going to get stuck and
57:15like you have to look in this menu and
57:16that's where the button you're looking
57:17for like straightforward advice that's
57:20like obviously useful. But when it comes
57:22to higher-level strategic
57:24thinking kind of hard to bake that into
57:26the harness
57:28where or or or like where where the the
57:31board game plays like like has this um
57:34you know sort of brings this out where
57:36it's like there's not any perfect advice
57:40that like you have to find the advice
57:42for yourself in a sense. Like you have
57:44to learn the game. Like so I will say if
57:46we give it extremely detailed strategy
57:48guides for here's how you beat this
57:50game. Like here's everything you could
57:52possibly like it's a cheat by human like
57:54no one would play the game that way.
57:56They do very They do very well. They do
57:59much better than if they basically have
58:01to find that for themselves.
58:02>> right creating the strategy guide.
58:04>> They can't create the strategy guide for
58:06themselves which is like pretty
58:07interesting. So so like we do expect
58:10eventually they'll get better at this
58:12but it's again it's a big deal if that
58:14if that happens because now it means
58:16okay
58:18go back to that first benchmark suite.
58:20Okay, they're not great at making charts
58:23for us or coming up with data insights
58:25for us you know without guidance. But
58:29now put them in on the team for a little
58:31while and let them learn on the fly and
58:34maybe they do become good. So you sort
58:36of see these as a combination like if on
58:38our board game where it's easy to
58:40measure they're not getting better over
58:41time then it's okay for us over in the
58:43real world like human in the loop
58:45evaluation
58:47studies for us to just do one try and
58:49say okay well they weren't great and we
58:51don't think they learn on the so that
58:52you know but but really you know those
58:54go together.
58:55>> And is the board game a synthetic one
58:58that you created for this challenge or
59:01is it a a real board game that that
59:03people play?
59:04>> It's a real board game. It's called
59:06the first one we did this with we'll do
59:07this with more
59:08is called Earthborn Rangers.
59:10It's fairly obscure. If you go to like
59:12boardgamegeek
59:14.com it's like 500th most popular or
59:17something. So it's not like up there.
59:19There's a small devoted community to it
59:22and
59:23AI systems don't seem to know that much
59:25about it. Like they've heard of it. They
59:26can give you some facts about it, uh
59:28but it doesn't seem like something
59:30they've been trained on specifically.
59:32It's impossible for us to tell this with
59:34with high certainty, with high you know,
59:35high confidence, but but that's sort of
59:37why we chose it. Um
59:39it's just hard to make these things from
59:40scratch and know whether they're hard or
59:42whether they're broken. Uh you know, so
59:44that's why we But but the risk which you
59:46might have in mind is what if AI systems
59:48like you know, just know about it from
59:50their training data. That's a risk on
59:52our mind. It's just a risk. One thing we
59:54will do in the future is we're moving on
59:56to video games. Incidentally, it's
59:57another area where AI is like very much
1:00:00not like human reaction time and visual
1:00:02and spatial reasoning. Again, coming
1:00:04along fast, but not uh not really there
1:00:07yet. Um
1:00:09that So, video games, you can always
1:00:11test on a new game, like a a game that
1:00:12just came out, uh not in the training
1:00:14data probably.
1:00:16Um or at least, you know, not as
1:00:17robustly in the training data.
1:00:20Uh and so we'll be testing can AI
1:00:21systems do well on those? Do they do
1:00:23better on like, you know, the older
1:00:26version of a video game than like, you
1:00:28know, Grand Theft Auto 4 vs. 5 or
1:00:30whatever. Uh like when GTA 6, whatever
1:00:33it is, comes out, like I I don't think
1:00:35we'll we'll be doing this one, but but
1:00:37you know, are they worse at that?
1:00:38>> It's interesting when you when you bring
1:00:41up video games, it immediately calls to
1:00:43mind kind of the classical reinforcement
1:00:46learning approaches, which is kind of
1:00:49learning on the fly, but kind of
1:00:51different.
1:00:51>> Yeah, we we call the second sometimes in
1:00:53context learning, where the only like
1:00:55lever the AI system really has is
1:00:57managing its context. And like look,
1:01:00when we give it the strategy guides and
1:01:01they do much better, clearly this is a
1:01:03powerful lever, but it's not as powerful
1:01:06as updating the weights. And like So,
1:01:08yeah, I I I mean like what's the big
1:01:10picture here? It's it's if you have this
1:01:12fast loop of in context on the fly
1:01:14learning, uh then AI capabilities might
1:01:18sort of improve much more rapidly than
1:01:19the slower loop of uh collecting more
1:01:23training data, doing more reinforcement
1:01:24learning, which which is done on
1:01:27language models as well these days.
1:01:29Um
1:01:30and then deploying a new model. Now, AI
1:01:32might automate that whole loop, but
1:01:33it'll still be slower than if it can
1:01:35just, you know, work out its
1:01:37its own context on the fly, yeah.
1:01:40>> Yeah, it's interesting when you zoom out
1:01:42on some of the the questions that you
1:01:45focus on, and we haven't talked
1:01:47explicitly about it, but you have a blog
1:01:49post where you kind of
1:01:52uh write out these nine big hairy
1:01:55questions that you're tracking.
1:01:57They kind of group into like, how do I
1:02:00assess where AI is now? That's like, can
1:02:02it do my job? Like, how is it, you know,
1:02:05uh competitively?
1:02:07Uh and you know, where is the puck
1:02:10going? Like, can it learn on the fly?
1:02:12You know, can it do research? Can it
1:02:14come up with new ideas? These are all
1:02:16It's kind of where it is now, but also
1:02:18like, what what you know, is it
1:02:21positioned to like dramatically
1:02:23um
1:02:25you know, self-improve or like, you
1:02:27know, what's the trajectory?
1:02:29>> That's exact That's exactly right.
1:02:31The um
1:02:33I mean, I think
1:02:34for for people thinking more broadly
1:02:36about what they need to know about AI,
1:02:39that this is the second topic is
1:02:43uh
1:02:43you know, just as important as the
1:02:45first. Uh but
1:02:47it's really
1:02:49you know, even if you take some of the
1:02:50recent incidents like the the hugging
1:02:52face incident, like
1:02:54the sort of think of two axes.
1:02:56Alignment of is the AI doing what I want
1:02:59it to do holistically? Obviously,
1:03:01hacking into some other company's
1:03:03computers is not well aligned. Uh and
1:03:05then capabilities, just like, what can
1:03:07it get done? Uh can it hack into uh you
1:03:10know, reasonably well-secured other
1:03:12companies'
1:03:13uh computers?
1:03:15Um
1:03:16what that incident was was like an early
1:03:18example, very like, I think pretty
1:03:20robust example of well, we don't have
1:03:22the alignment thing solved and capab-
1:03:25and capabilities are getting so big, so
1:03:28strong that this is becoming a problem.
1:03:31Uh, and so paying attention to
1:03:32capabilities like what we should expect
1:03:35is becoming more important sort of
1:03:37day-to-day and we're just trying to
1:03:38produce benchmarks that both say, where
1:03:42are we now? Exactly like you said, like
1:03:44what can it do now? Uh,
1:03:45this is maybe relevant for more, you
1:03:49know, mundane economic utility or
1:03:51whatever, as well as, uh,
1:03:53you know, do we see the the dynamics of
1:03:56the system changing in a way that might
1:03:58be alarming, basically.
1:04:00>> You're kind of also thinking about it as
1:04:01like first and second derivative of
1:04:03capability.
1:04:04>> Yeah, I think that's right. Uh, some of
1:04:06these things make it, uh,
1:04:08the slope goes, you know, much, uh, you
1:04:11know,
1:04:12faster, uh, hyp- hyperbolic growth or
1:04:14whatever in some of these cases versus
1:04:17merely just like, yep, it's, uh, it's
1:04:19growing and it continues to grow. Both
1:04:21are important. I mean, I think we've
1:04:22seen
1:04:23the disruption we've seen to date has
1:04:26come from these kind of unpredictable
1:04:29thresholds. Like when, you know, GPT-2,
1:04:32GPT-3, suddenly you had the ChatGPT
1:04:35moment and you're like, oh, this thing,
1:04:37I want to talk to this thing. I might
1:04:38use this thing instead of a search
1:04:39engine. This thing is useful to me for
1:04:40finding information. And then sud- and
1:04:42then like, you know, a little while,
1:04:44years later,
1:04:45whatever, 2 years later, you had the
1:04:47it's getting better slowly and steadily
1:04:49at helping me with coding or with like
1:04:51using my computer, but suddenly you have
1:04:53>> models.
1:04:54>> With the thinking models and then you
1:04:55had the Claude code moment where it was
1:04:58like, oh, suddenly this thing can
1:04:59actually do software projects for me
1:05:01even if I'm not a very not an engineer
1:05:03myself. Uh, and it's like hard to
1:05:05predict when these thresholds will be
1:05:07crossed. So, it's useful to still track
1:05:09but like the it's useful to track the
1:05:11mundane capabilities today and say,
1:05:13look, you know, cyber is maybe the
1:05:14latest example where it's like, look, we
1:05:16see their ability in controlled settings
1:05:18to do hacking going up
1:05:21smoothly and at some point it's going to
1:05:23cross some threshold where, oh shoot, it
1:05:25can hack hugging face or whatever. Hard
1:05:27to predict where those thresholds are,
1:05:29but still very useful to track those
1:05:32capabilities. So so even the first
1:05:34derivative as as you said version of
1:05:36this, we find very useful. We've got
1:05:40So a fun benchmark we hope hope to
1:05:42release maybe by the time this is
1:05:44published. We'll we'll we'll have it out
1:05:45on a furniture assembly where we you we
1:05:49build some IKEA furniture, make some
1:05:50mistakes and see if the AI system can
1:05:52like just given photos like say, oh like
1:05:55wait, you made a mistake in step back in
1:05:56step four like stop, go back.
1:05:59Um
1:06:00And they've you know, we see the same
1:06:01smooth capabilities like Astra GPT-6
1:06:04Astra pretty good at this actually.
1:06:06Uh and you know, but it wasn't out of
1:06:08nowhere. It was like from a smooth and
1:06:10this is you know, important because we
1:06:12haven't seen AI have much of an impact
1:06:14in physical industry yet. Like it's been
1:06:16mostly digital.
1:06:18Uh and even before robotics hit the
1:06:19scene hits the scene, we we might expect
1:06:22that you know, Claude in your glasses or
1:06:24something is saying like, hey, you know,
1:06:26I'm going to guide you through repairing
1:06:28your car or whatever. I'm going to help
1:06:29you fix this machine in the factory that
1:06:31broke. So so you know, we want to track
1:06:33those capabilities too just for the
1:06:34mundane impact. Anyway, impact all over
1:06:37the place coming. Benchmarks help us
1:06:39help us say so.
1:06:41>> Awesome. Awesome. Well, Greg, thanks so
1:06:42much for jumping on and sharing a bit
1:06:45about what you're working on. It's super
1:06:47interesting stuff and [music]
1:06:49very thought-provoking way of
1:06:51thinking about you know, AI and where it
1:06:53is, where it's going.
1:06:55>> Thanks, Sam. Really appreciate it.
1:06:57>> Awesome. Thank you.
1:07:18>> Mhm.

About this transcript

This page contains the full transcript of AI Progress Explained: From Math Olympiads to Navier-Stokes by The TWIML AI Podcast with Sam Charrington, generated from the public captions YouTube serves with the video. The transcript has 11,860 words across 1,835 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.