AI Just Reached A Tipping Point — Transcript
Full transcript
- 0:00Something strange just happened in the
- 0:01AI world. Over the last two weeks, some
- 0:04of the most careful, most skeptical
- 0:06people in AI all changed their minds at
- 0:08the same time. Duesh Patel, who spent
- 0:10the last year arguing that AGI is not
- 0:13right around the corner, now says we
- 0:15might be only a couple of months away
- 0:16from AI that's as good as the best
- 0:18humans at AI research. The team behind
- 0:21AI 2027 just published an update with a
- 0:23title that says it all. Looks like the
- 0:26intelligence explosion will happen in
- 0:272027 after all. And Anthropic quietly
- 0:31told the US government in its own IPO
- 0:33filing that it has now crossed its own
- 0:35safety threshold for AI, speeding up AI
- 0:38research. One researcher at Meta put it
- 0:40bluntly. I was seriously wrong. So, in
- 0:43this video, I'm going to break down what
- 0:44Anthropic just admitted, the numbers
- 0:46that changed everyone's minds, and why
- 0:48so many people now think AI has reached
- 0:51a tipping point.
- 0:56Which leads me to point number one,
- 0:58which is that Anthropic just crossed its
- 1:00own line. Anthropic has something called
- 1:02a responsible scaling policy. It's
- 1:04basically a list of trip wires, lines
- 1:06that, if their AI ever crosses them,
- 1:08mean the company has to put stronger
- 1:10safety measures in place. And one of
- 1:12those trip wires is about AI research
- 1:14itself. It triggers when AI is speeding
- 1:17up Anthropic's own progress by around
- 1:19two times. [music] And when that speed
- 1:20up is substantially because of
- 1:22automation. In other words, the moment
- 1:24where AI starts meaningfully building
- 1:26the next AI. Now, for most of AI's
- 1:29history, that line felt like science
- 1:31fiction. But on the 10th of November,
- 1:32Anthropic amended its IPO filing with
- 1:35the SEC, and Eli Lifeland, one of the
- 1:37authors of AI 2027, [music]
- 1:39spotted something in it. Anthropic now
- 1:41says it has met that threshold. Here's
- 1:44the key part of the filing. The company
- 1:46says its use of AI systems has already
- 1:48been accelerating its research and
- 1:49development and that its internal
- 1:51estimates indicate a further increase in
- 1:53the rate of progress. [music] And
- 1:55because the threshold applies where it
- 1:56observes or expects a certain level of
- 1:58acceleration, Anthropic says it made a
- 2:00determination that it has met this
- 2:02threshold based on what it expects from
- 2:04future models and its overall rate of
- 2:06progress. Now, that wording matters.
- 2:09Anthropic is not saying it has watched a
- 2:11current model cross the line. It's
- 2:13saying it expects its future models to,
- 2:15so it's treating the line as crossed
- 2:17[music] now. But the filing also says
- 2:19something that should make everyone pay
- 2:21attention. It [music] says Anthropic's
- 2:23evidence about the pace of AIdriven
- 2:25acceleration comes mostly from lagging
- 2:27indicators and that it may not identify
- 2:29a significant acceleration in its rate
- 2:31of development until after it has
- 2:33already happened. Read that again. The
- 2:36company building some of the most
- 2:38powerful AI in the world is telling
- 2:40investors that it might only notice the
- 2:42speed up after it's already here and
- 2:44people noticed. The account AI not kill
- 2:47everyone is in memes posted it in the
- 2:49most dramatic way possible. Anthropic
- 2:51have crossed their threshold for RSI.
- 2:54Recursive self-improvement is here. And
- 2:56they added that RSI was long dismissed
- 2:58by very serious people as impossible and
- 3:01that they were wrong. So what does AI
- 3:04building AI actually look like inside
- 3:06Anthropic? Anthropic now publishes
- 3:08something called the R&D automation
- 3:10index. It rates every model development
- 3:13task on a scale from no AI at all to AI
- 3:16assists to AI collaborates to AI leads
- 3:19all the way up to fully automated. And
- 3:22as of the latest version, Claude now
- 3:24leads 26% of Anthropic's model R&D work.
- 3:27Back in February, that number was under
- 3:301%. That's from basically nothing to a
- 3:33quarter of the work in about 6 months.
- 3:35And that lines up with what Anthropic
- 3:37has already said publicly. In a report
- 3:39called When AI builds itself, written by
- 3:41Jack Clark and Marina Favaro, Anthropic
- 3:43said that more than 80% of the code
- 3:45merged into its codebase was written by
- 3:47Claude. And in one AI safety research
- 3:50problem, two human researchers working
- 3:52for about a week recovered around 23% of
- 3:55the performance gap. Claude agents
- 3:57working on the same problem recovered
- 3:5997%.
- 4:00Anthropic also tested how often Claude
- 4:03picks a better next research step than
- 4:04the human researcher would. Claude Opus
- 4:074.5 won 51% of the time. Claude Mythos
- 4:10preview won 64% of the time. So this
- 4:13isn't just AI writing code faster. It's
- 4:16starting to show research judgment. And
- 4:18it gets more interesting. Just 2 days
- 4:21after that filing, Anthropics founders
- 4:23wrote a letter ahead of the IPO. In it,
- 4:25they said that building AI safely may
- 4:27require Anthropic and ideally the rest
- 4:29of the industry to modulate the
- 4:31extremely fast pace [music] at which AI
- 4:33capabilities are compounding, especially
- 4:35when this is driven by AI systems
- 4:37building [music] themselves. And they
- 4:39added that there may be times when they
- 4:41choose to advance more slowly than they
- 4:42could, even if that puts them at a
- 4:44competitive disadvantage. One person on
- 4:46X summed it up perfectly. Seems like
- 4:49they're preparing for RSI. Think about
- 4:51how unusual that is. a company that is
- 4:53about to go public telling future
- 4:55shareholders that it might deliberately
- 4:57slow down and let competitors catch up.
- 5:00You don't write that in an IPO letter
- 5:01unless you genuinely think the speed is
- 5:03becoming a problem. And if you want the
- 5:05biggest AI news delivered straight to
- 5:07your inbox, check out the AI Grid
- 5:09newsletter. The link is in the
- 5:11description. Which leads me to point
- 5:12number two, which is the numbers that
- 5:15changed everyone's minds. The trigger
- 5:17for all of this was a report from Meta,
- 5:19the independent group that tests what
- 5:21Frontier AI models can actually do. Meta
- 5:24spent months working with internal
- 5:25anthropic data, and it found that since
- 5:27October last year, Anthropic's internal
- 5:30capability scores have been growing at
- 5:31around 1.9 times the earlier rate. Meta
- 5:35says the real number could be anywhere
- 5:36from about 1.5 to 2.8 times, and it
- 5:40guessed that perhaps around 80% of that
- 5:42acceleration is because of AI itself.
- 5:44Now, two times faster might not sound
- 5:46like the end of the world, but here's
- 5:48why researchers freaked out. Ryan
- 5:50Greenblat, a well-known AI safety
- 5:52researcher who also works at MET, wrote
- 5:55that this report is a big update towards
- 5:57super intelligence within 1 to2 years.
- 5:59And then he wrote something you almost
- 6:01never see from someone in his position.
- 6:03I was seriously wrong about RSI speed
- 6:06up. He said he had previously expected
- 6:08only around a 1.15 to 1.3 time speed up
- 6:12in 2026. Instead, it looks like around
- 6:15two times, and it gets bigger. Greenblat
- 6:17points out that not all AI progress
- 6:19comes from better software. Some of it
- 6:21just comes from using more compute. So,
- 6:24if overall progress is going two times
- 6:26faster, the software side has to be
- 6:28going even faster than that. When he
- 6:30plugged the numbers into his own public
- 6:31model, he got a very rough estimate that
- 6:33AI is now speeding up AI software
- 6:36progress by around 4.5 times. And here's
- 6:39the kicker. In the AI 2027 scenario,
- 6:42that level of acceleration doesn't
- 6:44happen until around March 2027. So, by
- 6:47this estimate, reality is running about
- 6:495 months ahead of the most famous AI
- 6:51takeoff scenario ever written. Now, to
- 6:54be fair, Greenblat also gave a big
- 6:56caveat. Much of the underlying evidence
- 6:58in Meter's report was redacted, so a lot
- 7:01of the conclusions depend on trusting
- 7:02the authors. He says he does trust them
- 7:05a lot, but he also says less redaction
- 7:08would greatly reduce the need for trust.
- 7:10So, we're seeing the conclusion, but not
- 7:12all of the proof. And it's not just
- 7:14anthropic where this is showing up. Open
- 7:16AAI published a post in September about
- 7:18how its own researchers use AI. The
- 7:21heaviest 10% of AI users among OpenAI's
- 7:23researchers went from spending about
- 7:25$1,000 a day on internal coding agents
- 7:28at the beginning of the summer to over
- 7:29$7,000 a day by the end of August.
- 7:32That's a seven times jump in about 3
- 7:34months. When your researchers are each
- 7:36spending $7,000 a day on AI agents, it's
- 7:39because those agents are doing a huge
- 7:41amount of the work. But the speed up
- 7:43isn't the only number that's moving.
- 7:45Remember Meta's famous chart, the one
- 7:47that measures how long a task an AI
- 7:49agent can complete on its own. From 2019
- 7:52to 2025, that number doubled roughly
- 7:54every 7 months. From 2024 to early 2026,
- 7:58it was doubling every 3.3 months. And
- 8:01according to a new analysis using the
- 8:03estimates from meter's report, since
- 8:04Claude Opus 4.5, it's been doubling
- 8:07roughly every 1.7 months. The last model
- 8:10meter actually measured was an early
- 8:11version of Claude Mythos preview at
- 8:13around 3.1 hours. But Anthropic's secret
- 8:16internal model, which meter calls model
- 8:184, is estimated at around 38 hours.
- 8:22That's almost a full working week of
- 8:24human effort done by an AI on its own.
- 8:26And here's the craziest part about that
- 8:28chart. When someone asked Benjamin Todd
- 8:30for the official link, he said, "Meter
- 8:32isn't updating the official graph
- 8:34anymore because they can no longer
- 8:35measure the time horizons. The models
- 8:37have outgrown the test." Then Andrew
- 8:40Curran took Anthropic's own automation
- 8:42index and fitted a trend line to it. If
- 8:44the current curve keeps going, he
- 8:46estimates that 95% of Anthropic's model
- 8:49R&D would be fully automated by November
- 8:512027. That's what Anthropic describes as
- 8:54level five, fully autonomous RSI. Now, I
- 8:58want to be clear about this one. It's an
- 8:59extrapolation and no task has actually
- 9:02been rated fully automated yet. So, that
- 9:04part of the curve is based on
- 9:05assumptions. But Curran makes a point
- 9:07that I think is really important. This
- 9:09will eventually happen at every lab, not
- 9:12just anthropic. Which leads me to point
- 9:14number three, which is the great update.
- 9:17What makes this moment feel different
- 9:18isn't just the numbers, it's who is
- 9:21changing their mind. Duesh Patel runs
- 9:23one of the most respected podcasts in
- 9:25AI. He's interviewed basically every
- 9:27major AI leader. And last year, he wrote
- 9:29a famous essay arguing that AGI is not
- 9:32right around the corner because AI
- 9:34models can't learn on the job like
- 9:35humans can. On the 13th of November, he
- 9:38released a new video essay with a very
- 9:40different message. The avalanche that
- 9:42hit math is coming for AI development.
- 9:45And this is how it opens. If you had
- 9:47asked me earlier this year how far away
- 9:49we are from super intelligence, I'd
- 9:50probably have said at least the better
- 9:52part of a decade. But [music] over the
- 9:53last three months, I've updated my views
- 9:55a ton. I've been debating people both on
- 9:58and off the podcast about this question.
- 10:00And of course, I've been watching the
- 10:01progress of AI on verifiable domains and
- 10:04thinking about what that implies for AI
- 10:06research itself. And I've come to the
- 10:08view that we're only a couple years away
- 10:10and possibly only a couple months away
- 10:12from AIs that are as [music] good as the
- 10:15best humans at AI research. And I've
- 10:17also come to believe that automating AI
- 10:19research will compress so much progress
- 10:22into a single year that soon thereafter,
- 10:24so by 2030 and plausibly as early as
- 10:27next [music] year, we'll have AIs that
- 10:30are at least as good as the best human
- 10:32experts in every field. I know this is a
- 10:34wild claim and it's one that I myself
- 10:36didn't believe even a couple months ago.
- 10:38That post got over 1.5 million views and
- 10:41Dwarkkesh isn't alone. On the 10th of
- 10:43November, the AI Futures Project, the
- 10:46team behind AI 2027, published their
- 10:48latest timelines update. They say, "We
- 10:51are now about 5 months ahead of the AI
- 10:532027 scenario." Previously, Daniel
- 10:56Kokilo, the most bullish of the group,
- 10:58gave a 50% chance of a superhuman AI
- 11:01researcher before 2029. Now, all of them
- 11:04forecast that a superhuman AI researcher
- 11:06will likely arrive at some point in
- 11:082027, assuming there's no deliberate
- 11:11slowdown. They describe that as roughly
- 11:13the point where a company would rather
- 11:14fire all of its human researchers than
- 11:16lose its AI researchers. And they give a
- 11:1935 to 65% chance of AI that can automate
- 11:22all white collar work arriving before
- 11:242028. And they say the single biggest
- 11:27piece of evidence behind the update was
- 11:29the meter report. And then there's
- 11:31Anthropic's own co-founder Jack Clark.
- 11:33In his newsletter, Import AI. He wrote
- 11:36that he now thinks it's more likely
- 11:37we'll see full RSI become possible in
- 11:402027, whereas before he thought it was
- 11:42more likely in 2028. He said the big
- 11:45thing he was waiting for was creativity
- 11:47and that since then we've seen a
- 11:49tremendous advancement in AI making
- 11:51creative seeming math breakthroughs and
- 11:53signs that AI is getting increasingly
- 11:55good at AI research too. Jeremy Harris
- 11:58put it perfectly. Many of the smartest
- 12:00people who thought RSI wasn't near are
- 12:02now changing their minds. [music] And if
- 12:04you think very powerful AI is still a
- 12:06ways off because someone you respect
- 12:07told you so, it might be worth checking
- 12:09in with them again. They may have
- 12:11changed their mind. And this is why
- 12:13people keep bringing up AI 2027. In that
- 12:16scenario, June 2027 is the month of
- 12:18self-improving AI. The fictional lab has
- 12:21a country of geniuses in a data center.
- 12:23Most of the human researchers can't
- 12:25usefully contribute anymore. They go to
- 12:27bed every night and wake up to another
- 12:29week's worth of progress made mostly by
- 12:31the AIS. When that scenario came out, a
- 12:33lot of people called it science fiction.
- 12:35Now, its authors think the real world is
- 12:37running ahead of it. Which leads me to
- 12:40point number four, which is why AI
- 12:41research could go the way of math. This
- 12:44is the heart of Dwarkh's argument, and I
- 12:46think it's the most important idea in
- 12:48this whole video. Look at what just
- 12:50happened to mathematics. It's crazy how
- 12:52fast this progress has happened. As
- 12:54recently as 2023, these models couldn't
- 12:56reliably add four-digit numbers. In
- 12:582024, they could barely solve any
- 13:01problems in a high school math
- 13:02competition. In 2025, they could win a
- 13:05gold medal in the International Math
- 13:06Olympiad. [music] To earlier this year,
- 13:08where they finally started to solve some
- 13:10niche open problems to a month ago where
- 13:13OpenAI solved a millennium prize problem
- 13:16in 88 hours. [music] I'm reminded of a
- 13:18line from the novel East. Quote, "And
- 13:22there is one sure thing about the fall
- 13:24of gods. They do not fall a little. they
- 13:26crash and shatter." End quote. That's
- 13:29the idea. OpenAI's model reportedly
- 13:31solved a Millennium Prize problem, one
- 13:33of the hardest open problems in
- 13:35mathematics, in 88 hours. And it wasn't
- 13:37trained by solving other Millennium
- 13:39Prize problems. It was trained on lots
- 13:41of smaller, more tractable math
- 13:42problems, and it generalized all the way
- 13:44up. So, why does that matter for AI
- 13:47research? Because the key to all of this
- 13:49is something called verifiability. When
- 13:52an AI can check whether its answer is
- 13:54right, you can give it thousands of
- 13:55rounds of practice and it gets better
- 13:57and better. That's exactly what happened
- 13:59with competitive programming and with
- 14:01math. They went from worse than most
- 14:03humans to better than the best humans in
- 14:05just a few years. And Dwarkh's argument
- 14:08is that AI research is surprisingly
- 14:10verifiable, too. Did your new training
- 14:12trick lower the loss on a test set? Did
- 14:15the model get faster? Did it use less
- 14:17memory? Those are all things you can
- 14:19measure, which means [music] they're all
- 14:20things AI can practice. And here's a
- 14:23wild example he gives. The Transformer,
- 14:25the architecture behind every modern
- 14:27chatbot, wasn't really one giant leap of
- 14:29genius. Most of the pieces were already
- 14:32known. What it took was combining half a
- 14:34dozen ideas in just the right way and
- 14:36running tons and tons of small
- 14:38experiments. And that's exactly the kind
- 14:40of persistent combine everything, test
- 14:43everything work that AI is already
- 14:45extremely good at. Dwaresh even says the
- 14:47transformer is the kind of thing that a
- 14:49future Claude could have invented if a
- 14:51researcher asked it to try really hard
- 14:53to get attention to work better. Then
- 14:56there's the speed. Dwarsh says today's
- 14:58AIS already work at about 10 times human
- 15:01speed. And an automated researcher
- 15:03doesn't sleep. It works nights and
- 15:05weekends. So each one could cram roughly
- 15:0850 times more work into every calendar
- 15:10week than a human researcher. On top of
- 15:12that, every one of them could be as good
- 15:14as the very best researchers at the top
- 15:16labs and there could be thousands of
- 15:18them working in parallel. And a lot of
- 15:20people have argued that compute will
- 15:21stop this, that even with brilliant AI
- 15:24researchers, you can only run so many
- 15:26experiments. But there's a thought
- 15:28experiment that changed a lot of minds
- 15:29on this. Dan Robinson said it hit him
- 15:32like a sledgehammer. Imagine a slow
- 15:34version of open AI. Let's call it sloth
- 15:37AI. It has exactly the same compute as
- 15:39the real open AI, but its researchers
- 15:41only work 1 hour every week. How much
- 15:44slower would it make progress? Probably
- 15:46a lot slower, because with just 1 hour
- 15:48of thinking a week, [music] they can't
- 15:50design and analyze many careful
- 15:52experiments. But here's the twist.
- 15:54Compared to a company full of automated
- 15:56researchers thinking 10 times faster
- 15:58around the clock, the real open AI is
- 16:01sloth AI. Human thinking speed is a
- 16:04biological accident. It would be a
- 16:06strange coincidence if it happened to be
- 16:07the exact point where thinking faster
- 16:09stops helping. And what about data?
- 16:11Isn't AI going to run out of training
- 16:13data? Dwarvesh's answer is that during
- 16:16RSI, the AIs will be the ones making the
- 16:18data. And there's already evidence of
- 16:20this. Deepseek just published a paper
- 16:22about how it post-trained [music] its
- 16:24newest model and it said that changes to
- 16:26the basic training algorithm haven't
- 16:28mattered nearly as much as improvements
- 16:30in the scale diversity and verifiability
- 16:32of synthesized data and environments.
- 16:35And it said it uses large-scale
- 16:37automated pipelines to build those
- 16:39environments, training the model to
- 16:40construct better and harder tasks for
- 16:42itself. So the AI is already helping to
- 16:45build its own classroom which leads me
- 16:48to point number five which is that every
- 16:50lab is now saying the loop has started.
- 16:53This isn't just an anthropic story.
- 16:54Listen to how Google's CEO Sundar Pichai
- 16:57described the way Google is building its
- 16:59newest Gemini model being
- 17:03you know critical contributor as we are
- 17:05developing Argon and but to be very
- 17:08clear today you know in a continuum of
- 17:11we're pretty far from uh you know the
- 17:14end state of RSI or something like that
- 17:16right so today you have the the right
- 17:19way to describe it is you have RSI loops
- 17:22along with AI researchers in various
- 17:24aspects of model development, right? So,
- 17:27you know, I would say you almost have
- 17:30agents helping AI researchers uh in in
- 17:33some automated way uh work together to
- 17:36build these models. Though Google is
- 17:38openly saying RSI loops are part of how
- 17:40it builds its models and Google
- 17:42co-founder Sergey Brin went even further
- 17:45in
- 17:45>> I'm very excited about the cadence that
- 17:47we have nowadays and uh if you look at
- 17:51even before Gemini 4 when we had um
- 17:54flash 3.6 3.7 3.8 eight. Uh we're
- 17:58starting to get onto that RSI treadmill
- 18:01and now the you know our researchers and
- 18:04the model work hand in hand to make the
- 18:07next version of the model. Actually I
- 18:09just popped in here kind of randomly
- 18:11because Melvin and I were talking about
- 18:13the next version of the model and you
- 18:15know probably by the time folks are
- 18:17viewing this uh internally we're already
- 18:19using a couple models ahead.
- 18:21>> Internally they're already using a
- 18:23couple of models ahead. That means the
- 18:25Gemini you use today is not the best
- 18:27Gemini that exists and the models inside
- 18:29the lab are helping build the ones after
- 18:31them. Here's how Google deep mind
- 18:33researcher Mustafa Dani described it
- 18:36>> like and and and it's somehow like most
- 18:38of the people don't realize that this is
- 18:39like already uh happening especially
- 18:41over the past few months in almost every
- 18:44lab the new generation of the models are
- 18:46built heavily using the previous
- 18:48generation of the models. I think that's
- 18:49basically like like the case again
- 18:52everywhere. Uh and uh it's not fully
- 18:55automatic yet. Uh but the direction is
- 18:58like super clear and it's like easy to
- 19:00imagine that you know like we're going
- 19:02to get to a situation with full
- 19:04automation. These models are going to
- 19:06improve themselves and keep learning
- 19:09from the word
- 19:09>> and at OpenAI the focus is already on
- 19:12what comes next. Here's OpenAI's head of
- 19:15applied research Boris Power. So I would
- 19:17say still 80 90% of the research at
- 19:20OpenAI is focused at GPT7, GPT8 and
- 19:24beyond because we believe that's where
- 19:26fundamentally most of the value comes
- 19:27from.
- 19:28>> And it's not just the big labs. In
- 19:29August, Jeff Dean, one of the most
- 19:31legendary engineers in Google's history,
- 19:34left Google after 27 years to co-found a
- 19:37startup called Discovery Loop alongside
- 19:39Sanjay Gamat, Oriel Vineyles, and Quac
- 19:41Lair. Here's how one of its lead
- 19:43investors, Kla Ventures, described it on
- 19:45TBPN.
- 19:47>> You know, we invested in, we were the uh
- 19:49co-leads of Discovery Loop. This is Jeff
- 19:51Dean and his gang from Google. The
- 19:53legend was 27 years at Google.
- 19:56Everything from, you know, TPUs,
- 19:58TensorFlow,
- 19:59uh, Google Brain. I mean, you you know,
- 20:02really every almost all the significant
- 20:04achievements at Google, these four folks
- 20:06had their fingers in. and they finally
- 20:08leave after 27 years to do an RSA RSI
- 20:11company looking at how can we improve
- 20:14the scientific method. So you now are
- 20:16designing experiments getting results in
- 20:18real time and then continually
- 20:21in parallel designing thousands of
- 20:24follow-on experiments. People like that
- 20:26leave the biggest AI lab on earth to
- 20:28build an automated researcher. It tells
- 20:30you where they think the action is.
- 20:32Which leads me to point number six,
- 20:34which is spark or singularity. So let's
- 20:37say AI does become as good as the best
- 20:39human AI researchers. What happens next?
- 20:42Dwaresh says there are two competing
- 20:44forces. On one side, AI researchers keep
- 20:47getting faster, smarter, and more
- 20:49numerous because each new version helps
- 20:51build the next. On the other side, AI
- 20:54research gets harder because the easy
- 20:56ideas get used up. Companies already
- 20:58have to roughly quadruple their R&D
- 21:00compute and double their workforce every
- 21:02year just to keep the same rate of
- 21:04progress. If AI gets smarter faster than
- 21:07ideas get harder to find, you get what's
- 21:09called a softwareonly singularity.
- 21:12Without even adding new compute, the AI
- 21:14swarm makes smarter and smarter versions
- 21:16of itself until [music] within a matter
- 21:18of months you hit a super intelligence
- 21:20that runs into the physical limits of
- 21:22our current chips. But if ideas get
- 21:24harder to find faster than AI improves,
- 21:26you get what Dwarvesh calls a spark.
- 21:29something like 3 to 6 years of progress
- 21:31at today's rate packed into a single
- 21:33calendar year before things simmer down
- 21:35until more compute is built and he says
- 21:37it's basically a coin toss 50% spark 50%
- 21:41singularity but here's the part that I
- 21:43think people miss think but to be clear
- 21:46even in this spark scenario where we
- 21:47make something like 4 years of progress
- 21:49in the first calendar year after we
- 21:51automate AI research things would feel
- 21:54insane that's a jump as big as GPT 3.5
- 21:57to GPT6 six Astra in just 12 months. To
- 22:01remind you of the scale of progress that
- 22:03this leap would compress, GBT3.5 was
- 22:06able to write a passable limmerick and
- 22:08autocomplete a trivial Python function.
- 22:10But otherwise, it was mostly a useless
- 22:12toy. Whereas Aster can build whole video
- 22:14games from scratch and turns out so many
- 22:17fields medal worthy mathematical
- 22:18contributions that we become totally
- 22:20desensitized. [music] In the last 4
- 22:22years, we've gone from a preschooler to
- 22:25a savant intern. Think about that. A
- 22:27jump as big as GPT3.5 to GPT6 Astra in
- 22:3112 months. And Darkh says a mind like
- 22:34that would probably be vastly superhuman
- 22:37in any domain that's even slightly
- 22:39verifiable. Better than Nvidia at
- 22:41designing the next generation of AI
- 22:43chips. Better than the top trading firms
- 22:45at making money in the markets. Better
- 22:47than the rest of humanity at finding a
- 22:49room temperature superconductor if you
- 22:51gave it access to a materials lab. But
- 22:53what about things that can't be checked
- 22:54inside a data center, like running a
- 22:57company or winning a court case?
- 22:59Dwarvesh thinks the AIS will train on
- 23:01enormous virtual worlds. He describes
- 23:03simulated economies with millions of
- 23:05other AIs where an AI is told to co-ound
- 23:07a successful company and endless back
- 23:10tests of history. For example, can an AI
- 23:13figure out general relativity if it's
- 23:15only given information from before the
- 23:16year 1700? And then there's how fast an
- 23:19AI could learn a real job. Imagine an AI
- 23:22shadowing the CEO of a Fortune 500
- 23:25company. On its first morning, it reads
- 23:27every email, every Slack message, and
- 23:29every document the company has ever
- 23:31produced. By the end of its first day,
- 23:33copies of it have interviewed every
- 23:35employee, and every major client, all in
- 23:37parallel. That's years of context built
- 23:39up in a few days. And even in the slower
- 23:42Spark scenario, he thinks AI gets to top
- 23:44human performance across basically all
- 23:46fields by around 2028 or 2029. And
- 23:50here's the thing, some people think we
- 23:51get there even without RSI. Benjamin
- 23:54Todd wrote a post arguing that even
- 23:56without recursive self-improvement, dumb
- 23:58trend extrapolation gets to AGI by 2030.
- 24:01His logic is simple. AI revenue has
- 24:04tripled every year for more than 4
- 24:06years, and that growth is actually
- 24:08accelerating. According to Epoch AI
- 24:10data, the leading AI companies grew
- 24:12their revenue three times in 2024, 4.7
- 24:15times in 2025, and around six times this
- 24:18year. If revenue keeps growing six times
- 24:21a year, it reaches $10 trillion by
- 24:23around March 2029. That's roughly 10% of
- 24:26the entire world economy. And AI task
- 24:29length has been growing around eight
- 24:31times a year since 2024, which would
- 24:33reach multi-year tasks by early 2030. As
- 24:36he puts it, every exponential becomes an
- 24:39S-curve eventually, [music] but right
- 24:41now, revenue is still accelerating. So,
- 24:43you have two separate paths that both
- 24:45point to roughly the same place. Even if
- 24:48RSI turns out to be slower than people
- 24:50fear, the plain old trend lines are
- 24:52already pointing at something enormous
- 24:54before 2030. Which leads me to point
- 24:56number seven, which is the fight over
- 24:58the brakes. Because if this really is
- 25:01the tipping point, then the most
- 25:02important question isn't how fast AI can
- 25:05go. It's who decides how fast it should
- 25:07go. Even Sam Olman is now framing it
- 25:10this way. Here's what he said when asked
- 25:11about the biggest problem he's thinking
- 25:13about. I think that one that I just gave
- 25:15the the like this question of how do we
- 25:18think about
- 25:20the pace of progress as we enter this
- 25:21RSI era and how do we avoid these two
- 25:25ways that [music] things could go wrong
- 25:27either way from a loss of control
- 25:29incident or a concentration of power
- 25:31incident and what is the right
- 25:33governance mechanism? How like how do we
- 25:35think about how [music] smart an AI
- 25:38system should be allowed to be relative
- 25:40to a person? How quickly can that
- 25:41improve? how smart an AI system should
- 25:44be allowed to be relative to a person,
- 25:46how quickly it can improve. These are
- 25:48the questions the CEO of OpenAI is
- 25:50openly asking. And Microsoft's AI CEO,
- 25:53Mustafa Sullean, has been even more
- 25:55direct about where the real danger is.
- 25:58>> Um, the next phase, which I think
- 26:00carries the most risk, is closed RSI,
- 26:02closed loop, where you basically manage
- 26:04to somehow figure out how to do it
- 26:06without the human reviewer process uh in
- 26:09this in the in the whole procedure. And
- 26:12that um will obviously be faster, but I
- 26:14think is also a very dangerous way to
- 26:16get to full AGI because we're likely for
- 26:19two reasons. It's it's sometimes used to
- 26:21be referred to 20 years ago as like an
- 26:22intelligence explosion because you don't
- 26:25quite know uh how you know how do you
- 26:27control keep control of such a system uh
- 26:30such a process and then the end result
- 26:33will likely be quite alien. It's going
- 26:35to be like an alien intelligence.
- 26:36>> That's the key distinction right now.
- 26:38Humans are still in the loop. They
- 26:40review the work and set the direction.
- 26:42The scary version is when that human
- 26:44reviewer disappears. And notably, in an
- 26:47earlier version of its IPO filing,
- 26:49Anthropic said it is considering updates
- 26:51to its responsible scaling policy to
- 26:53keep humans in control of Frontier model
- 26:55development. So, even the company
- 26:57crossing the threshold is thinking about
- 26:59how to keep a human hand on the wheel.
- 27:01But not everyone in power sees
- 27:03anthropics warnings the same way. The
- 27:05White House's AI adviser, David Sax, had
- 27:07a very different take on all-in. So
- 27:09they're running around saying that this
- 27:11is a winner take all market. What does
- 27:13that sound like? Monopoly. They're not
- 27:14even saying that it's going to be a
- 27:16duopoly with anthropic and open AI.
- 27:18They're saying they're going to win the
- 27:19whole thing. They definitely are saying
- 27:22that recursive self-improvement has
- 27:24kicked in [music] despite the obvious
- 27:26safety problems with that. I mean,
- 27:29recursive self-improvement is the most
- 27:30dangerous thing anybody can be working
- 27:32on in [music]
- 27:34SI. If you're worried about rapid
- 27:36takeoff, for example, if that's [music]
- 27:38your if that's your greatest fear, then
- 27:40I mean, they are racing towards that.
- 27:42But recursive self-improvement would be
- 27:44the key to getting to true next level
- 27:48super intelligence, and they're treating
- 27:50it like a business advantage.
- 27:51>> But from Sax's point of view, Anthropic
- 27:53is warning about RSI while racing
- 27:55towards it and treating it like a
- 27:57business advantage. One person on X
- 27:59summed up the irony of it with three
- 28:01words. David Saxs says, "Ban RSI." And
- 28:04in Congress, some people literally are
- 28:06saying that. After New York Assembly
- 28:08member Alex Bores posted that every AI
- 28:11company is trying to fully automate AI
- 28:13research, Congressman Greg Casar replied
- 28:15with two words: ban RSI. That post got
- 28:19over 800,000 [music]
- 28:20views. And it's also worth hearing from
- 28:22people who think everyone is getting a
- 28:24little carried away. Richard No, an
- 28:26independent AI researcher who used to
- 28:28work at OpenAI, said he thinks people
- 28:30are jumping further than the evidence
- 28:32warrants towards expecting a very fast
- 28:34RSI over the next couple of years. But
- 28:36listen to how he finished that thought.
- 28:38I I I don't really see the world looking
- 28:40normal over the next decade or so. Um
- 28:43and and so the the default sense I I
- 28:45have for what's going to happen is that
- 28:48uh we're not going to get uh like yeah
- 28:53things going vertical, if you will. uh
- 28:55we're not going to get like sudden and
- 28:57extreme um self-improvement that goes
- 29:00far beyond uh that goes very quickly far
- 29:03be into the super intelligent regime but
- 29:05at the same time things are going to go
- 29:07much uh things are going to look much
- 29:09weirder than almost anyone expects
- 29:12honestly I think that's the most
- 29:13realistic way to look at it nobody
- 29:15actually knows if this ends in a
- 29:17software only singularity a spark or
- 29:19something slower the evidence is partly
- 29:21redacted the forecasts have wide error
- 29:24bars Dwaresh himself puts a 25% chance
- 29:27on AI not matching the best human
- 29:29researchers until after 2032. But what
- 29:32has changed is the direction. A year
- 29:34ago, recursive self-improvement was a
- 29:36thought experiment. Today, Anthropic has
- 29:39told the SEC it's crossed its own
- 29:41threshold. Google says RSI loops are
- 29:43part of how it builds Gemini. OpenAI CEO
- 29:46is talking about the RSI era. And the
- 29:49people who spent years saying, "Slow
- 29:50down and look at the evidence," are now
- 29:52saying, "We looked at the evidence and
- 29:54we changed our minds." That's what a
- 29:56tipping point looks like. Not one big
- 29:58announcement, but a lot of careful
- 30:00people quietly updating at the same
- 30:02time. And if they're right, 2027 might
- 30:05be the year that AI stops being a tool
- 30:07we build and starts becoming the thing
- 30:09that builds itself. So, let me know what
- 30:11you think. Are we really at the tipping
- 30:13point, or is everyone getting carried
- 30:14away? Drop your thoughts in the
- 30:16comments, and I'll see you in the next
- 30:17one.
About this transcript
This page contains the full transcript of AI Just Reached A Tipping Point by TheAIGRID, generated from the public captions YouTube serves with the video. The transcript has 5,515 words across 817 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.