YT YouTube2Text

YouTube2Text → Transcripts → ChatGPT Offered Me $2m To Keep Quiet: No One Is Ready For What's Coming!

ChatGPT Offered Me $2m To Keep Quiet: No One Is Ready For What's Coming! — Transcript

by The Diary Of A CEO · 25,468 words · 4,147 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00The scary open secret in the AI industry
0:02right now is that it's possible that
0:03we'll end up essentially creating a new
0:05species that ends up ruling the world
0:07with a 70% chance that this goes
0:08horribly wrong like human extinction.
0:10That's one possibility. There's many
0:11more.
0:12>> It's quite chilling what you're saying.
0:13>> Yeah, it's uh
0:15gets me down sometimes.
0:18I basically told my wife like let's not
0:19have any more kids. It's too uncertain.
0:21I don't think they'll ever join the
0:22workforce.
0:24Everybody should be afraid that their
0:25jobs are going to be lost. And I know
0:26this because I went to OpenAI in 2022.
0:28What I did there was forecasting what
0:30the what the next couple years might
0:31look like. And unfortunately, most of
0:33the world is kind of asleep at the wheel
0:34and doesn't really realize what's going
0:35on with AI. So, I resigned.
0:37>> I read it somewhere that you lost $2
0:39million for not signing an
0:41anti-disparagement clause, meaning you
0:42couldn't criticize the company.
0:44>> Yes, for reasons I'm happy to get into.
0:45But, the main thing I've learned is when
0:47I go talk to people at Anthropic and
0:48OpenAI about forecasting, they're like,
0:50"It's not going to take that long. You
0:51need to shorten them again. Get them
0:52back to 2027 or 2028." Because these
0:54powerful CEOs, Dario or Sam or Elon, are
0:57racing each other to be in control of
0:59the most powerful AIs. And are literally
1:01afraid that if the other guy gets there
1:03first, he might become dictator. I mean,
1:04Anthropic is on track to be the entire
1:07economy by 2030. But, none of these
1:09people should be trusted with that much
1:10power. So, this is the most important
1:12thing happening in our lifetimes,
1:13probably in all of history, in fact. And
1:15it's very important that it go well. So,
1:17I think that there's a lot we can do to
1:18like steer things in a better direction.
1:19There's loads of benefits that we could
1:20get from AI if we do it right. And if we
1:22do solve the problems, then things could
1:24be absolutely amazing for everyone.
1:26>> Well, this report here in 2021, it was
1:28remarkably [music] accurate. And then
1:30just published this one.
1:30>> Yeah. So, this is our new scenarios.
1:32>> So, let's go through these slowly and
1:33one at a time.
1:34>> I would be incredibly happy if all my
1:35predictions turn out to be wrong.
1:40>> This is super interesting to me. My team
1:41gave me this report to show me how many
1:43of you that watch this show subscribe.
1:44And some of you have told us, according
1:46to this, that you are unsubscribed from
1:48the channel randomly. So, favor to ask
1:50all of you, please could you check right
1:51now if you've hit the subscribe button.
1:53If you are regular viewer of this show
1:54and you like what we we here. We're
1:55approaching quite a significant landmark
1:57on this show in terms of the subscriber
1:59number. So, if there was one simple free
2:01thing that you could do to help us, my
2:03team, everyone here, to keep this show
2:05free, to keep it improving year over
2:07year and week over week, it is just to
2:09hit that subscribe button and to
2:10double-check if you've hit it. Only
2:11thing I'll ever ask of you.
2:13Do we have a deal?
2:14If you do it, I'll tell you what I'll
2:15do. I'll make sure
2:17every single week, every single month,
2:18we fight harder and harder and harder
2:19and harder to bring you the guests and
2:21conversations that you want to hear. I
2:22stay true to that promise since the very
2:24beginning of the Diary of a CEO, and I
2:25will not let you down. Please help us.
2:28Really appreciate it. Let's get on with
2:29the show.
2:31>> [music]
2:34>> Daniel Kokotajlo.
2:36At the very heart of what you do,
2:38um what is your mission? And why?
2:41>> So, what would you do if you thought
2:43that superintelligence was coming in a
2:44few years?
2:46>> I guess it depends
2:48what the consequences were.
2:51>> Well, let's talk about it. So,
2:52superintelligence, AIs that are better
2:54than the best humans at everything,
2:56while also being faster and cheaper,
2:57also able to
2:59operate robots that can do everything in
3:00the physical world that humans can do,
3:02but better, faster, and cheaper. If that
3:04really is coming in a few years,
3:07then we need to prepare, and we need to
3:08think about how to make it go well
3:10instead of poorly. So, that's sort of my
3:12answer is like, I'm doing that to the
3:13best of my ability.
3:14>> So, you believe it's coming in a few
3:16years?
3:16>> Yes.
3:17>> How could you be so sure?
3:19>> I spend a lot of time trying to forecast
3:20this sort of thing. My sort of median
3:22estimate, a 50% chance, is currently in
3:252029. Maybe it'll slip to 2028. It's
3:28possible that it'll take significantly
3:29longer, like maybe 10 years or something
3:31like that. But, uh you know, for reasons
3:35I'm happy to get into, seems to me like
3:37it's probably happening by the end of
3:38the decade. Which less important is the
3:41the sense of how close we are. What's
3:43more important is the pace of the
3:45trends.
3:46Anthropic
3:48this time last year was making something
3:50like a billion dollars a year.
3:52And they're making something like 60
3:54billion dollars a year.
3:55So that's
3:5660x growth in 1 year,
3:59which is extremely impressive even for
4:01very small startups, but for a company
4:04of their size, it might be the fastest
4:06growth in history.
4:07Um we expect that rate of growth to slow
4:10down,
4:11but even if it slows down quite a lot,
4:15they're still on track to be,
4:17you know, the entire economy by 2030 or
4:20so.
4:20>> Why should the average person care?
4:22>> The high-level thing is absolutely
4:23everything is going to change for the
4:25whole world, and including therefore for
4:27them and their families. Um could change
4:29for the better, could change for the
4:30worse, depending on the details of how
4:31it's done. So for example,
4:34everyone could die,
4:35you know? Um this is the classic loss of
4:37control scenario, or one version of it.
4:40If we do build these super
4:42intelligences, and we
4:44use them to automate all the jobs, and
4:46we put them in the military, and we, you
4:48know, have them giving advice to
4:49politicians, and so forth, they will
4:51eventually have accumulated enough
4:52real-world power
4:54that they don't need humans anymore. And
4:57they're smarter than us, they're more
4:58strategic, etc. At that point, we sort
5:00of have to hope that they are virtuous,
5:02that they have, you know, the goals that
5:04we wanted them to have, the values that
5:05we wanted them to have, etc.
5:07And the sort of
5:09scary open secret in the AI industry
5:11right now is that right now that is kind
5:12of just a hope. It's not something that
5:14we can
5:15be at all confident in, and in fact,
5:16there's lots of evidence and arguments
5:18that
5:19it we're not on track to achieve that.
5:20So there's lots of reason Like current
5:22AIs, for example, will often lie uh to
5:25people, or they will like you tell them
5:26to do something and they go do something
5:28else, and then pretend that they did it,
5:29right? So
5:31it's an inherently difficult problem to
5:32make something that's super intelligent
5:34and also
5:35has the values and virtues that you want
5:36it to have, and it doesn't seem like
5:38we're on track to solve that problem.
5:40Also, it seems like the sort of problem
5:41that you could think you solved when you
5:43haven't actually solved it, right? Uh
5:45that's a big reason why this is scary.
5:47So, for all those reasons, it's possible
5:49that we'll end up essentially creating a
5:51new species that ends up ruling the
5:53world instead of us. And then maybe we
5:56go the way of other extinct species in
5:57the past that were outcompeted by
5:58humans. That's one possibility. There's
6:01many more. Even if you're not worried
6:03about that and you think that the AIs
6:04will be totally controlled,
6:06there's the question of who controls the
6:07AIs,
6:08right?
6:09When there's a couple corporations that
6:11have made these superintelligences and
6:12are using them to automate all the jobs,
6:15well, that's a lot of power, you know?
6:17That's a lot of money. It's a lot of
6:18political power. They'll have the best
6:20strategists, the best advisers, you
6:22know, they'll think faster. Militarily,
6:25uh, the countries that has these AIs
6:27will be able to absolutely wipe the
6:28floor with all the other countries. The
6:30AIs themselves, it's it's kind of a
6:32single point of failure like central
6:34uh, control system where,
6:36you know, the CEO of Anthropic, Dario,
6:40he coined this phrase, "The country of
6:41geniuses in the giant data center." That
6:43was his
6:44phrase to describe what they're trying
6:45to build, you know?
6:47I think that's a little bit misleading.
6:49I think it would be more accurate to
6:50describe it as army of geniuses in the
6:52data center because
6:53it's not like it's a bunch of diverse
6:54different AIs,
6:56you know, living in their different
6:57parts of the data center. They're all
6:58copies
7:00of the same big model and they're owned
7:02by the company. And so,
7:04they all follow the orders given by the
7:06company, right? People should be asking
7:07questions of like, who controls this
7:09army or these armies and what are they
7:10going to be doing with them?
7:12I think that we could very easily end up
7:13in a sort of
7:15uh, a situation where
7:18some tiny group of people are
7:19essentially oligarchs or dictators. And
7:22ironically,
7:24both of these risks, the loss of control
7:26and the constitution of power,
7:28are things that people in the industry
7:30have been thinking about for decades.
7:32Um, even before the AI industry existed,
7:34you know, people thinking about AI were
7:36talking and writing about these things.
7:38And then part of the founding narrative,
7:39the founding myth of DeepMind and OpenAI
7:42and Anthropic is these problems are
7:44real.
7:46So, we need to get there first so that
7:48we can handle it responsibly. Those are
7:51I think the big two reasons, but then I
7:52can go on. There's lots more reasons as
7:54well. So, one thing is
7:55you know, World War III, geopolitical
7:57conflict. Um if AI does in fact get
8:00incredibly powerful, that's going to
8:02change the balance of power between
8:03nations. That's going to disrupt a lot
8:04of things.
8:06That puts us at increased risk of crisis
8:08more generally, right? Another one, what
8:10about those jobs?
8:11You you're going to lose your taxi job,
8:14but not just the taxi driver, everybody
8:15pretty much.
8:16Um there might be a few exceptions like
8:18people whose jobs for legal reasons are
8:20only allowed to be done by humans, but
8:23for the most part, everybody should be
8:24afraid that their jobs are going to be
8:25lost even if we manage to avoid all the
8:27other problems, right?
8:29>> This narrative has started to emerge and
8:31I've had several interviews on the show
8:32where I've interviewed people who are
8:34very very scared and anxious about AI.
8:35And these are people that have worked in
8:36the industry for sometimes decades.
8:38>> Yeah.
8:38>> Um the counter narrative coming over the
8:40hill is that this is doomerism.
8:42That these people are for whatever
8:44reason just trying to scare people and
8:46that they don't really understand what
8:47they're talking about. How do you
8:48respond to that sort of counter
8:49narrative? And you must have seen this
8:51emerging yourself, especially from
8:53people who stand to benefit, dare I say?
8:55>> Yeah, exactly. This counter narrative is
8:58fairly recent and it's been pushed by
8:59the people who stand to benefit
9:01um from it and it's not true. Like these
9:04these concerns have been around for
9:06decades since before the AI industry
9:07existed.
9:08They're actually pretty reasonable
9:09concerns. Like if you take the companies
9:11at their word and imagine that they are
9:12in fact going to build
9:13superintelligence,
9:14well, it raises a lot of questions. Like
9:16who's going to control it? Will anybody
9:18control it? What about the jobs? You
9:20know, like th- these are just kind of
9:21obvious
9:22implications to be thinking about and
9:23worrying about.
9:24>> Who are you and what's your story?
9:26>> My name is Daniel Kokotajlo.
9:28Um
9:29I currently run the AI Futures Project,
9:32which is a small nonprofit that
9:34mostly focuses on forecasting the future
9:36of AI.
9:38Before that, I worked at OpenAI.
9:40>> AI forecasting?
9:42>> Yeah, so
9:44think about how like
9:45you know, industry analysts who work for
9:47hedge funds and stuff will make these
9:49forecasts of like
9:50here is, you know, how many cars Tesla
9:52will be selling 5 years from now or like
9:55here's what the price of electricity
9:56will be in 2 years, right? That's
9:59forecasting. I was doing that but
10:01specifically focused on AI.
10:03The reason I was doing it is because
10:04it's incredibly important to to see
10:05where this is all headed.
10:06>> Why did you go to OpenAI? What did you
10:09do there? What did you observe while you
10:11were there and how did it change your
10:12perspective on the future of
10:15AI but also I guess OpenAI as a company
10:17and for anybody that doesn't know OpenAI
10:19are the company that produced ChatGPT.
10:21>> Yeah, so I went to OpenAI in 2022.
10:24Uh a large part of what I did there was
10:25more forecasting. AI 2027 is a scenario
10:27that you may have heard of. I did like
10:29smaller
10:30you know, lower effort versions of them
10:33internally for just internal circulation
10:34of like here's some guesses as to what
10:36the next couple years might look like. I
10:38also worked on evaluations for dangerous
10:40capabilities. So
10:42you know, trying to measure the AI's
10:43cyber abilities or persuasion abilities
10:46or situational awareness and I also
10:49briefly was on a
10:51uh a capabilities team doing
10:52reinforcement learning to create agents.
10:54AI is in fact getting
10:56uh a lot better and I can say more about
10:58why, you know, scaling laws, um deep
11:01neural nets bigger, trained on more
11:03data, become more efficient, more
11:04competent at those things.
11:06I also
11:08became a bit more disillusioned with the
11:11AI industry. So
11:13OpenAI, Anthropic, and DeepMind all had
11:15these sort of founding narratives of
11:17like yes, these risks are real but
11:19we've thought about them and we're going
11:21to try to handle them responsibly and
11:22that's why it's important for us to
11:24keep doing what we're doing and I
11:27increasingly came to think that these
11:28were rationalizations
11:31to justify what they were rather than
11:33sort of like deeply guiding their actual
11:35behavior and that when push comes to
11:37shove they'll follow their incentives
11:39rather than
11:41do what's actually good.
11:43>> So you're inside OpenAI at the time and
11:45you start to believe that they're
11:47following commercial incentives versus
11:49the I guess social or societal
11:52incentives that they founded themselves
11:53on.
11:53>> Sort of. I mean what I wouldn't actually
11:55describe it as commercial incentives. I
11:56think I would describe it as
11:58um
12:00power-seeking incentives. So
12:02like [clears throat]
12:03it's true that the companies care a lot
12:04about making a lot of money
12:06but especially at the very top of these
12:08companies like the leaders
12:11they understand that this is about more
12:12than just money. You know?
12:14There are these emails that came up in
12:15you know the the lawsuit between Musk
12:17and and um OpenAI.
12:20A bunch of emails were surfaced in that
12:21lawsuit which you can go read and in
12:24some of them
12:25the founders of OpenAI were talking back
12:27in like 2017 about how the reason why we
12:29made OpenAI
12:30was because we were worried that
12:33Demis Hassabis at Google was going to
12:34become dictator with AGI. Even back then
12:37they were this obviously about more than
12:38just money. Like these these powerful
12:40CEOs are literally afraid that
12:44if the other guy gets there first he
12:45might become dictator and they don't
12:48trust each other and so that's why
12:50they are racing as hard as they can so
12:52that they're the ones who get there
12:53first so to speak.
12:55>> Have you met Sam Altman?
12:57>> Yeah.
12:58>> And did did that shape your opinion of
13:00his incentives or what why he's doing
13:01what he's doing? Cuz there's a lot you
13:02know speculated about what his
13:04incentives are.
13:05I mean his most recent narrative says
13:07for the good of humanity. I think that's
13:09what
13:09>> Yeah, I mean I think the main thing I've
13:10learned is don't pay attention to the
13:11narratives. You know like uh what they
13:14say to one person is just different from
13:15what they can say to some other person
13:17at the same time and what they say in
13:19public is a third thing entirely. I
13:21think you should judge people by their
13:22actions not by their words.
13:25>> And why are you no longer at OpenAI?
13:27>> Largely the reason that I mentioned. So,
13:28I became gradually disillusioned with
13:30how the company was going to behave.
13:32For example,
13:33when I first joined in 2022, at least
13:36the people I talked to, my colleagues at
13:37the company, there was this general
13:39sense of like, of course we wouldn't
13:41actually just build super intelligence
13:44as soon as possible. Once we started
13:45getting really close, like once we
13:46started getting to AIs that could
13:48maybe automate the AI research process,
13:51we would pause and figure out how to
13:53make it safe.
13:55That's cuz we're the good guys and
13:56that's obviously the safe thing you
13:57should do rather than just going full
13:59speed ahead. But, we're worried about
14:01other people who might not pause, you
14:03know, our competitors, Google, for
14:04example. And so, that's why we need to
14:07be in the lead so that we have that room
14:09to do the safe stuff, right? That was
14:11sort of like a thing that seemed like
14:14maybe like the median position or
14:15something among the colleagues I talked
14:17to when I was there when I started,
14:18including people like Sam, you know,
14:20including the leadership. And then by
14:22the time I left, I was like, "Oh man,
14:23they're really not going to do that, are
14:24they?" Like
14:24>> [laughter]
14:25>> Like they they've sort of
14:27you know, partly because this has become
14:28more politicized and they've become
14:30bigger and been under more scrutiny,
14:32people have started asking like, "Why
14:33are you doing this in the first place if
14:34it's so risky?" And so, they've pivoted
14:36their narrative to being more like,
14:37"Actually, it's not that risky, you
14:38know?"
14:39Um
14:41and so, yeah, I mean, it seems like
14:42they're just going to keep going
14:44roughly as fast as they can and hope
14:46that they can figure it out on the way.
14:47>> How did your time at OpenAI come to an
14:49end?
14:49>> Uh I resigned in 2024. I had a nice
14:52goodbye party.
14:54>> What were the reasons you gave for
14:55quitting OpenAI?
14:56>> I thought that we were rationalizing too
14:58much and that we needed to think more
14:59about what would actually be good for
15:00the world. Um I wanted more freedom to
15:03publish.
15:05So, at OpenAI, as it became a bigger
15:07company,
15:09it became more of a normal tech company
15:11with incentives and, you know, a PR
15:14department and things like that. And so,
15:15it started becoming more difficult to um
15:19to publish the sort of research that I
15:20was doing. For example, those scenarios
15:22that I mentioned, couldn't uh couldn't
15:23publish those, right? They're just for
15:25internal use.
15:27I thought that that was a shame because
15:29right now most of the world is kind of
15:31asleep at the wheel and doesn't really
15:32realize what's going on with AI and
15:34doesn't really realize what's coming in
15:35the pipeline a couple years from now.
15:37And the companies aren't really
15:39incentivized to tell people that much
15:41about it. I mean,
15:42they say some vague stuff in a sort of
15:44hypey way, but
15:46um
15:48you know, well, they didn't want me to
15:49publish the scenario, for example,
15:50laying out like here's
15:52how things might actually look.
15:54>> I'm just kind of super curious as to
15:55what it's like being in a company like
15:56that when they you know, chat GPT-3 is
15:59released. You were there at that time,
16:00right?
16:01>> Mhm.
16:01>> Um which was a moment where I think the
16:03whole world stood up and realized that
16:04this technology was
16:06powerful.
16:08>> Yeah.
16:08>> Um and the conversation really began
16:09from a society level.
16:11Um company starts growing super quickly.
16:14>> Yeah.
16:14>> Quicker than I think anybody could ever
16:16have imagined.
16:17And what what was it like inside there?
16:19What did you see change um over over
16:21that period of time?
16:23>> I remember one all-hands meeting where
16:24Ilya said something like
16:25>> Ilya being
16:26>> Ilya Sutskever, who was um head of
16:28research at that time. He said something
16:30like, "Okay, now the world is starting
16:32to pay attention. Each of you is going
16:33to be the most popular person at every
16:35party
16:36uh for the next year.
16:38Don't let it get to your head. Focus on
16:39the mission. Got to build AGI."
16:41>> [laughter]
16:42>> The company grew a lot. It already
16:43wasn't really feeling like a nonprofit
16:45when I joined, but it definitely didn't
16:47feel like a nonprofit by the time I
16:48left. Um lots of new people came in.
16:52Ironically, the like
16:54amount of conversation about
16:57superintelligence and the implications
17:00of superintelligence arguably you sort
17:02of went down over time
17:04due to this growth, right? So, because
17:07the company would like double and then
17:08double again and then double again, all
17:10these new people were coming in from
17:12other parts of the tech industry who
17:13hadn't really been thinking about these
17:14things and were attracted by the high
17:16salaries.
17:16>> You lost $2 million
17:18for not signing an anti-disparagement
17:20clause,
17:21which would mean you could speak you
17:23couldn't criticize the company.
17:25>> Ah, yes. Well, so um I got to keep the
17:27money.
17:28>> Oh, you got to keep the money?
17:28>> what happened was after I had left, said
17:31my goodbyes, etc.
17:33Um I got the the exit paperwork and it
17:36included this clause that said you
17:38basically have to agree not to criticize
17:39the company again.
17:40Um and also a clause saying you can't
17:42tell anyone about this.
17:43And so
17:45I thought that was kind of
17:47rich coming from a nonprofit that's
17:49supposed to be,
17:50you know, for the benefit of all
17:51humanity. So, I didn't sign it. And if
17:54you don't sign, you don't get to keep
17:55your equity. So, your compensation, you
17:58know, what what they pay you is a bunch
17:59of money and then also a bunch of
18:02stock, basically. But then they had this
18:04stuff in the contract that
18:06they get to yank back your your stock if
18:09you don't sign this thing.
18:11Um
18:12and my wife and I, you know, were
18:15uh upset about this. We talked about it
18:17for like a month or two, consulted some
18:18lawyers, um and then ultimately decided
18:20to just refuse to sign.
18:22>> Which would mean you lost you would have
18:24lost $2 million.
18:25>> That's right. Which was like 80% of our
18:27net worth at the time.
18:29Um fortunately, uh
18:32it didn't go the way we expected. It
18:33blew up basically on the internet. Like
18:36when people heard that that we had done
18:37this and that we had said no, it became
18:40like this huge scandal. Employees at the
18:42company started like asking questions in
18:43Slack and like asking leadership like,
18:45wait, what? Like why are you going to
18:47take away our equity? What is this? You
18:49know, cuz a lot of people hadn't really
18:50noticed this before. It had been
18:52whispered about, but it hadn't been sort
18:53of like
18:54a thing that most employees knew about.
18:57Um and so they backtracked and they
18:58said, "Never mind, never mind. We'll
18:59change the paperwork. You can keep the
19:00equity.
19:01It's fine."
19:02>> And so management came out and said he
19:04was embarrassed that he didn't realize
19:05this was going
19:06>> Yeah, he had no idea, apparently.
19:08>> You don't believe him?
19:09>> No.
19:10I think he probably knew. And if he
19:11didn't know, then people close to him
19:12probably did, such as his head lawyer.
19:14>> Why did you decide not to take the $2
19:17million?
19:19I mean,
19:20most people would have, I think.
19:22>> It's true, most people would have, and
19:23most people did.
19:24And you know, money is nice, but like
19:27it's not the only thing, you know?
19:29Sometimes it's good to take a stand on
19:31principle.
19:32I I keep mentioning superintelligence.
19:33Perhaps I should say more about like
19:35the
19:36the sequence of events that the
19:38companies are planning to do.
19:40So,
19:41right now, they're focusing on
19:42automating coding. They're taking their
19:44AIs, they're making them bigger, they're
19:46training them for longer, and they're
19:48especially focusing the training on
19:50getting them to be good at autonomously
19:51writing and editing code. Because
19:55uh that will help the companies go
19:57faster, right? If they can automate the
19:58code, then they can do their own work
20:01better and faster, and accelerate
20:03progress.
20:04The next step, which they've already
20:05begun, is to
20:07look at the rest of the research process
20:09as well. Coming up with ideas,
20:11um analyzing experiments, communicating
20:13those results.
20:15All the other parts of of the research
20:17process, they're trying to figure out
20:18how to train AIs to be good at those as
20:19well.
20:20So that they can have AIs do the entire
20:22thing autonomously.
20:24>> When you say do the entire thing, what
20:26you mean [clears throat]
20:26do the entire thing?
20:27>> So like Anthropic and OpenAI in
20:29particular are trying to automate
20:31themselves. Like they're trying to make
20:32it the case that
20:34um they don't really need human
20:35employees anymore. Uh they just have a
20:37giant army of AIs that's
20:40churning away,
20:41doing all this autonomous research to
20:43make better AIs, to train the new AIs,
20:46put them in charge, so they can make
20:48even better AIs and so forth. And of
20:50course, not just not all just happening
20:52internally, but also like interfacing
20:54with the world, right? Like going out
20:55and talking to people, collecting the
20:56data, setting up the training
20:57environments,
20:58doing the business deals, and so forth.
21:00Like they're they're trying to automate
21:02all of that. The reason why they're
21:04doing this is because they're trying to
21:06get to a position where they have
21:09AIs that are superhuman
21:11at everything, superintelligence, and
21:13they're trying to get there before their
21:14competitors do.
21:16Needless to say, this is incredibly
21:17dangerous, I would say, you know. And in
21:20addition to being dangerous,
21:22it's a power grab, right? Like if they
21:24actually succeed at this, then they'll
21:26be sitting on top of this army of
21:28superhuman AIs that will give them
21:31immense leverage over all sorts of other
21:33actors in the economy in so far as they
21:35can work out something with the
21:36presidents and, you know, integrate it
21:38into the military or whatever, then that
21:40would give the US immense hard power
21:42over all of the countries, right?
21:44Obviously, nobody knows exactly when
21:46this is happening.
21:47But a very disquieting thing has
21:49happened over the last year to me,
21:51which is that when we published AI 2027,
21:55people were generally of the opinion
21:57that my timelines were too short.
21:59And that like probably it would take
22:01more than 2027 until we got to
22:04the sort of events that I was just
22:06mentioning, you know, uh recursive
22:07self-improvement, AIs automating the
22:09whole research process,
22:10superintelligence.
22:12These These types of milestones
22:14um they happen in 2027 in AI 2027,
22:18>> which is this research paper you
22:19published.
22:19>> That's right. It's It's a scenario
22:21forecast that sort of lays out like
22:23month by month a possible future
22:25trajectory. There was sort of like At
22:27the time that we started writing, it was
22:28my best guess as to what would actually
22:30happen. Obviously, there's lots of
22:31uncertainty, but, you know, I thought
22:33it's valuable to make a concrete guess
22:35just to sort of see what it might look
22:36like.
22:37And at the time we were writing this, a
22:38lot of my friends in the AI industry and
22:41in nonprofits and so forth that work on
22:44AI, a lot of people were saying like,
22:45"Yeah, that stuff's going to happen, but
22:47like it'll probably take a couple years
22:48longer than you think."
22:50And now
22:53it's more 50/50, especially when I go
22:55talk to people at Anthropic and OpenAI.
22:58They're often like,
23:00"Yeah, no, 2027, that's basically what's
23:02going to happen.
23:03Just like you wrote. Why did you Why did
23:06you become Why did you update your
23:07timelines? Oh, yeah, context for this is
23:10after after writing AI 2027,
23:13I shifted my timelines to be a little
23:14bit more conservative. So, at the time
23:15that we published, my 50% mark was in
23:182028, not in 2027.
23:20And then after we published, progress
23:22just seemed like it was going a bit
23:24slower, and so I updated to 2030.
23:27Which is, you know, still could happen
23:28sooner, could happen later. 2030.
23:31Um but now, when I talk to people in in
23:33the company, they're like, "It's not
23:35going to take that long."
23:36They're like, "Oh, you need to shorten
23:38them again. Like, get them back to 2027
23:40or 2028, you know."
23:42Um so, that's a bit disquieting. Um
23:45again, don't know how long it's going to
23:46take, but this is the stated plans of
23:49the uh companies is to do this
23:50incredibly dangerous thing, and they
23:51think that they're just a few years
23:53away.
23:53>> So, you wrote this um report here, What
23:562026 Looks Like, and you wrote this in
23:582021,
24:00and it was remarkably accurate. Helped
24:02make a name for yourself amongst um
24:05amongst uh everybody in AI. And I Which
24:07one was it that J.D. Vance, the vice
24:08president, read? I think it was this
24:09one, wasn't it? Yeah, this one. Um
24:12and then so, then you published this
24:13one, AI 2027, and this was published, I
24:15believe, in 2025.
24:17>> Uh yes, that's right. April.
24:18>> Yeah.
24:19>> What were you forecasting in here? What
24:21are What are the key things that you
24:22said in here for people that haven't
24:23read it?
24:24>> The high-level version of it is
24:26they automate the coding, then they
24:28automate the rest of the research
24:29process, then the pace of progress
24:31accelerates dramatically. They get to
24:32superintelligence. They're working with
24:34the government, specifically the
24:35president, the executive branch
24:37naturally wants to control this
24:38technology, in other words, wants to use
24:40it to beat China and integrate it into
24:41the military and so forth. By this
24:43[snorts] point, it's sort of
24:45doing basically all the work itself. I
24:46mean, it's it's superintelligence, so
24:49it's coming up with all these great
24:50ideas for how to integrate itself into
24:52everything and all these new
24:52technologies it's invented and so forth.
24:55And uh because of the race dynamics and
24:57because of the profit motive, they end
24:58up deploying it everywhere. And it
25:00builds robot factories that build more
25:01robots that build more robot factories,
25:02etc. Transforms the world entirely.
25:05And then at some point it has enough
25:07power it, meaning the AIs, have enough
25:10power that they don't have to pretend to
25:13to be aligned anymore.
25:15Right? Um then they
25:17stop listening to orders.
25:19That's the race ending
25:22of the 2027.
25:24We also wrote a sort of different
25:25branch, which is the slow down ending,
25:27which is intended to sort of illustrate
25:30the concentration of power issues um
25:33that I mentioned previously. So,
25:35what if hypothetically
25:36the alignment issues get sorted out
25:38sufficiently quickly? Like what if it
25:40turns out that like
25:41it's not too hard. With 2 months of slow
25:43down, we can figure out how to make the
25:45AIs robustly do what we want um and have
25:48the values that we want them to have.
25:49So, that's one possible branch. And in
25:51that branch, uh it looks pretty similar,
25:53you know, they take the jobs, beat
25:56China, etc. Um
25:58but instead of the AIs ultimately
26:00killing everyone, they create this sort
26:02of amazing utopia. But the amazing
26:05utopia is
26:06whatever the people who control the AIs
26:08want it to be, right? And so that would
26:10be a very small group of people, like
26:11the presidents, some CEOs, etc.
26:15>> There should be a button just down below
26:17here. And if it says subscribe, you're
26:19already subscribed. If it says subscribe
26:21buh, that means you're not yet. And if
26:23you're not subscribed, please could you
26:25do us a favor and hit that button. It
26:26helps to show more than you know. And
26:28according to the algorithm, you're
26:29someone that watches our show, but you
26:31haven't yet hit that button. Thank you
26:32so much. Is there any possibility, do
26:34you think, that we never get to this
26:36thing called AGI? And and how do we
26:38distinguish AGI from this term super
26:40intelligence? What's the difference?
26:42>> Yeah, so the difference is that AGI is a
26:43more vague uh and weak term.
26:46>> Okay.
26:46>> So, super intelligence is a bit more
26:48precisely defined. It's better than the
26:49best humans at everything, faster and
26:51cheaper. Um AGI is more like it stands
26:53for artificial general intelligence,
26:55which means AIs that can do things in
26:57general rather than like some specific
26:58task. Yeah. And so arguably we've
27:00already achieved AGI, right? If you use
27:02cloud code or something like that, it's
27:04like it can do a lot of stuff. It's it's
27:06almost kind of like a little employee
27:07that you can like have go do stuff. So
27:09it's it is quite general.
27:11It's not maximally general though. Can't
27:13do everything. Whereas super
27:14intelligence by definition
27:15can do all the things that a human can
27:16do but better.
27:17>> And how does this sort of overlap with
27:19robotics? Because obviously that we're
27:21seeing this huge robotics boom at the
27:22moment. There are some real world things
27:24that humans can still do because these
27:26AIs are still stuck in my computer.
27:28>> The way that people talk about this is
27:29that they
27:30basically just say we've achieved super
27:31intelligence for cognitive tasks. Then
27:33you can talk about like
27:35full super intelligence that can do the
27:37physical stuff.
27:38>> And are we going to get there? Are we
27:39going to get there with both?
27:40>> I think so. I mean again, this is not
27:42something that we can be certain about.
27:43Um, you asked like is it possible we'll
27:45never get there? Yes, it's possible
27:46we'll never get there.
27:47I don't think it's likely though.
27:49I think that
27:50there's nothing sort of like magical
27:51about the human brain. It's
27:54you know, um, it's just a bunch of
27:55neurons. It is possible for a digital
27:58system to
28:00do similar functions in the same way
28:01that like,
28:03you know, a plane can fly
28:05just like a bird. Not in the same way as
28:06a bird necessarily. Like it doesn't have
28:09it's not flying in the same way that a
28:10bird flies, but it flies, you know?
28:12Um, so so it does seem like yeah, like
28:15seems possible.
28:16>> You've written all these, you know,
28:16these research reports. You're working
28:18on another one that'll be released um,
28:19likely on the 9th of July.
28:22You have worked inside OpenAI. You then
28:25quit OpenAI because you were concerned
28:27about what was going on there and about
28:28the future of the industry. You know
28:30more than I do.
28:32Are you optimistic about the future or
28:35pessimistic? Are we heading to a bad
28:36place if things don't change um, based
28:39on everything that you know?
28:40>> I think we are headed to a bad place if
28:42things don't change. Um, I'm not
28:43confident in that. I would say something
28:45like 70%. It's very very hard to
28:47predict, of course, but yeah, it seems
28:49like the current default path is heading
28:51towards a very, very scary place.
28:53>> How do you contend with that personally
28:54and emotionally?
28:55>> Um
28:57it's rough. I mean, I think it It's the
28:58sort of thing that like
29:01gets me down
29:04on a regular basis, but also I've been
29:06dealing with this for so many years now
29:08that
29:09I've sort of gotten used to it, if that
29:10makes sense. Um
29:15yeah. Yeah, I I'll put it this way. I
29:17would be incredibly happy if all my
29:19predictions turn out to be wrong and
29:22uh and AI hits the wall, for example.
29:23>> It gets you down on a regular basis.
29:25>> I used to be known as a pretty chipper
29:27and optimistic person, but
29:30um in 2020
29:31my AI timelines predictions started
29:34collapsing due to GPT-3 and the scaling
29:37laws papers and um the bio anchor
29:39report, which I I can talk about if
29:41you're interested, but basically some
29:42events happened in 2020 that convinced
29:44me that actually this stuff was like
29:47quite plausibly coming by the end of the
29:48decade.
29:49And
29:50humanity is very obviously not ready for
29:52this, you know, in a whole bunch of
29:53different ways. And so that's obviously
29:55very scary.
29:56>> And that's a extremely scary world
29:58because of all the things you've said,
29:59but but again, because of this recursive
30:00self-improvement where AIs can train
30:02themselves. And at such point we're
30:04starting to lose hold of what's going on
30:06here.
30:06>> I mean, the AIs are already training
30:07themselves, to be clear. It's more like
30:10closing the entire research loop, right?
30:11So
30:12>> everything.
30:12>> Yeah, like right now a lot of the
30:14training data is generated by AIs. A lot
30:17of the reinforcement, like the grading
30:20that happens, doling out of positive and
30:21negative reinforcement, is itself done
30:23by AIs.
30:24>> Can you explain that in layman's terms
30:25for
30:25>> Yeah, so an important thing for
30:27everybody to understand is that modern
30:29AI systems are not software in the
30:31normal sense. I mean, they are
30:33technically software, but
30:34they're not lines of code, you know?
30:36It's not like some engineers at
30:38Anthropic went and wrote lines of code
30:41that basically says like, you know, when
30:43the user asks for this type of thing,
30:46then go do this type of thing for this
30:48many steps or whatever. There's nothing
30:50like that. Instead, it's a neural net,
30:51you know?
30:52>> What's that?
30:53>> Well,
30:54think about how the brain is a bunch of
30:55neurons connected to each other
30:56>> Yeah.
30:57>> that are firing
30:58um signals back and forth. The brain
31:00learns over time
31:02the types of patterns of firing that
31:05caused success, that caused a dopamine
31:08rush, or various other types of feedback
31:10get reinforced and fire more often. And
31:13the types of patterns that caused
31:14failure, like touching a hot stove, get
31:17anti-reinforced, they get, you know,
31:19um destroyed, so that they fire less
31:21often. And as a result of all of that,
31:24you over the course of years learn to
31:27act in the world, and you learn all
31:28sorts of skills, and you learn world
31:30models, you learn like beliefs about the
31:32world, and you can sort of like mentally
31:33simulate how it's going and stuff like
31:35that. So, artificial neural nets are
31:37like that, except artificial. So, it's
31:39it starts off as a giant
31:42tangled spaghetti mess of randomly
31:45generated uh
31:47artificial
31:48connections called parameters.
31:50These days, they might be something like
31:5210 trillion parameters
31:54uh it in the biggest AIs.
31:57So, it starts off randomly generated.
31:58So, it's of course completely useless.
32:00Like, if you
32:01give it some input, it'll just produce
32:03gibberish as an output. But then they
32:04train it, and they
32:07start with pre-training, which is where
32:09you give it a bunch of internet text,
32:12and you show it the first piece of text,
32:14and you put that in as the input, and
32:16then it gives a gibberish output,
32:18and then you positively or negatively
32:20reinforced it based on how accurate that
32:22output was at predicting the next piece
32:24of text. Um so, it's basically playing
32:27this game of like predict the next word.
32:29>> Isn't that how it happens with babies? I
32:31had a I think I had a neuroscientist
32:32tell me that babies have more neural
32:34connections
32:35um than adults. And yeah, it says yeah,
32:38toddlers have twice as many neural
32:39connections as adults. And they, I guess
32:42they whittle down through reinforcement.
32:44Yep. We have more pathways when we're
32:46younger. And just like the process of
32:48training an AI, we're trained down to
32:50like remove the ones that aren't useful
32:51and build up on the ones that are.
32:53>> Yeah, it's both pruning and
32:54strengthening. And it seems like in
32:56humans it's actually more pruning than
32:57strengthening, but it's both. Uh, and in
32:59AI it's the same thing, it's both. So,
33:02the first portion of training is where
33:03they train the AI to predict text, which
33:06is kind of like training it to read. Um,
33:08and it it's a similar thing does happen
33:09in humans. So, basically,
33:11the the random tangle gradually takes
33:14shape and gradually sort of coalesces
33:17into more useful circuitry that has
33:19stored lots of facts about the world and
33:21has stored lots of skills for how to,
33:24you know, process information and
33:26transform it and then produce
33:28predictions.
33:29That's just the first step. After they
33:31do the pre-training, then they
33:33try to teach it more useful skills
33:35besides just predicting text. And so,
33:38you know, by the end of the process,
33:39they've thrown lots of coding problems
33:42at it. And they've said like, here's a
33:43coding problem, go. Here's a coding
33:45problem, here's an environment, you have
33:47access to this virtual computer, here's
33:49like the code base you're working with.
33:50You can write code, you can edit the
33:52code, you can run the code, you can read
33:53it, you can use the internet.
33:56Go, go, go. And it does that for a while
33:58and then based on how successful it is,
34:00reinforcement happens and they have
34:03thousands, maybe millions of examples of
34:05coding problems like that that they
34:06trained it on. And that's why they're so
34:08good at coding now.
34:09>> So, what does superintelligence look
34:11like in this regard? Is it just more of
34:12these connections? And how would they
34:14get more connections? Can you explain
34:16that to me like I'm
34:17>> So, there's different AI models, right?
34:19So, there's like,
34:20you know, GPT-3 and GPT-4 and GPT-4.5
34:23and GPT-5 and GPT-5.5 and 5.6, right?
34:26Sometimes they're just the same previous
34:28model but with extra training. Sometimes
34:31they're are new model that's been
34:32trained from scratch, including starting
34:34the whole pre-training process again.
34:36Over the last couple years, they've done
34:38several new rounds of starting over from
34:40scratch. And typically when they start
34:41over from scratch, they make the whole
34:44thing bigger, the the artificial brain
34:45much bigger. Right now they're at
34:47something like 10 trillion parameters.
34:49Back in 2020, um
34:51it was more like 175 billion.
34:55So, we've grown like two orders of
34:56magnitude
34:57uh in 6 years.
34:58>> Two orders of magnitude.
34:59>> Yeah, like two 10 x's. So, 100 x, right?
35:03So, that process is continuing. Um
35:06they're also improving the algorithms
35:08themselves. So, they're not literally
35:09just the same type of AI but bigger.
35:12They've also come up with all sorts of
35:13ideas for how to change the structure of
35:16the of the connections in the neurons
35:18and so forth and change the like
35:19reinforcement
35:21algorithms that they're using and to
35:22change the training data that they're
35:25training on.
35:26All sorts of tweaks that have made this
35:27whole thing more efficient.
35:29>> We're literally building a brain.
35:30>> Basically, yeah. As they make more
35:32brains, they're getting better at making
35:33They're making them bigger and making
35:35them more efficient and so forth.
35:37>> And it's literally modeled on the brain,
35:38like the way it works, right?
35:39>> It's It's certainly heavily inspired by
35:41the brain, but I I shouldn't overstate
35:43the the analogy. Like there's lots of
35:44differences, too. So, for example, the
35:46transformer architecture um
35:48>> Which is
35:49>> Which is the architecture that they use
35:50for for these LLMs
35:52uh is not really recurrent. So, the
35:55information sort of flows one way rather
35:57than allowing all these sort of little
35:58loops on the inside. Also, the the
36:00backpropagation algorithm is different
36:02from the sort of um learning that
36:04naturally happens in human brains. So,
36:06there are some differences, but yes,
36:07like broadly speaking, uh we are sort of
36:10making artificial brains. It's kind of
36:11like for brains what like a plane is for
36:14a bird.
36:14>> Mhm. Yeah, that's a [clears throat]
36:15really good analogy.
36:16>> Yeah.
36:16>> That that analogy helped me think
36:18through a bunch of questions people
36:19often ask about AI when they said, "Can
36:21it be creative?"
36:22But actually that analogy kind of helps
36:24me understand that actually that maybe
36:25that's not the question.
36:27It's can it produce something that you
36:29would consider to be creative because
36:31[clears throat] creativity is people
36:32think of it as like a process, but
36:33actually it's it's judged based on the
36:35output, isn't it?
36:36>> I mean you you can get philosophical
36:38about like is it truly creativity that
36:39they have, but you can also be like
36:41well, I mean just look at all the stuff
36:42they're accomplishing,
36:43>> [laughter]
36:44>> you know, and it seems like they're
36:46going to be accomplishing a lot more in
36:47the near future.
36:48>> Yeah, I do I I asked the question about
36:50how this weighs on you personally
36:51because I can I can sense that you're
36:53actually personally bothered.
36:55>> I mean that I think the situation is
36:56crazy. Like
37:00first of all, it's very exciting. Like
37:01AI is really fascinating and interesting
37:02stuff. I've been following the field for
37:04more than a decade now.
37:05I've been part of it
37:07for some years and um
37:09it's really cool, really interesting and
37:11it's really fun to think about what's
37:13going on inside these artificial brains
37:15and why they are the way that they are
37:16and it's really cool to see all the
37:18applications of this technology out in
37:20the world.
37:21But it really seems like we're on a
37:23pretty scary path and the more you think
37:25about it, the more worried you get and
37:28you know, in stories
37:30it always ends well, but this is real
37:31life.
37:32And I I think we have to sort of
37:36stare reality in the face and tell it
37:38and realize that like it might not
37:39actually end well, you know.
37:41>> Were there any recent
37:43dare I say I was going to say eureka
37:45moments, but paradigm shifting moments
37:46where even your own sort of mental model
37:49of what's going on here and how this is
37:50going to look were changed for better or
37:52for worse?
37:53>> For better or for worse and probably for
37:54worse, things are kind of on track for
37:56AI 2027. There are a few things that
37:58have been different not exactly like
38:00paradigm shift differences, but like
38:02there have been some differences from
38:03what we expected at the time we wrote
38:05this. So
38:06the government has actually got involved
38:07faster than we expected and has been
38:09more aggressive than we expected. So the
38:10export controls on mythos being the
38:13biggest example and also threatening
38:15Anthropic with
38:16being destroyed by the defense
38:17production production act.
38:19Um
38:19>> [clears throat]
38:20>> Another thing that's been surprising to
38:21us is that Anthropic in particular has
38:24gone from second place to first place in
38:26the sort of in the race basically.
38:29>> Why do you think that happened? Because
38:30it seemed like ChatGPT were out front
38:33and clear as it relates relates to
38:34OpenAI were out front and clear but
38:36suddenly Anthropic have uh
38:38lapped them.
38:40>> Yeah, I mean I guess they have um
38:42probably higher talent density
38:44um and better strategy
38:46but not by a lot but enough to make the
38:48difference.
38:49>> Why do you think they have more talent?
38:51>> Well
38:53they don't have more compute. Like what
38:54are the inputs, right? Like they're in
38:56the lead now, they used to be behind.
38:58What are the possible explanations for
38:59this? Well, it could have been that they
39:01had more resources like more compute
39:03more money but that's not true. They
39:04have less resources less money, right?
39:07So then I guess talent's what is is the
39:10next best alternative. You could maybe
39:12say strategy.
39:13Some combination of those things, yeah.
39:15Something that wasn't just like the
39:16amount of resources they had.
39:18>> Just like Jon Jones where marginal
39:20improvements in your cognitive
39:21performance can have a massive impact.
39:23Sometimes I podcast for 10 hours a day.
39:25Over the last couple of weeks I've been
39:26in filming for a TV show and then I have
39:28like one or two days off to get all of
39:30my work done which means there's lots of
39:31cognitive load. And so I turn to ketones
39:34because I find myself more articulate,
39:36able to think more clearly, able to work
39:38out better when I'm fueled by ketones.
39:41And so the reason I became a convert of
39:42this company and the reason why they now
39:43are sponsoring this podcast is because I
39:46remember one of my team members called
39:47Christiana, she tried it once and came
39:48up to my desk and she goes this is the
39:50best product ever made. And I think in
39:52part that's because she really cares
39:54about those cognitive benefits as I do
39:55as Jon Jones does and as I think most of
39:57my listeners probably will. So if you
39:59haven't tried these yet, all you have to
40:01do is go to ketone.com/steven
40:05and you'll also get 30% off your first
40:06subscription order. You'll get exclusive
40:08ketone IQ merch and of course cognitive
40:11benefits that might just change your
40:13life.
40:14Much of the reason most people haven't
40:16posted content or built their personal
40:17brand is because it's hard and it's time
40:20consuming and we're all very very busy
40:22and if you've never posted something
40:23before,
40:25there's so many factors in your
40:27psychology that stop you wanting to
40:28post. What people will think of you. Am
40:30I doing this right? Is the thing I'm
40:32saying absolutely stupid? All of these
40:34result in paralysis which means you
40:36don't post and your feed goes bad.
40:39I'm an investor in a company called
40:41Stand Store which you've probably heard
40:42me talk about and what they've been
40:43building is this new tool called Stanley
40:45that uses AI, looks at your feed, looks
40:48at your tone of voice, looks at your
40:49history, looks at your best performing
40:50posts and tells you what you should
40:52post, makes those posts for you. You can
40:54also you just use it for inspiration and
40:57sometimes what we need when we're
40:58thinking about doing a post for our
40:59social media channels is inspiration.
41:02Building an audience has fundamentally
41:03changed my life and I think it could
41:05change yours too. So, I'm inviting you
41:07to give this new tool a shot and let me
41:09know what you think. All you have to do
41:11is search coach.stand.store
41:13now to get started.
41:16>> Uh a friend of mine who knows some of
41:17these people sat me down once upon a
41:19time in London. He's actually said this
41:21a few times to me but I remember one
41:23particular conversation where he says
41:25that
41:26some of these AI CEOs predict the
41:29probability of extinction at being I
41:31think he said 7%. I don't know why I
41:33have that number in my head but I
41:33remember it being less than 10% and the
41:35point he was making to me was that even
41:37if it was 1%. Like if there was 100
41:40buttons on this table now
41:41>> Yeah.
41:42>> and one of them would end the world.
41:44Would I dare
41:45>> I wouldn't press any of them
41:46>> you know. [laughter]
41:47Um
41:48>> No.
41:48>> I wouldn't press any of them but he made
41:50the case to me that these AI CEOs are
41:52very smart and they understand super
41:53intelligence and that they think
41:54actually if there was 100 buttons on
41:55this table right now, maybe 10 of them
41:58could end the world. I've heard you say,
41:59I think it was on the the the Daily
42:01Show, the interview you did, you said
42:02that you think there's a 70% chance of
42:04human extinction due to AI.
42:06>> I wouldn't say human extinction exactly.
42:08I'd say something like 70% chance that
42:10this goes horribly wrong like human
42:11extinction but that's just one of
42:12several possibilities. But yeah,
42:13basically
42:15Like for example, possibly the AIs take
42:17over and then don't actually kill
42:20everyone.
42:21You know, maybe they do something else.
42:23Like just just cuz they've taken over
42:24doesn't mean they're
42:25definitely going to kill us, right? They
42:26might, but they could do something else.
42:28So that's what that's that's why I don't
42:29usually say like
42:3070% chance of like actual human
42:32extinction, but 70% chance of like
42:34something like AIs taking over, some
42:36some sort of very big catastrophe like
42:38that that could lead to human
42:39extinction.
42:39>> I see what you mean. So two points
42:40there, which is you've been around these
42:42CEOs. I mean you've worked for Sam
42:44Altman at OpenAI before you quit.
42:46Do you think that they think there's a
42:48chance of human extinction?
42:49>> Yes.
42:50But
42:51I think that the important thing to
42:52understand is that
42:54like people sort of believe what they
42:56need to believe in order to think that
42:58they're great people and that they need
43:00to keep doing what they're doing. This
43:01is what rationalization is. And so I
43:04think that the tech CEOs have like
43:05genuinely convinced themselves that like
43:08probably things are going to be fine and
43:10that the way to make things fine is for
43:12them to keep doing what they're doing.
43:13And like they need to like make sure
43:14that like, you know, Sam needs to make
43:16Sam's probably thinking like can't let
43:18Dario or Elon
43:19get there first, you know, I know
43:21Dario's thinking Sam can't get there
43:23first. Elon's thinking that like, you
43:25know, they they they've all probably
43:26convinced themselves that like, oh yeah,
43:27like maybe it'll go horribly wrong, but
43:28like
43:30probably it's going to be fine and
43:31probably
43:32you know,
43:33I should be the one in charge.
43:35>> It appears to me that Anthropic are the
43:36only ones that are all talking about the
43:38potential chance of extinction or
43:40catastrophic event or
43:42um the down the real downside still.
43:44They seem to be the only ones that are
43:45still publishing on it and now they're
43:47actually becoming the enemy in many
43:49respects of the
43:50>> Yeah.
43:50>> the tech industry in San Francisco. I'm
43:52watching a lot of interviews and it's
43:53everyone's attacking Dario because he's
43:55saying, "Listen, things could go bad."
43:56They're calling him a doomer uh and
43:58questioning his incentives. Even with
43:59Mythos, which is a an a Claude model
44:01that they started to warn the world
44:03about, again, he is attacked immediately
44:05for saying that.
44:06>> Yeah.
44:07>> My question is, do you see him as being
44:09slightly different from Sam in this
44:10regard?
44:11>> Yeah, I mean, it seems like Anthropic
44:14and Stereo have been more willing to
44:17say and do things that are costly to the
44:19bottom line.
44:21Uh and at least in the last year or so.
44:23That's an example of it. Um like I don't
44:25think that really wins them favors in
44:26the administration or among their
44:29investors to say that type of thing. And
44:32you know, a better example is just the
44:34whole fight between the Department of
44:35War and Anthropic was an example of them
44:37doing something that like cost them a
44:38lot of money and even more importantly
44:40cost them a lot of power
44:41for
44:43something that like like they could have
44:44just signed the contract, you know.
44:46That said, I really don't want to be in
44:48a situation where we're like, which CEO
44:50is the least bad CEO? Let's support that
44:52one. You know, like none of these people
44:54should be trusted
44:55uh with that much power, basically.
44:57>> Nobody should.
44:58>> Nobody should.
44:59>> Regardless.
44:59>> Regardless, yeah.
45:00>> Mhm. So, uh on this point of the
45:02buttons, you you you do believe that
45:04they think there's a credible chance of
45:06extinction.
45:06>> Yeah, but they've [clears throat]
45:07convinced themselves that like it's
45:08probably fine and also it'll be even
45:10worse if I'm not doing it, you know.
45:13Like that's that's what they'll say
45:14inside the companies, too. Like the two
45:15people will be like, okay, well, if we
45:17stop,
45:18what about the other guys? Like they're
45:19not going to stop, you know?
45:21>> Yeah, this is this is always been why
45:22I've had this outstanding question,
45:23which is how does this not go bad when
45:25human incentives seem to rule the day
45:27when you look at history and all of the
45:28human incentives are saying, well, if
45:30you you're damned if you do,
45:32I you're damned if you carry on
45:33developing these bigger and bigger
45:34bigger AI brains, but you're also then
45:37damned if you don't from an a
45:38geographical perspective cuz the United
45:40States will lose to that country or this
45:41company will lose to that company. So,
45:43when you just look at human incentives
45:45and goes, how does how does if just you
45:46purely incentives and disincentives, how
45:48does this end? Well, it carries on
45:49going.
45:50>> Seems like it. I mean, there there is a
45:52caveat to that, which is a hopeful
45:53caveat, which is that
45:55first of all, if the world wakes up to
45:56all of this,
45:57then there can be a more serious
45:59conversation about regulation and
46:02international treaties and things like
46:04that. And that can change the
46:05incentives, right? So, the government
46:08could come in and say like actually
46:10here's some rules that you all have to
46:11follow. And because they're rules that
46:13you all have to follow, then you're not
46:14incentivized to like
46:17break them anymore because you get
46:18punished if you break them and
46:20everyone else is also following them,
46:21too. And so, you know, it's fine. So, so
46:24there is that sort of like ray of hope
46:25that like we can change the incentives
46:28if the government and especially the US
46:31government but then later other
46:32countries act to to change the
46:34incentives. But that's not going to
46:35happen until people sort of wake up to
46:37all of this.
46:38The second thing is that even
46:39individually
46:41at some point
46:43you know, Dario or Sam or Elon might
46:45realize that like actually it's like not
46:48even in their own interest
46:50to to keep racing unilaterally.
46:52And it it on the problem with that is
46:54it's only if it gets extremely obvious
46:55and extremely dire. So, like
46:57in in AI 2027, in that scenario, there's
47:00this choice point that I mentioned. And
47:02in one case the AIs are misaligned and
47:04the other case the AIs are aligned.
47:06At that choice point, we have like one
47:08branch that depicts the the misalignment
47:10ending and one branch that depicts like
47:11they they slow down a bit and solve the
47:13alignment issues.
47:13>> Mhm.
47:14>> The instigator for that choice point is
47:16they see some evidence that their AI
47:18might be misaligned and plotting against
47:20them.
47:20Right? So, if you actually see that
47:22evidence
47:23then it's like
47:24oh gosh, uh
47:26maybe we shouldn't put it in charge of
47:28everything and let it rip, you know?
47:31Because that evidence is staring us
47:32right in the face that it's this
47:33untrustworthy, you know? But if they
47:35don't see that sort of very clear
47:37evidence, then
47:39I think they're going to convince
47:40themselves that they need to keep going,
47:41you know? But maybe they will see very
47:43clear evidence like that. In which case,
47:45even if we don't have regulation, they
47:46might just sort of voluntarily stop.
47:48Um so, that's the second ray of hope.
47:51Like overall, I don't think that we're
47:52like definitely doomed, you know? Like
47:54[snorts] I said 70% but like
47:56I could see it working out pretty well
47:57as well.
47:58>> Hm.
48:00What about uh jobs?
48:02>> Yeah.
48:03So, I think I think I'm excited to at
48:06some point get into the new thing which
48:08is the more optimistic
48:09>> [clears throat]
48:09>> positive vision.
48:10Uh and that will have a lot to say about
48:12this.
48:13Because in the in the in the prediction,
48:16you know, in the year 2027, by the time
48:18everyone loses their jobs, there are
48:20worse things happening. Or like it's
48:22it's kind of like too late by that
48:23point. Um but yes, like once if I mean
48:26just just think about it. If the
48:27companies do manage to build
48:28superintelligence, then by definition,
48:31they're going to be able to take almost
48:33all the jobs or all the jobs, right? Cuz
48:35it's better, faster, and cheaper than
48:37the best humans at everything.
48:38>> And that's I mean, the timeline is by
48:39the end of sort of 2030, you reckon you
48:42think superintelligence might arrive.
48:43I'm trying to think about when we could
48:44start to see job displacement in the
48:46economy.
48:47>> We're already starting to see a little
48:48bit of it now, but not very much.
48:49>> Why?
48:49>> Um cuz the AIs aren't good enough yet.
48:52Like they're they're they're they're
48:53impressive, but they're not like
48:55they're not just a a drop-in replacement
48:58for a human worker in almost any field.
49:00>> And do you think that will be sudden?
49:02>> I think it'll be sudden because of the
49:05intelligence explosion dynamics or
49:07recursive self-improvement dynamics. So,
49:09you could imagine a different world
49:11where
49:12it's gradual.
49:13>> Mhm.
49:13>> And and this [clears throat] is this is
49:14maybe how it is in a lot of science
49:15fiction is,
49:17you know, the AIs gradually get better
49:19at a bunch of things and
49:20you know, they gradually automate like
49:22this one industry like pharma, then they
49:23automate like
49:25you know, steering drones, then they
49:27automate like driving cars or something
49:29like that. Um
49:31but what's different about the real
49:32world is that the companies have
49:35converged on this strategy of automating
49:37themselves first.
49:39You know, automating the AI research
49:40process.
49:41And so,
49:44if they are allowed to continue with the
49:45strategy,
49:47we're not going to see like,
49:49you know, the robot taxis and like the
49:52plumber robots and
49:54you know, the lawyer AIs. We're not
49:56going to see that sort of like broad
49:57diffusion of AI into the economy
49:59happening first because that's not what
50:02they're focusing on first. They're
50:03focusing on automating themselves,
50:05automating their own research so that
50:06they can do everything that they're
50:08doing faster.
50:09And they want that to sort of get going
50:11and get to
50:13you know,
50:14very high levels of intelligence, very
50:16high levels of general intelligence um
50:18and then deploy more out to the economy.
50:20economy. Right? So,
50:22by the time it's actually coming for
50:24like all these different jobs,
50:26they will have had fully autonomous AI
50:28research happening for months, maybe
50:31years, you know?
50:32And that means that like the AIs will be
50:34vastly superhuman at AI research and
50:37probably also vastly superhuman at lots
50:39of other things just as a side effect,
50:40you know?
50:42If you're wondering what this looks
50:43like, well,
50:44we wrote about what it looks like. It's
50:45sort of like this this wave smashing
50:48through the economy after they do the
50:50intelligence explosion internally.
50:52>> What I'm hearing there is that because
50:54the AI will be able to improve itself
50:56and train itself, it'll be getting
50:58better at everything at once and then
50:59it'll be released at kind of once.
51:02Is that accurate?
51:03>> it's it's not it's not even exactly that
51:05because even if it's mostly just getting
51:06better at the things that it's doing
51:07like research,
51:09that'll have some spillover effects
51:11to other skills as well.
51:13And then when it turns to the focusing
51:14on the those other skills, it'll be able
51:16to do them very fast.
51:17>> What jobs remain in such a scenario, do
51:19you think?
51:20>> I think that's actually a political
51:22question, not a technical question.
51:23>> Because
51:24>> Because on a technical level, all the
51:26jobs can be done by the AIs
51:29if they've reached that level.
51:30And so, it's a question of what jobs are
51:33allowed
51:34for them to do.
51:35>> And what kind of jobs wouldn't be
51:36allowed, do you think?
51:37>> That depends on who's in charge. So,
51:39there'd be some sort of political
51:40conversation about like what we're going
51:41to allow and disallow.
51:43>> I mean, in this scenario, the humans are
51:44still controlling them, the AIs.
51:46>> Depends on what you mean by control,
51:47right? So, there's like
51:49there's do the AIs actually have the
51:50goals and values that you want them to
51:52have, and are they going to robustly
51:54do that and behave as intended into the
51:56future? And then there's like are they
51:57obeying your orders for now?
51:59>> Are they obeying the orders is really
52:00what I'm saying.
52:01>> Yeah. So, like even in AI 24/7 in the
52:03scenario where the AIs take over and
52:04kill everyone, there's a period of like
52:06several years where they're still
52:07obeying orders,
52:09and they're, you know,
52:10taking some jobs but not other jobs, and
52:12they're helping to make better weapons
52:14that the US government can use to like
52:17do its arms race with China and so
52:18forth. And that's why they're able to
52:22get so much power so quickly is because
52:26the governments and the corporations and
52:27so forth trust them and is deliberately
52:30deploying them into all of these
52:32positions because it thinks that things
52:34are fine.
52:35But because these things are neural
52:37nets,
52:38you can't just like look inside and see
52:39what it's really thinking. You can't
52:41really tell.
52:42>> I think this is a really important point
52:43because unlike software where we can
52:45look at the code and see what's going
52:46on, theoretically, with AI you're saying
52:49that we don't know what why it's making
52:51the decisions that it's making cuz we
52:52can't get inside.
52:53>> One note of optimism is that it doesn't
52:55necessarily have to be that way. Like
52:57there's a a subfield of machine learning
52:59called mechanistic interpretability, and
53:01a a broader subfield called
53:02interpretability more generally that's
53:04trying to solve that problem and trying
53:06to take these these trained artificial
53:08neural nets and piece [snorts] them
53:10apart and understand
53:11like how the information is flowing and
53:13how the decisions are being made, so to
53:15speak. Um the problem is just it's a
53:17very inherently hard problem. If you
53:18have 10 trillion connections to look at,
53:21you can look at any particular group of
53:23them and be like, "Okay, so this is how
53:24like this particular connection works."
53:26But like how do you get a sense of the
53:28whole, you know? How do you get a sense
53:29of like
53:30what's happening at a high level? And
53:31the answer is, "Well, it might be
53:32impossible." But people are working on
53:34it and they are making progress, and
53:36if they can make enough progress, then
53:38we're in a very different and much
53:39brighter world. I think that it would be
53:42much less likely for us to get into
53:44those loss of control scenarios if we
53:46could just actually see what our AIs
53:47were thinking and why and how at any
53:50given time.
53:51Right?
53:51>> Yeah.
53:52>> So, we would still have the other
53:53problems to worry about, but at least we
53:55could mostly solve that one.
53:56>> It is pretty crazy to think that we're
53:57building a technology, a brain that we
53:59don't understand.
54:00>> Yeah, it's pretty crazy. I mean, it's
54:01one of those things where like
54:03>> In a movie, like a sci-fi movie, a bunch
54:05of scientists sit around this big brain
54:06and they're all just like they're
54:07they're making it more they're feeding
54:08it.
54:09>> Yeah.
54:09>> And they don't really know what the
54:10it is.
54:11>> Yeah, I mean, it's it's kind of just
54:12like obviously a dangerous thing to be
54:13doing.
54:14>> Yeah.
54:14>> Um but we're doing it anyway because of
54:16this history of how the field has
54:18developed in the last 10 years where
54:20you know, people were like, "Oh wow,
54:21yeah, that's obviously dangerous. Oh no,
54:23what if someone else did it and did a
54:24bad job of it? Therefore, we should do
54:26it and do a good job of it and now
54:29they're in this race where
54:30where they're racing each other and
54:32they're also under all sorts of
54:33political pressure to like pretend that
54:34it's not as bad as it seems because
54:36they don't want to like
54:38anger their investors, they don't want
54:39to anger the White House.
54:41>> One of the the key questions we had from
54:43our audience was which and I kind of
54:44asked you this in part, but which jobs
54:46are genuinely likely to survive AI and
54:49what skills should people {slash}
54:51students focus on over the next 10
54:53years?
54:53>> That's kind of like
54:55like imagine if you were someone living
54:57in Mexico
54:59in like 1500 and then you hear that like
55:03the conquistadors are coming.
55:05You could be asking yourself like,
55:06"Okay, well, what sort of job should I
55:08be switching to to like survive this
55:10transition?"
55:11But like, you have a lot more to worry
55:13about besides that. But yes, I think I
55:15would say that like if we managed to
55:17avoid the loss of control problem
55:19and we end up with humans still
55:21in charge of the AIs and humans can like
55:23say what the AIs goals and values are
55:25supposed to be even as they become much
55:27smarter than humans and even as they run
55:28the whole economy
55:30then probably there will be regulation
55:32that protects some areas
55:34and you can try to guess at what those
55:36areas might be. Maybe stuff that's more
55:37like
55:39like like judges potentially.
55:41>> What about podcasters?
55:44Be honest.
55:44>> Probably not podcasters, I think. Um
55:47stuff like
55:49you know, being a nanny
55:51maybe, right? Like I think that even if
55:53there's a robot nanny that's like really
55:55really good, I think a bunch of people
55:56might prefer to have an actual human
55:57because they might be creeped out by the
55:59idea of a really good robot nanny. So,
56:01you can sort of you can sort of reason
56:02like that. There's also like
56:05stuff that might be legally protected.
56:06Like maybe judges, for example, like are
56:08going to be legally required to be
56:09humans and not robots.
56:10>> Some people say though there's going to
56:12be so many jobs created that we can't
56:13foresee right now like there was in the
56:15industrial revolution or the internet
56:17boom or whatever.
56:18>> The problem with that is that
56:20um past technological advancements have
56:23been more narrow. They've like automated
56:25some things but not everything.
56:27But we are talking about a hypothetical
56:29future situation in which everything
56:31gets automated. So, there isn't any new
56:33job that you could do that AI couldn't
56:35also do.
56:37Except if it's like protected by
56:39regulation or something. That's that's
56:40that's also a thing. But so like for
56:43example, right now there's this sort of
56:44like cycle where
56:47you know
56:48the AI's learn to do a certain thing
56:50like write copy or like draft code or
56:54like debug something.
56:56And then humans who used to do that
56:57thing switch to managing AIs or switch
57:00to doing the other stuff that the AIs
57:01can't do.
57:03And that's why there's been this dynamic
57:04historically of
57:06you know, new jobs opening up and people
57:08flooding to them. But
57:10if it gets to the point where the AIs
57:11can do everything that humans can do and
57:13better and faster and cheaper, then
57:15whatever that new job is that you might
57:16have switched to, that the AIs can
57:17switch to that too and they'll already
57:18be be better at it than you.
57:21>> Because we haven't seen widespread
57:22unemployment yet in the economy, do you
57:24think people are getting a little bit
57:25complacent because what I'm seeing on my
57:26timeline is a lot of people saying I
57:28told you so, I told you everything would
57:29be fine. And when you look at the the US
57:32unemployment rate, currently the it's
57:34flat to slightly down. If you look at
57:36the UK, it is up. The trend is up
57:39compared to last year. We're at about 5%
57:41unemployment. The US is at 4.2%
57:43unemployment.
57:44>> Yeah. Basically, nobody has said that
57:46there would be mass unemployment by now.
57:48Or at least we didn't say that. You
57:49know, and we were historically one of
57:51the more bullish people on AI progress.
57:53In AI 2027, because of the dynamics that
57:55we just described, the mass unemployment
57:57doesn't happen until 2028 or 2029 after
57:59they already have superintelligence.
58:01Because, again, the companies aren't
58:02trying to cause mass unemployment as
58:04step one. That's like step three after
58:08you know, it's like step one, automate
58:09themselves.
58:10Step two,
58:12have this recursive self-improvement to
58:13get to superintelligence. Step three,
58:15expand out into the economy and automate
58:17everything. And so,
58:18this is really unfortunate from
58:20humanity's perspective, because one
58:22might have hoped that
58:24if there was this broad wave of
58:26automation going through the economy,
58:27people would sit up and pay attention
58:29and think about where all this is headed
58:31and demand good regulations from the
58:34government.
58:35But,
58:36that's not actually what the strategy of
58:37the companies are taking. You know,
58:39they're going to be getting the
58:39superintelligence first and then doing
58:41the broad wave of automation, which
58:42means that by the time they're actually
58:44doing all of that,
58:45uh well, it's already going to be moving
58:47very fast and the AIs will already be
58:49very powerful.
58:50>> In your 2027 report, so you wrote that
58:52in 2025, but it is called AI 2027, you
58:56said that in mid-2025 we'd have the
58:58autonomous employee, which is sort of
58:59like AI agents taking instructions over
59:01Slack or Teams.
59:04That happened. I've actually got an AI
59:06agent in my WhatsApp I can talk to. Of
59:07course, you've got Claude by exploded,
59:09obviously, around the world. And and
59:10now, um you know, Claude have talked
59:12about uh their new Slack integration.
59:14But, lots of people are using agents
59:15now. And that happened, I'd say for us
59:17at the We really sort of caught onto it
59:19at the the start of 2026.
59:21You also said by 2026 companies begin
59:24replacing entire corporate departments
59:25with AI agent subscriptions. 2027, the
59:28final job. AI automates the job of the
59:31human AI researchers themselves and
59:32begins the machine learning research to
59:34upgrade and build the next generation of
59:35AIs.
59:36>> Yeah, yeah. So, again, timelines.
59:39We are uncertain about how long it will
59:41take to achieve these milestones. In
59:42this scenario, they happen at those
59:44times, but
59:46by the time we had actually published
59:47this scenario, our timelines had shifted
59:49back a little bit. Specifically, mine
59:51had. So, like
59:53my 50% mark was 2028.
59:55>> Mhm.
59:55>> For that for the full automation of AI
59:57research milestone, not 2027.
59:59Uh
1:00:00and then other people on my team had
1:00:02more like 2030, 2031, things like that.
1:00:05So, I I I kind of want to like
1:00:07maybe try to illustrate this with the
1:00:08you know we have like this probability
1:00:09distribution. It's like a
1:00:11smeared out probability mass. And like
1:00:13the 50% mark is this particular year,
1:00:16but there's like a lot of possibility
1:00:17that it happens
1:00:18>> Later.
1:00:19>> years earlier or years later, right?
1:00:21>> Got you. What is this AI 2040?
1:00:24>> So, AI 2027 was our best guess
1:00:26prediction as to how things would
1:00:27actually go.
1:00:28>> Yeah.
1:00:28>> AI 2040 plan A is our recommendation for
1:00:31how things should go. So, we called it
1:00:34AI 2040 because in this scenario, uh
1:00:37they build superintelligence in 2040
1:00:39instead of much sooner because they
1:00:41delay things.
1:00:42>> Why do they delay things?
1:00:44>> To manage the risks and make sure that
1:00:46power is distributed equitably.
1:00:48They basically like
1:00:50regulate AI development so that it still
1:00:52continues, but at a slower, more
1:00:54reasonable pace uh in a more transparent
1:00:56and safe way
1:00:58and spread out over more countries and
1:00:59companies. And as a result, they get to
1:01:02superintelligence in 2040 instead of in
1:01:04say 2030.
1:01:06And then we call it plan A because
1:01:08well, it's our recommendation. Like
1:01:10we've we've come up with a plan for
1:01:12what government should do. And uh
1:01:15the scenario is an illustration of what
1:01:17it might look like to implement that
1:01:18plan. In a similar way to how AI 2027 is
1:01:20kind of an an illustration of what it
1:01:22would might look like
1:01:24to do with the companies are currently
1:01:25planning to do. If that makes sense.
1:01:27>> And is this wishful thinking or is this
1:01:29what you think is going to happen?
1:01:30>> No, it's definitely not what we think is
1:01:32going to happen.
1:01:33>> It's not what you think is going to
1:01:34happen?
1:01:34>> No, no, what we think is going to happen
1:01:35is still
1:01:36something more like this, right? We we
1:01:38don't expect the world to listen to us,
1:01:40right? This is our recommendation, but
1:01:43we we we hope that that people do
1:01:44something like this and we think it's
1:01:45possible, but it's not our like
1:01:48prediction for what's going to happen by
1:01:49default, you know.
1:01:51>> So, I do want to run through the plans,
1:01:53the potential plans, and also plan A,
1:01:55but um just to close off on how things
1:01:57might look after the year cuz I think I
1:01:58wanted to touch on robotics, too, and
1:02:00I've got this graph here which talks
1:02:02about share of labor output.
1:02:04>> Yes.
1:02:04>> Yeah.
1:02:04>> Um which I found to be quite striking.
1:02:06I've been sat here wondering as an
1:02:07employer who employs hundreds and
1:02:09hundreds of people
1:02:10when when all this stuff is going to
1:02:11happen. And you know, we're still hiring
1:02:13more people as things stand. There are
1:02:16some roles where our consideration is
1:02:18changing, shifting considerably.
1:02:21And I'd have to say that, you know,
1:02:22we're probably in the phase where our
1:02:23teams are AI-powered and they're using
1:02:25agents to do some of their work now.
1:02:27But I'm wondering as an employer like
1:02:29when is it
1:02:30when does this happen?
1:02:31>> Yeah, great question. So, if we could
1:02:33maybe zoom in on this a little bit.
1:02:35>> it on the screen.
1:02:36>> So, this is in the AI 2040 plan A
1:02:38scenario. And notably in that scenario,
1:02:41there's significant regulation
1:02:42introduced in 2029 that slows down the
1:02:45pace of AI development.
1:02:46In the scenario, they do that sort of at
1:02:48the last moment. So, in the scenario, if
1:02:51they hadn't done that, then it was about
1:02:52to take off similar to how it does in
1:02:54the AI 2027.
1:02:56Um but as you can see like in the
1:02:57scenario, there's still
1:02:59a bunch of jobs
1:03:02at the point that they implement it. And
1:03:04this gets back to what I was saying
1:03:04earlier is that if you wait until most
1:03:06people have lost their jobs
1:03:08to regulate the AI companies, that's
1:03:10already too late because
1:03:12they will probably already have super
1:03:14intelligent AI by then because their
1:03:16strategy is to first get super
1:03:17intelligent AI and then do all that
1:03:18stuff.
1:03:19>> think you say that it would collapse the
1:03:20economy and cause even more harm to
1:03:22suddenly regulate something that all of
1:03:23us and all of our lives were then at
1:03:24that point relying on.
1:03:26>> Oh, but it's a risk well worth taking. I
1:03:27mean, we It's true that right now a lot
1:03:30of people use AI for a lot of things,
1:03:31but like if we could somehow slow or
1:03:34halt AI development now to set up a
1:03:36better way to do it, that would be well
1:03:37worth it. Um even though there would be
1:03:39significant costs.
1:03:41>> But you can't over here, right? Can you?
1:03:42At this point where AI and robotics are
1:03:44doing most of the labor output.
1:03:46>> That's right. But in but in but in in
1:03:47this scenario, in the AI 2040 Plan A
1:03:49scenario, they put in the regulations in
1:03:512029.
1:03:52And then they slowly and carefully
1:03:54develop AI
1:03:56in a way that avoids all the problems,
1:03:58which we can get into in a little bit.
1:03:59And so eventually, yes, eventually the
1:04:01AIs take the jobs. Eventually
1:04:03basically the whole economy is run by
1:04:05AIs and robots, but it it happens
1:04:07gradually over the course of
1:04:09the 2030s instead of happening in this
1:04:11sort of crazy shock,
1:04:13you know, a year later.
1:04:15Right? Because in this scenario, they
1:04:17don't let the companies
1:04:19recursively self-improve and get to
1:04:21super intelligence as fast as possible.
1:04:23Instead, they regulate AI development so
1:04:25that the core capabilities of the AIs
1:04:27are improving at a more reasonable pace
1:04:29and also in a more transparent way so
1:04:32that the scientific community can see
1:04:34what's going on and help make it safe.
1:04:36>> But it's
1:04:37I guess I noticed here that in both your
1:04:39scenarios, eventually AI and robotics do
1:04:42pretty much all the jobs.
1:04:43>> Yes.
1:04:44>> So you kind of side there with Elon when
1:04:46Elon says that working will be a choice.
1:04:50>> Uh
1:04:53>> Because I mean we're going to have to
1:04:54>> I mean, if [laughter] it by definition
1:04:55if it can do all the things, then
1:04:57it can do all the things. I think that
1:05:00there's a question of like should we
1:05:01allow there to be AIs that can do all
1:05:02the things, right? Some people think
1:05:05that the answer is no and we should just
1:05:07shut it all down and prevent these types
1:05:09of AIs from being created in the first
1:05:10place. And we're actually kind of
1:05:13sympathetic to that. We we have our
1:05:15Should we bring out the plans diagram?
1:05:16>> Yeah.
1:05:18>> Thanks. Yeah. So,
1:05:21our scenario is called AI 2040 plan A.
1:05:24It's a scenario in which they slow down
1:05:25AI development to make a super
1:05:27intelligence happen in 2040 instead of
1:05:28earlier. And plan A is our
1:05:30recommendation. So, this is sort of
1:05:31illustrating our recommendation. But,
1:05:33for comparison, we made like mini
1:05:34scenarios illustrating different
1:05:36alternative plans, which we call plan S,
1:05:39plan B, plan C, and plan D.
1:05:41Plan D is basically
1:05:44the same thing that happens in AI 2027.
1:05:45Like, the race continues. There's very
1:05:47little regulation.
1:05:49Um you can read about that in AI 2027.
1:05:51Plan C also very similar to what happens
1:05:54in the slow down ending of AI 2027 where
1:05:55they solve the alignment problems. So,
1:05:57in that ending,
1:05:58they like slow down a little bit,
1:06:01pivot more resources to AI alignment and
1:06:03AI safety research,
1:06:05get lucky and succeed, and now they have
1:06:07aligned AIs,
1:06:08and then they speed up again and take
1:06:11all the jobs and beat China and all
1:06:12those things.
1:06:13Plan B is
1:06:16it's kind of like plan C in that
1:06:20well,
1:06:21basically in plan B, you're
1:06:23uh being more aggressive towards China
1:06:25and you're like
1:06:26taking actions to sabotage or cyber
1:06:28attack them to like keep them behind so
1:06:30that you have more breathing room to to
1:06:32solve the alignment problems yourself.
1:06:34Plan A is our recommendation. It's uh
1:06:37domestic regulation and then an
1:06:39international deal
1:06:40to continue building AI, but in a much
1:06:42better way.
1:06:43Plan S is shut it all down.
1:06:46If you want to have a future where
1:06:48there aren't AIs running around that can
1:06:50do everything better and faster than
1:06:52humans, you kind of want something like
1:06:54plan S. What What do you want?
1:06:56Plan A is our recommendation.
1:06:58I think that I'm sympathetic to plan S,
1:07:00but for reasons we explained, we
1:07:03recommend plan A instead.
1:07:04>> And And do you think is most probable?
1:07:06If you're being honest?
1:07:07>> Plan D.
1:07:08>> Which is that they just
1:07:09>> yeah, 24/7 type of thing where they keep
1:07:11racing. They don't really slow down
1:07:12significantly.
1:07:14Um
1:07:15and uh
1:07:16things happen extremely fast.
1:07:18The diagram sort of explains like
1:07:19roughly the reasoning behind this, too.
1:07:21So, like there's this high-level thing
1:07:22of like
1:07:24do you want to keep racing
1:07:26as fast as possible to make the AI
1:07:27smarter and smarter, to put them in
1:07:29charge of more things so that we can
1:07:30beat China?
1:07:31You know,
1:07:32if you're happy with that, then
1:07:35you get down and it says variation of
1:07:36happens here.
1:07:37If you are worried about that, well
1:07:41you get to something like this.
1:07:43There's more different options besides
1:07:44these, but this is kind of like the ones
1:07:46that we could compress onto a screen.
1:07:50>> Do you have children?
1:07:51>> Yeah, I have two children.
1:07:55It's kind of sad.
1:07:56Like
1:07:58I think that one way or another this
1:07:59will probably all be over by the time
1:08:01they're old enough to
1:08:02join the workforce.
1:08:05So, I don't think they'll ever join the
1:08:05workforce.
1:08:07>> When you say this will be all over by
1:08:08the time they join the What do you mean
1:08:09by this will be all over?
1:08:13>> So, these milestones that I described,
1:08:15like AIs automating the AI research, AIs
1:08:17getting super intelligent. Um
1:08:20AIs then exploding onto the economy,
1:08:23taking the jobs, building robot
1:08:24factories to build more robots to build
1:08:25more factories,
1:08:27etc. GDP starting to
1:08:29go vertical.
1:08:30That sort of thing is what I mean. Like
1:08:32all of those events transpiring.
1:08:34Maybe there's like you know, 10, 20%
1:08:35chance or something that
1:08:37hits the wall
1:08:38and and none of this comes to pass even
1:08:41if you don't do anything.
1:08:43>> How old is your oldest?
1:08:45>> Six.
1:08:45>> Six.
1:08:46Boy or girl?
1:08:47>> Girl.
1:08:48>> Girl. So, your daughter comes to you and
1:08:49says, "Dad, what should I um what should
1:08:51I study in school?"
1:08:52>> I mean, again, like if these radical
1:08:55transformations happen, then
1:08:57the world will just look completely
1:08:58different and
1:09:00what sort of jobs you set yourself up
1:09:01for basically, won't matter that much,
1:09:03probably. I would say um that the thing
1:09:06to do is
1:09:08well, A, try to make it actually go
1:09:09well. Like, if you can exert any
1:09:10influence at all on history and how this
1:09:12all develops, you should be trying very
1:09:14hard to steer the future in better
1:09:16directions.
1:09:17And then separately from that, on a
1:09:18personal level, you should focus on
1:09:21well,
1:09:23being a good person and doing things
1:09:25that are sort of good in their for their
1:09:26own sake, rather than good because
1:09:28they'll set you up for later employment
1:09:30because that later employment is going
1:09:31to be very uncertain um basically.
1:09:34>> Elon talks about this age of abundance
1:09:35we're heading towards.
1:09:37Age of abundance
1:09:38>> There'll definitely be abundance.
1:09:40The question is who controls the
1:09:42abundance?
1:09:43And what do they do with it?
1:09:45Right? Are the AIs controlled by anyone?
1:09:48Or are they doing their own thing?
1:09:49And then if they are controlled by
1:09:51people, who controls them? And what do
1:09:53they do? And what's the sort of like
1:09:55political structure governing how they
1:09:57make those decisions?
1:09:58>> I think it was Geoffrey Hinton that said
1:09:59to me, he said there's no example in
1:10:01nature where a more intelligent species
1:10:05is has less control than a less
1:10:09intelligent species. Thus saying that
1:10:12we're quite arrogant to think that in a
1:10:13world where there's this artificial
1:10:16brain that's a gazillion times the size
1:10:18of mine, that I'm going to give it
1:10:19orders.
1:10:20>> Yeah. I mean, that that's the thing is I
1:10:22I think it's like
1:10:24that should be our default assumption.
1:10:26Is that like, well, there's these
1:10:27brains, we can't see exactly what
1:10:29they're thinking. We're going to make
1:10:30them smarter than us and put them in
1:10:31charge of everything.
1:10:33>> And then we're going to give them
1:10:33bodies.
1:10:34>> Yeah. And then they're going to be
1:10:35autonomously building new factories and
1:10:36so forth. And like, how is this supposed
1:10:38to end well again? Like, isn't this just
1:10:40exactly like us picking a new species
1:10:43that's then going to outcompete us when
1:10:45it doesn't need us anymore? Like, I
1:10:47think that is just the default
1:10:48trajectory. Now, there's a whole
1:10:50argument we can get into about like ways
1:10:52that we could get off of that default
1:10:53trajectory. So, for example, there's
1:10:55research into interpretability that I
1:10:56described previously. And if that
1:10:58research bears fruit, then you will be
1:11:00able to actually see what they're
1:11:01thinking. And then that would be an
1:11:02excellent tool for shaping them and
1:11:04controlling them and making sure that
1:11:05they do what we want, right? There's
1:11:07other sorts of um
1:11:08AI alignment research agendas that are
1:11:11making progress. And if enough of those
1:11:13agendas succeed sufficiently, we can
1:11:15avoid this problem. Of course, also
1:11:17there's the regulatory side, too, where
1:11:18like part of what makes this difficult
1:11:20is that we're building these AIs in race
1:11:22conditions, you know? Like the the
1:11:24companies are secretive about their
1:11:26recipes for making these AIs because
1:11:28it's secrets that they want to protect
1:11:30so that other people can't copy them.
1:11:32And so a lot of this is happening, you
1:11:34know, behind closed doors. Only a few
1:11:35people can really see
1:11:37the recipes that they're using to train
1:11:39these AIs and and so forth. And then
1:11:41oftentimes when the AIs
1:11:43behave in unexpected ways or even just
1:11:44like blatantly misaligned ways,
1:11:46sometimes that information doesn't
1:11:47really flow out to the public because
1:11:49the companies are not really
1:11:50incentivized to tell everyone about how
1:11:52they messed up and how their AI is evil.
1:11:54It's just not very conducive to
1:11:55scientific progress on these issues. If
1:11:58the regulatory system was different,
1:11:59then perhaps we could be in a better
1:12:00situation, make faster progress. Also,
1:12:02of course, we wouldn't be planning to
1:12:05put these AIs in charge of everything as
1:12:06fast as possible. And we wouldn't be
1:12:08planning to like let them self-improve,
1:12:10you know? Like the these are choices
1:12:12that we could not make, you know?
1:12:17>> I don't speak Vietnamese, but this show
1:12:19can because of AI video technology from
1:12:21our sponsor, HeyGen. I get messages
1:12:23every single week from those of you
1:12:24listening to the Current CEO all around
1:12:26the world, and you express how much
1:12:28impact it's had on you and your life.
1:12:29And if that's true, then those
1:12:31conversations shouldn't only reach
1:12:32people in English. HeyGen can take one
1:12:34recording of me and deliver in any
1:12:36language while keeping my voice, timing,
1:12:38and expressions intact. But you don't
1:12:41need a studio like this to make it work
1:12:43for you. Record 15 seconds of yourself
1:12:46and get an AI avatar that delivers
1:12:48studio-quality video in over 175
1:12:51languages. We're up to 20 languages now,
1:12:54and we're not the only ones using it.
1:12:56HeyGen is already used by 30 million
1:12:58people, including 85% of the Fortune
1:13:01100. Whether you're building an audience
1:13:02on social media, launching an online
1:13:04course, or rolling out training across
1:13:05your team, check out HeyGen now. Your
1:13:08first three videos are totally free at
1:13:10heygen.com/doac.
1:13:13That's h e y g e n dot com slash d o a
1:13:16c. See you there.
1:13:19>> Ilya was As you said, he was one of the
1:13:20leaders at OpenAI, and he left and he
1:13:22started his own company now, Safe
1:13:23Superintelligence.
1:13:25Very curious name of a company, Safe
1:13:27Superintelligence, after leaving OpenAI.
1:13:29Did you ever get to work with him?
1:13:30>> Uh I wasn't directly working with him. I
1:13:31had a couple chats with him.
1:13:33>> Do you think he's he's genuinely
1:13:35concerned as well?
1:13:36>> I think he is, but I think it's I think
1:13:39he's similar to these other CEOs, where
1:13:43I mean, just think about the sort of
1:13:44incentives that they're under, right?
1:13:45Like
1:13:47they can sort of see the problem,
1:13:49and then they can
1:13:51be like, okay, but like if I don't if I
1:13:52stop, if I quit my job, and or do
1:13:55something else,
1:13:56that's not going to solve the problem,
1:13:57cuz the other CEOs are going to keep
1:13:59going.
1:14:00And even if all of us didn't go, then
1:14:01maybe China would keep going. So, like,
1:14:03man, it seems like this is just going to
1:14:04happen one way or another, whether I do
1:14:05anything about it or not.
1:14:08I guess I should be involved, you know,
1:14:09and like maybe I can make it go well,
1:14:11and at any rate, like I don't want to be
1:14:12out in the cold while these other people
1:14:14I don't trust are in charge of
1:14:15everything. So, they all sort of like
1:14:16reason through all of this and then
1:14:17convince themselves that like the thing
1:14:19to do is for them
1:14:20>> to build their AI.
1:14:21>> build it and to do it better. And I
1:14:22think Ilya's just the latest example of
1:14:25this. Elon's another example. Dario's
1:14:28another example.
1:14:29You know, arguably OpenAI at the
1:14:30beginning, Sam was an example, although
1:14:32like Elon and Dario were at OpenAI early
1:14:34on, so
1:14:35>> What do you think they should all do
1:14:36then?
1:14:37>> So, I think what should happen is some
1:14:38sort of international regulation, or at
1:14:40least domestic regulation, similar to
1:14:42what we described in plan A.
1:14:43>> Okay, so talk me through plan A.
1:14:45>> Yeah.
1:14:46So, in this scenario
1:14:48AI takes longer to the to get to
1:14:51recursive self-improvement and full
1:14:52automation of AI research than it does
1:14:54in 2027. We figured that we should try
1:14:56to illustrate like a range of different
1:14:57possibilities because we do have those
1:14:59sort of uncertainty intervals. So, we
1:15:01chose 2030 as the
1:15:04moment when full automation would
1:15:06finally be achieved and things would
1:15:07really kick off.
1:15:08And then working backwards from that
1:15:11when's the last moment you could really
1:15:12have good regulation? 2029. So, in this
1:15:15scenario
1:15:16AI progress slows down a little bit
1:15:17naturally and the AI companies keep keep
1:15:20racing, but they don't quite succeed in
1:15:22automating uh themselves in 2027 or in
1:15:252028 or in 2029, but they're getting
1:15:27really close and they're going to do it
1:15:28in 2030.
1:15:30And then in 2029, the government steps
1:15:31in and regulates them. What regulations
1:15:34do they do? Well, they basically just
1:15:35shut it down temporarily.
1:15:37>> Can I ask um
1:15:39how does the elections overlay with your
1:15:42time frames here? Because there's going
1:15:43to be a big election, isn't there in
1:15:442028?
1:15:46And it seems now that sentiment has
1:15:47really really turned against AI in in
1:15:49sort of in the general public and that
1:15:51it will be one of the big ticket items
1:15:53on the on the ballot.
1:15:54>> We think that it'll be maybe the most
1:15:55important issue in the presidential
1:15:57election in 2028. Um I think a lot of
1:16:00people most people will be quite
1:16:02concerned about where things are headed
1:16:03and that's part of why we we chose
1:16:06to depict things the way they were doing
1:16:08in this scenario because that helps
1:16:09explain why they might do this sort of
1:16:10regulation in 2029 is that the voters
1:16:13have been demanding it and the
1:16:14presidential candidates have been
1:16:14promising it.
1:16:15>> And in this scenario and then 2027,
1:16:18would the general public have felt the
1:16:19consequences of AI much more severely
1:16:21than they have now by then?
1:16:23>> Yes.
1:16:24Although still even in 2029 in this
1:16:26scenario, they still mostly have the
1:16:27jobs as as depicted here, right? So, in
1:16:29in 2029 in this scenario, lots of jobs
1:16:32now involve managing AI agents.
1:16:34You you mentioned you have an AI agent,
1:16:36right? Well, in 2029 in this scenario,
1:16:38the AI agents will be much better. Still
1:16:40though, not enough to just completely do
1:16:42everything. You know, that was the sort
1:16:44of thing that would come in 2030
1:16:45in this in this timeline. Again, we're
1:16:47uncertain about timelines.
1:16:49Things could go faster than depicted in
1:16:50this scenario, and in fact, I think
1:16:51things probably will go a bit faster
1:16:53than depicted in this scenario, but
1:16:55we're uncertain. We already did the very
1:16:56fast timeline scenario, so now we're
1:16:57doing the slower timeline scenario. But,
1:16:59maybe we should talk about the
1:17:00high-level goals. So,
1:17:03they want to have AI continue, but in a
1:17:05slower pace so that they can make it
1:17:07safe.
1:17:07>> The politicians, you know, the president
1:17:09and the people who voted for the
1:17:11president and, you know, the heads of
1:17:13other governments and so forth. So, goal
1:17:15one, slow things down.
1:17:17Um goal two, make it more transparent
1:17:20so that the scientific community can
1:17:22catch up to this stuff and make more
1:17:23progress. And also, so that we don't
1:17:24have to take the company's word for it
1:17:26when they say that their systems are
1:17:27safe and when they say that they
1:17:28haven't, you know,
1:17:30put in any biases into their systems,
1:17:32for example. That's a constitutional
1:17:33power issue. We also want to avoid a
1:17:36situation where there's an intense
1:17:37concentration of power. So, in addition
1:17:39to these
1:17:40the transparency and the slowdown,
1:17:43we actually think it's actively good for
1:17:44there to be multiple AI companies
1:17:46across multiple different countries that
1:17:48have similar levels of very advanced AI
1:17:50capability and for there to be like
1:17:53broad diffusion of AI into society
1:17:56rather than, you know, a single mega
1:17:57project that has all the best AIs, for
1:17:59example. And the another thing about
1:18:01that is you kind of get that by default
1:18:02if you do the first two things. If you
1:18:04slow it down and if you make it more
1:18:05transparent, then that means there's
1:18:07breathing room
1:18:08for other projects to sort of catch up,
1:18:11right? And the transparency just like
1:18:12literally helps them catch up because
1:18:14then they can like copy
1:18:15copy some of the ideas. And then I think
1:18:17the fourth thing would be reversibility.
1:18:19So, in what follows in the scenario, we
1:18:22are going to be building up a lot of
1:18:23data centers, a lot of robots. We're
1:18:25going to be transforming the world at a
1:18:27at a sort of like slower pace, though
1:18:29still a very fast pace, but slower. And
1:18:32if things go wrong and the deal breaks
1:18:34down and everyone starts racing each
1:18:35other again to get to super intelligence
1:18:37as fast as possible.
1:18:39That would be very scary. And so, the
1:18:41fourth principle is basically build the
1:18:44new data centers in such a way that if
1:18:46everything
1:18:47breaks down and everyone starts racing
1:18:48again, the newly built data centers get
1:18:50destroyed so that we're sort of back to
1:18:52square one again instead of in an even
1:18:54worse race where there's even more AIs
1:18:56and robots and compute everywhere. Um
1:18:59So, I can sort of walk you through the
1:19:00timeline if you're interested. Sure. Or
1:19:02the president talks to China, talks to
1:19:04the leaders of a bunch of other
1:19:05countries
1:19:06and says
1:19:07we're going to basically
1:19:09halt AI development until we can figure
1:19:10out a a plan for how to do it in the way
1:19:12in the ways that achieve these goals.
1:19:14So, they basically send inspectors to
1:19:17each other's data centers. Like Chinese
1:19:19inspectors come to US data centers, US
1:19:20inspectors go to Chinese data centers
1:19:22and verify that they are doing inference
1:19:24and not training. Developing new AIs,
1:19:27that's that involves training them. But,
1:19:30just taking existing AIs and using them
1:19:32to serve customers, that's called
1:19:34inference.
1:19:35And so, the sort of like solution they
1:19:37come up with here in this scenario is
1:19:39we'll allow them to keep doing inference
1:19:41but not training for now until we can
1:19:43get the new training data centers set
1:19:45up. So, they retrofit the existing data
1:19:48centers to serve inference. People can
1:19:50still keep talking to their AI agents
1:19:52but they're going to stop getting better
1:19:54and better
1:19:55for like 6 months to a year while they
1:19:57build the new data centers that are
1:19:59going to be the transparent data
1:20:00centers. And that's where the training's
1:20:01going to happen.
1:20:03Once they get those new data centers set
1:20:04up in 2030,
1:20:06then AI research continues. This is a
1:20:08bit spicy. We advocate for total
1:20:10research transparency, which means that
1:20:12on the training data centers that are
1:20:13training the new models,
1:20:15they basically have to publish
1:20:16everything.
1:20:17Which means you get to see all the
1:20:18details of the recipes for training
1:20:19these models. You get to see the
1:20:20architectures, etc. We think that's sort
1:20:23of open science is really important for
1:20:25solving the alignment problem fast
1:20:27enough because you don't want to have to
1:20:28sort of biased companies making the
1:20:30decisions about whether the AIs are
1:20:32safe. Um and we also think it's
1:20:34important for just good regulations more
1:20:36generally because right now most of the
1:20:38expertise in the world on AI is sort of
1:20:40concentrated in Silicon Valley and the
1:20:42the governments in particular kind of
1:20:45are don't really understand AI that well
1:20:47and imagine an alternative instead of
1:20:49total research transparency you had like
1:20:51an auditor system where the government
1:20:53says here are some rules for how to make
1:20:56the AI safe
1:20:57and then we're going to have like an
1:20:58agency that like goes into the companies
1:21:01and ask them questions and tries to make
1:21:02sure that they're following the rules.
1:21:03That creates this sort of adversarial
1:21:05dynamic where the company is
1:21:06incentivized to like fool the the
1:21:09regulator, you [clears throat] know, and
1:21:10and also if they if they discover some
1:21:12new problem that's not even on the
1:21:14government's radar
1:21:15they might be incentivized to like not
1:21:16tell the government about it, right? So
1:21:18if you have the total transparency it
1:21:19helps the government make better
1:21:20decisions faster.
1:21:22>> But it kills that competitive advantage.
1:21:24>> Yes. Prophetic's not going to like this,
1:21:26you know, OpenAI's not going to like
1:21:27this. This would be
1:21:29probably bad for the valuations. I don't
1:21:31think it would kill them completely but
1:21:33it means that it would commoditize more,
1:21:35right? So it means that there'd be like
1:21:37a bunch of AI companies that would catch
1:21:38up to the frontier, they would train AIs
1:21:40that are like roughly similar, roughly
1:21:42equivalent. They could still make money
1:21:44by doing that and then selling their AIs
1:21:46but they wouldn't have a monopoly, they
1:21:48wouldn't have anything close to a
1:21:49monopoly which I think is good for
1:21:50humanity although it's bad for the
1:21:52bottom line of those particular
1:21:53companies. Notably it's good for the
1:21:55bottom line of lots of other companies.
1:21:56Like if you're a company that's behind
1:21:58and you don't you're not Anthropic or
1:22:00you're not OpenAI then you would love
1:22:02this because this helps you catch up,
1:22:04you know, or this this helps you to like
1:22:06um capture more of the value from the
1:22:08chips you're selling for example or from
1:22:09the like downstream product that you're
1:22:11making.
1:22:11>> And by 2031 then you have 1/5 of all
1:22:15cognitive labor done by AI.
1:22:17>> Yeah, so what's happening here is that
1:22:19we're imagining that the government of
1:22:20the United States and the government of
1:22:22these other countries that are involved
1:22:23in this agreement that are sort of
1:22:24implementing similar regulations
1:22:26um they don't have to be exactly the
1:22:27same,
1:22:28uh, but that's another thing that's nice
1:22:30about the transparency is that if you
1:22:31have this sort of transparency, then
1:22:34if two governments are
1:22:36implementing different regulations, like
1:22:38if one of them is like
1:22:39telling their companies to go slower or
1:22:41like banning more stuff than the other
1:22:43one is, they can both see
1:22:45>> Yeah.
1:22:45>> like, "Oh, you're letting them do that
1:22:46sort of thing?
1:22:47And you're not? Like, maybe we should
1:22:49let them do this, too, you know?" So, it
1:22:51helps to sort of naturally equalize the
1:22:53regulations to some extent without
1:22:56having there to be a central power that
1:22:57just gets to make regulations for
1:22:59everybody.
1:22:59>> Mhm.
1:23:00>> So, anyhow, we're imagining that when
1:23:01they when they get this transparency set
1:23:03up, they basically agree to ban the
1:23:05dangerous stuff, to allow the
1:23:07not-so-dangerous stuff, and there's a
1:23:08constant ongoing conversation about
1:23:10like, "Well, what's dangerous and what's
1:23:11not? What should we ban? What should we
1:23:12allow? What about this country? What
1:23:14about that country?" That conversation
1:23:16evolves over time, but the gist of it
1:23:17is, at least if they do it the way that
1:23:19we recommend it, is that they don't do
1:23:21an intelligence explosion. They don't
1:23:22let the AIs, you know, autonomously
1:23:24self-improve. Instead,
1:23:26they slowly and carefully scale up the
1:23:29AIs that they currently have, and invest
1:23:31lots into finding ways to make them more
1:23:33interpretable,
1:23:34uh, to make them more easy to control,
1:23:36to understand better how they work, and
1:23:37so forth. The result is that AI progress
1:23:39continues, but it's
1:23:41not quite as fast,
1:23:43and it's much, much, much safer and more
1:23:45transparent.
1:23:45>> But still through these, you know, are
1:23:47we seeing job disruptions?
1:23:48>> continuing cuz they are building more
1:23:49data centers, right? Like, this whole
1:23:51time, they're building more and more
1:23:53data centers, more and more chips, and
1:23:55they're continuing to like
1:23:57make there be a a larger and larger
1:23:59population of AIs, so to speak, and that
1:24:01causes this huge transformation over the
1:24:04course of the 2030s. So, the big thing
1:24:05that we sort of want people to take away
1:24:07is that even if you heavily restrict AI
1:24:09progress,
1:24:11you still get this sort of crazy
1:24:12transformation. Yeah, in this scenario,
1:24:14they basically
1:24:16allow progress to continue, but at a
1:24:17slower, more safe pace here in 2030,
1:24:20and then it as a result, it takes until
1:24:222035
1:24:24to get to top expert level AI. So,
1:24:26remember they were on track to do that
1:24:28in 2030, but then sort of at the last
1:24:30moment they stopped. But because they
1:24:32were sort of so close to the last
1:24:33moment, that means that like they can
1:24:35sort of get there pretty soon if they
1:24:36want to, and it's just a matter of like
1:24:38how long they they allow it to go,
1:24:40right? So, they sort of they sort of
1:24:42slow it down, and spread it out,
1:24:44leisurely arrive at this level after 5
1:24:46years. By this point they've built up
1:24:49massive amounts of data centers
1:24:50everywhere. So, it's not just that the
1:24:51AIs are smarter and able to do all the
1:24:54things that humans can do, but also
1:24:55there's a lot more of them. And there's
1:24:57a lot of robots and so forth. So, by
1:24:59this by this point you kind of have the
1:25:02economy that a lot of people would have
1:25:03imagined with AGI, where there's AIs,
1:25:06there's lots of them, they're able to do
1:25:07all sorts of jobs, there's robots,
1:25:09there's lots of them, they're able to do
1:25:10all sorts of physical work, and
1:25:12basically the economy is being run by
1:25:14these machines.
1:25:15>> So, in 20
1:25:1631, you you have the 1/5 of all
1:25:19cognitive labor done by AI. In 2023, you
1:25:21have 60 million AIs running at 100x
1:25:24speed. In 2033,
1:25:27there's cash dividends to all Americans.
1:25:29>> Mhm.
1:25:30>> Um I've got to
1:25:32explain explain this to me.
1:25:35>> Yeah, so if the AIs are going to be
1:25:37taking people's jobs, then it's very
1:25:39important that people not starve to
1:25:40death, and still have money.
1:25:43And if
1:25:45companies are going to be using AIs and
1:25:46robots to take all these jobs, then that
1:25:48means that there needs to be some sort
1:25:49of taxation scheme, or something, to
1:25:51like
1:25:52make sure that people still have a a
1:25:54slice of that pie. Mhm. The pie is going
1:25:56to grow huge, but you still need to
1:25:57actually give people a slice of the pie.
1:25:59And our proposal for how to do that, we
1:26:01call it the citizens dividend, basically
1:26:04people have shares in a agency that
1:26:07sells permits to the robot companies,
1:26:10and to the compute companies,
1:26:12and makes profit from selling those
1:26:14permits, and then those are people have
1:26:17shares in that entity. It starts off
1:26:19small. It starts off something like
1:26:20$25,000 per person.
1:26:22Uh and then by the end, it's something
1:26:24like $10 million
1:26:25per citizen.
1:26:26>> per person?
1:26:27>> Per person per year.
1:26:29>> Factoring in inflation, like what you
1:26:30mean?
1:26:31>> in inflation.
1:26:31>> So, we're going to be
1:26:32multi-millionaires.
1:26:33>> Yes, if this happens, which it probably
1:26:36won't, but if it happens, this is where
1:26:37it will go. And again, this is the thing
1:26:39I want to emphasize is that if you get
1:26:40to the point where your AIs are close to
1:26:42being able to do
1:26:43all the research, and then you sort of
1:26:45pause and slow down,
1:26:47that means that like you still have a
1:26:49lot of transformation ahead of you
1:26:50because if you allow those AIs to like
1:26:52still proceed slowly and like start to
1:26:54automate various jobs and so forth,
1:26:56after some years, they will in fact have
1:26:58done that. And
1:26:59they will have, you know, built huge
1:27:01amounts of new data centers, huge
1:27:02amounts of new chip fabs, huge amounts
1:27:04of new robots, robot factories, etc.
1:27:06You know, we're not sure obviously how
1:27:08fast this will go exactly, but we've
1:27:10thought about it a lot and we have our
1:27:11our guesses and this is sort of like our
1:27:12median guess.
1:27:13>> What does this mean, 2037? The
1:27:15apocalyptic arrival of truth on Earth?
1:27:18>> Yeah, so like
1:27:19this is the point where we say they get
1:27:20to top expert level AI. So,
1:27:23it's not super intelligence in the sense
1:27:25that it's not like vastly smarter than
1:27:26humans at things because they
1:27:28deliberately pause it at the level of
1:27:30top experts. So, so here they're going
1:27:32slow. Here they've just actually
1:27:33stopped.
1:27:35But they stopped at a point where the
1:27:36AIs are just actually really good at
1:27:37everything. So, kind of they've
1:27:39definitely got AGI, maybe they got like
1:27:41weak super intelligence.
1:27:43Because they have so many these AIs and
1:27:45because they think faster than humans,
1:27:47you know, they just run much faster,
1:27:49that's going to transform society
1:27:50dramatically. So,
1:27:53we talk about some of the ways in which
1:27:54it transforms society. Like this is sort
1:27:55of life after work. We talk about what
1:27:57it would be like to be living on your
1:27:58citizens citizens dividend and not have
1:28:00a job anymore in this sort of world. Um
1:28:03here we talk about all the scientific
1:28:04changes and all the social changes that
1:28:06would come from all of the
1:28:09intellectual progress and activity that
1:28:10would be generated by all of these AIs.
1:28:13So,
1:28:14for example, here is things like cancer
1:28:16cures and like, you know, people living
1:28:18in apartments that were built by robots
1:28:202 years ago.
1:28:22>> Mhm.
1:28:23>> Providing again we stop in 2029.
1:28:25>> Yeah.
1:28:26>> And providing, I mean, a conservative
1:28:27This is a conservative time frame.
1:28:29>> Yeah, like unfortunately, I actually
1:28:31think that things will happen faster
1:28:32than this by default and that if we
1:28:34don't slow down, things will happen much
1:28:35faster than this. Once you get to the
1:28:37point where you've got, you know, a
1:28:38billion AIs running day and night and
1:28:42they're each better than the best humans
1:28:43at everything and so they're doing a lot
1:28:45of science, they're doing a lot of
1:28:47talking to each other, they're doing a
1:28:48lot of thinking, everyone's constantly
1:28:50talking to their AI assistants and so
1:28:51forth.
1:28:52There's going to be a lot of scientific
1:28:53progress. There's going to be a lot of
1:28:54changes to politics, to ideologies. It's
1:28:58going to be very disruptive and crazy
1:29:00and we get into some of the ways in
1:29:02which it is
1:29:03uh later, basically.
1:29:04>> I I'm still not super clear on what this
1:29:06means, the apocalyptic arrival of truth
1:29:08on Earth.
1:29:09It's just It's just because there's so
1:29:10many eyes AIs that are so smart that
1:29:12they're uncovering making new
1:29:13discoveries in sciences.
1:29:15>> Let me give you an example, lie
1:29:16detectors.
1:29:16>> Yeah.
1:29:17>> So,
1:29:18that's an example of a a technology that
1:29:20might be invented.
1:29:21>> Yeah.
1:29:21>> You know, right now we don't have good
1:29:22lie detectors, we have very bad lie
1:29:24detectors that like sort of work but
1:29:25don't don't fully work. But once you've
1:29:28had these top expert level AIs thinking
1:29:31for many years at you know, 100x human
1:29:33speed and there's billions of them and
1:29:35they have access to robot factories to
1:29:36do research and stuff,
1:29:38they'll probably invent a ton of
1:29:39technologies. Maybe they'll invent lie
1:29:40detectors that actually work on real
1:29:42humans.
1:29:43That'll have big social effects, right?
1:29:45Imagine a presidential candidate who's
1:29:46like, "Those allegations are false
1:29:49and to prove them, I will go under a lie
1:29:50detector and say that they're false."
1:29:52>> I was just thinking about the whole like
1:29:54justice system and
1:29:55how that would be overturned. Um in
1:29:57fact, you could, you know, theoretically
1:29:59walk down the street and be
1:30:01Yeah.
1:30:02>> It's both
1:30:03terrifying and exciting.
1:30:05One thing that we talk about in this
1:30:06sec- in this section like the invention
1:30:08of lie detectors could be really bad.
1:30:10Like it could be that it enables a new
1:30:11form of totalitarianism where the
1:30:14powerful people, you know, the CEOs and
1:30:15the politicians
1:30:17force the people under them to go under
1:30:19lie detectors and say like yes, I'm
1:30:20loyal to the dear leader. I would never
1:30:22do anything against the dear leader,
1:30:23right?
1:30:24>> you're lying then you're in
1:30:25>> And then if you're lying you get fired,
1:30:26right? So like there's there's a ton of
1:30:27like very harmful uses of lie detector
1:30:29technology. There's also the good uses
1:30:31and broadly speaking I would say the
1:30:33good uses are when lie detectors are
1:30:35used on the powerful instead of by the
1:30:37powerful.
1:30:37>> What's this? 2040 passing the torch to
1:30:40AIs.
1:30:41>> Yeah, great. So
1:30:42here they pause at the top expert AI
1:30:44level. And the reason why they pause is
1:30:46because
1:30:47their safety cases aren't good enough
1:30:49for going beyond that level. Um so in
1:30:51the sort of regulatory systems that they
1:30:53set up over the course of these years,
1:30:55roughly speaking the way they would work
1:30:57is when you're making a new AI and then
1:30:59when you're trying to deploy the AI into
1:31:01something, you have to have some sort of
1:31:03safety case explaining like
1:31:05what your intentions are and like why
1:31:07you think it's going to work the way
1:31:08that you want it to work. And in
1:31:09particular why the AI is going to like
1:31:12do as it's told, for example, and why
1:31:14nothing super terrible's going to happen
1:31:15like AI takeover.
1:31:17It's relatively easy to make safety
1:31:18cases like this when your AIs are still
1:31:21not capable of automating everything.
1:31:24But the more powerful they get, the more
1:31:26difficult it is to actually argue that
1:31:28things are going to be fine because the
1:31:29AIs are just more capable and they can
1:31:31they can get up to more stuff. And if
1:31:32you if they're actually untrustworthy,
1:31:34the the possible downsides are bigger.
1:31:36So that's why they stop at this level is
1:31:38that they they realize that if they keep
1:31:40going then they might actually lose
1:31:41control of everything. But at the
1:31:43current level they're convinced by
1:31:45safety cases that it's fine. But then
1:31:47they don't want to go further. So they
1:31:48stop there.
1:31:49And then what happens in 2040 is they've
1:31:51made significant progress scientifically
1:31:54including on alignment and they figured
1:31:56out how to make AIs that are actually
1:31:57aligned in a robust way.
1:31:59>> With humans?
1:32:00>> With humans. So they can actually trust
1:32:02those AIs and they can allow them to
1:32:03become much smarter again. So, that's
1:32:05why we call the whole thing AI 2040 cuz
1:32:07in 2040 they sort of let off the brakes
1:32:11and allow the AIs to become
1:32:13significantly smarter than humans.
1:32:14>> I guess you know, this is a this is a
1:32:16plan and this is a hope.
1:32:18>> Yes.
1:32:20>> But in reality, this is not what you
1:32:21think probabilistically if you had to
1:32:24>> That's right. It's important to
1:32:25distinguish like this is what we
1:32:26recommend. This is what we want to
1:32:27happen from like this is what we
1:32:30actually think will happen by default.
1:32:32Now, we do think it's possible for this
1:32:33to happen, but you know, that will
1:32:35require a lot of people to sort of wake
1:32:36up and pay more attention and advocate
1:32:40for something like this to happen. So,
1:32:41our main scenario is mostly talking
1:32:44about the policy choices made and the
1:32:46broad scale effects on society. We
1:32:48figured it would also be nice to
1:32:49accompany this with a little mini
1:32:51scenario that describes what it would
1:32:53actually feel like to live through this
1:32:56from an ordinary person's perspective.
1:32:57>> Okay.
1:32:58>> Um 2029, everyone's yelling at each
1:33:00other, the presidents are negotiating
1:33:01something and they've paused AI, but you
1:33:04still have access to the existing AIs,
1:33:06so it doesn't really feel that different
1:33:07although it definitely is like something
1:33:09exciting happening. 2031, they've
1:33:11started progress again, the AIs are
1:33:12really smart, more people have lost
1:33:14their jobs, it's like really starting to
1:33:15actually affect things, but I think
1:33:16still most people have their jobs, but
1:33:18their jobs are sort of transformed. So,
1:33:19like by 2031 it's like
1:33:21most white collar jobs involve working
1:33:23with AIs to a large extent or managing
1:33:25teams of AIs or collaborating with them
1:33:27somehow.
1:33:27>> And what was
1:33:28>> Also, there are some things like robo
1:33:29taxis that are basically just working.
1:33:31Citizens dividend, you know, ideally
1:33:33this would happen sooner. Like in our
1:33:34scenario, they kind of do things at the
1:33:36last minute.
1:33:37You know, so like a lot of these policy
1:33:39things are like happening kind of like
1:33:41just in time. Obviously, we would
1:33:42recommend that you do them sooner and
1:33:44and do a better job of them, too. But
1:33:46so, 2033, you start getting your your
1:33:48checks from your dividend.
1:33:49>> So, you're forecasting that there will
1:33:51be a citizen's check. The your model
1:33:53says it could be around 25,000 at the
1:33:55start per person.
1:33:56>> And then it would grow as the economy
1:33:57grows.
1:33:58>> But also as like as job displacement
1:34:00takes hold, they're going to need to to
1:34:01grow that check and make sure you can
1:34:02>> And that's why it's kind of the last
1:34:03possible moment because if you waited to
1:34:05implement this until like 2037, then
1:34:08like everyone would have already lost
1:34:09their jobs by the time that happens,
1:34:11right?
1:34:12>> People losing their jobs, especially if
1:34:14it happens
1:34:16quickly like like we see on this sort of
1:34:17graph here,
1:34:18is going to cause lots of problems in
1:34:20terms of civil unrest, social unrest,
1:34:21purpose, mental health, these kinds of
1:34:23things theoretically.
1:34:25>> Yes.
1:34:26>> How do you think about that?
1:34:27>> Uh it's it's going to be rough and
1:34:29hopefully we can navigate that well. We
1:34:31think that at a high level, people need
1:34:33to have money
1:34:34and also people need to have power. And
1:34:36I think these are like somewhat
1:34:37different things. It's like why are jobs
1:34:39important? Well, there's a lot of
1:34:40reasons why jobs are important, but I
1:34:41think the main ones are
1:34:42um well, it's how people get money so so
1:34:44they can survive and get things that
1:34:45they want by buying the things that they
1:34:46want. So if people are going to be
1:34:48losing their jobs, you need some other
1:34:49way of people getting money.
1:34:51And then there's also the power thing,
1:34:52which is that right now people have
1:34:55political power in part due to their
1:34:57economic power. People can threaten to
1:34:59go on strike, for example, or you know,
1:35:01countries that are ruled by dictators
1:35:04can't
1:35:06just completely,
1:35:07you know, genocide an entire
1:35:09subpopulation, or they can, but like
1:35:11it's costly for them to do so because
1:35:14then they'll have less money because
1:35:15that subpopulation is contributing to
1:35:16their economy and contributing tax
1:35:18revenue and so forth. But if you end up
1:35:20in a world where actually nobody's
1:35:21contributing tax revenue revenue except
1:35:23for the AI companies and the robot
1:35:25companies, then you're you, the
1:35:26government, are less incentivized to
1:35:29care about what, you know, the common
1:35:31people think. So so
1:35:32when people lose their jobs, they're not
1:35:34just threatened with lack of loss of
1:35:36income, they're also threatened with
1:35:37loss of political power.
1:35:39And so we think that it's important to
1:35:41like do things to push against that.
1:35:43>> What does that look like? How do you How
1:35:45do people have power in such a world?
1:35:47>> Well, in democracies at least they still
1:35:48have votes.
1:35:49>> Okay.
1:35:50>> So I think that it's very important for
1:35:52there to be uh regulations on the use of
1:35:56AI that help make
1:35:59the public discourse more sane
1:36:02and more
1:36:03um
1:36:04actually giving the people what is in
1:36:05their interest and what they want and
1:36:07avoiding a sort of um opposite outcome
1:36:10where
1:36:11you know, the masses are easily
1:36:14manipulated by AI-powered media, for
1:36:16example. Or where everyone's talking all
1:36:19day to their AI advisers, and the AI
1:36:21advisers are like subtly steering them
1:36:24away from voting for the candidate that
1:36:27would
1:36:28not be what the AI companies want
1:36:29because the AI companies have this other
1:36:32candidate that they like better, and
1:36:33they're like secretly biasing their AIs
1:36:35to like steer people towards voting for
1:36:36that candidate, right? So, so we want to
1:36:38be in a situation where
1:36:40um
1:36:41people have AIs that are actually
1:36:43trustworthy and that are truth-seeking
1:36:45AIs, honest AIs, and that don't have any
1:36:49sort of like political agendas put into
1:36:50them by the AI companies or by the
1:36:52government. You know, you want to avoid
1:36:53a situation where the AI company where
1:36:54where the government has issued some
1:36:55sort of secret order that like
1:36:58the AIs have to be such and such a way.
1:37:00Yeah, the Department of War dispute
1:37:01versus Anthropic is like a an
1:37:03interesting sort of foreshadowing of
1:37:04this,
1:37:05right? Where um Anthropic was giving
1:37:08their AIs to the Department of War.
1:37:10Department of War wanted to use them
1:37:12for certain things and was upset that
1:37:14Anthropic's AIs were like
1:37:16not supposed to be used for those
1:37:17things. Uh the things in particular were
1:37:19domestic surveillance and
1:37:21uh
1:37:23autonomous robots.
1:37:25There's going to be a lot more issues
1:37:26like that coming up, and you want it to
1:37:27be the case that like people know what
1:37:29they're getting, and that if people are
1:37:30like spending hours a day talking to
1:37:31their chatbot, that chatbot doesn't have
1:37:34political biases put into it or a secret
1:37:35agenda or things like that, and instead
1:37:37has been trained to like give honest,
1:37:39true answers to things. And I think if
1:37:40you can do that, it can improve the
1:37:42discourse and help people to use their
1:37:44votes to put even better regulations and
1:37:46even better politicians in place, and so
1:37:48forth. And you can sort of potentially
1:37:50bootstrap this to having something where
1:37:53people's power is even more secure than
1:37:54it is today.
1:37:55>> A lot of this stuff we've we've covered
1:37:57in part. So, you know, the wars and
1:37:59drones and missiles, we're already
1:38:00seeing this around the world at the
1:38:01moment, which is really, really
1:38:02interesting. Um
1:38:04and we've talked about robots
1:38:06outnumbering humans as well, which is
1:38:08part of this prediction. Some of the
1:38:09ones down here I found to be really
1:38:10curious, which is
1:38:12people will be protected by AIs wherever
1:38:14they go.
1:38:15>> Mm, yeah. In this scenario,
1:38:18they delay the creation of
1:38:19superintelligence until 2040,
1:38:21and they in fact they pause in 2035, but
1:38:23then they let it go after that. And then
1:38:25they let the AIs become vastly
1:38:26superintelligent.
1:38:28And we think that once the AIs are
1:38:30vastly superintelligent,
1:38:32the world will transform even more
1:38:34radically than
1:38:35what happens in the 2030s in this
1:38:37scenario. So, in the 2030s in this
1:38:39scenario, it's more like human level,
1:38:41you know, the AIs are not
1:38:43they're they're doing the same sorts of
1:38:44things that human experts would have
1:38:45done, they're just doing it a little bit
1:38:47better, a bit faster, and a lot cheaper.
1:38:49And there's a lot more of them.
1:38:50And the robots are still, you know,
1:38:52doing the same sorts of things that
1:38:53human workers would have done. They're
1:38:54just more of them, and they're cheaper.
1:38:57And because of exponential growth, uh
1:39:00you start with a world that looks not
1:39:01that different from today in 2029, and
1:39:03then by 2039, you end in a world that's
1:39:05radically transformed, where everyone's
1:39:07living in these like fancy new
1:39:08apartments that were built by robots 2
1:39:09years ago. There's like giant special
1:39:12economic zones that are full of robots
1:39:14and solar panels and factories producing
1:39:16more robots and solar panels and
1:39:17factories, and so forth. Most of the
1:39:19economy is AIs and robots, and people
1:39:21don't have jobs anymore. That sort of
1:39:23transformation is what you get if you
1:39:24pause at human level.
1:39:26But if you go beyond the
1:39:27superintelligence,
1:39:29there's a whole 'nother transformation
1:39:30coming that's going to look more like
1:39:31magic. Think about how the technology of
1:39:33today
1:39:34would look like magic to someone from
1:39:36500 years ago.
1:39:37>> Mhm.
1:39:38>> You know? And that's without even like a
1:39:40qualitative improvement in intelligence,
1:39:42right? Like the humans of today aren't
1:39:44like qualitatively smarter than the
1:39:45humans from 500 years ago. It's just
1:39:47that we've had more time to do research
1:39:49and we have more like money and
1:39:50resources to build,
1:39:52you know, prototypes and experiments and
1:39:53run experiments and so forth. But if you
1:39:55had a point where there were billions
1:39:57and billions of AIs that were not only
1:40:00faster than humans, but like
1:40:01qualitatively way, way, way better at
1:40:04everything and in particular at doing
1:40:05scientific research, we should expect
1:40:07that some of the things that they
1:40:08develop will seem like magic to us and
1:40:11we'll just completely like we did not
1:40:13think that was even possible, you know?
1:40:15People don't want to die. People don't
1:40:16want to be hit by cars. People don't
1:40:17want to be like attacked by a random
1:40:19mass murderer.
1:40:20>> Cancer's gone?
1:40:22>> I mean, not just cancer, like
1:40:24>> [snorts]
1:40:24>> you know, all all a lot of the stuff
1:40:25that happens in science fiction will
1:40:26probably have happened by then. So,
1:40:28things like people scanning their brains
1:40:29and uploading into into computers,
1:40:32right? Or self-replicating robots
1:40:35in the asteroid belt
1:40:36uh creating more and more satellites to
1:40:40uh produce more and more power to
1:40:41produce more and more self-replicating
1:40:42robots and so forth.
1:40:43>> Most people still live on Earth, but the
1:40:45trend is to move to space?
1:40:46>> That's right. Yeah. So, like if
1:40:49if you end up in the situation where the
1:40:50entire
1:40:51human economy
1:40:54is just like a tiny drop in the bucket
1:40:56that is the entire economy and it's just
1:40:58like a huge amounts of robots and AIs
1:41:01that are
1:41:02moving incredibly quickly, then what you
1:41:04want is Earth to be
1:41:07mostly left as something like a
1:41:08preserve,
1:41:09you know? I think a lot of people are
1:41:12worried about the environment being
1:41:12destroyed,
1:41:14which it totally would be if it wasn't
1:41:15protected. And uh
1:41:17you know, there's a lot of people who
1:41:18sort of like their lives as it is
1:41:20and don't want to be uploaded or live in
1:41:23some crazy new future thing. And it
1:41:25seems to us like the reasonable solution
1:41:27to these issues is
1:41:29uh create new living spaces off the
1:41:31planet with some of that vast
1:41:34economic wealth and activity that's
1:41:35happening
1:41:36for the people who want that sort of
1:41:37thing. And then that way the Earth can
1:41:39be preserved.
1:41:41>> Data center picture here of data centers
1:41:43in the ocean. Uh I mean, there's three
1:41:46images there of
1:41:48different environments where humans
1:41:49might live.
1:41:50>> Again, like our proposal was you
1:41:53preserve like 99% of the Earth
1:41:55uh mostly as is as historic or
1:41:58environmental from as historic or
1:41:59environmental reasons, but then like
1:42:01some parts of it you designate as
1:42:02special economic zones where the robots
1:42:04can go crazy and dig giant pit mines and
1:42:07produce factories and so forth.
1:42:09Um
1:42:10we were thinking it would be good to
1:42:11build the data centers on the ocean
1:42:12instead of um on land for a variety of
1:42:14reasons, although later space would be
1:42:17better and
1:42:19I could see that being reasonable as
1:42:20well.
1:42:21>> What about immortality in a world of AI?
1:42:24Um 20 Well, 30, 45, you say you've lived
1:42:27a dozen lifetimes and are immortal
1:42:29passing from life to life
1:42:31as if by reincarnation.
1:42:34I mean, there's a lot of billionaires at
1:42:35the moment that are focused on
1:42:36longevity. I mean, Brian Johnson's said
1:42:38he's got this central rule, which is do
1:42:40not die right now Yeah. Because we're in
1:42:42the age of AI and it's conceivable that
1:42:44with superintelligence we'll be able to
1:42:46choose when we die.
1:42:47>> Yep. I think that's probably right. We
1:42:49don't depict that happening in this part
1:42:51because at this part they only have, you
1:42:53know, human-level AIs, but that's one of
1:42:55those things that seems quite plausible
1:42:57that superintelligence could achieve um
1:43:01through a variety of means.
1:43:05>> What is your hope with all of this
1:43:06stuff?
1:43:08And why did you do this? Why did you
1:43:09make this 2040 plan A?
1:43:11>> In the like first week after we
1:43:13published AI 2027, it it blew up a lot
1:43:15bigger than we expected, by the way.
1:43:16Like after we published AI 2027, it it
1:43:20blew up a lot bigger than we expected,
1:43:21by the way. Like we actually made
1:43:23forecasts beforehand of like
1:43:26how many views it would get and stuff
1:43:27like that and it was like
1:43:2890th percentile outcome. So, like
1:43:31um very much not what we expected. Um
1:43:34but in like the Twitter storm that
1:43:35happened various people were like
1:43:38all right, why are you giving us all
1:43:39this like doom and gloom uh
1:43:41predictions? Like how about a more
1:43:43positive vision of like what you think
1:43:45we should do instead? And I think that
1:43:46that seed sort of like
1:43:48implanted in us and then we were like,
1:43:50yeah, that's reasonable. Like we've sort
1:43:52of depicted what we think the default
1:43:54path looks like and why we think it's
1:43:55pretty scary.
1:43:57Now maybe we should switch tacks and
1:44:00come up with some actual recommendations
1:44:01and then depict that as well.
1:44:02>> Even though you don't believe they're
1:44:03pro-probable.
1:44:05>> Yeah, I mean you can vote for a
1:44:06political candidate even if you aren't
1:44:07confident that they're going to win, you
1:44:09know? And and you can say like here's
1:44:11what I think we should do even if you
1:44:13think that people are probably not going
1:44:14to do it.
1:44:15You shouldn't say this if you think it's
1:44:16completely unlikely. Like if you think
1:44:17there's no chance, then like maybe you
1:44:19shouldn't bother. But we think there's a
1:44:20chance. Like in particular, for the
1:44:22reasons that we described in the
1:44:24scenario we think that people are going
1:44:26to wake up to the
1:44:28power of AI over the next few years.
1:44:30>> Because of something happens?
1:44:32>> The companies are saying that they're
1:44:33going to do this.
1:44:34>> Mhm.
1:44:34>> And [clears throat]
1:44:36they are kind of on track and it just
1:44:39sort of makes sense that like if they
1:44:41get anywhere close
1:44:42to this level of AI, then there's like
1:44:45big issues and big problems and like we
1:44:46need to like do something about this.
1:44:48And so I think that even if there's not
1:44:51any like very dramatic warning shot or
1:44:53something
1:44:54I think that just naturally people are
1:44:56going to start paying more attention to
1:44:57this and reasoning through the
1:44:58implications and trying to predict
1:45:00>> what's going to happen.
1:45:01>> And so naturally people are going to be
1:45:03more interested in regulation of AI for
1:45:06example. And in fact
1:45:09there's actually like there's there's
1:45:11actually more of this happening than we
1:45:12predicted.
1:45:13>> More of what happening?
1:45:14>> Serious interest in reg- AI regulation.
1:45:17So at the time that we published AI 2047
1:45:19the sort of like mainstream position of
1:45:22the tech companies and in the government
1:45:23was kind of like AI regulation bad idea.
1:45:26>> Free for all.
1:45:27>> Free for all.
1:45:27>> Yeah.
1:45:28>> In fact, there was even an attempt to um
1:45:31preemptively ban states from regulating
1:45:33AI.
1:45:33>> Yeah.
1:45:34>> You remember that? Now it seems like the
1:45:35conversation has changed a lot. Like now
1:45:37that the US government just told
1:45:39Anthropic they have to shut down
1:45:41their AI because they were worried that
1:45:43bad actors would use it for cyber
1:45:44attacks, you know? The government
1:45:47is like waking up and doing more stuff
1:45:49than we expected already. And
1:45:52we're actually hopeful that that trend
1:45:54will just continue and that
1:45:55before it's actually too late, there
1:45:57will be very serious conversations
1:45:59happening inside the government and
1:46:00outside the government and in the
1:46:01broader society about all of these
1:46:03issues and trying to uh
1:46:05chart a course that is um avoids the
1:46:09loss of control and concentration of
1:46:10power risks that we mentioned.
1:46:12>> You um you've spent what must be almost
1:46:15coming up to 15 years thinking about
1:46:16this stuff.
1:46:18Um if this here was a button
1:46:21and if you press that button, your plan
1:46:23S would occur and it would shut down
1:46:26every data center that is currently
1:46:29training a frontier AI model uh for
1:46:31good.
1:46:33There would never be any other
1:46:35>> Mhm.
1:46:35>> AI labs um working on these problems,
1:46:38would you press that button?
1:46:40>> I was I was about to slam it until you
1:46:42said for good.
1:46:43>> Oh, okay.
1:46:44>> Like I think I think if it was a sort of
1:46:45temporary shut down, I would totally
1:46:47slam that button. Because we are not
1:46:49ready to do this, you know? Like what
1:46:52civilization is not ready to have these
1:46:53companies
1:46:55automate themselves and then get smarter
1:46:57and smarter and then have the super
1:46:57intelligent. Like no, there's a bunch of
1:46:59reasons why that's really uh dangerous.
1:47:02But I would be at least hesitant to
1:47:04press this button
1:47:06if it permanently foreclosed the
1:47:08possibility of ever doing it again for
1:47:09sure.
1:47:10>> But but if you think that plan D is
1:47:12probable, which is this race we're on to
1:47:14super intelligent
1:47:15>> If I had a choice between D and S, I
1:47:17think I would press it.
1:47:18>> Well, it's it comes down to what you
1:47:19think, right? Cuz if you think that's
1:47:21that is what's going to happen, plan B.
1:47:23And the only alternative
1:47:26>> I didn't say this is what's going to
1:47:27happen.
1:47:28>> Probabilistically.
1:47:28>> Yeah, yeah, yeah. Like like I'd be like
1:47:29this is the most likely, maybe this is
1:47:31the second most likely, maybe this is
1:47:33the third most likely. They are all
1:47:34possible.
1:47:35>> So with your current perspective on
1:47:36whatever one you think is going to
1:47:37happen, would you press the button? I'm
1:47:39giving you a an S, a definite S, or
1:47:41whatever you think is going to happen.
1:47:42>> That's tough.
1:47:45>> [sighs]
1:47:48>> What is the scope of the shutdown? So is
1:47:50it
1:47:51>> It's no one can train an AI model again.
1:47:54Ever again.
1:47:57>> That's real rough cuz like I said,
1:47:58there's loads of benefits that we could
1:47:59get from AI if we do it right. Um
1:48:01>> I think I I've almost put you in the
1:48:03position of Sam Altman.
1:48:04>> Yeah. [laughter]
1:48:05>> To some degree.
1:48:06>> Yeah.
1:48:08Um let me Do you mind if I just take a
1:48:10moment to think about this?
1:48:10>> think about it. Perfectly to think.
1:48:12>> Yeah.
1:48:21I think I would not press
1:48:23the button, but I'm I feel very torn
1:48:25about it.
1:48:26Um the reason why I think I would not
1:48:27press the button is that
1:48:29I still have substantial hope that we
1:48:31can get something much better than this,
1:48:32something more like this.
1:48:34And I think that
1:48:37Basically, I think that if we don't
1:48:38build powerful AI systems eventually,
1:48:41then
1:48:43we're probably going to die as a
1:48:45civilization
1:48:47eventually, you know, like 100 years
1:48:48from now, 200 years from now, something
1:48:49like that. Like nuclear war, pandemic,
1:48:52you know.
1:48:54I I don't think human civilization right
1:48:56now is like super super stable.
1:48:59Um
1:49:00and so
1:49:01I think that
1:49:03basically, what I was about to say was
1:49:05the possible benefits for posterity and
1:49:07for all the billions and billions of
1:49:09people who could live in the future
1:49:10outweigh the like
1:49:14the current level of risk, but actually
1:49:17>> I've heard that narrative before. Yeah,
1:49:18I don't know. Like
1:49:20Yeah, like maybe maybe it's just like
1:49:22nope.
1:49:23The people right now
1:49:24are the people we should prioritize.
1:49:26People right now are in grave danger.
1:49:29They're going to be fine for at least
1:49:30the next couple of decades.
1:49:32So,
1:49:34never mind posterity.
1:49:36Prioritize the people right now.
1:49:38Um and people right now definitely don't
1:49:39want
1:49:41to do this lottery,
1:49:42I would say.
1:49:44Um
1:49:45>> [sighs and gasps]
1:49:46>> Yeah, you've really asked me a tough
1:49:47question. So, would you press the button
1:49:50if that was the button?
1:49:52Probably not, but I would feel very
1:49:54torn.
1:49:55>> Okay.
1:49:56So, what I I always think about the
1:49:58personas of like the audience that are
1:49:59watching. And these are, you know,
1:50:01they're they're very curious people,
1:50:02especially on the subject of AI as we've
1:50:03seen, but they they want to know like
1:50:06what it means for them. I think a lot of
1:50:07them also want to know what they can do.
1:50:09>> Uh yes. Yeah, what can people do? Well,
1:50:12I think that if you either have
1:50:15talent or passion, you can get directly
1:50:18involved. There's lots of organizations
1:50:20that are worried about these things and
1:50:21that are trying to do something about
1:50:22it, like political advocacy or technical
1:50:25research or like building useful tools
1:50:28that will hopefully help people be
1:50:30better and stuff. But if you don't want
1:50:31to like make any major career changes or
1:50:34or things like that, then
1:50:36I would say just pay more attention to
1:50:38these issues and talk about it more with
1:50:40people. Do stuff like, you know,
1:50:42emailing your congressman or whatever.
1:50:44It doesn't change things that much, but
1:50:46it does help. I think that especially
1:50:49for this particular issue, the core
1:50:51problem is that people aren't taking it
1:50:52seriously yet.
1:50:54Like if the sorts of things that I was
1:50:55just saying to you for the last hour or
1:50:57two were just like
1:50:59top of everybody's mind,
1:51:02we wouldn't even be here. Like there
1:51:03would there would already be much more
1:51:04significant regulation in place, you
1:51:07know? And not only would there be more
1:51:09heavy regulation in place, but there
1:51:11would have been better regulation in
1:51:12place that's less, you know, less like a
1:51:15cudgel and more like a scalpel and
1:51:16that's like more sensitive to what's
1:51:19actually bad and what's not so bad and
1:51:21so forth. And there'd be more expert
1:51:22people in the government and advising
1:51:24the government and so forth. So just in
1:51:26general like
1:51:28the more people wake up to these
1:51:30concerns and to these projections,
1:51:32I think the more likely it is that we
1:51:34can do good stuff before it's too late.
1:51:36>> What about how they should vote at the
1:51:37polls? We've got an election coming up
1:51:40in the United States in a couple of
1:51:41years time, but there's elections
1:51:42happening all over the world all the
1:51:43time.
1:51:44>> You should ask your candidates what they
1:51:46think about all this AI stuff. You
1:51:47should try to get them to like have
1:51:49opinions and then you should vote for
1:51:50the candidates whose opinions are better
1:51:52on this topic. This is the most
1:51:53important thing happening
1:51:55in our lifetimes, probably in all of
1:51:57history in fact, and it's very important
1:51:59that it go well. And so this is what all
1:52:01the all the leaders of all the countries
1:52:03should be thinking about and making
1:52:04plans for.
1:52:05>> Isn't it such a weird thing to be alive
1:52:06at this moment in time?
1:52:08Like I was thinking about all the times
1:52:09that I could have been born. And I guess
1:52:10my ancestors probably thought the same,
1:52:12but I was thinking as you were speaking
1:52:13I was like, I think it's when you
1:52:14referred to it as like the final show.
1:52:16>> Yeah.
1:52:17>> What was the phraseology you used?
1:52:18>> I said the the climate it was the run-up
1:52:20to the climax or something.
1:52:21>> Yeah. I mean what a what a crazy thing
1:52:24to be born in the run-up to the climax
1:52:26where everything you're describing here
1:52:28is within my lifetime conceivably
1:52:30hopefully.
1:52:30>> Yeah.
1:52:31>> Um or maybe not hopefully.
1:52:33What a crazy time to be alive.
1:52:35>> Certainly.
1:52:36>> I noticed that when I meant asked you if
1:52:37you had kids your demeanor changed quite
1:52:39considerably.
1:52:40>> Well, it's yeah.
1:52:42>> It's like you dropped into a different
1:52:43state.
1:52:44Obviously that's been central to the
1:52:48rumination that you've been
1:52:49experiencing.
1:52:50>> Well, it is a sad topic, right? Like
1:52:52when when I had kids
1:52:54like the reason to have kids is in large
1:52:56part about the future, you know?
1:52:58Like it's not just like a cuddly thing
1:53:00to have with you in the moment. It's cuz
1:53:02you have all these hopes and dreams
1:53:03about how they'll grow up and how
1:53:04they'll go to their own thing and be
1:53:05their own person and stuff. And
1:53:08because of what's happening with AI, I
1:53:10think a lot of those dreams are in
1:53:11jeopardy.
1:53:12>> Presumably you still would have had
1:53:13kids?
1:53:14>> I've actually flip-flopped on this
1:53:15occasionally. Yeah. Basically the top
1:53:17line answer is I'm not sure. The
1:53:20my first child was had we we had her
1:53:22when we were um in 209 she was born in
1:53:242019. Yeah. So this is before my
1:53:26timeline shortened a lot. So at that at
1:53:28this point I was interested in AI, I was
1:53:29tracking the field, I was making
1:53:30forecasts,
1:53:31but I didn't like actually expect it to
1:53:33happen soon.
1:53:34You know?
1:53:36And then this caused like
1:53:38when I when I did start thinking like oh
1:53:39my gosh, it's going to be happening like
1:53:40real soon. Um like by 2030, you know?
1:53:44Um that caused
1:53:46some reconsidering. And so
1:53:49I basically told my wife like let's not
1:53:50have any more kids. It's too uncertain,
1:53:52you know?
1:53:54But that turned out to be really hard
1:53:55because
1:53:56especially for my wife. Like we already
1:53:58had one kid and like
1:54:00no siblings.
1:54:01Um so eventually I sort of gave in and
1:54:04was like okay, well, you know what? We
1:54:05already have one.
1:54:07It's going to be all right. Like
1:54:09maybe maybe the future will be good and
1:54:11even if it's not like
1:54:13well, we're all in the same boat
1:54:13together.
1:54:15>> It's quite chilling what you're saying.
1:54:17It's chilling because you know more than
1:54:18me.
1:54:19And if you're at home saying to your
1:54:20wife, "Listen, maybe we should pause on
1:54:22having more children and building a
1:54:23family because of what's going on with
1:54:24AI."
1:54:26>> To be clear, is it Yes, I mean yes, it's
1:54:27very concerning.
1:54:29I am I am chilled.
1:54:31Uh this is bad. This is what I've been
1:54:32saying.
1:54:33I hope things go well. I think things
1:54:35might go well. Um I think that there's a
1:54:37lot we can do to like steer things in a
1:54:38better direction.
1:54:39>> I mean one of those things as well I
1:54:40have to say is just speaking about it.
1:54:43It's I think a lot of the progress we've
1:54:45seen with governments waking up and
1:54:48you know, we've seen certain things with
1:54:49people booing certain people at certain
1:54:50events. Yeah. Um is it is it downstream
1:54:53from people like yourself actually
1:54:55coming on shows like this and all the
1:54:57other podcasts and
1:54:58Yeah. telling us what's going on. Yeah.
1:55:00Because else we're to be fair, we're
1:55:02going to be gaslighted by the people
1:55:03that have the biggest PR machines.
1:55:05>> Yeah.
1:55:06>> So, um I often I think it's probably
1:55:07worth me saying I find myself kind of in
1:55:09two minds cuz I'm an entrepreneur and
1:55:11I'm an I'm an investor. I'm an investor
1:55:13in probably more than 100 companies now
1:55:14and well so many of those companies are
1:55:16using AI. I invested in Grok, the
1:55:18inference chip company. Invested in
1:55:20SpaceX which now own another Grok and
1:55:22they're doing AI. I use AI every day in
1:55:24my life. I've been using it through this
1:55:25conversation to understand different
1:55:26things that you've said. So, that's one
1:55:28side of me which is like business
1:55:30builder, entrepreneur who has seen the
1:55:32benefits of AI in my own life and then
1:55:34there's the other side of me. And it's
1:55:35funny cuz I think sometimes people think
1:55:37you have to pick a camp.
1:55:38But through all of my life, even when I
1:55:40was a social media CEO and I was saying
1:55:41by the way listen I'm building a social
1:55:42media business but I think there's some
1:55:43downsides to social media. Find myself
1:55:45at the same moment where I'm like I
1:55:46build with AI. I have AI investments.
1:55:49And at the same time as a civilian I'm
1:55:51like
1:55:52>> Yeah.
1:55:53I mean I think that is a tension. I
1:55:54think that there's there's different
1:55:57way ways you can draw the line. So, and
1:55:59I know lots of people who draw the line
1:56:01in lots of different ways. So, like
1:56:02there's some people who just like I'm
1:56:03not going to use AI. I think this stuff
1:56:05is bad um and on a bad trajectory so I'm
1:56:07going to like boycott AI, right? I'm not
1:56:09one of those people. I use AI a lot. We
1:56:11all do at AI Futures Project. Um it's
1:56:13helpful for a lot of our work.
1:56:15The opposite end of the spectrum is
1:56:18you
1:56:19people being like
1:56:21well, it seems like it's on a trajectory
1:56:22to happen so the thing to do to make it
1:56:24go well is to like
1:56:26get involved and accumulate power and
1:56:27try to like steer it from the inside.
1:56:29>> Mhm.
1:56:29>> And so I'm going to go work at OpenAI or
1:56:31Anthropic and like try to like climb the
1:56:33ranks and then like you know, be someone
1:56:35who matters when the important decisions
1:56:37are being made. And I know loads of
1:56:38people like that. That was like what I
1:56:40was doing when I was
1:56:41That wasn't what I was doing exactly but
1:56:42like
1:56:43>> That was the path.
1:56:44>> That was like that was a I mean this In
1:56:45some sense this is what the whole
1:56:46narrative of the companies are, right?
1:56:47Like this is why they tell themselves
1:56:48it's okay to do what they're doing is
1:56:50that they're worried about the other
1:56:50guys, you know? And so like all these
1:56:53people are deciding like we're going to
1:56:55like lean really hard into it. We're
1:56:56going to like be there in the room when
1:56:59the when decisions are being made, you
1:57:00know? So, there's a whole spectrum and
1:57:02I'm sort of like somewhere in the
1:57:03middle. Like I'm not at the at
1:57:04companies, I'm not helping them
1:57:06go faster.
1:57:07Instead, I'm talking to the broad public
1:57:09and trying to advocate for what I think
1:57:11is the
1:57:13my current best guess as to the way out,
1:57:15you know, the way forward.
1:57:17Um but, I'm not like boycotting all the
1:57:19AIs. I'm I'm not like, you know,
1:57:21uh trying to I'm not refusing to like
1:57:23engage with it in that way.
1:57:25>> Do you think it's too late?
1:57:27>> No.
1:57:29I don't think it's too late. If I
1:57:30thought it was too late, I wouldn't be
1:57:31here.
1:57:31>> Hm. Where would you [clears throat] be?
1:57:33>> With my family.
1:57:36>> What's your closing message to the
1:57:38general public if you had to have a
1:57:40closing statement to them? Maybe I would
1:57:42say that like
1:57:44>> you're going to hear a lot of things and
1:57:45you already have been hearing a lot of
1:57:46things about
1:57:48AI and it's going to sound like science
1:57:50fiction,
1:57:51but sometimes things which sound like
1:57:53science fiction happen in reality.
1:57:56And in fact, many times historically
1:57:58things which used to be science fiction
1:57:59have then become reality. And people
1:58:02need to
1:58:03stop thinking about what does or doesn't
1:58:04sound like science fiction and just
1:58:05start thinking about like the trends
1:58:08and,
1:58:09you know, the actual trends that this
1:58:11technology is on and
1:58:13reading and forecasting how it's going
1:58:14to go and then taking seriously the
1:58:16possibility that it could go something
1:58:18like this and then thinking about what
1:58:20should be done about that.
1:58:21>> And where would you direct them to get
1:58:23more information? You can go
1:58:25>> to ai2047.com to read our previous
1:58:27scenario. You can go to ai2040.com plan
1:58:30A to read our new proposal for what is
1:58:33to be done. Um these things are not just
1:58:36a sci-fi story. They also have lots of
1:58:39like explainers and links to other
1:58:40things. And so, they're kind of like a
1:58:42nice jumping off point to to learn about
1:58:45all of this stuff. Um
1:58:48If you want, I could um after this is
1:58:49over, like give a reading list of like
1:58:51other papers and articles and
1:58:55>> Please do.
1:58:55>> blogs to follow and so forth.
1:58:57>> And I'll link them all below in the
1:58:58comment section. So, if you're listening
1:58:59now, go ahead and take a look at the
1:59:01comment sec the description of this
1:59:03episode and you'll see a bunch of links
1:59:05which is Daniel's recommendations of
1:59:06what you should read. You know, I think
1:59:08it's it's just a really really great
1:59:09moment in time to get educated on this
1:59:11stuff. Um humans have a an inclination
1:59:14because of cognitive dissonance where we
1:59:15feel uncomfortable about something to
1:59:17bury our heads in the sand and avoid it.
1:59:20>> Yeah.
1:59:20>> But actually, I think this is one such
1:59:22time to do the very opposite. For many
1:59:23reasons, to to inform yourself so you
1:59:26know what actions to take, but also
1:59:27because AI
1:59:29you know, unavoidably is going to be a
1:59:30huge part of all of our lives and
1:59:31careers.
1:59:32>> Yeah. Yeah, thank you. And that that's
1:59:34the good way to
1:59:35to say it. It's going to matter a lot.
1:59:37It's going to It's going to be
1:59:38everywhere soon and um
1:59:41we need to do something about it before
1:59:42it's too late.
1:59:42>> What about AI Future Project?
1:59:44>> That's our organization. We spent a year
1:59:46writing a 2047 after I left OpenAI and
1:59:48then we spent another year writing a
1:59:492040 Plan A.
1:59:52>> Daniel, thank you.
1:59:53>> Thank you.
1:59:53>> Thank you for all the work that you do.
1:59:54I can see how much you care about this
1:59:55stuff and it's your care it's funny care
1:59:57itself makes others feel care. And
2:00:00seeing how personal this is for you and
2:00:01seeing how much you've dedicated your
2:00:02life to this, but also hearing that you
2:00:05you basically walked away from $2
2:00:07million to be able to speak to the
2:00:09public about this information
2:00:10[clears throat] is incredibly admirable
2:00:11and uh I I think voices like yours are
2:00:15more important now than they've ever
2:00:16been on this subject. So, please do keep
2:00:17fighting the fight that you're fighting
2:00:18and that's one of information, it is of
2:00:20honesty, and it is uh of saying what
2:00:23what is often the quiet part out loud.
2:00:25>> Thank you.
2:00:26>> doing really really smart research. I'll
2:00:27link everything we've discussed today
2:00:29below and I hope we can chat again
2:00:30sometime soon.
2:00:31>> Thank you.
2:00:32>> YouTube have this new crazy algorithm
2:00:33where they know exactly what video you
2:00:36would like to watch next based on AI and
2:00:38all of your viewing behavior. And the
2:00:40algorithm says that this video is the
2:00:43perfect video for you. It's different
2:00:45for everybody looking right now. Check
2:00:46this video out. I bet you you might love
2:00:48it.

About this transcript

This page contains the full transcript of ChatGPT Offered Me $2m To Keep Quiet: No One Is Ready For What's Coming! by The Diary Of A CEO, generated from the public captions YouTube serves with the video. The transcript has 25,468 words across 4,147 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.