YT YouTube2Text

YouTube2Text → Transcripts → The AI Safety Expert: These Are The Only 5 Jobs That Will Remain In 2030! - Dr. Roman Yampolskiy

The AI Safety Expert: These Are The Only 5 Jobs That Will Remain In 2030! - Dr. Roman Yampolskiy — Transcript

by The Diary Of A CEO · 15,152 words · 2,524 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00You've been working on AI safety for two
0:01decades at least.
0:02>> Yeah. I was convinced we can make safe
0:04AI, but the more I looked at it, the
0:06more I realized is not something we can
0:08actually do. You have made a series of
0:10predictions about variety of different
0:12dates. So, what is your prediction for
0:152027?
0:19Dr. Roman Yampolskiy is a globally
0:21recognized voice on AI safety and
0:23associate professor of computer science.
0:25He educates people on the terrifying
0:27truth of AI
0:28>> and what we need to do to save humanity.
0:30In 2 years, the capability to replace
0:32most humans in most occupations will
0:34come very quickly. And then in 5 years,
0:37we're looking at a world where we have
0:39levels of unemployment we've never seen
0:40before. Not talking about 10% but 99%.
0:45And that's without super intelligence, a
0:47system smarter than all humans in all
0:49domains. So, it would be better than us
0:51at making new AI. But it's worse than
0:54that. We don't know how to make them
0:55safe. And yet we still have the smartest
0:57people in the world competing to win the
0:59race to super intelligence. But what do
1:01you make of people like Sam Altman's
1:02journey with AI? So, a decade ago, we
1:05published guardrails for how to do AI
1:07right. They violated every single one.
1:10And he's gambling 8 billion lives on
1:12getting richer and more powerful. So, I
1:14guess some people want to go to Mars.
1:16Others want to control the universe.
1:18But it doesn't matter who builds it. The
1:20moment you switch to super intelligence,
1:22we will most likely regret it terribly.
1:24And then by 2045?
1:27Now, this is where it gets interesting.
1:30Dr. Roman Yampolskiy, let's talk about
1:32simulation theory. I think we are in
1:34one. And there is a lot of agreement on
1:36this. And this is what you should be
1:37doing in it so they don't shut it down.
1:40First,
1:42I see messages all the time in the
1:44comment section that some of you didn't
1:45realize you didn't subscribe. So, if you
1:47could do me a favor and double-check if
1:49you're a subscriber to this channel,
1:50that would be tremendously appreciated.
1:52It's the simple, it's the free thing
1:54that anybody that watches this show
1:55frequently can do to help us here to
1:57keep everything going in this show in
1:58the trajectory it's on. So, please do
2:00double-check if you subscribed and uh
2:02thank you so much because in a strange
2:04way you are you're part of our history
2:06and you're on this journey with us and I
2:07appreciate you for that. So, yeah, thank
2:09you.
2:13Dr. Roman Yampolskiy.
2:17What is the mission that you're
2:18currently on? Cuz it's quite clear to me
2:20that you are on a bit of a mission and
2:22you've been on this mission for, I
2:23think, the best part of two decades at
2:24least.
2:26I'm hoping to make sure that super
2:29intelligence we are creating right now
2:31does not kill everyone.
2:37Give me some Give me some context on
2:39that statement cuz it's quite a shocking
2:40statement.
2:41Sure. So, the last decade, we actually
2:44figured out how to make artificial
2:46intelligence better.
2:48Turns out if you add more compute, more
2:51data,
2:52it just kind of becomes smarter.
2:55And so now, smartest people in the
2:57world, billions of dollars, all going to
3:00create the best possible super
3:02intelligence we can.
3:04Unfortunately, while we know how to make
3:07the systems much more capable,
3:09we don't know how to make them safe.
3:11How to
3:13make sure they don't do something we
3:14will regret.
3:16And that's the state of the art right
3:18now. When we look at
3:20just prediction markets, how soon will
3:22we get to advanced AI?
3:25The timelines are very short, couple
3:26years.
3:28Two, three years according to prediction
3:30markets, according to CEOs of top labs.
3:34And at the same time,
3:36we don't know how to make sure that the
3:40systems are aligned with our
3:41preferences.
3:43So, we are creating this alien
3:45intelligence.
3:46If aliens were coming to Earth and
3:50you had 3 years to prepare,
3:53you would be panicking right now.
3:55But most people don't don't even realize
3:57this is happening.
4:00So, some of the counterarguments might
4:01be, well, these are very, very smart
4:03people. These are very big companies
4:05with lots of money. They have a
4:06obligation and a moral obligation but
4:09also just
4:10a legal obligation to make sure they do
4:12no harm. So, I'm sure it'll be fine. The
4:14only obligation they have is to make
4:16money for their investors. That's the
4:17legal obligation they have. They have no
4:19moral or ethical obligations. Also,
4:22according to them, they don't know how
4:24to do it yet. The state-of-the-art
4:26answers are, we'll figure it out when we
4:28get there or AI will help us control
4:30more advanced AI.
4:33That's insane.
4:34In terms of probability, what do you
4:35think is the probability that something
4:37goes catastrophically wrong?
4:40So, nobody can tell you for sure what's
4:42going to happen. But if you're not in
4:44charge, you're not controlling it, you
4:46will not get outcomes you want. The
4:49space of possibilities is almost
4:50infinite. The space of outcomes we will
4:53like is tiny.
4:56And
4:57who are you and how long have you been
4:59working on this?
5:01I'm a computer scientist by training. I
5:03have a PhD in computer science and
5:05engineering.
5:06I probably started work in AI safety,
5:10mildly defined as control of bots at the
5:13time,
5:1515 years ago.
5:1715 years ago. So, you've been working on
5:19AI safety before it was cool. Before the
5:21term existed. I coined the term AI
5:23safety. So, you're the founder of the
5:25term AI safety? The term, yes, not the
5:27field. There are other people who did
5:29brilliant work before I got there.
5:31Why were you thinking about this 15
5:32years ago? Because most people have only
5:34been talking about the term AI safety
5:35for the last two or three years. Yeah,
5:37it started very mildly just as a
5:40security project. I was looking at poker
5:43bots.
5:44And I realized that the bots are getting
5:46better and better.
5:48And if you just project this forward
5:50enough,
5:51they're going to get better than us,
5:53smarter, more capable. And it happened.
5:55They are playing poker way better than
5:57average players.
5:59But
6:00more generally, it will happen with all
6:01other domains, all the other cyber
6:03resources.
6:05I wanted to make sure AI is a technology
6:07which is beneficial for everyone. So, I
6:09started work on making AI safer.
6:14Was there a particular moment in your
6:15career where you thought,
6:17"Oh my god."?
6:19First 5 years at least, I was working on
6:22solving this problem. I was convinced we
6:24can make happen, we can make safe AI.
6:27That was the goal. But the more I looked
6:29at it, the more I realized every single
6:31component of that equation is not
6:33something we can actually do.
6:35And the more you zoom in, it's like a
6:37fractal. You go in and you find 10 more
6:39problems and then 100 more problems. And
6:43all of them are not just difficult,
6:45they're impossible to solve. There is no
6:48seminal work in this field where like,
6:51we solved this. We don't have to worry
6:52about this. There are patches. There are
6:55little fixes we put in place and quickly
6:57people find ways to work around them.
7:00They jailbreak whatever safety
7:02mechanisms we have. So, while progress
7:05in
7:06AI capabilities is exponential or maybe
7:09even hyper exponential,
7:11progress in AI safety is linear or
7:13constant.
7:14The gap is increasing.
7:16The gap between
7:18the the how capable the systems are and
7:21how well we can control them, predict
7:23what they're going to do, explain their
7:25decision-making.
7:26I think this is quite an important point
7:28because you said that we're basically
7:30patching over the issues that we find.
7:32So, we're developing this this core
7:34intelligence and then to stop it doing
7:36things
7:38or to stop it showing some of its
7:40unpredictability or its threats,
7:43the companies that are developing this
7:45AI are programming in code over the top
7:47to say, "Okay, don't swear. Don't say
7:49that rude word. Don't do that bad
7:50thing." Exactly. And you can look at
7:52other examples of that. So, HR manuals,
7:55right? We have those humans, they're
7:57general intelligences, but you want them
7:59to behave in a company. So, they have a
8:01policy. No sexual harassment. No this,
8:03no that. But if you're smart enough, you
8:06always find a workaround. So, you're
8:08just pushing behavior into a different,
8:10not yet restricted subdomain.
8:14We We should probably define some terms
8:16here.
8:17So, there's narrow intelligence which
8:19can play chess or whatever. There's
8:21artificial general intelligence which
8:22can operate across domains. And then
8:24super intelligence which is smarter than
8:26all humans in all domains.
8:27And where are we?
8:29So, that's a very fuzzy boundary, right?
8:32We definitely have many excellent narrow
8:35systems, no question about it. And they
8:37are super intelligent in that narrow
8:38domain. So,
8:40protein folding is a problem which was
8:42solved using narrow AI and it's superior
8:44to all humans in that domain.
8:46In terms of AGI, again I said, if we
8:49showed what we have today to a scientist
8:52from 20 years ago, they would be
8:54convinced we have full-blown AGI. We
8:56have systems which can learn. They can
8:58perform in hundreds of domains and
9:00they're better than human in many of
9:02them.
9:03So, you can argue we have a weak version
9:06of AGI.
9:08Now, we don't have super intelligence
9:09yet. We still have brilliant humans who
9:12are completely dominating AI, especially
9:14in science and engineering.
9:16But that gap is closing so fast. You can
9:19see
9:20especially in the domain of mathematics.
9:233 years ago,
9:25large language models couldn't do basic
9:27algebra.
9:28Multiplying three-digit numbers was a
9:30challenge. Now, they're helping with
9:32mathematical proofs. They're winning
9:34mathematics Olympiads, competitions.
9:37They're working on solving millennial
9:39problems, hardest problems in
9:41mathematics. So, in 3 years, we closed
9:43the gap from subhuman performance to
9:46better than most mathematicians in the
9:48world. And we see the same process
9:50happening in science and engineering.
9:54You have made a series of predictions
9:56and they correspond to a variety of
9:58different dates and I have those dates
10:00in front of me here.
10:02What is your prediction for the year
10:042027?
10:07We're probably looking at AGI as
10:10predicted by prediction markets and tops
10:13of the labs.
10:14So we'd have artificial general
10:15intelligence by 2027.
10:18And how would that make the world
10:19different
10:21to how it is now?
10:22So if you have this concept of a drop-in
10:26you have free labor, physical and
10:28cognitive, trillions of dollars of it.
10:30It makes no sense to hire humans for
10:32most jobs.
10:34If I can just get, you know, a $20
10:36subscription or free model to do what an
10:38employee does,
10:40first, anything on a computer will be
10:41automated.
10:43And next thing, humanoid robots are
10:45maybe 5 years behind, so in 5 years all
10:48the physical labor can also be
10:49automated.
10:51So we're looking at a world where we
10:53have levels of unemployment we've never
10:55seen before. Not talking about 10%
10:57unemployment, which is scary, but 99%.
11:01All you have left is jobs where, for
11:03whatever reason, you prefer another
11:06human would do it for you.
11:08But anything else
11:10can be fully automated. It doesn't mean
11:12it will be automated in practice. A lot
11:14of times
11:15technology exists, but it's not
11:17deployed. Video phones were invented in
11:20the '70s. Nobody had them until iPhones
11:22came around.
11:25So we may have a lot more time with jobs
11:28and with world which looks like this.
11:30But capability
11:32to replace most humans in most
11:34occupations will come very quickly.
11:38Okay, so let's try and drill down into
11:40that and and stress test it.
11:43So
11:46a podcaster like me,
11:48would you need a podcaster like me?
11:52So let's look at what you do. You
11:54prepare, you
11:57ask questions, you ask follow-up
11:59questions, and you look good on camera.
12:01Thank you so much. Let's see what we can
12:03do. Large language model today can
12:05easily read everything I wrote Yeah. and
12:07have very solid understanding. Better, I
12:10assume you haven't read every single one
12:11of my books. I haven't. Yeah. That thing
12:13would do it.
12:14It can train on every podcast you ever
12:16did, so it knows exactly your style, the
12:19types of questions you ask. It can also
12:22find correspondence between what worked
12:24really well, like this type of question
12:26really increased viewers, this type of
12:29topic was very promising, so it can
12:31optimize, I think, better than you can
12:33because you don't have a data set.
12:35Of course, visual simulation is trivial
12:38at this point. So it can you can make a
12:40video within seconds of me sat here and
12:42So we can generate videos of you
12:44interviewing anyone on any topic very
12:47efficiently and you just have to get
12:51likeness approval, whatever.
12:53Are there many jobs that you think would
12:56remain in a world of AGI? If you're
12:57saying AGI is potentially going to be
12:59here, whether it's deployed or not, by
13:002027,
13:02what kind and then okay, so let's take
13:04out of this any physical labor jobs for
13:07a second. Are there any jobs that you
13:09think a human would be able to do better
13:11in a world of AGI
13:13still? So that's the question I often
13:15ask people. In the world with AGI, and I
13:18think almost immediately we'll get
13:20superintelligence as a side effect. So
13:22the question really is, in a world of
13:24superintelligence, which is defined as
13:26better than all humans in all domains,
13:29what can you contribute?
13:31And so you know better than anyone what
13:33it's like to be you.
13:35You know what ice cream tastes to you.
13:38Can you get paid for that knowledge? Is
13:40someone interested in that?
13:43Maybe not, not a big market.
13:45There are jobs where you want a human.
13:47Maybe you're rich and you want a human
13:49accountant for whatever historic
13:51reasons.
13:53Old people like
13:55traditional ways of doing things. Warren
13:58Buffett would not switch to AI. He would
14:00use his human accountant.
14:02But it's a tiny subset of a market.
14:05Today we have products which are
14:07man-made
14:08in US as opposed to mass-produced in
14:11China, and some people pay more to have
14:13those.
14:14But it's a small subset. It's a almost a
14:16fetish.
14:18There is no practical reason for it.
14:20And I think anything you can do on a
14:22computer could be automated
14:24using that technology.
14:27You must hear a lot of rebuttals to when
14:29this when you say it because people
14:31experience a huge amount of mental
14:33discomfort when they hear
14:35that their job, their career, the thing
14:36they got a degree in, the thing they
14:38invested $100,000 into is going to be
14:40taken away from them. So their natural
14:42reaction, some for some people is that
14:43cognitive dissonance that no, you're
14:45wrong. AI can't be creative. It's not
14:48this, it's not that. It will never be
14:50interested in my job. I'll be fine
14:52because
14:53you hear these arguments all the time,
14:55right?
14:55>> It's really funny. I ask people and I
14:57ask people in different occupations.
14:59I'll ask my Uber driver, are you worried
15:01about self-driving cars? And they go,
15:03"No. No one can do what I do. I know the
15:06streets of New York. I can navigate like
15:08no AI.
15:10I'm safe." And it's true for any job.
15:12Professors are saying this to me. Oh,
15:14nobody can lecture like I do. Like this
15:16is so special.
15:17But you understand it's ridiculous. We
15:19already have self-driving cars replacing
15:21drivers.
15:23That is not even a question
15:25if it's possible. It's like how soon
15:27before you're fired.
15:30Yeah, I mean, I've just been in LA
15:31yesterday and my car drives itself. So I
15:34get in the car, I set I put in where I
15:36want to go, and then I don't touch the
15:38steering wheel or the brake pedals, and
15:39it takes me from A to B, even if it's an
15:41hour-long drive without any intervention
15:44at all. I actually still park it,
15:46but other than that, I'm not I'm not
15:47driving the car at all. And then
15:49obviously in LA we also have Waymo now,
15:51which means
15:52you order it on your phone and it shows
15:55up with no driver in it and takes you to
15:56where you want to go. Oh, yeah. So it's
15:59quite clear to see how that is
16:00potentially a matter of time. For those
16:02people, cuz we do have some of those
16:04people listening to this conversation
16:05right now, that their occupation is
16:07driving,
16:08to offer them a I think driving is the
16:10biggest oc-
16:11occupation in the world, if I'm correct.
16:15I I'm pretty sure it is the biggest
16:16occupation in the world.
16:17>> of the top ones, yeah.
16:19What would you say to those people?
16:21What what should they be doing with
16:22their lives? What should they should
16:23they be retraining in something or
16:25what time frame? So that's the paradigm
16:27shift here. Before we always said this
16:29job is going to be automated, retrain to
16:31do this other job. But if I'm telling
16:33you that all jobs will be automated,
16:36then there is no plan B.
16:38You cannot retrain.
16:41Look at computer science.
16:442 years ago, we told people, learn to
16:46code. Mhm. You are an artist, you cannot
16:49make money, learn to code.
16:51Then we realized, oh, AI kind of knows
16:54how to code and getting better. Become a
16:56prompt engineer.
16:58You can engineer prompts for AIs. It's
17:01going to be a great job. Get a 4-year
17:02degree in it. But then we're like, AI is
17:05way better at designing prompts for
17:06other AIs than any human.
17:08So that's gone. So I can't really tell
17:10you right now, the hottest thing is
17:12design AI agents for practical
17:14applications. I guarantee you in a year
17:17or two it's going to be gone just as
17:18well.
17:20So I don't think there is a
17:22this occupation needs to learn to do
17:24this instead. I think it's more like,
17:25where's the humanity when we all lose
17:28our jobs? What do we do? What do we do
17:31financially?
17:32Who's paying for us?
17:34And what do we do in terms of
17:36meaning? What do I do with my extra 60,
17:4080 hours a week?
17:42You've thought around this corner,
17:44haven't you?
17:45A little bit.
17:46What is around that corner in your view?
17:49So the economic part seems easy. If you
17:51create a lot of free labor, you have a
17:53lot of free wealth, abundance, things
17:56which are right now
17:57not very affordable become dirt cheap,
18:00and so you can provide for everyone's
18:01basic needs. Some people say you can
18:03provide
18:06beyond basic needs. You can provide very
18:09good existence for everyone. The hard
18:11problem
18:12what do you do with all that free time?
18:14For a lot of people, their jobs are what
18:17gives them meaning in their lives, so
18:19they would
18:20retire or do early retirement. And for
18:24so many people who hate their jobs,
18:26they'll be very happy not working. But
18:28now you have people who are chilling all
18:30day.
18:31What happens to society? How does that
18:33impact crime rate, pregnancy rate, all
18:36sorts of issues?
18:37Nobody thinks about. Governments don't
18:39have programs prepared to deal with 99%
18:43unemployment.
18:47What do you think that world looks like?
18:50Again, I I think
18:52>> you going to be doing? very important
18:53part to understand here is the
18:56unpredictability of it.
18:58We cannot predict what a smarter than us
19:00system will do.
19:02And the point when we get to that is
19:04often called singularity, by analogy
19:06with physical singularity. You cannot
19:09see beyond the event horizon. I can tell
19:11you what I think might happen, but
19:13that's my prediction. It is not what
19:16actually is going to happen because I
19:18just don't have cognitive ability to
19:20predict a much smarter agent impacting
19:23this world.
19:25When you read science fiction,
19:27there is never a superintelligence in it
19:29actually doing anything because nobody
19:31can write believable science fiction at
19:33that level. They either banned AI, like
19:36Dune, because this way you can avoid
19:38writing about it, or it's like Star
19:40Wars. You have this really dumb bots,
19:43but not nothing superintelligent ever.
19:45Cuz by definition, you cannot predict at
19:48that level.
19:50Because by definition of it being
19:51superintelligent, it will make its own
19:52mind up. By definition, if it was
19:55something you could predict, you would
19:56be operating at the same level of
19:58intelligence, violating our assumption
20:00that it is smarter than you.
20:02If I'm playing chess with super
20:03intelligence and I can predict every
20:05move, I'm playing at that level. It's
20:07kind of like my French bulldog trying to
20:08predict
20:10exactly what I'm thinking and what I'm
20:12going to do. That's a good cognitive
20:13gap. And it's not just he can predict
20:15you going to work, you coming back, but
20:17he cannot understand why you doing a
20:18podcast. That is something completely
20:20outside of his model of the world.
20:25Yeah, he doesn't even know that I go to
20:26work. He just sees that I leave the
20:27house and doesn't know where I go.
20:30By food for him. What's the most
20:32persuasive argument against
20:34your own perspective here? That we will
20:37not have unemployment due to advanced
20:39technology?
20:41That there won't be this
20:43French bulldog human gap in
20:46understanding and
20:49I guess like power and control.
20:53So, some people think that we can
20:55enhance human minds either through
20:57combination with hardware, so something
20:59like Neuralink, or through genetic
21:02engineering to where we make smarter
21:04humans.
21:06Yeah. It may give us a little more
21:09intelligence. I don't think we are still
21:11competitive in biological form with
21:13silicon form. Silicon substrate is much
21:16more capable for intelligence. It's
21:18faster. It's more resilient, more energy
21:21efficient in many ways. Which is what
21:23computers are made out of the brain.
21:25Yeah.
21:26So, I don't think we can keep up just
21:28with improving our biology. Some people
21:31think maybe, and this is very
21:32speculative, we can upload our minds
21:35into computers.
21:36So, scan your brain, connectome of your
21:39brain, and have a simulation running on
21:42a computer and you can speed it up, give
21:44it more capabilities. But to me, that
21:46feels like you no longer exist. We just
21:48created software by different means and
21:50now you have AI based on biology and AI
21:54based on some other forms of training.
21:57You can have evolutionary algorithms.
21:59You can have many paths to reach AGI.
22:01But at the end, none of them are humans.
22:04I have a another date here, which is
22:092030.
22:11What's your prediction for 2030? What
22:13will the world look like?
22:15So, we probably will have
22:17humanoid robots with enough flexibility,
22:20dexterity to compete with humans in all
22:23domains, including plumbers.
22:25We can make artificial plumbers.
22:28Not the plumbers. We That was That felt
22:30like the last
22:32bastion of
22:33human employment. So, 2030, 5 years from
22:36now, humanoid robots So, many of the
22:38companies, the leading companies,
22:39including Tesla, are developing humanoid
22:41robots
22:42at light speed and they're getting
22:43increasingly more effective. And these
22:46humanoid robots will be able to move
22:47through physical space,
22:49for you know, make an omelet,
22:52do anything humans can do, but obviously
22:54have
22:56be connected to AI as well.
22:58So, they can think, talk,
23:00Like they're controlled by AI. They're
23:02always connected to the network, so they
23:04are already dominating in many ways.
23:08Our world will look remarkably different
23:11when humanoid robots are functional and
23:13effective. Because that's really when,
23:16you know, I start to think, "Crap." Like
23:18the combination of intelligence and
23:21physical ability
23:23is really really doesn't leave much,
23:26does it, for
23:27us um
23:29human beings.
23:31Not much. So, today, if you have
23:33intelligence through internet, you can
23:34hire humans to do your bidding for you.
23:36You can pay them in Bitcoin, so you can
23:38have bodies, just not directly
23:41controlling them. So, it's not a huge
23:43game changer to add direct control of
23:46physical bodies. Intelligence is where
23:48it's at. The important component is
23:50definitely higher ability to optimize,
23:53to solve problems, to find patterns
23:55people cannot see.
23:57And then by 2045,
24:01I guess the world looks even even more
24:03um
24:05which is 20 years from now. So, if it's
24:07still around, If it's still around,
24:09>> Ray Kurzweil predicts that that's the
24:11year for the singularity. That's the
24:13year where progress becomes so fast, so
24:16this AI doing science and engineering
24:19work makes improvements so quickly we
24:21cannot keep up anymore. That's the
24:23definition of singularity, point beyond
24:25which we cannot see, understand,
24:28predict.
24:29See, understand, predict the
24:31intelligence itself or
24:33What is happening in the world? The
24:34technology is being developed. So, right
24:36now, if I have an iPhone, I can look
24:38forward to a new one coming out next
24:40year and I'll understand it has slightly
24:42better camera. Imagine now this process
24:45of researching and developing this phone
24:47is automated. It happens every 6 months,
24:50every 3 every month, week, day, hour,
24:53minute, second.
24:54You cannot keep up with
24:5630 iterations of iPhone in 1 day. You
24:59don't understand what capabilities it
25:01has,
25:02what
25:04proper controls are. It just escapes
25:06you. Right now, it's hard for any
25:08researcher in AI to keep up with the
25:10state of the art. While I was doing this
25:13interview with you, a new model came out
25:15and I will no longer know what the state
25:17of the art is.
25:18Every day, as a percentage of total
25:20knowledge, I get dumber.
25:22I may still know more because I keep
25:23reading, but as a percentage of overall
25:26knowledge, we all getting dumber.
25:29And when you take it to extreme values,
25:33you have zero knowledge, zero
25:34understanding of the world around you.
25:37Some of the arguments against this
25:39eventuality are that when you look at
25:41other technologies like the Industrial
25:43Revolution, people just found new ways
25:46to
25:47to work and new careers that we could
25:50never have imagined at the time were
25:51created.
25:52How do you respond to that in a world of
25:54super intelligence?
25:56It's a paradigm shift. We always had
25:58tools, new tools which allowed some job
26:01to be done more efficiently. So, instead
26:02of having 10 workers, you could have two
26:04workers and eight workers had to find a
26:07new job. And there was another job. Now
26:09you can supervise these workers or do
26:11something cool. If you creating a meta
26:15invention, you inventing intelligence,
26:17you inventing a worker, an agent, then
26:20you can apply that agent to the new job.
26:23There is not a job which cannot be
26:25automated. That never happened before.
26:28All the inventions we previously had
26:30were kind of
26:31a tool for doing something. So, we
26:33invented fire. Huge game changer. But
26:36that's it. It stops with fire. We invent
26:39a wheel. Same idea. Huge implications,
26:42but wheel itself is not an inventor.
26:45Here we are inventing
26:47a replacement for human mind, a new
26:50inventor capable of doing new
26:52inventions. It's the last invention we
26:54ever have to make. At that point, it
26:56takes over and the process of doing
26:58science, research, even ethics research,
27:02morals, all that is automated at that
27:04point.
27:06Do you sleep well at night? Really well.
27:09Even though you you spent the last 15,
27:1220 years of your life working on AI
27:14safety and it's suddenly
27:16among us in a in a way that I don't
27:19think anyone could have predicted 5
27:20years ago. When I say among us, I really
27:21mean that the amount of funding and
27:23talent that is now focused on reaching
27:26super intelligence faster has made it
27:28feel more inevitable and more
27:30soon
27:32than any of us could have possibly
27:34imagined.
27:35We as humans have this built-in bias
27:37about not thinking about really bad
27:39outcomes and things we cannot prevent.
27:42So, all of us are dying.
27:44Your kids are dying, your parents are
27:46dying, everyone's dying, but you still
27:48sleep well, you still go on with your
27:50day. Even 95-year-olds are still doing
27:53games and playing golf and whatnot, cuz
27:56we have this ability to not think about
27:59the worst outcomes, especially if we
28:01cannot actually modify the outcome. So,
28:04that's the same
28:05infrastructure being used for this.
28:07Yeah, there is
28:09humanity level
28:11death-like event. We happening to be
28:14close to it probably, but unless I can
28:18do something about it, I I can just keep
28:21enjoying my life. In fact, maybe knowing
28:24that you have limited amount of time
28:25left gives you more reason to have a
28:27better life. You cannot waste any.
28:30And that's the survival trait of
28:32evolution, I guess, because those of my
28:34ancestors that spent all their time
28:35worrying
28:36wouldn't have spent enough time having
28:38babies and hunting to survive.
28:40>> Suicidal ideation. People who really
28:42start thinking about how horrible the
28:43world is usually escape pretty soon.
28:46Mhm.
28:51One of the You co-authored this paper
28:54um analyzing the key arguments people
28:56make against the importance of AI
28:57safety.
28:58And one of the arguments in there is
29:00that there's other things that are of
29:02bigger importance right now. It might be
29:04world wars, it could be nuclear
29:05containment, it could be other things.
29:07There's other things that the
29:08governments and podcasters like me
29:10should be talking about that are more
29:11important. What's your rebuttal to that
29:14argument?
29:15>> So, super intelligence is a meta
29:17solution. If we get super intelligence
29:20right, it will help us with climate
29:22change, it will help us with wars, it
29:24can solve all the other existential
29:26risks. If we don't get it right, it
29:30dominates. If climate change will take
29:32100 years to boil us alive and super
29:35intelligence kills everyone in five, I
29:37don't have to worry about climate
29:38change. So, either way, either it solves
29:41it for me or it's not an issue.
29:44So, you think it's the most important
29:45thing to be working on? Without
29:47question, there is nothing more
29:48important than getting this right.
29:54And I know everyone says it. You take
29:56any class but you take English
29:57professor's class and he tells you this
29:59is the most important class you'll ever
30:00take. But
30:02you can see the meta level differences
30:05with this one.
30:07Another argument in that paper is that
30:09we will be in control and that the
30:11danger is not AI.
30:13This particular argument asserts that AI
30:14is just a tool. Humans are the real
30:16actors that present danger and we can
30:19always maintain control by simply
30:21turning it off. Can't we just pull the
30:23plug out? I see that every time we have
30:24a conversation on the show about AI,
30:26someone says can't we just unplug it?
30:27Yeah, I get those comments on every
30:29podcast I make and I always want to like
30:31get in touch with the guy and say this
30:33is brilliant. I never thought of it.
30:35We're going to write a paper together
30:36and get a Nobel Prize for it. This is
30:38like let's do it.
30:40Because it's so silly. Like can you turn
30:42off a virus? You have a computer virus
30:43you don't like. You turn it off.
30:46How about Bitcoin? Turn off Bitcoin
30:47network.
30:48Go ahead. I'll wait.
30:50This is silly. Those are distributed
30:51systems. You cannot turn them off and on
30:54top of it they're smarter than you. They
30:55made multiple backups. They predicted
30:58what you're going to do. They will turn
30:59you off before you can turn them off.
31:02The idea that we will be in control
31:05applies only to pre super intelligence
31:08levels. Basically what we have today.
31:09Today humans with AI tools are
31:12dangerous. They can be hackers,
31:13malevolent actors. Absolutely. But the
31:16moment super intelligence becomes
31:18smarter, dominates, they no longer be
31:20important part of that equation. It is
31:22the higher intelligence I'm concerned
31:24about, not the human who
31:27may add additional malevolent payload
31:29but at the end still doesn't control it.
31:32It is tempting
31:35to
31:36follow your the next argument that I saw
31:38in that paper which basically says
31:39listen
31:40this is inevitable.
31:42So there's no point fighting against it
31:44because there's really no hope here. So
31:46we should probably give up even trying
31:48and be faithful that it will work itself
31:50out.
31:51Because everything you've said sounds
31:53really inevitable.
31:54And if with China working on it, I'm
31:56sure Putin's got some secret division.
31:57I'm sure Iran are doing some bits and
31:59pieces. Every European country is trying
32:02to get ahead of AI. The United States is
32:04leading the way.
32:05So it's it's inevitable. So we probably
32:08should just have faith and pray.
32:11Praying is always good but incentives
32:13matter.
32:14If you
32:15looking at what drives these people. So
32:18yes, money is important. So there is a
32:20lot of money in that space and so
32:22everyone's trying to be there and
32:24develop this technology. But if they
32:26truly understand the argument, they
32:28understand that you will be dead.
32:31No amount of money will be useful to
32:32you.
32:33Then incentives switch. They would want
32:35to not be dead. A lot of them are young
32:37people, rich people. They have their
32:39whole lives ahead of them. I think they
32:41would be better off not building
32:43advanced super intelligence,
32:45concentrating on narrow AI tools for
32:48solving specific problems. Okay, my
32:50company cures breast cancer. That's all.
32:53We make billions of dollars. Everyone's
32:54happy. Everyone benefits.
32:57It's a win.
32:59We are still in control today. It's not
33:01over until it's over. We can decide not
33:04to build general super intelligences.
33:07I mean the United States might be able
33:09to conjure up enough enthusiasm for
33:12that. But if the United States doesn't
33:14build general super intelligences, then
33:16China are going to have the big
33:17advantage, right?
33:18So right now at those levels, whoever
33:21has more advanced AI has more advanced
33:23military. No question. We see it with
33:25existing conflicts. But the moment you
33:27switch to super intelligence and control
33:30super intelligence, it doesn't matter
33:31who builds it, us or them. And if they
33:34understand this argument, they also
33:36would not build it. It's a mutually
33:38assured destruction on both ends.
33:41Is this technology different than say
33:43nuclear weapons which require a huge
33:45amount of investment and you have to
33:47like enrich the uranium and you need
33:51billions of dollars potentially to even
33:54build a nuclear weapon.
33:56But it feels like this technology is
33:58much cheaper to get to super
34:00intelligence potentially or at least it
34:02will become cheaper. I wonder if it's
34:04possible that some some guy, some
34:06startup is going to be able to build
34:08super intelligence in
34:10you know, a couple of years without the
34:11need of
34:12you know, billions of dollars of compute
34:14or or electricity power. That's a great
34:16point. So every year it becomes cheaper
34:18and cheaper to train sufficiently large
34:20model. If today it would take a trillion
34:23dollars to build super intelligence,
34:24next year it could be 100 billion and so
34:27on. At some point a guy in a laptop
34:29could do it.
34:31But you don't want to wait four years to
34:33make it affordable. So that's why so
34:35much money is pouring in. Somebody wants
34:37to get there this year and lock in all
34:39the winnings. Litecoin level award.
34:43So in that regard they're both very
34:45expensive projects like Manhattan level
34:48projects. Which is the nuclear bomb
34:50project.
34:51>> Right.
34:51The difference between the two
34:53technologies is that nuclear weapons are
34:55still tools.
34:57Some dictator, some country, someone has
35:00to decide to use them, deploy them.
35:03Whereas super intelligence is not a is
35:05not a tool. It's an agent.
35:07It makes its own decisions and no one is
35:09controlling it. I cannot take out this
35:10dictator and now super intelligence is
35:12safe.
35:14So that's a fundamental difference to
35:15me.
35:17But if you're saying that it is going to
35:18get
35:19incrementally cheaper like I think it's
35:21Moore's law, isn't it? The technology
35:22gets cheaper. It does.
35:24Then there is a future where some guy on
35:27his laptop is going to be able to create
35:28super intelligence without
35:30oversight or regulation or employees,
35:32etc. Yeah, that's why a lot of people
35:34suggesting we need to build something
35:36like um
35:38surveillance planet where you are
35:42monitoring who's doing what and you're
35:44trying to prevent people from doing it.
35:46Do I think it's feasible? No. At some
35:48point it becomes so affordable and so
35:50trivial that it just will happen. But at
35:53this point we're trying to get more
35:54time. We don't want it to happen in five
35:56years. We want it to happen in 50 years.
36:01I mean that's not very hopeful. Depends
36:03on how old you are.
36:05Depends on how old you are.
36:08I mean
36:09if you're saying that
36:10you believe in the future people will be
36:12able to make super intelligence
36:14without the resources that are required
36:16today, then it is just a matter of time.
36:18Yeah, but so will be true for many other
36:21technologies. We're getting much better
36:22in synthetic biology where today someone
36:25with a bachelor's degree in biology can
36:27probably create a new virus.
36:29This will also become cheaper. Other
36:31technologies like that. So we are
36:34approaching a point where it's very
36:36difficult to make sure no technological
36:39breakthrough is the last one. So
36:42essentially in many directions we have
36:45this
36:46pattern of making it easier in terms of
36:49resources, in terms of intelligence to
36:51destroy the world.
36:52If you look at I don't know, 500 years
36:55ago, the worst dictator with all the
36:57resources could kill couple million
36:59people. He couldn't destroy the world.
37:01Now we know nuclear weapons we can blow
37:03up the whole planet multiple times over.
37:06Synthetic biology we saw with COVID, you
37:08can very easily create a combination
37:12virus which impacts billions of people.
37:15And all those things becoming easier to
37:17do.
37:18In the near term you talk about
37:20extinction being a real risk, human
37:21extinction being a real risk. Of all the
37:23the pathways to human extinction that
37:25you think are
37:27most likely, what what is the leading
37:29pathway? Because I know you talk about
37:31there being some issue pre-deployment of
37:33these AI tools like you know, someone
37:35makes a mistake when they're
37:38designing a model or other issues
37:41post-deployment. When I say
37:42post-deployment, I mean once the chat
37:44GPT or something, an
37:46agent is released into the world and
37:47someone hacking into it and changing it
37:49and reprogram reprogramming it to be
37:51malicious. Of all these potential paths
37:54to human extinction, which one do you
37:56think is the highest probability?
37:59So I can only talk about the ones I can
38:01predict myself. So I can predict even
38:03before we get a super intelligence,
38:05someone will create a very advanced
38:06biological tool, create a novel virus
38:09and that virus gets everyone or most
38:11everyone.
38:12I can
38:13envision it. I can understand the
38:15pathway. I can say that. So just
38:17assuming on that then, that would be
38:19using an AI to make a virus and then
38:21releasing it. Yeah.
38:22And would that be
38:24intentional or
38:26There is a lot of psychopaths, a lot of
38:29terrorists, a lot of doomsday cults.
38:31We've seen historically again, they
38:33tried to kill as many people as they
38:35can. They usually fail. They kill
38:36hundreds of thousands.
38:38But if they get technology to kill
38:39millions or billions, they would do that
38:41gladly.
38:44The point I'm trying to emphasize is
38:47that it doesn't matter what I can come
38:48up with. I am not a malevolent actor
38:51you're trying to defeat here. It's the
38:53super intelligence which can come up
38:54with completely novel ways of doing it.
38:57Again, you brought up example of your
38:59dog.
39:01Your dog cannot understand all the ways
39:03you can take it out.
39:06It can maybe think you'll bite it to
39:08death or something. But that's all.
39:10Whereas you have
39:12infinite supply of resources.
39:15So if I asked your dog exactly how
39:18you're going to take it out, it would
39:19not give you a meaningful answer. It can
39:21talk about biting.
39:23And this is what we know. We know
39:25viruses. We experienced viruses. We can
39:27talk about them. But what
39:31an AI system capable of doing novel
39:33physics research can come up with is
39:35beyond me.
39:37One of the things that I think most
39:38people don't understand is how little we
39:40understand about how these AIs are
39:43actually working. Cuz one would assume,
39:45you know, with computers, we kind of
39:46understand how a computer works. We we
39:48know that it's doing this and then this
39:49and it's running on code. But,
39:52from reading your work, you describe it
39:54as being a black box. We actually So, in
39:57the context of something like ChatGPT or
39:59an AI we know, you're telling me that
40:01the people that have built that tool
40:02don't actually know
40:04what's going on inside there.
40:07That's exactly right. So, even people
40:08making those systems have to run
40:11experiments on their product to learn
40:13what it's capable of. So, they train it
40:16by giving it all of data, let's say all
40:18of internet text.
40:20They run it on a lot of computers to
40:22learn patterns in that text. And then we
40:25start experimenting with that model. Oh,
40:27do you speak French? Or, can you do
40:29mathematics? Or, are you lying to me
40:31now? And so, maybe it takes a year to
40:34train it and then 6 months to get some
40:37fundamentals about what it's capable of.
40:40Some safety overhead.
40:43But, we still discover new capabilities
40:45in old models. If you ask a question in
40:48a different way, it becomes smarter.
40:51So, it's
40:52no longer
40:54engineering how it was the first 50
40:56years where someone was a knowledge
40:58engineer programming an expert system AI
41:01to do specific things. It's a science.
41:03We are creating this artifact, growing
41:06it. It's like a alien plant. And then we
41:09study it to see what it's doing.
41:11And just like with plants, we don't have
41:13100% accurate knowledge of biology.
41:16We don't have full knowledge here. We
41:17kind of know some patterns. We know,
41:20okay, if we add more compute, it gets
41:22smarter most of the time. But,
41:24nobody can tell you precisely what the
41:26outcome is going to be given a set of
41:29inputs.
41:31I've watched so many entrepreneurs treat
41:32sales like a performance problem. When
41:34it's often down to visibility. Because
41:36when you can't see what's happening in
41:38your pipeline, what stage each
41:40conversation is at, what's stalled,
41:42what's moving, you can't improve
41:43anything. And you can't close the deal.
41:46Our sponsor, Pipedrive, is the number
41:48one CRM tool for small to medium
41:50businesses. Not just a contact list, but
41:53an actual system that shows your entire
41:55sales process end-to-end. Everything
41:57that's live, what's lagging, and the
41:59steps you need to take next. All of your
42:01teams can move smarter and faster. Teams
42:04using Pipedrive are on average closing
42:06three times more deals than those that
42:08aren't. It's the first CRM made by sales
42:11people for sales people that over
42:13100,000 companies around the world rely
42:16on, including my team who absolutely
42:18love it. Give Pipedrive a try today by
42:21visiting pipedrive.com/ceo.
42:24And you can get up and running in a
42:25couple of minutes with no payment
42:27needed. And if you use this link, you'll
42:29get a 30-day free trial. What do you
42:32make of OpenAI and Sam Altman and what
42:35they're doing?
42:36And obviously, you're aware that one of
42:38the co-founders, was it um This was Ilya
42:40Sutskever?
42:42Ilya yeah. Ilya left and he started a a
42:44new company called Superintelligence
42:46Safety. Superintelligence Safety.
42:48>> AI safety wasn't challenging enough, he
42:50decided to just jump right to the hard
42:52problem.
42:54As an onlooker, when you see that people
42:57are leaving OpenAI to to start
43:00superintelligent safety companies,
43:04what was your read on that situation?
43:06So, a lot of people who worked with Sam
43:10said that maybe he's not the most direct
43:13person in terms of being honest with
43:15them and they had concerns about his
43:18views on safety.
43:20That's part of it. So, they wanted more
43:22control, they wanted more concentration
43:25on safety. But, also it seems that
43:27anyone who leaves that company and
43:29starts a new one gets a $20 valuation
43:32just for having it started. You don't
43:34have a product, you don't have
43:35customers, but
43:36if you want to make many billions of
43:38dollars, just do that. So, it seems like
43:41a very rational thing to do for anyone
43:43who can.
43:44So, I'm not surprised that there is a
43:46lot of attrition.
43:48Meeting him in person, he's super nice,
43:51very smart.
43:53Absolutely
43:55perfect public interface. You see him
43:57testify in the Senate, he says the right
44:00thing to the senators. You see him talk
44:02to the investors, they get the right
44:04message.
44:05But, if you look at what people who know
44:07him personally are saying,
44:10it's probably not the right person to be
44:13controlling a project of that impact.
44:17Why?
44:19He puts safety second.
44:23Second to
44:25winning this race to superintelligence,
44:27being the guy who created God and
44:29controlling light cone of the universe.
44:31He's worse.
44:34Do you suspect that's what he's driven
44:35by is by the the legacy of being an
44:37impactful person that did a
44:41remarkable thing versus
44:43the consequence that that might have on
44:45for society?
44:46Because it's interesting that his his
44:47other startup is Worldcoin, which is
44:49basically a platform to create universal
44:51basic income. I a platform to give us
44:54income in a world where
44:57people don't have jobs anymore. So, on
44:58one hand you're creating an AI company,
44:59on the other hand you're creating a
45:00company that is preparing for people to
45:02not have employment.
45:05It also has other
45:07properties. It keeps track of everyone's
45:10biometrics.
45:12It
45:13keeps you in charge of a world's
45:15economy, world's wealth. They are
45:16retaining a large portion of Worldcoins.
45:20So, I I think it's kind of very
45:23reasonable part to integrate with world
45:26dominance. If you have a
45:28superintelligence system and you control
45:30money,
45:32you're doing well.
45:36Why would someone want world dominance?
45:40People have different levels of
45:41ambition. Then you are very young person
45:43with billions of dollars, fame, you
45:45start looking for more ambitious
45:47projects. Some people want to go to
45:49Mars, others want to control light cone
45:51of the universe.
45:53What What did you say, light cone of the
45:55universe?
45:55>> Light cone. So, every part of the
45:58universe light can reach from this
46:00point, meaning anything accessible you
46:01want to grab and
46:03bring into your control. You think Sam
46:05Altman wants to control
46:07every part of the universe?
46:10I I suspect he might, yes.
46:12Hmm.
46:13It doesn't mean he doesn't want a side
46:15effect of it being a very beneficial
46:17technology which makes all the humans
46:19happy.
46:20Happy humans are good for control.
46:24If you had to guess
46:27what the world looks like in
46:302100,
46:32if you had to guess,
46:35it's either
46:36free of human existence or it's
46:39completely not comprehensible to someone
46:41like us.
46:44It's one of those extremes. So, there's
46:46either no humans It's basically the
46:48world is destroyed or it's so different
46:51that I cannot envision those
46:54predictions.
46:56What can be done to turn this ship to a
47:00more certain positive outcome at this
47:02point?
47:04Is Is there still things that we can do
47:06or is it too late? So, I believe in
47:08personal self-interest. If people
47:11realize that doing this thing is really
47:13bad for them personally, they will not
47:15do it. So, our job is to convince
47:16everyone with any power in this space
47:19creating this technology working for
47:20these companies,
47:22they are doing something very bad
47:24for them. Not just forget about eight
47:27billion people you're experimenting on
47:28with no permission,
47:30no consent, you will not be happy with
47:33the outcome. If we can get everyone to
47:35understand that's the default, and it's
47:37not just me saying it. You had Geoff
47:39Hinton on
47:40him. Nobel Prize winner, founder of the
47:42whole machine learning space. He says
47:44the same thing. Bengio, dozens of
47:46others, top scholars. We had a statement
47:49about dangers of AI signed by thousands
47:51of scholars, computer scientists. This
47:54is basically what we think right now and
47:57we need to make it a universal. No one
47:59should disagree with this. And then, we
48:01may actually make good decisions about
48:04what technology to build. It doesn't
48:06guarantee long-term safety for humanity,
48:09but it means we're not trying to get
48:11there as soon as possible to the worst
48:12possible outcome.
48:14And do you Are you hopeful that that's
48:16even possible?
48:18I want to try. We have no choice but to
48:21try.
48:22And what would need to happen and who
48:24would need to act? What is it government
48:25legislation? Is it
48:27Unfortunately, I don't think making it
48:29illegal is sufficient. There are
48:31different jurisdictions, there is, you
48:33know, loopholes. And what are you going
48:35to do if somebody does it? You're going
48:37to fine them for destroying humanity?
48:38Like very steep fines for it? Like what
48:40are you going to do? It's not
48:41enforceable. If they do create it, now
48:44the superintelligence is in charge. So,
48:46the judicial system we have is not
48:48impactful. And all the punishments we
48:51have are designed for punishing humans.
48:53Prisons, capital punishment doesn't
48:55apply to AI.
48:56Here's the problem I have is when I have
48:57these conversations, I never feel like I
48:59walk away with
49:03hope that something's going to go well.
49:05And what I mean by that is I never feel
49:07like I walk away with clear some kind of
49:10a clear set of actions that can course
49:12correct what might happen here. So, what
49:15should What should I do? What should the
49:17person sat at home listening to this do?
49:19You You talked to a lot of people who
49:21are building this technology. Mhm.
49:24Ask them precisely to explain some of
49:28those things they claim to be
49:29impossible, how they solved it or going
49:31to solve it before they get to where
49:33they're going. Do you know, I don't
49:35think Sam Altman wants to talk to me.
49:37I don't know. He seems to go on a lot of
49:39podcasts. Maybe he does.
49:40>> wants to go on mine.
49:43I wonder why that is.
49:47I'd love to speak to him, but I don't I
49:48don't think he wants to
49:50I don't think he wants me to
49:54interview him. Have an open challenge.
49:56Maybe money is not the incentive, but
49:58whatever attracts people like that,
50:00whoever can convince you that it's
50:02possible to control and make safe super
50:04intelligence gets the prize. They come
50:07on your show and prove their case.
50:10Anyone. If no one claims the prize or
50:12even accepts the challenge after a few
50:14years, maybe we don't have anyone with
50:16solutions.
50:17We have companies valued again at
50:20billions and billions of dollars working
50:21on safe super intelligence.
50:24We haven't seen their output yet.
50:29Yeah, I'd like to speak to Ilya as well
50:31cuz I know he's he's working on safe
50:32super intelligence, so
50:34Notice the pattern too. If you look at
50:36history of AI safety organizations
50:39or
50:40departments within companies,
50:42they usually start well, very ambitious,
50:44and then they fail and disappear.
50:47So,
50:48OpenAI had super intelligence alignment
50:51team.
50:52The day they announced it, I think they
50:54said we're going to solve it in 4 years.
50:56Like half a year later they canceled the
50:58team.
50:59And there is dozens of similar examples.
51:02Leading
51:04a perfect safety for super intelligence,
51:06perpetual safety as it keeps improving,
51:08modifying, interacting with people.
51:11You're never going to get there. It's
51:12impossible.
51:14There is a big difference between
51:16difficult problems in computer science
51:18and be complete problems and impossible
51:20problems. And I think control indefinite
51:23control of super intelligence is such a
51:25problem. So, what's the point trying
51:27then if it's impossible? Well, I'm
51:29trying to prove that it is specifically
51:31that. Once we establish something is
51:32impossible, fewer people will waste
51:34their time claiming they can do it and
51:36find looking for money. So many people
51:38go and give me a billion dollars in 2
51:40years and I'll solve it for you.
51:42Well, I don't think you will.
51:44But people aren't going to stop striving
51:46towards it. So, if there's no attempts
51:48to
51:49make it safe and there's more people
51:51increasingly striving towards it, then
51:53it's inevitable. But it changes what we
51:55do. If we know that it's impossible to
51:57make it right, to make it safe, then
51:59this direct path of just build it as
52:01soon as you can become suicide mission.
52:03Hopefully fewer people will pursue that.
52:06They may go in other directions like
52:08again,
52:09I'm a scientist, I'm an engineer. I love
52:11AI. I love technology. I use it all the
52:13time. Build useful tools. Stop building
52:16agents.
52:17Build narrow super intelligence, not a
52:19general one. I'm not saying you
52:21shouldn't make billions of dollars. I
52:22love billions of dollars.
52:25But
52:26don't kill everyone, yourself included.
52:33They don't think they're going to
52:34though.
52:35Then tell us why.
52:37I hear things about intuition. I hear
52:39things about we'll solve it later. Tell
52:41me specifically in scientific terms.
52:42Publish a peer-reviewed paper explaining
52:45how you're going to control super
52:46intelligence.
52:48It's strange. It's strange to it's
52:49strange to even bother if there was even
52:51a 1% chance of human extinction. It's
52:53strange to do something. Like if there
52:54was a 1% chance Someone told me there
52:56was a 1% chance that if I got in a car,
53:00I might not I might not be alive, I
53:02would not get in the car. If you told me
53:03there was a 1% chance that if I drank
53:05whatever liquid is in this cup right
53:07now, I might die, I would not drink the
53:08liquid. Even if there was
53:12a billion dollars
53:14if I survived. So, the 99% chance is I
53:16get a billion dollars, the 1% is I die.
53:17I wouldn't drink it. I wouldn't take the
53:19chance. It's worse than that. Not just
53:21you die, everyone dies. Yeah. Yeah. Now,
53:25would we let you drink it at any odds?
53:27That's for us to decide. You don't get
53:29to make that choice for us.
53:31To get consent from human subjects,
53:34you need them to comprehend what they
53:36are consenting to.
53:38If those systems are unexplainable,
53:40unpredictable, how can they consent?
53:42They don't know what they are consenting
53:43to. Mhm.
53:44So, it's impossible to get consent by
53:47definition.
53:48So, this experiment can never be run
53:49ethically.
53:50By definition, they are doing unethical
53:53experimentation on human subjects. Do
53:55you think people should be protesting?
53:57There are people protesting. There is
53:59Stop AI. There is Pause AI. They block
54:01offices of OpenAI. They do it weekly,
54:04monthly. There are quite a few actions
54:06and they're recruiting new people. You
54:08think more people should be protesting?
54:10Do you think that's an effective
54:11solution?
54:12If you can get it to a large enough
54:14scale to where majority of population is
54:17participating, it would be impactful. I
54:19don't know if they can scale from
54:20current numbers to that, but I support
54:23everyone trying everything peacefully
54:25and legally.
54:27And for the for the person listening at
54:28home, what should they what should they
54:30be doing? What what
54:31cuz they they don't want to feel
54:32powerless. None of us want to feel
54:34powerless.
54:35So, it depends on what scale we are
54:37asking about time scale. I was saying
54:40like this year your kid goes to college,
54:41what major to pick? Should they go to
54:43college at all? Yeah. Should you switch
54:45jobs? Should you go into certain
54:47industries? Those questions we can
54:48answer. We can talk about immediate
54:50future.
54:51What should you do in 5 years with
54:55this being created? For an average
54:56person, not much. Just like they can't
54:59influence World War III nuclear
55:01holocaust, anything like that. It's not
55:04something anyone's going to ask them
55:06about.
55:07Today, if you want to be a part of this
55:09movement, yeah, join Pause AI, join Stop
55:12AI. Those are organizations currently
55:14trying to build up momentum to
55:17bring
55:18democratic powers to influence those
55:21individuals.
55:23So, in the meantime,
55:25not a huge amount. I was wondering if
55:26there there are any interesting
55:27strategies in the meantime. Like should
55:28I be thinking differently about my
55:31family, about I mean, you've got kids,
55:33right? You've got three kids. That I
55:35know about, yeah.
55:37How are you thinking about parenting in
55:39this world that you see around the
55:41corner? How are you thinking about what
55:42to say to them, the advice to give them,
55:43what they should be learning? So, there
55:45is general advice.
55:47I would say that there is the main that
55:48you should live your everyday as if it's
55:51your last.
55:52It's a good advice no matter what. If
55:53you have 3 years left or 30 years left,
55:55you live your best life.
55:57So,
55:59try to not do things you hate for too
56:01long.
56:03Do interesting things. Do impactful
56:05things.
56:06If you can do all that while helping
56:08people, do that.
56:10Simulation theory
56:12is a interesting sort of adjacent
56:14subject here because as computers begin
56:17to accelerate and get more intelligent
56:18and we're able to
56:21you know, do things with AI that we can
56:23never have imagined in terms of like
56:25imagine the worlds that we could create
56:27with virtual reality. I think it was
56:28Google that recently released
56:30what was it called? Um
56:33like the AI worlds. You take a picture
56:36and it generates a whole world. Yeah.
56:38Yeah. You can move through the world.
56:39I'll put it on the screen for people to
56:40see, but Google have released this
56:42technology which allows you, I think
56:43with a simple prompt actually, to make a
56:46three-dimensional world that you can
56:48then navigate through. And in that world
56:51it has memory. So, in the world if you
56:52paint on a wall and turn away, you look
56:54back, the wall
56:55>> Yeah, it's persistent. And when I saw
56:57that, I thought oh God, Jesus, bloody
56:58hell, this is
57:00this is like the foothills of being able
57:03to create a simulation that's
57:04indistinguishable from everything I see
57:06here. Right.
57:08That's why I think we are in one. That's
57:10exactly the reason. AI is getting to the
57:12level of creating human agents, human
57:15level agents, and virtual reality is
57:17getting to the level of being
57:19indistinguishable from ours.
57:20So, you think this is a simulation? I'm
57:22pretty sure we are in a simulation,
57:24yeah.
57:26For someone that isn't familiar with the
57:27simulation arguments, what are what are
57:29the first principles here that convince
57:31you that we are currently living in a
57:32simulation?
57:34So,
57:35you need certain technologies to make it
57:37happen. If you believe we can create
57:39human level AI, Yeah. and you believe we
57:42can create virtual reality as good as
57:43this in terms of resolution, haptics,
57:46whatever properties it has,
57:49then I commit right now, the moment this
57:51is affordable, I'm going to run billions
57:53of simulations of this exact moment
57:55making sure you are statistically in
57:57one.
57:59Say that last part again. You're going
58:01to run you're going to run I'm going to
58:03commit right now when it's very
58:04affordable. It's like 10 bucks a month
58:06to run it. I'm going to run a billion
58:08simulations of this interview.
58:11Why?
58:12Because statistically that means you are
58:14in one right now. The chance of you
58:16being in the real one is one in a
58:17billion.
58:19Okay. So,
58:21to make sure I'm clear on this, It's a
58:22retroactive placement. Yeah. So, the
58:24minute it's affordable,
58:26then
58:28you can run billions of them
58:30and they would feel and appear to be
58:31exactly like this interview right now.
58:33>> Right. So, assuming that AI has internal
58:37states, experiences, qualia. Some people
58:39argue that they don't. Some say they
58:41already have it. That's a separate
58:42philosophical question, but if we can
58:44simulate this, I will.
58:48Some people might misunderstand. You're
58:50not
58:51you're not saying that you will.
58:53You're saying that someone will. I can
58:55also do it. I don't mind.
58:58Okay. Of course, others will do it
59:00before I get there. If I'm getting it
59:01for $10, somebody got it for $1,000.
59:03That's not the point. If you have
59:05technology, we're definitely running a
59:07lot of simulations for research, for
59:10entertainment, games,
59:12all sorts of reasons.
59:14And the number of those greatly exceeds
59:16the number of real worlds we're in.
59:18Look at all the video games kids are
59:20playing. Every kid plays 10 different
59:22games. You know, billion kids in the
59:24world. So, there is 10 billion
59:26simulations in one real world. Mhm.
59:31Even more so when we think about
59:33advanced AI super intelligent systems.
59:35Their thinking is not like ours. They
59:37think in a lot more detail. They run
59:39experiments. So, running a detailed
59:42simulation of some problem at the level
59:45of creating artificial humans and
59:47simulating the whole planet would be
59:49something they'll do routinely.
59:51So, there is a good chance this is not
59:53me doing it for $10. It's a future
59:55simulation thinking about something in
59:58this world.
1:00:03So, it could be the case that
1:00:06a species of humans or a species of
1:00:09intelligence in some form got to this
1:00:12point where they could affordably run
1:00:16simulations that are indistinguishable
1:00:18from this
1:00:19and they decided to do it
1:00:21and this is it right now.
1:00:25And it would make sense that they would
1:00:26run simulations as experiments or for
1:00:28games or for entertainment. And also,
1:00:31when we think about time in the world
1:00:33that I'm in in this simulation that I
1:00:34could be in right now, time feels long
1:00:36relatively. You know, I have 24 hours in
1:00:38a day, but on there in their world it
1:00:41could be
1:00:43Time is relative. Relative, yeah. It
1:00:44could be a second. My whole life could
1:00:46be a millisecond in there.
1:00:48>> Right. You can change speed of
1:00:50simulations you're running for sure.
1:00:53So, your belief is that this is probably
1:00:55a simulation. Most likely. And there is
1:00:57a lot of agreement on that if you look
1:00:59again returning to religions. Every
1:01:00religion basically describes what? A
1:01:03super intelligent being
1:01:05an engineer, a programmer creating a
1:01:07fake world for testing purposes or for
1:01:11whatever. But, if you took the
1:01:13simulation hypothesis paper
1:01:15you go to jungle, you talk to primitive
1:01:18people, a local tribe and in their
1:01:20language you tell them about it.
1:01:23Go back two generations later. They have
1:01:25religion. That's basically what the
1:01:27story is.
1:01:29Religion, you know, it describes a
1:01:31simulation theory basically. Somebody
1:01:33creates
1:01:33>> So, by default that was the first theory
1:01:35we had and now with science more and
1:01:37more people are going like I'm giving it
1:01:39non-trivial probability. A few people as
1:01:41high as I am, but a lot of people give
1:01:44it some credence. What percentage are
1:01:45you at in terms of believing that we are
1:01:47currently living in a simulation? Very
1:01:49close to certainty.
1:01:52And what does that mean for
1:01:54the nature of your life? If you're close
1:01:56to 100% certain that we are currently
1:01:58living in a simulation
1:02:00does that change anything in your life?
1:02:02So, all the things you care about are
1:02:04still the same. Pain still hurts. Love
1:02:06still love, right? Like those things are
1:02:08not different, so it doesn't matter.
1:02:09They're still important. That's what
1:02:11matters.
1:02:12The
1:02:13little 1% difference is that I care
1:02:16about what's outside the simulation. I
1:02:17want to learn about it. I write papers
1:02:19about it. So, that's the only impact.
1:02:22And what do you think is outside of the
1:02:23simulation? I don't know.
1:02:26But, we can
1:02:27look at this world and derive some
1:02:30properties of the simulators.
1:02:32So, clearly brilliant engineer,
1:02:34brilliant scientist, brilliant artist.
1:02:37Not so good with morals and ethics.
1:02:40Room for improvement.
1:02:42In our view of what morals and ethics
1:02:44should be. Well, we we know there is
1:02:46suffering in the world. So, unless you
1:02:48think it's ethical to torture children,
1:02:51then
1:02:52I'm questioning your approach. But, in
1:02:55terms of incentives to create a positive
1:02:57incentive, you probably also need to
1:02:58create negative incentives. Suffering
1:03:00seems to be one of the negatives and
1:03:02incentives built into our design to stop
1:03:04me doing things I shouldn't do. So, like
1:03:06put my hand in a fire, it's going to
1:03:07hurt. But, it's all about levels, levels
1:03:10of suffering, right? So, unpleasant
1:03:12stimuli, negative feedback doesn't have
1:03:14to be at like negative infinity hell
1:03:17levels. You don't want to burn alive and
1:03:19feel it. You want to be like, "Oh, this
1:03:21is uncomfortable. I'm going to stop."
1:03:24It's interesting because we we assume
1:03:26that they don't have great moral morals
1:03:27and ethics, but we too would we take
1:03:30animals and cook them and eat them for a
1:03:32dinner and
1:03:33we also take conduct experiments on mice
1:03:35and rats
1:03:35>> But, to get university approval to
1:03:37conduct an experiment you submit a
1:03:39proposal and there is a panel of
1:03:41ethicists who would say, "You can't
1:03:43experiment on humans. You can't burn
1:03:45babies. You can't eat animals alive."
1:03:47All those things would be banned.
1:03:50In most parts of the world. Where they
1:03:52have ethical boards. Yeah. Cuz some
1:03:54places don't bother with it, so they
1:03:56have easier approval process.
1:03:59It's funny when you talk about the
1:04:00simulation theory, there's a there's an
1:04:02element of the conversation that makes
1:04:04life feel less meaningful in a weird
1:04:06way.
1:04:08Like
1:04:09I know it doesn't matter but whenever I
1:04:12have this conversation with people not
1:04:14on the podcast about are we living in a
1:04:16simulation
1:04:17you almost see a little bit of meaning
1:04:20come out of their life for a second and
1:04:22then they forget and then they carry on.
1:04:23But, the the the thought that this is a
1:04:25simulation almost
1:04:27posits that it's not important
1:04:30or that I I think humans want to believe
1:04:32that this is the highest level and we're
1:04:34that the most important and we're the
1:04:36it's all about us. We're quite
1:04:37egotistical by design.
1:04:40And yeah, I just a interesting
1:04:42observation I've always had when I have
1:04:43these conversations with people that it
1:04:44it seems to strip something out of their
1:04:45life. Do you feel religious people feel
1:04:48that way? They know there is another
1:04:50world and the one that matters is not
1:04:52this one. Do you feel they don't value
1:04:55their lives the same?
1:04:56I guess in some religions. Think um they
1:05:00think that this world is being created
1:05:01for them and that they are going to go
1:05:03to this heaven or or hell and that still
1:05:06puts them at the very center of it. But,
1:05:08it but if it's a simulation, you know,
1:05:10we could just be
1:05:12some computer game that a 4-year-old
1:05:14alien has is messing around with and
1:05:16while he's got some time to burn.
1:05:18But, maybe there is, you know, a test
1:05:21and there is a better simulation you go
1:05:23to and a worse one. Maybe there are
1:05:25different difficulty levels. Maybe you
1:05:27want to play it on a harder setting next
1:05:29time.
1:05:30I've just invested millions into this
1:05:33and become a co-owner of the company.
1:05:34It's a company called KetoneIQ and the
1:05:37story is quite interesting. I started
1:05:38talking about ketosis on this podcast
1:05:40and the fact that I'm very low carb,
1:05:42very very low sugar and my body produces
1:05:44ketones which have made me incredibly
1:05:46focused, have improved my endurance,
1:05:48have improved my mood and have made me
1:05:50more capable at doing what I do here.
1:05:51And because I was talking about it on
1:05:52the podcast, a couple of weeks later
1:05:55these showed up on my desk in my HQ in
1:05:57London, these little shots. And oh my
1:06:00god. The impact this had on my ability
1:06:03to articulate myself, on my focus, on my
1:06:06workouts, on my mood, on stopping me
1:06:08crashing throughout the day was so
1:06:11profound that I reached out to the
1:06:12founders of the company and now I'm a
1:06:14co-owner of this business. I highly
1:06:16highly recommend you look into this. I
1:06:18highly recommend you look at the science
1:06:19behind the product. If you want to try
1:06:21it for yourself, visit ketone.com/steven
1:06:24for 30% off your subscription order and
1:06:26you'll also get a free gift with your
1:06:28second shipment. That's
1:06:29ketone.com/steven.
1:06:32And I'm so honored that once again a
1:06:34company I own can sponsor my podcast.
1:06:37I've built companies from scratch and
1:06:39backed many more. And there's a blind
1:06:40spot that I keep seeing in early stage
1:06:42founders. They spend very little time
1:06:44thinking about HR. And it's not because
1:06:46they're reckless or they don't care,
1:06:48it's because they're obsessed with
1:06:50building their companies and I can't
1:06:51fault them for that. At that stage
1:06:52you're thinking about the product, how
1:06:54to attract new customers, how to grow
1:06:55your team, really how to survive. And HR
1:06:58slips down the list because it doesn't
1:07:00feel urgent. But, sooner or later it is.
1:07:03And when things get messy, tools like
1:07:05our sponsor today JustWorks go from
1:07:06being a nice-to-have to being a
1:07:08necessity. Something goes sideways and
1:07:10you find yourself having conversations
1:07:12you did not see coming. This is when you
1:07:13learn that HR really is the
1:07:15infrastructure of your company and
1:07:16without it things wobble. And JustWorks
1:07:18stops you learning this the hard way. It
1:07:20takes care of the stuff that would
1:07:22otherwise drain your energy and your
1:07:24time automating payroll, health
1:07:25insurance benefits and it gives your
1:07:27team human support at any hour. It grows
1:07:30with your small business from startup
1:07:32through to growth even when you start
1:07:34hiring team members abroad. So, if you
1:07:35want HR support that's there through the
1:07:37exciting times and the challenging
1:07:39times, head to justworks.com now. That's
1:07:42justworks.com.
1:07:44And do you think much about longevity? A
1:07:46lot, yeah. It's probably the second most
1:07:48important problem because if AI doesn't
1:07:50get us, that will.
1:07:52What do you mean? You're going to die of
1:07:54old age.
1:07:56Which is fine. That's not good. You want
1:07:58to die?
1:07:59I mean You don't have to. It's just a
1:08:01disease. We can cure it.
1:08:05Nothing stops you from living forever.
1:08:08As long as universe exists unless we
1:08:10escape the simulation.
1:08:12But, we wouldn't want a world where
1:08:13everybody could live forever, right?
1:08:15That would be Sure we do. Why? Who do
1:08:17you want to die?
1:08:19Well, I don't know. I
1:08:20I mean I say this because it's all I've
1:08:22ever known that people die, but wouldn't
1:08:23the world become pretty overcrowded if
1:08:25>> No, you stop reproducing if you live
1:08:27forever. You have kids because you want
1:08:28a replacement for you. If you live
1:08:30forever, you're like, "I'll have kids in
1:08:32a million years. That's cool." I'll go
1:08:34explore universe first.
1:08:36Plus, if you look at actual population
1:08:38dynamics outside of like one continent,
1:08:41we're all shrinking. We're not growing.
1:08:43This is crazy. It's crazy that the more
1:08:45rich people get, the less kids they they
1:08:47have which aligns with what you're
1:08:49saying and I do actually think I think
1:08:51if I'm going to be completely honest
1:08:52here, I think if I knew that I was going
1:08:54to live to 1,000 years old, there's no
1:08:57way I'd be having kids at 30. Right.
1:08:59It's a Biological clocks are based on
1:09:01terminal points. Whereas, if your
1:09:03biological clock is infinite, you'd be
1:09:05like one day.
1:09:07And you think that's close?
1:09:09Uh Being able to extend our lives?
1:09:11It's one breakthrough away. I think
1:09:13somewhere in our genome we have this
1:09:15rejuvenation loop and it's set to
1:09:17basically give us at most 120. I think
1:09:20we can reset it to something bigger.
1:09:23AI is probably going to accelerate that.
1:09:26That's one very important application
1:09:28area, yes, absolutely.
1:09:30So, maybe Brian Johnson's right when he
1:09:32says don't die.
1:09:33Now, he keeps saying to me, he's like,
1:09:35don't die now. Don't die ever. Because
1:09:38he's saying like don't die before we get
1:09:40to the technology.
1:09:40>> Right. Longevity escape velocity. You
1:09:42want to long live long enough to live
1:09:44forever. If at some point we
1:09:47every year of your existence add 2 years
1:09:50to your existence through medical
1:09:52breakthroughs, then you live forever.
1:09:53You just have to make it to that point
1:09:55of longevity escape velocity.
1:09:58And he thinks that longevity escape
1:10:00velocity, especially in the world of AI,
1:10:01is pretty
1:10:02is pretty
1:10:03is decades away, minimum, which means
1:10:06As soon as we fully understand human
1:10:08genome, I think we'll make amazing
1:10:10breakthroughs very quickly. Because we
1:10:12know some people have genes for living
1:10:14way longer. They have generations of
1:10:16people who are centenarians. So, if we
1:10:18can understand that and copy that or
1:10:20copy it from some animals which will
1:10:22live forever,
1:10:23we'll get there. Would you want to live
1:10:25forever?
1:10:27>> Reverse reverse the question. Let's say
1:10:29we lived forever and you ask me, do you
1:10:31want to die in 40 years? Why would I say
1:10:33yes? I don't know. Maybe
1:10:34>> You just used to the default. Yeah, I am
1:10:36used to the default. And nobody wants to
1:10:38die. Like, no matter how old you are,
1:10:39nobody goes, yeah, I want to die this
1:10:40year. Everyone's like, ah, I want to
1:10:41keep living.
1:10:43I wonder if life and everything would be
1:10:46less special if I lived for
1:10:4910,000 years. I wonder if going to
1:10:51Hawaii for the first time or, I don't
1:10:54know,
1:10:55a relationship. All of these things
1:10:57would be way less special to me if
1:10:59they were less scarce.
1:11:02And if that I just, you know, It could
1:11:03be individually less special, but there
1:11:05is so much more you can do. Right now,
1:11:07you can only make plans to do something
1:11:09for a decade or two. You cannot have an
1:11:11ambitious plan of working on this
1:11:13project for 500 years. Imagine
1:11:15possibilities open to you with infinite
1:11:17time in an infinite universe.
1:11:21Gosh.
1:11:22Well, you can't.
1:11:23>> Because it's exhausting, I guess. It's a
1:11:24big amount of time.
1:11:26I don't know about you, but I don't
1:11:28remember like 99% of my life in detail.
1:11:31I remember big highlights. So, even if I
1:11:33enjoyed Hawaii 10 years ago, I'll enjoy
1:11:35it again.
1:11:37Are you thinking about that really
1:11:38practically as as in terms of, you know,
1:11:40if in the same way that Brian Johnson is
1:11:42Brian Johnson is convinced that we're
1:11:43like maybe two decades away from being
1:11:45able to extend life. Are you thinking
1:11:47about that practically? And are you
1:11:48doing anything about it?
1:11:49>> Diet, nutrition. I try to think about
1:11:52investment strategies which pay out in
1:11:54the long years, yeah.
1:11:56Really? Yeah, of course. What do you
1:11:58mean of course? Of course Why wouldn't
1:11:59you if you think this is what's going to
1:12:01happen? You you should try that. So, if
1:12:03we get AI right, now, what happens to
1:12:05economy? We talked about Worldcoin. We
1:12:08talked about free labor.
1:12:10What's money? Is it now Bitcoin? Do you
1:12:12invest in that? Is there something else
1:12:14which becomes the only resource we
1:12:16cannot fake? So, those things are very
1:12:19important research topics. So, you're
1:12:20investing in Bitcoin, aren't you?
1:12:23Yeah.
1:12:26Because it's a It's the only scarce
1:12:28resource. Nothing else has scarcity.
1:12:31Everything else, if price goes up, will
1:12:33make more. I can make as much gold as
1:12:35you want given a proper price point.
1:12:38You cannot make more Bitcoin.
1:12:41Some people say Bitcoin is just this
1:12:42thing on a computer that we all agreed
1:12:43was valuable.
1:12:44>> Yeah, a thing on a computer.
1:12:48Remember?
1:12:49Okay, so, I mean, not investment advice,
1:12:53but investment advice. It's hilarious
1:12:55how that's one of those things where
1:12:56they tell you it's not, but you know it
1:12:58is immediately. There is a your call is
1:13:00important to us. That means your call is
1:13:02of zero importance. And investment is
1:13:04like that. Yeah, yeah, when they say no
1:13:06investment advice, it's definitely
1:13:07investment advice. Um, but it's not
1:13:09investment advice. Okay, so, you're
1:13:11bullish on Bitcoin because it's
1:13:13it can't be messed with.
1:13:15It is the only thing which we know how
1:13:18much there is
1:13:20in the universe. So, gold, there could
1:13:22be an asteroid made out of pure gold
1:13:24heading towards us, devaluing it.
1:13:27Well, also killing all of us, but
1:13:30Bitcoin, I know exactly the numbers. And
1:13:32even the 21 million is an upper limit.
1:13:35How many are lost? Passwords forgotten.
1:13:37I don't know what Satoshi's doing with
1:13:39his million.
1:13:40It's getting scarcer every day while
1:13:43more and more people are trying to
1:13:44accumulate it.
1:13:47Some people worry that it could be
1:13:48hacked with a supercomputer. A quantum
1:13:51computer can break that algorithm. There
1:13:53is
1:13:54strategies for switching to quantum
1:13:57resistant cryptography for that. And
1:13:59quantum computers are still kind of
1:14:01weak.
1:14:02Do you think there's any changes to my
1:14:04life that I should make
1:14:06following this conversation? Is there
1:14:07anything that I should do differently
1:14:09the minute I walk out of this door?
1:14:11I assume you already invest in Bitcoin
1:14:13heavily. Yes, I'm an an investor in
1:14:15Bitcoin. Is this financial advice?
1:14:17Uh, no, just you seem to be winning.
1:14:19Maybe it's your simulation. You're rich,
1:14:21handsome. You have famous people hang
1:14:24out with you like that's pretty good.
1:14:28Keep it up.
1:14:33Robin Hanson has a paper about how to
1:14:35live in a simulation, what you should be
1:14:36doing in it.
1:14:38And your goal is to do exactly that. You
1:14:40want to be interesting. You want to hang
1:14:41out with famous people so they don't
1:14:42shut it down. So, you are part of a part
1:14:45someone's actually watching on
1:14:46pay-per-view or something like that.
1:14:48Well, I don't know if you want to be
1:14:49watched on pay-per-view because then you
1:14:51would be the same
1:14:52>> Then they shut you down. If no one's
1:14:53watching, why would they play it?
1:14:57I'm saying you don't you want to fly
1:14:58under the radar? Don't you want to be
1:14:59the the guy just living a normal life
1:15:01that the the masters Those are NPCs.
1:15:03Nobody wants to be an NPC.
1:15:07Are you religious?
1:15:08Not in any traditional sense, but I
1:15:10believe in simulation hypothesis which
1:15:12has a super intelligent being. So,
1:15:14But you don't believe in the like,
1:15:16you know, the religious books.
1:15:18So, different religions. This religion
1:15:20will tell you don't work Saturday. This
1:15:22one, don't work Sunday.
1:15:24Don't eat pigs. Don't eat carbs. They
1:15:26just have local traditions on top of
1:15:28that theory. That's all it is. They're
1:15:29all the same religion. They all worship
1:15:31super intelligent being.
1:15:33They all think this world is not the
1:15:35main one.
1:15:37And they argue about which animal not to
1:15:39eat.
1:15:41Skip the local flavors. Concentrate on
1:15:43what do all the religions have in
1:15:45common?
1:15:46And that's the interesting part.
1:15:49They all think there is something
1:15:50greater than humans. Very capable,
1:15:52all-knowing, all-powerful. Then they run
1:15:54a computer game.
1:15:56For those characters in the game, I am
1:15:57that.
1:15:58I can change the whole world. I can shut
1:16:00it down. I know everything in the world.
1:16:05It's funny. I was thinking earlier on
1:16:06when we started talking about the
1:16:07simulation theory that there's there
1:16:09might be something in a in us that has
1:16:11been left from the creator, almost like
1:16:13a clue. Like a like an intuition.
1:16:16Cuz that's what we we tend to have
1:16:17through history. Humans have this
1:16:18intuition. Yeah. That all the things you
1:16:21said are true. That there's this
1:16:22somebody above and that We have
1:16:24generations of people who were
1:16:26religious, who believed God told them
1:16:28and was there and gave them books. And
1:16:31that has been passed on for many
1:16:32generations. This is probably one of the
1:16:34earliest generations not to have
1:16:36universal religious belief.
1:16:40What if those people are telling the
1:16:41truth?
1:16:42What if there's people there's people
1:16:43that say God came to them and said
1:16:44something. Imagine that. Imagine if that
1:16:45was part of the I'm looking at the news
1:16:47today. Something happened an hour ago
1:16:49and I'm getting different conflicting
1:16:51results. I can't even get with cameras,
1:16:53with drones, with like guy on Twitter
1:16:56there. I still don't know what happened.
1:16:58And you think 3,000 years ago we have
1:17:00accurate record of translations? No, of
1:17:03course not.
1:17:05You know, these conversations you have
1:17:06around AI safety.
1:17:08Do you think they make people feel good?
1:17:12I don't know if they feel good or bad,
1:17:13but people find it interesting. It's one
1:17:16of those topics where I can't have a
1:17:18conversation about different cures for
1:17:20cancer with an average person, but
1:17:22everyone has opinions about AI. Everyone
1:17:24has opinions about simulation. It's
1:17:26interesting that you don't have to be
1:17:28highly educated or a genius to
1:17:30understand those concepts.
1:17:33Cuz I tend to think that it makes me
1:17:34feel
1:17:36not positive.
1:17:38And I understand that, but I've always
1:17:42been of the opinion that
1:17:47you shouldn't live in a world of
1:17:48delusion where you're just seeking to be
1:17:50positive have sort of
1:17:53positive things said and avoid
1:17:55uncomfortable conversations. Actually,
1:17:57progress often in my life comes from
1:17:59like having uncomfortable conversations,
1:18:01becoming aware about something, and then
1:18:03at least being informed about how I can
1:18:05do something about it.
1:18:07And so,
1:18:09I think that's why that's why I asked
1:18:10the question cuz I think I assume most
1:18:11people will should, if they're, you
1:18:13know, if they're normal human beings,
1:18:15listen to these conversations and go,
1:18:18gosh, that's scary.
1:18:20And this is concerning.
1:18:24And and then I can come back to this
1:18:25point which is like, well, what do I do
1:18:27with that energy?
1:18:28Yeah, but I'm trying to point out this
1:18:31is not different than so many
1:18:33conversations. We can talk about, oh,
1:18:35there is starvation in this region,
1:18:37genocide in this region. You're all
1:18:39dying. Cancer is spreading. Atheism is
1:18:42up. You can always find something to be
1:18:45very depressed about and nothing you can
1:18:47do about it. And we're very good at
1:18:49concentrating on what we can change,
1:18:52what we are good at, and
1:18:55basically
1:18:57not trying to embrace the whole world as
1:18:59a local environment. So, historically,
1:19:01you grew up with a tribe. You had a
1:19:03dozen people around you. If something
1:19:04happened to one of them, it was very
1:19:06rare. It was an accident. Now, if I go
1:19:08on the internet, somebody gets killed
1:19:10everywhere all the time. Somehow,
1:19:13thousands of people are reported to me
1:19:14every day. I don't even have time to
1:19:16notice.
1:19:17It's just too much. So, I have to put
1:19:19filters in place.
1:19:21And I think this topic is what
1:19:25people are very good at filtering as
1:19:27like this was this entertaining
1:19:29talk I went to, kind of like a show, and
1:19:32the moment I exit, it ends. So, usually
1:19:35I would go give a keynote at a
1:19:37conference and
1:19:39I tell them basically you're going to
1:19:40die, you have 2 years left, any
1:19:42questions?
1:19:43And people be like,
1:19:45will I lose my job?
1:19:47How do I lubricate my sex robot? Like
1:19:49all sorts of nonsense, clearly
1:19:51understanding what I'm trying to say
1:19:53there.
1:19:54And those are good questions,
1:19:55interesting questions, but not fully
1:19:58embracing the result. They're still in
1:20:00their bubble of local versus global.
1:20:03And the people that disagree with you
1:20:04the most as it relates to AI safety,
1:20:07what is it that they say?
1:20:10What are their counterarguments
1:20:11typically?
1:20:13So, many don't engage at all. Like they
1:20:16have no background knowledge in the
1:20:18subject. They never read a single book,
1:20:20single paper, not just by me, by anyone.
1:20:23They may be even working in a field, so
1:20:26they are doing some machine learning
1:20:27work for some company maximizing ad
1:20:30clicks.
1:20:31And to them, those systems are very
1:20:33narrow.
1:20:35And then they hear that all this AI is
1:20:37going to take over the world like has no
1:20:39hands. How would it do that? It's
1:20:42nonsense. This guy is crazy, has a
1:20:43beard, why would I listen to him, right?
1:20:46That's uh
1:20:47Then they start reading a little bit.
1:20:49They go, oh okay, so maybe I can be
1:20:52dangerous, yeah, I see that, but we
1:20:54always solve problems in the past, we're
1:20:56going to solve them again. I mean, at
1:20:58some point we fixed the computer virus
1:21:00or something, so it's the same.
1:21:02And basically, the more exposure they
1:21:05have, the less likely they are to keep
1:21:08that position. I know many people who
1:21:10went from
1:21:12super
1:21:13careless developer to safety researcher.
1:21:17I don't know anyone who went from I
1:21:19worry about AI safety to like there is
1:21:21nothing to worry about.
1:21:29What are your closing statements?
1:21:31Uh let's make sure that it's not a
1:21:32closing statement we need to give for
1:21:34humanity. Let's make sure we stay in
1:21:36charge, in control.
1:21:38Let's make sure we only build things
1:21:40which are beneficial to us.
1:21:42Let's make sure people who are making
1:21:44those decisions are remotely qualified
1:21:46to do it.
1:21:48They are good, not just at science,
1:21:51engineering, and business, but also have
1:21:52moral and ethical standards.
1:21:55And uh if you're doing something which
1:21:57impacts other people, you should ask
1:21:59their permission before you do that. If
1:22:02there was one button in front of you
1:22:04and it would
1:22:07shut down every AI company in the world
1:22:09right now
1:22:10permanently, with the inability for
1:22:12anybody to start a new one,
1:22:14would you press the button? Are we
1:22:15losing narrow AI or just
1:22:17superintelligent AGI part? Losing all of
1:22:19AI.
1:22:21That's a hard question because AI is an
1:22:23extremely important, it controls stock
1:22:26market, power plants, it controls
1:22:28hospitals. It would be a devastating
1:22:31accident. Millions of people would lose
1:22:34their lives. Okay, we can keep narrow
1:22:36AI. Oh, yeah.
1:22:38That's what we want. We want narrow AI
1:22:40to do all this for us, but not God we
1:22:42don't control doing things to us. So,
1:22:45you would stop it, you would stop AGI
1:22:47and superintelligence.
1:22:48>> We have AGI. What we have today is great
1:22:51for almost everything. We can make
1:22:53secretaries out of it. 99% of the
1:22:55economic potential of current technology
1:22:58has not been deployed. We make AI so
1:23:00quickly, it doesn't have time to
1:23:01propagate through the industry, through
1:23:03technology. Something like half of all
1:23:06jobs are considered BS jobs. They don't
1:23:08need to be done, jobs.
1:23:10So, those can be not even automated,
1:23:12they can just gone. But I'm saying we
1:23:14can replace 60% of jobs today with
1:23:18existing models.
1:23:19We've not done that. So, if the goal is
1:23:21to grow economy, to develop, we can do
1:23:24it for decades without having to create
1:23:26superintelligence as soon as possible.
1:23:28Do you think globally, especially in the
1:23:30Western world, unemployment's only going
1:23:31to go up from here?
1:23:33Do you think relatively this is the low
1:23:34of unemployment?
1:23:36I mean, it fluctuates a lot with other
1:23:38factors. There are wars, there is
1:23:40economic cycles, but overall, the more
1:23:42jobs you automate and the higher is the
1:23:44intellectual necessity to start a job,
1:23:47the fewer people qualify.
1:23:50So, if we plotted it on a graph over the
1:23:53next 20 years, you're assuming
1:23:55unemployment's gradually going to go up
1:23:57over that time. I think so. Fewer and
1:23:59fewer people would be able to
1:24:01contribute. Already, we kind of
1:24:03understand it because we created minimum
1:24:05wage. We understood some people don't
1:24:07contribute enough economic value to get
1:24:09paid
1:24:10anything really. So, we had to force
1:24:13employers to pay them more than they're
1:24:15worth.
1:24:17And we haven't updated it. It's what,
1:24:18725 federally in US?
1:24:21If you keep up with the economy, it
1:24:23should be like $25 an hour now.
1:24:26Which means all these people making less
1:24:29are not contributing enough economic
1:24:31output to justify what they're getting
1:24:33paid.
1:24:35We have a closing tradition on this
1:24:36podcast where the last guest leaves a
1:24:37question for the next guest not knowing
1:24:38who they're leaving it for.
1:24:40And the question left for you is, what
1:24:41are what are the most important
1:24:45characteristics
1:24:47for a friend,
1:24:48colleague,
1:24:50or mate?
1:24:52Those are very different types of
1:24:54people.
1:24:55Mhm. But for all of them, loyalty is
1:24:58number one.
1:25:00And what does loyalty mean to you?
1:25:03Not betraying you,
1:25:05not screwing you, not cheating on you.
1:25:10Despite the temptation.
1:25:12Despite the world being as it is,
1:25:15situation, environment.
1:25:17Dr. Roman, thank you so much. Thank you
1:25:19so much for doing what you do because
1:25:21you're you're starting a conversation
1:25:22and pushing forward a conversation and
1:25:24doing research that is
1:25:25incredibly important and you're doing it
1:25:27in the face of a lot of um
1:25:29a lot of skeptics.
1:25:30I'd say there's a lot of people that
1:25:31have a lot of incentives to discredit
1:25:34what you're saying and what you do
1:25:36because they have
1:25:37their own incentives and they have
1:25:38billions of dollars on the line and they
1:25:40have their jobs on the line potentially
1:25:41as well, so
1:25:43it's really important that there are
1:25:44people out there that are willing to
1:25:47I guess stick their head above the
1:25:48parapet and
1:25:50come on shows like this and go on big
1:25:52platforms and talk about
1:25:54the unexplainable, unpredictable,
1:25:56uncontrollable future that we're heading
1:25:57towards.
1:25:59So, thank you for doing that. This book,
1:26:00which which I think everybody should
1:26:02should check out if they want a
1:26:03continuation of this conversation,
1:26:05I think it was published in 2024,
1:26:07gives a holistic view on many of the
1:26:09things we've talked about today, um
1:26:10preventing AI failures and much, much
1:26:12more. And I'm going to link it below for
1:26:14anybody that wants to read it. If people
1:26:16want to learn more from you, if they
1:26:17want to go further into your work,
1:26:18what's the best thing for them to do?
1:26:20Where do they go? They can follow me,
1:26:21follow me on Facebook, follow me on X,
1:26:23just don't follow me home. Very
1:26:25important.
1:26:26>> do it.
1:26:26Okay, so I'll put your Twitter, your X
1:26:28account um as well below so people can
1:26:30follow you there.
1:26:31And yeah, thank you so much for doing
1:26:32what you did. Remarkably eye-opening and
1:26:34it's given me so much food for thought
1:26:36and it's actually convinced me more that
1:26:37we are living in a simulation. But it's
1:26:39also made me think quite differently of
1:26:41religion, I have to say,
1:26:42because um you're right, all the
1:26:43religions, when you get away from the
1:26:45sort of the local traditions, they do
1:26:46all point at the same thing.
1:26:48And actually, if they are all pointing
1:26:50at the same thing, then maybe the
1:26:51fundamental truths that exist across
1:26:52them should be something I pay more
1:26:54attention to. Things like loving thy
1:26:56neighbor, things like the fact that we
1:26:57are all one, that there's a a divine
1:26:59creator, and maybe also they all seem to
1:27:02have consequence beyond this life.
1:27:05So, maybe I should be thinking more
1:27:06about
1:27:07how I behave in this life and and where
1:27:09I might end up thereafter.
1:27:11Roman, thank you. Amen.
1:27:23Oh
1:27:23oh oh oh oh.
1:27:35Oh
1:27:36oh oh oh oh.

About this transcript

This page contains the full transcript of The AI Safety Expert: These Are The Only 5 Jobs That Will Remain In 2030! - Dr. Roman Yampolskiy by The Diary Of A CEO, generated from the public captions YouTube serves with the video. The transcript has 15,152 words across 2,524 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.