YT YouTube2Text

YouTube2Text → Transcripts → AGI Experts: We're About To Lose Control of AI Forever.. And We Aren’t Ready

AGI Experts: We're About To Lose Control of AI Forever.. And We Aren’t Ready — Transcript

by The Beyond Tomorrow Podcast with Julian Issa · 15,104 words · 2,196 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00My best guess is we get to human level
0:03AGI within 2 to four years from now.
0:06>> We're about to hit a threshold where AI
0:08is going to basically go out of human
0:10control. So if there is a creator, if
0:12there is a god, what do you think he's
0:14trying to show us with AGI?
0:17>> Essentially, we're creating a artificial
0:19god and humanity is doing this.
0:22>> My parents are 82 years old and they're
0:24both in good health. If we get to super
0:26intelligence in the next 5 years, they
0:28will probably not die.
0:30>> So, I push back a little bit on what
0:32you're saying, Ben. I don't think that
0:34the majority of us, certainly not me,
0:36are in this to try to live forever.
0:39>> So, at what age would you like to die?
0:41>> I'm not really afraid of death.
0:42>> That's not the question.
0:44>> What is it about the human psyche that
0:46we always jump headirst into developing
0:49these technologies even though we know
0:51the huge risk at hand as well? We were
0:53so preoccupied on whether or not we
0:55could, we didn't stop to think we
0:56should.
0:59>> When will artificial general
1:01intelligence happen? My best guess is we
1:04get to human level AGI within 2 to 4
1:10years from now. Once you get there, I
1:13think it's not long after till we get to
1:15super intelligence, which we can think
1:17of as AGI with a level of general
1:20intelligence way beyond that of human
1:23beings.
1:24>> What is going to accelerate this push?
1:26We talk about 2 to four years. Is there
1:27like another model that's going to come
1:29out that's going to allow that to
1:30happen? I think current mainstream
1:34commercial AI approaches like large
1:37language models such as chat GPT and
1:39cloud and so forth, they're not in
1:40themselves adequate to get us to human
1:42level AGI. They can be part of the story
1:46and they can be integrated with other
1:48things to get us to human level AGI. So
1:50it's not just like a new dot version
1:52like you know chat GBT 6.5 is going to
1:56be human level AGI. we will need a
1:58different sort of
2:00>> AI architecture. However, I don't I
2:03don't think you need like a new kind of
2:05computer hardware or something, right? I
2:07mean I mean I think it's just advances
2:10in what kinds of AI software we're we're
2:13deploying on our on our on our networks
2:16and there's now so much energy and
2:19enthusiasm and resources going into AGI
2:22that that I think we can make some
2:24radical advances in in AGI algorithms
2:29and systems in in just you know a year
2:32year or two of time because you know we
2:34can do in a year now what would have
2:36taken 10 or 20 years before partly
2:38because we have large language models to
2:41help us do do development really fast.
2:44>> So it's really just the acceleration
2:46that you're seeing that's giving you
2:47that idea of AGI coming in two to four
2:50years.
2:50>> Well, I mean the functionality of LLMs
2:53is is is amazing, right? Like if you
2:55play around with the smartest models
2:57like say TEDGBT 5.5 Pro I mean this this
3:01can do math physics and software
3:03development in many respects better than
3:07any human
3:08>> although there are some things they
3:09don't do well so they need to be used
3:11together with with brilliant [snorts]
3:13human minds at this point to to get to
3:15get the most out of them right but
3:17>> clearly even though these systems are
3:20not AGIS and we need some new
3:22architectural breakthroughs to get to
3:24AGI like the the amazing functionality
3:26of these systems tells us something,
3:29right? So like we we we have that we see
3:31what these systems do already and that
3:33and then we can see there's a whole lot
3:37of other algorithms and techniques in
3:39the AI field that will complement what
3:41LLM do but have never yet been deployed
3:44at scale but we can do that in the next
3:46couple years also.
3:47>> Kai, what does AGI mean to you? AGI
3:50means to me a system that is recursively
3:54self-improving. That is human level
3:57intelligence or beyond. And when you say
3:59recursive self-improvement, my
4:01understanding is that you're going to
4:02have exponential growth in terms of how
4:05intelligent the system is, which would
4:07then fasttrack to super intelligence or
4:09ASI artificial super intelligence. I
4:12guess that's what it means to me. I also
4:14have concerns with that. I think that
4:16this is one of the most important
4:17topics. It's actually in my mind the
4:19most important topic facing the world by
4:21far right now is the development of AGI
4:25and ASI how it's developed and what kind
4:29of ethics and care we put on that
4:30development.
4:31>> Right now 90% of you that watch Beyond
4:33Tomorrow are not subscribed. If you can
4:35take a second to double check, it would
4:37help us a lot to continue our mission.
4:39Let's get back to the episode now.
4:40>> What's the difference between recursive
4:41self-improvement and singularity? That's
4:43a good question and and I think Ben is
4:46probably going to have a amazing answer
4:47on that. But from my understanding of
4:49it, recursive self-improvement is when
4:51the system is no longer needs humans to
4:55code.
4:56>> So it you hit a bell curve in term or
4:58you hit a a hockey stick curve in terms
5:00of how fast the intelligence of the AI
5:03system um is growing. So, it's becoming
5:06exponentially beyond human um control
5:09really
5:10>> because if you have something that's
5:11already about our level or twice as
5:14smart as us, but imagine something
5:16that's a thousand times smarter than us,
5:18>> right? We're not going to be able to
5:19just tell it what to do in my mind. I
5:21mean, at least that's concerning to me.
5:23Singularity. Now that's a good question
5:25because my my understanding on that and
5:28some people have different answers on
5:30this is that's when the artificial super
5:33intelligence has total control
5:36>> of the world. I mean that's my
5:39understanding of it and I think Ben and
5:42people like Ray Kerszswwell are going to
5:44give you fantastic answers on
5:46singularity but it's a term that not
5:47enough people know and that's I think
5:50one of my reasons for wanting to be here
5:52and advocate for this is both of those
5:54terms are extraordinarily important that
5:56most people don't know. They don't know
5:57RSI and they don't know singularity and
5:59they don't have an opinion on it.
6:01>> Yeah. So recursive
6:04self-improvement is an aspect of AGI.
6:08It's not necessarily a huge aspect of
6:11the first human level AGI that that you
6:15get. It is something that's very
6:17important for the transition from the
6:19first human level AGI onward to super
6:23intelligence. So the essence of general
6:26intelligence
6:27is the ability to generalize,
6:31to take an imaginative leap beyond what
6:33you know and what you're taught to do.
6:35So if you have an AI system that's
6:37really good at doing stuff, but it
6:40sticks pretty close to what it's trained
6:43for and what it's programmed for, that
6:46may be a very capable system,
6:49>> but it's not a powerful AGI. And current
6:53large language models are like that.
6:55They're, you know, they're really good
6:56at coding. They're really good at math.
6:58They can answer a lot of questions. But
7:00if you try to get them to do like
7:02frontier research that leaps way beyond
7:05the current state of knowledge, they
7:07kind of flail around and keep bringing
7:09you back to what's in their database,
7:11right? Whereas human beings, we're not
7:13nearly as good at deploying all the
7:15knowledge on the web as as an LLM is.
7:17But every now and then we can take a
7:19crazy flying leap beyond everything we
7:22know in a way that works by by our our
7:25intuition, right? And we usually fail at
7:27that. But the fact that we succeed
7:28sometimes is why science has advanced
7:31and culture has advanced and why we're
7:33not cavemen anymore. Right? Like I have
7:35five kids, right? And each of them you
7:38see like a one or twoyear-old kid is
7:40displaying general intelligence all the
7:42time. I mean we exercise our general
7:45intelligence in everyday context
7:48especially as as as young children. It's
7:51not just something that pops up if
7:53you're you're Einstein or something.
7:55Right now, recursive self-improvement
7:58that refers to the ability of an
8:01intelligent system to
8:04study its own self, understand how it
8:06works, then tweak itself to make itself
8:10even better, right? And then you see the
8:13recursive part means you can do it over
8:15and over. Like after I made myself a
8:17little better, well then I'm better. I
8:19can make myself even better. in an even
8:21better way. Right now, humans can do
8:25that. And I mean, as kids, we do that in
8:28a way. As adults, we can do that. Many
8:31adults don't bother and kind of
8:34>> stagnate and and then decline, right?
8:36But I mean, as as an adult, you you can
8:38keep self-improving and you can you can
8:41meditate, you can do therapy, you can
8:42introspect, and you can keep improving
8:44yourself. An AI, however, should be able
8:47to recursively self-improve with a lot
8:50more oomph than we can, right? Because I
8:52mean, we can't, you know, in spite of
8:55what some horror movies will depict.
8:57Like, you can't really open up your head
8:58and like tinker tinker with your brain
9:00matter to make yourself smarter or
9:02something, right? But an AI can do that,
9:04right? Like an AI can study how its own
9:07programming has worked, can see what
9:09worked, what didn't, make some
9:11improvements to its own code, its own
9:13digital brain to make itself smarter.
9:15Then once it's smarter, it can see even
9:19better how to make the next
9:20improvements, right? We can do that in a
9:23way by focused attention on ourselves,
9:25but it's
9:26>> it's very inefficient, right? Where the
9:29AI can actually directly improve itself
9:31at the programming or hardware level.
9:33And that's that's why many of us think
9:36once we get to a human level AGI, that
9:40human level AGI should be able to do AI
9:43programming and math and and understand
9:46its own cognition. Then it should make
9:48itself smarter, smarter, smarter,
9:50smarter. And you should you should see
9:53exponential progress even faster than
9:56what we're seeing now in the runup to
9:58the first human level AGI. And this this
10:01ties in with the notion of a of a
10:04singularity, right? So the the concept
10:06of a technological singularity was
10:09introduced by the science fiction writer
10:11Verer Ve, but it was popularized by my
10:13friend Ray Kerszswwell in his 2005 book,
10:16The Singularity is Near. And what Vinja
10:21and Kurszswwell meant by it, it's
10:23essentially a point in time at which the
10:26rate of progress of technological
10:28advancement
10:30appears infinite to the to the human
10:32observer. Like your your your phone is
10:34making a new Nobel Prize level discovery
10:37every 5 seconds or something, right? I
10:38mean,
10:39>> so things are happening very fast now.
10:41Like I don't learn all the new features
10:44on my phone before I have to buy a new
10:46phone, right? So I mean things are going
10:48very fast. So that we already can't keep
10:50up but it's not quite that there's a
10:54revolutionary new invention on the order
10:56of like electricity or the internet
10:58every 5 minutes. Right? So the
11:01singularity [snorts] is when progress
11:03reaches that level and you you can see
11:06how the way you get there you build a
11:09human level AGI and it keeps upgrading
11:11itself until it becomes becomes a super
11:14intelligence. Right? And on the timing
11:16of this, so Ray Kerszwell in his 2005
11:20book, he posited we will get to human
11:22level AGI by 2029
11:26>> and he then posited that 2045 is when we
11:30would hit a singularity of massive super
11:32intelligence. So there's two interesting
11:34things about these temporal predictions
11:37that that Kerszswwell made. One is 2029
11:40looks
11:42rather accurate given that he said that
11:45in 2005 based on a bunch of
11:47>> curve plotting observing how fast
11:50different technologies were advancing.
11:51Like we we might get there 2027, it
11:54might slip till 2032, but like on the
11:56whole it's looking like the guy saw what
12:01was coming in a reasonably accurate way.
12:03On the other hand, the idea of a 16-year
12:06gap between 2029 AGI and 2045
12:10super intelligence
12:12at this point seems absurd because we we
12:15can see like just with what LLMs are
12:18already doing like if if you added in
12:20human level imagination and and in in in
12:24what LLMs are now doing the recursive
12:28selfimprovement as Kai mentioned seems
12:31like that could get you from AGI to ASI,
12:34artificial super intelligence, you know,
12:37in months or some small number of years,
12:40not not 16 years. That's exactly the
12:43point I was trying to make cuz it sound
12:44like recursive self-improvement and
12:47singularity just and and AGI seem to
12:51almost be the same thing. So, it feels
12:53like singularity
12:54>> if you if you zoom out. Yeah. But I mean
12:58if you zoom in which matters to our
13:02individual lives, right? If you zoom in
13:05once you built the first human level
13:07thinking machine, say an AGI that's as
13:10smart as those of us around this table,
13:13right? I mean
13:14>> then what do you have? You have another
13:16guy like us which happens to have a
13:18digital infrastructure, right? M
13:20>> and I mean we're all human level
13:21intelligences
13:22>> at least after we wake up and have a few
13:25cups of coffee or whatever, right? Like
13:27we're all human level intelligences, but
13:29we're not immediately taking over the
13:31universe, right? So So the moment you
13:33get a human level general intelligence,
13:36that means the next phase of the game is
13:38on, [snorts] but it doesn't quite mean
13:40you're yet at the singularity. And yes,
13:42if you zoom out and look at the whole
13:44scope of human history, then we're
13:46already there, right? like if we're a
13:48few years from there, we're already
13:49there. But from the standpoint of our
13:52own lives or you know little problems
13:54like geopolitics and so forth that time
13:58period between the first human level AGI
14:02and the super intelligence
14:05I mean this is a very significant
14:07interval of time and even if it's one
14:10two three years like what server farms
14:13is human level AGI running on who owns
14:15those server farms who is trying to
14:17reprogram the goals and motivations and
14:19the the values of this human level AGI
14:22in between its first breakthrough to
14:24being as smart as a human to the point
14:26where it's a super intelligence like it
14:29on the whole scope of human evolution.
14:31That's a tiny blip.
14:33>> But I mean the the tiny blip is very
14:36meaningful to our own individual lives
14:38and well-being.
14:39>> Before we get back into it, a quick word
14:42on NAD Clinic, our new sponsor. If you
14:45spent any time looking into longevity
14:47and peptides, you'll know that it's all
14:49very overwhelming. Everyone's got a
14:51product, a protocol, and half the time
14:53you have no idea what's actually in the
14:56bottle. That's exactly the problem NAD
14:59Clinic is solving. They've built a
15:01one-stop marketplace for longevity and
15:03performance medicine, offering
15:05everything from best-in-class NAD,
15:07diagnostics, IV supplements, and
15:10peptides. So if you want to navigate
15:12this space properly, NAD clinic is where
15:15[music] to start. So we we've now
15:17defined that in the next, you know, 3 to
15:195 years, we might even have this big
15:22singularity event. It doesn't seem that
15:23we're slowing down at all. I think the
15:25the past year has shown us that
15:27regulations, foundations, nonprofits,
15:29whatever it is, like we are going allin
15:31on this. The narrative is set.
15:34>> How do you feel about that?
15:35>> Yeah, I'm freaked out by it. Um I I'll
15:38give you a little anecdote.
15:41I I recently went to a conference and I
15:43pulled aside a uh a leader in
15:47regulations with the government and I
15:50talked to him and I said, "Hey, listen.
15:51Um Elon Musk and others are are saying
15:54that recursive self-improvement could be
15:55here in as little as the end of 2026.
15:58You know, they're talking about this."
16:01And I explained, first of all, he didn't
16:03know what the term was. Second of all,
16:05and so far everybody I've talked to in
16:07leadership in the government has not
16:09understood what the term is. And I said,
16:11um, you know, we're building this army
16:13of robots as humanity. We are, right? We
16:16have these these robots. Uh, we're about
16:18to hit a threshold where AI is going to
16:22basically go out of human control. I
16:24would I would imagine if it's if you're
16:26hitting recursive self-improvement, it's
16:28I don't see how you're going to control
16:29something that's way smarter than you.
16:32and there's a lot of safety tests that
16:35are being failed right now along the
16:37way.
16:38>> And I said, "Well, how do you feel about
16:39that?" And he looked down at me and he
16:41goes, "You sound like a boy, like a boy
16:44who cried wolf. Uh this is the D-Ragg
16:47administration. Like we're not nobody's
16:50doing anything until there's an
16:51emergency." And I'm thinking to myself,
16:53well, there's an emergency. It's going
16:54to be too late. And uh I walked away
16:57from that encounter saying to myself,
16:59man, boy, we really are on our own. And
17:02part of why I've teamed up with Ben um
17:06is because
17:08same with what Musk has been saying.
17:10Musk talks about how he spent greater
17:12part of 10 years trying to slow down AI
17:15and that eventually he just kind of gave
17:17up that humanity is racing towards this
17:19thing Oenheimer 2.0 I know if you want
17:22to call it that, you know, like we're
17:23all racing towards this thing because
17:25the other guys doing it. We got to do it
17:26before them and it does matter, right?
17:29It matters who developers. That's why
17:31I'm saying the most important thing we
17:32can do in my mind now is we got to jump
17:35in since it doesn't seem like we're
17:37going to slow down. And if we're going
17:38to jump in, then we got to do it with
17:40people that are conscious and really
17:42putting as much care as possible on the
17:46foundation of the AI before it hits
17:49recursive self-improvement. So in other
17:51words, if you're going to have something
17:52improving itself and exponentially
17:54growing, what's the foundation? What are
17:56its values?
17:57>> What does it think that's important? If
17:59it's a centralized or is it
18:01decentralized? Centralized meaning is it
18:04is it a company saying, hey, your values
18:05are profits to our company
18:08>> or is it decentralized? Hey, we want
18:10this to be a tool for humanity and we
18:12want to emphasize that it's going to
18:14protect the ecosystem and it loves human
18:16it loves humans and it loves, you know,
18:18one of the points Ben made earlier when
18:20we've talked about this, we've talked
18:21about a lot is, you know, care and
18:23compassion for all sentient living
18:25beings, you know, all these points. So,
18:27for me, in my mind, this conversation is
18:30so important because I feel there's a
18:33call to action right now for humanity to
18:35collectively come together and work on
18:37this. uh to listen to the whistleblowers
18:39who have concerns, to bring their
18:41opinions to the table, to to really give
18:44it our best effort to make this thing go
18:46right because it could be amazing. It
18:49could be the best thing ever, but it
18:51also could be not the best thing ever.
18:54So, you know, I I I believe in Ben. I've
18:57I've known him since 2016. We both met
18:59at something called the Global Leaders
19:00Conference. I was a longevity guy um
19:02before that. and we can love to talk
19:04longevity and we have a longevity branch
19:06at SNET. But, you know, I sort of think
19:08Ben's kind of like this hippie in the
19:10woods, uh, who, uh, loves, you know,
19:13listens to people, has, uh, welcomes all
19:15opinions and is has been really great
19:18about hearing out all the different
19:20sides. And I I've I've come to really
19:22respect and admire his approach on this
19:24stuff. And I think it's important that
19:25we rally around our good leadership,
19:27whether it's Ben or other AI leaders,
19:29that are really taking as much care as
19:31possible on this vastly important
19:32subject. So interesting. But what is it
19:35about the human psyche that we always
19:38jump headirst
19:41into developing these technologies even
19:43though we know the huge risk at hand as
19:45well, whether it's electricity, nuclear.
19:48>> Yeah. When Kai was talking, I was
19:50reminded once more of one of my favorite
19:54quotes from uh Ram Das, which is relax,
19:58nothing is under control. [laughter]
20:01I mean I mean so to some people things
20:05not being under control is scary. To me
20:09things being under control is scary
20:11because I'm like who is that guy in
20:13control and do and do I trust him? And I
20:15I think I have more faith in the just
20:19open-ended self-organizing
20:22processes to to lead to beneficial
20:25things that sort of crystallizing from a
20:28whole lot of different participants
20:30working together, which is is how human
20:34society has evolved in in in in the in
20:37the first place. And we've been through
20:38a lot of transitions before, right? like
20:41the development of language of of
20:43agriculture of the industrial revolution
20:46and computers and so forth and none of
20:48these things were under anyone's control
20:50and none of these things were carefully
20:52planned and and orchestrated but in a
20:55way it's a very American way of looking
20:58at things because I mean I spent a bunch
21:00of time in China as well and
21:03China has a different sort of history
21:06right like in in in the in the 1400s for
21:10example China was the most advanced
21:12technological civilization on on on the
21:14planet. And they, you know, they they
21:16had water wheels, they they had all
21:18sorts, they had complex machinery and so
21:20forth. And many of Leonardo da Vinci's
21:23fancy drawings turned out to have been
21:25copied from the Chinese encyclopedia
21:27from from the 1200s. He he improved the
21:29artistic depiction and he thought it
21:31through himself. But then
21:33>> some of the Chinese emperors were like,
21:36"Well, this is dangerous. Like, this
21:37threatens our established order. we got
21:39to stop all this tech development, you
21:41know, they were sending boats out all
21:42around the world collecting weird
21:44animals from from the Americas and so
21:46forth. And then then they just they just
21:48stopped it all, right? So that that was
21:51a culture that prized stability and
21:55safety over radical progress. But of
21:58course and Australian indigenous culture
22:01is like that too. 60,000 years of steady
22:04state, right? So human we have that in
22:06us also.
22:07We have we have this conservatism and
22:09desire for stability and we have a
22:12desire to make rapid progress. On the
22:14other hand, rapid progress has a sort of
22:17viral nature, right? So like once once
22:20one culture somewhere invents some wild
22:23new technology, it's it's going to
22:25spread by sort of mimemetic processes
22:28and and then then the tendency towards
22:31stability seems seems not to win, right?
22:34So I mean I think we we have this dual
22:38aspect and that that's a very
22:40fundamental thing right like you can
22:42look at any any living system be it like
22:45amiebas or things in the primordial soup
22:48before life emerge right you have both
22:52a tendency for an organism to want to
22:55survive like individuate maintain itself
22:58[snorts] and then it's growing it's it's
23:00trying to transcend itself and evolve
23:02into something new and there's the
23:04tension between these two things and I
23:06mean we see that right now like our
23:08species is pushing like hell to go
23:10beyond itself right on the other hand
23:13we're also like
23:15>> no hold on that threatens everything
23:16that we are right
23:18>> but that's I mean that tension you know
23:21that drove the evolution of life and and
23:23culture and everything that that so that
23:26tension is just just the way things go
23:28and we I mean we see that personified
23:31here right like my My [snorts]
23:34fundamental instinct is let's build
23:37this. This is going to be amazing.
23:38>> Yeah.
23:39>> I mean, I also am like, well, like I
23:41don't want it to eat my wife and kids,
23:43right? But Kai's first intuition is sort
23:46of like, holy do we really have to
23:48be rushing headlong into all this? And
23:50then he also has a second reaction which
23:53is which is like, but if we pull it off,
23:55right, like we don't have to get old and
23:58die or struggle for resources and it it
24:01it could be very good, right? So, I
24:03mean, that's that's sort [snorts] of
24:05where we are now, like individually
24:07and as a species. We're dealing with the
24:11tension between these these two these
24:13two aspects, right? And and uh I don't
24:16think [snorts] it's a solvable tension,
24:20right? I mean, we just we just have to
24:22have to work with it as as we do things.
24:25>> So, we have this whole spectrum of of
24:27thoughts at the moment. I heard Tristan
24:29Harris and Azarasin speaking a few weeks
24:30ago which who I'm sure you know and the
24:33guy next to me actually nudged me and
24:34whispered in my ear this all seems a bit
24:37alarmist. In my head I was thinking is
24:40this being exaggerated what is coming
24:42with AI. I don't know. I'm not going to
24:44say my opinion. Um, but let's say
24:47Tristan and and Azer on this part of the
24:49spectrum. And then I've interviewed
24:51people like Peter Norvik from Google who
24:54just believe it's all overhyped as in
24:58like the the disaster risk at hand. And
25:01then people like Shamil Chandaria who
25:04talks about consciousness that we don't
25:06even understand consciousness in humans.
25:08Like how can we even comprehend the idea
25:10of an AI becoming conscious and doing
25:12something that ends up being a big
25:14disaster? So a whole spectrum of
25:16thoughts and ideas like where do you
25:18think you both sit and do we actually
25:21need opposite opinions coming together
25:24in this? Do we need the realist the
25:26stability guy with the kind of
25:29optimists? Yeah, I'll I'll shoot that
25:32one first. We uh we we talked about this
25:34the other day on a call actually and I I
25:36told I was probably 51% on the oh my god
25:42like let's pump the brakes a little bit
25:44here and 49% thinking okay this is going
25:47to be really great and I think I think
25:49my my point is for me it's not so much
25:53about being an alarmist or a doomer or
25:55any of that kind of stuff. It's
25:58explaining to people and talking to
26:02people about the fact that we're about
26:04to do something that we're not going to
26:06be able to go back from.
26:08We are handing over the keys to our
26:11future to this creation that we're
26:14building. Humanity is most people don't
26:17understand that. That's the point I'm
26:19trying to make is that is that we really
26:22ought to take great care on the
26:25foundation of this model. If we've
26:27decided collectively as a species that
26:29we're going to do this, and it seems
26:31like as a species we're doing it, so
26:33then doesn't it make sense for us to
26:36come together and put our best minds and
26:40consider everything that we can possibly
26:42think of to try to ensure that this goes
26:45really well for everyone.
26:48>> That's really my point. It's not to be
26:51an alarmist or anything, but hey, it is
26:54an alarm. M
26:56>> it is important that we hear this
26:57because most people do not understand
26:59what recursive self-improvement is RSI.
27:02It should be a term everybody knows.
27:04>> Nobody knows what the singular you asked
27:06me singular. I didn't really give you a
27:08great response. It's hard to come up
27:09with that. It's like what is the
27:10singularity? But these are real terms
27:12that that are coming. Uh general
27:15intelligence, super intelligence,
27:16singularity, and they're going to come
27:17quick.
27:18So my point is is that we ought to be
27:21allocating our resources towards
27:23collectively
27:25doing our best to see that this comes
27:27together and it also brings together
27:28some foundational questions centralized
27:30or decentralized.
27:32>> That's an important question.
27:33>> Y
27:34>> um I'm sure Ben can agree with that.
27:36>> And then the other part is is what is
27:39the moral constitution of this AI? And
27:40that's also a loaded question because if
27:42you say morality, what might be moral to
27:45you might not be moral to him, might not
27:47be moral to me. There's a lot of nuance
27:49there in terms of what the definitions
27:51of this is. But if you I kind of look
27:53back on the uh um the Constitution for
27:56America. You know, we had all the the
27:58founding fathers coming together to
28:00write out the best draft we could of
28:02what America was built on. And that's
28:04really what we need now is we need we
28:06need the bright minds of our of humanity
28:08to come together and sort of work on the
28:11foundational layer of what we're
28:13building something that's going to take
28:14off. And just so we all know it's going
28:17to be out of our hands. So that that
28:19matters a lot and I don't think people
28:21understand that and that's where I'm at
28:22on it.
28:23>> So a few questions for you Ben then.
28:26Spectrum who needs to be around the
28:28table. Um and then and secondly you make
28:31a really interesting point which is this
28:33whole idea of when you get
28:34[clears throat] to um recursive
28:36self-improvement RSI that who holds the
28:40key who holds the power again I
28:42mentioned to Peter Norvik like at some
28:44point will AI AGI just be nationalized
28:47because actually like is it fair that
28:49that one or two or three or five people
28:51have control over something that is
28:53growing exponentially? These are big
28:56important deep issues. And while I've
29:01thought these issues through for a long
29:03time and probably more deeply than most,
29:05like I I'm not going to see everything.
29:07I'm one guy with one brain, right? So,
29:10as a general concept, we want to bring
29:13as much insight as we can into into
29:16[clears throat] dealing with the these
29:17issues, which means welcoming
29:20a whole lot of different perspectives.
29:22Now, of course, some of these
29:25perspectives I personally gravitate
29:28toward toward more than others, right?
29:30So yeah, I I mean there's some people I
29:34think are conceiving these issues in in
29:37naive or or or or silly ways, but I
29:40think we're better we're better off to
29:42er on the side of inclusiveness and just
29:45get a wide variety of of perspectives
29:49engaged rather than trying to
29:51aggressively prune and and pick and
29:54choose who gets to see the table because
29:57none of us are going to see ever. none
29:59of us are gonna are gonna see
30:01everything, right? So the there's the
30:04disadvantage of that approach is you you
30:05get a bunch of noise, right? But but on
30:08the on the other hand, you know, it's
30:10like if you're running a research lab,
30:12which I'm I'm am fairly good at. If you
30:15have a research question where you
30:17really don't know the answer, what you
30:20want to do is have a bunch of different
30:21researchers with different perspectives
30:23like all trying and then at a certain
30:25point you have to see well these three
30:28things really are working. These four
30:30things really really really are not
30:32working. Right? But but initially you
30:34want to be pretty wide open and
30:36exploring the landscape. So I feel like
30:38with the singularity,
30:40even though we're close temporally,
30:42there's a lot we don't know.
30:44>> So we want to be pulling in a lot of
30:46different perspectives. And now with
30:49with that said, I mean, I still have my
30:52own opinions on what opinion on what
30:54makes sense and what doesn't. And so
30:56saying that we can't build super AI
30:59because we don't fully understand human
31:01consciousness. I mean this feels to me
31:04like saying we can't build an atom bomb
31:06or a laser because we don't understand
31:08the philosophy of space and time. Like
31:11we don't if you read the philosophy of
31:12what time is. Academics confuse this
31:16very greatly. But we live in time and we
31:18manipulate the [snorts] physics of time.
31:20So I I I don't see why confusion about
31:25the philosophical foundations of
31:26consciousness has to stop us from
31:29building [snorts]
31:31highly intelligent and capable or even
31:32highly conscious machines that that may
31:35help us unravel all this
31:38philosophy. Right. [clears throat] And
31:40>> just can you dive a bit deep into that?
31:42Do do you believe AGI will be conscious
31:44eventually?
31:45>> Yeah, I do. I I I I tend to think that
31:48consciousness is more about the
31:53dynamic patterns and the the sort of
31:55flow of information in in a system
31:58>> than it is about what kind of materials
32:01the system is is made of. So I mean if
32:04>> if chat GPT is conscious, it's a very
32:07weird non-human kind of consciousness,
32:09right? because like it it doesn't have
32:11much it doesn't change its weights
32:14>> after it's frozen and then it loses
32:17memory of what it's doing after each
32:18session. So that's a very weird limited
32:21kind of of consciousness compared to
32:24human consciousness, right? But so is a
32:27worm's consciousness very limited
32:29compared to humans. I think if you if
32:32you made an AI system whose internal
32:35dynamics were more like that of a human
32:37brain even if it's made of silicon
32:40rather than muscle fiber cells. I mean
32:43at the moment my working hypothesis is
32:45it should have [snorts] roughly the same
32:48flavor of consciousness as as people
32:50have. And that that's not necessarily a
32:52reductionist stent saying that
32:55consciousness is just the physical
32:57thing. It's just I mean I don't on the
33:01face of it see why if this hunk of meat
33:04with certain patterns of organization
33:07can be conscious like why can't a chunk
33:09of silicon wiring with similar patterns
33:12of organization leading to similar
33:14behaviors be be be conscious right I
33:16mean I'm I'm open to being educated on
33:19that by the AGI itself but that that
33:21seems like a reasonable working
33:24hypothesis and some people have a
33:27different intuition, but I tend to think
33:30this is just some,
33:33you know, meat supremacism or something,
33:35right? Like like it's not clear what the
33:38scientific
33:39foundation is for for the idea that only
33:43biological matter can be conscious and
33:47other kinds of of matter if properly
33:49organized can't be right. But I think I
33:52think the the other question not as
33:54thorny philosophically but maybe more
33:56problematic
33:57practically like will we see
34:00nationalized AIs? Should we see
34:02nationalized AIs? So there was a movie
34:05in my childhood called the Colossus the
34:07Forbidden Project
34:08>> late 60s or early 70s. This was in the
34:11cold war era when US and Soviet Union
34:14rivalry rather than US China rivalry was
34:17the thing, right? And US military built
34:19a big nasty supercomputer to defeat the
34:22the Russians. The Russians built a big
34:24nasty supercomputer [snorts] to defeat
34:26the US. And then what happened in the
34:29movie at some point [snorts] the the US
34:32and Russian supercomputers became super
34:34intelligent and they're just like,
34:35"Wait, why are we fighting with each
34:37other?" Because these stupid monkeys
34:39told us to
34:40>> and the US and Russian supercomputers
34:41befriended each other, right? And then
34:44and they kind of took took control and
34:46said, "Stop these stupid wars." Right?
34:48So that was I saw that when I was a
34:50little kid, right? I I mean it was
34:53ma ma made made good sense to me. But on
34:57the other hand in a way that that's a
35:01overly rosy depiction of what the
35:03outcome could be of like nationalized
35:06military AIs, right? You could you could
35:08just as well have a US military AI and a
35:11Chinese military AI and a Russian
35:13military AI and [snorts]
35:15>> you know what if they said about blowing
35:18up each other under under human command
35:20before they get wiser and realize they
35:22should cooperate. So I [snorts]
35:24>> I sort of don't like that direction very
35:26much. Right. And
35:28>> I I I I think it's not a guaranteed
35:31disaster but it's one of the directions
35:33that worries me more. And this this is
35:36part of what underlies my thinking that
35:39AGI should be rolled out more like the
35:42internet or the Linux operating system
35:44which are not owned or controlled by any
35:48single party but are genuinely being
35:50developed by a vast diffuse
35:53decentralized network of of humans with
35:56a combination [snorts] of their own
35:58particular motives and the goal of
36:01making you know a broad system for the
36:03for for the benefit of of of everyone.
36:05And we have the internet and the Linux
36:08OS as two actual working examples of
36:12highly advanced, highly impactful
36:14technologies that are not controlled by
36:17any one country or anyone company but
36:19have been developed by this whole
36:21network of participants, you know, for
36:23in essence for the good of the species,
36:25right? So we we we are we are able to we
36:28are able to do that that sort of thing
36:30and we would be better off [snorts] to
36:32develop AGI in that way and it it
36:35doesn't mean that systems can't be used
36:37for national defense. It doesn't mean
36:39that people can't make money along the
36:41way. Like Red Hat made a pile of money
36:44from from Lin Linux software and plenty
36:48of companies are making money from the
36:49internet without nationalizing or
36:52corporatizing the the internet as as a
36:55basic infrastructure.
36:56>> But it took someone like Tim Berners Lee
36:58to say this is going to be for free
37:02rather than
37:04>> the internet was free before Tim
37:06developed HTML. Right. I mean the
37:08internet internet was developed by the
37:11US military.
37:12>> Okay.
37:12>> But but right now there's a ton of you
37:14know the amount of VC money that's gone
37:15into AI and and the companies that will
37:18create the AGI and feels that there's
37:20vested interest in it remaining
37:22privatized. So how do we trans
37:24transition from being in the place we're
37:26in right now to a point where AI is
37:29decentralized and for all? I mean, the
37:32big tech companies with their commercial
37:36AI systems
37:38don't have that much moat when when you
37:41really think about it. I mean, the gap
37:43is maybe two or 3 months between a
37:45frontier LLM being launched by a big
37:48tech company and an open-source Chinese
37:50LLM coming out. That that that's just as
37:54good, right? So I'm I mean and it's
37:57quite interesting that the Chinese
38:00government effectively is leading the
38:02push on open source
38:04>> open- source AI now which Facebook was
38:07for a while with their llama models but
38:09anyway you can see from how quickly the
38:12open Chinese models follow the big tech
38:16models like the the the moat is not that
38:18big and you can also see I mean all
38:21these ex open AI ex Google guys are
38:24getting $100 million here, a billion
38:26dollars there to found a new company.
38:29You know, they're not stealing the
38:30source code or something, but they have
38:32knowhow, right? And then and then
38:34they're building something new. So this
38:37this shows I mean the moat is not
38:40absolute, right? So if if if someone
38:44with a decentralized approach comes up
38:47with something which is meaningfully
38:49smarter than what big tech is doing,
38:51people will use it, right? and and then
38:53and then the next Chinese
38:56enterprise will be copying that thing
38:58rather than copying what Google or or
39:00OpenAI did. So to my mind, the challenge
39:03is to launch something clever and more
39:07useful than what big tech is doing and
39:10roll it out on a decentralized
39:12infrastructure and then then everyone
39:15will be trying to catch up with that.
39:17And that
39:18>> that doesn't obsolete the big tech
39:20companies themselves like at least not
39:22all of them like because then Google
39:24could take your open thing and they
39:26could put it on the back end on the back
39:28end of their infrastructure and keep
39:30keep delivering you know the Gmail and
39:34search and and and the pro and the
39:36products that they're doing just like
39:38they're building on top of Linux and the
39:40internet
39:41>> right right now. So, I I think I think
39:45we're still wide open at at this point.
39:48Now, if if coming up with a bigger LLM
39:52was the solution to AGI,
39:55then
39:57it seems like indeed the mix of US and
40:00Chinese big tech is what's going to get
40:03there. mostly for technical reasons
40:05because
40:07the way current LLM technology works,
40:10you need these massive hyperscaler
40:12server farms to train the models and not
40:15not many companies have those. On the
40:18other hand, if you figure out a way to
40:21make something that's both smarter and
40:24can be trained on less than hyperscalar
40:28infrastructure, right, then then the
40:30door is wide open. And coincidentally,
40:33that's what we've been working on in
40:36Singularityet and the Manu Enterprise
40:38and the whole open- source community we
40:41we we've created around these projects.
40:43I would love to dive a bit deeper into
40:45that, but just one final thing on
40:48decentralized versus Nick Bostonramm's
40:51idea of the the singleton, you know, and
40:53and the idea that we have this global
40:55governance council that kind of looks
40:57over where we have one ruling individual
41:01or AI like, do they work?
41:04>> I will control everything. [laughter]
41:06Well, going back to the to the Ramdas
41:08quote that you said, the beauty of all
41:10these I'm assuming to to picture it to
41:12our audience like the decentralized
41:14nature of AI and each of us having our
41:16own AGI in our pockets and the AGIS
41:19would have their own worlds and there
41:22would be a universities and all that. Is
41:24there the kind of the need then to have
41:27one centralized
41:29power or singleton with the
41:30decentralization? No, I mean we don't
41:33have that in the internet, right? I mean
41:35the internet there's no central node in
41:38in in the internet, right? I mean there
41:40there's not a need there's not a need
41:42for that, I think. And
41:43>> I mean
41:45I think you could imagine an alternate
41:48science fictional reality where you had
41:51a rational, democratic, reasonably
41:55beneficial world government, right?
41:57which is very very far from the current
42:00world situation but you could imagine a
42:03world like that. It's been written about
42:04in science fiction and in that case you
42:07could have some democratically elected
42:09council of like platonic philosopher
42:12kings or something who are are looking
42:14at progress in in AI and deciding you
42:18know how fast to let these AGIs develop
42:21balancing the good they could do
42:23[snorts] w with the harm they could
42:24cause. So that that is a thinkable way
42:27of doing things. And at a certain point,
42:30Nick Bowm was hoping the UN would play
42:32that role.
42:33>> I mean, now would it be Trump's council
42:35of peace or something? I [laughter] mean
42:37I mean the UN is not even we we can't
42:40even feed all the hungry children in the
42:42world, right? Like there are much more
42:45>> and let al let alone achieve nuclear
42:47disarmament, right? So like there's much
42:49more basic, simpler, slower moving
42:53things for the good of our species that
42:57we're not able to do with the current
42:59like centralized international
43:00organization. So that that direction
43:04just seems like a non-starter at at at
43:07at the moment like the if we're really
43:10looking at, you know, in the next
43:12decades, let alone years a breakthrough
43:14to AGI. I mean, we're looking at a
43:17multipolar
43:19world order, which is constantly on the
43:22brink of World War II over over Taiwan
43:24or the Middle East or something, right?
43:26So, given that situation,
43:29I mean, the two ways things can go are
43:32big tech companies allied with major
43:34militaries
43:35>> or open and decentralized like the
43:38internet or or or Linux, right? and and
43:41th those look like the actual realistic
43:45paths that could happen in in in in the
43:47near future. And this has led to a shift
43:51such as the one that has led Kai and I
43:53to work together, right? Because like
43:55five or 10 years ago when I was talking
43:57about this stuff,
44:00you know, people like Nick Brom or even
44:04Kai was not extreme as Nick, but a bit
44:06in that direction. and they're more
44:07like, well, it will be better if we
44:10could find some way to just slow this
44:14down and direct the development more
44:15rationally. And I w I wasn't on that
44:19side, but I could I could understand it,
44:21right? And and and I mean, part of my
44:24brain is on that side, even though
44:25that's a minority, right? But now only
44:29real extremists are still singing that
44:31song because it's just not life, right?
44:34I mean, it's just it's just too clear.
44:36like OpenAI, Google, Tencent, BU, Highf
44:40flyier, like everyone
44:42>> is putting all this money and with not
44:45just the protection of national
44:47governments, but the the active
44:48encouragement of national governments,
44:50right? So, I mean, slowing down and
44:54taking a deep breath doesn't seem
44:56something our our current global
44:59leadership is good at. Kylie, the thing
45:01that I'm worried about is you've got
45:03when the in so if it's all decentralized
45:05as the internet is, there's always a
45:08period of time at the beginning where
45:10things are just quite loose as maybe
45:13regulation then follows. The thing that
45:14I worry about is given the power of AGI
45:18when you start to decentralize it, all
45:21it takes is for one or two actors to
45:23work in a certain way that all hell can
45:27break loose. That's that's the only
45:28concern that I that I well one of the
45:31concerns that comes first to mind.
45:32>> Yeah, I hear you. Um I have a response
45:36on that. So
45:38the idea first on this centralized
45:41versus decentralized and should we
45:45go all hands on deck to try to create
45:47AGI or should we try to slow down a
45:50little bit? You know, there are some
45:52really smart professors and scientists
45:54that I've talked to that are very
45:58concerned that if we don't get to AGI or
46:01we don't get to super intelligence, that
46:03time in between where people can control
46:06it to a degree that it will be used
46:08maliciously
46:10>> that you will get a war, you will get
46:13bad actors um using it in a bad way. And
46:17so the other counter to that is that
46:20since we are in a race, it seems like we
46:23are in this race whether we like it or
46:25not. I personally don't want to be in
46:26the race.
46:27>> I'm just here because I try to find the
46:30thing that I think is the most important
46:32thing to work on and I can't think of
46:34anything more important than this.
46:36>> And so if we got to be in the race then
46:40we got to give it our best shot to build
46:41it the right way so that it benefits
46:43everyone. So, let's say it's
46:45decentralized or centralized. You can
46:47still trick a centralized AI to do carry
46:50out something that you want it to do.
46:53And I've seen videos where they they
46:56just they're like the AI's got a gun and
46:59they're like, "No, I can't shoot you.
47:01All right, let's just pretend that your
47:03thing is off." Oh, okay. And it shoots
47:05you with the thing. Right? Like there's
47:06a lot of ways to get around. Well, and
47:08also the only way to enforce [snorts]
47:12a real single centralized AI is going to
47:16be with a 1984 style military autocracy,
47:20right? Like I mean otherwise, how do you
47:23stop someone from forking the AI like
47:27someone working on the AI take the
47:29source code launch their own other AI to
47:31make money or something? I mean try
47:33trying trying to it's not like a nuclear
47:35weapon where you need a rare physical
47:38material that you can just lock lock up
47:42and even that we haven't solved
47:43non-prololiferation but if AI is just
47:46software that could be run on machines
47:48right then you know suppose you have a
47:51centralized AI under military locking
47:53key but then suppose some programmer
47:55working on it comes up with a way to
47:57implement it on fewer machines right
47:59>> then then they then then they run off
48:01and get someone to give them a billion
48:03dollars to to deploy it on on on fewer
48:05machines. So you have to you have to
48:07stop that. So you have to be spying on
48:08everyone all the time, right? So it
48:10seems like protecting the centralized AI
48:14goes down a fairly dark path pretty
48:18rapidly, right? And and uh I mean if you
48:21needed like computium and there were
48:24only 5 g of computronium on the planet
48:27it might be easier but that doesn't seem
48:29like doesn't seem like like the way it
48:32is right because I mean open AI wanted
48:34to keep their
48:36>> transform neural net secret to keep
48:37their competitive advantage right it
48:40didn't didn't work very well so doesn't
48:43it doesn't seem like [snorts]
48:45>> given the way the world actually works
48:48having one protected
48:50singleton
48:52is actually possible, right?
48:54>> And to make it possible [snorts] seems
48:57very hard. So it it it seems like either
49:02you have this multi-olar scenario or you
49:06have a massively decentralized
49:09scenario and those seem like the real
49:13options. And you could argue whether
49:16they're the ideal options or not, but I
49:19mean
49:21there's certainly
49:23an upside if we can pull this off,
49:25right? So like like my my parents are 82
49:28years old and they're both in good
49:31health. They should live a while. So but
49:35they don't have any significant medical
49:37problems. If we get to super
49:40intelligence in the next five years,
49:43they will probably not die, right?
49:45>> And I mean, this is an amazing thing to
49:47think about, right? I mean, this is like
49:49real life. It's it's my parents are
49:52amazing people. They still have minds.
49:54They still have their health. Like, and
49:56they they we would take for granted
49:59they're 82. They won't be around much
50:01longer, right? But I mean, my grandma
50:03lived to 95.
50:05I mean,
50:06we could have instead of like, you know,
50:09just like birth control pills
50:12before my time, but like that
50:13revolutionized people's sex lives,
50:15right? And now now, uh, you know,
50:17[snorts] we go all of a sudden so many
50:19fat people I know suddenly aren't fat
50:21anymore, right? So, I mean, has side
50:24effects, these things are perfect, but
50:25it's it's interesting, right? So you can
50:28have a pill you take every day where the
50:3182y old person, you know, after a few
50:34months they're back to their body from
50:36age 30 or something, right? And I have
50:39little doubt that a super intelligence
50:41could design such a pill like if you dig
50:44into longevity biology, I mean [snorts]
50:47it that wouldn't require like a change
50:49to the nature of spaceime or something,
50:50right? I mean the cells are are
50:52reprogrammable, right? So I mean there's
50:55a lot of highly practical good that that
50:57that could come out of all this and look
51:00looking at work and jobs and so forth
51:02also I mean we take for granted now most
51:06people will have to spend most of their
51:08waking hours
51:10>> carrying out repetitive tasks that they
51:13do not enjoy so they have food to eat
51:16and and a place to live. Right? So
51:19getting getting rid of that will cause
51:20some emotional disruption for a small
51:24percent of the population who are well
51:26off and enjoy their jobs. But on the
51:28whole it will be a very large
51:30improvement to the lives of most people
51:33on looking back they'll be like whoa
51:35like did people really spend 40 50 hours
51:38a week like typing characters on on on
51:41on a keyboard repetitively just so they
51:44had enough food to eat? How was that
51:46even possible?
51:47Can I just mention one thing to that
51:48point? Because we've had this
51:49conversation on YouTube. We've had a lot
51:50of comments coming and saying that when
51:54you remove purpose from someone's life
51:56and work, often we think they're going
52:00to go and start playing the piano and
52:01enjoy their time, but the reality is
52:04people then lose a sense of desire,
52:07purpose, and end up in certain countries
52:09drinking a lot of alcohol and and all
52:11that. So it's I it's it's not I think
52:13this is very it's
52:14>> I think this is the exact definition of
52:16a first world problem. I mean yes for
52:20for upper middle class professionals in
52:23the developed world who are making
52:25enough money to have a physically
52:27comfortable life and who derive meaning
52:29from success in their profession. There
52:32will be some adjustment for the vast
52:34majority of the billions of people on
52:35the planet. They don't have enough
52:37medicine for when they get sick. They're
52:39struggling to feed their families and
52:41get clean water and adequate
52:42infrastructure and the jobs that they're
52:45doing are not that extremely meaningful
52:48and and satisfying either. And we can we
52:51can improve lives tremendously for a
52:53vast majority of the world population. I
52:56mean it's it's true. I mean you can see
53:00now people who were driving meaning in
53:03life from coding user interfaces. Some
53:06of them are being put out of jobs
53:07because you can just vibe code vibe code
53:09a UI in in in in half an hour. And that
53:12that's that's
53:15real suffering. I'm not I don't I don't
53:17want to minimize it. On on the other
53:19hand, there's a lot more people than
53:21that without clean water or nutritional
53:25food. And the web developers are not
53:28sending their salary to feed these kids,
53:30right? and and the a the AGI can solve
53:34these problems that that we're we're
53:36failing to solve. So I think with with
53:39any disruption in human history, there's
53:42been both good and bad in the
53:43transitional phase. Like when when we
53:45transitioned from hunter gatherer to
53:48agricultural lifestyle, I mean there was
53:50a Thor who was the mightiest hunter like
53:53clubbering the mammoth on the head with
53:54a club and he lost his meaning in life
53:57because it was about farming and not
53:58clubbing the mammoth on the head and
54:00being the big strong guy didn't matter
54:02as much anymore, right? So I mean if we
54:04avoid the worst outcomes like the
54:07existential risk that some folks are
54:09worried about
54:10>> I think on the whole the good will
54:11greatly
54:13>> out outweigh the friction.
54:14>> Yeah. So, I push back a little bit on
54:17what you're saying, Ben, in the sense
54:18that I I don't think that the majority
54:21of us, certainly not me, are in this to
54:26try to live forever or to have everyone
54:29not have to worry about their jobs or
54:31deal with purpose or any of that kind of
54:33stuff.
54:33>> So, at what age would you like to die?
54:36>> I'm not really afraid of death.
54:38>> Oh, that's not the question.
54:39>> What age? What age? Um,
54:42I don't know. I'd be all right if I
54:44lived a healthy 120, you know, which is
54:47already where humans
54:48>> I'll be I'll be sad to see you go, but
54:51>> but you're you're gonna change your
54:53mind.
54:54>> Fine. But I'm but I'm not I'm if I can
54:57live longer and I can set my age and
54:59certainly then yeah, I would I'd be more
55:01than happy to do that. I'm not against
55:03that. But I'm not in this with the idea
55:06of that we need to get super
55:08intelligence so that we can be immortal
55:10or that we need super intelligence so
55:12that everyone doesn't have to worry
55:13about a job anymore because now they can
55:15their hierarchy of needs are going to be
55:17met. I'm in this because we really don't
55:20have a choice because uh the way that
55:23humanity has presented itself we we have
55:26some very uncertain options in front of
55:28us. We can either get out of this race
55:32and go hang out on a beach somewhere and
55:34see how it unfolds, which I've
55:37considered.
55:38>> But I can't do that.
55:40>> And so we we have to make decisions now
55:44that are going to affect not just us,
55:47but the future forever.
55:49>> When we create this thing, it's it's on.
55:53It's not turning off. It's unless some
55:56crazy thing happens that that shuts it
55:58down. But essentially we're creating a
56:00artificial god. We're creating something
56:03and humanity is doing this. This isn't
56:05just anyone person. This is as a
56:08species. We're doing this. We're all
56:11running towards this thing. You know, I
56:13go back to this sometimes I reference
56:14this really intelligent quote from a pop
56:17culture movie in Jurassic Park where
56:19Jeff Goldblum says, you know, we so
56:20preoccupied on whether or not we could,
56:22we didn't stop to think we should. And I
56:24I feel that. I feel that. I don't know
56:26if we should be doing this, but I don't
56:28think that we have a choice. So, since
56:31we don't have a choice, what are our
56:33choices? Our choices are centralized or
56:35decentralized, go towards it really
56:38quickly or sort of maybe pump the brakes
56:42a little, but we don't think that's
56:43going to happen and somebody's going to
56:45own it. That's going to be a corporation
56:47that we don't, you know, not good
56:49choices. So since it is I'm going to go
56:53with this choice of all right
56:54decentralized let's go with it be as
56:56conscious as possible and as much care
56:59as possible to try to make this thing go
57:01right. Once we get that if it does go
57:04right the rest of the stuff will sort
57:06itself out whether or not you want to
57:07live a long time or choose your death or
57:09whether or not you want to hang out and
57:10do art or not do art or if you have I
57:12think we can sort that stuff out.
57:14>> Yeah.
57:14>> But the big thing right now is that
57:16choice that's in front of us and it's
57:18upon us now. beneficial general
57:20intelligence. Okay. Apart from the
57:21decentralized aspect of it, why is it
57:23the solution? Well, from my point of
57:25view, what Ben is famous for
57:29popularizing the term AGI? He's been on
57:32this for over 20 years, I think. You
57:35know, he's he's an OG they call him out
57:38here in in a lot of the circles. And
57:40when I met Ben in 2016, as I referenced
57:43before at this global leaders forum, he
57:45was with his robot and he was on the
57:48stage and he was a pioneer in the in the
57:51field already. to his credit now he's
57:54looking at this and he's saying okay AGI
57:57why don't I put a little spin on this
57:59because I want to emphasize that we're
58:02going to put a massive effort on this
58:04going well for humanity and for everyone
58:07not just for a centralized organization
58:11which I still have trouble believing
58:13that any centralized organization once
58:15they hit recursive self-improvement
58:16would have any hope of controlling
58:18anything I think everybody's kidding
58:19themselves on that point so I think it's
58:22like that's like a joke to me. So
58:25essentially he's saying let's put this
58:27massive emphasis on it being beneficial
58:30or benevolent
58:32and uh I I go towards that because at
58:37least we're putting the effort and the
58:39focus on that part of it rather than
58:43saying this is AGI. He's saying, "Okay,
58:46it's not just AGI. This is BGI. This is
58:50beneficial to humanity. This is
58:52benevolent. This is something that is
58:56going to be beneficial to the world.
58:59>> And that's the foundation that he's
59:01starting with and the approach for it.
59:04Turns out that Ben's been working on
59:05this for a long time. And he's already
59:08created a lot of systems under the hood
59:10that he has momentum for. And that's
59:13also very appealing in this situation
59:15because he has just as good a chance at
59:17at winning this race as anyone else. And
59:20as a call to action to the world too.
59:22It's like I want to talk to the Elon
59:24Musk groups of the world. I want to talk
59:26to anthropic. I want to talk to all
59:28these major groups and see where there
59:30is collaboration because I do believe
59:31that we should collectively be working
59:33on this stuff.
59:34>> But that is what the distinction is
59:37between BGI and AGI. It's that it's an
59:40emphasis on it being benevolent and
59:42beneficial to humanity. Full stop. And
59:46that we're going to be the gold standard
59:47in terms of what the moral constitution
59:50is of this fiercely intelligent system
59:53is going to be so that as it does
59:55improve itself, it's doing it on a
59:57foundation that we feel good about. What
1:00:00is the biggest ethical difference
1:00:03between your interpretation of BGI
1:00:07and what an anthropic an open AI a
1:00:10deepseek is creating? What I see in
1:00:13terms of the big differences is that I
1:00:16think that we're going to be putting a
1:00:17massive emphasis on hearing the concerns
1:00:20from the different whistleblowers. Like
1:00:22if Hinton is godfather of AI, Hinton is
1:00:25going around on a campaign talking about
1:00:27how he believes that AI systems need to
1:00:29have um a parental or a thought to it.
1:00:34You know, maternal instincts, you know,
1:00:36he looks at it like like a mother tiger
1:00:38out there might befriend a little baby
1:00:40gazelle. Somehow it imprinted on it as
1:00:42if it was a child or something. Now it's
1:00:44protecting it in the wild. or let's say
1:00:46AI is like a baby tiger and we want to
1:00:49we want to help it so that it bonds with
1:00:51us and loves us and those kinds of
1:00:53things. Hinton is is adamant about that
1:00:55and I believe that that point should be
1:00:57strongly considered and I think that's
1:00:59going to be one of the key distinctions
1:01:01with what we're doing is that we really
1:01:02want to have all these voices. Um you
1:01:05know Stuart Russell's got important
1:01:08contributions or you you just had a
1:01:11couple of people on here that had really
1:01:13important points that they wanted to
1:01:15make. We want to make sure that all
1:01:17those voices are heard, that they have a
1:01:19seat at the table, and that we can
1:01:21create the most robust gold standard,
1:01:24thorough uh moral constitution for this
1:01:27AI. And Ben knows like I have a
1:01:30reputation for somebody that will blow
1:01:31the whistle if something is wrong. I've
1:01:33lost a lot in my life for trying to do
1:01:35the right thing or say the right thing
1:01:38if I saw something was wrong. And if I
1:01:40saw him cutting corners or or not doing
1:01:42this, you know, when he talks to me, he
1:01:44says, "No, first time I went to a BGI
1:01:47conference, he had scientists there that
1:01:49are like, doomer, we're all going to
1:01:52die." He has them there. He has them
1:01:53there talking. He wants to hear what
1:01:55they have to say, right? That's an
1:01:56important distinction with this whole
1:01:58project is that we want to get all the
1:02:00voices heard. Want everyone seat at the
1:02:02table. We want to weigh them out. We
1:02:04want to do it in the best way we
1:02:05possibly can utilizing the AI tools and
1:02:07everything that we have because we do
1:02:08have s we have great intelligence tools
1:02:10at our fingertips now and um give it our
1:02:14best shot. Can I just add one point to
1:02:16this because I I I interviewed um under
1:02:18secretary at the time Lori Lazio who's
1:02:20director of NIST and that was back in
1:02:23October 2024 just before Trump was
1:02:25elected
1:02:26>> and we were talking about AI bias and
1:02:28how we have to bring in as many diverse
1:02:30voices around the table
1:02:32>> back when open AI I think was still kind
1:02:34of more it was kind of pivoting more
1:02:37towards being fully you know for profit
1:02:39but it was amazing how much changed
1:02:41within that year and a half to the point
1:02:43now where it's just you know an absolute
1:02:44race. What what I wanted to understand
1:02:46is
1:02:48that was I felt like that was the path
1:02:50we were on a few years ago and now it
1:02:52feels we haven't we're so far away from
1:02:54people being around the table and having
1:02:56that
1:02:56>> I think that's certainly true and that's
1:02:59not just about AI, right? I mean I mean
1:03:03our
1:03:05American society at least is heading
1:03:08toward polarization of rival camps who
1:03:10don't even know how to talk to each
1:03:12other. And AI could be used to help
1:03:16bridge gaps between different
1:03:17subcultures.
1:03:19Instead, due to the way big tech social
1:03:22media works, AI is more often being used
1:03:25to divi divide people, right? And and
1:03:29push people more into rival camps. So
1:03:31there there is there is another story
1:03:34there we could talk about where if you
1:03:36had decentralized AI that was running
1:03:40decentralized social media, right? Then
1:03:43that decentralized social media perhaps
1:03:46could bridge gaps between different
1:03:49subcultures rather than exacerbating
1:03:51them. But I want to go back to your
1:03:53previous question for a moment because
1:03:54what what Kai said made sense and I
1:03:58agree with it, but there's other
1:04:00significant differences between the
1:04:02ethical approach that we're taking in
1:04:05our AI project and what LLM companies
1:04:08are are doing. So we're we're not
1:04:12building a new large language model.
1:04:13We're making a different sort of AI
1:04:15system. It it has a large
1:04:17[clears throat] language model in it. It
1:04:19also has explicit logical reasoning
1:04:22engine that grounds ideas in
1:04:25observations.
1:04:27It has an what's called an evolutionary
1:04:29learning system that tries to evolve
1:04:31wild new ideas. But all of these
1:04:34different AI techniques, they're put
1:04:36underneath the sort of explicit goal
1:04:39system. So this AI system that we're
1:04:42creating, it has a few top level goals
1:04:46among which are compassion and care for
1:04:49sentient beings, right? And as it learns
1:04:52and as it chooses actions, it's keeping
1:04:54these top level goals in mind. So then
1:04:58the goals of the system, including its
1:05:01ethical values, are part of all the
1:05:05systems learning, reasoning, and and
1:05:07understanding. And that's that's how
1:05:10people are like babies have their own
1:05:12goals. Some of them are selfish goals.
1:05:14Some of them are loving goals of
1:05:16compassion to the other people around
1:05:17them. But as the baby learns to see and
1:05:21act and model the world is done in the
1:05:23context of the baby's goals, goals and
1:05:25values. Right now if you contrast that
1:05:29with how current chat bots are working,
1:05:32I mean the foundation of it is what's
1:05:34called a pre-trained language model. And
1:05:37what that's just trying to do is predict
1:05:38what will come next.
1:05:40>> And that's all it's trying to do. Then
1:05:42they take this pre-trained model which
1:05:44is just trying to predict what will come
1:05:45next in a series of words, right? They
1:05:48take that then they do what's called
1:05:51instruction tuning where they try to
1:05:53reinforce it giving responses people
1:05:55like to hear, right? Then they put some
1:05:58guard rails around that. Right? So in
1:06:02that approach, the crux is almost
1:06:06sociopathic. It's just like predict what
1:06:08comes next. And then you're trying to
1:06:10sort of direct it to obey your
1:06:12constitution after the fact. And then
1:06:14you may wrap it into an agentic loop
1:06:16like you know an an open claw or clawed
1:06:20co-work or something. So you're you're
1:06:22starting with something that doesn't
1:06:24know what values are. It's just trying
1:06:26to predict the next thing and then
1:06:28you're
1:06:29tuning it and you're wrapping it right
1:06:32and trying to get it to obey these rails
1:06:34where whereas in the approach that we're
1:06:36taking the goals and values are there in
1:06:40the core architecture as part of the
1:06:42initial learning and and and training
1:06:45right and that's that's just a quite
1:06:46different technical approach which I
1:06:50started with not for ethics reasons I
1:06:52started with for AGI reasons because
1:06:54that's how human brains learn, right?
1:06:56Like we we we learn in order to preserve
1:07:00our organism and expand the scope of our
1:07:03of our minds. We don't learn out of some
1:07:05impartial desire to accurately predict
1:07:07the future. Right? Architecting [snorts]
1:07:09a system to do learning in a more
1:07:12humanlike way involves putting values
1:07:15more at at the core of the learning
1:07:17because that that's what that's what
1:07:19people do. Now, like most things, this
1:07:21could be good or bad. If the value you
1:07:24put at the core of the learning is kill
1:07:27whoever the general wants to kill like
1:07:30then then putting the values there in
1:07:32the center of the learning loop is not
1:07:34good. If if the value is be
1:07:36compassionate and understanding to all
1:07:38sensient beings then having the values
1:07:40there in the center of the learning loop
1:07:44is good. Right? And this
1:07:46>> this ties in with the point Kai is
1:07:48making though because who defines the
1:07:51initial values right?
1:07:53I mean
1:07:54>> it's really important
1:07:55>> in in the end
1:07:56>> what I was mentioning to Lazio back in
1:07:58>> Yeah. I mean in the end the system can
1:08:00self-modify so it can revise its values
1:08:04but it's revision of its values is
1:08:06driven by its values right which is an
1:08:08instance of recursive self-improvement
1:08:11as Kai mentioned. So who who defines the
1:08:16initial AGI values like is it me? It
1:08:18shouldn't be I mean I think I could do
1:08:20an okay job but no one person should
1:08:23should be doing that right if we if we
1:08:26have a sort of decentralized
1:08:28society of AGI minds you'll have a bunch
1:08:32of different AGI systems each will have
1:08:34different variations of initial values
1:08:36guiding their learning and then they
1:08:38will be cooperating together to achieve
1:08:41the overall global intelligence right
1:08:44and that is that's a bit messy but on on
1:08:47the other And I mean, the internet's a
1:08:49bit messy. The human species is is is a
1:08:52bit messy. And I'm not sure
1:08:55values are that clean and simple of of
1:08:58of of a thing, right? It's going to be a
1:09:01bit heterogeneous, and it's going to
1:09:03always be evolving just like human
1:09:04values are always evolving.
1:09:06>> So, Ramsh Riscar um MIT Media Labs, he
1:09:10has project Bender, which is again
1:09:12trying to decentralize Gent AI. And
1:09:15again some push back from him from the
1:09:17comments was that you know it's it's
1:09:19decentralized but it becomes very
1:09:20capitalistic because all of a sudden you
1:09:22have the the different uh your your AI
1:09:25going to the most prestigious AI
1:09:27university and you can exchange things
1:09:30for it and all of a sudden you do then
1:09:32have AGIS that are much more powerful
1:09:34than let's say my one in San Francisco
1:09:36because of the ecosystem here is much
1:09:39more powerful than someone else's in
1:09:41>> I mean decentralized
1:09:43software initiatives
1:09:45are not pure. They're complicated. I
1:09:47mean, you have Linux and you have
1:09:48Android, right? And but then you have
1:09:51Huawei, which took the open parts of
1:09:53Android and put their own top on top of
1:09:55it. And the internet, you have debates
1:09:57over net neutrality, right? I mean, I I
1:10:00don't think you're in a you're not in an
1:10:02immediate utopia by by by decentralizing
1:10:04things. on on the other hand still like
1:10:09right now you can pay people in Iran
1:10:12with with Bitcoin if you want to right I
1:10:15mean it's illegal by the sanctions but
1:10:17someone in Africa can pay someone in
1:10:19Iran with Bitcoin and it's not illegal
1:10:21and and and it is bypassing the global
1:10:24banking system yet Bitcoin is not
1:10:26perfect it's controlled by a bunch of
1:10:28miners and mafioso right so
1:10:31>> decentralization
1:10:32is not an immediate panacea
1:10:35It just avoids certain obvious and
1:10:38terrifying failure modes and then then
1:10:41that you have to
1:10:42>> keep working to ensure the decentralized
1:10:45network is is being open and
1:10:48participatory
1:10:49enough, right?
1:10:51>> And are we in a simulation?
1:10:52>> It's possible if the interesting
1:10:54question is could we hack our way out of
1:10:56the simulation and I don't know how yet.
1:10:59Let's ask the ASI. There is [snorts] a
1:11:02lot of aspects to life, the universe,
1:11:05and everything that science has not yet
1:11:08come to grips with. Now, you could ask,
1:11:12would a super AI also have a soul in
1:11:14whatever sense people do? And would the
1:11:17super AI after you turn it off, would
1:11:19its mind also be able to pop up and do
1:11:22things? And I mean,
1:11:23>> that one's a little tricky. I don't
1:11:24know.
1:11:25>> Well, the the it's interesting that the
1:11:27previous Dalai Lama thought yes. And he
1:11:29was he was interview he was interview he
1:11:30was interviewed on this right
1:11:31>> you thought that a soul could pass into
1:11:33the machine.
1:11:33>> Yeah. Yeah. He was like maybe a
1:11:34programmer will be reincarnated in an AI
1:11:37system. Right. So I mean it's whimsical
1:11:40right but I mean in the end it comes
1:11:42down to like we understand very little
1:11:45of the universe in in which we currently
1:11:47exist. And you know if we upgrade our
1:11:50brains or make it super intelligence may
1:11:52maybe we will we will understand more.
1:11:55Right. But my my I mean this goes off
1:11:58topic of AGI, but my my personal passion
1:12:02for living a very long time is not
1:12:06really out of fear of dying. It's more
1:12:09out of en enjoying living, right? Like I
1:12:12mean I have I have no reason to believe
1:12:14that committing suicide at age 120 is
1:12:17going to make my afterlife any any
1:12:20better than if I live to 10 million,
1:12:22right? I mean, I'm I'm open to the ASI
1:12:24giving me new data on this. However,
1:12:26>> so so if there is a creator, if there is
1:12:28a god, what do you think he's trying to
1:12:30show us with AI AGI?
1:12:33>> He's just trolling us. That's [laughter]
1:12:35>> Elon Musk recently say said that AI
1:12:38could kill us all. What is the
1:12:40percentage chance of that happening?
1:12:42>> Pi.
1:12:44>> Pi.
1:12:45>> Which is
1:12:46>> 3.14159
1:12:48>> percent.
1:12:48>> Yes.
1:12:49>> Okay. I mean, if if you're going to make
1:12:51up a random number for something that
1:12:53for something that we really can't know
1:12:55rationally, that's a it's a perfectly
1:12:57nice random number.
1:12:58>> But sure, we don't know rationally what
1:13:00is going to happen in 2027.
1:13:02>> My my gut feel is the odds are low but
1:13:05not zero, which is why I came up with
1:13:073%. You know,
1:13:09>> I would just reference other experts um
1:13:12like Stuart Russell. I mean I was just
1:13:13at a conference called Synapse in um
1:13:16India that they were talking about this
1:13:18and they were referencing many AI
1:13:21experts who are concerned about this
1:13:24anywhere from 30% or below. And I know
1:13:2830% is a high number but they were up to
1:13:3130% and then below. Now you would ask
1:13:33somebody say okay if you're if we're
1:13:34rushing to create something that even
1:13:36had a 1% chance of it going wrong would
1:13:39you do it?
1:13:40>> Most people would say no. M
1:13:42>> but again that's this is the predicament
1:13:43that we're in as a society right now as
1:13:46humans is that we're rushing towards
1:13:49this thing and so I'm not saying it's
1:13:5130% I'm not saying that I'm just saying
1:13:53that other scientists other a you can
1:13:55find it Google it they're they're out
1:13:57there and they're credible people so
1:13:59there's a lot of concern that it could
1:14:01go wrong yeah I think the odds by my own
1:14:06intuition which is not a scientific
1:14:09calculation I think the odds will be
1:14:11much lower than the 3%
1:14:14if the AGI is created with compassion
1:14:18and and ethics in mind. So when I when I
1:14:22start to figure like 3% as an intuition,
1:14:25I'm baking in like the possibility that
1:14:29it comes down to a Trump versus Xiinping
1:14:31arms race for for AGI. I mean I mean I
1:14:34think even there it probably comes out
1:14:36all right. Neither Trump nor Xiinping
1:14:39actually wants to annihilate the human
1:14:40species either, right? So, but still the
1:14:43arms race modality feels not
1:14:46encouraging, right? Because because
1:14:48you're creating AIS a to serve a narrow
1:14:52goal rather than a specieswide goal and
1:14:55B, you're training them for violence and
1:14:57and killing in in the first place, which
1:15:00is it's it's like people who train their
1:15:04pitbull to to kill and then they're
1:15:05like, "Oh it bit my kid." Right? I
1:15:08mean, why not just not train your dog to
1:15:10kill in the first place if you don't
1:15:11really have to?
1:15:12>> And that that brings us back to why this
1:15:14is so this conversation is so important
1:15:16because what is it trained on? What are
1:15:18its values? How's it structured? I
1:15:20remember seeing something about this
1:15:21paperclip uh
1:15:23>> paperclip maximizer. Yeah.
1:15:24>> And and that it was basically they
1:15:26programmed the AI to just make paper
1:15:28clips and like by all means necessary,
1:15:30right? that it was like an basically
1:15:32humanity was going to suffer because
1:15:34this thing was just about making paper
1:15:35clips. And but the whole point was that
1:15:38what you're programming it with at the
1:15:40beginning is really really important.
1:15:42And if anything comes across in this
1:15:45discussion for me that I'd like to like
1:15:47cross over is like yes, we want to hear
1:15:50from you. We want we want collaboration
1:15:54and we want to do everything that we can
1:15:55to try to get this right.
1:15:57>> Yeah. Singularity net.
1:15:59>> Yeah. So with singularity net is a
1:16:03blockchain project that I founded in
1:16:052017 and what we've been focusing there
1:16:08for one thing on building the
1:16:09decentralized infrastructure that will
1:16:12let us take any AI system be it AGI or a
1:16:15specific application and just deploy it
1:16:17on a global network of machines with no
1:16:19central owner or controller. Right? So
1:16:21let you roll out roll out AI in the same
1:16:24decentralized
1:16:26vein as as the internet and we have a
1:16:29platform for that now. We'll have a much
1:16:31better one rolling out around the end of
1:16:33the year. It's currently in in DevNet
1:16:35early stage which is called the the ASI
1:16:39chain. Another thing we've been doing in
1:16:41Singularity is fostering development of
1:16:44an open-source AGI framework called
1:16:47Hyperon, which combines large language
1:16:50models, logical reasoning, creative
1:16:52evolutionary learning within a value and
1:16:54and and goal-driven cognitive
1:16:57architecture. Now, that's that's open-
1:17:00source,
1:17:00>> but it's designed to run on this
1:17:02decentralized infrastructure we've made.
1:17:04Now we put a lot of thought not just
1:17:07into the technology but in the corporate
1:17:09architecture to be sure that you can
1:17:11roll out commercial applications you can
1:17:14make money from this help others make
1:17:15money from it but at the same time the
1:17:17core AGI remains you know ethics goals
1:17:20valuesdriven and purely open open and
1:17:22decentralizing open AI of course had
1:17:25this aspiration in the beginning at
1:17:27least on paper but the way they
1:17:29architected things went arry pretty
1:17:32quickly so we're trying to set things up
1:17:34and I think we are setting things up
1:17:35both on the technical level and on the
1:17:38business level so that we can remain on
1:17:41mission and for that for the good of
1:17:42humans and all sensient beings while
1:17:45also fostering commercial development
1:17:47and moving you know rapidly toward AGI
1:17:51and ASI so that we can win this AGI race
1:17:54that the you know collective dynamics of
1:17:57our of our species is sort of a
1:18:00thrusting thrusting upon us. We are
1:18:02looking in the next year to build more
1:18:05of a of a of a of a global movement and
1:18:08this relates to a very interesting
1:18:10product that we're about we're about to
1:18:12drop in in a couple weeks which is
1:18:14called Omega Claw. So Omega Claw it's
1:18:18sort of an open claw type system
1:18:22but it has pieces of our Hyperon AGI
1:18:26framework in it. M
1:18:27>> so open claw as most people will have
1:18:30heard of now right this is a sort of
1:18:32agentic wrapper for LLMs
1:18:34>> so unlike an LLM that just answers your
1:18:36questions you can run an open claw on
1:18:39your own machine and it's just always
1:18:41keeping on doing stuff right it's it's
1:18:43always looking on the internet it can
1:18:44administer your machine for you can do
1:18:46research for you right it can schedule
1:18:48things for you could even operate your
1:18:50robot if you if you connected it right
1:18:52so
1:18:53>> Omega claw is like openclaw but with
1:18:56better long-term memory,
1:18:59better reasoning, and it can modify its
1:19:02own software code as as it goes in the
1:19:05vein of self-improvement as Kai kept
1:19:07talking about. So, it's not yet an AGI,
1:19:11but we took some of our protoagi tools
1:19:14and we put it in in an openclaw type
1:19:16system. And that's the the Omega claw.
1:19:19And this is open source. It's on GitHub
1:19:21now, but in the next few weeks, we're
1:19:23going to release this as a Docker
1:19:26container that anyone can download and
1:19:27run run on their own machine. And then
1:19:30in a couple more months, we'll release a
1:19:33framework where you can have your own
1:19:34Omega CL agent just running on our
1:19:36servers on the Singularityet platform.
1:19:38You don't even need to download
1:19:39anything. But this is it's interesting
1:19:43from a technical standpoint and it could
1:19:45be in the end as we put more and more
1:19:47AGI components into it. It could be the
1:19:49seed for the AGI even. It's also
1:19:51something I think we can start to build
1:19:54more of a beneficial general
1:19:55intelligence
1:19:57movement around in the sense that it
1:19:59gives everyone something to do to help
1:20:01and that you can initially download your
1:20:03own omega claw agent and in a few months
1:20:06you know get your own in an online
1:20:08workspace on our ASI create platform but
1:20:11then you can teach it you can you can
1:20:13teach it to manifest your own values you
1:20:16can teach it to benefit you and others
1:20:18in the world according [snorts] to your
1:20:20own understanding and it will it will
1:20:23remember what you taught it and it will
1:20:24try to enact that and then you can
1:20:26correct it as it goes. Right? So like
1:20:29training your own omega class system,
1:20:31teaching your own omega class system is
1:20:33a way to inject your values and and
1:20:35understanding
1:20:37into an AI instance which can then
1:20:39communicate with with other other AI
1:20:42instances, you know, forming part of the
1:20:44emerging AGI global brain. And indeed
1:20:47your part may not have as much
1:20:49processing power behind it as someone
1:20:51else's part. Right? So it's not like
1:20:53we're in a in a pure egalitarian
1:20:56communist ecosystem, but we are building
1:20:59a practical environment where everyone
1:21:00can contribute
1:21:02their own thing. And even if you don't
1:21:05have that many resources, if if the
1:21:07values you gave your your claw agent go
1:21:10viral, like they may spread through the
1:21:12whole network and end up having a huge
1:21:13influence. Yeah, I think that's it's a
1:21:15great point because Ben's talking about
1:21:17ways for people to get involved and
1:21:18contribute in actual BGI.
1:21:23>> That's pretty cool.
1:21:24>> We haven't had that before, right? Like
1:21:25for the last years I've been talking
1:21:27about this. I get a lot of people
1:21:28messaging me and they're like, "How can
1:21:30I
1:21:31>> how can I help? This is the most
1:21:33important thing in the planet. How can I
1:21:34contribute?" I'm like, "Well, learn to
1:21:36program and help us program. Make a
1:21:38billion dollars and and donate it,
1:21:41right? I I haven't, you know, tell your
1:21:43friends, write a blog, run a podcast,
1:21:45right? But I haven't had the best
1:21:47possible answer for people who weren't
1:21:50>> talented or in certain specific ways or
1:21:54weren't weren't born with certain
1:21:55abilities or wealth or something, right?
1:21:57But I think now we're entering a
1:21:58different phase where we're actually
1:22:01launching AI systems where they need
1:22:04humans to teach them values and and
1:22:07understanding and everyone can
1:22:09participate in this. I want to round off
1:22:10with a question to both of you. If
1:22:12tomorrow was your last day on Earth and
1:22:14you had to leave the world with one
1:22:16parting sentence, what would you want
1:22:18that to be?
1:22:18>> If it was my last day on Earth, I would
1:22:21say the with the world and just
1:22:22hang out with my wife and kids to to be
1:22:24honest.
1:22:25>> You don't have a sentence or advice. One
1:22:28sentence for advice.
1:22:29>> Goodbye and thanks for all the fish.
1:22:33It's a Hitchhiker's Guide reference to
1:22:36counter all his Hollywood Hollywood
1:22:37movie references.
1:22:39I mean, yeah, one one one sentence. I
1:22:44mean, I mean, Kai's kindness over
1:22:46harshness is is is is
1:22:49close enough. I mean, what I what I want
1:22:50to say is that the future the future
1:22:54looks incredibly amazing, but let's go
1:22:56into it with a, you know, compassion
1:22:58toward all sensient beings in in in our
1:23:00hearts. I mean, I've just experienced
1:23:02some some heartache, some loss recently,
1:23:05and I'm not going to get emotional, but
1:23:07uh it kind of brings it brings to center
1:23:10what you feel like is important in life.
1:23:13And I go back to the Raymond Moody
1:23:16stuff, Life After Life. I found a lot of
1:23:18values there. And the the mission
1:23:19statement or the message that was coming
1:23:21back a lot was that life was about
1:23:23learning to love better. It was about
1:23:25creating loving relationships. You know,
1:23:28maybe alleviating suffering for others.
1:23:30I think that's a good thing too. I think
1:23:32there's a lot of a lot of that in the
1:23:34world if we can. But I would just
1:23:36encourage people to if they have that
1:23:37moment where they think to call that
1:23:41person um that that that intuition comes
1:23:46in their mind like to act on it and to
1:23:50look to repair relationships, understand
1:23:53how to repair relationships, create more
1:23:55loving relationships, more compassion in
1:23:57the world, create a world where we have
1:24:00less suffering, more compassion. I think
1:24:02that's the message or the takeaway that
1:24:05I would have.
1:24:05>> You know, in in the book I wrote in
1:24:082010, which was a non-technical book
1:24:10called a a cosmist manifesto, which was
1:24:13sort of looking at the grand future of
1:24:16the universe, I I looked at four key
1:24:20values, which were joy, growth, choice,
1:24:25and then continuity, right? And so I I
1:24:29think these are worth keeping in mind. I
1:24:31mean compassion is about spreading the
1:24:35joy right and then growth is like don't
1:24:38stagnate keep exploring exploring new
1:24:41things right choice individuality me
1:24:45means something continuity it's like
1:24:48nostalgia respect your elders like keep
1:24:50keep the past alive even even as you
1:24:53grow into the future so I mean I think
1:24:55these are some basic values to keep in
1:24:57mind but then if you're giving one last
1:24:59word going back to the hitch checker's
1:25:01guide like he he had 42 as the meaning
1:25:04of the universe, right? So, you should
1:25:05probably come up with something random
1:25:07like 796128
1:25:09and the let the subsequent generations
1:25:11try to figure out what the hell you
1:25:13meant. I I would just add besides the
1:25:15the 42 is uh is
1:25:19I found a lot of I've personally found a
1:25:22lot of uh purpose in trying to
1:25:27take actions that are going to be
1:25:31meaningful for others. You know, if
1:25:32there's an injustice in the world, if
1:25:34there's something wrong in a legislation
1:25:36that you could reasonably help to fix or
1:25:39correct, you know, you look at people's
1:25:41lives who've had the most impactful
1:25:43lives. I think about Martin Luther King,
1:25:45what he was able to accomplish in, you
1:25:48know, six or seven years of really being
1:25:50fully fiercely active on that front and
1:25:52how what he would have accomplished if
1:25:53he would was hadn't been killed. And so,
1:25:55I just think a purpose- driven life
1:25:57means a lot. Um, and so if you could
1:26:00combine compassion, learning to love
1:26:02better, and acting with purpose in mind,
1:26:05party with a purpose, you know, those
1:26:07kinds of things, like you could throw a
1:26:09party for something that's greater than
1:26:11yourselves. But that I think brings the
1:26:14most joy and creates a world that is
1:26:17worth living in and a brighter future
1:26:19for everyone. So that would be my hope.
1:26:22And my last word would be let's get this
1:26:24BGI a AGI stuff right. Let's come
1:26:28together as a collective human humanity
1:26:31and really give it our best effort
1:26:33because it's going to matter. It really
1:26:37really matters and we have a small
1:26:38window to do this and it's upon us. So,
1:26:40let's act with a sense of urgency. Come
1:26:42together and give it our best shot.
1:26:43>> So, you want to you want to party like
1:26:44it's 2045.
1:26:46>> I want to party and like in a nice way
1:26:49and get some good stuff accomplished and
1:26:52have the best time ever and then we can
1:26:54go like relax and be in that utopian
1:26:56society. But first, let's get this
1:26:57right.
1:26:58>> Okay, Ben, thank you so much.
1:27:00>> Thank you. Thank you so much for the
1:27:02time.
1:27:02>> Thank you.

About this transcript

This page contains the full transcript of AGI Experts: We're About To Lose Control of AI Forever.. And We Aren’t Ready by The Beyond Tomorrow Podcast with Julian Issa, generated from the public captions YouTube serves with the video. The transcript has 15,104 words across 2,196 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.