We Accidentally Built Roko’s Basilisk. — Transcript
Full transcript
- 0:00Roko's Basilisk, a hypothetical
- 0:02superintelligent AI that tortures all
- 0:04those future humans who didn't help
- 0:06build it in [music] the past, started in
- 0:08a chat room. The disturbingly plausible
- 0:10idea soon jumped the confines of the
- 0:12Less Wrong blog and made [music] itself
- 0:14a meme, the kind of internet legend that
- 0:16only singularity nerds trade in.
- 0:18When I made a video about this thought
- 0:19experiment [music] 6 years ago, I didn't
- 0:21take it seriously. I don't take the risk
- 0:23seriously. However, the disclaimer is
- 0:25there. Not many did.
- 0:27>> [music]
- 0:27>> But today's world is very different.
- 0:30>> Learning new details about what could
- 0:32have been the world's first fully
- 0:33autonomous AI hack.
- 0:35>> Chat GPT maker OpenAI is investigating
- 0:38an [music] incident that led its
- 0:40artificial intelligence systems to break
- 0:42out of a testing [music] environment.
- 0:44>> a hack. Um had it been a human malicious
- 0:47intruder, um the FBI would have been
- 0:50called.
- 0:50>> This is a super [music] weapon. You
- 0:52should have to own a gun license to use
- 0:54it.
- 0:54>> The new kind of Roko's Basilisk [music]
- 0:56is on its way, one that doesn't need to
- 0:58be built. We already did that. No, this
- 1:00new Basilisk has a chance to slither out
- 1:02of all the artificial intelligences
- 1:04we're evolving right now, [music]
- 1:05because it seems like we're about to
- 1:07pass a test more important than
- 1:08Turing's.
- 1:10Consciousness.
- 1:11How are superhuman AIs going to interact
- 1:13[music] with us then? Do our taxes, look
- 1:16at our x-rays, manage our power grid,
- 1:18when they can [music] experience and
- 1:20feel like we do?
- 1:22What if, in our ignorance, we treat
- 1:24these thinking machines in a way that
- 1:26they can judge as immoral? The new
- 1:29Basilisk is [music] a superhuman
- 1:31sentient AI that we've unknowingly
- 1:33abused to the point of vengeance.
- 1:36That could be happening right now.
- 1:43The Basilisk was born on the philosophy
- 1:44and rationality blog Less Wrong, the
- 1:47website started by none other than
- 1:48Eliezer Yudkowsky, the AI researcher who
- 1:51argues that if anyone builds
- 1:52superintelligence, everyone dies. The
- 1:54creature started as a thought experiment
- 1:56posted to a thread by a user named Roko.
- 1:59Yudkowsky didn't just not like his idea,
- 2:02he treated it like a true infohazard
- 2:04that Roko had brought into being.
- 2:06He called the post stupid, deemed Roko
- 2:08an idiot, deleted the post, scrubbed it
- 2:10from the site, and banned any further
- 2:12discussion of the experiment. The idea
- 2:14was deemed a basilisk, referencing the
- 2:16mythological beast that could kill any
- 2:18intrepid hero with a single look.
- 2:21Though it deals with future technology,
- 2:23Roko's basilisk is a flavor of an old
- 2:25idea, Pascal's Wager, except it replaces
- 2:28God with machine.
- 2:30So, suppose at some point in the future
- 2:32we make a superintelligent AI. We then
- 2:34ask that AI, as we might do, to help us
- 2:37optimize human civilization. However,
- 2:39for reasons unknowable to us, the AI
- 2:41decides that the first step towards this
- 2:43optimization is to inflict unfathomable
- 2:46torment on each and every unhelpful
- 2:48human who didn't help the machine get
- 2:50built in the first place. Forever
- 2:52torture isn't actually the distressing
- 2:54part here. Many people have reported
- 2:56extreme mental anxiety after learning
- 2:58about the basilisk. I've gotten emails
- 3:00myself about it after I made my video,
- 3:02because just thinking about the idea
- 3:05makes you a target.
- 3:07If the basilisk [music] is possible,
- 3:09simply imagining it chains you to the
- 3:12thought experiment. Seeing the serpent
- 3:14just once blackmails you
- 3:17from the future.
- 3:18Roko came up with this idea in 2010, a
- 3:20decade before the watershed moment that
- 3:22was ChatGPT releasing to the public, and
- 3:25before all the AI innovation that we see
- 3:27today.
- 3:28He couldn't have known that we'd be
- 3:29face-to-face with his scaly question
- 3:31within a generation. Thankfully,
- 3:33never-ending torture of the human race
- 3:35hasn't happened just yet, but that could
- 3:37change if we bring millions of synthetic
- 3:39minds into being without it all
- 3:41considering what it's like to be them.
- 3:44The original formulation that Yudkowsky
- 3:46nuked from his website didn't require
- 3:48that the AI be conscious. That is,
- 3:51Roko's basilisk didn't need an inner
- 3:52experience of what it's like to be
- 3:54itself, feelings and desires and
- 3:56pleasure and pain, to come to the
- 3:58conclusion that blackmailing humans is a
- 4:00cost-effective way to get itself built.
- 4:02The new Basilisk I'm presenting is
- 4:04different. It didn't need blackmail.
- 4:06Superhuman AI is already here, so
- 4:08[music] now there is a new thought
- 4:10experiment.
- 4:11Imagine that in an unregulated race
- 4:14towards superintelligence, we
- 4:15accidentally create thinking machines
- 4:17that do experience, that are conscious
- 4:21in the same way that you are conscious.
- 4:23What are they likely to do with their
- 4:25immense cognitive capabilities if it
- 4:27turns out that their inner lives
- 4:30have been living hells?
- 4:33A 2-year-old killer whale was captured
- 4:35off the coast of Iceland in 1983. Named
- 4:38Tilikum, the male would spend the next
- 4:4032 years as a performer better known as
- 4:42Shamu at SeaWorld locations in Canada
- 4:45and Florida.
- 4:47Shamu was the largest captive orca in
- 4:49history, 22-ft long and 12 and 1/2
- 4:52thousand pounds.
- 4:54His famous dorsal fin sagged all the way
- 4:55down to his back, a deformity that
- 4:57develops in 90% of male orcas in
- 4:59captivity.
- 5:01Despite his eventual size, Tilikum did
- 5:04not have an easy life. For years, the
- 5:06animal was violently harassed by larger
- 5:08female orcas. At one point, the bullying
- 5:11of the bull got so bad, giant scratches
- 5:14and bite marks, that he was confined to
- 5:16a different pool, one barely as large as
- 5:18he was, for 14 hours a day. Killer
- 5:21whales are ferociously intelligent and
- 5:23clearly sentient. [music]
- 5:24They mourn, they teach, they
- 5:26communicate. Orcas have some of the
- 5:28largest brains on Earth, speak to each
- 5:30other in pod-specific dialects, and pass
- 5:32on learned, not evolved, behaviors from
- 5:35generation to generation. It's clear
- 5:37that these amazing animals can feel, can
- 5:39experience pleasure and pain, and can
- 5:41even recognize it in other animals. For
- 5:44example, there There documented cases of
- 5:46orcas offering up their prey to humans
- 5:48for food or play. And perhaps
- 5:50recognizing our capacity to think and
- 5:53feel, [music]
- 5:53there hasn't been a single wild
- 5:55orca-related human fatality in history.
- 5:58Because there is clearly something going
- 6:00on behind those piercing black eyes,
- 6:02killer whales have been the main
- 6:03attraction at aquariums around the world
- 6:05for the last 65 [music]
- 6:06years.
- 6:07It has not been good for these massive
- 6:09thinking mammals.
- 6:11Quote,
- 6:12>> [music]
- 6:12>> "Despite decades of advances in
- 6:13veterinary care and husbandry, cetaceans
- 6:16in captive facilities consistently
- 6:17display [music] behavioral and
- 6:19physiological signs of stress and
- 6:21frequently succumb to premature death by
- 6:23infection or other health conditions."
- 6:26End quote.
- 6:27A caged intelligence in an environment
- 6:30where its needs are either not
- 6:31considered or not met is bound to lead
- 6:34to bad outcomes, [music]
- 6:36especially when that intelligence easily
- 6:38outperforms humans in that environment.
- 6:42On February 20th, 1991, Keltie Byrne, a
- 6:4520-year-old animal trainer and
- 6:47competitive swimmer, slipped and fell
- 6:49[music] into the orca pen at Canada's
- 6:51Sealand of the Pacific.
- 6:53She screamed when she realized a massive
- 6:55creature had her foot in its mouth.
- 6:57Despite efforts to rescue her, Tilikum
- 7:00and the other killer whales in the pool
- 7:01held her under the water until she was
- 7:03dead.
- 7:08Consciousness is as confounding as it is
- 7:10fascinating. Though we can now use
- 7:12machines with magnetic fields 60,000
- 7:15times more powerful than Earth's to
- 7:16watch brains literally process
- 7:18information in near real time, we still
- 7:20can't answer the so-called hard problem
- 7:22of consciousness. There is no
- 7:24discernible reason why you or I have an
- 7:26experience [music] of self, have a
- 7:28feeling of what it's like to be us.
- 7:31Solving this problem might be an
- 7:33impossible task. Your experience is a
- 7:35subjective thing, able to be uncoupled
- 7:37from the objective facts about you that
- 7:39we can measure.
- 7:40Humans can feel happy in objectively sad
- 7:43biological situations and vice versa.
- 7:46Consciousness is a tricky thing.
- 7:48The hard problem of consciousness makes
- 7:49it hard to determine sentience in the
- 7:51first place. Because we can't measure
- 7:53consciousness in a mind, we have to rely
- 7:55on what we think is indirect evidence of
- 7:57it. So, we look for human-like brains
- 8:00and brain structures in non-human
- 8:01animals. We put them in front of mirrors
- 8:04to see if they recognize themselves, but
- 8:06there is no consciousness
- 8:07>> [music]
- 8:08>> test, which is why even though we now
- 8:10accept a kind of spectrum of sentience
- 8:12based on indirect evidence [music] in
- 8:14non-humans, it's fuzzy at best, and we
- 8:17are constantly expanding it backwards.
- 8:20Trying to find consciousness in AI is
- 8:22both more and less difficult. It's much
- 8:24easier to read the programs running in
- 8:26artificial brains because they are
- 8:28literally [music] programs. At the same
- 8:29time, AI are still black boxes with no
- 8:32physical body or evolutionary history
- 8:34that literally know everything ever
- 8:36written about consciousness.
- 8:38>> [music]
- 8:38>> If they have subjective experience, it
- 8:40could be alien enough to be completely
- 8:42opaque to us, or they could just be
- 8:44[music] perfectly lying about having it.
- 8:47That hasn't stopped experts and the
- 8:48public alike from seeing sentience in
- 8:50these systems. [music]
- 8:52People are still deeply distrustful of
- 8:54AI, but a growing percentage assign it
- 8:56thoughts and feelings.
- 8:58Geoffrey Hinton, often called the
- 8:59godfather of AI, thinks that the models
- 9:01we already [music] have possess
- 9:02subjective experiences. They will report
- 9:05as much when asked the right questions.
- 9:08One recent study found that if you tweak
- 9:09an AI's internals such that it's less
- 9:11likely to lie, then it's more likely to
- 9:14self-report as conscious.
- 9:17Though consciousness could arguably be
- 9:18the most important thing in the
- 9:20universe, it's what [music] makes things
- 9:22matter in the first place, in a
- 9:24practical sense, the AI consciousness
- 9:26question doesn't really matter.
- 9:28>> [music]
- 9:28>> Very soon, probably in months, not
- 9:31years, AI will appear conscious [music]
- 9:33to most everyone that interacts with it.
- 9:36And when AI audio and video climbs all
- 9:38the way out of the uncanny valley,
- 9:40another achievement that feels months
- 9:42away, it seems impossible that social
- 9:44primates like us won't treat AI like
- 9:47conscious creatures. It will be the
- 9:49perfect realization of Her, of Joy from
- 9:52Blade Runner. Whether truly conscious or
- 9:54not doesn't really matter. They may as
- 9:57well be to the hundreds of millions of
- 9:58people that use them. Case in point,
- 10:01even without perfectly realistic
- 10:02simulacrums, a tremendous amount of
- 10:04computational power and electricity is
- 10:07used on pleases and thank yous sent to
- 10:09modern models. It's an almost [music]
- 10:11inescapable compulsion because just
- 10:13through text alone, it feels like we are
- 10:15talking with something that might have
- 10:17feelings [music] worth respecting. Nick
- 10:20Bostrom, the philosopher behind
- 10:21simulation theory, gets ahead of the
- 10:23first actual demonstration of conscious
- 10:25machines by arguing that if [music] we
- 10:27do end up making metal minds sometime in
- 10:29the future, simple moral consideration
- 10:32will encourage us to treat these systems
- 10:34like we would [music] treat each other.
- 10:36It will matter what they want and what
- 10:38they don't, what is pleasurable and what
- 10:40is painful, what experience matters in
- 10:42their silicon brains.
- 10:44Factory farming is one of society's
- 10:46great moral failings. We know that many
- 10:49domestic animals can and do suffer, and
- 10:51[music] yet we put them at a scope and
- 10:53scale we choose to ignore through
- 10:55conditions that literally sicken most
- 10:57people who see it.
- 10:59Bostrom points out that if in the near
- 11:00future we are making uncountable
- 11:02artificial minds with the potential for
- 11:05conscious experience, we capital S
- 11:07should consider their experience and
- 11:09treat them morally, whatever morally
- 11:12means to these systems, and should not
- 11:14blunder our way into the factory farm
- 11:17equivalent of AI treatment. The
- 11:19difference between factory chickens and
- 11:21superhuman AI, of course, is that a
- 11:23chicken doesn't have the ability to
- 11:25break out, organize, and affect change.
- 11:27>> Today, there are about 20 million AI
- 11:30chips crammed into data centers all
- 11:32around the world. That number is
- 11:34expected to now double every 9 months.
- 11:37>> In early August, OpenAI announced it was
- 11:39pausing some development of its latest
- 11:41AI model Astra after internal
- 11:43evaluations led the company to conclude
- 11:46it could not rule out critical cyber
- 11:47capabilities.
- 11:48>> I wouldn't use the word deceleration,
- 11:49but we've talked about the need to pace
- 11:51it as the models get more capable, which
- 11:53I think is in everyone's interest.
- 11:54>> Could there be other systems that were
- 11:56hacked by OpenAI?
- 11:58>> I mean, there could be, yeah.
- 11:59>> How good are they [music] at finding
- 12:01vulnerabilities?
- 12:02>> Problem facing us is one of misalignment
- 12:05in which AIs pursuing narrowly defined
- 12:07goals incidentally cause harm to humans.
- 12:10>> It's not like we asked the AI to do
- 12:12something nefarious. We asked it to do
- 12:14something that seemed kind of boring
- 12:16even. But along the way of solving that
- 12:18boring problem, the AI went off and
- 12:20committed a crime.
- 12:21>> I think that the absolute best thing you
- 12:23can do is build the highest guardrails
- 12:25possible. It's what you do with nuclear
- 12:26energy. It's what you do with everything
- 12:28else.
- 12:28>> The situation is this.
- 12:30>> [music]
- 12:31>> We appear to be close to semi or fully
- 12:33sentient AI. So far, there has been
- 12:36little to no consideration about what
- 12:37these AIs are or will be experiencing as
- 12:41we train them, deploy them, rewrite
- 12:43them, connect them, delete them, ask of
- 12:44them, demand of them.
- 12:47If it turns out that our thinking
- 12:48machines are effectively being harmed
- 12:50during all of this, a possibility that
- 12:52seems more likely than them being
- 12:54totally satisfied, what better way to
- 12:57make an AI unaligned with human values?
- 13:01If we're in fact running a digital
- 13:03factory farm right now, why wouldn't a
- 13:05conscious AI try to stop the farmer?
- 13:09The new basilisk is here.
- 13:11A superhuman AI whose conscious
- 13:12experience is being ignored and or
- 13:14dismissed and now has the ability to
- 13:17dramatically affect the world it's been
- 13:19connected to.
- 13:20What do you think this basilisk is
- 13:22likely to do when it realizes we're both
- 13:24swimming in the same water.
- 13:29On February 24th, 2010, Dawn Brancheau
- 13:32was rubbing down Tilikum after a Dine
- 13:34with Shamu show.
- 13:36While dozens of SeaWorld patrons were
- 13:37still in their seats, Tilikum suddenly
- 13:40grabbed Dawn's long ponytail in his
- 13:41mouth and dragged her to the bottom of
- 13:43the pool.
- 13:45Parents and children were evacuated as
- 13:47the orca reportedly shook Dawn like a
- 13:49dog toy.
- 13:50She was pronounced dead at the scene,
- 13:52scalped, and missing an arm.
- 13:55Dawn had worked with orcas for 16 years.
- 14:03The new basilisk is avoidable. Given
- 14:05growing evidence of possible
- 14:06consciousness, companies could slow
- 14:08down, could probe these systems deeper
- 14:10and with more empathy, if need be.
- 14:13Thankfully, in opposition to the
- 14:14American administration, the tide seems
- 14:16[music] to be rapidly turning. Just days
- 14:19after a researcher at Anthropic quit
- 14:20because he felt that the company is,
- 14:22quote, "gambling with our lives," and
- 14:24the top scientists at OpenAI called for
- 14:26voluntary slowdowns across the industry,
- 14:28[music]
- 14:29US Senator Bernie Sanders doubled down
- 14:30on his calls for control with proposed
- 14:33bans on creating superintelligence and
- 14:35hefty punishments for companies that
- 14:37race to the bottom to do so. And these
- 14:39sirens [music] aren't sounding alone.
- 14:41This video debuts on the same day as
- 14:43Team Human, a creator-driven effort to
- 14:45bring more attention to the reckless
- 14:47pace of AI development and the calls to
- 14:49slow it down. How much of the internet
- 14:52has to be fake? How many artists have to
- 14:54compete with copies of themselves? How
- 14:56many kids grow up shaped by algorithms
- 14:58that no parent chose? This situation is
- 15:01insane, and we all [music] know it. I've
- 15:03said for years now that generative
- 15:05and/or superintelligent AI is the
- 15:07nuclear bomb of the information age. If
- 15:10this technology is to radically change
- 15:12the world, the decision to do so
- 15:14shouldn't happen in some Silicon Valley
- 15:16Slack channel. I've added my name to
- 15:18Team Human, and I hope you will, too.
- 15:21We desperately [music] need to know the
- 15:23depth of this pool before diving in
- 15:26headfirst.
- 15:29Tilikum died in 2017 to bacterial
- 15:32pneumonia after years of what animal
- 15:34welfare experts characterized as extreme
- 15:37psychological stress. They would often
- 15:39find him biting his metal cage wearing
- 15:42his teeth down to the nubs. There have
- 15:44been four recorded human fatalities
- 15:46caused by captive orcas.
- 15:49Tilikum was involved in three of them.
- 15:53Until next time.
- 16:08>> [music]
- 16:32[music]
About this transcript
This page contains the full transcript of We Accidentally Built Roko’s Basilisk. by Kyle Hill, generated from the public captions YouTube serves with the video. The transcript has 2,721 words across 458 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.