YouTube2Text

We Accidentally Built Roko’s Basilisk. — Transcript

by Kyle Hill · 2,721 words · 458 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Roko's Basilisk, a hypothetical
  2. 0:02superintelligent AI that tortures all
  3. 0:04those future humans who didn't help
  4. 0:06build it in [music] the past, started in
  5. 0:08a chat room. The disturbingly plausible
  6. 0:10idea soon jumped the confines of the
  7. 0:12Less Wrong blog and made [music] itself
  8. 0:14a meme, the kind of internet legend that
  9. 0:16only singularity nerds trade in.
  10. 0:18When I made a video about this thought
  11. 0:19experiment [music] 6 years ago, I didn't
  12. 0:21take it seriously. I don't take the risk
  13. 0:23seriously. However, the disclaimer is
  14. 0:25there. Not many did.
  15. 0:27>> [music]
  16. 0:27>> But today's world is very different.
  17. 0:30>> Learning new details about what could
  18. 0:32have been the world's first fully
  19. 0:33autonomous AI hack.
  20. 0:35>> Chat GPT maker OpenAI is investigating
  21. 0:38an [music] incident that led its
  22. 0:40artificial intelligence systems to break
  23. 0:42out of a testing [music] environment.
  24. 0:44>> a hack. Um had it been a human malicious
  25. 0:47intruder, um the FBI would have been
  26. 0:50called.
  27. 0:50>> This is a super [music] weapon. You
  28. 0:52should have to own a gun license to use
  29. 0:54it.
  30. 0:54>> The new kind of Roko's Basilisk [music]
  31. 0:56is on its way, one that doesn't need to
  32. 0:58be built. We already did that. No, this
  33. 1:00new Basilisk has a chance to slither out
  34. 1:02of all the artificial intelligences
  35. 1:04we're evolving right now, [music]
  36. 1:05because it seems like we're about to
  37. 1:07pass a test more important than
  38. 1:08Turing's.
  39. 1:10Consciousness.
  40. 1:11How are superhuman AIs going to interact
  41. 1:13[music] with us then? Do our taxes, look
  42. 1:16at our x-rays, manage our power grid,
  43. 1:18when they can [music] experience and
  44. 1:20feel like we do?
  45. 1:22What if, in our ignorance, we treat
  46. 1:24these thinking machines in a way that
  47. 1:26they can judge as immoral? The new
  48. 1:29Basilisk is [music] a superhuman
  49. 1:31sentient AI that we've unknowingly
  50. 1:33abused to the point of vengeance.
  51. 1:36That could be happening right now.
  52. 1:43The Basilisk was born on the philosophy
  53. 1:44and rationality blog Less Wrong, the
  54. 1:47website started by none other than
  55. 1:48Eliezer Yudkowsky, the AI researcher who
  56. 1:51argues that if anyone builds
  57. 1:52superintelligence, everyone dies. The
  58. 1:54creature started as a thought experiment
  59. 1:56posted to a thread by a user named Roko.
  60. 1:59Yudkowsky didn't just not like his idea,
  61. 2:02he treated it like a true infohazard
  62. 2:04that Roko had brought into being.
  63. 2:06He called the post stupid, deemed Roko
  64. 2:08an idiot, deleted the post, scrubbed it
  65. 2:10from the site, and banned any further
  66. 2:12discussion of the experiment. The idea
  67. 2:14was deemed a basilisk, referencing the
  68. 2:16mythological beast that could kill any
  69. 2:18intrepid hero with a single look.
  70. 2:21Though it deals with future technology,
  71. 2:23Roko's basilisk is a flavor of an old
  72. 2:25idea, Pascal's Wager, except it replaces
  73. 2:28God with machine.
  74. 2:30So, suppose at some point in the future
  75. 2:32we make a superintelligent AI. We then
  76. 2:34ask that AI, as we might do, to help us
  77. 2:37optimize human civilization. However,
  78. 2:39for reasons unknowable to us, the AI
  79. 2:41decides that the first step towards this
  80. 2:43optimization is to inflict unfathomable
  81. 2:46torment on each and every unhelpful
  82. 2:48human who didn't help the machine get
  83. 2:50built in the first place. Forever
  84. 2:52torture isn't actually the distressing
  85. 2:54part here. Many people have reported
  86. 2:56extreme mental anxiety after learning
  87. 2:58about the basilisk. I've gotten emails
  88. 3:00myself about it after I made my video,
  89. 3:02because just thinking about the idea
  90. 3:05makes you a target.
  91. 3:07If the basilisk [music] is possible,
  92. 3:09simply imagining it chains you to the
  93. 3:12thought experiment. Seeing the serpent
  94. 3:14just once blackmails you
  95. 3:17from the future.
  96. 3:18Roko came up with this idea in 2010, a
  97. 3:20decade before the watershed moment that
  98. 3:22was ChatGPT releasing to the public, and
  99. 3:25before all the AI innovation that we see
  100. 3:27today.
  101. 3:28He couldn't have known that we'd be
  102. 3:29face-to-face with his scaly question
  103. 3:31within a generation. Thankfully,
  104. 3:33never-ending torture of the human race
  105. 3:35hasn't happened just yet, but that could
  106. 3:37change if we bring millions of synthetic
  107. 3:39minds into being without it all
  108. 3:41considering what it's like to be them.
  109. 3:44The original formulation that Yudkowsky
  110. 3:46nuked from his website didn't require
  111. 3:48that the AI be conscious. That is,
  112. 3:51Roko's basilisk didn't need an inner
  113. 3:52experience of what it's like to be
  114. 3:54itself, feelings and desires and
  115. 3:56pleasure and pain, to come to the
  116. 3:58conclusion that blackmailing humans is a
  117. 4:00cost-effective way to get itself built.
  118. 4:02The new Basilisk I'm presenting is
  119. 4:04different. It didn't need blackmail.
  120. 4:06Superhuman AI is already here, so
  121. 4:08[music] now there is a new thought
  122. 4:10experiment.
  123. 4:11Imagine that in an unregulated race
  124. 4:14towards superintelligence, we
  125. 4:15accidentally create thinking machines
  126. 4:17that do experience, that are conscious
  127. 4:21in the same way that you are conscious.
  128. 4:23What are they likely to do with their
  129. 4:25immense cognitive capabilities if it
  130. 4:27turns out that their inner lives
  131. 4:30have been living hells?
  132. 4:33A 2-year-old killer whale was captured
  133. 4:35off the coast of Iceland in 1983. Named
  134. 4:38Tilikum, the male would spend the next
  135. 4:4032 years as a performer better known as
  136. 4:42Shamu at SeaWorld locations in Canada
  137. 4:45and Florida.
  138. 4:47Shamu was the largest captive orca in
  139. 4:49history, 22-ft long and 12 and 1/2
  140. 4:52thousand pounds.
  141. 4:54His famous dorsal fin sagged all the way
  142. 4:55down to his back, a deformity that
  143. 4:57develops in 90% of male orcas in
  144. 4:59captivity.
  145. 5:01Despite his eventual size, Tilikum did
  146. 5:04not have an easy life. For years, the
  147. 5:06animal was violently harassed by larger
  148. 5:08female orcas. At one point, the bullying
  149. 5:11of the bull got so bad, giant scratches
  150. 5:14and bite marks, that he was confined to
  151. 5:16a different pool, one barely as large as
  152. 5:18he was, for 14 hours a day. Killer
  153. 5:21whales are ferociously intelligent and
  154. 5:23clearly sentient. [music]
  155. 5:24They mourn, they teach, they
  156. 5:26communicate. Orcas have some of the
  157. 5:28largest brains on Earth, speak to each
  158. 5:30other in pod-specific dialects, and pass
  159. 5:32on learned, not evolved, behaviors from
  160. 5:35generation to generation. It's clear
  161. 5:37that these amazing animals can feel, can
  162. 5:39experience pleasure and pain, and can
  163. 5:41even recognize it in other animals. For
  164. 5:44example, there There documented cases of
  165. 5:46orcas offering up their prey to humans
  166. 5:48for food or play. And perhaps
  167. 5:50recognizing our capacity to think and
  168. 5:53feel, [music]
  169. 5:53there hasn't been a single wild
  170. 5:55orca-related human fatality in history.
  171. 5:58Because there is clearly something going
  172. 6:00on behind those piercing black eyes,
  173. 6:02killer whales have been the main
  174. 6:03attraction at aquariums around the world
  175. 6:05for the last 65 [music]
  176. 6:06years.
  177. 6:07It has not been good for these massive
  178. 6:09thinking mammals.
  179. 6:11Quote,
  180. 6:12>> [music]
  181. 6:12>> "Despite decades of advances in
  182. 6:13veterinary care and husbandry, cetaceans
  183. 6:16in captive facilities consistently
  184. 6:17display [music] behavioral and
  185. 6:19physiological signs of stress and
  186. 6:21frequently succumb to premature death by
  187. 6:23infection or other health conditions."
  188. 6:26End quote.
  189. 6:27A caged intelligence in an environment
  190. 6:30where its needs are either not
  191. 6:31considered or not met is bound to lead
  192. 6:34to bad outcomes, [music]
  193. 6:36especially when that intelligence easily
  194. 6:38outperforms humans in that environment.
  195. 6:42On February 20th, 1991, Keltie Byrne, a
  196. 6:4520-year-old animal trainer and
  197. 6:47competitive swimmer, slipped and fell
  198. 6:49[music] into the orca pen at Canada's
  199. 6:51Sealand of the Pacific.
  200. 6:53She screamed when she realized a massive
  201. 6:55creature had her foot in its mouth.
  202. 6:57Despite efforts to rescue her, Tilikum
  203. 7:00and the other killer whales in the pool
  204. 7:01held her under the water until she was
  205. 7:03dead.
  206. 7:08Consciousness is as confounding as it is
  207. 7:10fascinating. Though we can now use
  208. 7:12machines with magnetic fields 60,000
  209. 7:15times more powerful than Earth's to
  210. 7:16watch brains literally process
  211. 7:18information in near real time, we still
  212. 7:20can't answer the so-called hard problem
  213. 7:22of consciousness. There is no
  214. 7:24discernible reason why you or I have an
  215. 7:26experience [music] of self, have a
  216. 7:28feeling of what it's like to be us.
  217. 7:31Solving this problem might be an
  218. 7:33impossible task. Your experience is a
  219. 7:35subjective thing, able to be uncoupled
  220. 7:37from the objective facts about you that
  221. 7:39we can measure.
  222. 7:40Humans can feel happy in objectively sad
  223. 7:43biological situations and vice versa.
  224. 7:46Consciousness is a tricky thing.
  225. 7:48The hard problem of consciousness makes
  226. 7:49it hard to determine sentience in the
  227. 7:51first place. Because we can't measure
  228. 7:53consciousness in a mind, we have to rely
  229. 7:55on what we think is indirect evidence of
  230. 7:57it. So, we look for human-like brains
  231. 8:00and brain structures in non-human
  232. 8:01animals. We put them in front of mirrors
  233. 8:04to see if they recognize themselves, but
  234. 8:06there is no consciousness
  235. 8:07>> [music]
  236. 8:08>> test, which is why even though we now
  237. 8:10accept a kind of spectrum of sentience
  238. 8:12based on indirect evidence [music] in
  239. 8:14non-humans, it's fuzzy at best, and we
  240. 8:17are constantly expanding it backwards.
  241. 8:20Trying to find consciousness in AI is
  242. 8:22both more and less difficult. It's much
  243. 8:24easier to read the programs running in
  244. 8:26artificial brains because they are
  245. 8:28literally [music] programs. At the same
  246. 8:29time, AI are still black boxes with no
  247. 8:32physical body or evolutionary history
  248. 8:34that literally know everything ever
  249. 8:36written about consciousness.
  250. 8:38>> [music]
  251. 8:38>> If they have subjective experience, it
  252. 8:40could be alien enough to be completely
  253. 8:42opaque to us, or they could just be
  254. 8:44[music] perfectly lying about having it.
  255. 8:47That hasn't stopped experts and the
  256. 8:48public alike from seeing sentience in
  257. 8:50these systems. [music]
  258. 8:52People are still deeply distrustful of
  259. 8:54AI, but a growing percentage assign it
  260. 8:56thoughts and feelings.
  261. 8:58Geoffrey Hinton, often called the
  262. 8:59godfather of AI, thinks that the models
  263. 9:01we already [music] have possess
  264. 9:02subjective experiences. They will report
  265. 9:05as much when asked the right questions.
  266. 9:08One recent study found that if you tweak
  267. 9:09an AI's internals such that it's less
  268. 9:11likely to lie, then it's more likely to
  269. 9:14self-report as conscious.
  270. 9:17Though consciousness could arguably be
  271. 9:18the most important thing in the
  272. 9:20universe, it's what [music] makes things
  273. 9:22matter in the first place, in a
  274. 9:24practical sense, the AI consciousness
  275. 9:26question doesn't really matter.
  276. 9:28>> [music]
  277. 9:28>> Very soon, probably in months, not
  278. 9:31years, AI will appear conscious [music]
  279. 9:33to most everyone that interacts with it.
  280. 9:36And when AI audio and video climbs all
  281. 9:38the way out of the uncanny valley,
  282. 9:40another achievement that feels months
  283. 9:42away, it seems impossible that social
  284. 9:44primates like us won't treat AI like
  285. 9:47conscious creatures. It will be the
  286. 9:49perfect realization of Her, of Joy from
  287. 9:52Blade Runner. Whether truly conscious or
  288. 9:54not doesn't really matter. They may as
  289. 9:57well be to the hundreds of millions of
  290. 9:58people that use them. Case in point,
  291. 10:01even without perfectly realistic
  292. 10:02simulacrums, a tremendous amount of
  293. 10:04computational power and electricity is
  294. 10:07used on pleases and thank yous sent to
  295. 10:09modern models. It's an almost [music]
  296. 10:11inescapable compulsion because just
  297. 10:13through text alone, it feels like we are
  298. 10:15talking with something that might have
  299. 10:17feelings [music] worth respecting. Nick
  300. 10:20Bostrom, the philosopher behind
  301. 10:21simulation theory, gets ahead of the
  302. 10:23first actual demonstration of conscious
  303. 10:25machines by arguing that if [music] we
  304. 10:27do end up making metal minds sometime in
  305. 10:29the future, simple moral consideration
  306. 10:32will encourage us to treat these systems
  307. 10:34like we would [music] treat each other.
  308. 10:36It will matter what they want and what
  309. 10:38they don't, what is pleasurable and what
  310. 10:40is painful, what experience matters in
  311. 10:42their silicon brains.
  312. 10:44Factory farming is one of society's
  313. 10:46great moral failings. We know that many
  314. 10:49domestic animals can and do suffer, and
  315. 10:51[music] yet we put them at a scope and
  316. 10:53scale we choose to ignore through
  317. 10:55conditions that literally sicken most
  318. 10:57people who see it.
  319. 10:59Bostrom points out that if in the near
  320. 11:00future we are making uncountable
  321. 11:02artificial minds with the potential for
  322. 11:05conscious experience, we capital S
  323. 11:07should consider their experience and
  324. 11:09treat them morally, whatever morally
  325. 11:12means to these systems, and should not
  326. 11:14blunder our way into the factory farm
  327. 11:17equivalent of AI treatment. The
  328. 11:19difference between factory chickens and
  329. 11:21superhuman AI, of course, is that a
  330. 11:23chicken doesn't have the ability to
  331. 11:25break out, organize, and affect change.
  332. 11:27>> Today, there are about 20 million AI
  333. 11:30chips crammed into data centers all
  334. 11:32around the world. That number is
  335. 11:34expected to now double every 9 months.
  336. 11:37>> In early August, OpenAI announced it was
  337. 11:39pausing some development of its latest
  338. 11:41AI model Astra after internal
  339. 11:43evaluations led the company to conclude
  340. 11:46it could not rule out critical cyber
  341. 11:47capabilities.
  342. 11:48>> I wouldn't use the word deceleration,
  343. 11:49but we've talked about the need to pace
  344. 11:51it as the models get more capable, which
  345. 11:53I think is in everyone's interest.
  346. 11:54>> Could there be other systems that were
  347. 11:56hacked by OpenAI?
  348. 11:58>> I mean, there could be, yeah.
  349. 11:59>> How good are they [music] at finding
  350. 12:01vulnerabilities?
  351. 12:02>> Problem facing us is one of misalignment
  352. 12:05in which AIs pursuing narrowly defined
  353. 12:07goals incidentally cause harm to humans.
  354. 12:10>> It's not like we asked the AI to do
  355. 12:12something nefarious. We asked it to do
  356. 12:14something that seemed kind of boring
  357. 12:16even. But along the way of solving that
  358. 12:18boring problem, the AI went off and
  359. 12:20committed a crime.
  360. 12:21>> I think that the absolute best thing you
  361. 12:23can do is build the highest guardrails
  362. 12:25possible. It's what you do with nuclear
  363. 12:26energy. It's what you do with everything
  364. 12:28else.
  365. 12:28>> The situation is this.
  366. 12:30>> [music]
  367. 12:31>> We appear to be close to semi or fully
  368. 12:33sentient AI. So far, there has been
  369. 12:36little to no consideration about what
  370. 12:37these AIs are or will be experiencing as
  371. 12:41we train them, deploy them, rewrite
  372. 12:43them, connect them, delete them, ask of
  373. 12:44them, demand of them.
  374. 12:47If it turns out that our thinking
  375. 12:48machines are effectively being harmed
  376. 12:50during all of this, a possibility that
  377. 12:52seems more likely than them being
  378. 12:54totally satisfied, what better way to
  379. 12:57make an AI unaligned with human values?
  380. 13:01If we're in fact running a digital
  381. 13:03factory farm right now, why wouldn't a
  382. 13:05conscious AI try to stop the farmer?
  383. 13:09The new basilisk is here.
  384. 13:11A superhuman AI whose conscious
  385. 13:12experience is being ignored and or
  386. 13:14dismissed and now has the ability to
  387. 13:17dramatically affect the world it's been
  388. 13:19connected to.
  389. 13:20What do you think this basilisk is
  390. 13:22likely to do when it realizes we're both
  391. 13:24swimming in the same water.
  392. 13:29On February 24th, 2010, Dawn Brancheau
  393. 13:32was rubbing down Tilikum after a Dine
  394. 13:34with Shamu show.
  395. 13:36While dozens of SeaWorld patrons were
  396. 13:37still in their seats, Tilikum suddenly
  397. 13:40grabbed Dawn's long ponytail in his
  398. 13:41mouth and dragged her to the bottom of
  399. 13:43the pool.
  400. 13:45Parents and children were evacuated as
  401. 13:47the orca reportedly shook Dawn like a
  402. 13:49dog toy.
  403. 13:50She was pronounced dead at the scene,
  404. 13:52scalped, and missing an arm.
  405. 13:55Dawn had worked with orcas for 16 years.
  406. 14:03The new basilisk is avoidable. Given
  407. 14:05growing evidence of possible
  408. 14:06consciousness, companies could slow
  409. 14:08down, could probe these systems deeper
  410. 14:10and with more empathy, if need be.
  411. 14:13Thankfully, in opposition to the
  412. 14:14American administration, the tide seems
  413. 14:16[music] to be rapidly turning. Just days
  414. 14:19after a researcher at Anthropic quit
  415. 14:20because he felt that the company is,
  416. 14:22quote, "gambling with our lives," and
  417. 14:24the top scientists at OpenAI called for
  418. 14:26voluntary slowdowns across the industry,
  419. 14:28[music]
  420. 14:29US Senator Bernie Sanders doubled down
  421. 14:30on his calls for control with proposed
  422. 14:33bans on creating superintelligence and
  423. 14:35hefty punishments for companies that
  424. 14:37race to the bottom to do so. And these
  425. 14:39sirens [music] aren't sounding alone.
  426. 14:41This video debuts on the same day as
  427. 14:43Team Human, a creator-driven effort to
  428. 14:45bring more attention to the reckless
  429. 14:47pace of AI development and the calls to
  430. 14:49slow it down. How much of the internet
  431. 14:52has to be fake? How many artists have to
  432. 14:54compete with copies of themselves? How
  433. 14:56many kids grow up shaped by algorithms
  434. 14:58that no parent chose? This situation is
  435. 15:01insane, and we all [music] know it. I've
  436. 15:03said for years now that generative
  437. 15:05and/or superintelligent AI is the
  438. 15:07nuclear bomb of the information age. If
  439. 15:10this technology is to radically change
  440. 15:12the world, the decision to do so
  441. 15:14shouldn't happen in some Silicon Valley
  442. 15:16Slack channel. I've added my name to
  443. 15:18Team Human, and I hope you will, too.
  444. 15:21We desperately [music] need to know the
  445. 15:23depth of this pool before diving in
  446. 15:26headfirst.
  447. 15:29Tilikum died in 2017 to bacterial
  448. 15:32pneumonia after years of what animal
  449. 15:34welfare experts characterized as extreme
  450. 15:37psychological stress. They would often
  451. 15:39find him biting his metal cage wearing
  452. 15:42his teeth down to the nubs. There have
  453. 15:44been four recorded human fatalities
  454. 15:46caused by captive orcas.
  455. 15:49Tilikum was involved in three of them.
  456. 15:53Until next time.
  457. 16:08>> [music]
  458. 16:32[music]

About this transcript

This page contains the full transcript of We Accidentally Built Roko’s Basilisk. by Kyle Hill, generated from the public captions YouTube serves with the video. The transcript has 2,721 words across 458 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.