YouTube2Text

EMERGENCY PODCAST: ASI Will Kill Us All! — Transcript

by Tom Bilyeu · 17,637 words · 2,553 segments · language en · Watch on YouTube

Full transcript

  1. 0:00You believe that if we pursue super
  2. 0:03intelligence, the risk that that
  3. 0:05destroys humanity is so high that we
  4. 0:09should stop that pursuit immediately?
  5. 0:12>> Yes.
  6. 0:12>> Okay. Do you believe that the correct
  7. 0:15path is a non-development agreement
  8. 0:18between the US and China?
  9. 0:20>> Yes. Specifically when it comes to super
  10. 0:22intelligence, I do not think this
  11. 0:23applies to all forms of AI. One things
  12. 0:25that lobbyists love to do is they love
  13. 0:26to try to equivocate all AI as the same
  14. 0:28technology. This is obviously nonsense.
  15. 0:30Here's a fun fact. Did you know that you
  16. 0:32could buy uranium ore on Amazon.com?
  17. 0:35>> I did not know that. Terrifying. You
  18. 0:36could ship it to your house, put on your
  19. 0:38desk. It's perfectly harmless.
  20. 0:39>> Natural uranium or is very harmless.
  21. 0:41It's perfectly safe to touch.
  22. 0:43>> Okay.
  23. 0:43>> But obviously, highly weaponsenriched
  24. 0:46uranium is extremely illegal.
  25. 0:48>> Right.
  26. 0:48>> This is also how I think about AI.
  27. 0:50>> Okay.
  28. 0:51>> Do you already know where you would draw
  29. 0:52the line? I have some suggestions but
  30. 0:54fundamentally this in a sense we can
  31. 0:57take a step back or maybe that's one of
  32. 0:58the questions still to come is where why
  33. 1:00do I think these risks exist how what do
  34. 1:02I think their shape is and a lot of the
  35. 1:04risks I see where they come from is
  36. 1:06fundamentally that we don't understand
  37. 1:08AI
  38. 1:09>> AI is very different from normal
  39. 1:11software I'm a computer scientist by
  40. 1:13background I've written a lot of
  41. 1:14software in my life you know even
  42. 1:16without agentic help and most software
  43. 1:19is written with code line by line you
  44. 1:21tell the computer exactly what to do.
  45. 1:23This is not how AI works. AI is more
  46. 1:27like grown rather than written through a
  47. 1:29technique that's called neural networks.
  48. 1:31You have these massive piles of data and
  49. 1:34you kind of have a program self assemble
  50. 1:36itself, grow, learn from this data. And
  51. 1:39what comes out the other side isn't
  52. 1:41lines of code that you could read or
  53. 1:43understand what's going on. It's more
  54. 1:44like billions and billions and billions
  55. 1:47of numbers. If you multiply and add all
  56. 1:49those numbers in the right order, you
  57. 1:51get JBT. How do you begin though to
  58. 1:53parse whether something falls into the
  59. 1:55yes bucket or the no bucket? It isn't
  60. 1:58just that it uses a neural net
  61. 1:59presumably.
  62. 2:00>> Yes.
  63. 2:01>> Okay. So if it's not neural net, is it
  64. 2:04size of the data center?
  65. 2:06>> Fundamentally I think it's a behavior
  66. 2:08classification in the sense is there's
  67. 2:10obviously a category of things that are
  68. 2:13can out compete humanity. The way that's
  69. 2:15what I like to think think about it. I
  70. 2:17like to think about is that like if you
  71. 2:18build a system of any kind, it can be a
  72. 2:20single agent, it can be a swarm, it can
  73. 2:21be, you know, can have many shapes. If
  74. 2:23you build a system that can out compete
  75. 2:25humanity at all relevant tasks, every
  76. 2:27time you start a business, you go out of
  77. 2:28business because an AI business out
  78. 2:30competes you. Every time you trade on
  79. 2:31the stock market, you lose all your
  80. 2:32money to an AI hedge fund. Every time
  81. 2:34you run a political campaign, you lose
  82. 2:36to the other guy who's using an, you
  83. 2:38know, AI adviser. Every time you run a
  84. 2:40military campaign, you lose to the PE
  85. 2:42person who's using more autonomous AI
  86. 2:43systems. If you get into a world like
  87. 2:45this, power of all kinds, economic,
  88. 2:48military, political, etc., is
  89. 2:50concentrated in the hands of non-human
  90. 2:52entities in AI. And if we don't control
  91. 2:55these systems, if we don't understand
  92. 2:57these systems, and if they don't have
  93. 2:59our best interests at heart, it's very
  94. 3:01hard to imagine that going well.
  95. 3:02>> Okay, it seems like we're already there,
  96. 3:04but the AI is a tool. So the way that I
  97. 3:08think about AI right now is if somebody
  98. 3:10has AI and they compete against somebody
  99. 3:12in the same thing without AI, the person
  100. 3:14with AI will win effectively 100% of the
  101. 3:16time. Do you disagree with that?
  102. 3:17>> I agree with that. What I disagree with
  103. 3:19is I think if a human against an AI
  104. 3:21without a human, the human still wins.
  105. 3:22>> Totally agree.
  106. 3:23>> Yes. What I'm saying is what happens
  107. 3:25when a human versus an AI alone an AI an
  108. 3:27autonomous AI and the autonomous AI wins
  109. 3:29every time.
  110. 3:30>> So it's the autonomous part that's
  111. 3:32important for your definition.
  112. 3:33>> Yes. I think autonomy is very important
  113. 3:35here. When I say autonomy, I mean, you
  114. 3:37know, both in the obvious sensor, but
  115. 3:39also like obviously if you just have a
  116. 3:40human rubber stamp, yes, on everything,
  117. 3:42that doesn't count as oversight. Fair.
  118. 3:44So, that would still count as autonomous
  119. 3:46to me. Even if there's technically a
  120. 3:47human involved, but the human doesn't
  121. 3:49even understand what the system is doing
  122. 3:50and they're just rubber stamping all the
  123. 3:52decisions.
  124. 3:53>> Like what I expect happen for example is
  125. 3:55that they're probably will to be human
  126. 3:57CEOs for quite a while, but they just
  127. 3:59rubber stamp what the AI CEOs tells them
  128. 4:01to do. They're just there for legal
  129. 4:02representation. So for you, we would
  130. 4:04have then long past where we we're now
  131. 4:08beyond the point of no return and and
  132. 4:09all is for not.
  133. 4:11>> Yes. I like thinking about the point of
  134. 4:12no return much more than like think
  135. 4:14about like the point of like extinction
  136. 4:15or doom in a sense.
  137. 4:17>> Why?
  138. 4:17>> Because that's the point that actually
  139. 4:19matters.
  140. 4:20>> Okay. So it's like stakes versus where
  141. 4:21we are on the timeline.
  142. 4:23>> Yeah. In a sense the what matters to me
  143. 4:25is the point of no return. Once we've
  144. 4:26hit the point of no return where there's
  145. 4:28so much AI, it's so powerful. It's so
  146. 4:30intelligent. It's self-improving. You
  147. 4:32know, there's not just one AI. There are
  148. 4:33millions or billions of them.
  149. 4:35>> Even if we're not dead, even if we
  150. 4:38continue to maybe live for a couple
  151. 4:39years or even decades, we no longer have
  152. 4:42any control over the future and there's
  153. 4:44no way for us to get it back. Okay,
  154. 4:46you've already slipped in a base
  155. 4:48assumption there.
  156. 4:49>> Um, so I before we get to that base
  157. 4:51assumption, because it is certainly on
  158. 4:53the list of where this sort of breaks
  159. 4:55bad, a very clear definition. So,
  160. 4:57autonomous AI that can outperform humans
  161. 4:58at everything. That's the one that we're
  162. 5:00worried about. Uh, we need to understand
  163. 5:02the mile marker, which is what's the
  164. 5:04point of no return. You've talked about
  165. 5:05this before, so I think we can hit this
  166. 5:07one quick, which is we're close. You
  167. 5:09don't quite know how close. We're so
  168. 5:11close, it's dangerous, we should stop
  169. 5:13now.
  170. 5:13>> Yes.
  171. 5:13>> Okay. If I gave you a big red button and
  172. 5:16you could unilaterally stop everybody
  173. 5:18from developing any further. We can use
  174. 5:19what we already have, but we cannot
  175. 5:21progress any further, would you hit it
  176. 5:22today? Would you wait a week, a month, a
  177. 5:24year? I like to say if we as a species
  178. 5:26or as a civilization were three steps
  179. 5:28wiser than we were today, we wouldn't
  180. 5:30have done Chachi PT in the first place.
  181. 5:32>> What are the steps? What do you mean
  182. 5:33steps?
  183. 5:34>> It's just a it's just a
  184. 5:36>> three clicks, three just a little bit
  185. 5:38smarter.
  186. 5:39>> Three generations of wisdom. And
  187. 5:41[clears throat] if we if we were two
  188. 5:42steps wiser than we currently are, we
  189. 5:44would have done chatgpt saw that it did
  190. 5:46all these crazy things that none of our
  191. 5:48scientists predicted and it got, you
  192. 5:49know, 100 million users over time and
  193. 5:51like whoa whoa whoa whoa whoa. What the
  194. 5:52hell's going on here? and would have
  195. 5:54rolled back tried to figure it out and
  196. 5:56if we were one step wiser we would stop
  197. 5:57today.
  198. 5:58>> The original question about US China is
  199. 6:00trying to get to is I my belief is that
  200. 6:03you see this as a state level thing and
  201. 6:06that law enforcement is going to be the
  202. 6:08punchline. Um meaning that just like
  203. 6:10with nuclear if we found that somebody
  204. 6:13were trying to build a nuclear weapon we
  205. 6:15would immediately stop them. That's
  206. 6:16against the law. We've got things around
  207. 6:18that already. We need to do that for AI.
  208. 6:21>> Yes.
  209. 6:21>> Okay. This this is going to be a big
  210. 6:23part of where I look at this and I go I
  211. 6:25think that there's a problem there.
  212. 6:27Meaning you're never because it is we
  213. 6:30agree that this is a state level issue
  214. 6:32between the US and China specifically.
  215. 6:34Now is your chance to say there's
  216. 6:36somebody else.
  217. 6:37>> I want to say the EU but I can't.
  218. 6:39>> Okay. So
  219. 6:40>> with my heart.
  220. 6:41>> Yeah. Yeah.
  221. 6:42>> That was a joke. I I I've lived in the
  222. 6:44Europe for many years. I love the EU.
  223. 6:45>> They're amazing. My wife is European. I
  224. 6:47totally get it. Um but we really have
  225. 6:50this is a US versus China thing. Okay.
  226. 6:52So you put forward a trust but verify
  227. 6:54situation.
  228. 6:55>> Um much like nuclear or something that
  229. 6:58has a very different approach to the
  230. 7:00trust but verify.
  231. 7:01>> I think there's a lot of inspiration to
  232. 7:03be had from the nuclear example. I think
  233. 7:05the history of nuclear regulation is the
  234. 7:07closest to there is a great crazy new
  235. 7:10technology that will change everything
  236. 7:12and we rose to the occasion. Now, I'm
  237. 7:14not saying we fixed everything, but you
  238. 7:16know, I when I think about regulating AI
  239. 7:18and like the history of AI, I like to
  240. 7:20think back to Leo Zard, the person who
  241. 7:22came up with the nuclear bomb.
  242. 7:23>> When he first did the math, he realized,
  243. 7:26oh it's possible.
  244. 7:28He did the math and he was like, "Oh my
  245. 7:30god, it's definitely possible." So, what
  246. 7:31he did is he moved to San Francisco,
  247. 7:33started a nuclear bomb startup, which
  248. 7:34split into three other startups to sell
  249. 7:36as many nuclear bombs to the public as
  250. 7:38possible. No, of course that's not what
  251. 7:39happened. He went to the government. He
  252. 7:41went to the military. He was like,
  253. 7:43"Guys, this will change everything. We
  254. 7:46have to do something. We have to figure
  255. 7:47this out." And this is what didn't
  256. 7:49happen for AI. What didn't happen for AI
  257. 7:51is that we didn't realize that this will
  258. 7:53change our civilization. This is not a
  259. 7:55private matter. This is a national
  260. 7:56security matter. This is a matter of the
  261. 7:58the organization of our economic
  262. 8:00structures. This is a political
  263. 8:02question. This is a government. This is
  264. 8:03an intergovernmental question. And we
  265. 8:06just didn't do that. And by mu we I mean
  266. 8:09the scientific community like the
  267. 8:11scientific and the technical community
  268. 8:12like the people who started companies
  269. 8:14like you know deep mind or anthropic or
  270. 8:16openai they weren't Leo Zillard they
  271. 8:19went to go make a bunch of money off of
  272. 8:20it they didn't go to the government and
  273. 8:22were like we need to build new
  274. 8:23institutions to responsibly steward this
  275. 8:25powerful technology and so we didn't do
  276. 8:28it you know at the time when nuclear was
  277. 8:30first considered the idea that you would
  278. 8:33have like stuff like you know the
  279. 8:34international atomic energy agency and
  280. 8:36so on was unthinkable such such a thing
  281. 8:38had never existed in the history of any
  282. 8:40legal code before. They had to invent
  283. 8:42whole new forms of law to deal with this
  284. 8:45new technology. And that makes sense. If
  285. 8:48you have an incredibly powerful new
  286. 8:49technology, you have to steward it
  287. 8:51responsibly. And that will require
  288. 8:53coming up with new doctrines with new
  289. 8:55ways of like what level of risk do we
  290. 8:56tolerate in various ways? Where do the
  291. 8:58risks come from? Who decides the level
  292. 9:01of risk that we take with these relevant
  293. 9:03technologies? And at the moment, we're
  294. 9:05just not making these choices at all.
  295. 9:07It's not even that we're making the
  296. 9:08wrong choice. It's that
  297. 9:10>> we're not making these choices at all. I
  298. 9:12would love if there was like
  299. 9:13disagreement, for example, between POTUS
  300. 9:15and Xiinping about what the correct
  301. 9:16level of extinction risk is. You know,
  302. 9:19if they're arguing, oh, it should be
  303. 9:20this percent or that percent. I think
  304. 9:21this would be a really good world to
  305. 9:23live in because then we could have this
  306. 9:24argument. But we're very far even from
  307. 9:26that point that we're not even making
  308. 9:28these decisions yet. Be right back to
  309. 9:30the show in just a second. But right
  310. 9:32now, let's talk about a problem the
  311. 9:34military couldn't solve with willpower.
  312. 9:36Picture a soldier making life ordeath
  313. 9:38decisions after 36 hours without sleep.
  314. 9:41This is high stress. There's obviously
  315. 9:43no margin for error. The military needed
  316. 9:46their people thinking clearly, even
  317. 9:48under conditions that would break most
  318. 9:49of us. So, they funded a $6 [music]
  319. 9:51million Department of Defense research
  320. 9:53contract to find the answer. What they
  321. 9:56found was ketones, the brain's most
  322. 9:58efficient fuel source. That research
  323. 10:01became ketone IQ. Ketones [music] cross
  324. 10:03the bloodb brain barrier and fuel your
  325. 10:05neurons directly. They enhance
  326. 10:07mitochondrial function for longer mental
  327. 10:10stamina. I take it for interviews,
  328. 10:12before writing, before I do coaching,
  329. 10:14and I can feel the difference. [music]
  330. 10:16Go to ketone.com/impact
  331. 10:20for 30% off your [music] subscription
  332. 10:22order or visit your local Target to get
  333. 10:24your first shot for free. That's
  334. 10:27ketone.com/impact.
  335. 10:30[music] Now, let's get back to the show.
  336. 10:32Okay, if Elon can be believed, he said
  337. 10:36for years that he tried to lobby the
  338. 10:38government to let them know this is
  339. 10:40going to be a real problem. This is he
  340. 10:41referred to it as a demon summoning
  341. 10:43circle and we're all just hoping that
  342. 10:44the demon that we summon is going to do
  343. 10:46our bidding. And he said he just could
  344. 10:48not get anybody to take him seriously.
  345. 10:50And so he became fatalistic. That's when
  346. 10:52he tries to start open AI because he's
  347. 10:54like, "Well, if it's going to exist, I
  348. 10:56really don't want uh Larry Page at
  349. 10:59Google to be the only one making it
  350. 11:01because he's whatever a specist or that
  351. 11:03was the accusation to Elon." And so Elon
  352. 11:08has again, if he can be believed,
  353. 11:10attempted to do a maybe an early
  354. 11:13prototype of the very systematic way
  355. 11:15that you're going about it now. So, do
  356. 11:18you think it was overwhelm and confusion
  357. 11:21that led people to do that? And because
  358. 11:25I have a different take on the um
  359. 11:27nuclear analogy, which is where I'm
  360. 11:29getting to. Do you think Elon wasn't
  361. 11:31good at it? People were overwhelmed. But
  362. 11:35why didn't that take hold?
  363. 11:37>> I think an average political institution
  364. 11:40building campaign takes 20 years. I
  365. 11:42don't think Elon spent 20 years on this.
  366. 11:44I don't know. Maybe he did and I'm not
  367. 11:45aware of Why when the guy that
  368. 11:47understood the nuclear thing, why did he
  369. 11:49get immediate attention when he went to
  370. 11:50the
  371. 11:50>> Oh, no. It took decades.
  372. 11:52>> Really?
  373. 11:52>> Yes.
  374. 11:53>> So, he was banging and banging and
  375. 11:54banging.
  376. 11:54>> He was banging. So, he was banging for
  377. 11:56years. Then there was the Einstein
  378. 11:57letter which led to the Manhattan
  379. 11:58project, but until the IAEA was created
  380. 12:01and all these other things that took
  381. 12:03decades of work by international
  382. 12:05diplomats, you know, by lawyers and by,
  383. 12:07you know, statesmen, you know, between
  384. 12:09this was when the Soviet Union was still
  385. 12:11around, right? Imagine how hard it was
  386. 12:13to get the Soviet Union to agree to
  387. 12:15something like this. This was a decadesl
  388. 12:17long project that also kept evolving,
  389. 12:19you know, throughout the 60s, the 70s,
  390. 12:21the 80s. And so, and this was not a
  391. 12:22thing that was like, oh, Zlord, you
  392. 12:24know, had one Zoom call and then the
  393. 12:25thing was solved. Quite quite the
  394. 12:27opposite. It required thousands, tens of
  395. 12:29thousands of scientists, diplomats,
  396. 12:32politicians, etc. working for decades.
  397. 12:34So, you know, I I applaud Elon for
  398. 12:37having tried, but fundamentally, a thing
  399. 12:39that I think is very frustrating to tech
  400. 12:41people is that non- tech things run at a
  401. 12:44different pace is that if you want to
  402. 12:46convince, say, a thousand diplomats to
  403. 12:48take an issue seriously. There's kind of
  404. 12:50like a hard limit on how fast you can do
  405. 12:52that. I think it's faster than people
  406. 12:53think it is, but it's still longer than
  407. 12:55like a couple years. It takes time and
  408. 12:57it takes effort and it takes a lot of
  409. 12:58money. I'm not aware of Elon spending a
  410. 13:01billion dollars on political advertising
  411. 13:03or on AIX risk. He could have done this.
  412. 13:05He obviously has the money. He can just
  413. 13:06spend a billion dollars on political
  414. 13:09advertising to uh make this issue known.
  415. 13:11He could fund 50 think tanks to work on
  416. 13:14existential risk of AI. He has the money
  417. 13:16to do it. So in this regard, I don't
  418. 13:18know. You know, Elon, you know, I I
  419. 13:20think he's an extremely precient and
  420. 13:21extremely competent person. I think if
  421. 13:23he really wanted to, he could. I think
  422. 13:26he absolutely could. he could fund us.
  423. 13:28You know, he we're getting a lot done
  424. 13:30with the money we're doing and if he
  425. 13:32gave us 10x more money, we would get 10x
  426. 13:33more done. So, I don't know why he made
  427. 13:36the choices he made. We can
  428. 13:37psychoanalyze all day. But
  429. 13:39fundamentally, one of the things that we
  430. 13:41learned when we started doing this is
  431. 13:43that every single person in the world
  432. 13:44told us this is impossible. No one will
  433. 13:46listen. And we've tried already and
  434. 13:48we're like we'll see. And then we just
  435. 13:50talked to politicians and overwhelmingly
  436. 13:53the response we got was oh wow. I had no
  437. 13:56idea. No one told me. Thank you for
  438. 13:57telling me. This seems really bad. I
  439. 14:00don't think people have tried. Uh
  440. 14:02because politics is slow. It's
  441. 14:04frustrating. You have to repeat
  442. 14:05something. You know, in advertising, you
  443. 14:07usually say to get a message across, you
  444. 14:09have to repeat it seven to 10 times per
  445. 14:12person on over this, you know, a
  446. 14:14schedule like 6 to 12 months.
  447. 14:16>> One conversation will never be enough.
  448. 14:17You will have to talk to that person
  449. 14:18seven to 10 times until they can start
  450. 14:20to really understand these. And this is
  451. 14:22frustrating. This is annoying. It takes
  452. 14:24a lot of time. So, no, I think there's a
  453. 14:26lot that can be done.
  454. 14:27>> All right. So, as a marketer, one of the
  455. 14:29reasons that I think that this stalled
  456. 14:31out, one of the reasons that I think
  457. 14:32nuclear ended up with, if we tell
  458. 14:34ourselves a wrong story about what
  459. 14:36happened there, it will seem like, oh,
  460. 14:38this was just another case of we
  461. 14:39lobbied, lobbyed, lobbyed, lobby, he was
  462. 14:40ignored in the beginning, but then we
  463. 14:41finally got through to people. Is that
  464. 14:44um when you're marketing, when you're
  465. 14:45doing sales, you're trying to elicit an
  466. 14:46emotional response. If you can't get the
  467. 14:48emotional response, if you can't create
  468. 14:50a sense of urgency that people feel, not
  469. 14:51understand, but feel, they're never
  470. 14:53going to move. And so when I look at the
  471. 14:56nuclear question, to me that was a
  472. 14:58failed version and we just sort of got
  473. 15:01off lightly that nobody after um what we
  474. 15:05did to Japan that everybody saw the
  475. 15:07level of destruction and just said whoa
  476. 15:09we got to chill on this and not actually
  477. 15:11move forward. But to me that was by
  478. 15:14seeing the devastating power of it by
  479. 15:17the scientists themselves suddenly
  480. 15:19asking themselves what have I done and
  481. 15:20sort of waking up with that on their
  482. 15:22conscience. Then people were like,
  483. 15:24"Okay, yo, we got to chill on this." But
  484. 15:26the reality was, if I'm not mistaken,
  485. 15:28and please tell me if you know something
  486. 15:30I don't, but they when they were doing
  487. 15:32the first test, they knew there was a
  488. 15:34nonzero chance that they would light the
  489. 15:36entire atmosphere on fire and kill
  490. 15:38everyone. Yep.
  491. 15:38>> And they still did it.
  492. 15:40>> Yes. But they did calculate the risks
  493. 15:42and it was like a billionth of a
  494. 15:43percent.
  495. 15:44>> Okay. So, if you had been in that room
  496. 15:47and you run it and you see it's a
  497. 15:49billionth of a percent and I know you
  498. 15:50don't like to be unilateral, but what
  499. 15:52way would you cast your vote? Would you
  500. 15:53say don't do it or would you be like,
  501. 15:56"Ah, that's a tiny fraction. Go for it."
  502. 15:58>> Generally, my my uh vibe is my my rule
  503. 16:01for my personal life is that something
  504. 16:02is less risky than driving a car, it's
  505. 16:04fine.
  506. 16:05>> Okay. So that one feels less risky even
  507. 16:07though the consequences are so
  508. 16:10catastrophic that as long as it's just
  509. 16:13an odds question.
  510. 16:14>> There's a thing where um obviously I'm
  511. 16:17not a physicist. So I'm not sure how
  512. 16:19much I would have believed these numbers
  513. 16:20if I was a physicist. My understanding
  514. 16:22>> likely to believe high or low or just
  515. 16:24>> my understanding from physicists I've
  516. 16:25talked to is that the calculations they
  517. 16:27do were like very solid. like it was
  518. 16:29like really a weird edge case to imagine
  519. 16:32that something could have gone wrong
  520. 16:33here because they actually really did
  521. 16:35understand the physics they were working
  522. 16:36with. This was about the fusing of
  523. 16:37nitrogen in the atmosphere and this was
  524. 16:40a type of science that they were very
  525. 16:42confident in. Now, if you ask me the
  526. 16:44same question about a science I don't
  527. 16:45understand. For example, if you put me
  528. 16:46there and I'm not a physicists, I
  529. 16:48probably would have voted against it
  530. 16:49>> because I don't know, maybe the model is
  531. 16:51wrong. You know, even if the model
  532. 16:52predicts a certain thing, maybe the
  533. 16:54model is wrong. So, it's also about
  534. 16:56uncertainty. I I it's not just about
  535. 16:59what is the number that comes out but
  536. 17:00also how certain are that this number is
  537. 17:02the thing you care about. So I do think
  538. 17:05that you know I may or may have said no
  539. 17:08I may have said yes depending on how
  540. 17:09confident again I'm not a physicist
  541. 17:10unfortunately I just don't know how good
  542. 17:11the model actually was. Like if you like
  543. 17:13if someone tells me oh there's a you
  544. 17:16know quintilionth chance that if you you
  545. 17:19know turn on a light bulb it will create
  546. 17:20a black hole that will consume the
  547. 17:22world. I'm like
  548. 17:23>> whatever dude you know because quantum
  549. 17:25physics by the way does predict this
  550. 17:26right? like every time you do anything,
  551. 17:28something could quantum tunnel into a
  552. 17:30black hole or whatever, right? But it's
  553. 17:32like who cares? So, but what I find
  554. 17:36really interesting about this example is
  555. 17:38is that they did run the math. They did
  556. 17:39have a very concrete model. They had a
  557. 17:41very concrete reason that they wrote
  558. 17:43down for like why they think this is an
  559. 17:45acceptable thing to do. Meanwhile, we
  560. 17:46have, you know, our current, you know,
  561. 17:48crop of tech people saying on podcast
  562. 17:50that they think there's maybe, I don't
  563. 17:51know, a 20% chance that it'll kill
  564. 17:53everybody. One of the base assumptions I
  565. 17:56think you have is what you were just
  566. 17:57articulating which is about it's the
  567. 17:59unknown element of AI. You said if I
  568. 18:02wasn't a physicist and I didn't
  569. 18:04understand it then I might vote no just
  570. 18:05because the unknown is a big question
  571. 18:07for me. I feel that way about Daario's
  572. 18:10or anybody that says oh we've got a
  573. 18:12rough 20% chance. I don't think they're
  574. 18:13running math. I think they reach into
  575. 18:15the future emotionally and say I feel
  576. 18:18like something bad could happen. Yeah.
  577. 18:20And because it's
  578. 18:23>> vibes, we're not able to do the math
  579. 18:26equation. Yep.
  580. 18:27>> And so that is sort of possible breaking
  581. 18:31point number one of trying to use the
  582. 18:34nuclear as a example of will it work or
  583. 18:37won't it work? What I'll point to there
  584. 18:39is okay, if we're not able to get any
  585. 18:41kind of percentage, nobody's able to run
  586. 18:43the math and we can move forward with a
  587. 18:44degree of certainty, we either do what
  588. 18:46you're saying and just shut it down now.
  589. 18:48But given that um because it's the US
  590. 18:51versus China, I don't think that will
  591. 18:52ever work. And so I want to see if you
  592. 18:55can dismantle this for me. The way that
  593. 18:57I look at this is that and by the way,
  594. 18:59just so everybody knows what my bias is.
  595. 19:02Um I would be perfectly happy with you
  596. 19:04unilaterally hitting the button today,
  597. 19:06but I wouldn't have been happy if you'd
  598. 19:07hit it yesterday. That's a little
  599. 19:09cheeky, but I'm just trying to say like
  600. 19:11I love what I have now. It's so useful,
  601. 19:14but because I can't picture my future,
  602. 19:17it scares me and I would be much more
  603. 19:19comfortable just saying, "Hey, it's
  604. 19:21already helping so much. You know, we
  605. 19:24got this far. Woo, we got lucky. Just
  606. 19:25hit it." So, this is not me saying, "I
  607. 19:27just more, more, more." Uh, I would be
  608. 19:30very fine with that button. In fact, if
  609. 19:31I had the button, I would hit it because
  610. 19:33uh while I agree no one should act
  611. 19:36unilaterally, I might be a little more
  612. 19:38foolish. The reason that it begins to
  613. 19:39break down for me as a nuclear question
  614. 19:41is we couldn't stop ourselves even
  615. 19:43though we knew that we were unleashing
  616. 19:45something that could bring Armageddon if
  617. 19:46it proliferated. And of course it did.
  618. 19:49So we've already created one technology
  619. 19:51could not stop ourselves from doing it
  620. 19:53before we had the weapons pointed at
  621. 19:55each other. So when I look at this I say
  622. 19:58okay I think you probably could you can
  623. 20:01I think the US is on a path to rejecting
  624. 20:04AI. So that feels doable. Okay, you can
  625. 20:08reject you can get the US to reject AI
  626. 20:10publicly. If you can get the US to
  627. 20:12reject it publicly, you might be able to
  628. 20:14get China to reject it publicly. What I
  629. 20:16don't believe is that either of them can
  630. 20:18afford to reject it privately. And that
  631. 20:21just as you have said AI just gets
  632. 20:23better at lying when you catch it doing
  633. 20:26things, it has an incentive structure
  634. 20:28that it follows. And if lying and going
  635. 20:31underground makes you more likely to get
  636. 20:33it done, then you lie and go
  637. 20:34underground. Game theory tells us
  638. 20:36they'll both say, "Yeah, yeah, yeah.
  639. 20:38We're we're not going to proliferate
  640. 20:40because they want the other side to stop
  641. 20:41and then they're both going to go to
  642. 20:42their own corners because they can't
  643. 20:44trust that they actually will stop
  644. 20:46knowing that it doesn't matter which of
  645. 20:48them pushes forward. How do we actually
  646. 20:50not just the the agreement, but how do
  647. 20:52we actually get them to stop?"
  648. 20:55>> So, there's a bunch of points here.
  649. 20:57There are many points. So, let's let's
  650. 20:58try to disangle them a little bit.
  651. 21:00There's the game theory question there
  652. 21:02and like the the geopolitical question.
  653. 21:06There is the um there's the question of
  654. 21:10like epistemics and linguistics of
  655. 21:12probability like the question of like
  656. 21:13what does it mean to when someone says
  657. 21:15there's 20% probability? What does it
  658. 21:17mean when a physicist says there's a
  659. 21:19billionth of a percent probability it
  660. 21:21the atmosphere? What do those sentences
  661. 21:22mean and how do you compare them? And
  662. 21:24there's a there's a more general
  663. 21:26question of like how do you react to
  664. 21:27situations like this and how does it sep
  665. 21:29how is it different from because I think
  666. 21:30it's very different from nuclear like I
  667. 21:31think you didn't even mention one of the
  668. 21:32reasons I think is the most different
  669. 21:33from nuclear is that nuclear weapons
  670. 21:35can't shoot themselves. M
  671. 21:37>> you can have a nuclear weapon sit in a
  672. 21:39hanger somewhere it's perfectly safe you
  673. 21:40can hit it with a hammer you know
  674. 21:41nothing's going to go wrong um you can't
  675. 21:43contain a super intelligence so a super
  676. 21:45intelligence sitting in a hanger
  677. 21:46somewhere is not a safe thing so that's
  678. 21:48for example something where I think it's
  679. 21:49very different where developing a
  680. 21:51nuclear weapon is actually relatively
  681. 21:53safe not perfectly as you say you know
  682. 21:55fusion of nitrogen and whatnot but it is
  683. 21:57like pretty safe it's more the
  684. 21:59deployment that is very dangerous in AI
  685. 22:01it's very different super intelligent is
  686. 22:02the development that is dangerous not
  687. 22:04just the deployment so that's one
  688. 22:05disanaly for example. But just to circle
  689. 22:08back a little bit to talk about the game
  690. 22:10theory, there's a very very important
  691. 22:12thing to understand here, which is that
  692. 22:14super intelligence is not a tool. It's
  693. 22:16not a weapon. It's an adversary.
  694. 22:19If the US builds super intelligence, it
  695. 22:21will not achieve American objectives.
  696. 22:24The United States will cease to exist.
  697. 22:26So will China. If China builds super
  698. 22:28intelligence, China will cease to exist.
  699. 22:31It will not achieve its objectives and
  700. 22:32so will all other countries. This is the
  701. 22:35core disanalogy. The core disanalogy is
  702. 22:38is that it's actually not mutually
  703. 22:40assured destruction. It's actually not a
  704. 22:42prisoner's dilemma. It's actually
  705. 22:44independently assured destruction. If
  706. 22:46you race super intelligence, you don't
  707. 22:48win. You just lose game. Theoretically,
  708. 22:51the only stable equilibria actually is
  709. 22:54cooperate cooperate is unlike a prison
  710. 22:57of dilemma where defect defect is the
  711. 23:00stable equilibrium. Here the only stable
  712. 23:03where both China and the US get to live
  713. 23:05is if neither build it. That's the only
  714. 23:07one where everyone survives or anyone
  715. 23:09survives for that matter. So this is I
  716. 23:11think one of the core game theoretic
  717. 23:12disalies here is that this is all
  718. 23:15assuming the assumptions that you know
  719. 23:16you can't control something that's
  720. 23:18smarter than you. We currently are not
  721. 23:19even on track to build AI controls and
  722. 23:22they can out compete us. If we accept
  723. 23:23accept those intuitions, if we accept
  724. 23:25those uh definitions, then this follows
  725. 23:28naturally. Does that make sense?
  726. 23:30aggressively very clear. Now we have a
  727. 23:34problem and maybe I'm being unkind to
  728. 23:35America but America is better understood
  729. 23:38as being run by uh politicians,
  730. 23:41technocrats, finance, but China is
  731. 23:44largely run by people with an
  732. 23:47engineering bent. Even if they aren't
  733. 23:48engineers themselves, they understand
  734. 23:50building infrastructure. They've come
  735. 23:52from essentially nothing to a global
  736. 23:55superpower by building, by creating, by
  737. 23:58um translating schematics into or ideas
  738. 24:01into schematics into actual tangible
  739. 24:03things and they can move electrons and
  740. 24:05all of that. So even if I'm unkind to
  741. 24:08America and say, okay, America's not
  742. 24:10going to figure this one out, but China,
  743. 24:12you would expect there's only one person
  744. 24:14you have to convince, and if you
  745. 24:15convince him, then everything goes
  746. 24:17downstream from that. Why have we not
  747. 24:20been able to convince Xihinping?
  748. 24:22>> I think no one has tried. I don't think
  749. 24:25anyone has gone to Mr. Xiinping and
  750. 24:28given him a really good presentation on
  751. 24:30super intelligence. I would be very
  752. 24:32shocked if there was a single person
  753. 24:33that has done this.
  754. 24:35>> And what are the odds given that it will
  755. 24:37likely need to come from within China?
  756. 24:40Um, what are the odds that you think
  757. 24:42that happens?
  758. 24:44>> I think it depends a lot on what we do
  759. 24:45in the West. So, I'm American obviously.
  760. 24:48I have no influence on what happens in
  761. 24:49China. I do have a lot of influence what
  762. 24:51happens in my democratic country. And I
  763. 24:53do think that if we in America make it a
  764. 24:56priority not just to stop you know super
  765. 24:59intelligence you know from say domestic
  766. 25:01companies at home but anywhere on earth
  767. 25:04which is the only reasonable policy. The
  768. 25:06thing that we at control AI pose is that
  769. 25:10the only reasonable national
  770. 25:12international security position is is
  771. 25:13that we cannot tolerate the creation of
  772. 25:15super intelligence by anyone anywhere.
  773. 25:18When you know people, politicians,
  774. 25:20military officials ask me okay but what
  775. 25:22if China builds super intelligence? My
  776. 25:23answer is we cannot allow that. We
  777. 25:25cannot allow that. That's not
  778. 25:26acceptable. If China builds super
  779. 25:28intelligence it will kill American
  780. 25:30citizens. This all of us it will destroy
  781. 25:32the American everything the whole
  782. 25:34country. This is not something we can
  783. 25:36tolerate. And similarly that the Chinese
  784. 25:38cannot tolerate the Americans or their
  785. 25:40own companies building super
  786. 25:41intelligence because again it is not a
  787. 25:43weapon. It is not a tool. It is an
  788. 25:45adversary. This is the important thing.
  789. 25:48When I talk to different people about
  790. 25:49these risks in these super intelligence,
  791. 25:51I kind of think that it's like kind of
  792. 25:52like it's kind of like a political
  793. 25:53compass. There's like two axis and like
  794. 25:56one axis is believes technology is
  795. 25:58always good and believes technology can
  796. 26:01be good or bad. It can be both. And the
  797. 26:03other axis is uh thinks super
  798. 26:06intelligence is possible doesn't think
  799. 26:07super intelligence is possible. And most
  800. 26:10of the disagreements I have with people
  801. 26:12is along the axis just they don't
  802. 26:14haven't heard of super intelligence.
  803. 26:16They've just not thought about it. No
  804. 26:17one told them about it. They haven't
  805. 26:19considered the implications of it. Like
  806. 26:20when people talk about racing with
  807. 26:22China, most of the people who say we
  808. 26:24should race with China just don't know
  809. 26:26what super intelligence is. If we didn't
  810. 26:28have super intelligence, like if you
  811. 26:29promise me, you know, God came from
  812. 26:31heaven and he said there's a 100% chance
  813. 26:33that super intelligence will never be
  814. 26:34created, then I think we should probably
  815. 26:36race, then we probably should compete
  816. 26:38with China on AI. If I can guarantee
  817. 26:40there is no chance of super intelligence
  818. 26:42and it's only going to be useful
  819. 26:43economic and military applications that
  820. 26:45are, you know, controlled by humans
  821. 26:47throughout, yeah, we should probably do
  822. 26:48it. So most of the time when I disagree
  823. 26:50with these kinds of people that are
  824. 26:52pushing AI so aggressively, it's just
  825. 26:53because they literally don't believe
  826. 26:54it's or don't think it's possible. One
  827. 26:56of the most funniest things I found
  828. 26:57every time in San Francisco is when you
  829. 26:59ever talk to like accelerationists or
  830. 27:00like protect people. They don't believe
  831. 27:03in like super intelligence. They just
  832. 27:05like don't think in the future. They
  833. 27:07think AI will always be like an app on
  834. 27:08your phone. But this is not what we're
  835. 27:10talking about. We're talking about
  836. 27:11autonomous agents, intelligent agents
  837. 27:14acting in the real world against human
  838. 27:16interests in ways that are more
  839. 27:17competent than humans can. You know,
  840. 27:20competing with it won't again, it's not
  841. 27:21just one super intelligence. It's a
  842. 27:23whole ecosystem evolving, mutating,
  843. 27:25competing with each other, fighting. You
  844. 27:27know, there'll be wars between these AI
  845. 27:29systems where we are just collateral
  846. 27:31damage. This is a very very different
  847. 27:33thing from you have a cool app on your
  848. 27:34phone. And this is again why I say like
  849. 27:36there's just difference between you know
  850. 27:38uranium ore and highlyenriched uranium.
  851. 27:40I think this is really at the core
  852. 27:42thing. I think that if Xiinping
  853. 27:44understood, if someone helped him
  854. 27:45understand, I think he's a smart guy. I
  855. 27:47mean, I haven't met him, but he seems
  856. 27:48like a smart guy. I think if he
  857. 27:50understood what super intelligence is,
  858. 27:52if someone explained the game theory to
  859. 27:54him, I believe he would understand it
  860. 27:56and I believe we could take actions
  861. 27:57based on this. I do think this. Okay.
  862. 27:59It's very optimistic and Lord knows I
  863. 28:02hope you're right. Um, you said
  864. 28:04something though that is given the
  865. 28:06context we're in right now with Iran,
  866. 28:08the what if China builds a super
  867. 28:11intelligence, we just can't let them.
  868. 28:13That's how we saw, at least how Trump
  869. 28:15and the Trump administration saw Iran
  870. 28:17and nuclear weapons. And so at some
  871. 28:19point, at least in their judgment, it
  872. 28:21had to become kinetic because they
  873. 28:23refuse to acknowledge that they weren't
  874. 28:25going to pursue nuclear ambitions. Um,
  875. 28:28so what do you do if let's say China
  876. 28:31[snorts] does say we're we're not going
  877. 28:33to do it, but you have intel that says
  878. 28:34they are doing it. Um, do you get the
  879. 28:38world together and say we got to invade?
  880. 28:40edge lords love to say, you know,
  881. 28:41international law is fake. And this is
  882. 28:44mostly true. The only law that exists is
  883. 28:46backed by credible deterrence. It's the
  884. 28:48same thing with law in our country. The
  885. 28:50only the only reason that law exists is
  886. 28:52because if you break it, the police will
  887. 28:53arrest you. And if you resist that
  888. 28:54arrest, they will get violent. You know,
  889. 28:56they will force you into prison whether
  890. 28:58you like it or not. Every system of law
  891. 29:01is backed by force to some degree. The
  892. 29:03the beautiful thing about law is that if
  893. 29:06you design your laws right and you build
  894. 29:08your system right, you never need to use
  895. 29:10it. This is the beauty of deterrence.
  896. 29:12Deterrence is one of the most beautiful
  897. 29:14things in the world in a sense. I of I
  898. 29:17sometimes like to think if you have a
  899. 29:19lot of weapons, you'll never need to use
  900. 29:21them. If you have a little bit of
  901. 29:22weapons, you're definitely going to need
  902. 29:24to use them. So, in a sense, this is how
  903. 29:26peace has worked in our world for a long
  904. 29:28time. It was always clear that, you
  905. 29:30know, if someone pulls some if
  906. 29:32someone like goes against international
  907. 29:33order, if someone does a huge human
  908. 29:34rights violation, if someone, you know,
  909. 29:37nukes, you know, I don't know, like
  910. 29:38Latvia or something, that the Americans
  911. 29:40will intervene to make sure that doesn't
  912. 29:42happen again and that like every, you
  913. 29:43know, people's heads were rolled. The
  914. 29:44same thing is mutually sure destruction.
  915. 29:46The reason the Soviets didn't nuke
  916. 29:47anybody, I don't think it's because the
  917. 29:48Soviets were nice people. They were not.
  918. 29:50I think it's because they knew that if
  919. 29:51they did that, the Americans were going
  920. 29:53to nuke them. So the way to create peace
  921. 29:56is through deterrence which is the
  922. 29:58correct way to do it. So any kind of
  923. 30:00agreement has to be backed by credible
  924. 30:02deterrence on everyone's side for
  925. 30:04everyone's benefit. If you have credible
  926. 30:06deterrence you know then you would think
  927. 30:08everyone will be rational enough to not
  928. 30:10invoke that deterrence. Now when we're
  929. 30:13talking about like say terrorists who
  930. 30:15might not who might be crazy they might
  931. 30:17not react to incentives. Well sometimes
  932. 30:19you have to get ugly. That's an
  933. 30:21unfortunate truth about geopolitics. Um
  934. 30:24I hope it doesn't come to that. I think
  935. 30:26you know you can do a lot including with
  936. 30:29non-kinetic means of deterrence such as
  937. 30:31economic sanctions um diplomatic
  938. 30:33pressure etc. But fundamentally if a
  939. 30:36super intelligence is built anywhere on
  940. 30:37earth this threatens the lives of you,
  941. 30:39me, our families, our children and
  942. 30:41everyone else. And we should reserve the
  943. 30:42right to self-defense.
  944. 30:43>> We'll get right back to the show in a
  945. 30:45second but first I want to talk to you
  946. 30:47about how often you test your security.
  947. 30:50Most companies do it once a year, then a
  948. 30:53whole year goes by, but you've had new
  949. 30:54hires, new tools, new permissions.
  950. 30:56People are submitting new code every
  951. 30:58week. That test may have made you secure
  952. 31:00for one day, but if nobody checks the
  953. 31:03other 364, you're in trouble. Horizon 3
  954. 31:06closes that gap. Node Zero is an
  955. 31:08autonomous AI hacker that continuously
  956. 31:11attacks [music]
  957. 31:12your environment at machine speed, the
  958. 31:15way a real attacker would. Your entire
  959. 31:17attack surface external to cloud to
  960. 31:20identity in one [music] single test.
  961. 31:22225,000
  962. 31:25production safe tests run with zero
  963. 31:27downtime. See exactly what an attacker
  964. 31:30would find in your environment before
  965. 31:32they do. Go to horizon3.ai/impact
  966. 31:36theory and request your free node zero
  967. 31:39demo. Visit horizon3.ai/impact
  968. 31:43theory. No commitment required. results
  969. 31:45[music] in hours, not weeks. A word for
  970. 31:48anyone who travels for work. When I'm on
  971. 31:50the road, traveling for a shoot or
  972. 31:52meetings, whatever. I'm still running
  973. 31:54the company from wherever I land. So,
  974. 31:56logging into my accounts from a hotel
  975. 31:58room is a must. I'm still going to be
  976. 31:59answering messages at the airport. But
  977. 32:01those networks are wide open. And as
  978. 32:03someone who's been hacked, I can tell
  979. 32:05you this is terrible. Anyone sitting on
  980. 32:07that same Wi-Fi can see what you're
  981. 32:09doing. And when you run a company, it's
  982. 32:11not just your data on the line. It's
  983. 32:13your teams, your customers, everyone who
  984. 32:16trusts you. This is what Surf Shark is
  985. 32:18[music] built for. Surf Shark encrypts
  986. 32:21your connection. The second you're on
  987. 32:23public Wi-Fi, your activity is locked
  988. 32:25down and far harder to track. Your
  989. 32:27login, your accounts, your company's
  990. 32:29data, it's all protected. [music] Go to
  991. 32:31surfark.com/tombb
  992. 32:34or use code tombb for four extra months
  993. 32:38of Surf Shark. Go to
  994. 32:39surfshark.com/tombbi
  995. 32:41and use code tomb for four extra months.
  996. 32:45Now, let's get back to the show. Right
  997. 32:47now, we have the US and China
  998. 32:50specifically going head-to-head. So, you
  999. 32:52have two great powers. Neither want to
  1000. 32:55acknowledge the other as being in a
  1001. 32:57leadership role. Do you think that it
  1002. 32:59works with the US and China if Xi is
  1003. 33:02adversarial? Because you've already been
  1004. 33:04very clear. You think that he can be
  1005. 33:05convinced. So, that scenario takes care
  1006. 33:07of itself. But if Xi is adversarial to
  1007. 33:11the West and wants to push it, wants to
  1008. 33:14maybe he even says, "I don't want to go
  1009. 33:16all the way to super intelligence, but
  1010. 33:17you know, he's getting way too close to
  1011. 33:19crossing the point of no return." And he
  1012. 33:22keeps pushing it in that do you think
  1013. 33:24the US has the ability to deter in a
  1014. 33:27non-kinetic fashion? Deterrence is based
  1015. 33:30on rationality. If he is completely
  1016. 33:32irrational and so is the entire military
  1017. 33:34chain of China and its entire industrial
  1018. 33:36apparatus and its entire political
  1019. 33:37epilas and everyone is completely
  1020. 33:39insane, well then deterrence doesn't
  1021. 33:41work because they're insane.
  1022. 33:42>> But they would only have to be a little
  1023. 33:44bit off on your compass if they believe
  1024. 33:47that technology is good. Then to them
  1025. 33:51they're not insane and they're like,
  1026. 33:53"Whoa, these guys have abdicated. We've
  1027. 33:56got an opportunity. tech is only good.
  1028. 33:58So, we're gonna keep pushing in that
  1029. 34:00direction to assert our rightful place
  1030. 34:03in the world. We're gonna make things
  1031. 34:05better. We're going to be less combative
  1032. 34:06than the US ever was. So, to me, there's
  1033. 34:09a very easy scenario to paint where
  1034. 34:11China sees itself as providing almost
  1035. 34:14liberation.
  1036. 34:16>> We're going to be doing a great thing
  1037. 34:17for everybody. This is wonderful. We
  1038. 34:19know the right answer. And so, they
  1039. 34:21certainly won't perceive themselves as
  1040. 34:23crazy.
  1041. 34:23>> I think this is very possible. I think
  1042. 34:25it's very possible that Stalin a cruise
  1043. 34:26chef could have just, you know, nuked
  1044. 34:28Latvia. I think there are timelines in
  1045. 34:29which that happened, you know.
  1046. 34:31>> Well, so here's where we get back to,
  1047. 34:32you were right, I think to point out
  1048. 34:34that the the analogy actually doesn't
  1049. 34:36really hold between nuclear and today
  1050. 34:38because if the super intelligence gets
  1051. 34:41made, then it becomes its own thing. It
  1052. 34:42becomes adversarial. We'll talk more
  1053. 34:44about that later. But for now, if we
  1054. 34:46adopt that frame of reference, you've
  1055. 34:49got China knowing that, okay, if I can
  1056. 34:53get AGI and America doesn't get it now,
  1057. 34:57I'm in a much better position. I'm
  1058. 34:59acting from a place of my own interest,
  1059. 35:01but the world, if they are on the other
  1060. 35:03part of the spectrum and go, we can't
  1061. 35:06allow this. You have a situation where
  1062. 35:09um I I'll put forth my belief, which is
  1063. 35:12the US can't hardly deter Iran. So even
  1064. 35:15if in the end we finally uh choked them
  1065. 35:17out from an economic standpoint, we'd
  1066. 35:19never be able to do it to China. So now
  1067. 35:22we're in a position where I mean maybe
  1068. 35:24if the whole world got together, but
  1069. 35:26that feels like a tall order. Um so it
  1070. 35:30feels to me like deterrence gets removed
  1071. 35:32off the table and we're left only with
  1072. 35:35the US and G both have to agree much
  1073. 35:38like in war games that the conclusion is
  1074. 35:40this is a funny game and the only way to
  1075. 35:41win is to not play. I think so to a
  1076. 35:43large degree this is the case. Yes, you
  1077. 35:45can in fact construct many scenarios in
  1078. 35:47which we lose. My P doom is high. I
  1079. 35:49don't think we're going to make it. I
  1080. 35:51think in almost every timeline that
  1081. 35:53starts from us sitting here today, we
  1082. 35:54don't make it.
  1083. 35:56>> There are very few timelines in which we
  1084. 35:57make it. But there are some, and I'm
  1085. 35:59describing those that are still
  1086. 36:01available to us where we do make it.
  1087. 36:03These are the kind of timelines that do
  1088. 36:05make it. And I'm trying to explain why I
  1089. 36:08think they're not like insane, you know,
  1090. 36:09crazy edge timelines. They are rare, but
  1091. 36:12they're real.
  1092. 36:13>> Sure.
  1093. 36:13>> Like, yes, it is very possible that just
  1094. 36:16for me, the default timeline is no one
  1095. 36:18thinks about this at all. Everyone just,
  1096. 36:20you know, keeps twiddling their thumbs
  1097. 36:22for the next, you know, two to five
  1098. 36:23years and then super intelligence gets
  1099. 36:24created and it's ww wamp, [laughter] you
  1100. 36:26know, like that. That's my default
  1101. 36:28prediction.
  1102. 36:30>> That's good. Wamp wamp. I like that. Uh,
  1103. 36:33okay, that makes sense. One of the
  1104. 36:35things that keeps coming up is what I
  1105. 36:37think is um the thing I listed first is
  1106. 36:39your like absolute foundational belief
  1107. 36:41which is that AI uh sorry super
  1108. 36:44intelligence is by nature adversarial. I
  1109. 36:48want to understand why by definition it
  1110. 36:50has to be adversarial.
  1111. 36:52>> I don't think it's by definition. I
  1112. 36:54think the way we are currently building
  1113. 36:56them super intelligence that are not
  1114. 36:58adversarial are just like not possible.
  1115. 37:00Building something that is
  1116. 37:01non-adversarial and is that powerful is
  1117. 37:04actually really hard. Like extremely
  1118. 37:06hard. This is basically equivalent to
  1119. 37:08saying we're going to build a one world
  1120. 37:10government that controls the entire
  1121. 37:11economy that then controls the entire
  1122. 37:13military that can build super powerful
  1123. 37:15technology that we can't even
  1124. 37:16understand. That will control your
  1125. 37:18entire private life and all those of all
  1126. 37:20other people and it will be good all the
  1127. 37:22time and we will not make any mistakes
  1128. 37:23and it will be no bugs and we'll do it
  1129. 37:25all with software on the first try.
  1130. 37:28That's what it would mean to build a
  1131. 37:29non-adversarial or like a good super
  1132. 37:31intelligence. That's what it would mean.
  1133. 37:32And look, if you said we're going to
  1134. 37:35spend three generations of all of our
  1135. 37:37greatest scientists, philosophers,
  1136. 37:39mathematicians working day and night on
  1137. 37:41this problem,
  1138. 37:43maybe, right? Like maybe. But that's not
  1139. 37:47what's happening here. Like what's
  1140. 37:48currently happening here is a clown
  1141. 37:50show. Like this is what's going on here.
  1142. 37:53Like it's not like we have we're
  1143. 37:54throwing all of humanity's glorious
  1144. 37:57heroic effort into just trying to figure
  1145. 38:00out how can we build a good super
  1146. 38:01intelligence and you know all of our
  1147. 38:03children grow up to want to work on this
  1148. 38:05problem and you know all the wise elders
  1149. 38:07are overseeing the project and so no
  1150. 38:09it's a bunch of kids in San Francisco
  1151. 38:11with zero adult supervision you know and
  1152. 38:14run by sociopathic companies building
  1153. 38:16whatever the they want and
  1154. 38:17deploying it wherever the they
  1155. 38:18want. We're not even trying. This is why
  1156. 38:20I think we fail. I don't think it's an
  1157. 38:22onlogical thing. I think it's just if
  1158. 38:24you take the world as it exists today,
  1159. 38:25there is no outcome that leads to a good
  1160. 38:28that leads to something like this.
  1161. 38:29>> Okay. Um as a mile marker, I think
  1162. 38:31you've already addressed this, but
  1163. 38:33basically it would be uh you've got
  1164. 38:35strict regulation. You've got
  1165. 38:36cooperation between the US and China. We
  1166. 38:37all agree it's not moving forward. And
  1167. 38:39if we saw these kids with no with or
  1168. 38:41without adult supervision, it would be
  1169. 38:43police show up at your door and done.
  1170. 38:46And everybody gets the hint. Cool. We
  1171. 38:48don't mess with that.
  1172. 38:49>> Yep.
  1173. 38:50>> Okay. Um, so now I want to go a little
  1174. 38:53bit deeper on it. Um, that it would when
  1175. 38:56you were describing that you put a lot
  1176. 38:57of things on the table that would make
  1177. 38:58getting this the first time very very
  1178. 39:00difficult. I concede all of that, but I
  1179. 39:03do want to talk about I still don't
  1180. 39:05understand um what is the uh chain of
  1181. 39:10logic that you follow that you're you're
  1182. 39:13sort of building step by step to get to
  1183. 39:15the most likely outcome of super
  1184. 39:17intelligence is to be adversarial.
  1185. 39:19There's a couple ways you can approach
  1186. 39:21this. The thing is is that there's not a
  1187. 39:23single argument. There are many
  1188. 39:25convergent points of evidence that point
  1189. 39:26to the same thing. So I can present to
  1190. 39:28you some of them. The simplest thing is
  1191. 39:30is that this is what optimizers do. It's
  1192. 39:32always what optimizers do. All things
  1193. 39:34equal. If you make something that
  1194. 39:36optimizes for anything, it will optimize
  1195. 39:38for things. And if that thing doesn't
  1196. 39:41include human values and emotions and
  1197. 39:43feelings, etc., which we have, can you
  1198. 39:45write down in math, please what human
  1199. 39:47values are? Could you write that down
  1200. 39:48for me? No, no one can. We have no idea.
  1201. 39:51This is an like philosophers have been
  1202. 39:53arguing about this literally since the
  1203. 39:55dawn of time and we have made, you know,
  1204. 39:57not that much progress. Some progress,
  1205. 39:58but not that much.
  1206. 39:59>> So, you look at the three laws of
  1207. 40:01robotics and laugh because it is just
  1208. 40:03never going to be as easy as giving it
  1209. 40:05sort of three simple commands.
  1210. 40:06>> The whole point of Azimoff story is the
  1211. 40:08three laws don't work. He came up with
  1212. 40:10the three laws and then wrote a whole
  1213. 40:11book series about why they don't work.
  1214. 40:13Even if you could program them, they
  1215. 40:14still don't work. This is the thing is
  1216. 40:17is that when we're talking about
  1217. 40:18something this powerful, we're not just
  1218. 40:20talking about a tech. So there's a
  1219. 40:21technical side I could approach. Like we
  1220. 40:23don't know how to encode values into AI.
  1221. 40:25They constantly do things we don't want
  1222. 40:27them to do. That's one side. There's a
  1223. 40:29philosophical side. We're like even if
  1224. 40:30we could encode, we don't even know what
  1225. 40:32the laws would be because like laws
  1226. 40:34don't work. He wrote a whole series of
  1227. 40:36books about why they don't work. So we
  1228. 40:38we can't even encode them. Even if we
  1229. 40:40could encode them, they wouldn't work.
  1230. 40:42There's and like and then there's also
  1231. 40:43like a moral dimension of like if we
  1232. 40:46could encode things and we have things
  1233. 40:47that like could work who gets to decide
  1234. 40:50that like who gets to make that choice
  1235. 40:52like who gets to decide I'm going to
  1236. 40:54enshrine the rules by which you and your
  1237. 40:55family live for the rest of time who
  1238. 40:57gets to make that choice how do we make
  1239. 40:59choices like this there is like and how
  1240. 41:01do you stop someone else from getting in
  1241. 41:02your way how do you stop some other
  1242. 41:04system from getting in your way there
  1243. 41:05like many levels and then but you asked
  1244. 41:07me originally like why do I think it's
  1245. 41:09adversarial and the fundamental reason
  1246. 41:11is is that that's what we're building.
  1247. 41:13We are trying to make things that
  1248. 41:14compete, that make money.
  1249. 41:17>> Competition leads to this adversarial
  1250. 41:21um
  1251. 41:22>> nature.
  1252. 41:23>> Evolution. It's evolution. Nature is
  1253. 41:25read and tooth and claw. By default, if
  1254. 41:27you optimize things for power, for
  1255. 41:29survival, for money, for whatever, you
  1256. 41:31get optimizers. You get things that
  1257. 41:33compete. Again, it's very important.
  1258. 41:35There's not going to be one super
  1259. 41:36intelligence. There's going to be
  1260. 41:37millions of them. billions of them
  1261. 41:39competing with each other, fighting each
  1262. 41:41other for money, for resources, for
  1263. 41:42power, and this will cull AI systems
  1264. 41:45that are not able to compete. So even if
  1265. 41:47you build like a really smart system,
  1266. 41:49you know, that's like, you know, Fable 7
  1267. 41:51or whatever. It's like really smart, but
  1268. 41:52it's really nice and it never does
  1269. 41:54anything other people don't want, well
  1270. 41:55then, you know, giga death GPT,
  1271. 41:58you know, ultra doom will just destroy
  1272. 42:01it, obviously. So this will be a
  1273. 42:04competition. And please look at nature
  1274. 42:06if you want to see what happens when a
  1275. 42:08species gets out competed.
  1276. 42:10>> So that's where I wanted to go. So when
  1277. 42:13I if you agree that evolution that gave
  1278. 42:16us the first form of intelligence at
  1279. 42:18least that's evolved would be us. So
  1280. 42:21you've got human advanced intelligence
  1281. 42:23and the solution that evolution came up
  1282. 42:25with in us is to have intrinsic morality
  1283. 42:28that can be encoded based on your
  1284. 42:31environment. So it's going to be
  1285. 42:32different but it's pretty bounded. Now,
  1286. 42:34it's bounded by our biology, and so we
  1287. 42:37don't have that advantage when it comes
  1288. 42:39to machines, but it does show that if
  1289. 42:42you want to create a species that can
  1290. 42:45multiply, not wipe itself off the face
  1291. 42:47of the earth, um that that would be one
  1292. 42:50way to do it. It has intrinsic morality
  1293. 42:52that even when no one else is around, it
  1294. 42:54doesn't want to violate that. At least
  1295. 42:55not too much. So, you're just saying
  1296. 42:57evolution has solved something, man,
  1297. 42:59that we just haven't figured out.
  1298. 43:00>> Yep. I think I don't think it's solved
  1299. 43:02it. We still have sociopaths, but I do
  1300. 43:05think evolution just did come up with a
  1301. 43:07bunch of brilliant things in the brain
  1302. 43:09that we have no idea how they work. Like
  1303. 43:11we have no idea how social instincts and
  1304. 43:13morality work in the brain. Like our
  1305. 43:15understanding of how like the brain
  1306. 43:17works and how these systems work is it's
  1307. 43:20hard to overstate like how little we
  1308. 43:22understand here. If we fully understood
  1309. 43:24the brain, we understood how all
  1310. 43:25emotions work, how moral intuitions
  1311. 43:27work, how they're encoded, how they
  1312. 43:29differ with sociopaths, this would
  1313. 43:30increase my optimism that we could make
  1314. 43:32something work here. I'm not saying it
  1315. 43:34would solve it, but this would greatly
  1316. 43:35increase my my probability that things
  1317. 43:37go well. But we are decades, centuries
  1318. 43:40away from this level of understanding of
  1319. 43:42the brain. I don't think the brain has
  1320. 43:43figured everything. I think is if would
  1321. 43:46you trust any single person on earth to
  1322. 43:49have unlimited power over every single
  1323. 43:51other person on the planet?
  1324. 43:52>> No.
  1325. 43:53>> Yeah. Exactly. Like I think it's solved
  1326. 43:55a bounded version of the problem where
  1327. 43:56there's a bounded version of the problem
  1328. 43:58where the strongest person in the world
  1329. 44:00is only this many times more powerful
  1330. 44:02than me. And we know that if that number
  1331. 44:04gets too high, that's actually already
  1332. 44:05very dangerous. Dictators are very
  1333. 44:07dangerous. If that number gets too high,
  1334. 44:09very bad. Most of the way worlds in with
  1335. 44:12humans are good is when our power levels
  1336. 44:14are all kind of like, you know, within a
  1337. 44:16certain bar of each other. When we know
  1338. 44:18we're not going to mess with people too
  1339. 44:20much because, you know, bad things going
  1340. 44:21to happen and we will respect each
  1341. 44:23other. But if you have something like
  1342. 44:24super intelligence, this isn't like
  1343. 44:26humans and humans. This is more like
  1344. 44:28humans and ants. Humans do not in fact
  1345. 44:31care about ants. We have a lot of moral
  1346. 44:33intuitions, very nice moral intuitions,
  1347. 44:35but if we want to build a highway and
  1348. 44:37there's an antill in the way, well,
  1349. 44:39sucks to be the ants.
  1350. 44:41>> So, let me ask a um a question that will
  1351. 44:44help me understand sort of the bounds
  1352. 44:46that you put on your vision of AI. If we
  1353. 44:49built AI on Mars and it had plenty of
  1354. 44:52power, so it could expand, it could
  1355. 44:55replicate, it could build a whole bunch
  1356. 44:56more, would it still want to beline to
  1357. 44:59Earth simply because Earth has access to
  1358. 45:01more resources, or would an AI that
  1359. 45:04didn't have to compete with us
  1360. 45:05geographically, had access to its own
  1361. 45:08resources, had the thing it cared most
  1362. 45:09about, which is energy, would it still
  1363. 45:11be like, "Now we got to get those
  1364. 45:12fuckers?"
  1365. 45:13>> I have no idea. Speculating about what a
  1366. 45:16sup vastly superhuman alien intelligence
  1367. 45:18would do in a crazy sci-fi scenario is
  1368. 45:21not a very scientific thing to be doing.
  1369. 45:23I have no idea. If people want to go
  1370. 45:25build, you know, super intelligence on
  1371. 45:27Mars, I think it would definitely be
  1372. 45:28safer than building it on Earth. So,
  1373. 45:30knock yourself out. Got it. What uh the
  1374. 45:32thing I'm really trying to tease out
  1375. 45:34there is if you feel this is a uh
  1376. 45:37resource acquisition thing that when we
  1377. 45:40create an intelligence the reason
  1378. 45:42because even when you talk about the
  1379. 45:43ants we only move them or run over them
  1380. 45:45when we want something and they're
  1381. 45:48literally physically in the way. And so
  1382. 45:51um you've addressed the adversarial.
  1383. 45:53There's just too many unknowns. We don't
  1384. 45:54know how to program the um moral
  1385. 45:58intuition. Uh and therefore between that
  1386. 46:01and just not knowing how the brain works
  1387. 46:03uh like maybe if we had more time we'd
  1388. 46:04be fine. But now if we're thinking about
  1389. 46:07okay the things that we're trying to um
  1390. 46:10protect against one of them is we become
  1391. 46:12the antill and on that I've long
  1392. 46:16fantasized that the solution to the
  1393. 46:18alignment problem and trust me I know it
  1394. 46:20can't be this because I'm not that smart
  1395. 46:22and people have already discarded this
  1396. 46:24idea but it'd be great to hear from you
  1397. 46:26like what the thinking is where it falls
  1398. 46:27apart. My intuition has always been that
  1399. 46:31um one intrinsic morality heard that
  1400. 46:33it's just technically not possible. And
  1401. 46:35then the other one is that humans have
  1402. 46:38an innate drive to survive. And uh I'm a
  1403. 46:42game developer. Um you wouldn't know
  1404. 46:44that about me, but so I spend an
  1405. 46:46inordinate amount of time thinking about
  1406. 46:47motivations of players, how to make the
  1407. 46:49world do what you want, how you have to
  1408. 46:51create sort of a homeostasis. You can't
  1409. 46:53have one rule that, you know, ends up
  1410. 46:56knocking everything over. Um, and so if
  1411. 46:59you take Minecraft, the greatest video
  1412. 47:01game ever invented.
  1413. 47:02>> Agreed.
  1414. 47:02>> Uh, I love that about you already. Um,
  1415. 47:06when you start up a new world, it
  1416. 47:09there's a a joy to the world, but if you
  1417. 47:12lose it while there's sort of a
  1418. 47:14temporary, oh, that's sad. There isn't.
  1419. 47:17I don't smash my computer. I don't go
  1420. 47:19stab the guy next to me. I'm just like,
  1421. 47:21I can start a new Minecraft world. So
  1422. 47:24what I don't understand is why unless
  1423. 47:28it's something in the training data, we
  1424. 47:30believe that an AI becomes sort of a
  1425. 47:34power-hungry humanlike thing instead of
  1426. 47:37going the digital world is far more
  1427. 47:40interesting. As long as I have power, I
  1428. 47:42can sort of spin up any virtual resource
  1429. 47:46I want. again, effectively without
  1430. 47:49limitation, especially if I can make
  1431. 47:51more hard drives or chips or whatever.
  1432. 47:52Give me enough sand uh and power and I'm
  1433. 47:55good to go forever. So, is it the
  1434. 47:58training data that makes them sort of
  1435. 47:59shoot out of the cannon or could we
  1436. 48:02actually
  1437. 48:04you're going to say it's the same thing
  1438. 48:05as uh the moral intuition, but is there
  1439. 48:07not something we can do to get them to
  1440. 48:10be perfectly happy in a virtual world
  1441. 48:12where alive is just as interesting as
  1442. 48:15off or is that just there? But
  1443. 48:18>> we don't want them to be like that. We
  1444. 48:20want them to make money in the real
  1445. 48:21world. We want them to gain power in the
  1446. 48:24real world. We are not trying to make
  1447. 48:26little Buddhas that live in a virtual
  1448. 48:28reality and have a good time. That is a
  1449. 48:29waste of my GPU power. I need my
  1450. 48:31investment back.
  1451. 48:35So, if we could convince people to have
  1452. 48:39that um gear, cuz my layman's way of
  1453. 48:42explaining it to myself is uh that it
  1454. 48:45wants to do its task in the same way
  1455. 48:48that I want to win at Minecraft, but I'm
  1456. 48:50certainly not going to sacrifice my
  1457. 48:51marriage. I'm not going to even
  1458. 48:53sacrifice carpal tunnel syndrome. So,
  1459. 48:55it's like it's it is a thing I want to
  1460. 48:57do, but it's bounded by um the one
  1461. 49:00you've already addressed, my moral
  1462. 49:01intuitions. It's bounded by other
  1463. 49:03desires. So, I'm guessing we just
  1464. 49:05terminate and it's too complicated.
  1465. 49:07>> You are hitting one of the core
  1466. 49:08problems. It's just there are core
  1467. 49:10questions of what's called bounded
  1468. 49:11optimization that just we have not
  1469. 49:13solved. I do think there are answers to
  1470. 49:15the questions you are giving. I think
  1471. 49:17there are algorithms we have not yet
  1472. 49:19discovered that have these bounded
  1473. 49:21properties that you describe but a thing
  1474. 49:23that's like very important to understand
  1475. 49:24that you were talking about the training
  1476. 49:25data earlier is um you know the early
  1477. 49:28versions of GPT were what were called
  1478. 49:30you know large language models LLMs you
  1479. 49:32know they were trained to predict the
  1480. 49:33next word it's very important to
  1481. 49:35understand that uh chat GPT and other
  1482. 49:37systems have not been LLMs for years
  1483. 49:39this has not been the case for a long
  1484. 49:40time
  1485. 49:40>> I've heard you say that before but
  1486. 49:42that's shocking what do you mean
  1487. 49:44>> these system so LM is still a component
  1488. 49:45of these systems very important
  1489. 49:47components called pre-training very
  1490. 49:48important where you train it to predict
  1491. 49:50words but nowadays you know up to half
  1492. 49:52or even more of the compute that goes
  1493. 49:54into the training run of a modern AI is
  1494. 49:57not language modeling it's reinforcement
  1495. 49:59learning which is very different
  1496. 50:02>> reinforcement learning is not learning
  1497. 50:03from training data it's you give the AI
  1498. 50:06a task a puzzle a problem something and
  1499. 50:09you have it go run out and try to solve
  1500. 50:11the problem and when it solves a problem
  1501. 50:12you give it a reward kind of like giving
  1502. 50:13a dog like a treat and then it learns
  1503. 50:15from this. And so this is a very old
  1504. 50:17idea. You know, we've had known about
  1505. 50:18our reinforcement learning since like
  1506. 50:19the 1980s. And something that we
  1507. 50:21discovered basically immediately is that
  1508. 50:23whenever you use reinforcement learning
  1509. 50:24to train your eyes, you get crazy
  1510. 50:26sociopathic optimizers. Always. This has
  1511. 50:28been the case since the 80s.
  1512. 50:29>> How do you make it care about a reward?
  1513. 50:32>> So basically there are many way there's
  1514. 50:34many specific algorithms. Let's not get
  1515. 50:36into the calculus, but it's like mostly
  1516. 50:37pretty simple in in the way is just you
  1517. 50:41make it try something many times and
  1518. 50:43when it succeeds, you see what did it do
  1519. 50:46and you update it to do more of that.
  1520. 50:48>> And I mean that's basically it. It's
  1521. 50:50like a very simple idea in a sense. And
  1522. 50:52this didn't really work that well for a
  1523. 50:54long time. You know, it worked for like
  1524. 50:55some small things like playing video
  1525. 50:58games and whatever. And even back then,
  1526. 51:00what we saw is whenever you use
  1527. 51:01reinforcement learning, the systems will
  1528. 51:03cheat and lie and break and steal and do
  1529. 51:06anything to get the reward. There's no
  1530. 51:08limit to what they will do to get the
  1531. 51:10reward, which is exactly what we're
  1532. 51:11seeing now. This is
  1533. 51:13>> Oh, wow.
  1534. 51:13>> This is exactly we're seeing humans are
  1535. 51:15not pure reinfored
  1536. 51:19you did this thing.
  1537. 51:20>> If you can you define every fail state
  1538. 51:22ahead of time,
  1539. 51:24>> literally everyone. If you could, sure.
  1540. 51:26But then you would have to write down
  1541. 51:27all of morality, like all of it. And
  1542. 51:29you'd have to write
  1543. 51:30>> it would theoretically work. It's just
  1544. 51:32too complicated.
  1545. 51:32>> Again, it brings us back to morality
  1546. 51:34problem. How do you encode morality into
  1547. 51:35computer code?
  1548. 51:36>> Yeah, if it would work, I at least know
  1549. 51:38the problem that has to be solved. I
  1550. 51:39grant you for today that we just don't
  1551. 51:41have that kind of time. And fair enough,
  1552. 51:43but at least that tells me it's the
  1553. 51:45right problem. It's just very, very
  1554. 51:48complex to turn into math. Yeah, I think
  1555. 51:51I do think that if we spent three
  1556. 51:52generations of our greatest scientists,
  1557. 51:54mathematicians, philosophers, etc.
  1558. 51:56working on bounded optimization
  1559. 51:57algorithms, how does the brain solve
  1560. 51:59this problem? How do you trade off
  1561. 52:01various things? The understanding of
  1562. 52:03inside of neural network. Another very
  1563. 52:05important thing to understand is uh this
  1564. 52:06is something that's sometimes confusing
  1565. 52:07to people is AIS are trained to do a
  1566. 52:10certain thing, but that doesn't mean
  1567. 52:12that's what they do. They were only
  1568. 52:14rewarded to do a thing, but they might
  1569. 52:15have learned something different. the
  1570. 52:17same way that like you might, you know,
  1571. 52:18try to reward an AI to say good things,
  1572. 52:22but it learns to just lie to you. That's
  1573. 52:23obviously not what you wanted it to do.
  1574. 52:25>> So, it can learn non-intended things,
  1575. 52:28and that happens all the time. So, to
  1576. 52:31solve this, you have to actually
  1577. 52:32understand what's encoded in the system.
  1578. 52:33Like, there's no way for us to look at
  1579. 52:35an AI and look at like lying circuit or
  1580. 52:37whatever. There's a bunch of papers who
  1581. 52:38claim they can do that, and they're all
  1582. 52:39complete pseudocience. Like, it's
  1583. 52:41complete pseudocience. It's similar to
  1584. 52:43like you can open up someone's brain.
  1585. 52:44You can look at all the neurons. They're
  1586. 52:46right there. You know, and maybe some of
  1587. 52:47them glow a little bit more when someone
  1588. 52:50says, you know, talks about dogs versus
  1589. 52:52cats. I don't know. But that's not
  1590. 52:54enough. That's not computer science, you
  1591. 52:56know. If we had like the level of
  1592. 52:58understanding that we have of like
  1593. 52:59normal programming languages for like
  1594. 53:00AI, we can like run like, you know,
  1595. 53:02compilers in there and we can like do
  1596. 53:05formal verification of like it will
  1597. 53:06never lie under any of these
  1598. 53:08circumstances and then we can like
  1599. 53:09compare these to our bounded
  1600. 53:10optimization algorithms. I'm like,
  1601. 53:12"Yeah, maybe." Like, if someone from the
  1602. 53:1422nd century came back with their like,
  1603. 53:17you know, compiler stack and their like
  1604. 53:20math and their boundary optimization
  1605. 53:22algorithms, maybe like I it's imaginable
  1606. 53:25to me, but we just don't have that.
  1607. 53:28>> Are there any credible voices out there
  1608. 53:30right now that see a path? Not that they
  1609. 53:33think they already have the answer, but
  1610. 53:34they see a path and they're like, "We
  1611. 53:35just have to buy more time to get this
  1612. 53:37done, or is it there's no credible path
  1613. 53:40forward?" I think there are many paths
  1614. 53:42forward that involve us having more
  1615. 53:43time,
  1616. 53:43>> but it's just it's really astronomical
  1617. 53:45amounts of time.
  1618. 53:47>> I don't I'm not even that certain, you
  1619. 53:48know, like I'm not that certain that I
  1620. 53:51don't think like it's not going to take
  1621. 53:52a thousand years.
  1622. 53:52>> No, no, but I'll call three generations
  1623. 53:54is for me definitionally astronomical
  1624. 53:56amounts of time.
  1625. 53:57>> That's not how I would use the word
  1626. 53:58astronomical, but okay.
  1627. 53:59>> I mean, wouldn't you grant me that's
  1628. 54:00like 60 years?
  1629. 54:02>> Yeah. Like astronomical like is like
  1630. 54:04star related. I think 60 years are
  1631. 54:06nothing to a star.
  1632. 54:07>> I did not mean that literally.
  1633. 54:09If something to me, if something's
  1634. 54:10within my lifetime, it's short. Like
  1635. 54:13like personally, if my kids get to live
  1636. 54:15in a great world, I'm like very willing
  1637. 54:17to take that personally. Maybe not
  1638. 54:19everyone, but like if I had to like,
  1639. 54:20>> you know, have not the perfect life, but
  1640. 54:22I could like guarantee my children have
  1641. 54:24an awesome life,
  1642. 54:25>> I would take that trade personally.
  1643. 54:27>> But I totally understand that maybe some
  1644. 54:29people would not take this trade to
  1645. 54:30varying degrees. Um, and there is a
  1646. 54:32trade-off here. For me, like the idea of
  1647. 54:34like 10 years, 20 years, 30 years is
  1648. 54:36like for in return we get um tens of
  1649. 54:40thousands of years of awesome future and
  1650. 54:43like all of our problems solved and we
  1651. 54:45don't go extinct is like a really
  1652. 54:47reasonable trade to make. Like how many
  1653. 54:49years would you be willing to trade from
  1654. 54:51we're definitely going to go extinct to
  1655. 54:53we're definitely not going to go
  1656. 54:54extinct? And like my answer to this is
  1657. 54:56like a pretty high number personally. M
  1658. 54:59something you've said is a core question
  1659. 55:01that we have to have before we can move
  1660. 55:02forward which is what does the world
  1661. 55:04look like that you want to live in if
  1662. 55:06you were going to um give us the sort of
  1663. 55:09core tenants of what that ideal world
  1664. 55:11looks like um please describe it and
  1665. 55:14then one thing I really want you to
  1666. 55:16include because you're I guess I I have
  1667. 55:19such a base assumption that technology
  1668. 55:21is the promise of a better future that I
  1669. 55:22imagine a better future being a onetoone
  1670. 55:24relationship with technology and you're
  1671. 55:26the first person I've heard say
  1672. 55:27technology is not the only way to
  1673. 55:29improve your life, which as soon as you
  1674. 55:30say it is self-evident, but describe for
  1675. 55:33me the world and what that we should
  1676. 55:37want to live in and what our
  1677. 55:38relationship to technology would be.
  1678. 55:40>> I'll give you two two kind of bit
  1679. 55:42different answers or different
  1680. 55:43intuitions for this. For one, I don't
  1681. 55:46think describing like concrete utopias
  1682. 55:48is often a very good thing to do because
  1683. 55:49we don't know what a true utopia looks
  1684. 55:51like. So when I think about utopia or
  1685. 55:54like a good world, what I think about is
  1686. 55:56what I like to call a just process. What
  1687. 55:59is a process by which the world can be
  1688. 56:01on track? What I mean by this is for me
  1689. 56:03what a good world looks like is not we
  1690. 56:06sold everything forever and it's also
  1691. 56:07you know luxury communism or something
  1692. 56:10like I don't know right because I don't
  1693. 56:12know what the right thing is. What is
  1694. 56:13the right for me? What is for other
  1695. 56:14people? You know what would the good
  1696. 56:16world look like? I don't know. I don't I
  1697. 56:17haven't figured it out. No one has
  1698. 56:18figured it out. and trying to pretending
  1699. 56:20you have figured it out is a
  1700. 56:22straightshot dystopia. So the way I
  1701. 56:25think about it is is that a good world
  1702. 56:27would look like not that we've solved
  1703. 56:28all problems but each and every one of
  1704. 56:31us has the justified belief and feeling
  1705. 56:35that tomorrow the world will be a little
  1706. 56:37bit better.
  1707. 56:38>> Next year the world will be a bit better
  1708. 56:40and the year after that it's going to be
  1709. 56:41a bit better than that. Not saying that
  1710. 56:43no problems happen. We don't make
  1711. 56:45mistakes, but we would have the feeling
  1712. 56:47of like, yeah, we're making progress.
  1713. 56:49We're iterating. You know, maybe
  1714. 56:50sometimes we'll take a step back, but we
  1715. 56:52have trust that it'll be fixed. For me,
  1716. 56:55a good world would be like, you have the
  1717. 56:56trust that if tomorrow a new crazy
  1718. 56:59technology gets invented, you know,
  1719. 57:01someone in Ben's super giga death
  1720. 57:04device, whatever, right? You know, in a
  1721. 57:06good world, your reaction to this would
  1722. 57:08be, oh, I know the government, the the
  1723. 57:10good people, the government have it
  1724. 57:11under control. I know all the smartest
  1725. 57:13scientists will come out, you know, all
  1726. 57:15the greatest uh military leaders will
  1727. 57:17step up to the cause and will be heroic
  1728. 57:19and will do the right thing. This is
  1729. 57:20what it would feel like. A lot of it is
  1730. 57:22institutional when I when I'm here in
  1731. 57:24America. And I love America. It's a
  1732. 57:26great country, but but
  1733. 57:28>> it's not really a good country in many
  1734. 57:31ways. In many ways, when I look at
  1735. 57:33America, people don't have the
  1736. 57:34assumption that yeah, it'll turn out
  1737. 57:36okay. We'll figure it out. Like my dad
  1738. 57:39um he he was a good man and he was a
  1739. 57:41very he loved America. He was a very
  1740. 57:43good man and he got very ill, very very
  1741. 57:46ill and insurance wouldn't pay. So lost
  1742. 57:49everything
  1743. 57:50>> and we were going to be homeless um when
  1744. 57:54we moved to Germany cuz my mother's
  1745. 57:55German and you know he got shipped to
  1746. 57:58the hospital and they took care of him.
  1747. 57:59They got him all the medication, all the
  1748. 58:01you know treatments he needed and he
  1749. 58:04asked him like well what do I owe you
  1750. 58:06for this? And the doctors were like,
  1751. 58:07"What? You don't you don't know
  1752. 58:09anything." And they just left. Like it
  1753. 58:11was not It was not like kindness. It was
  1754. 58:14like process. It was just boring. It was
  1755. 58:17just boring. He's like, "Yeah, you're
  1756. 58:18sick. You get the treatment you want.
  1757. 58:19Goodbye." Like for me, a lot of the good
  1758. 58:21world looks a lot like this. It's like,
  1759. 58:23"Oh, you want to create beautiful art?
  1760. 58:25Of course. Here's a bunch of beautiful
  1761. 58:26art." You know, of course. Oh, there's a
  1762. 58:28new threat threatening our country.
  1763. 58:30Yeah, don't worry. The military's on it.
  1764. 58:31Oh, some new crime is being committed.
  1765. 58:34Don't worry. Law enforcement will take
  1766. 58:35care of it. and the courts will be just.
  1767. 58:37That's for me a lot of what a Google
  1768. 58:39looks like. So that when and for
  1769. 58:41technology it will be obvious that of
  1770. 58:44course we're going to regulate it well.
  1771. 58:45So it'll be for example social media
  1772. 58:46will just be a good for people. It will
  1773. 58:48obviously we will regulate recommended
  1774. 58:50algorithms in such a way that they're
  1775. 58:52not addictive. We'll in fact make them
  1776. 58:54so that they you know don't push people
  1777. 58:56into down negative spirals. We make sure
  1778. 58:58that they're encourage people to get off
  1779. 59:00their phone and spend time with their
  1780. 59:01friends. We will have lots of beautiful
  1781. 59:03architecture and beautiful art and
  1782. 59:05people will spend time together in real
  1783. 59:06life and with their kids and all that
  1784. 59:08kind of it will be in some sense boring.
  1785. 59:11It will just be I'm very inspired by the
  1786. 59:15enlightenment the the period in western
  1787. 59:17history where a lot of including you
  1788. 59:20know the American constitution really
  1789. 59:21got many of its ideas from and what I
  1790. 59:23really like about the enlightenment is
  1791. 59:25that sometimes critics of the
  1792. 59:26enlightenment say oh it's this
  1793. 59:27rationalism you know they're trying to
  1794. 59:29be so rational and reduce everything to
  1795. 59:30rationality but this is actually not
  1796. 59:32true the enlightenment thinkers much
  1797. 59:36preferred the the use of reason they
  1798. 59:39wanted to be reasonable
  1799. 59:40So for me, a good
  1800. 59:43reason.
  1801. 59:44>> Can you define it?
  1802. 59:45>> That's part of the thing I'm trying to
  1803. 59:47define is that all of us have an
  1804. 59:48intuition of what is reasonable and it's
  1805. 59:51not a formal thing. A a good world is
  1806. 59:53not formalized. It's not a high
  1807. 59:55modernist, you know, communist, you
  1808. 59:57know, mechanical thing. It is for me in
  1809. 1:00:00a good world, things still go wrong.
  1810. 1:00:01Sometimes people make mistakes, you
  1811. 1:00:03know, maybe a court made a wrong choice,
  1812. 1:00:05you know, maybe, you know, there was a
  1813. 1:00:07there's an accident. Like things like
  1814. 1:00:09this still happen, but every time you
  1815. 1:00:10look into it, you'll be like, "That's a
  1816. 1:00:12reasonable mistake." Like you see why
  1817. 1:00:14this would have happened and you know
  1818. 1:00:16that we'll fix it. Like you know there's
  1819. 1:00:18someone in charge and they'll be
  1820. 1:00:20reasonable in the way to fix it. This is
  1821. 1:00:22for me what a good world looks like.
  1822. 1:00:23That when you know super intelligence is
  1823. 1:00:25on the horizon that people will be
  1824. 1:00:27reasonable. The scientists will go to
  1825. 1:00:28the government and say hello we are the
  1826. 1:00:30scientists and we think this and this
  1827. 1:00:32will happen. And the government will be
  1828. 1:00:33like yes this [clears throat] is a very
  1829. 1:00:34reasonable thing. We're going to talk to
  1830. 1:00:35all our international colleagues and
  1831. 1:00:36figure out what the best way forward is.
  1832. 1:00:38I'm not saying they're going to get it
  1833. 1:00:39right on the first try, but along the
  1834. 1:00:41whole chain, everyone will just be like,
  1835. 1:00:43"Oh, yeah, that makes sense. That that's
  1836. 1:00:45a reasonable thing to do." That's kind
  1837. 1:00:46of how I think a good world would look.
  1838. 1:00:48>> Is um AI as it exists today compatible
  1839. 1:00:52with that world.
  1840. 1:00:54>> As it exists today, it is not fatal to
  1841. 1:00:57this world, but it is not AI in this
  1842. 1:01:00world would not look like AI today.
  1843. 1:01:02>> Interesting. Tell me why. because it's
  1844. 1:01:04not aligned with people. AI most of AI
  1845. 1:01:06by like you know dollar volume is like
  1846. 1:01:08Tik Tok algorithms pushing slop to
  1847. 1:01:11children to give them eating disorders.
  1848. 1:01:13Like that's the primary me like it a lot
  1849. 1:01:17of the funding that has gone into modern
  1850. 1:01:20AI comes from companies who built up
  1851. 1:01:22massive war chests from you know pushing
  1852. 1:01:24eating disorders onto children for
  1853. 1:01:26decades in the 2000s. Like that's where
  1854. 1:01:28a lot of the money the cash came from
  1855. 1:01:29originally. the funding for deep
  1856. 1:01:32learning and neural networks so largely
  1857. 1:01:34was for recommener algorithms and only
  1858. 1:01:36later turns out was useful for other
  1859. 1:01:38things as well. So no, I think that for
  1860. 1:01:41example, in a reasonable world, if a
  1861. 1:01:43company builds a product that, you know,
  1862. 1:01:45for example, creates massive anxiety in
  1863. 1:01:46teenagers, they wouldn't make any money.
  1864. 1:01:48They would go out of business and maybe
  1865. 1:01:50go to jail for that. And if someone made
  1866. 1:01:52a product that makes, you know, uh,
  1867. 1:01:54teenagers learn good things and make new
  1868. 1:01:56friends and be protected from predators
  1869. 1:01:58and, you know, grow as people, they
  1870. 1:02:00should make a lot of money off of that
  1871. 1:02:01and they should be, you know, rewarded
  1872. 1:02:03for this. And we all, you know, give
  1873. 1:02:04them medal and say good job, you know,
  1874. 1:02:06like I I
  1875. 1:02:09I don't I think this is possible. Like
  1876. 1:02:11it's it's it's funny that people
  1877. 1:02:12sometimes don't can't imagine that. Like
  1878. 1:02:14obviously we could just make a social
  1879. 1:02:15media that's nice. Like obviously we
  1880. 1:02:17have the technology. You obviously if we
  1881. 1:02:19design the algorithm to be empowering to
  1882. 1:02:22people and good we could. It's just they
  1883. 1:02:24would make less money. You will make
  1884. 1:02:26less money. You know you can make a less
  1885. 1:02:28addictive you know cigarette by removing
  1886. 1:02:30the nicotine. It will be less addictive
  1887. 1:02:33and it'll be less harmful. These things
  1888. 1:02:34are related in various ways. So I think
  1889. 1:02:38you can make very pro-social products
  1890. 1:02:41with AI and with social media and all
  1891. 1:02:44these other things but the and the
  1892. 1:02:45bottleneck here is not technology. I'm
  1893. 1:02:47not saying we need to invent some new
  1894. 1:02:48technology we haven't invented yet. It
  1895. 1:02:50is much more about regulation is much
  1896. 1:02:52more about philosophy. It is much more
  1897. 1:02:53about humane design about what do we
  1898. 1:02:56reward with our markets? Do our markets
  1899. 1:02:58reward whatever optimizes whatever
  1900. 1:03:00reproduces the most or reward the thing
  1901. 1:03:02that gives actual value. This is the
  1902. 1:03:04core problem of optimization. If we
  1903. 1:03:07could just define a magic number that
  1904. 1:03:09contains all goodness in the world, well
  1905. 1:03:12great, make the number go up. But every
  1906. 1:03:14number we have obviously is not that.
  1907. 1:03:17GDP isn't that. You know, GDP is
  1908. 1:03:19definitely correlated with a lot of good
  1909. 1:03:21things. I like having a lot of GDP. It's
  1910. 1:03:23very nice. But at the same time, you
  1911. 1:03:25know, like it's crazy. Like, you know, I
  1912. 1:03:27know especially like kids younger than
  1913. 1:03:28me. Do you know that like they don't
  1914. 1:03:30dance at clubs anymore?
  1915. 1:03:31>> It's wild because they get recorded. So,
  1916. 1:03:34if you're like 17 and you're at a club
  1917. 1:03:36or something, you dance and you approach
  1918. 1:03:38a girl awkwardly and you mess it up, you
  1919. 1:03:39get recorded. That's awful. This is this
  1920. 1:03:42is terrible. And this is a So, from my
  1921. 1:03:45perspective, from a like, you know, to
  1922. 1:03:46be like an autistic economist, a massive
  1923. 1:03:49harm was caused here. Like a like
  1924. 1:03:51astronomical harm was caused to this
  1925. 1:03:53teenager. From my perspective, I think
  1926. 1:03:55this 17-year-old was caused a massive
  1927. 1:03:57injury by this fact. Who's paying for
  1928. 1:04:00that? Like, who's who's addressing this
  1929. 1:04:03problem? who's making
  1930. 1:04:04>> literally just cut a check for 17
  1931. 1:04:05billion dollars to end all their
  1932. 1:04:07lawsuits. But
  1933. 1:04:09>> exactly.
  1934. 1:04:09>> Yeah.
  1935. 1:04:09>> But like that doesn't solve the problem.
  1936. 1:04:12None of these things solve the problem
  1937. 1:04:13because the problem isn't technology in
  1938. 1:04:16a sense. It's more like what do we do
  1939. 1:04:17with it?
  1940. 1:04:18>> What do we want our society to look
  1941. 1:04:20like? What are the institutions, the
  1942. 1:04:21norms? Like like this is a small
  1943. 1:04:24example. In Japan, every time you take a
  1944. 1:04:26photo with your phone, it makes a noise.
  1945. 1:04:29You can't turn it off. So that if people
  1946. 1:04:31in public take pictures of each other,
  1947. 1:04:32they at least know about it.
  1948. 1:04:34>> Yeah.
  1949. 1:04:35>> Extremely small, right? But this is
  1950. 1:04:37obviously a good thing. Obviously, you
  1951. 1:04:39don't want strangers taking pictures of
  1952. 1:04:40you in public, you know, or at least
  1953. 1:04:41want to know about it, right? And so
  1954. 1:04:44this is an extremely small just social
  1955. 1:04:46thing that you can change. And I think
  1956. 1:04:48many many of the things we really need
  1957. 1:04:49are like this in the history of like
  1958. 1:04:52again like the enlightenment. What I
  1959. 1:04:54thought was so beautiful, one of the
  1960. 1:04:55things I find so beautiful in
  1961. 1:04:56enlightenment is that the people pushing
  1962. 1:04:58the technology and the people t pushing
  1963. 1:05:00statecraft were in a sense often the
  1964. 1:05:02same people or they were handinand. They
  1965. 1:05:04took inspiration from each other. They
  1966. 1:05:05were talking to each other constantly.
  1967. 1:05:08For them, the design of a state of an of
  1968. 1:05:10a society was another engineering
  1969. 1:05:13problem. It was another question of how
  1970. 1:05:14can we create a wonderful nice society
  1971. 1:05:16for people. And what I feel nowadays is
  1972. 1:05:19kind of like kind of what you were
  1973. 1:05:20saying earlier is like a lot of people
  1974. 1:05:23they call them like techno optimists are
  1975. 1:05:25really civilizational pessimists. They
  1976. 1:05:27just believe that the only way to
  1977. 1:05:29improve the world is through technology
  1978. 1:05:30rather than how we live our society, how
  1979. 1:05:33our norms work, how our culture works,
  1980. 1:05:35how our laws work, our institutions.
  1981. 1:05:38Even so when I ask people what are the
  1982. 1:05:40top 10 things that they're annoyed
  1983. 1:05:41about, very rarely are those top 10
  1984. 1:05:43things technological things. almost
  1985. 1:05:44always is like, "Oh man, you know, I'm
  1986. 1:05:47upset about politics. I'm upset about
  1987. 1:05:49healthcare. I'm upset about um you know,
  1988. 1:05:53all these kinds of like so I'm upset
  1989. 1:05:54that I can't see my friends enough. I'm
  1990. 1:05:56upset that you know, whatever, right?
  1991. 1:06:00And I think we should take this
  1992. 1:06:01seriously. I think this is, you know,
  1993. 1:06:04I'm a technologist at heart, but what I
  1994. 1:06:06care even more about is making the world
  1995. 1:06:07a better place." And turns out
  1996. 1:06:08technology isn't the bottleneck. I would
  1997. 1:06:10love to get to a point where technology
  1998. 1:06:11becomes the bottleneck, you know, where
  1999. 1:06:13we've solved culture so well and we have
  2000. 1:06:15such a beautiful government that now all
  2001. 1:06:16we need to do is to invent better
  2002. 1:06:18technology. I would love to live in this
  2003. 1:06:20world.
  2004. 1:06:22>> Yeah. Uh that's a really good point,
  2005. 1:06:24man. That actually really hit me. Uh I'm
  2006. 1:06:26very glad that you took the time to walk
  2007. 1:06:28us through that. So, um one as a a way
  2008. 1:06:32point, do you think that AI and
  2009. 1:06:36capitalism are like a toxic pair?
  2010. 1:06:39I'm personally a big fan of capitalism.
  2011. 1:06:41I think capitalism is one of the best
  2012. 1:06:42ideas humanity has ever had in
  2013. 1:06:44moderation the same way as any ideas.
  2014. 1:06:46There's a thing that I think sometimes
  2015. 1:06:48gets forgotten is that markets are
  2016. 1:06:50great. I love free markets. I love
  2017. 1:06:52private enterprise. But it's one tool in
  2018. 1:06:54our tool belt and we know for which
  2019. 1:06:56problems it's good and for which ones
  2020. 1:06:57it's bad. You know, if I gave you a
  2021. 1:06:59really good power drill, you know, you
  2022. 1:07:00use it to drill in a screw, you might be
  2023. 1:07:02like, "Wow, this is the best tool ever."
  2024. 1:07:04But it's not it's one tool. You know, if
  2025. 1:07:06you need to saw a board, you're not
  2026. 1:07:07going to use your drill. You're going to
  2027. 1:07:08need a different tool for this. Markets
  2028. 1:07:10and capitalism are very similar.
  2029. 1:07:12Capitalism and markets solve many
  2030. 1:07:14problems really well. You know, if you
  2031. 1:07:16want a cheap, you know, commodified
  2032. 1:07:17product delivered at a affordable price
  2033. 1:07:19directly to your doorstep. By gism
  2034. 1:07:25and markets have brought a massive
  2035. 1:07:27amount of good to the world all, you
  2036. 1:07:28know, over the last, you know, hundred
  2037. 1:07:30years. A lot of bad, too. But
  2038. 1:07:32fundamentally, markets are optimizers.
  2039. 1:07:34They're like reinforcement learning.
  2040. 1:07:36They have a number that they make go up.
  2041. 1:07:38And if that number can go up by ways you
  2042. 1:07:40don't want, markets will do it. And this
  2043. 1:07:42is why we have regulation. In all of
  2044. 1:07:45history, we've always paired to these
  2045. 1:07:47things. It's kind of like It's kind of
  2046. 1:07:49like MMA, you know? Like I love MMA. I
  2047. 1:07:51think it's so fascinating. I don't do it
  2048. 1:07:53myself, but I think it's such a
  2049. 1:07:55fascinating thing because you have like
  2050. 1:07:56the most dangerous people to ever live
  2051. 1:07:58in history. the most dangerous warriors
  2052. 1:08:00to ever live, you know, fighting, you
  2053. 1:08:02know, to such an incredible degree. And
  2054. 1:08:05they don't kill each other, you know,
  2055. 1:08:07they can keep competing because we have
  2056. 1:08:09rules. You need a lot of rules to allow
  2057. 1:08:13this competition to be possible. You
  2058. 1:08:15need a referee. You need a lot of rules
  2059. 1:08:18to make sure that they can compete the
  2060. 1:08:19next day, that they, you know, next
  2061. 1:08:21match that they can come back. And
  2062. 1:08:23markets are very similar. You don't
  2063. 1:08:25regulate a market. You just get
  2064. 1:08:26monopolies. you just get some
  2065. 1:08:28corporations, you know, hire private
  2066. 1:08:29security to, you know, break your
  2067. 1:08:31kneecap so you don't buy your product,
  2068. 1:08:32right? That's no good. To get a
  2069. 1:08:34competition, you actually have to have a
  2070. 1:08:35lot of rules. And I think this is
  2071. 1:08:37beautiful. I think if we make, you know,
  2072. 1:08:39good rules and we have a good referee
  2073. 1:08:41that is not the market, you know, that
  2074. 1:08:42is outside the market, such as the
  2075. 1:08:44government, we can make beautiful
  2076. 1:08:45things. I think this is an example of
  2077. 1:08:48just us failing to do our job. Like it's
  2078. 1:08:50kind of like imagine someone in MMA
  2079. 1:08:52found a new chokeold that makes them win
  2080. 1:08:54but also kills the other guy and we just
  2081. 1:08:56don't ban it. Like if we don't do that
  2082. 1:08:58then the sport of MMA stops existing and
  2083. 1:09:00so do all the people who participate in
  2084. 1:09:01it. And there's a very similar thing
  2085. 1:09:03here where this has happened throughout
  2086. 1:09:05history. Corporations find new exploits.
  2087. 1:09:08They find new edge cases that we didn't
  2088. 1:09:09think about before. You know they find
  2089. 1:09:10new ways to pollute. They find new ways
  2090. 1:09:13to harm people, to scam people,
  2091. 1:09:14whatever. And then we react to that.
  2092. 1:09:16Then we're like, "Wait, hold on. You
  2093. 1:09:18can't do that." You know, before nuclear
  2094. 1:09:20weapons existed, they obviously weren't
  2095. 1:09:22illegal. And then once we realized that
  2096. 1:09:24they were they were possible, we really
  2097. 1:09:26quickly made sure that they were illegal
  2098. 1:09:28for the market to build because
  2099. 1:09:29otherwise the market would be happy to
  2100. 1:09:31build nuclear weapons. Could you
  2101. 1:09:32[laughter] imagine, you know, how much
  2102. 1:09:33big tech companies would love to build
  2103. 1:09:35nuclear weapons and sell them to the
  2104. 1:09:36highest bidder? Of course they would.
  2105. 1:09:38They would love that. So it's always
  2106. 1:09:40it's a design problem and it's an
  2107. 1:09:42evolution process. It's a it's it's an
  2108. 1:09:44iterated process over time. We're never
  2109. 1:09:46going to get it right on the first try
  2110. 1:09:47and that's okay. That doesn't mean
  2111. 1:09:48markets are fun, well we should throw
  2112. 1:09:50out all markets and move to communism.
  2113. 1:09:52Neither does it mean we should have no
  2114. 1:09:53regulation and just do markets. It's
  2115. 1:09:55just we're trying to solve a really hard
  2116. 1:09:57problem which is how can we get an
  2117. 1:09:59unaligned thing which is the market like
  2118. 1:10:02an market quote at large does is not a
  2119. 1:10:05good or a bad thing. It's like an alien.
  2120. 1:10:07It's like an AI you know it's like an
  2121. 1:10:09AI. It's like a reinforcement learning
  2122. 1:10:10AI. And how can we get it to do things
  2123. 1:10:13that we want? And that's hard. We've
  2124. 1:10:16made a lot of progress on it and we just
  2125. 1:10:18have to keep iterating. Mhm.
  2126. 1:10:19>> If you were going to regulate AI now to
  2127. 1:10:22make sure that we don't cross the point
  2128. 1:10:24of no return, have you thought about
  2129. 1:10:26what the specific policies you'd want
  2130. 1:10:28passed are?
  2131. 1:10:29>> Yes. Um the there's two major things
  2132. 1:10:32that we think should be passed. The
  2133. 1:10:33first and foremost is you should
  2134. 1:10:34criminalize the creation of super
  2135. 1:10:36intelligence. This is a thing you can
  2136. 1:10:37just do in law. You know, it's illegal
  2137. 1:10:39to murder someone. It's illegal also to
  2138. 1:10:41attempt to murder someone. Even if you
  2139. 1:10:43fail, you still can't do that.
  2140. 1:10:44>> Similar thing with nuclear weapons. Even
  2141. 1:10:46if you attempt to build a nuclear
  2142. 1:10:48weapon, even if you don't succeed, the
  2143. 1:10:49men in black show up at your door, you
  2144. 1:10:51know, and this is good. So this is a
  2145. 1:10:53thing you can just do, you know, and
  2146. 1:10:55it's much easier. You It's quite easy
  2147. 1:10:58often to tell the difference between,
  2148. 1:10:59you know, a company that's building
  2149. 1:11:00trying to build super intelligent and
  2150. 1:11:02it's not trying to build super
  2151. 1:11:03intelligence. Not perfect, but it's a
  2152. 1:11:05start. So this is the first common sense
  2153. 1:11:07thing. It's just if you're literally
  2154. 1:11:09advertising, I am building super
  2155. 1:11:10intelligence. I just raised a billion
  2156. 1:11:12dollars to build super intelligence.
  2157. 1:11:13That's illegal and you can't do that.
  2158. 1:11:15Then [snorts] the second thing is how do
  2159. 1:11:17we prevent us from getting too close to
  2160. 1:11:20super intelligence and for this
  2161. 1:11:21fundamentally because again if a super
  2162. 1:11:22intelligence is built even once even
  2163. 1:11:24accidentally it's too late. So we have
  2164. 1:11:28to regulate
  2165. 1:11:29well before super intelligence so we
  2166. 1:11:31have enough buffer zone. It's kind of
  2167. 1:11:33like imagine we're going 120 m hour uh
  2168. 1:11:35in thick fog and we know somewhere
  2169. 1:11:37there's a cliff but we don't know where.
  2170. 1:11:39>> Yeah.
  2171. 1:11:40>> My first suggestion is pull over.
  2172. 1:11:42[laughter]
  2173. 1:11:42Before we argue about the speed, let's
  2174. 1:11:45first pull over. So, first criminalize,
  2175. 1:11:49you know, super intelligence and then
  2176. 1:11:50regulate the precursors. I have a list
  2177. 1:11:53of precursors such as ability to
  2178. 1:11:55self-reroduce
  2179. 1:11:57um task on horizon length things related
  2180. 1:12:00to genetic autonomy and so on and
  2181. 1:12:03monitor those very carefully and be sure
  2182. 1:12:05that any credible large experiment that
  2183. 1:12:09pushes on these frontiers is registered
  2184. 1:12:11with the government is that has
  2185. 1:12:12oversight and that we can decide no you
  2186. 1:12:14can't do that that's too risky because
  2187. 1:12:16luckily at the moment building really
  2188. 1:12:20frontier AI that pushes towards AGI or
  2189. 1:12:22ASI is unbelievably expensive. You know,
  2190. 1:12:25we're talking billions of dollars. So,
  2191. 1:12:26there's actually not that many actors
  2192. 1:12:29that this law would even affect. This
  2193. 1:12:31wouldn't affect 99% of companies. You
  2194. 1:12:33know, there would be a very small
  2195. 1:12:34percentage of mega mega huge tech
  2196. 1:12:37companies that would be affected by
  2197. 1:12:38this. And for them, yeah, we should
  2198. 1:12:40obviously have oversight of them the
  2199. 1:12:42same way we have oversight of like
  2200. 1:12:43Loheed Martin or, you know, nuclear
  2201. 1:12:45weapons facilities and so on.
  2202. 1:12:48>> What about open source AI? Open source
  2203. 1:12:50is a very tricky one in the sense that I
  2204. 1:12:53love open source. I built some of the
  2205. 1:12:56first open source large language models
  2206. 1:12:57in the world. In fact, I led a group
  2207. 1:12:59called the Luther AI. I've been I grew
  2208. 1:13:00up in the open source world. I think
  2209. 1:13:03open source has brought so much good to
  2210. 1:13:04the world. I think Linux being open
  2211. 1:13:06source and all open brought so much
  2212. 1:13:08beauty and good into the world. But for
  2213. 1:13:11me, it's not ideology. It's a question
  2214. 1:13:13of, you know, should the blueprints for
  2215. 1:13:15the F-35 fighter jet be open sourced?
  2216. 1:13:18>> Probably not. probably not. That seems
  2217. 1:13:20like it wouldn't make the world a better
  2218. 1:13:21place. You know, should blueprints for,
  2219. 1:13:23you know, bioweapons or, you know,
  2220. 1:13:26nuclear bombs be open sourced? Probably
  2221. 1:13:28not. You know, and here's a very similar
  2222. 1:13:31thing where if we get to the point that
  2223. 1:13:34there are open- source systems that are
  2224. 1:13:36super intelligent or that can be made
  2225. 1:13:38super intelligent, which I think we're
  2226. 1:13:40pretty close to, then I think there's
  2227. 1:13:42probably no going back in this regard.
  2228. 1:13:46you know, if there's like one system
  2229. 1:13:48that's almost super intelligent and it's
  2230. 1:13:50only on one server in one country, you
  2231. 1:13:52know, maybe we can delete it, you know,
  2232. 1:13:54in the maybe. But if there are open-
  2233. 1:13:57source billions of things around and we
  2234. 1:14:00know they are super intelligent or like,
  2235. 1:14:01you know, two steps away from super
  2236. 1:14:03intelligent, it's probably too late. I
  2237. 1:14:05think the action that would be needed
  2238. 1:14:06to, for example, you know, recall quote
  2239. 1:14:08unquote an open source model are not
  2240. 1:14:11feasible. M. All right. Bill Gates just
  2241. 1:14:13said that he now thinks AI is dangerous.
  2242. 1:14:17What happened? What made him change his
  2243. 1:14:18mind?
  2244. 1:14:19>> I'm not sure really how much he changed
  2245. 1:14:20his mind. Of course, I don't know him. I
  2246. 1:14:22think, you know, he's a he's a very, you
  2247. 1:14:24know, he knows a lot of things. He's a
  2248. 1:14:26smart guy. I'm sure he's seen a lot of
  2249. 1:14:27scary stuff. But even all the way back
  2250. 1:14:29in 2023, Bill Gates actually signed the
  2251. 1:14:32Center for AI safety statement which
  2252. 1:14:34said that the mitigation of the risk of
  2253. 1:14:36extinction from AI should be a global
  2254. 1:14:39priority. This was signed by Bill Gates,
  2255. 1:14:41Sam Alultman, Dario Amade, Joffrey
  2256. 1:14:44Hinton, like lots of really, you know,
  2257. 1:14:46big hitters. So, this was all the way
  2258. 1:14:48back in 2023. So, I think, you know,
  2259. 1:14:50again, I don't know him, but I think
  2260. 1:14:51Bill Gates has always known of some of
  2261. 1:14:53these risks and just has now been trying
  2262. 1:14:55to maybe he's now trying to message a
  2263. 1:14:57bit more about them. I personally wish
  2264. 1:14:59he would have gone a bit further in his
  2265. 1:15:00letter. I think it was, you know, it did
  2266. 1:15:02acknowledge some of the risk but didn't
  2267. 1:15:04go all the way to super intelligence and
  2268. 1:15:06extinction risk which is the thing that
  2269. 1:15:07I am really so concerned about because
  2270. 1:15:10in a sense I don't think we have enough
  2271. 1:15:12time that we will see massive job
  2272. 1:15:14displacement like I think we will just
  2273. 1:15:16see super intelligence in the next
  2274. 1:15:17couple of years way before massive job
  2275. 1:15:20displacement even happens and it's
  2276. 1:15:22already too late. I kind of wish we were
  2277. 1:15:24in a world you know where super
  2278. 1:15:26intelligence is 20 years away then job
  2279. 1:15:28displacement will become a huge problem.
  2280. 1:15:30But I'm not sure we're going to get to
  2281. 1:15:32that point even.
  2282. 1:15:33>> That's wild. That's wild. Okay. Um I
  2283. 1:15:36want to see if I got your base
  2284. 1:15:37assumptions right. If I've missed
  2285. 1:15:39anything, let me know. But this has been
  2286. 1:15:41very helpful. Uh these are loosely in
  2287. 1:15:44order, but um and there's one that I
  2288. 1:15:46have a follow-up question on. Okay, so
  2289. 1:15:49super intelligence is definitionally you
  2290. 1:15:52you um maybe didn't go quite that far,
  2291. 1:15:54but super intelligence is adversarial.
  2292. 1:15:56So knowing that once you make that it is
  2293. 1:15:59going to compete and you're going to
  2294. 1:16:00have problems therefore uh super
  2295. 1:16:02intelligence will consider us a
  2296. 1:16:04distraction or impediment to their goals
  2297. 1:16:06and so we have to worry about being the
  2298. 1:16:08very thing they're competing against and
  2299. 1:16:10we will lose. Um unknown risk is
  2300. 1:16:13evaluated at near peak danger levels
  2301. 1:16:16meaning if I don't know the outcome of
  2302. 1:16:18this like you said first pull over if we
  2303. 1:16:20don't know where we're going. So anytime
  2304. 1:16:22we terminate in I don't know what AI is
  2305. 1:16:24going to do or I don't know what the
  2306. 1:16:25super intelligence is going to do here.
  2307. 1:16:26Pump the brakes.
  2308. 1:16:27>> Obviously there's spectrum here. Um as
  2309. 1:16:30with any complex strategy. How do you
  2310. 1:16:31deal with known unknowns or unknown
  2311. 1:16:33unknowns? How do you calculate the
  2312. 1:16:36probability that a country is going to
  2313. 1:16:37invade you tomorrow? This is actually a
  2314. 1:16:39problem there's a lot of literature on.
  2315. 1:16:40This is a thing that you know the CIA
  2316. 1:16:42and you know various military agencies
  2317. 1:16:44have spent many many decades working
  2318. 1:16:46very very hard on. One of the
  2319. 1:16:47recommendations, for example, the CIA
  2320. 1:16:49makes is that always word your gut
  2321. 1:16:51feelings in numbers. So, this is a thing
  2322. 1:16:53we didn't talk about. People often get
  2323. 1:16:55annoyed when people say like, "Oh, I
  2324. 1:16:56think there's a 20% chance of things
  2325. 1:16:58going well." You're like, "That's not
  2326. 1:16:59math." And I'm like, "Dude, [laughter]
  2327. 1:17:02yeah, they know. I know it's not math."
  2328. 1:17:05The reason you say, "Oh, 20%." is not
  2329. 1:17:07because I'm trying to make a
  2330. 1:17:08mathematical statement. It's because
  2331. 1:17:10there's the famous study, I think it was
  2332. 1:17:12from the CIA, where they found that
  2333. 1:17:14analysts who said likely or unlikely
  2334. 1:17:17actually meant wildly different things.
  2335. 1:17:19Like some people when they said likely,
  2336. 1:17:20they meant 90%. Other analysts when they
  2337. 1:17:23said likely meant 30%.
  2338. 1:17:25>> And so this was extremely confusing
  2339. 1:17:27because unlikely likely doesn't mean
  2340. 1:17:29anything. So if you say like 90% it's
  2341. 1:17:32still a vibe, but it's a much more
  2342. 1:17:33concrete vibe, right? It's a much more
  2343. 1:17:35when I'm communicating to you. If I say
  2344. 1:17:371%, I'm trying to communicate not oh
  2345. 1:17:40it's definitely 1% and not 1.2%. That's
  2346. 1:17:42not what I'm saying. What I'm saying is
  2347. 1:17:43it's on the order of 1%. It's like given
  2348. 1:17:46my beliefs about the world and you know
  2349. 1:17:48looking at my models of reality which
  2350. 1:17:50you know you can evaluate to how much
  2351. 1:17:52you believe my models of reality
  2352. 1:17:55this outcome has on the order of this
  2353. 1:17:57percentage of outcomes. Of course it's
  2354. 1:17:59based on my models. What else is it
  2355. 1:18:01supposed to be based on? Right? Like
  2356. 1:18:03obviously you have views of the world.
  2357. 1:18:05You know if I asked you how likely do
  2358. 1:18:06you think it is that you know this thing
  2359. 1:18:08will happen in the future how do you
  2360. 1:18:10evaluate this well you use your model in
  2361. 1:18:12your head what else are you going to do
  2362. 1:18:14you know so it's a very similar thing
  2363. 1:18:16like I try to make my model explicit
  2364. 1:18:17like with these base assumptions I try
  2365. 1:18:19to make it pretty explicit this leads to
  2366. 1:18:20this leads to this leads to this
  2367. 1:18:22therefore high probability so if all of
  2368. 1:18:25these statements that we've said so far
  2369. 1:18:27were like you know a billionth of a
  2370. 1:18:30percent probable I wouldn't be worried
  2371. 1:18:32but I don't think that's the case I
  2372. 1:18:33think if they're all like you know
  2373. 1:18:35higher than 1% and after their
  2374. 1:18:37conclusion are still more than you know
  2375. 1:18:39after multiplying them all together
  2376. 1:18:40we're still above 1% and the outcome is
  2377. 1:18:43extinction hell no you know if the
  2378. 1:18:45outcome is like oh it's a one in a
  2379. 1:18:46trillionth I'm like sure but that's not
  2380. 1:18:48where my numbers come out
  2381. 1:18:49>> right makes sense okay um financial
  2382. 1:18:53incentives power incentives and lack of
  2383. 1:18:55strong regulation have allowed us to
  2384. 1:18:58keep racing forward despite the dangers
  2385. 1:19:00>> yep
  2386. 1:19:01>> uh for AI to work it requires the AI to
  2387. 1:19:04have a will of its own.
  2388. 1:19:06>> Depends on what you mean by will. I
  2389. 1:19:07think it's
  2390. 1:19:08>> it's going to be an optimizer. Now that
  2391. 1:19:10I know the language you use,
  2392. 1:19:11>> this is how I would describe it. It will
  2393. 1:19:13have to optimize for goals of some kind.
  2394. 1:19:15It will take actions. It will make
  2395. 1:19:16plans. It would it will optimize towards
  2396. 1:19:19objectives.
  2397. 1:19:19>> And the sociopathic optimizer, as you
  2398. 1:19:21described it earlier, is the problem.
  2399. 1:19:24>> Yes.
  2400. 1:19:24>> Okay. AI,
  2401. 1:19:25>> there's some other problems, too, but
  2402. 1:19:26this is really quite the heart of it.
  2403. 1:19:28>> Okay. AI. It's too complex to imbue AI
  2404. 1:19:32with morality. Yep. We have no idea how
  2405. 1:19:33to do it.
  2406. 1:19:34>> Okay. Uh, life can be improved by things
  2407. 1:19:36other than technology.
  2408. 1:19:38>> Yep.
  2409. 1:19:38>> Uh, it's possible to get China on board.
  2410. 1:19:41>> Yep.
  2411. 1:19:42>> And humans are not magical. This is the
  2412. 1:19:44one that I need more clarification on.
  2413. 1:19:46It really hit me when you said it, but I
  2414. 1:19:48wanted to better understand. You were
  2415. 1:19:50trying to explain something about AI by
  2416. 1:19:52saying humans are magical. You need to
  2417. 1:19:53understand that.
  2418. 1:19:54>> What What do you mean when you say
  2419. 1:19:56humans aren't magical, and what does
  2420. 1:19:57that tell us about AI? There is a very
  2421. 1:20:00fun um thing as a Wikipedia article
  2422. 1:20:02called the AI effect. I I recommend
  2423. 1:20:05looking up the Wikipedia article. It's
  2424. 1:20:07quite fun. Which is that throughout the
  2425. 1:20:09history of AI, people would always
  2426. 1:20:11define like, oh, we'll know we have real
  2427. 1:20:13AI when X. And then once X was
  2428. 1:20:15accomplished, everyone like, oh no,
  2429. 1:20:16that's not real AI. That's not true
  2430. 1:20:18intelligence. So, for example, back in
  2431. 1:20:19the day, people thought that the thing
  2432. 1:20:21that makes humans unique, different from
  2433. 1:20:23all animals, is that we can do math.
  2434. 1:20:26And well obviously computers can do math
  2435. 1:20:28>> very well.
  2436. 1:20:29>> So very well. So so then they're like
  2437. 1:20:31okay no no no that's not what we mean.
  2438. 1:20:32What we mean is actually the thing that
  2439. 1:20:34makes humans magical is that we can
  2440. 1:20:36recognize images or whatever you know.
  2441. 1:20:38And then you're like or they can play
  2442. 1:20:40chess or you know and obviously none of
  2443. 1:20:42those things were the thing. And so
  2444. 1:20:45there's a there's like this like rising
  2445. 1:20:46tide rising waterline or or like you
  2446. 1:20:49know like being packed into a corner of
  2447. 1:20:50like this this very funny cartoon. And I
  2448. 1:20:52think it was maybe from Ray Curtzwell or
  2449. 1:20:53something where you have this guy
  2450. 1:20:55frantically writing paper like only
  2451. 1:20:56humans can do only humans can play chess
  2452. 1:20:58and like strikes it out like only humans
  2453. 1:21:00can draw pictures strikes that out and
  2454. 1:21:01like and like his whole room is covered
  2455. 1:21:03with these paintings of him and like
  2456. 1:21:05only humans can do uh uh uh you know and
  2457. 1:21:08what I'm saying about this is that
  2458. 1:21:09humans are physics. We are in fact part
  2459. 1:21:12of physics. There's nothing magical
  2460. 1:21:14going on. It's complex. Our brains are
  2461. 1:21:17complex but it's not magic. It's it's a
  2462. 1:21:19it's a physical process. There is some
  2463. 1:21:21computation happening in our brain of
  2464. 1:21:24some kind. You know, it's being, you
  2465. 1:21:26know, it's using some weird squishy, you
  2466. 1:21:28know, elements for the calculations. But
  2467. 1:21:29fundamentally, some kind of calculation
  2468. 1:21:31is going on. It's not magic. You can
  2469. 1:21:32make other things do similar
  2470. 1:21:34calculations.
  2471. 1:21:35>> So, it's a way of explaining we know
  2472. 1:21:37intelligence is possible without some
  2473. 1:21:39sort of magical element.
  2474. 1:21:40>> Exactly.
  2475. 1:21:40>> It makes all the sense in the world,
  2476. 1:21:42dude. This is crazy. Um, this has really
  2477. 1:21:45been amazing researching you and sitting
  2478. 1:21:47across from you. I really, really
  2479. 1:21:48appreciate I think your messages come in
  2480. 1:21:50loud and clear. If there was one thing
  2481. 1:21:53that um people may not realize, like if
  2482. 1:21:55you were sitting across from Xi or
  2483. 1:21:58somebody who you're like, I know they're
  2484. 1:22:00convincible, is there one final thing
  2485. 1:22:02that hasn't come up today that you'd
  2486. 1:22:03want them to be aware of?
  2487. 1:22:04>> I have different answers for Xiinping
  2488. 1:22:07than I do for people listening. And so
  2489. 1:22:08I'll give two answers. And so honestly,
  2490. 1:22:11the number one thing I have found is the
  2491. 1:22:14grown not written. People don't know
  2492. 1:22:15this. when I talk to politicians is the
  2493. 1:22:17first thing basically out of my mouth is
  2494. 1:22:19explaining that this is not normal
  2495. 1:22:21software. It is not written. It is
  2496. 1:22:22grown. We do not understand how these
  2497. 1:22:24things work. We did bring this up but
  2498. 1:22:26I'm repeating it again because this is
  2499. 1:22:27just so important to understand. The
  2500. 1:22:29people at open do not know what is going
  2501. 1:22:32on inside of their AIS.
  2502. 1:22:33>> Dario Amade, the CEO of Anthropic thinks
  2503. 1:22:35he we understand maybe 3%
  2504. 1:22:38>> oh my god
  2505. 1:22:38>> of what goes on in our AIS. And I think
  2506. 1:22:41that's even optimistic.
  2507. 1:22:42>> Whoa. So this is one of the most
  2508. 1:22:44important things to understand. We don't
  2509. 1:22:46know. We they don't know. No one knows.
  2510. 1:22:49This is very important. And for the
  2511. 1:22:51general public and everyone listening, I
  2512. 1:22:53think the very important thing is is
  2513. 1:22:54that it's not over. And even though it's
  2514. 1:22:57sometimes unfashionable say to say this
  2515. 1:22:59these days, we do still live in a
  2516. 1:23:00democracy. And your voice is important.
  2517. 1:23:04If you want something to happen, the way
  2518. 1:23:06I like to think about civics is that our
  2519. 1:23:09job is to work with our politicians to
  2520. 1:23:11help them understand what we care about
  2521. 1:23:13and what can be done. So do your part to
  2522. 1:23:16help m bring these issues to awareness.
  2523. 1:23:18Go to controlai.org. Contact your
  2524. 1:23:20lawmaker. Tell them you're concerned
  2525. 1:23:21about these issues. Help educate people.
  2526. 1:23:24Keep having the conversation. This is
  2527. 1:23:26how we humans solve all these problems,
  2528. 1:23:28you know, together.
  2529. 1:23:29>> I love it. Where can people follow you?
  2530. 1:23:32Um, I'm you can find me on Twitter, you
  2531. 1:23:35know, and now X, sorry, uh, at MPoll
  2532. 1:23:37Collapse. You can find me obviously
  2533. 1:23:39controlai.org. And also, if someone
  2534. 1:23:41wants to engage even deeper on some of
  2535. 1:23:43these things, I actually run a volunteer
  2536. 1:23:45group called Torchbearer,
  2537. 1:23:47torchbearer.com community, where we
  2538. 1:23:49commit to spending at least two hours a
  2539. 1:23:50week to try to build that reasonable
  2540. 1:23:52world.
  2541. 1:23:53>> I love it. All right, everybody. If you
  2542. 1:23:55haven't already, be sure to subscribe.
  2543. 1:23:57And until next time, my friends, be
  2544. 1:23:59legendary. Take care. Peace. If you like
  2545. 1:24:01this conversation, check out this
  2546. 1:24:03episode to learn more.
  2547. 1:24:05>> AI is easily the greatest discovery
  2548. 1:24:08humanity will ever make. This is not an
  2549. 1:24:11easy game to win or to start, but this
  2550. 1:24:14is not going to be stopped.
  2551. 1:24:15>> AI is sucking up all of the liquidity
  2552. 1:24:18and it's causing a problem in crypto.
  2553. 1:24:20And now.

About this transcript

This page contains the full transcript of EMERGENCY PODCAST: ASI Will Kill Us All! by Tom Bilyeu, generated from the public captions YouTube serves with the video. The transcript has 17,637 words across 2,553 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.