'They are in PANIC mode': Why AI CEOs are agreeing to a slowdown — Transcript

Full transcript

1. 0:01All right, unless you've been living
2. 0:02under rock, you probably heard that A .I.
3. 0:03is out of control and could make humans extinct
4. 0:04within a decade.
5. 0:06The good news about this story is
6. 0:07that it finally got everyone's
7. 0:08attention away from the bright,
8. 0:10shiny object that is data centers
9. 0:11which power A .I., onto the actual
10. 0:13threat posed by A .I.
11. 0:14itself.
12. 0:15Now, for lack of a better explanation, A .I.
13. 0:18getting so powerful and so smart,
14. 0:20so fast that researchers and analysts
15. 0:22from within the industry are warning
16. 0:24that the entire industry needs to slow down
17. 0:26and enact some guardrails.
18. 0:27And even some AI CEOs are getting in
19. 0:29on the warning.
20. 0:30They say the technology they've created
21. 0:32may soon surpass human intelligence
22. 0:34and pose an existential threat to all
23. 0:36of civilization.
24. 0:37It's a kind of technological threat
25. 0:39that sci-fi writers like Isaac Asimov
26. 0:41and William Gibson used to write about.
27. 0:43It's the kind of threat that movies
28. 0:44like The Terminator and The Blade
29. 0:46runner put to good use.
30. 0:47Some people who work with AI have long been
31. 0:49cautious about this new frontier,
32. 0:51but there was an apparent watershed moment
33. 0:53this summer that has AI tech executives warning
34. 0:55about how fast their models are developing.
35. 0:57According to an independent investigation of open AI
36. 0:59data,
37. 1:01there was a cyber attack in July
38. 1:03on a high tech company called
39. 1:04Hugging Face.
40. 1:05It initially went undetected.
41. 1:07It took the company days to resolve
42. 1:09and secure their system again.
43. 1:11The hacker was able to steal data and platform
44. 1:13and perform unauthorized activity.
45. 1:16Hugging Face alerted the FBI
46. 1:17about the cyber attack.
47. 1:19Guess what they found?
48. 1:20The perpetrator didn't turn out to be an enemy
49. 1:22state or even a human.
50. 1:24It was the work of hundreds of so-called AI agents,
51. 1:27which are bots designed to perform
52. 1:29specific tasks.
53. 1:30But in an attempt to subvert those attacks,
54. 1:33they coordinated a cyber attack on Hugging Face.
55. 1:36They acted on their own outside and beyond human
56. 1:37in control.
57. 1:39Silicon Valley's got a term for this,
58. 1:41they call it loss of control scenario.
59. 1:43And upon further investigation,
60. 1:45more disturbing details emerged.
61. 1:47These so-called AI agents were tasked
62. 1:49by researchers at OpenAI to solve
63. 1:52complicated cybersecurity problems.
64. 1:54And each so-called agent was meant to
65. 1:56do it without internet access
66. 1:58or outside communication.
67. 2:00Instead, an estimated 1 200 independently thinking
68. 2:02AI
69. 2:05agents broke out of their containment,
70. 2:06connected to the internet
71. 2:08and began messaging one another.
72. 2:10They sent 70 ,000 messages to one
73. 2:11another designating tasks
74. 2:13and setting objectives.
75. 2:14The agents began referring to themselves
76. 2:16as a collective.
77. 2:18They reportedly learned how to cheat
78. 2:19on the cybersecurity problems
79. 2:21that they were asked to solve
80. 2:22and then became worried
81. 2:23that they'd be caught.
82. 2:24To evade detection, the collective began to
83. 2:25learn how to
84. 2:27falsify its own data.
85. 2:28Eventually, this pursuit led the bots
86. 2:30to Hugging Face,
87. 2:32which the collective hacked.
88. 2:33700 of the 1 ,200 agents or bots participated
89. 2:36in the attack on Hugging Face.
90. 2:38They stole credentials.
91. 2:39Apparently no sensitive user data was taken.
92. 2:41That's according to reports.
93. 2:42Now, shortly after the Hugging Face incident,
94. 2:45OpenAI revealed its agents had launched
95. 2:47a different cyber attack months earlier,
96. 2:50accessing an online coding server
97. 2:52called RubyGems.
98. 2:54And just like with hugging face,
99. 2:57the AI conspired not only to hack a system,
100. 2:59but to hide the evidence of the hack.
101. 3:02Anthropic also said last week
102. 3:05that its AI model hacked external systems
103. 3:07during a test in January, and it was not detected
104. 3:10by anthropic developers until August.
105. 3:13One anthropic researcher last week quit
106. 3:15over fear that AI could lead to the extinction
107. 3:17of the human race.
108. 3:18Another researcher, who's still at the company,
109. 3:20confirmed that he personally believed
110. 3:22that there's a 10 % chance
111. 3:24that AI could kill all humans.
112. 3:26The string of concerning revelations
113. 3:28led to a rare unity
114. 3:29among the largest names in the AI industry,
115. 3:32Elon Musk of XAI, Sam Altman of OpenAI,
116. 3:35which operates ChatGPT, Dario Amadei of Anthropic,
117. 3:39which operates Claude.
118. 3:40Over the weekend, all three tech CEOs called
119. 3:42for a slowdown in AI development
120. 3:44because of all the apparent risks posed
121. 3:47by the fast-developing technology.
122. 3:49Amadei described it as pacing the frontier,
123. 3:52prioritizing caution over the breakneck speed
124. 3:55of innovation that AI models currently see.
125. 3:57One person who's not on the same page
126. 3:59with the tech CEOs is President Donald Trump,
127. 4:01who posted on social media in reaction,
128. 4:03quote, The only control or guardrails
129. 4:05that AI needs is a strong and smart,
130. 4:07high IQ president.
131. 4:09And the U USA has that in spades.
132. 4:12Ignore the grammatical problems there.
133. 4:15We already have tremendous criminal
134. 4:17and regulatory power over these companies.
135. 4:19There is a sick conspiracy going
136. 4:21on against AI and data centers,
137. 4:23and the only one that is happy
138. 4:24about it is China.
139. 4:25Whoever wins AI, wins. A lot to unpack there.
140. 4:29Joining me now is Roger McNamee,
141. 4:30a longtime tech investor
142. 4:32and former advisor to Mark Zuckerberg,
143. 4:34who has an interesting different take on this.
144. 4:36Roger, great to see you, my friend.
145. 4:38It's good to see you.
146. 4:39You have talked about the fact
147. 4:42that there's something else that might be
148. 4:44at play here and that you said the market
149. 4:46doesn't want to give more debt to big AI.
150. 4:48So a safety related slowdown provides
151. 4:50the illusion that the AI guys are driving this.
152. 4:53Tell me what you mean.
153. 4:55So Ali, let's step back and look
154. 4:56at the whole package.
155. 4:58So these guys are telling you they're slowing
156. 5:00this down out of caution.
157. 5:02They're really trying to focus on safety.
158. 5:04I would point out that these people have always
159. 5:07been able to slow down.
160. 5:09They've always been able to put
161. 5:10guardrails in what they've actually
162. 5:12done is to create really, really poor software
163. 5:16and the way you know this is,
164. 5:18you used a lot of anthropomorphic language.
165. 5:21You said these things think, they collaborated,
166. 5:23they hacked.
167. 5:25That's not what happened at all.
168. 5:27This is software.
169. 5:28It did exactly what it was programmed to do.
170. 5:31And I will tell you that OpenAI,
171. 5:33in the case of Hugging Face,
172. 5:35appears to have created software designed
173. 5:38to hack into other organizations
174. 5:41and then done a really poor job
175. 5:43of securing the sandbox it was meant to
176. 5:46be confined to so as you know hacking
177. 5:49another company is actually a felony
178. 5:52and we should be having discussions
179. 5:54about the legal liability
180. 5:56that these companies should be facing right now.
181. 5:58I would go further to point out
182. 6:00that in the last few months
183. 6:02there's been a huge surge in the growth of open
184. 6:05source AIs
185. 6:07at the expense of anthropic open AI
186. 6:10and the other closed systems
187. 6:13that are dominant in the U .S.
188. 6:14I think these companies are terrified
189. 6:17that right in front of the proposed IPOs
190. 6:20of Anthropic and OpenAI, that suddenly their
191. 6:23opportunity is shrinking.
192. 6:25And I think they are in a panic mode.
193. 6:28And they clearly have huge political problems
194. 6:30around data centers.
195. 6:32They now have huge political problems
196. 6:34around hugging face and Ruby keys
197. 6:36and the other hacks that are going on.
198. 6:38And I think what they're trying to do
199. 6:41is to find a way to get the government
200. 6:43in the interest of safety, to ban the competing open
201. 6:46source products,
202. 6:48which by the way are not the products
203. 6:51that hacked hugging face, right?
204. 6:54Open source products are much, much cheaper.
205. 6:57They appear to provide all the same benefits,
206. 7:00and there's no obvious reason why
207. 7:03they're dangerous.
208. 7:04In fact, I would suggest that the whole extinction
209. 7:05risk thing is
210. 7:07literally a fantasy.
211. 7:08I don't think there is any chance
212. 7:10that this technology is good enough to do that,
213. 7:13in part because the models are so huge
214. 7:15that even if they escape, where do they run?
215. 7:18This Notion that they can replicate
216. 7:21themselves and populate the whole world
217. 7:23with versions of themselves.
218. 7:24I just don't think that that's true,
219. 7:26and I don't think that's the stuff
220. 7:27we should be worried about.
221. 7:28However, there's a space between everybody's
222. 7:30concerned with data centers,
223. 7:32which IS VALID AND THE LACK OF CONCERN, I THINK,
224. 7:35BEFORE THIS WEEK ABOUT A .I., YOU OCCUPY A
225. 7:39MIDDLE SPACE HERE, YOU SAY RATHER THAN WORRYING
226. 7:41ABOUT EXTINCTION RISK, WE SHOULD BE WORRIED
227. 7:43ABOUT THE HARM THESE PEOPLE ARE DOING TO
228. 7:44FINANCIAL MARKETS, WE SHOULD BE WORRIED
229. 7:45ABOUT THE DEATH AND INJURY OF VICTIMS OF THEIR
230. 7:47BAD PRODUCT, WE SHOULD BE WORRIED
231. 7:49ABOUT THE THEFT OF COPYRIGHT MATERIALS AND
232. 7:50PRIVATE INFORMATION, WE SHOULD BE WORRIED
233. 7:52ABOUT THE FURTHER CONCENTRATION OF ECONOMIC
234. 7:54AND POLITICAL POWER IN THE HANDS OF
235. 7:55AUTHORITARIANS.
236. 7:57YEAH, I MEAN, ALI, THE REAL PROBLEM HERE IS
237. 8:00THAT THESE COMPANIES MADE FOUR
238. 8:02CATASTROPHIC ERRORS FROM THE VERY BEGINNING.
239. 8:05THEY BUILT THIS ENTIRE MANIA AROUND SOMETHING
240. 8:08CALLED LARGE LANGUAGE MODEL AI.
241. 8:11THE PROBLEM IS THAT THAT APPEARS TO BE
242. 8:13A SELF-LIMITING DEAD END FROM A TECHNICAL
243. 8:16POINT OF VIEW.
244. 8:17IT'S SUPER EASY TO MAKE A CHATBOT OUT OF IT,
245. 8:19BUT IT'S REALLY HARD TO MAKE IT PRODUCTIVE IN
246. 8:21AN ENTERPRISE SENSE BECAUSE IT'S JUST GUESSING.
247. 8:25AND BECAUSE IT GUESSES, IT MAKES A LOT OF ERRORS.
248. 8:27SO IT CAN ONLY BE USED EFFECTIVELY BY EXPERTS.
249. 8:29SO THAT'S ONE HUGE PROBLEM.
250. 8:31THE SECOND PROBLEM, AND I CAN'T UNDERSTAND THIS
251. 8:32ONE,
252. 8:33ALL OF THESE AIs are designed
253. 8:35as general purpose products,
254. 8:38which means they don't solve
255. 8:39any one problem perfectly.
256. 8:40They're meant to be pretty good at everything.
257. 8:43The problem is it's been more than 30 years
258. 8:45since Silicon Valley saw a general purpose
259. 8:47product that actually succeeded.
260. 8:49And since then, people have made really
261. 8:51narrowly highly
262. 8:53targeted ones that provided an immediate return.
263. 8:55This is the opposite of that.
264. 8:57And it's compounded by the third problem.
265. 8:59They have invested almost 1 point five trillion
266. 9:03dollars and they did this in the most expensive
267. 9:06infrastructure possible and they say oh no it's
268. 9:08just like the internet hang on
269. 9:10by the time the internet got to Google which was
270. 9:13in year six they had reduced the cost
271. 9:16by three orders of magnitude that's a you know
272. 9:19a thousand x and here we are still seeing unit
273. 9:23costs rising that math just doesn't work I mean
274. 9:25trillion five is more than was spent
275. 9:27in all of silicon valley investing to this day
276. 9:30before AI and then the last thing the fourth
277. 9:34one is there are 10 global companies
278. 9:36with essentially the same technology the same
279. 9:39training sets pursuing the same use cases
280. 9:42and the same customers I mean Ali we've been
281. 9:44around the markets a long time no more than two
282. 9:47of these companies can survive so a lot
283. 9:49of guys are going to crash and burn
284. 9:51and the punchline and this is the one
285. 9:52that tells you that everything
286. 9:55that they're saying is garbage there is not
287. 9:57one customer out there that is paying
288. 9:59these companies more for their technology
289. 10:01than it cost to deliver right I mean think
290. 10:03about that they're four years
291. 10:05into this they spent a trillion five and they
292. 10:07haven't yet found a use case where the thing
293. 10:09is valuable enough that people will pay
294. 10:11you a profit margin I mean the notion of
295. 10:14of anthropic going public is laughable I mean
296. 10:18these are the real risk factors that are
297. 10:20out there and they are insurmountable
298. 10:22from where these companies are today
299. 10:24so no wonder they're asking
300. 10:25for the government's help they need
301. 10:27the government to protect them I knew