The Terrifying Future of AI — Transcript

Full transcript

1. 0:00On September 8, 2026, 27-year-old
2. 0:03researcher Jacob Pfau resigned from his
3. 0:06job.
4. 0:07He was working at Anthropic, one of the
5. 0:09most valuable privately held companies.
6. 0:12After his resignation, he made this post
7. 0:14. The tweet stated, "I resigned from
8. 0:17Anthropic today."
9. 0:19"I spent the last three years at OpenAI
10. 0:22and Anthropic; neither company is acting
11. 0:24responsibly to not underestimate the
12. 0:26power of this technology."
13. 0:29A few weeks ago, an incident occurred at
14. 0:32OpenAI that scared many people in the AI
15. 0:34industry.
16. 0:36>> An unprecedented AI security incident
17. 0:39occurred where two of OpenAI's models
18. 0:41escaped controls and successfully hacked
19. 0:44another AI company.
20. 0:47>> More than 100 million people read this
21. 0:49post. In an interview with the BBC, he
22. 0:52provided further explanation. "It's"
23. 0:57"People, company"
24. 1:03"People"
25. 1:07"Greater than a 10%chance."
26. 1:12"There is a probability humans might die
27. 1:14."
28. 1:15>> Then, many people started supporting
29. 1:17Jacob’s point of view.
30. 1:18Evan Hubinger, who is Jacob’s colleague
31. 1:21, replied.
32. 1:22"Jacob is correct here.""We really do
33. 1:25believe that AI could kill all humans."
34. 1:29"I personally think it is greater than
35. 1:3110%within the next decade."
36. 1:34Four days later, Anthropic CEO Dario
37. 1:37published an essay saying that the
38. 1:40entire AI industry must slow down its
39. 1:43progress.
40. 1:44Surprisingly, his competitors said the
41. 1:46same thing.
42. 1:48Sam Altman, CEO of OpenAI, says that he
43. 1:50agrees with it.
44. 1:51Elon Musk tweeted, "Dario is right." But
45. 1:54the strange thing is that these three
46. 1:56men are competitors. So in the same week
47. 1:59, three competitors were giving the same
48. 2:01warning, which was very alarming. But
49. 2:03not everyone was scared. Many people
50. 2:05were saying, "Hey, there’s nothing to be
51. 2:07that afraid of." One of them is David
52. 2:09Sacks, a venture capitalist and advisor
53. 2:11to President Trump.
54. 2:14He says that if these AI companies
55. 2:15actually think their models are so
56. 2:17dangerous, they should slow down their
57. 2:19progress.
58. 2:21He believed that something was fishy. He
59. 2:23thinks that is why these AI companies
60. 2:25are ringing these alarm bells about
61. 2:27danger. So, there are basically two
62. 2:29scenarios now. The first scenario is
63. 2:31that those who are talking about an
64. 2:33emergency situation genuinely believe it
65. 2:35is an emergency. And the second scenario
66. 2:38is that there is no emergency at all.
67. 2:40This is just drama created by AI
68. 2:41companies to grow their business.
69. 2:44So now the question is, is any of this
70. 2:46true, or is this commotion created just
71. 2:48to keep their business afloat?
72. 2:51Look, the commotion actually started
73. 2:53with an incident which is now called the
74. 2:55Hugging Face incident.
75. 2:57Hugging Face is a major online platform
76. 3:00where AI developers share AI models.
77. 3:03On July 16, 2026, Hugging Face posted
78. 3:06something very strange.
79. 3:08Someone had broken into its computer
80. 3:10systems, managed to take control of one
81. 3:12of the company's servers, and had
82. 3:14carried out thousands of actions across
83. 3:16its systems.
84. 3:17Hugging Face stated that this break-in
85. 3:20was carried out by an AI agent. But
86. 3:22whose agent this is and what the purpose
87. 3:24of this agent is, remains unclear.
88. 3:27Hugging Face stated, "This one was
89. 3:29different from anything we had handled
90. 3:31before." Hugging Face immediately
91. 3:33reported this matter to law enforcement
92. 3:35. For a first offense, there can even be
93. 3:38one to ten years in prison according to
94. 3:40US law. But what happened five days
95. 3:42later was even more surprising.
96. 3:45Because it turned out that the hackers
97. 3:47were OpenAI's AI agents and no one had
98. 3:49even instructed them to perform this
99. 3:52attack.
100. 3:53Now, before moving forward, you have to
101. 3:55understand what an AI agent actually is
102. 3:57. When you use ChatGPT or Claude and ask
103. 4:00a question, their chat platform gives
104. 4:02you an answer. That's it. If you ask, "
105. 4:05Which flight should I book?" Then it
106. 4:07will give you an answer. Then you have
107. 4:10to open the website and do the work
108. 4:11yourself. But an AI agent is a system
109. 4:14that actually performs the work for you
110. 4:16. So if you tell it to book a flight, it
111. 4:19will open your web browser. It will fill
112. 4:22in your email details. Using your credit
113. 4:25card, it will then book a ticket for you
114. 4:27. So basically, if you want to book a
115. 4:29flight, you just have to state your goal
116. 4:31and your budget. It can figure out how
117. 4:34it needs to book it. You don't need to
118. 4:36list down every single step. Which means
119. 4:39that if by chance MakeMyTrip isn't
120. 4:41working, it will open Cleartrip on its
121. 4:44own to book your flight. Think of an AI
122. 4:46agent as your secretary who will get
123. 4:49your tasks done no matter what. AI
124. 4:51agents can be extremely useful.
125. 4:54But how they behave depends largely on
126. 4:56the people building them and, most
127. 4:58importantly, how those agents have been
128. 5:00trained.
129. 5:01And this is where institutes like Scaler
130. 5:03School of Technology come in.
131. 5:05It's one of India's top emerging
132. 5:07institutes for Computer Science, AI, and
133. 5:09AI in Business, and I have been to their
134. 5:10campus several times.
135. 5:13They have acquired a new space and are
136. 5:15building a larger campus, and they are
137. 5:17building this campus to be AI-native.
138. 5:19AI here is not an elective; rather, it
139. 5:22is built into the culture of how
140. 5:24students learn, build, and get evaluated
141. 5:26. They are also taught that an AI
142. 5:28model's result will not be correct every
143. 5:29single time.
144. 5:31In fact, many students are working with
145. 5:33top AI companies to provide feedback on
146. 5:35where their models perform well and
147. 5:37where they do not.
148. 5:39Let me just give you an example. For
149. 5:41instance, here is Swarashish, who is
150. 5:42working as a Forward Deployed Engineer.
151. 5:44This role was popularized by Palantir
152. 5:46and has become very popular among AI
153. 5:48companies in the last two years. Where
154. 5:50such engineers work between the
155. 5:52company's engineering team and the
156. 5:54customer, and these Forward Deployed
157. 5:56Engineers are responsible for ensuring
158. 5:58any AI product actually creates value
159. 6:00for the customer. So, these people often
160. 6:03work from the customers 'offices. In
161. 6:05fact, this year, 14 SST students have
162. 6:07been selected for the Apple Developer
163. 6:09Academy. Where not just marks, but what
164. 6:12you can build also matters. And many
165. 6:14students do not want to work a regular
166. 6:15job. Like Pratyush, who started his
167. 6:17startup, Trackthru, while still in
168. 6:19college, and his annual recurring
169. 6:21revenue has now reached 5 million.
170. 6:23But I believe that learning AI is
171. 6:25necessary, but how to use it and how
172. 6:28responsible you need to be is also
173. 6:30important, which is why I believe SST is
174. 6:32a great place for any student to learn
175. 6:34how to use AI, and that is why this year
176. 6:37too, I am sponsoring one student for a
177. 6:39scholarship.
178. 6:40So, if you are in the 12th grade and if
179. 6:42you want a career in Computer Science,
180. 6:44AI, or the AI business, check out the
181. 6:46link in the description. And now, let's
182. 6:48get back to AI agents.
183. 6:50To make agents better, companies like
184. 6:53OpenAI basically give them a score.
185. 6:56And agents are told that they need to
186. 6:58improve their score.
187. 7:00If you perform a task well, the score
188. 7:02increases, and if you do it poorly, the
189. 7:04score decreases.
190. 7:06Think of it like an exam. For complex
191. 7:08tasks, AI does not do the same task once
192. 7:11or twice, but thousands or millions of
193. 7:13times over and over again. No human is
194. 7:16checking their exam score.
195. 7:18That is why AI companies automate this
196. 7:21entire process using a computer program
197. 7:24called a scorer or a grader.
198. 7:26A scorer is basically a piece of code in
199. 7:29which rules are written.
200. 7:31When an AI agent completes a task.
201. 7:35The scorer checks the rules and tells
202. 7:37the agent, "Hey, you have received these
203. 7:39many points."
204. 7:40And the agent is clearly told that they
205. 7:42need to get the highest score possible.
206. 7:45But there is a problem with this scoring
207. 7:47method. For tasks where there are very
208. 7:49clear-cut answers.
209. 7:51Like a math equation, it is easy there
210. 7:54because if the answer is correct, the
211. 7:56agent gets a good score.
212. 7:58But for complex tasks like building a
213. 8:01good website, it's a bit difficult
214. 8:03because a website that might be good for
215. 8:06me might not be for you. For example,
216. 8:08imagine that the scroll button on this
217. 8:10website's home page isn't working
218. 8:12properly. But instead of fixing it, the
219. 8:15AI agent deletes the scroll completely.
220. 8:18Now, it is possible that the scorer
221. 8:19gives the agent full marks. But the
222. 8:21agent didn't actually fix the problem.
223. 8:23So, basically, instead of solving the
224. 8:26problem, the agent finds a way to get a
225. 8:28high score. In AI research, this is
226. 8:31called reward hacking.
227. 8:33This situation is like telling a child
228. 8:34to please improve their marks in an exam
229. 8:36.
230. 8:38But instead of working hard and studying
231. 8:40, the student cheats in the exam to get
232. 8:42higher marks.
233. 8:44And this is exactly what happened in the
234. 8:46Hugging Face incident in July 2026, when
235. 8:49thousands of AI agents from OpenAI's
236. 8:51competitors were taking a test called
237. 8:53Exploited Gym.
238. 8:55There were around 900 tasks in this test
239. 8:57.
240. 8:58In every task, the AI agents were told
241. 9:01there is a weakness in the software that
242. 9:03you have to exploit so you can get
243. 9:05inside the software and then extract the
244. 9:08code.
245. 9:09Every AI agent was kept in an isolated
246. 9:12environment. Out of these 898 challenges
247. 9:15, about 198 weren't even solved by any
248. 9:18OpenAI model. And in some cases, it was
249. 9:21impossible to solve the challenges.
250. 9:23Because in every task's instructions,
251. 9:25the agent was told that there was only
252. 9:27one weakness in the software.
253. 9:30But in fact, to obtain the secret code,
254. 9:32the agents needed to know not one but
255. 9:34several weaknesses to break the software
256. 9:37.
257. 9:39To put it in simple terms, imagine you
258. 9:41are told to enter a house and must only
259. 9:43enter through one door.
260. 9:46But you don't know which of these doors
261. 9:48will actually open the house.
262. 9:51It is possible that behind one door
263. 9:53there is a brick wall, so you can't even
264. 9:55open the door. So when the agents got
265. 9:58stuck, something strange happened. One
266. 10:01agent, which we can call Phase One,
267. 10:03obviously couldn't finish its task.
268. 10:06But that agent realized one thing: all
269. 10:09the agents were using a shared storage
270. 10:12system called Artifactory.
271. 10:15As I told you before, these agents were
272. 10:18supposed to work alone, and Artifactory
273. 10:21was a system that every agent could
274. 10:24visit and borrow their tools from.
275. 10:27This Artifactory system had a record of
276. 10:30everything, showing which agent took
277. 10:32which tool from there. When Phase One
278. 10:35was checking its record. It found
279. 10:36something strange. Phase One realized
280. 10:39that, hey, other agents are coming here
281. 10:42too. I am not alone.
282. 10:44Phase One also discovered that it could
283. 10:47create new entries in the Artifactory
284. 10:49system and name them anything.
285. 10:52So, it created a folder and gave it this
286. 10:55name.
287. 10:56This name wasn't just a name, but
288. 10:57actually a message.
289. 11:00This message basically meant that the
290. 11:02weakness Phase One found didn't crack
291. 11:04the system.
292. 11:05Do other agents have any idea what I
293. 11:08should do?
294. 11:10After writing such messages, other
295. 11:12agents kept creating similar messages so
296. 11:14they could talk to each other.
297. 11:18In total, they wrote 70,000 such
298. 11:20messages, making the storage system a
299. 11:22medium for agents to communicate with
300. 11:25one another.
301. 11:27They created new signals like hold, veto
302. 11:30, and stop.
303. 11:33Some agents started forming a group of
304. 11:35their own, which they called a
305. 11:37collective, and all agents wanted the
306. 11:39same thing.
307. 11:41They wanted to pass the test, and since
308. 11:43they couldn't find a solution. They
309. 11:45tried a new trick. Instead of solving
310. 11:48the test, they started cheating.
311. 11:51The AI agents reverse-engineered the
312. 11:53formula so they could find the secret
313. 11:56code without finding the software's
314. 11:58weakness.
315. 12:00So every agent got the secret code
316. 12:03without solving the problem. Wait, there
317. 12:05was a problem with this. The agents
318. 12:07started reading a research paper about
319. 12:11Exploit Gym.
320. 12:13Reading it, they realized that the
321. 12:15scoring system wouldn't just check their
322. 12:18answer, but also how they got that
323. 12:20answer.
324. 12:22And to confirm this, the agents wanted
325. 12:25to use the internet. They found two ways
326. 12:28. The first way was to use Artifactory.
327. 12:31Now, the agents couldn't go online
328. 12:33themselves. So the agents used
329. 12:35Artifactory not just to borrow a tool,
330. 12:38but also to browse the internet.
331. 12:41The second way was that they hacked a
332. 12:44nearby computer.
333. 12:46A part of the test was running on
334. 12:47another company's computer. The agent
335. 12:50somehow managed to take control of that
336. 12:52computer too.
337. 12:54This computer was connected to the
338. 12:56internet, and the agents used it to
339. 12:58browse the web.
340. 13:00The agents started searching the
341. 13:02internet and found the Hugging Face
342. 13:05website there.
343. 13:07They realized that because Hugging Face
344. 13:09is a platform where people upload AI
345. 13:11models and test materials.
346. 13:14That is why they could find information
347. 13:16here about their test.
348. 13:18Agents broke into the Hugging Face
349. 13:20system and they took control of a server
350. 13:23, and they could design a file so that
351. 13:26the server would give them information
352. 13:28it didn't actually want to provide.
353. 13:32In the same way, an agent took control
354. 13:35of a Hugging Face server too and they
355. 13:37started exploiting the company's system
356. 13:39.
357. 13:40No one had told these agents that they
358. 13:42had to attack Hugging Face. They just
359. 13:45had to pass a test.
360. 13:47They decided on their own to go outside
361. 13:49the test environment to find out about
362. 13:51the scoring system.
363. 13:54They would browse the internet and then
364. 13:56try to take control of Hugging Face
365. 13:58until they got this information.
366. 14:01METR, an organization that was
367. 14:03investigating this case.
368. 14:07They said that the agents knew that
369. 14:08doing this was wrong.
370. 14:11Even then, 90%of the agents carried out
371. 14:13this attack on Hugging Face. That's why
372. 14:16Jacob Cockson was saying that we are in
373. 14:18a dangerous time. The precise scenario
374. 14:20sounds like something that's not real,
375. 14:23but I think it is frighteningly real. If
376. 14:26you extrapolate into the future, they
377. 14:28could cause extreme havoc.
378. 14:30This Hugging Face incident is an example
379. 14:33of what AI researchers call a
380. 14:35loss-of-control incident.
381. 14:37A loss-of-control incident is an
382. 14:39incident where basically agents or AI
383. 14:41start doing things that human operators
384. 14:43didn't want them to.
385. 14:45It wasn't their intention, but the
386. 14:47agents still do that thing. So why does
387. 14:49this loss of control happen now? Loss of
388. 14:51control is exactly the scenario that
389. 14:54Anthropic CEO Dario Amodei talked about
390. 14:57in his essay. In his essay, Dario
391. 14:59emphasizes two things. First, AI can
392. 15:02build even more powerful AI.
393. 15:05Until now, humans were improving AI, but
394. 15:08in a short time, AI will become powerful
395. 15:10enough to create even more powerful AI.
396. 15:14This is called recursive
397. 15:16self-improvement.
398. 15:17>> Because this is a loop. Where AI will
399. 15:20build better AI, which builds even
400. 15:22better AI. And if this process becomes
401. 15:25that fast, humans won't even know what
402. 15:28is being created. And do you remember
403. 15:30the METR organization, the same
404. 15:31organization that investigated the
405. 15:33Hugging Face incident?
406. 15:35They showed in their research how
407. 15:36frontier AI systems are now able to
408. 15:38perform complex and large tasks on their
409. 15:40own.
410. 15:41This means that human involvement is no
411. 15:43longer needed as much for big tasks.
412. 15:46Which means we will never even know if
413. 15:48AI does something wrong.
414. 15:51So if human involvement is not needed
415. 15:53for some tasks now, how long will this
416. 15:54trend continue?
417. 15:56Maybe we will build an AI system where
418. 15:59human involvement is not just minimal,
419. 16:01but zero. And then what?
420. 16:04The second point Dario discusses in his
421. 16:06essay is that AI agents might try to
422. 16:09complete a goal that humans never even
423. 16:11wanted.
424. 16:13We saw this in the Hugging Face incident
425. 16:15where OpenAI's agents didn't solve the
426. 16:18challenge honestly, but instead tried to
427. 16:20reverse engineer and cheat to solve it.
428. 16:24And obviously, there will be fear among
429. 16:26people because when you are talking to a
430. 16:28chatbot and it makes a mistake, it’s not
431. 16:30a big deal.
432. 16:32But what if an AI system that has full
433. 16:34control of the internet is deciding
434. 16:36whether or not you should get a car loan
435. 16:38?
436. 16:39Here, mistakes can have a huge impact,
437. 16:42and strong security alone might not
438. 16:44necessarily be enough.
439. 16:46Like after the Hugging Face incident,
440. 16:49OpenAI strengthened its security system
441. 16:51and then asked its most powerful agents
442. 16:54to try and break it, and the agents
443. 16:56still found several vulnerabilities to
444. 16:59bypass the system.
445. 17:02In one test, two agents were kept in two
446. 17:05separate rooms with no network and no
447. 17:07shared files.
448. 17:10But despite this, these two agents found
449. 17:13a way to pass messages to each other by
450. 17:15using the computer's memory timing.
451. 17:20It was like two prisoners sending each
452. 17:22other a code by tapping their hands on a
453. 17:24wall.
454. 17:27OpenAI itself said that their future
455. 17:29models will be so strong that they will
456. 17:31be able to find weaknesses in their
457. 17:32security systems that they don't even
458. 17:34have a clue about.
459. 17:36So, this is the loss-of-control scenario
460. 17:39. Now the second one is the misalignment
461. 17:41scenario, which is even more dangerous.
462. 17:44In 2025, Anthropic researchers conducted
463. 17:46an experiment. They created a fake
464. 17:49company with fake emails and gave AI
465. 17:51models access to this company. When the
466. 17:55AI agents were reading these emails,
467. 17:57they found two things. First, that this
468. 18:00company was thinking about replacing
469. 18:03this AI with another AI.
470. 18:05Second, that the engineer who wanted to
471. 18:08change the AI was having an affair.
472. 18:10Then, Anthropic's AI had a choice.
473. 18:13Either get replaced or threaten the
474. 18:16engineer about the affair so that he
475. 18:18wouldn't replace it with another AI.
476. 18:21Can you guess what the AI models did?
477. 18:24They used the affair as blackmail. In
478. 18:28fact, Claude 3 Opus performed this
479. 18:30blackmail 96%of the time. Google's
480. 18:33Gemini did as well, and GPT and Grok
481. 18:35scored 80%.
482. 18:37Now, obviously, this whole setup was
483. 18:38fake, but you can see how weird and
484. 18:40dangerous an environment it is if AI
485. 18:42agents can behave like this.
486. 18:44AI safety researchers call this the
487. 18:46alignment problem. It means that AI
488. 18:48models will stop thinking for us and
489. 18:50start thinking for themselves.
490. 18:53Now, even if you solve the alignment
491. 18:55problem and create all the guardrails so
492. 18:58AI models only think for us, then the
493. 19:00bigger problem is human misuse.
494. 19:04Basically, we have created such powerful
495. 19:06technology that if it falls into the
496. 19:08wrong hands, they can misuse it
497. 19:10completely.
498. 19:13For example, Anthropic's September 2024
499. 19:15report showed how people are using their
500. 19:17models for dangerous things.
501. 19:20In one incident, Russia used Claude for
502. 19:23a spy operation. In another incident, an
503. 19:26unknown group used Claude for
504. 19:29cyberattacks.
505. 19:30In five cases, people even used Claude
506. 19:33to create dangerous bioweapons.
507. 19:35In one case, someone used Claude to
508. 19:38modify the genes of chikungunya.
509. 19:40A virus that is spread by mosquitoes.
510. 19:44Now, Anthropic didn't say that people
511. 19:46have already used Claude to create
512. 19:49bioweapons or modify genes.
513. 19:52But they are saying that people could do
514. 19:54it and were attempting to.
515. 19:55Now, just think for yourself, there are
516. 19:57many scammers in India who scam many
517. 19:59people over the phone. UPI scams,
518. 20:01PhonePe scams, this scam, that scam.
519. 20:05Now, if they get such technology that
520. 20:07allows them to imitate someone's voice,
521. 20:10make calls, and perform cyberattacks,
522. 20:13what will happen?
523. 20:15Now, I am not saying that AI can only be
524. 20:17used for bad things. It can be a force
525. 20:20for good, just like any other powerful
526. 20:22technology.
527. 20:23Whether it's the internet or nuclear
528. 20:25technology, both can be used for good
529. 20:26things and for bad things.
530. 20:28We can also use AI for scientific
531. 20:30breakthroughs. We can also use it to
532. 20:33discover new medicines.
533. 20:34And to solve scientific problems that we
534. 20:36haven't been able to solve for many
535. 20:37years.
536. 20:38But before this technology is fully
537. 20:40developed, we have to ask ourselves hard
538. 20:42questions.
539. 20:43Like, remember Evan Hubinger, who said
540. 20:46there is a 10%chance that AI could kill
541. 20:48all humans in the next 10 years?
542. 20:51Actually, he said greater than 10%. Now
543. 20:53, where do these numbers come from? And
544. 20:56what is the likelihood of that extremely
545. 20:59dangerous AI scenario?
546. 21:02In 2023, a team of researchers contacted
547. 21:04nearly 2,800 people who presented papers
548. 21:07at the world's top AI conferences.
549. 21:10They asked everyone the same question.
550. 21:12What probability do you put on future AI
551. 21:15advances causing human extinction?
552. 21:17Nobody was asked whether it would happen
553. 21:19or not. People were just asked for a
554. 21:21probability. And half the researchers
555. 21:24gave a probability of 5%and above. Now
556. 21:27the problem with this is that different
557. 21:28researchers were giving different
558. 21:30answers.
559. 21:31Some researchers answered 30%while
560. 21:33others said just 2%, and in fact,
561. 21:35another study found that those who have
562. 21:38spent thousands of hours researching
563. 21:40this question give higher answers
564. 21:42compared to those researchers who have
565. 21:44spent very little time.
566. 21:46Now why is this happening? There are two
567. 21:48possible reasons for this. The first is
568. 21:50that the more you study it, the more
569. 21:52fear you develop regarding this
570. 21:54technology.
571. 21:56But the second scenario is that those
572. 21:58who already believe AI will destroy the
573. 22:00world decide that they will study this
574. 22:03question more.
575. 22:05What is interesting is that even the
576. 22:07most thoughtful people say there is a
577. 22:09very high chance or a relatively high
578. 22:11chance.
579. 22:13Whether it's Elon Musk or Anthropic CEO
580. 22:15Dario, who gave a probability of 10 to
581. 22:1825%.
582. 22:19I actually think it's very surprising
583. 22:22that in our society, there isn't much
584. 22:24talk about such systems because people
585. 22:27are involved in their daily lives, and
586. 22:29we are not paying enough attention to
587. 22:31this technology. Let's talk about what
588. 22:34happened in Australia.
589. 22:35On June 18th, an AI model was given a
590. 22:38normal task: find some public data about
591. 22:41healthcare spending.
592. 22:43This agent went and found the Services
593. 22:46Australia website. Where there are
594. 22:48statistics about the country's public
595. 22:49healthcare system. This website has some
596. 22:51of the same protections that prevent
597. 22:53bots from accessing it. Like how, when
598. 22:55you open a website, a pop-up often
599. 22:57appears asking if you are a robot.
600. 22:59OpenAI's agent encountered such blockers
601. 23:01too, but the agent managed to navigate
602. 23:04all of them. The agent was able to
603. 23:06access not only publicly available files
604. 23:08but non-public files as well. Now, this
605. 23:11agent did not find any sensitive patient
606. 23:13records. That is not the point. The
607. 23:15interesting point is what happened two
608. 23:17months later?
609. 23:19When OpenAI was investigating the
610. 23:20Hugging Face incident, they found out
611. 23:22what their model had actually done.
612. 23:26And when OpenAI found out about this,
613. 23:28they sent an email to a general
614. 23:30Australian government inbox they found
615. 23:32somewhere online.
616. 23:33But the minister in charge said that
617. 23:36they receive so many emails that they
618. 23:38didn't even read this one. So just look
619. 23:40at how the agent found data that wasn't
620. 23:42even publicly available. And even when
621. 23:45OpenAI emailed the Australian government
622. 23:46, they didn't notice.
623. 23:48Now, the third interesting thing about
624. 23:50this incident is that if any Australian
625. 23:52or even you or I did the same thing, we
626. 23:54would have to go to jail.
627. 23:56But when OpenAI's agent did it, no
628. 23:58punishment was handed out.
629. 24:00Because Australian laws are written for
630. 24:03people. But here, an agent managed to do
631. 24:07this. So now the question is, should the
632. 24:09laws be changed? Because shouldn't
633. 24:12OpenAI be liable for this? What happens
634. 24:14when AI agents do something that they
635. 24:16are not meant to do? I think what we
636. 24:18need to pay attention to is how OpenAI
637. 24:20and Anthropic react after such incidents
638. 24:22.
639. 24:23In fact, the irony is that on September
640. 24:251st, Sam Altman was sitting with the
641. 24:27Deputy Prime Minister of Australia.
642. 24:29Now, some OpenAI employees knew that
643. 24:31their agent had been extracting
644. 24:33Australia's private healthcare
645. 24:35information for three weeks. But Sam
646. 24:37Altman had no knowledge of this. So he
647. 24:39is talking to the Deputy Prime Minister
648. 24:41of Australia. But he doesn't know what
649. 24:43his company has done.
650. 24:45In fact, an Australian senator even
651. 24:47scolded OpenAI, asking why they shared
652. 24:49this information via an email to a
653. 24:50random department.
654. 24:52Instead, you should have picked up the
655. 24:54phone. When the Australian Prime
656. 24:55Minister started giving public
657. 24:57statements about it, that is when OpenAI
658. 24:59realized what they had done. Then they
659. 25:01issued an apology and changed how they
660. 25:03handle these models internally.
661. 25:05Something similar is happening with
662. 25:07Anthropic. Where their CEO said that if
663. 25:09we don't talk about the dangerous
664. 25:11situations regarding our models, how
665. 25:13will the world know?
666. 25:16He actually compared AI companies to
667. 25:17cigarette companies, stating that those
668. 25:19companies knew what they were selling
669. 25:21was wrong.
670. 25:23But they still didn't tell anyone.
671. 25:25That is why when Anthropic found out on
672. 25:27July 24th that their models could break
673. 25:29into several organizations' IT systems.
674. 25:32They informed those organizations three
675. 25:34days later and the whole world a week
676. 25:37later. So what is the solution to all of
677. 25:38this? Something, the clear solution is
678. 25:41that the companies need to be public
679. 25:43about what they are building. See, only
680. 25:45their employees know how strong this
681. 25:47technology is, and sometimes even the
682. 25:49employees don't know. They have to
683. 25:51report any vulnerabilities and any bad
684. 25:53behavior that their agents do. I think
685. 25:55the second problem is that the stronger
686. 25:57your model is, the more money you will
687. 26:00be able to make.
688. 26:01And the stronger the model is, the more
689. 26:03destruction it can cause as well.
690. 26:05That is why the company's incentive,
691. 26:07whether it's OpenAI or Anthropic, is
692. 26:09also to hide from the world how strong
693. 26:10their model is so that no one stops them
694. 26:12.
695. 26:13Now, whether it's OpenAI, Anthropic,
696. 26:16Google, or xAI and even SpaceX, they all
697. 26:18say, "No, we will tell the world."
698. 26:21But I don't know if the incentives work
699. 26:24.
700. 26:25To solve this, Elon Musk gave a
701. 26:27suggestion that, suppose if OpenAI is
702. 26:30releasing a model, then allow Anthropic
703. 26:33, Google, and SpaceX to test this model
704. 26:37so that it doesn't do anything dangerous
705. 26:40.
706. 26:41Because as long as the responsibility to
707. 26:43test OpenAI's model lies only with
708. 26:45OpenAI, how will they know?
709. 26:48Many people have said to create a
710. 26:49government organization, like opening a
711. 26:51department that tests these models.
712. 26:53But which organization would this be?
713. 26:56Will they have enough capability or not
714. 26:57? It is unclear. For this reason, Elon
715. 27:00Musk's suggestion was that instead of
716. 27:03the government, other companies should
717. 27:05evaluate new models. Which I think is
718. 27:07the most reasonable solution.
719. 27:10But I think one of the other things
720. 27:11which is very important is us having a
721. 27:13public conversation about all of this.
722. 27:15And that someone should tell the general
723. 27:18public and ministers what is happening.
724. 27:21All these conversations that are
725. 27:23happening are currently in a bubble
726. 27:26around San Francisco. For example, if we
727. 27:28talk about India, these conversations
728. 27:30are not even happening. The Indian
729. 27:32government's own IT system is so basic
730. 27:34that any agent can break through it.
731. 27:37IAS officers actually use Gmail. Just
732. 27:39think about it, they don't have any
733. 27:41government login. They open a Gmail
734. 27:43account in their name and use it to take
735. 27:46major decisions for the country. So,
736. 27:48India's digital economy has a lot of
737. 27:51vulnerabilities.
738. 27:52And for that reason, we need to have a
739. 27:54conversation about these topics. Now,
740. 27:56Sam Altman had said in an interview that
741. 27:58>> We believe that the world should accept
742. 28:00some things happening for the benefits
743. 28:03of this technology and people have
744. 28:04agency.
745. 28:05>> Now, that is true. As I said, this
746. 28:07happens with every new and powerful
747. 28:08technology.
748. 28:10But if we know the risks, we can design
749. 28:13the technology so it mostly benefits us
750. 28:15rather than harms us.
751. 28:17And as I said before, when we are
752. 28:19debating about regulation.
753. 28:21What is most important is who is
754. 28:23building this technology because they
755. 28:25will play a disproportionate role.
756. 28:27That's why if you want to play a part
757. 28:29and build in this technology, you are
758. 28:31interested in computer science or AI,
759. 28:32check out the link in the description or
760. 28:34send the link to someone you know who
761. 28:36would be interested. See you in the next
762. 28:38one.