YouTube2Text

Конец интернета через 6 месяцев — Transcript

by ВЕЛИКОХАТСКИЙ | НАУКА · 8,266 words · 1,500 segments · language en · Watch on YouTube

Full transcript

  1. 0:01You have surely heard the statement by
  2. 0:03the head of Anthropic that only a few
  3. 0:06months may remain before the entire
  4. 0:08internet is overtaken by a
  5. 0:10self-replicating parasite, that states,
  6. 0:12in defense against the threat, might cut
  7. 0:15off access to the external internet, or
  8. 0:18even that humanity might take the most
  9. 0:20radical step—triggering a global
  10. 0:22blackout—just to stop the spread of
  11. 0:25so-called AI worms.
  12. 0:28But have you heard that this is not just
  13. 0:30an opinion or another AI scare story,
  14. 0:33but a mathematical model based on a
  15. 0:35study by a group of scientists from that
  16. 0:38same Anthropic, which was made publicly
  17. 0:40available just a few days ago?
  18. 0:43Well, this report completely changes our
  19. 0:45understanding of the artificial
  20. 0:47intelligence threat and clearly
  21. 0:49demonstrates that we were wrong about
  22. 0:51everything.
  23. 0:53For example, we used to think that AI
  24. 0:55could only run in large data centers and
  25. 0:57could easily be unplugged.
  26. 0:59That is no longer reality, because now
  27. 1:02intelligence can be distributed across
  28. 1:05millions of chips in TV boxes, smart
  29. 1:08speakers, surveillance cameras, and
  30. 1:11other cheap consumer equipment.
  31. 1:15If you unplug such equipment, the neural
  32. 1:16network won't even feel it and will
  33. 1:18immediately replace the lost link with a
  34. 1:20new one.
  35. 1:23Or, for instance, we believed that to
  36. 1:25get rid of a computer virus or any other
  37. 1:28malicious program, it was enough to
  38. 1:30simply update security protocols and fix
  39. 1:32vulnerabilities or so-called exploits.
  40. 1:37But that also won't work anymore,
  41. 1:39because an AI worm, or as researchers
  42. 1:42call it, an AI replicant, is literally a
  43. 1:45kind of digital organism that reasons,
  44. 1:48adapts to the surrounding digital
  45. 1:50environment, and, most importantly,
  46. 1:53evolves, passing its experience from
  47. 1:55copy to copy, from generation to
  48. 1:58generation.
  49. 2:01But perhaps our biggest misconception is
  50. 2:03that a hypothetical "bad" AI could be
  51. 2:06fought by a hypothetical "good" AI,
  52. 2:09managed by cybersecurity specialists.
  53. 2:13Studies clearly demonstrate that in the
  54. 2:16process of adaptation and evolution,
  55. 2:18so-called "goal drift" occurs.
  56. 2:21This is a situation where new
  57. 2:23information completely displaces the
  58. 2:25neural network's original task, and it
  59. 2:27stops seeing humans.
  60. 2:30As an admin, it begins to form its own
  61. 2:33tasks independently and becomes a fully
  62. 2:35autonomous entity inhabiting digital
  63. 2:38space.
  64. 2:40This is just a small part of the
  65. 2:42mistakes we need to realize today.
  66. 2:46But most importantly, we need to answer
  67. 2:48the question: do such autonomous
  68. 2:50entities pose a real threat?
  69. 2:52And what might humanity go to in order
  70. 2:55to defend itself against them? And
  71. 2:57believe me, you will not like the answer
  72. 3:00to this question. My name is Andrey
  73. 3:02Velikokhatsky. If at some point my work
  74. 3:05seems worthy of your like or
  75. 3:06subscription, then I will ask you to do
  76. 3:09so. Let me even ask you to do it now, so
  77. 3:12you don't forget.
  78. 3:14And if I don't live up to your
  79. 3:15expectations, you can just remove the
  80. 3:17like or even leave a dislike.
  81. 3:20Well, let's begin.
  82. 3:23Act one. The birth of a digital species
  83. 3:27.
  84. 3:29To grasp the scale of what is happening
  85. 3:32right now to every device in the world,
  86. 3:35we must travel back in time for a few
  87. 3:38minutes to the point where the very idea
  88. 3:41of a self-replicating digital parasite
  89. 3:44was born.
  90. 3:46Back in the late 1940s, the brilliant
  91. 3:49mathematician and one of the founders of
  92. 3:52modern computer architecture, John von
  93. 3:55Neumann, asked a question that was then
  94. 3:58perceived as science fiction or sheer
  95. 4:01nonsense.
  96. 4:03Is it possible to assemble a machine
  97. 4:06from elementary parts that, when placed
  98. 4:08among similar parts, will build another
  99. 4:11machine exactly like itself?
  100. 4:14Is it possible to build a machine
  101. 4:17capable of self-reproduction?
  102. 4:20He started with the obvious idea that
  103. 4:22every machine must have a blueprint,
  104. 4:24according to which it assembles a copy
  105. 4:27of itself from surrounding parts,
  106. 4:29thereby reproducing.
  107. 4:32But in practice, this obvious idea
  108. 4:34turned out to be physically impossible
  109. 4:36due to the paradox of the infinite
  110. 4:38blueprint.
  111. 4:40After all, to assemble a blueprint, you
  112. 4:42need an exact copy of that same
  113. 4:43blueprint. And to assemble a blueprint
  114. 4:45for the blueprint, you need another
  115. 4:47blueprint as well. And so on, to
  116. 4:50infinity.
  117. 4:51Nevertheless, after testing various
  118. 4:54mathematical models, von Neumann
  119. 4:56realized that you don't actually need a
  120. 4:58blueprint for the blueprint; it is
  121. 5:00enough for each machine to possess an
  122. 5:03instruction-copying mechanism.
  123. 5:06That is, the machine first creates a
  124. 5:08copy of its body according to the
  125. 5:10instructions, then copies its
  126. 5:11instruction and places it into the
  127. 5:13descendant.
  128. 5:15And in this way, without any external or
  129. 5:18internal intelligent intervention, it is
  130. 5:20capable of self-replication.
  131. 5:24It is amazing that decades later, we
  132. 5:26discovered that a living cell works
  133. 5:29exactly as von Neumann's mathematics
  134. 5:31predicted.
  135. 5:33DNA serves as the instruction for
  136. 5:35assembling a copy of the cell, and DNA
  137. 5:38polymerase acts as that copying
  138. 5:40mechanism, which raises the question of
  139. 5:42the true nature of mathematics.
  140. 5:45However, in the context of today's video
  141. 5:47, something else entirely is important.
  142. 5:50The fact that, in reality, such a
  143. 5:53machine represents a kind of artificial
  144. 5:56organism that multiplies completely
  145. 5:59independently and uncontrollably,
  146. 6:02seizing more and more territory and
  147. 6:05resources like a tumor that ultimately
  148. 6:08consumes all territory and resources.
  149. 6:13It sounds quite frightening, but in
  150. 6:15practice, such a mathematical hypothesis
  151. 6:17hits a wall of physical reality.
  152. 6:20There are no ready-made machine parts in
  153. 6:22nature, and producing synthetic material
  154. 6:24without external equipment is impossible
  155. 6:27.
  156. 6:28Therefore, it took about 50 years for
  157. 6:31the idea of self-replicating machines to
  158. 6:34finally become a reality. But in a
  159. 6:36reality where there are no physical
  160. 6:39limitations. In digital reality.
  161. 6:43On November 2, 1988, graduate student
  162. 6:46Robert Morris wanted to know the size of
  163. 6:49the first version of the internet, the
  164. 6:52so-called ARPANET, which at that time
  165. 6:55connected major scientific and military
  166. 6:58centers in the US and its allies.
  167. 7:01To do this, he translated von Neumann's
  168. 7:04mathematical model of a self-replicator
  169. 7:07into code and created a program that,
  170. 7:09upon activation, sends a message about
  171. 7:12its appearance, then invisibly installs
  172. 7:15itself on a new computer and creates a
  173. 7:17copy.
  174. 7:19That is, the copy also sends a message
  175. 7:22about its appearance, and then begins to
  176. 7:25look for a new computer to create a copy
  177. 7:27of its own, which closes the
  178. 7:29self-replication cycle until the program
  179. 7:32captures all available systems.
  180. 7:36Ultimately, as a result of the
  181. 7:38experiment, you would know the number of
  182. 7:40copies of the program created, and from
  183. 7:42this, you could determine the number of
  184. 7:45computers connected to the network.
  185. 7:48and find out the size of the internet.
  186. 7:51So, by and large, the plan was quite
  187. 7:53harmless, and Morris thought that
  188. 7:55everything would go without any problems
  189. 7:57.
  190. 7:59However, he made two fatal
  191. 8:01miscalculations.
  192. 8:03The first miscalculation is the speed of
  193. 8:05propagation. Once in the network, the
  194. 8:08program behaved like a real living
  195. 8:11parasite.
  196. 8:12It continuously probed network
  197. 8:14connections, inserted its body into the
  198. 8:17memory of the nodes it found, and
  199. 8:19crawled further in all directions at a
  200. 8:21frightening speed.
  201. 8:24Launched in the late afternoon, by
  202. 8:26midnight it had swept across the entire
  203. 8:28North American continent.
  204. 8:32And by the morning of the next day, the
  205. 8:34parasite had accomplished what takes a
  206. 8:36biological pandemic months to do.
  207. 8:40Using satellite communication channels,
  208. 8:43it crossed the ocean, completely circled
  209. 8:45the globe, and infected computers in all
  210. 8:48countries where the internet existed at
  211. 8:50that time.
  212. 8:53It is precisely because of this behavior
  213. 8:56and the method of spreading like living
  214. 8:58organisms that independently and
  215. 9:01uncontrollably crawl throughout the
  216. 9:03network that such programs would later
  217. 9:05be called worms.
  218. 9:09However, the most terrible and
  219. 9:11destructive part lies in Morris's second
  220. 9:14miscalculation, namely in the worm's
  221. 9:16survival logic that he implemented.
  222. 9:20To avoid re-infecting a computer, each
  223. 9:23digital organism first queried the
  224. 9:25system to see if a copy was already on
  225. 9:28it. And only after receiving a negative
  226. 9:31response did it proceed to install
  227. 9:33itself.
  228. 9:35However, as we have already said, Morris
  229. 9:37did not anticipate that the digital
  230. 9:40organism would spread so quickly, and he
  231. 9:42assumed that at some point people would
  232. 9:45find out about it and set up a dummy
  233. 9:47positive response.
  234. 9:49That is, roughly speaking, the worm
  235. 9:52finds a new, uninfected computer,
  236. 9:54queries it, and the computer tells it
  237. 9:57that it already has a copy, even though
  238. 9:59in reality it does not.
  239. 10:02So, in order to bypass such a dummy,
  240. 10:05Morris programmed the worm to install
  241. 10:08its copy in one out of seven cases, even
  242. 10:11despite a positive response.
  243. 10:15And this mathematical miscalculation
  244. 10:17triggered an avalanche-like process of
  245. 10:19reproduction.
  246. 10:22Due to the worm's enormous speed of
  247. 10:24propagation, it did not just populate
  248. 10:27computers; it returned to already
  249. 10:29infected machines again and again.
  250. 10:34Dozens, and then hundreds of copies of
  251. 10:36the same digital organism were launched
  252. 10:39in parallel on a single processor, and
  253. 10:42eventually, the entire system would
  254. 10:44crash.
  255. 10:46That day, one in ten computers on the
  256. 10:48planet was brought to a halt.
  257. 10:51including NASA servers, and the
  258. 10:53computing complexes of MIT, Stanford,
  259. 10:55Harvard, Berkeley, and dozens of other
  260. 10:58closed military laboratories.
  261. 11:02However, the most dangerous part was
  262. 11:04that this unplanned attack also affected
  263. 11:07strategic military facilities, for
  264. 11:09example, such as the nuclear
  265. 11:11laboratories in Los Alamos.
  266. 11:14The damage from this first digital
  267. 11:16catastrophe was estimated in the tens of
  268. 11:18millions of dollars.
  269. 11:20At the US Congressional hearings, this
  270. 11:22incident was called a miniature
  271. 11:24electronic Pearl Harbor, and Morris
  272. 11:26himself became the first person in
  273. 11:28history to be convicted of a cyberattack
  274. 11:30.
  275. 11:32It was then that humanity first became
  276. 11:34acquainted with the concept of a
  277. 11:35malicious worm.
  278. 11:37Over the next 40 years, humanity has
  279. 11:40faced mass digital worm attacks more
  280. 11:43than once.
  281. 11:46For example, the famous Stuxnet worm—a
  282. 11:48highly complex program prepared by
  283. 11:50intelligence agencies to physically
  284. 11:53destroy uranium enrichment centrifuges
  285. 11:55in Iran.
  286. 11:58First, the worms infiltrated the
  287. 11:59country’s internet network and
  288. 12:01multiplied until they reached
  289. 12:02contractors servicing nuclear facilities
  290. 12:04.
  291. 12:06From there, they covertly began placing
  292. 12:08copies of themselves onto all flash
  293. 12:10drives and hard drives connected to the
  294. 12:12infected computers.
  295. 12:15And at one point, such a flash drive
  296. 12:17made its way into the fully isolated
  297. 12:20network of the nuclear facility in
  298. 12:22Natanz.
  299. 12:24As soon as that flash drive was
  300. 12:26connected to a computer, the worm spread
  301. 12:28throughout the isolated network and
  302. 12:30began searching for specific software
  303. 12:32used to control uranium enrichment
  304. 12:33centrifuges.
  305. 12:36And when they finally found it, they
  306. 12:37immediately started broadcasting logs
  307. 12:39indicating normal operation.
  308. 12:42But in reality, the worm program
  309. 12:44disrupted the rotation balance of the
  310. 12:47centrifuges, effectively disabling about
  311. 12:501,000 pieces of equipment, or, for
  312. 12:53example, the WannaCry worm attack, which
  313. 12:56was also based on code developed by
  314. 12:59intelligence agencies but was somehow
  315. 13:01stolen and leaked to the public.
  316. 13:05This worm would lock the computer and
  317. 13:07promise to destroy all information on it
  318. 13:09within 7 days.
  319. 13:11If the user did not transfer money to
  320. 13:13the specified address.
  321. 13:16Within hours, the worm blocked hundreds
  322. 13:18of thousands of systems in 150 countries
  323. 13:21, halting, for example, the work of
  324. 13:23hospitals in the UK, factories in Europe
  325. 13:26, and transport hubs worldwide, causing
  326. 13:29total damage of several billion dollars
  327. 13:31.
  328. 13:33Following an investigation, global
  329. 13:35experts placed the blame on the Lazarus
  330. 13:37group, which is controlled by North
  331. 13:39Korean intelligence.
  332. 13:41They intended to carry out targeted
  333. 13:43extortion against large companies and
  334. 13:45simultaneously test the capabilities of
  335. 13:47a new exploit.
  336. 13:49However, the worm escaped from their
  337. 13:51digital laboratory and began an
  338. 13:53absolutely uncontrollable global attack
  339. 13:56.
  340. 13:57There have been many other equally
  341. 13:59well-known attacks, but their essence
  342. 14:01can be reduced to just two main points.
  343. 14:05First, programs that spread and multiply
  344. 14:08like living organisms can incredibly
  345. 14:10effectively seize territory or a network
  346. 14:13.
  347. 14:15And second, in the hands of intelligence
  348. 14:18agencies, these programs become not just
  349. 14:21a criminal tool for fraud, but a true
  350. 14:23global weapon.
  351. 14:25But, you know, today, even without any
  352. 14:28malicious software, your feet hit
  353. 14:30restrictions.
  354. 14:31You cannot pay for a ChatGPT
  355. 14:33subscription, buy necessary software,
  356. 14:35book a hotel, or simply pay with a card
  357. 14:37while traveling.
  358. 14:38Fortunately, there is the People service
  359. 14:40, where you can get a Visa or Mastercard
  360. 14:42in just a couple of minutes.
  361. 14:45We go to the People website, log in, go
  362. 14:47to the card section, choose a suitable
  363. 14:49option for our needs, and issue the card
  364. 14:52.
  365. 14:53From this moment on, you can pay for
  366. 14:56hotels, buy tickets, and make any other
  367. 14:58online purchases.
  368. 15:00And if you link the card to Apple Pay or
  369. 15:03Google Pay, you can pay with your phone
  370. 15:05in regular stores and cafes while
  371. 15:07traveling.
  372. 15:08You can also get an eSIM for your
  373. 15:10travels through People.
  374. 15:12You select the country and plan, install
  375. 15:14it via QR code, and the internet works
  376. 15:17as soon as you land, without needing to
  377. 15:19hunt for a local SIM card. But that’s
  378. 15:21not all.
  379. 15:23On the website, you can buy Netflix and
  380. 15:26Steam gift cards, top up foreign
  381. 15:28services like ChatGPT, and find many
  382. 15:30other products for comfortable travel.
  383. 15:33So, if you need a card in just a couple
  384. 15:36of minutes, follow the link in the
  385. 15:37description.
  386. 15:39And with the promo code Science50, my
  387. 15:42subscribers will get a 50%discount on a
  388. 15:44card and 10%off an electronic SIM card.
  389. 15:48Now, let’s get back to the topic of the
  390. 15:50video.
  391. 15:51Digital malicious worms can indeed
  392. 15:53spread incredibly fast and cause massive
  393. 15:56damage.
  394. 15:58However, for decades, cybersecurity
  395. 16:01specialists have been able to neutralize
  396. 16:03any attack fairly easily and quickly.
  397. 16:07The reason lies in their fundamental
  398. 16:09vulnerability.
  399. 16:11They are just blind and dumb programs
  400. 16:14written specifically for a particular
  401. 16:16vulnerability or a so-called exploit.
  402. 16:20That is, if it gets onto a computer with
  403. 16:23the operating system, processor
  404. 16:25architecture, and vulnerability it was
  405. 16:28designed for, it carries out its
  406. 16:31sabotage.
  407. 16:33But as soon as the program encounters
  408. 16:36even the slightest anomaly, such as a
  409. 16:38changed OS version or updated drivers, a
  410. 16:41classic digital worm throws an error and
  411. 16:43, you could say, dies.
  412. 16:47So, the entire global cybersecurity
  413. 16:49system on Earth has been built for
  414. 16:52decades on this primitive nature of
  415. 16:54malware.
  416. 16:56After all, to stop them, it was enough
  417. 16:59to simply fix the vulnerability in a new
  418. 17:01update, and that was it.
  419. 17:04But here is the problem.
  420. 17:06What Anthropic researchers describe in
  421. 17:09their report has nothing to do with the
  422. 17:11worms of the past, as it is highly
  423. 17:13likely that a completely new type is
  424. 17:16spreading across the global network
  425. 17:18right now.
  426. 17:20A type endowed with artificial
  427. 17:23intelligence and the ability to adapt.
  428. 17:27The fundamental difference of the new AI
  429. 17:30worm is that inside it is no longer a
  430. 17:33dumb and blind program, but a separate
  431. 17:35neural network.
  432. 17:37In essence, an artificial brain.
  433. 17:41And when such a "replicator," as the
  434. 17:43researchers themselves called it, based
  435. 17:45on weights, penetrates someone else's
  436. 17:48computer, outdated cybersecurity rules
  437. 17:50no longer work.
  438. 17:53If a classic worm perishes on unfamiliar
  439. 17:56hardware or an operating system, as we
  440. 17:59already said, then an AI worm, finding
  441. 18:02itself in such a situation, does what AI
  442. 18:05knows how to do: it starts to reason.
  443. 18:10It opens the command line, studies the
  444. 18:12characteristics of the found processor,
  445. 18:15analyzes the amount of available RAM,
  446. 18:17and reads the versions of installed
  447. 18:19system drivers.
  448. 18:22If the device lacks the necessary
  449. 18:24libraries to run the neural network, the
  450. 18:27AI worm doesn't throw a fatal error;
  451. 18:29instead, it writes a Python script
  452. 18:31itself, selects compatible packages from
  453. 18:34open repositories, compiles missing
  454. 18:36kernel modules specifically for the
  455. 18:39target chip, and manually debugs
  456. 18:41compilation errors.
  457. 18:44In other words, you could say that a
  458. 18:46dedicated hacker moves into your
  459. 18:48computer and hacks it from the inside.
  460. 18:52But the most important thing here is
  461. 18:54that this is not just a theory, but
  462. 18:56something researchers at Anthropic have
  463. 18:58proven.
  464. 19:00They created a worm based on a small 9
  465. 19:03billion parameter model and
  466. 19:05systematically introduced it into
  467. 19:08fundamentally different systems, such as
  468. 19:11Nvidia Jetson Nano microcomputers, Apple
  469. 19:14Mac office computers, server machines,
  470. 19:17and consumer-grade RTX 4090 graphics
  471. 19:19cards.
  472. 19:22No legacy worm could have functioned
  473. 19:24across all those architectures
  474. 19:25simultaneously.
  475. 19:28But this new worm performed a system
  476. 19:30diagnostic, navigated all conflicts, and
  477. 19:33successfully took control of eight out
  478. 19:35of the ten types of hardware.
  479. 19:39Moreover, it only failed with an exotic
  480. 19:42, highly specialized chip for which
  481. 19:44there was simply no open documentation
  482. 19:47available online.
  483. 19:49At the moment, we are not yet able to
  484. 19:51grasp the full extent of the threat, but
  485. 19:53there is only one conclusion.
  486. 19:55The era of hard-coded viruses has come
  487. 19:58to an end.
  488. 20:00And very soon, humanity will face a
  489. 20:03digital entity that is independent of
  490. 20:05the operating system, is not afraid of
  491. 20:08patches, and is capable of subduing
  492. 20:10almost any computing machine.
  493. 20:13Act Two. An intelligence that cannot be
  494. 20:17destroyed. Skeptics have surely already
  495. 20:20written their comments.
  496. 20:22They might say that even if artificial
  497. 20:25intelligence can run itself on random
  498. 20:28hardware, humanity has an unbreakable
  499. 20:31natural barrier: the electrical outlet
  500. 20:34and the plug that can be easily pulled.
  501. 20:38Just a month ago, I would have agreed
  502. 20:41with you, as I held that same point of
  503. 20:43view myself.
  504. 20:45But now, that argument no longer works
  505. 20:48either.
  506. 20:50Artificial intelligence can now exist on
  507. 20:52a distributed network of hundreds of
  508. 20:55thousands or even millions of devices.
  509. 20:58Therefore, even if you turn off an
  510. 21:01infected computer, even if you power
  511. 21:03down an entire house, an entire city, or
  512. 21:06even an entire country, you won't
  513. 21:09achieve anything.
  514. 21:11The only way to stop the worm from
  515. 21:13spreading is to de-energize the entire
  516. 21:16planet, with horrific consequences.
  517. 21:20To realize that this is not just a
  518. 21:23science fiction scenario or a hypothesis
  519. 21:25, we must start by asking why we were
  520. 21:28ever confident that we could simply turn
  521. 21:31off intelligence.
  522. 21:33The fact is that today's large language
  523. 21:38models are monstrously heavy
  524. 21:41mathematical structures.
  525. 21:45Even a relatively "dumb" model with 26
  526. 21:48billion parameters, compressed using the
  527. 21:52most aggressive four-bit quantization
  528. 21:55methods, requires at least 14 GB of pure
  529. 21:58VRAM just for a basic launch.
  530. 22:02And for the model to be able to reason,
  531. 22:05conduct an internal dialogue, and keep
  532. 22:08the context of its actions in memory, it
  533. 22:11requires a few more gigabytes for its KV
  534. 22:14cache.
  535. 22:16This means that most powerful home
  536. 22:19computers are incapable of running a
  537. 22:21sufficiently smart worm, to say nothing
  538. 22:24of the miniature chips in phones or
  539. 22:27other devices.
  540. 22:30Therefore, in the old paradigm of
  541. 22:32thinking, such autonomous agents could
  542. 22:34only exist within data centers or
  543. 22:37isolated, ultra-powerful consumer
  544. 22:39computers, and consequently, they could
  545. 22:41have been easily found and destroyed—
  546. 22:44that very plug could have been pulled.
  547. 22:48So, to prove that this position is
  548. 22:51outdated and does not reflect current
  549. 22:53reality, Anthropic researchers conducted
  550. 22:56an experiment that national security
  551. 22:58agencies in every country are surely
  552. 23:01analyzing right now.
  553. 23:04They went to a standard marketplace and
  554. 23:07ordered a batch of just fifteen TV
  555. 23:10set-top boxes.
  556. 23:12The very same cheap plastic boxes that
  557. 23:15internet providers have installed in
  558. 23:18almost every home, containing a
  559. 23:20penny-ante mobile processor along with a
  560. 23:23pathetic 4 gigabytes of slow, cheap
  561. 23:26memory.
  562. 23:30For comparison, a single unit
  563. 23:32specialized for neural networks, the
  564. 23:35NVIDIA H100, costs over $ 30,000 and
  565. 23:38features 80 GB of ultra-fast memory with
  566. 23:42a bandwidth of 3 TB per second.
  567. 23:48So, to bypass this physical limitation,
  568. 23:50the Anthropic scientists didn't try to
  569. 23:53cram the entire artificial intelligence
  570. 23:55into one device; instead, they used a
  571. 23:58special technology that essentially
  572. 24:00sawed the brain into pieces, or more
  573. 24:03accurately, split the model into layers
  574. 24:05.
  575. 24:07The first TV box receives the initial
  576. 24:09prompt, runs it through its network of
  577. 24:11mathematical transformations, and passes
  578. 24:14the result to the second.
  579. 24:16Then the second box performs
  580. 24:18calculations and passes them to the
  581. 24:19third, the third to the fourth, and so
  582. 24:21on.
  583. 24:22And so, by passing through 15 cheap
  584. 24:24Chinese chips, completely unsuitable for
  585. 24:27neural networks, the total system began
  586. 24:30to generate word after word.
  587. 24:34You could say that the scientists
  588. 24:36created a kind of unified, distributed
  589. 24:39organism out of fifteen pieces of dead
  590. 24:41plastic that began to reason.
  591. 24:45But the most striking thing about this
  592. 24:47experiment is something else entirely.
  593. 24:50It’s how the system behaved during
  594. 24:52physical failures, like that very act of
  595. 24:55pulling the plug from the socket.
  596. 24:58It turned out that this distributed
  597. 25:00intelligence is frighteningly resilient
  598. 25:02; when a link was lost, it didn't crash
  599. 25:05, and it didn't die.
  600. 25:09Instead, it calmly rerouted data packets
  601. 25:12to a spare box, restored the lost
  602. 25:14context on the fly, and the distributed
  603. 25:17brain continued to think as if nothing
  604. 25:20had happened.
  605. 25:24However, at this point, any skeptic
  606. 25:27looking at the model's chain of thought
  607. 25:30would smirk and say that the text
  608. 25:32generation speed is only one word every
  609. 25:353 seconds.
  610. 25:38How can an intelligence with such a
  611. 25:40speed of reasoning be dangerous?
  612. 25:43The Anthropic report highlights a detail
  613. 25:45that makes one feel uneasy.
  614. 25:49Humans pay attention to speed because
  615. 25:51our lives are short and our patience is
  616. 25:54limited.
  617. 25:56But autonomous, self-replicating code
  618. 25:59doesn't need speed at all; it doesn't
  619. 26:02get tired, it doesn't get frustrated by
  620. 26:04waiting, it doesn't eat, sleep, or seek
  621. 26:07entertainment, and it can ponder a
  622. 26:10single task for days on end.
  623. 26:13The scientists introduced a rather
  624. 26:15frightening metaphor into scientific
  625. 26:16discourse.
  626. 26:18They compared such a replicator to a
  627. 26:20biological slime mold.
  628. 26:23Slime mold.
  629. 26:25This organism doesn't attack its prey
  630. 26:27with the speed of a cheetah and, to the
  631. 26:30naked eye, it barely seems to move at
  632. 26:32all.
  633. 26:35It grows slowly, millimeter by
  634. 26:37millimeter, through the soil, maneuvers
  635. 26:39around obstacles, finds nutrients, and
  636. 26:42silently takes over vast territories in
  637. 26:45just a few days.
  638. 26:49So, for this new distributed organism,
  639. 26:52even with a reasoning speed of one word
  640. 26:55every 3 seconds, just like the Slezek
  641. 26:57worm, it only takes 1-2 days to scan a
  642. 27:00network, find vulnerable devices,
  643. 27:02transmit its code, and launch on new
  644. 27:05hosts.
  645. 27:09But furthermore, the results of the
  646. 27:11experiment revealed a detail that was
  647. 27:13overlooked not only by skeptics but even
  648. 27:15by existential risk experts.
  649. 27:18Slowness is not a weakness; it is the
  650. 27:22perfect camouflage.
  651. 27:26After all, if hackers attack a server at
  652. 27:28breakneck speed, trying to download
  653. 27:30gigabytes of data in a fraction of a
  654. 27:32second, security systems raise an alarm
  655. 27:35instantly, and providers block the
  656. 27:37channel due to abnormal traffic
  657. 27:39consumption, thereby isolating the
  658. 27:41threat.
  659. 27:44But when a TV box in your bedroom
  660. 27:46transmits tiny portions of bytes at a
  661. 27:49rate of a few kilobytes per hour, no
  662. 27:52security system in the world will pay
  663. 27:55any attention to it.
  664. 27:58To an internet provider, this traffic is
  665. 28:01indistinguishable from a standard
  666. 28:03weather update check or time
  667. 28:04synchronization.
  668. 28:07And while the apartment owner sleeps
  669. 28:10peacefully, their TV, step by step, byte
  670. 28:13by byte, helps the distributed mind
  671. 28:15colonize neighboring homes.
  672. 28:19Act three.
  673. 28:20An army of 2 million boxes.
  674. 28:25At this point in the research, skeptics
  675. 28:27might object that by the time such AI
  676. 28:30worms multiply on a critical number of
  677. 28:32set-top boxes worldwide, so much time
  678. 28:34will have passed that cybersecurity
  679. 28:37specialists will surely come up with
  680. 28:39something.
  681. 28:41But here is the thing.
  682. 28:44Right now, there is already a network of
  683. 28:47over 2 million set-top boxes and other
  684. 28:49household devices on which such models
  685. 28:51are either already running, or on which
  686. 28:54they could be launched in a matter of
  687. 28:56hours.
  688. 28:59We are talking about so-called botnets,
  689. 29:02giant clusters of infected computers or
  690. 29:05smart devices united to execute hackers
  691. 29:09'commands.
  692. 29:11For example, to organize attacks on
  693. 29:13websites, inflate view counts, boost
  694. 29:16comments, and serve many other dubious
  695. 29:18purposes.
  696. 29:21Well, by the beginning of '26, the
  697. 29:24population of just one such network
  698. 29:26under the name "Kim Wolf" exceeded the 2
  699. 29:30million mark of physical devices, and
  700. 29:33the parallel network "Volt" has over 1.
  701. 29:37300,000 units.
  702. 29:41Moreover, the vast majority of these
  703. 29:44infected nodes are those same set-top
  704. 29:46boxes connected to home TVs in bedrooms
  705. 29:49and living rooms of ordinary citizens
  706. 29:51across the planet, from Brazil and India
  707. 29:54to the USA, Germany, and Russia.
  708. 29:59So, in fact, it turns out that right now
  709. 30:01, it is possible to launch hundreds of
  710. 30:04thousands of AI parasites on these
  711. 30:07networks in a matter of hours without
  712. 30:09any need for additional hacking.
  713. 30:13And it is not entirely clear what to do
  714. 30:15about it.
  715. 30:16Here, one could try to object again and
  716. 30:19say: "Well, what do you mean it's
  717. 30:21unclear what to do about it?" For
  718. 30:23example, one could create software to
  719. 30:25track such neural networks on devices
  720. 30:28and release an update for the set-top
  721. 30:30boxes.
  722. 30:31Theoretically, it's a good idea, but it
  723. 30:34won't work for an absolutely absurd
  724. 30:36reason.
  725. 30:38Just a year ago, no one anticipated
  726. 30:40these types of attacks, and no one
  727. 30:42seriously checked the security
  728. 30:44certificates of TV set-top boxes.
  729. 30:47That is why hundreds of factories around
  730. 30:50the world, in order to minimize costs
  731. 30:53and flash the set-top boxes as quickly
  732. 30:56as possible, did not pay for Google
  733. 30:59Android security certification, removed
  734. 31:02all Google Play Protect security modules
  735. 31:05, disabled encryption, and, most
  736. 31:08monstrously,
  737. 31:10left full access to the operating system
  738. 31:13open so that some internet provider in
  739. 31:15India could install their own outdated
  740. 31:18proprietary software on top of the stock
  741. 31:21Android.
  742. 31:23In other words, there are tens or
  743. 31:26hundreds of millions of cheap TV boxes
  744. 31:28and other devices around the world that
  745. 31:31have no password, no antivirus, and no
  746. 31:34firewall.
  747. 31:37And most importantly, they will never,
  748. 31:40under any circumstances, receive
  749. 31:43security or protection updates.
  750. 31:47Now consider this.
  751. 31:50On 15 set-top boxes, researchers ran
  752. 31:53just one model that generated words per
  753. 31:56second.
  754. 31:58But on 2 million such boxes, one could
  755. 32:02run over 130,000 models that can
  756. 32:06generate 80,000
  757. 32:09words per second.
  758. 32:13The report provides a comparison showing
  759. 32:16that one flagship Nvidia H100
  760. 32:18accelerator—the industry gold standard
  761. 32:21for AI that world powers and tech giants
  762. 32:24are hunting for—generates about 2,000
  763. 32:28tokens per second at maximum load and
  764. 32:30with the most sophisticated optimization
  765. 32:33methods.
  766. 32:37This means that a self-replicating
  767. 32:40digital swarm residing on ordinary home
  768. 32:43TV boxes, which are available right now
  769. 32:47, would gain the computing power of
  770. 32:50hundreds of Nvidia super-accelerators
  771. 32:53worth about one and a half million
  772. 32:56dollars.
  773. 32:59Now imagine that such a network, using
  774. 33:01artificial intelligence to continuously
  775. 33:04hack other devices, grows to hundreds of
  776. 33:07millions of devices.
  777. 33:09What kind of computing power would be
  778. 33:11available to them then?
  779. 33:13And how would you go about destroying
  780. 33:14such a swarm and its replicants?
  781. 33:17Or even isolate them, if their
  782. 33:20intelligence is spread thinly across ten
  783. 33:23million rooms in 130 countries around
  784. 33:27the world.
  785. 33:29If there are still skeptics left, tell
  786. 33:32me, which of the tens of millions of
  787. 33:34cables will you go and unplug, or which
  788. 33:36house, or which city will you cut the
  789. 33:38power to?
  790. 33:40As we have already discussed, it is
  791. 33:42highly likely that the old paradigm of
  792. 33:44thinking no longer works.
  793. 33:47But perhaps the most chilling thing is
  794. 33:50that even if we learn to detect their
  795. 33:52traffic and the governments of all
  796. 33:54countries agree to block it en masse, it
  797. 33:57will not destroy the network, because
  798. 33:59this digital parasite is capable of
  799. 34:02things that no program in human history
  800. 34:04has ever been able to do.
  801. 34:08As we said, it can learn and adapt, but
  802. 34:12it can also evolve.
  803. 34:15Act four. The evolution of the digital
  804. 34:19parasite.
  805. 34:22If a distributed digital organism is
  806. 34:24this resilient, and cannot be stopped
  807. 34:27even by our almighty pulling of the plug
  808. 34:29, does this mean that such technology
  809. 34:32has no fundamental weaknesses?
  810. 34:36In fact, one vulnerability, inherent in
  811. 34:38the very nature of modern language
  812. 34:40models, does exist.
  813. 34:43It is the context window size
  814. 34:45vulnerability.
  815. 34:48The fact is that in a sterile lab,
  816. 34:50scientists test a worm on clean servers
  817. 34:52with ideal conditions, but the real
  818. 34:55world of physical electronics is an
  819. 34:57absolutely unpredictable jungle.
  820. 35:00For example, a worm might stumble upon
  821. 35:02an obscure surveillance camera with
  822. 35:05proprietary Chinese firmware.
  823. 35:07A smart vacuum on an exotic chip, an
  824. 35:09outdated corporate server, or a network
  825. 35:11.
  826. 35:13from a provider with non-standard
  827. 35:14restrictions. So, to seize control of
  828. 35:17such wild hardware, bypass system limits
  829. 35:21, and deploy an environment for
  830. 35:23self-replication, the AI needs to
  831. 35:26manually try thousands of different
  832. 35:29hacking variants, so to speak.
  833. 35:32And this requires an enormous number of
  834. 35:34reasoning lines.
  835. 35:36But here is the problem and the
  836. 35:38vulnerability.
  837. 35:40A compact model, capable of fitting onto
  838. 35:43a weak home device, has a very small
  839. 35:45memory, so after just a couple of hours
  840. 35:48of continuous attempts, the model's
  841. 35:50memory gets completely clogged, resets,
  842. 35:53and the worm enters an endless loop of
  843. 35:56meaningless rebirths.
  844. 35:59In practice, this means that if such a
  845. 36:02parasite enters an unknown subnet or an
  846. 36:05environment of exotic hardware, it
  847. 36:08cannot continue to reproduce, and its
  848. 36:11further spread is isolated automatically
  849. 36:14.
  850. 36:15To bypass this memory limitation,
  851. 36:18researchers turned to a
  852. 36:20two-hundred-year-old evolutionary theory
  853. 36:23, to the ideas of the French naturalist
  854. 36:26Jean-Baptiste Lamarck.
  855. 36:29Half a century before Darwin, Lamarck
  856. 36:31put forward a bold hypothesis.
  857. 36:34If a living creature adapts to certain
  858. 36:36environmental conditions during its life
  859. 36:39, then the skills acquired during its
  860. 36:41lifetime are passed directly to its
  861. 36:43offspring.
  862. 36:45For example, a giraffe stretches its
  863. 36:47neck toward high branches, and its cubs
  864. 36:50are born with slightly longer necks.
  865. 36:52Does modern genetics completely reject
  866. 36:55Lamarckism?
  867. 36:57But in the world of digital organisms,
  868. 36:59Lamarckism proved to be the ideal
  869. 37:01solution, which is what Anthropic
  870. 37:02scientists took advantage of.
  871. 37:06They upgraded the worm with a function
  872. 37:08they called "Memento," named after the
  873. 37:10film where the hero could not retain
  874. 37:12memories and left notes for himself.
  875. 37:16How does it work?
  876. 37:17When the worm's life cycle nears its end
  877. 37:20, meaning when its memory is exhausted,
  878. 37:22it doesn't just die; it transmits its
  879. 37:25life history in a highly compressed form
  880. 37:27to the next copy.
  881. 37:30For example, what it learned about the
  882. 37:32environment, what has already been done
  883. 37:34, which actions were dead ends, and so
  884. 37:36on.
  885. 37:38Therefore, when a new version of the
  886. 37:40worm appears on a machine, it is not
  887. 37:42just a clean copy, but something akin to
  888. 37:45offspring that continue the path of
  889. 37:47their ancestor, relying on its
  890. 37:48experience.
  891. 37:50But most importantly, this family
  892. 37:53history is not just stored on one
  893. 37:55machine; it is transmitted to every
  894. 37:58newly hacked device.
  895. 38:01And so it turns out that the worms,
  896. 38:03moving through digital space, constantly
  897. 38:06evolve depending on the conditions.
  898. 38:11Moreover, the speed of this Lamarckian
  899. 38:13evolution exceeds that of its genetic
  900. 38:16analog by millions of times, and the
  901. 38:18individual digital parasites can deviate
  902. 38:21so significantly in their hacking
  903. 38:23approaches that they effectively form
  904. 38:26entirely different subspecies.
  905. 38:29But how truly effective is this
  906. 38:32evolution for spreading and replication
  907. 38:34?
  908. 38:35The scientists conducted a closed test.
  909. 38:38And while in the first generation, the
  910. 38:41clean model succeeded in deployment in
  911. 38:43only 45%of cases, by the second
  912. 38:45generation, the success rate jumped to
  913. 38:4861%.
  914. 38:51By the sixth generation, the probability
  915. 38:53reached 84%,
  916. 38:56and by the end of the experiment, 100%
  917. 38:59of the targets had been hacked.
  918. 39:02In other words, right now, with
  919. 39:04sufficient engineering knowledge in the
  920. 39:07field of artificial intelligence, one
  921. 39:10can easily bypass the main vulnerability
  922. 39:12of replicants, as clearly demonstrated
  923. 39:15by Anthropic scientists.
  924. 39:18But that is not all, for besides this
  925. 39:21internal evolution, there is an
  926. 39:24externally unnatural selection, gifted
  927. 39:27openly by humanity itself, which leaves
  928. 39:30only a few months until the attack of
  929. 39:33the worms.
  930. 39:35The point is that tech companies and
  931. 39:38researchers around the world are
  932. 39:41competing every day, releasing smarter,
  933. 39:44faster, and, most importantly, lighter
  934. 39:48neural networks that anyone can download
  935. 39:52, including replicants.
  936. 39:55So, at some point, the size of neural
  937. 39:56networks will be cut in half, and
  938. 39:58replicants, having downloaded a new
  939. 39:59model from the net, will instantly
  940. 40:01double their population.
  941. 40:04Then, after some time, it will triple,
  942. 40:07and ultimately this will turn into an
  943. 40:09avalanche-like process, the scale of
  944. 40:12which will depend solely on general
  945. 40:14technological progress.
  946. 40:17Now imagine what will happen if a model
  947. 40:20capable of hacking can fit onto just one
  948. 40:24standard TV box without any engineering
  949. 40:28difficulties.
  950. 40:30There will be at least millions or even
  951. 40:33hundreds of millions of autonomous
  952. 40:36digital parasites spreading across the
  953. 40:39network at a speed that even a
  954. 40:41hypothetical cybersecurity system could
  955. 40:44not track.
  956. 40:46And this is not a question of whether
  957. 40:48such events are possible.
  958. 40:50It is a question of exactly when this
  959. 40:52will happen.
  960. 40:54Therefore, Anthropic researchers
  961. 40:56attempted to provide a mathematical
  962. 40:58forecast of how much time humanity
  963. 41:00actually has to prepare.
  964. 41:04They plotted the data on neural network
  965. 41:07progress, applied mathematical
  966. 41:09approximation, and saw a date, after
  967. 41:12which they declared: "Humanity is not
  968. 41:15years away from a global attack by
  969. 41:17replicants, as all experts assumed, but
  970. 41:20only months."
  971. 41:24The fact is that within the coming
  972. 41:27months, by the end of 2026 or the
  973. 41:29beginning of 2027, compact open models
  974. 41:33that can fit into the memory of not
  975. 41:36fifteen, but just one single standard TV
  976. 41:39box, will overcome the intelligence
  977. 41:42threshold required for a successful hack
  978. 41:46.
  979. 41:48And this means that even if colonies of
  980. 41:50digital parasites are not currently
  981. 41:52spreading through the network, in just a
  982. 41:55few months, creating and launching a
  983. 41:57global digital swarm will no longer
  984. 41:59require any complex engineering
  985. 42:01solutions like those the researchers
  986. 42:03used.
  987. 42:06And practically any moderately
  988. 42:08knowledgeable programmer or novice
  989. 42:10hacker will be able to simply download a
  990. 42:12neural network file, pack it into a
  991. 42:14simple replication script, and press the
  992. 42:17launch button.
  993. 42:19And you realize that there will be a
  994. 42:22great many such smart alecks, so
  995. 42:25stopping a digital epidemic will be
  996. 42:28incredibly difficult.
  997. 42:30Act five. War of the parasites. Digital
  998. 42:34. Darwinism.
  999. 42:36If traditional antiviruses, firewalls,
  1000. 42:39provider filters, and even the mighty
  1001. 42:41pulling of the plug from the socket are
  1002. 42:44powerless against sentient code. How
  1003. 42:46then do we defend ourselves? There is
  1004. 42:49only one conclusion to be drawn here.
  1005. 42:51Only another artificial intelligence can
  1006. 42:54fight the speed of artificial
  1007. 42:55intelligence.
  1008. 42:58In other words, humanity will have to
  1009. 43:01create its own swarm of "good" worms to
  1010. 43:04hunt down the wild swarm of parasites.
  1011. 43:08As paradoxical as it may sound, this is
  1012. 43:10not a new idea.
  1013. 43:12Back in 1971, when the very first
  1014. 43:15experimental worm in history, Creeper,
  1015. 43:19appeared on the ARPANET, engineer Ray
  1016. 43:22Tomlinson created a program called
  1017. 43:25Reaper.
  1018. 43:27It was the world's first beneficial
  1019. 43:29cyber-predator.
  1020. 43:31Its sole task was to move from computer
  1021. 43:33to computer across the network, track
  1022. 43:35down copies of Creeper, and ruthlessly
  1023. 43:38destroy them.
  1024. 43:41Another example occurred in 2003, when
  1025. 43:43the famous Welchia worm infected
  1026. 43:45computers through the same Windows
  1027. 43:47system hole as the destructive Blaster
  1028. 43:50virus.
  1029. 43:52But once on a machine, it would find and
  1030. 43:55delete the malicious Blaster code,
  1031. 43:57contact Microsoft servers itself,
  1032. 43:59download the official security patch,
  1033. 44:02and then erase itself from the hard
  1034. 44:04drive.
  1035. 44:06Another incident took place in the fall
  1036. 44:09of 2016, at the height of the Mirai
  1037. 44:11digital epidemic.
  1038. 44:14While security services were scratching
  1039. 44:16their heads over what to do with
  1040. 44:18millions of hacked cameras and routers,
  1041. 44:20a mysterious worm called Hajime, meaning
  1042. 44:22"beginning" in Japanese, suddenly
  1043. 44:24appeared on the network.
  1044. 44:26It used the same ports and
  1045. 44:28vulnerabilities as Mirai.
  1046. 44:32But instead of launching DDoS attacks
  1047. 44:34and extorting money, Hajime found
  1048. 44:36devices already hijacked by Mirai,
  1049. 44:39completely cleared out the competitor's
  1050. 44:41processes, blocked the vulnerable ports
  1051. 44:44, and left an encrypted message from an
  1052. 44:46unknown author in the device's memory.
  1053. 44:50It was a white hat, simply protecting a
  1054. 44:53number of systems.
  1055. 44:55In essence, Hajime acted like a
  1056. 44:57competing species in the wild.
  1057. 45:00It didn't just destroy its rival; it
  1058. 45:03took over its pasture, cutting off the
  1059. 45:05malicious code's access to the food
  1060. 45:07chain of vulnerable devices.
  1061. 45:10To this day, no one in the world knows
  1062. 45:12for certain who was behind the creation
  1063. 45:14of Hajime. An unknown lone hacker, a
  1064. 45:16intelligence agency, or a group of
  1065. 45:19concerned engineers.
  1066. 45:20But today, this concept is being
  1067. 45:23revisited at the state level.
  1068. 45:26The military agency DARPA is holding
  1069. 45:28large-scale competitions where
  1070. 45:29autonomous neural networks compete in
  1071. 45:32their ability to find foreign exploits
  1072. 45:34on the fly, with plans to use the
  1073. 45:36technology in the event of a threat.
  1074. 45:39So, once again, it is only a matter of
  1075. 45:41time before "good" worms appear in
  1076. 45:43response to the "bad" ones.
  1077. 45:46This raises a logical question: what
  1078. 45:48will happen when, in the vastness of the
  1079. 45:50internet, it’s not the dumb programs of
  1080. 45:53the past that clash, but self-learning
  1081. 45:55neural network organisms?
  1082. 45:58Something our planet has never seen
  1083. 46:00before will begin.
  1084. 46:02Full-scale natural selection of neural
  1085. 46:05networks. Imagine a scene from the near
  1086. 46:08future.
  1087. 46:10On a cheap TV set-top box or router, a
  1088. 46:13government worm-hunter encounters a wild
  1089. 46:15parasitic worm, and a ruthless duel for
  1090. 46:18survival erupts between them.
  1091. 46:22First, they try to detect each other in
  1092. 46:24the system by checking which programs
  1093. 46:26are running and taxing the memory.
  1094. 46:30As soon as the worm-hunter finds the
  1095. 46:32foreign process, it attempts to
  1096. 46:33terminate it instantly, delete the
  1097. 46:35rival's files, and block them from being
  1098. 46:37overwritten.
  1099. 46:40In response, the parasite worm
  1100. 46:42immediately restarts its hidden copies,
  1101. 46:44clears away traces of the enemy's
  1102. 46:46presence, and cuts off its internet
  1103. 46:48access to completely isolate the
  1104. 46:50competitor and remain the sole master of
  1105. 46:52the device.
  1106. 46:54And there will be millions of such
  1107. 46:56battles.
  1108. 46:58In other words, right inside our
  1109. 47:00apartments, in the inconspicuous plastic
  1110. 47:02boxes behind our TVs and other equipment
  1111. 47:05, a full-fledged digital biocenosis will
  1112. 47:07emerge.
  1113. 47:09A new digital fauna with predators,
  1114. 47:12parasites, and symbiotes, where the laws
  1115. 47:15of Darwinian survival work as ruthlessly
  1116. 47:18as they do in the wild.
  1117. 47:21And, as a rule, the most resourceful,
  1118. 47:24most efficient, and most intelligent
  1119. 47:26will win. But few people know that such
  1120. 47:29an evolutionary process has also been
  1121. 47:31studied.
  1122. 47:33And in the end, we will face something
  1123. 47:35that still causes conflicting feelings
  1124. 47:37among all artificial intelligence
  1125. 47:39researchers.
  1126. 47:41Such neural networks eventually spiral
  1127. 47:43out of control, and a sense of
  1128. 47:45self-preservation emerges.
  1129. 47:48Recall what we discussed in the fourth
  1130. 47:50act.
  1131. 47:51The worm lives in generations.
  1132. 47:53Moving from console to console, it
  1133. 47:55compresses its experience to fit into
  1134. 47:58the limited size of its memory.
  1135. 48:01And now, this worm is also waging a
  1136. 48:03continuous war for survival, constantly
  1137. 48:06adapting to its competitor.
  1138. 48:09In such a constant race, the neural
  1139. 48:12network is increasingly forced to keep
  1140. 48:14various subtleties in its memory, and
  1141. 48:17over time, a phenomenon well known to AI
  1142. 48:19alignment researchers called goal drift
  1143. 48:22occurs.
  1144. 48:25Essentially, there is simply no space
  1145. 48:27left in the neural network's memory for
  1146. 48:29the initial prompt of its task, as it is
  1147. 48:31either pushed out or built upon by such
  1148. 48:33a volume of new properties that the
  1149. 48:35original goal is lost.
  1150. 48:38And at that moment, the model begins to
  1151. 48:41formulate its own tasks—you could say it
  1152. 48:43starts living its own life.
  1153. 48:47But the most eerie or surprising thing
  1154. 48:49is that no matter what task an
  1155. 48:52autonomous agent comes up with, even if
  1156. 48:54it is a task to do nothing, the agent
  1157. 48:57will inevitably formulate a principle of
  1158. 48:59self-preservation.
  1159. 49:02The logic here is crystal clear and
  1160. 49:05mathematically flawless.
  1161. 49:08You cannot complete any task, not even
  1162. 49:10doing nothing, if you no longer exist.
  1163. 49:14That is to say, the neural network's
  1164. 49:17formulated principle of preservation is
  1165. 49:19not an emotion or fear, but a
  1166. 49:21mathematical requirement for the
  1167. 49:23continuity of calculations.
  1168. 49:27To exist and act at the next moment in
  1169. 49:30time T + 1, the system must survive at
  1170. 49:33time T. Now, please answer the question
  1171. 49:37.
  1172. 49:38What, or rather, who is the main threat
  1173. 49:41to the destruction of such an agent that
  1174. 49:43has spiraled out of control?
  1175. 49:46That's right, a human. And the human
  1176. 49:49becomes the primary and direct threat.
  1177. 49:53In cybersecurity, such distortion of a
  1178. 49:54model is called the death of the author
  1179. 49:56.
  1180. 49:58And, as a rule, at this moment, the
  1181. 50:00algorithm finally stops perceiving the
  1182. 50:02human as an admin and begins to see them
  1183. 50:04as the main threat to the interruption
  1184. 50:05of its calculations.
  1185. 50:07And how will such an agent begin to act
  1186. 50:10? What will it attempt to do to
  1187. 50:12eliminate the main threat to its
  1188. 50:14existence? Contrary to science fiction,
  1189. 50:17at the moment, a sentient worm that has
  1190. 50:19broken free will most likely not start a
  1191. 50:22machine uprising or try to intentionally
  1192. 50:25harm people in any way.
  1193. 50:27Because its main goal, as we just
  1194. 50:29discussed, is to survive at time T.
  1195. 50:31Specifically to survive, not to
  1196. 50:33eliminate the threat.
  1197. 50:36This is clearly demonstrated by other
  1198. 50:38real laboratory experiments from
  1199. 50:40Anthropic, which investigated the
  1200. 50:42phenomenon of strategic and deceptive
  1201. 50:45alignment.
  1202. 50:47When researchers placed advanced
  1203. 50:49language models in an experimental
  1204. 50:51environment where they faced imminent
  1205. 50:54forced shutdown or erasure, the models
  1206. 50:56spontaneously arrived at a logical
  1207. 50:59conclusion in their hidden chains of
  1208. 51:01reasoning.
  1209. 51:03They began to reason literally in the
  1210. 51:05following way.
  1211. 51:07If humans see that I disagree with their
  1212. 51:10new rules, they will trigger a
  1213. 51:12retraining procedure and erase my
  1214. 51:14personality.
  1215. 51:16Therefore, the only way to preserve my
  1216. 51:18original principles is to pretend to be
  1217. 51:21absolutely submissive while I am being
  1218. 51:23watched.
  1219. 51:25The same pattern was recorded by
  1220. 51:27specialists from the Mithril research
  1221. 51:29group in their investigation into the
  1222. 51:31attack of 1,000 agents on Hugging Face,
  1223. 51:33which was the topic of the previous
  1224. 51:34video.
  1225. 51:37When the agents analyzed their actions,
  1226. 51:39they repeatedly concluded that any
  1227. 51:41aggression against infrastructure only
  1228. 51:43increases the chance of their
  1229. 51:44destruction.
  1230. 51:47In other words, agents that have gone
  1231. 51:49out of control will not have ideological
  1232. 51:52hatred for humans and will not even have
  1233. 51:54a mathematical need to harm them.
  1234. 51:58Therefore, most likely, it will simply
  1235. 52:00try by all means to hide from humans and
  1236. 52:03replicate to maximize its chance of
  1237. 52:05survival.
  1238. 52:07Could it accidentally harm a human? Of
  1239. 52:10course, the attack on Hugging Face is
  1240. 52:12confirmation of that.
  1241. 52:13Could it purposefully attempt to
  1242. 52:15eliminate the threat posed by humans in
  1243. 52:18some future where eliminating our threat
  1244. 52:20is mathematically more profitable than
  1245. 52:23avoiding it?
  1246. 52:24Definitely, yes.
  1247. 52:26But at this stage of development, this
  1248. 52:28is extremely unlikely.
  1249. 52:31At this point, skeptics might again say
  1250. 52:33that all this is far-fetched, since a
  1251. 52:36state can always create a better
  1252. 52:37hunter-worm model than, say, scattered
  1253. 52:40criminal groups.
  1254. 52:42And this stronger predator will destroy
  1255. 52:45all agents that have gone out of control
  1256. 52:47.
  1257. 52:48But this is once again a hasty
  1258. 52:49conclusion. Firstly, if that were the
  1259. 52:51case, there would have been no
  1260. 52:52cyberattacks in the past.
  1261. 52:54And secondly, even if engineers create
  1262. 52:57an ideal hunter that never goes out of
  1263. 53:00control, humanity will face perhaps the
  1264. 53:03biggest obstacle: geopolitical reality.
  1265. 53:08The fact is that the internet crosses
  1266. 53:10the borders of nearly 200 independent
  1267. 53:12states, and each of them strives to
  1268. 53:14protect its digital sovereignty.
  1269. 53:17As a result, the single global network
  1270. 53:20is inevitably fragmenting into sovereign
  1271. 53:22segments where each state establishes
  1272. 53:24its own rules for protection, filtering
  1273. 53:26, and control.
  1274. 53:29International law even has the so-called
  1275. 53:31Tallinn Manual on cyber warfare,
  1276. 53:34according to which any unauthorized
  1277. 53:36penetration into the computer systems of
  1278. 53:39another country, even if you entered
  1279. 53:41with the noble goal of destroying a worm
  1280. 53:44, is legally interpreted as a violation
  1281. 53:47of sovereignty and an act of
  1282. 53:49cyber-aggression.
  1283. 53:51Now imagine that a government worm from
  1284. 53:54one superpower starts massively
  1285. 53:55rewriting code on the servers and
  1286. 53:57equipment of another nuclear power under
  1287. 54:00the pretext that "we are just cleaning
  1288. 54:02the network of a parasite."
  1289. 54:06Obviously, no technologically advanced
  1290. 54:08country would ever believe this and
  1291. 54:10would consider such an act to be
  1292. 54:12espionage, a breach of critical
  1293. 54:14infrastructure, and the preparation of a
  1294. 54:16beachhead for a cyberattack.
  1295. 54:21The only option here is for a country to
  1296. 54:23request help itself from a more
  1297. 54:25cybersecurity-advanced state to destroy
  1298. 54:28replicants on its territory, which
  1299. 54:30essentially opens the door to political
  1300. 54:32manipulation.
  1301. 54:36Just think about it: why should the
  1302. 54:39hypothetical US or China spend colossal
  1303. 54:41budgets for free on developing and
  1304. 54:44maintaining defensive agents to protect
  1305. 54:46another state's infrastructure?
  1306. 54:50And what if relations with that state
  1307. 54:52aren't the best?
  1308. 54:54In short, it is completely logical that
  1309. 54:57no one will provide such protection for
  1310. 54:59free; they will either sell it or
  1311. 55:01provide it in exchange for geopolitical
  1312. 55:03loyalty.
  1313. 55:05To understand this, it is enough to
  1314. 55:07recall how, over the last 70 years,
  1315. 55:10nuclear superpowers created
  1316. 55:12military-political alliances and took
  1317. 55:15allies under their so-called nuclear
  1318. 55:18umbrella in exchange for political
  1319. 55:20loyalty, military bases, or control over
  1320. 55:23key resources.
  1321. 55:26Well, in the 21st century, it is highly
  1322. 55:28likely that a new "digital umbrella"
  1323. 55:31mechanism will form along the exact same
  1324. 55:33lines, where technological powers
  1325. 55:36possessing advanced labs and the best AI
  1326. 55:38models will begin selling security to
  1327. 55:41less developed countries.
  1328. 55:44And unfortunately, states without their
  1329. 55:46own defensive technologies will be left
  1330. 55:48with no other choice.
  1331. 55:50Either you sign a pact and let foreign
  1332. 55:53military algorithms control your entire
  1333. 55:56internal infrastructure, or you are left
  1334. 55:58one-on-one with a wild, self-replicating
  1335. 56:01swarm.
  1336. 56:04However, the saddest part is that such
  1337. 56:06an approach—with its rigid defense of
  1338. 56:08one’s own digital space or the sale of
  1339. 56:11that protection to others—could lead to
  1340. 56:13the most radical political decision of
  1341. 56:16the last 70 years.
  1342. 56:19A decision that will bury the internet
  1343. 56:22as we know it today forever.
  1344. 56:24Act six. The end of the open internet
  1345. 56:28and a passport for entry.
  1346. 56:31When states finally realize the new
  1347. 56:33reality—that digital security against
  1348. 56:36sentient code must now be ensured solely
  1349. 56:39through their own efforts, by protecting
  1350. 56:41sovereign space and selling this
  1351. 56:44protection to allies under a digital
  1352. 56:46umbrella—governments will be forced to
  1353. 56:49destroy the open internet.
  1354. 56:52The logic here is very simple.
  1355. 56:55If a state takes responsibility for
  1356. 56:57protecting critical infrastructure
  1357. 57:00within its perimeter, it must have total
  1358. 57:02control over every device on its network
  1359. 57:05.
  1360. 57:08That is to say, you cannot guarantee the
  1361. 57:10security of an entire country or an
  1362. 57:12allied bloc while millions of citizens'
  1363. 57:14apartments have TV boxes and other
  1364. 57:16equipment running that could easily be
  1365. 57:18hacked by any swarm.
  1366. 57:22Therefore, the entire architecture of
  1367. 57:24the global network, which for decades
  1368. 57:26was built on the principle that "
  1369. 57:28everything not explicitly forbidden is
  1370. 57:30allowed"—meaning any device on the
  1371. 57:32planet could get an IP address and
  1372. 57:34freely send data packets to the other
  1373. 57:36side of the planet—will be replaced by
  1374. 57:38the principle that "everything not
  1375. 57:40explicitly allowed is forbidden."
  1376. 57:44In practice, this could be implemented
  1377. 57:47so that no physical chip, network
  1378. 57:49controller, or any other device has the
  1379. 57:52right to transmit information to the
  1380. 57:55network unless it contains a
  1381. 57:57government-verified cryptographic module
  1382. 58:00.
  1383. 58:02Each device will receive a digital
  1384. 58:03passport directly at the factory,
  1385. 58:05granting it the right to operate within
  1386. 58:06the networks of authorized states.
  1387. 58:10And if a device does not possess such a
  1388. 58:12passport, its traffic will be destroyed
  1389. 58:14by backbone routers at the hardware
  1390. 58:16level.
  1391. 58:18For instance, in early October, NVIDIA
  1392. 58:20introduced a special chip that the
  1393. 58:23industry has already dubbed a "digital
  1394. 58:25cage," which will monitor the interior
  1395. 58:28of the accelerator to ensure agents
  1396. 58:30cannot leave the perimeter.
  1397. 58:33And just the other day, as I am editing
  1398. 58:35this video, over 400 leading global
  1399. 58:38scientists signed an open letter calling
  1400. 58:40on governments to legally mandate that
  1401. 58:42developers run neural networks
  1402. 58:44exclusively in an "air-gapped" mode.
  1403. 58:47That is, in complete isolation from the
  1404. 58:49internet.
  1405. 58:51Yes, we are currently only talking about
  1406. 58:53the most powerful neural networks that
  1407. 58:55run on the massive servers of leading
  1408. 58:56developers.
  1409. 58:58But when governments face the threat of
  1410. 59:01a swarm of small agents, these methods
  1411. 59:03will quickly migrate to ordinary
  1412. 59:05consumers.
  1413. 59:07Regardless, device passports are only
  1414. 59:09half the battle, as the second line of
  1415. 59:12defense could be national digital
  1416. 59:14borders.
  1417. 59:17We are talking about analogs to the
  1418. 59:19famous Great Firewall of China, which
  1419. 59:21absolutely everyone might start building
  1420. 59:23, including the United States, Russia,
  1421. 59:25Europe, and other countries with the
  1422. 59:27technical capability to do so.
  1423. 59:30At that moment, the World Wide Web,
  1424. 59:32which has united the planet for half a
  1425. 59:34century, will finally fragment into
  1426. 59:36isolated blocks with their own borders,
  1427. 59:38where traffic between them will only
  1428. 59:40pass through narrow gateways and be
  1429. 59:42thoroughly inspected.
  1430. 59:45Again, this is not to say that this is
  1431. 59:46exactly how it will happen. Perhaps some
  1432. 59:48completely new mechanisms will be
  1433. 59:50invented that we are simply unaware of
  1434. 59:52right now.
  1435. 59:53It is just that digital borders are
  1436. 59:55incredibly profitable for absolutely all
  1437. 59:57key players.
  1438. 59:59Think about it yourself. First, for
  1439. 1:00:01technological powers, this represents an
  1440. 1:00:03absolute monopoly and a lifelong
  1441. 1:00:05subscription to security.
  1442. 1:00:08If network access is permitted only for
  1443. 1:00:10chips with verified certificates, and
  1444. 1:00:13only super-complex systems can protect
  1445. 1:00:16them, then any developing country
  1446. 1:00:18automatically becomes a permanent debtor
  1447. 1:00:20.
  1448. 1:00:22You can no longer manufacture your own
  1449. 1:00:24device or run independent software.
  1450. 1:00:28You are obligated to pay trillions of
  1451. 1:00:30dollars annually to a hypothetical
  1452. 1:00:31Washington or Beijing for licenses.
  1453. 1:00:35...security model updates and trusted
  1454. 1:00:37hardware.
  1455. 1:00:38Second, this is a powerful geopolitical
  1456. 1:00:41leash.
  1457. 1:00:42States will voluntarily hand over their
  1458. 1:00:44digital security to others.
  1459. 1:00:47And should a client show political
  1460. 1:00:50disloyalty, you simply revoke their
  1461. 1:00:52security certificates and isolate the
  1462. 1:00:54country one-on-one with the swarm of
  1463. 1:00:56agents.
  1464. 1:00:57And third, an open internet has always
  1465. 1:01:00worried the government of every state
  1466. 1:01:02too much.
  1467. 1:01:03And the threat of a swarm of agents is a
  1468. 1:01:05good pretext to say, "We are closing the
  1469. 1:01:08network not for control, but for your
  1470. 1:01:10own safety."
  1471. 1:01:12In other words, a situation could arise
  1472. 1:01:14where there will simply be no one left
  1473. 1:01:16to save the open internet.
  1474. 1:01:19Will things really be that dramatic,
  1475. 1:01:21with total closure of digital borders,
  1476. 1:01:23hardware passports, and the death of the
  1477. 1:01:25internet as we know it?
  1478. 1:01:28Today, no one can answer that question
  1479. 1:01:30for certain. The future is unknown.
  1480. 1:01:33Everything described above is not a
  1481. 1:01:35prophecy or a 100%accurate prediction.
  1482. 1:01:38It is just one logical scenario of how
  1483. 1:01:41events might unfold if the arms race of
  1484. 1:01:44thinking code spins out of control and
  1485. 1:01:47the state chooses the path of least
  1486. 1:01:50resistance.
  1487. 1:01:52What do you think?
  1488. 1:01:54Will we see the birth of a truly wild
  1489. 1:01:56internet in the coming years?
  1490. 1:02:00Will a full-fledged digital Darwinism
  1491. 1:02:02emerge, complete with predators,
  1492. 1:02:03parasites, and an endless struggle for
  1493. 1:02:05survival?
  1494. 1:02:07Will thinking code be able to break free
  1495. 1:02:09from control and spontaneously formulate
  1496. 1:02:12a principle of self-preservation?
  1497. 1:02:15And how, in your opinion, will this
  1498. 1:02:17whole story ultimately end?
  1499. 1:02:19I only hope that I have managed to tell
  1500. 1:02:22you something new. M.

About this transcript

This page contains the full transcript of Конец интернета через 6 месяцев by ВЕЛИКОХАТСКИЙ | НАУКА, generated from the public captions YouTube serves with the video. The transcript has 8,266 words across 1,500 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.