AI Is Merging Video Games Now (here's how) — Transcript

Full transcript

1. 0:00This is real Minecraft running inside of
2. 0:02GTA V with Minecraft TNT blowing up GTA
3. 0:05cars. This is the Halo Warthog driving
4. 0:08around in Minecraft. And this is Call of
5. 0:09Duty Zombies but rebuilt inside of
6. 0:12Minecraft. All three of these were made
7. 0:13with AI coding agents and the developer
8. 0:15behind them has open-sourced the whole
9. 0:17kit called Universal Modder. You simply
10. 0:19point Cloud Code at any game that you
11. 0:21own and describe the mod that you want.
12. 0:22It works out the engine for you, reads
13. 0:24the code, even makes its own models, and
14. 0:26then tests the running game. So, I
15. 0:28decided to put this to the test by
16. 0:30asking it to put Black Ops 3 Zombies
17. 0:31inside of GTA V and add a new perk.
18. 0:35>> [music]
19. 0:39>> So, Universal Modder is built by Rayhan
20. 0:41and he's actually turned this into a
21. 0:42Cloud Code plugin now and it's simply a
22. 0:44set of skills, a command line tool, and
23. 0:45a knowledge base of notes from mods
24. 0:47other agents have already built. Now,
25. 0:49one thing to clear up straight away is
26. 0:50this is called Universal Modder, but it
27. 0:52is pretty clear about its limitations
28. 0:54when it comes to anti-cheat. So,
29. 0:55hopefully there won't be a surge of
30. 0:57aimbots from this. Now, I'll explain
31. 0:59more about how all of this works later,
32. 1:00but first, let's just jump in and try
33. 1:02this. I picked two games from my Steam
34. 1:03library. GTA V story mode is the host
35. 1:06because you can run this with anti-cheat
36. 1:08switched off and Black Ops 3 is going to
37. 1:09be my guest, which means read only, so
38. 1:11the agent will only ever read its files,
39. 1:13but it will never launch it or change
40. 1:15it. In Cloud Code, I used Opus 5.5 and
41. 1:17my prompt simply asked for Black Ops 3
42. 1:19Zombies at the Vinewood sign with
43. 1:21rounds, the Ray Gun off the wall,
44. 1:22Juggernog, the Mystery Box, and one
45. 1:24extra perk machine that you'll see in a
46. 1:26bit. With that prompt sent, my agent
47. 1:28started with recon. It checked my GTA V
48. 1:30version against Script Hook V, which is
49. 1:32the loader that GTA V script mods run
50. 1:33through, then it backed up my saves and
51. 1:35wrote its plan to a mod log file. Then
52. 1:37it started three parallel sub agents,
53. 1:39one for Black Ops 3 real zombie numbers
54. 1:41from the decompiled game scripts and the
55. 1:43wiki, one for our Black Ops 3 names and
56. 1:45sound files, and one to find a finished
57. 1:47GTA save. For the sound files, it
58. 1:49actually opened up Black Ops 3 sound
59. 1:50banks and worked out how they're stored
60. 1:52and wrote a converter so the round
61. 1:54change, the Juggernog jingle, the
62. 1:55kaboom, they're all going to be real
63. 1:57Black Ops 3 sounds pulled from my game
64. 1:59install. When it comes to the models
65. 2:01they used, apparently the tool that
66. 2:02extracts them needs Black Ops 3 running,
67. 2:04which I'd ruled out. So, the machines in
68. 2:06the box are actually look-alikes
69. 2:08generated using F or F generated some
70. 2:10concept art for them. In the future, I
71. 2:12think I would just tell it that it's
72. 2:13okay to launch the game to get the real
73. 2:15models. Once all of that was done, it
74. 2:16then got started writing about 4,000
75. 2:18lines of C++. Apparently, GTA V can't
76. 2:21load new models without digging into its
77. 2:22own archives, so the mod actually draws
78. 2:24them itself through ReShade using GTA's
79. 2:26camera and depth buffer. At some point,
80. 2:28Claude also realized that GTA V takes
81. 2:30forever to load, so actually split the
82. 2:32mod so the gameplay code could reload in
83. 2:34the running game in about 5 seconds, and
84. 2:36it used that hot reload nearly 50 times
85. 2:38while building this, checking every
86. 2:39feature with screenshots. Finally, after
87. 2:41testing all of these features itself, it
88. 2:43then decided to shoot its own trailer,
89. 2:45but I did take a look at that and it's
90. 2:46not so good. So, let's just jump in and
91. 2:48play this ourselves. So, I started out
92. 2:50in normal story mode, but I just asked
93. 2:51Claude to actually start the experience.
94. 2:53Immediately, you can see we're put into
95. 2:54an arena where we have two perk
96. 2:56machines. One's a custom better stack
97. 2:57one, you'll see what that does in a bit.
98. 2:59We also have a ray gun on the wall that
99. 3:01we can buy. Then there's also a mystery
100. 3:03box over here, but you will notice these
101. 3:04are generated assets, so it has messed
102. 3:06up sort of the Juggernog text that we
103. 3:08have on the machine. It didn't steal the
104. 3:10real ones from Black Ops 3, but
105. 3:11apparently it can do that. Same thing
106. 3:14goes for the zombies here. It's actually
107. 3:15just using real GTA V NPCs, and the gun
108. 3:18is also from GTA V.
109. 3:20It's also overlaying the score that we
110. 3:21can see down there, and the score is
111. 3:22working. We're getting 10 points for
112. 3:24each shot. And we can also see our round
113. 3:26number as well. Luckily, I have just the
114. 3:27exact amount of points for the mystery
115. 3:29box, so we can go ahead and try this
116. 3:30out. You can see it generated a model
117. 3:32from F, so it doesn't actually look like
118. 3:34the real mystery box, and it's also
119. 3:35spinning real Black Ops 3 guns, but then
120. 3:37when you pick it up, it just converts to
121. 3:39a GTA V gun. So, in this case, it's just
122. 3:41the RPG, but that is the end of round
123. 3:44one, and we have successfully done our
124. 3:45first GTA V zombies round. After playing
125. 3:48a few more rounds, I now have enough
126. 3:49points to actually buy the better stack
127. 3:51perk over here, so we'll go ahead and do
128. 3:53that. And this is a perk that should
129. 3:54actually randomly activate while we're
130. 3:56playing. So, we'll see what it does in a
131. 3:57bit. I'm actually going to go ahead and
132. 3:59get a better weapon from the mystery box
133. 4:00because the pistol is starting to be
134. 4:02less effective now. Okay, there we go.
135. 4:04An SMG should do the trick. I suppose I
136. 4:06should also buy Juggernog as well so we
137. 4:08can get a bit more extra health. And why
138. 4:10not try out the ray gun that's on the
139. 4:12wall cuz it actually tried to port the
140. 4:13ray gun mechanics into GTA V. So, you
141. 4:16can see it's sort of a GTA V recreation
142. 4:18of what the ray gun is. But it is using
143. 4:20the real sound effects. I've now gone to
144. 4:22round six and something funny, the
145. 4:24police have actually turned up but
146. 4:25they're shooting at the zombies and not
147. 4:26me. And it appears that I can't shoot
148. 4:28the police. So, the mod has added that
149. 4:30in for me. So, hopefully I'm getting a
150. 4:32little bit of extra help from the LSPD
151. 4:34here.
152. 4:35And there we go. The better stack perk
153. 4:36is activating. It opens up the phone
154. 4:38browser on the better stack YouTube
155. 4:39channel and subscribes. And that gives
156. 4:41us a kaboom. So, you should definitely
157. 4:43subscribe. So, yeah. In my opinion,
158. 4:44that's really not bad for a one-shot
159. 4:46mod. And if you're curious how much all
160. 4:48of this cost, it used about 210 million
161. 4:50tokens. But 99% of those were cash reads
162. 4:52which are pretty cheap. So, API prices,
163. 4:55this comes out to about $68 plus a few
164. 4:57dollars of the foul art that it
165. 4:58generated. I'm actually on the Claude
166. 5:00Max plan so obviously I didn't pay that
167. 5:02and it barely burned through my usage.
168. 5:04So, now let's talk about how this skill
169. 5:05actually knows how to mod so well. The
170. 5:07main part of it is called mod any game.
171. 5:09This is just a loop. It searches the
172. 5:10knowledge base, recon's the game, backs
173. 5:12up your saves, reads the real code,
174. 5:14builds one working slice, checks it in
175. 5:16the game, records it, and then writes a
176. 5:18field note. Breaking down some of those
177. 5:20steps, the recon is done with the CLI
178. 5:21tool which fingerprints the engine, the
179. 5:23anti-cheat, and where your saves live.
180. 5:25The art that it generates comes from
181. 5:26foul. So, sprites, textures, 3D models,
182. 5:29sound, even voice if you need it. It is
183. 5:31worth pointing out here, this was made
184. 5:32by an ex-foul engineer. So, you can
185. 5:34probably ask it to switch this part out
186. 5:35for your own tools. But that's probably
187. 5:37why foul is the default. The next clever
188. 5:39part is the knowledge base. There are
189. 5:40already 117 game notes in there plus 65
190. 5:43on techniques. Mostly written by Claude
191. 5:45Code and Open Code. And my build
192. 5:47actually started by reading the
193. 5:48Minecraft in GTA note, which used the
194. 5:50same GTA version, same loader, and
195. 5:52reshade trick that I did. And by the end
196. 5:54of it, it actually wrote its own note,
197. 5:55which has all of the list of gotchas, as
198. 5:57well as the cause and fix that it found.
199. 5:59And the idea is that when you use this
200. 6:00mod, if you want to help others out, you
201. 6:02just ask it to open up a PR adding these
202. 6:04notes to the knowledge base. So, that is
203. 6:06how Universal Modder works. And you can
204. 6:07see from my real testing that it does
205. 6:09actually seem to create a working mod to
206. 6:11some extent. If you're wondering what
207. 6:12the industry thinks of this, Kotaku
208. 6:14actually called this plague of vibe
209. 6:15coded mashups and says that these
210. 6:17mashups don't hold up as games. And
211. 6:19yeah, most of them are probably just
212. 6:2030-second viral clips that no one is
213. 6:22going to actually play. But I do think
214. 6:24this is really cool. When I was a kid, I
215. 6:26had to learn Java to add stuff to
216. 6:27Minecraft that could now be done in
217. 6:29seconds. Although, I must say I am glad
218. 6:31I did learn to code back then, as I
219. 6:33wouldn't know how today. Let me know
220. 6:34what you think about all of this in the
221. 6:35comments down below. While you're there,
222. 6:37subscribe. And as always, see you in the
223. 6:38next one.