Two AI Gladiators vs Zombies — Transcript

Full transcript

1. 0:00In the previous episode, we met Gary,
2. 0:02our AI gladiator. He was able to fight
3. 0:05up to 16 zombies at the same time thanks
4. 0:08to a fearsome technique, a perfect parry
5. 0:10followed by a backstab.
6. 0:13But despite his prowess, Gary ended up
7. 0:15overwhelmed by the horde.
8. 0:21So today, he won't be fighting alone in
9. 0:24the arena anymore. It's time to get a
10. 0:26little help. And that help is Larry.
11. 0:35Thanks to everyone for all the name
12. 0:37ideas in the comments. It was very
13. 0:40inspiring. Now the only question is
14. 0:43whether they can work together to take
15. 0:45on the Horde.
16. 0:47What brilliant strategies could possibly
17. 0:49emerge from these two incredible
18. 0:52fighters?
19. 0:56For deeper immersion, I completely
20. 0:58reworked the arena's ground system. The
21. 1:01sand is made up of hundred of thousands
22. 1:03of particles that can be moved and
23. 1:05tinted. I also fully overhauled the
24. 1:07physics of the various projections.
25. 1:13A diffusion mechanism lets all the
26. 1:15traces dissipate naturally.
27. 1:18What I really like is that the arena
28. 1:20floor tells a story all on its own.
29. 1:23Now, let's see how our two heroes do in
30. 1:26this all new environment. There's
31. 1:28nothing like a good zombie fight to
32. 1:29break the ice.
33. 1:33After 2 minutes, progress is looking
34. 1:35pretty good. Let's see how the boys are
35. 1:38doing.
36. 1:46Unfortunately, Gary has never had a
37. 1:48friend and he's not quite sure how to
38. 1:50handle it. Even worse, his brain isn't
39. 1:53wired for friendship. Gi will need to
40. 1:56learn trust and love if he's going to
41. 1:57succeed.
42. 1:59I had to rewrite the entire simulation
43. 2:02to introduce the concept of allies and
44. 2:04make the whole system more generic.
45. 2:07Until then, I was using ray casting for
46. 2:09perception. The advantage is that the
47. 2:12number of observations is fixed. But
48. 2:14there are several drawbacks. The number
49. 2:16of inputs is very high with either many
50. 2:19missing values or a huge amount of
51. 2:21redundancy. On top of that, detection is
52. 2:24also quite imprecise.
53. 2:27The new system is based on the proximity
54. 2:29of surrounding entities. The 10 nearest
55. 2:32ones are encoded and fed into the neural
56. 2:34network. Perhaps a mix of the two
57. 2:37systems could be used. This is something
58. 2:39I'm still exploring. Of course, the
59. 2:42rewards also need to be adjusted to
60. 2:44explicitly penalize attacks between
61. 2:46allies. Let's run the training again,
62. 2:48hoping that we can observe the early
63. 2:50signs of collaboration. AI is playing a
64. 2:52growing role in my projects, especially
65. 2:54when it comes to reviewing my code or
66. 2:57researching deep technical topics like
67. 2:59how to adapt PO to a multi- aents
68. 3:02problem. The challenge is that different
69. 3:04AI models excel at different tasks and
70. 3:06subscription are not cheap. That's why I
71. 3:09use Mammoth AAI which brings a wide
72. 3:11range of models together under one
73. 3:13subscription. It gives me much more
74. 3:15flexibility in how I use my credits. For
75. 3:18my projects, I mainly use Deep Search to
76. 3:20quickly get up to speed on a topic or
77. 3:22explore it in more depth. Perplexity is
78. 3:25great for this, delivering detailed,
79. 3:27wellsourced reports in just a few
80. 3:29minutes.
81. 3:31Opus 5.5 excels at coding questions and
82. 3:34explaining research papers, while GPT6
83. 3:36is my go-to for lighter, quicker
84. 3:38research. Mammoth gives me access to all
85. 3:41of these services for $10 a month. You
86. 3:44can also create custom Mammoth for
87. 3:46recurring tasks. For example, I've set
88. 3:48up a code reviewer, a researcher, and an
89. 3:50English translator.
90. 3:52If you want to cut your subscription
91. 3:54costs while accessing all the Frontier
92. 3:56models, try Mammoth starting at $10 a
93. 3:59month when build annually. Thanks again
94. 4:01to Mammoth AAI for their support.
95. 4:04Let's get back to training and see
96. 4:06whether the new reward improves the
97. 4:08situation. This interface lets me
98. 4:10monitor training progress, especially
99. 4:12through the event logger, which provides
100. 4:14a glimpse into the agents capabilities.
101. 4:17After 100 iterations, let's quickly
102. 4:20check whether our two companions are at
103. 4:22each other's throats before we continue
104. 4:24training.
105. 4:31It's obviously still far from perfect,
106. 4:34but things between them already seem
107. 4:36much calmer. We can also see that
108. 4:38they're managing to pull off a few
109. 4:40perfect parries. Let's hope it stays
110. 4:42that way and get back to training.
111. 4:56The perfect parries seem to be pretty
112. 4:58well mastered and we are starting to see
113. 5:00a few backstabs from time to time. Let's
114. 5:02take a closer look.
115. 5:22It's definitely much better already.
116. 5:25Gary and Larry are far less reckless and
117. 5:27managed to block many more hits.
118. 5:29However, they are quite uncomfortable
119. 5:31when they get close. It feels like they
120. 5:33are getting in each other's way.
121. 5:35Clearly, basic safety practices are not
122. 5:38being followed at all and inevitably an
123. 5:42accident eventually happens.
124. 5:48Despite a few minor mistakes and thanks
125. 5:50to Larry's courageous sacrifice, they
126. 5:53make it through the wave. That's a good
127. 5:55start, but there's still a long way to
128. 5:58go.
129. 6:22After 1,200 iterations, progress is
130. 6:25significant. Parries are becoming more
131. 6:27frequent, and our gladiators clear the
132. 6:29wave without any trouble. However, I
133. 6:32think there's a problem with the
134. 6:33rewards. Nothing really compensate for
135. 6:35the prospect of hurting an ally. Staying
136. 6:38close doesn't do much for the agents
137. 6:40other than giving them a chance to try
138. 6:42to steal each other's points.
139. 6:45To counterbalance this, I decided to add
140. 6:48an assist mechanic. Two situations
141. 6:50qualify as an assist. Attacking a zombie
142. 6:52that has been stunned by an ally or
143. 6:54attacking an enemy that is targeting an
144. 6:57ally. In addition, any elimination
145. 6:59achieved during an assist grants the
146. 7:01same rewards to both agents to force
147. 7:04Gary and Larry to target the same enemy.
148. 7:06I'll start by having them face a single
149. 7:08zombie. That will prevent them from
150. 7:10splitting up at the start of the fight
151. 7:12if the zombies are not grouped together.
152. 7:15Let's take a quick look at where things
153. 7:17stand after 200 iterations.
154. 7:32The agents have turned into a pair of
155. 7:34zombie bashing bullies. It's an original
156. 7:37strategy, though unfortunately it also
157. 7:40involves beating up their allies, and it
158. 7:42isn't very effective at racking up
159. 7:44eliminations. When it come to proximity,
160. 7:47however, it's a resounding success.
161. 7:50Let's see whether we can develop a
162. 7:52slightly more refined approach going
163. 7:53forward.
164. 7:57Soon after, eliminations start chaining
165. 7:59together and the score literally
166. 8:01skyrockets.
167. 8:26The progress really is impressive. The
168. 8:29agents are far more effective at
169. 8:30eliminating threats, and we can see that
170. 8:33they show a great deal of restraint
171. 8:35toward one another.
172. 8:53progress will continue more slowly until
173. 8:56it reaches a plateau after 600
174. 8:58iterations. Let's see how that goes.
175. 9:14The method is now finally honed and the
176. 9:17zombies barely get a chance to do
177. 9:19anything. There is the occasional bit of
178. 9:22friendly fire between Gary and Larry,
179. 9:24but that's surely just good camaraderie.
180. 9:26It's almost become too easy. Let's bump
181. 9:28up the number of zombies they're facing.
182. 9:34Let's see how they fare against four
183. 9:36zombies. They do attempt to cover each
184. 9:39other, but the two fighters are quickly
185. 9:41overwhelmed and eliminated within
186. 9:43seconds.
187. 9:45It's a rather tough start for them, but
188. 9:47it's also a situation they've never
189. 9:49faced before. Let's give them a little
190. 9:52time to get used to this new setup.
191. 9:57Fairly quickly, the agents adapt and the
192. 9:59score shoots back up. However, what's
193. 10:02happening here is a textbook example of
194. 10:05the deviousness of optimization
195. 10:07algorithms. They're a bit like evil
196. 10:09genies. We ask them for something and
197. 10:12they grant it in the absolute worst way
198. 10:15possible. Here's what Gary does when
199. 10:17asked not to hurt his teammate during
200. 10:20fights.
201. 10:27uh
202. 10:30no damage to
203. 10:32and this is not a one-off. It's
204. 10:35systematic.
205. 10:38On top of that, he's actually pretty
206. 10:40good. He's quite capable of controlling
207. 10:42the group of zombies and then stabbing
208. 10:43them one by one. Once we bumped it up to
209. 10:46four zombies, paying the price of
210. 10:48betrayal became more profitable. he
211. 10:50could rack up points four times as fast
212. 10:53rather than share them and risk getting
213. 10:55constantly punished whenever an accident
214. 10:57happened. At that point, I was a bit
215. 11:00stuck because I couldn't find a way to
216. 11:02get the agents to cooperate. My attempts
217. 11:05had even had the exact opposite effect.
218. 11:08The missing piece came from an OpenAI
219. 11:10paper about their famous Dota playing
220. 11:12AI. Of course, the agents had to work
221. 11:15together. And one of the things Open AI
222. 11:17tried was giving them a shared reward.
223. 11:20That should solve my problem. If Gary
224. 11:22takes out an ally, he misses out on a
225. 11:24big reward bonus. With this new system,
226. 11:28after 200 iterations, there were no
227. 11:30dubious strategies. They seem to be
228. 11:33playing by the rules.
229. 11:35However, there is no real attacks or
230. 11:37signs of cooperation, but it's still
231. 11:40very early.
232. 11:50Very quickly the attacks begins to
233. 11:52reappear. But the most interesting part
234. 11:55is yet to come.
235. 12:04Something very cool is happening here.
236. 12:07Gary pushes an enemy towards Larry who
237. 12:09can then eliminate it instantly. In
238. 12:12fact, this is the first glimmer of
239. 12:15teamwork. Finally.
240. 12:30This pattern clearly didn't escape them.
241. 12:33And even though it's still quite
242. 12:34hesitant, the intention is now
243. 12:37unmistakable.
244. 12:56After 1,500 iterations, their mastery is
245. 13:00much better, and they clear the wave
246. 13:02with ease. However, the agents are
247. 13:05clearly not very economical with their
248. 13:07attacks. They just strike widely and
249. 13:09continuously. To force them, to be more
250. 13:12precise, I added an energy pool that
251. 13:14every action draws from. This new
252. 13:17addition clearly didn't sit well with
253. 13:19our two heroes who quickly ran out of
254. 13:22energy and were swept aside.
255. 13:33Adaptation is very quick, however, and
256. 13:35they are becoming much more economical
257. 13:37with their attacks.
258. 13:56a thousand iterations later, they have
259. 13:58become killing machines.
260. 14:02To make things more challenging, I'll
261. 14:04double the energy cost of attacks and
262. 14:06half the regeneration rate. Successfully
263. 14:09performing a perfect parry restores 20%
264. 14:12of the energy.
265. 14:18Clearly, we're going to have to learn to
266. 14:20parry a bit more.
267. 14:27This time, adaptation takes longer.
268. 14:29After 600 iterations, they are mainly
269. 14:32learning to buy their time and operate
270. 14:34more efficiently.
271. 14:47Gary, however, pulls off a super perfect
272. 14:49parry at the end, which is an excellent
273. 14:52signal.
274. 15:01A little later, they clearly grasp the
275. 15:04value of the perfect parry and use it
276. 15:06whenever they run low on energy. The
277. 15:08execution is still a bit hesitant, but
278. 15:10it's an excellent direction.
279. 15:3230 minutes later, the strategy is nearly
280. 15:35flawless. The two cooperate seamlessly
281. 15:38and dispatch eight zombies with ease.
282. 15:41From here on, I'll disable all secondary
283. 15:44rewards and keep only eliminations, so
284. 15:47they stay focused on what matters most.
285. 15:49I'll also gradually add more zombies to
286. 15:51see just how far Gary and Larry can go.
287. 16:45Ha!
288. 17:21That
289. 17:44hooked.
290. 17:53Hat.
291. 18:12Help.
292. 18:27Ha!
293. 19:08Power.
294. 19:34Fire.
295. 20:03Hat.
296. 20:52Getting the two agents to collaborate
297. 20:54was no small feat. Still, I'm very
298. 20:56pleased with the result, and I would
299. 20:59like to try it with far more than just
300. 21:00two fighters. Also, for the next stage
301. 21:03of the project, I'm looking for someone
302. 21:05to create sprites. Anyone with skills in
303. 21:08sprite animation, pixel art, or
304. 21:10otherwise is welcome to contact me on
305. 21:12Discord or by email. All the details are
306. 21:15in the description. A huge thanks to all
307. 21:18the Patreon members for their support
308. 21:20and to Mammoth AI.