Gemini 4 Argon - Out and Tested!! — Transcript

Full transcript

1. 0:00Okay, so Gemini 4 Argon is now out. And
2. 0:04in this video, while I'm going to show
3. 0:05you a few demos, I'm going to really
4. 0:07talk about more about what this model
5. 0:09means and how you should actually think
6. 0:10about this model. So, I'm going to say
7. 0:12this right up front. This is a really,
8. 0:15really, really good model. But, is it
9. 0:18the best model out there? I actually
10. 0:19don't think it is. But, more
11. 0:21importantly, I don't think that that
12. 0:22really matters that much. And I'm going
13. 0:24to talk about why in this video. Cuz
14. 0:26what you're looking at here are some of
15. 0:27the demos that I basically put together
16. 0:29for this. Showing where this model is
17. 0:31genuinely strong. It's not always going
18. 0:34to be the stuff that tops leaderboards.
19. 0:36And in some ways, I think it's actually
20. 0:37more interesting to understand why
21. 0:39Google built this model. And that
22. 0:41requires looking past leaderboards
23. 0:43anyway. So, when Google first announced
24. 0:45Argon, I made a short video about it
25. 0:47then. Sure enough, everyone was
26. 0:48complaining that there weren't many
27. 0:50examples, the model wasn't available,
28. 0:52all that stuff. Totally fair comments,
29. 0:55by the way. So, full disclosure, I've
30. 0:57been an early tester on this model, and
31. 0:59I had to sign quite a few more NDAs than
32. 1:01normal. But, by the time you're watching
33. 1:03this video, Gemini now is part of the
34. 1:06paid tier of the Gemini API. It's going
35. 1:09to be in antigravity. It's going to be
36. 1:11in the Gemini app. It's going to be in
37. 1:13almost every Google surface where
38. 1:16they're putting models. Now, often it's
39. 1:17only going to be available if you're
40. 1:19paying for this model. But, for things
41. 1:21like AI mode and stuff like that, it's
42. 1:23not just available to ultra subscribers,
43. 1:25it's also going to be open to pro. But,
44. 1:27basically, the cool thing here is I can
45. 1:29finally talk about what it's like to
46. 1:31actually use this model. All right, so
47. 1:33where is this model actually strong? For
48. 1:36me, the standout things here is anything
49. 1:38that combines sort of visuals, code, and
50. 1:41media in one kind of task. If you take a
51. 1:44look at something like this, which I
52. 1:46chose to do not in antigravity, but just
53. 1:48in the pure Gemini app, you can see here
54. 1:51I'm basically saying, "Can you build me
55. 1:52a music maker in the style of ReBirth by
56. 1:55Propellerheads? Try to give it an 808, a
57. 1:57909, and a 303, and then put it
58. 2:00together, and make sure to include
59. 2:02filters for effects and a Roland style
60. 2:04step sequencer. Okay, I deliberately
61. 2:07made the prompt here very short, right?
62. 2:09To see, okay, what can this actually
63. 2:11come up with? So, I've got a reference
64. 2:13to something like Rebirth, I've got a
65. 2:15reference to some Roland synthesizers,
66. 2:18drum machines in there, and sure enough,
67. 2:20it went away and produced something
68. 2:23which is really pretty amazing in here.
69. 2:25And if I come in and run this,
70. 2:31>> [music]
71. 2:36>> straight away you can see that it's
72. 2:38built not only a very sophisticated
73. 2:40interface here, but it's also then gone
74. 2:43and actually made the sounds to this and
75. 2:46built a little synthesizer, built
76. 2:48something like an 808 drum machine, a
77. 2:49909, et cetera. And that's a single shot
78. 2:52prompt. Now, before you jump down my
79. 2:54throat about the real world is not about
80. 2:55using just single shot prompts. I agree
81. 2:57with you, right? I just felt that some
82. 2:59of these are actually really good to
83. 3:01look at and get a sense of what the
84. 3:03model can actually do with a limited set
85. 3:06of resources. And let me explain what I
86. 3:09actually mean by that. So, the model is
87. 3:11extremely good at doing things like
88. 3:14Blender and doing things like game
89. 3:16generation, those kinds of things. And
90. 3:19clearly that's come from a lot of the
91. 3:21reinforcement learning environments that
92. 3:23they've trained this model on and done a
93. 3:26lot of work post-training it on. And
94. 3:28it's not just one demo here. If you look
95. 3:30at the demos that Google put out with
96. 3:32the launch themselves, there's a pretty
97. 3:34clear pattern. In one of them, I have
98. 3:35gone basically scripts Blender
99. 3:38animations, generates custom audio,
100. 3:40pipes the whole thing through to an
101. 3:42actual vintage CRT V. In another one of
102. 3:45them, they build a multi-tiered Japanese
103. 3:47temple in the grounds using Blender. And
104. 3:50if you look at that, that's pretty
105. 3:52amazing to see what this thing can do
106. 3:55when it's given an agentic loop and a
107. 3:56proper harness that can use those kinds
108. 3:59of skills. And all of this feels like
109. 4:01it's had a lot of RL on these kind of
110. 4:02pipelines, being things like Blender,
111. 4:053JS, shaders, etc. And those are the
112. 4:08kind of tasks where the model writes the
113. 4:09code, renders something out, looks at
114. 4:11the result, then gradually fixes it.
115. 4:13It's that loop of sort of code, render,
116. 4:16look, fix something up that continuously
117. 4:19seems to be doing well here. Now, in
118. 4:21this video, I don't want to get into
119. 4:22benchmarks too much, but what I do want
120. 4:23to say is that from everything I'm
121. 4:25hearing, this is the biggest model that
122. 4:27they've done pre-training on. You'll
123. 4:28notice in the name, being Argon, that
124. 4:32it's not actually falling into the other
125. 4:34tiers that they've had before, whether
126. 4:36that was Flash, Pro, even Ultra, which
127. 4:39sort of got discontinued. And we do know
128. 4:41for a fact that this is just the first
129. 4:43Gemini 4 model. Now, the big question
130. 4:45people are going to ask, is it as good
131. 4:47as the Anthropic models or the OpenAI
132. 4:49models, or even things like the new Muse
133. 4:51wonder here. So, I'm going to say in my
134. 4:53testing, a lot of this now comes down to
135. 4:57how you're actually using the model. I
136. 4:59still don't think, for example,
137. 5:01antigravity is anywhere near as good as
138. 5:03Claude Code or Codex. And as of the time
139. 5:06recording this, I haven't tried this in
140. 5:07AI Studio build, for example. I've
141. 5:10actually been either using it from the
142. 5:11raw API in antigravity or in the Gemini
143. 5:15app. But clearly, the thing here is that
144. 5:17it's definitely good at a lot of these
145. 5:18multimodal tasks. Now, in the last
146. 5:21video, I did talk about some of the
147. 5:22benchmarks from the Artificial Analysis
148. 5:24Intelligence Index, where this model
149. 5:26scores a 53, which ties it with GPT-6
150. 5:30Astra and Fable 5.1, but quite a bit
151. 5:33behind Opus 5.5 and even the new Sonnet
152. 5:375.5. Added to that in the mix is now
153. 5:39Meta has released Muse wonder, which
154. 5:42also is quite a bit ahead of this model.
155. 5:44But one of the things I will say here is
156. 5:45that the benchmarks don't always tell
157. 5:47the full story, right? This model seems
158. 5:50to be very good at generalization. I
159. 5:52will say it's not the best coding model.
160. 5:55I think the other models are probably
161. 5:56going to do better at different kinds of
162. 5:58coding tasks and for the whole sort of
163. 6:00coding agent game. This model though
164. 6:02really isn't aimed at taking those
165. 6:05models out at their own game. So, I
166. 6:07think if your main job is picking the
167. 6:09strongest model for agentic coding,
168. 6:11you're probably going to look elsewhere.
169. 6:14And I'm sure a lot of AI engineers are
170. 6:15going to complain about things like
171. 6:16that. And that's a fair call, but I
172. 6:18don't think that's why Google built this
173. 6:20model. I don't think Google's actually
174. 6:22trying to be in the game of being a
175. 6:24standalone company that just sells
176. 6:26tokens. If you look at the Google for
177. 6:28business event last week, they basically
178. 6:30announced that they had over 500
179. 6:33companies now that had done more than a
180. 6:35trillion tokens with the Gemini models.
181. 6:38My bet is most of those are actually the
182. 6:39flash models, perhaps even the
183. 6:41flashlight models. What Google was
184. 6:43lacking was a frontier model that they
185. 6:45could basically put out there to their
186. 6:47customers and say, "We also have a very
187. 6:49good model." And that's clearly what
188. 6:51Argon delivers here. It allows their
189. 6:53customers to basically have a credible
190. 6:55option of using this model if they're
191. 6:58already a Google shop or they're looking
192. 7:00to use Google products. Now, Gemini for
193. 7:02Argon gives those people a credible
194. 7:05option without having to leave the
195. 7:07Google stack. And I think in many ways
196. 7:09that's the real customer here. And then
197. 7:11on the consumer product side, Google's
198. 7:13needed a frontier model underneath
199. 7:15everything that it ships. The Gemini app
200. 7:17now has tasks that keep running when
201. 7:20you're away from your computer. It can
202. 7:21connect to all these different apps.
203. 7:23You've got Notebook LM. You've got
204. 7:25things like Anti-gravity. And all those
205. 7:28things are going to benefit not only
206. 7:29from this model, but probably from
207. 7:31distillations of this model to make
208. 7:33other Gemini 4 models. Having Argon
209. 7:36allows the product people at Google to
210. 7:38really focus on building the products
211. 7:40that are working, things like Notebook
212. 7:42LM, and probably improving the ones that
213. 7:44perhaps haven't gotten as much traction
214. 7:47as they would like for things like
215. 7:49anti-gravity and putting this in the
216. 7:51Gemini app, the Google Workspace apps,
217. 7:54etc. So, I think in many ways, basically
218. 7:56Google here is focused on Argon as being
219. 7:58a foundation. And in that sense, this is
220. 8:00clearly a big win for them. While this
221. 8:03year they've lost a lot of the top
222. 8:05talent that came up with a lot of the
223. 8:07ideas for a lot of the early LLMs. Argon
224. 8:10really shows that they've still got
225. 8:12people there that know how to use the
226. 8:13TPUs, know how to spin up a lot of them,
227. 8:16know how to do a lot of pre-training,
228. 8:18know how to do the RL environments, and
229. 8:20know how to put it together to actually
230. 8:23make a frontier level model. So, I think
231. 8:25the best thing is rather than me show
232. 8:27you a lot of demos about what it can do,
233. 8:29what it can't do. Really, at this point
234. 8:31you should be taking your use cases into
235. 8:33either AI Studio or the Gemini app, or
236. 8:36if you want to anti-gravity, and
237. 8:38actually running those things yourself.
238. 8:40We'll say that in the first video, I
239. 8:42talked a lot about the million tokens in
240. 8:44a single response. In practice, it turns
241. 8:46out it's a lot more nuanced than I
242. 8:47originally had thought. The key thing
243. 8:49there is that's not going to be in the
244. 8:52Gemini app. I don't know if it's going
245. 8:54to be in anti-gravity. Really, it's more
246. 8:57for the API, and it's actually a flag
247. 8:59that you need to pass in to the config,
248. 9:02not to actually get it to keep going.
249. 9:04And really, you're only going to benefit
250. 9:06properly from it if you've got a task
251. 9:09that requires multiple hundreds of
252. 9:11thousands of tokens for reasoning. And
253. 9:15while I think that's going to be
254. 9:15interesting to see how people prompt it
255. 9:18to get it to just keep reasoning and
256. 9:20reasoning and reasoning and go into that
257. 9:22sort of loop in its reasoning, I don't
258. 9:24see that that's going to get a lot of
259. 9:26mainstream use. So, I'd still call this
260. 9:28sort of early days. If you're building
261. 9:30your own vertical harnesses and stuff
262. 9:31like that, I probably wouldn't go out
263. 9:33and redesign everything around the
264. 9:34million tokens yet. That said, if you do
265. 9:37have scientific use cases or things like
266. 9:39that where you can actually get really
267. 9:41long reasoning out of it. This is
268. 9:44certainly worth turning on for your long
269. 9:46reasoning tasks and measuring it to see
270. 9:48actually how it plays out for you.
271. 9:50Anyway, to wrap up, Argon is a really
272. 9:52good model. It may not be at the top of
273. 9:53the leaderboards and I'm sure many
274. 9:55people are going to criticize it for
275. 9:56that, but I think the real win for
276. 9:58Google here is just what they themselves
277. 10:01are able to build on it and also their
278. 10:04customers are able to build on it for
279. 10:06different use cases. And the Google
280. 10:08models have always been very good at
281. 10:10generalization for these kind of tasks.
282. 10:13They haven't gone out trying to make the
283. 10:14best coding model or to win every single
284. 10:16benchmark that they're going after. What
285. 10:19they've made is a very, very strong
286. 10:21multimodal model that has a really good
287. 10:24sense of a lot of intelligence that
288. 10:25perhaps was more on the fringe in the
289. 10:28previous models. Anyway, don't forget
290. 10:30that the price currently is $2 in and
291. 10:32$10 out. That's going to last 1 and
292. 10:36honestly, I'm going to say I don't think
293. 10:38you're going to see this model still
294. 10:39around by December 1. We know that this
295. 10:42is not the final checkpoint of this
296. 10:44model. It's still doing a lot of
297. 10:45training, doing a lot of tests how to
298. 10:47improve it. Just like you saw with GPT 6
299. 10:50Soul getting a massive bump when it went
300. 10:52from 6 to 6.1, my guess is that Argon is
301. 10:56also going to get that same kind of bump
302. 10:58in the next month or two. So anyway,
303. 10:59I've put the links to the Google blog
304. 11:01post and the artificial analysis in the
305. 11:03description. If you miss my first video
306. 11:05on Argon, check that out. Let me know in
307. 11:07the comments what you think and whether
308. 11:08you think it really matters that this is
309. 11:10not the number one model. And as always,
310. 11:13if you found the video useful, please
311. 11:14click like and subscribe and I will talk
312. 11:16to you in the next video. Bye for now.