Gemini Live Avatars — Transcript

Full transcript

1. 0:00Welcome. I'm Jennifer. I'm ready to dive
2. 0:02into the latest F1 action with you.
3. 0:05>> Hey Jennifer, who won the latest Grand
4. 0:06Prix?
5. 0:07>> George Russell won the latest Grand Prix
6. 0:09in Azerbaijan. He had a fantastic race,
7. 0:12holding off Max Verstappen to take the
8. 0:14win.
9. 0:14>> So, where did Kimi Antonelli come?
10. 0:17>> Kimi Antonelli finished in fifth place
11. 0:19after a really impressive recovery drive
12. 0:21from 16th on the grid.
13. 0:23>> Okay, so that was Google's new live
14. 0:26avatars with Gemini 3.8 live. So, I've
15. 0:29talked about the live models in the
16. 0:30past. These are the bidirectional models
17. 0:32that basically have multimedia going in,
18. 0:35being audio and video. And up until
19. 0:37recently, it could just produce text or
20. 0:39audio back. Well, Google's now updated
21. 0:41that so it can actually have a live
22. 0:43rendering face that kind of lip-syncs
23. 0:46what the actual audio is. So, up until
24. 0:48recently, if you wanted to do any sort
25. 0:50of AI agent with a face that could
26. 0:52actually hold a conversation, you would
27. 0:54generally sort of stitching together
28. 0:56four different things. You had some kind
29. 0:57of speech to text, you had an LLM, you
30. 1:00had a TTS model creating the voice back,
31. 1:03and then there was some sort of avatar
32. 1:04service on the end doing lip-sync. And
33. 1:06the challenge has always been with that
34. 1:08kind of thing is that every one of those
35. 1:10points adds a lot of latency. And by the
36. 1:12time the mouth starts moving, the
37. 1:14conversation has often already started
38. 1:16to fall apart. Now, I won't say that
39. 1:18Google's got this perfect, but what
40. 1:19they've done is pretty impressive.
41. 1:22They've made their Gemini 3.8 live
42. 1:24avatar system generally available. And
43. 1:26there are actually a few caveats to
44. 1:27that, which I'll mention later on in the
45. 1:29video. But this whole thing basically
46. 1:31collapses down those four models that
47. 1:33you had to take care of yourself into a
48. 1:35single stream. It can listen, it can see
49. 1:38what's on your screen, it can call tools
50. 1:40in the background. As you can see in the
51. 1:42example I just showed, it was using
52. 1:44Gemini search to basically get the
53. 1:46latest results from Saturday's Grand
54. 1:48Prix. And the cool thing here is it can
55. 1:50do all of this in 97 languages. So, I've
56. 1:53been playing with this for a while, and
57. 1:55I built some demos that I'm going to go
58. 1:56through in a second and show you. But
59. 1:58let's put this into a little bit of
60. 1:59context. So, while we're still yet to
61. 2:01see a Gemini Pro model come out now for
62. 2:04I think it's about 7 months, this has
63. 2:06still been a pretty busy week for Google
64. 2:08on the audio side. So, in my last video
65. 2:10I talked about the whole new TTS system
66. 2:12that they launched with the voice
67. 2:14cloning, with those sorts of things. And
68. 2:16actually before that, they updated
69. 2:17Gemini 3.8 live and live extended
70. 2:20thinking. And now they've released live
71. 2:22avatar. And like I said in the last
72. 2:24video, whatever you think about Google
73. 2:26is in the LLM race at the moment, the
74. 2:28multimodal and audio stuff has been
75. 2:31really strong. They've had Omni, live
76. 2:33translate, transcribe, the TTS models,
77. 2:36and now this live avatar. So, I guess
78. 2:38quickly what does generally available
79. 2:40actually mean here? Basically, it's
80. 2:42ready for production on Google Cloud.
81. 2:45But the sad thing here is that not
82. 2:46everything is generally available. You
83. 2:48get a library of pre-built avatars. If
84. 2:50you want a custom avatar, one made from
85. 2:53your own reference photo, your own sort
86. 2:55of voice, that kind of thing, that's
87. 2:56behind an allow list and a verification
88. 2:58process. And even things like the live
89. 3:01extended thinking version of live is
90. 3:02still in private preview. So, in this
91. 3:04video I'm just going to show what you
92. 3:05could go and do and build right now and
93. 3:08show that in my demos, etc. If you want
94. 3:10to see an example of the custom agents,
95. 3:12they've got a video in there with a demo
96. 3:14of making a custom avatar for an avatar
97. 3:17with a really bad Australian accent that
98. 3:19I think only a bank owned by Crocodile
99. 3:21Dundee would actually want to use. So,
100. 3:23with what they've actually made
101. 3:24generally available in here, you've got
102. 3:27a bunch of different visual avatars that
103. 3:29you can pick from. So, here I've made an
104. 3:30avatar studio, which is basically a
105. 3:33front end app that's running this and
106. 3:36streaming the live avatars through the
107. 3:37back end in here. So, here we can
108. 3:39basically just pick any of the avatars
109. 3:42that we want to. I can then basically
110. 3:44choose a voice that I've got and they've
111. 3:46got a set of voices that we can actually
112. 3:48pick. We can pick the sort of persona
113. 3:51preset that we want. Let's go for a time
114. 3:53traveling tour from 2126, and then we've
115. 3:56got the system instructions. Now,
116. 3:57obviously, the system instructions, you
117. 3:59can fully customize yourself. You can
118. 4:01get all of that set up. And then you can
119. 4:03add in tools if you want. So, here I've
120. 4:05basically got Google Ground Searching
121. 4:07for this. Now, since we're talking to
122. 4:09someone from the future, that actually
123. 4:10might not be good. I will leave that off
124. 4:12for now. And then basically, what I can
125. 4:14do is I can just start it. And then I've
126. 4:17got the choices. Once I start
127. 4:19>> Wow, it is so incredible to finally see
128. 4:22the quaint 21st century in person.
129. 4:25>> Okay, and you can see there that now I'm
130. 4:28talking to Vera. Vera, how are you doing
131. 4:30today?
132. 4:31>> I am doing absolutely wonderful. Thank
133. 4:33you. Exploring this era is just pure
134. 4:36magic. These are actually a perfect
135. 4:39vintage replica of 2026 fashion. So, you
136. 4:42could say I dressed the part to blend in
137. 4:44perfectly.
138. 4:45>> Okay, so Vera, what I want to show is
139. 4:46that we can actually type to you as
140. 4:48well. So, I'm going to ask you a
141. 4:50question in a second, and I only want
142. 4:52you to answer the question when you see
143. 4:53the typed question.
144. 4:55>> Understood. I am ready and waiting for
145. 4:57your typed question. I'm doing
146. 4:59absolutely fantastic. Just so clear the
147. 5:02incredible energy of 2026.
148. 5:06>> You can see there that basically, she
149. 5:07was able to take a type or voice, and
150. 5:10she's still listening to me, aren't you,
151. 5:11Vera?
152. 5:12>> Exactly. I'm processing everything
153. 5:14perfectly, no matter how it reaches me.
154. 5:16>> Okay, I'm going to stop there. So, you
155. 5:19can get a sense from that this is
156. 5:20actually pretty responsive, right? Now,
157. 5:23there is a delay, and obviously, we're
158. 5:25going to get the transcript back ever so
159. 5:27slightly before the actual voice comes
160. 5:30back, which kind of makes sense. But you
161. 5:32can see for something where now we can
162. 5:33go through and pick any of the different
163. 5:35avatars. You can pick photorealistic
164. 5:38avatars, or you can pick sort of
165. 5:40cartoonish avatars in here. And so, the
166. 5:43cool thing here is you can actually make
167. 5:44a bunch of fun demos with this. So, not
168. 5:47only can we do all of this in English,
169. 5:49we can actually come in here and we can
170. 5:51pick different languages. So, if we want
171. 5:54to have a cafe and we want to put it in
172. 5:56a different place, we can actually sort
173. 5:58of make this a conversational app for
174. 6:01teaching us to speak a different
175. 6:03conversation, etc. So, you can see here
176. 6:05we've basically got something where it's
177. 6:08going to teach me Let's try Japanese.
178. 6:11Let's see how that goes. And what I'm
179. 6:13going to do is try and have a
180. 6:14conversation with her. Hi there. Can you
181. 6:16tell me how to order a coffee in
182. 6:18Japanese?
183. 6:25Kohi is coffee?
184. 6:31Okay, so how do I say I would like one
185. 6:33coffee?
186. 6:38How do I pronounce it? Can you give me
187. 6:40the corrections for how I should
188. 6:42pronounce it?
189. 6:50Kohi o kudasi.
190. 6:56Okay, so enough of me embarrassing
191. 6:58myself Nothing going to speak Japanese,
192. 7:00but you can see here that we can
193. 7:02basically set this up to teach us
194. 7:05vocabulary, to do a whole bunch of
195. 7:06different things. And we could change it
196. 7:08so that we start with something that's
197. 7:10really beginner and something that then
198. 7:12gets right through to something much
199. 7:14more advanced in here. Another thing
200. 7:16that you can actually do for fun is that
201. 7:18you can actually have multiple
202. 7:20characters and have them communicate to
203. 7:22each other. So, here we're going to
204. 7:24start a debate over whether pineapple
205. 7:27belongs on pizza or not. And you can see
206. 7:30Moderator, esteemed guests, culinary
207. 7:33purists often mistake rigidity for
208. 7:35tradition, but love and gastronomy
209. 7:39thrives on daring unions. To deny the
210. 7:42pineapple its rightful throne upon the
211. 7:44pizza is to deny the very spirit of
212. 7:46romance, which seeks the unexpected
213. 7:49harmony of sweet and savory. Imagine the
214. 7:52sun-drenched tropics marrying the rustic
215. 7:54warmth of baked dough. It is not just a
216. 7:57topping, Carmen. It is poetry on a
217. 7:59crust.
218. 8:00>> Quickly, it is then. Leo, while I
219. 8:03appreciate your poetic vision, culinary
220. 8:06harmony requires more than just sweet
221. 8:09and savory colliding. From a practical
222. 8:11standpoint, the excessive moisture in
223. 8:14pineapple compromises the structural
224. 8:16integrity of the crust, turning a crisp
225. 8:18base into a soggy mess.
226. 8:21>> You are right, moderator. There is
227. 8:23wisdom in her words that I must respect.
228. 8:26Yet, Carmen, true passion often
229. 8:28withstands a few
230. 8:30>> Okay, I'm going to stop it there, but
231. 8:31you get a sense there of that these two
232. 8:33can actually sort of communicate to each
233. 8:35other. So, you can have some sort of fun
234. 8:37with that. Probably I'd go back and
235. 8:39actually change the system prompt to
236. 8:40make them a little bit more concise for
237. 8:42my demo. And it doesn't stop there. You
238. 8:44can also do things like where because
239. 8:46you can also activate the camera, you
240. 8:49can then hold up things and have a
241. 8:50conversation about them with the avatar.
242. 8:53Now, if you want to see a video of me
243. 8:54actually explaining the code and going
244. 8:56through it a little bit and stuff like
245. 8:58that, I showed you how to get started,
246. 8:59etc. I'm going to put the link for that
247. 9:01in the description. That's going to be
248. 9:02on my second channel. You can go through
249. 9:04and get this all started yourself. Okay,
250. 9:07so just to finish off, I would say that
251. 9:08while this doesn't have the flexibility
252. 9:10that perhaps we would like at the
253. 9:12moment, again, this is Google having to
254. 9:15be conservative with how they do things.
255. 9:17They don't want 500 people uploading
256. 9:20pictures of Trump or making a Trump
257. 9:21avatar or something like that. My guess
258. 9:23is there's going to be quite a number of
259. 9:24safety features around that kind of
260. 9:26avatar system. You can imagine also that
261. 9:28Google's going to be pretty strict about
262. 9:29the different use cases for this. If you
263. 9:31want to try it out, this is in Google
264. 9:33Cloud now. You can actually just come in
265. 9:35and look at the Gemini live avatar. You
266. 9:38can pick from the avatars that they've
267. 9:39actually got set up in here. So, you can
268. 9:41certainly get a demo of this up for a
269. 9:43business or to show people how to use
270. 9:45it, etc. Now, the pricing for this is
271. 9:47going to be pretty pricey coming out. I
272. 9:49think the simple fact that this just
273. 9:51uses so much compute means that we're
274. 9:54probably not going to see this get super
275. 9:55cheap in the really near term. But, it's
276. 9:57definitely going to be interesting to
277. 9:58see where this goes, and I'd love to
278. 10:00hear from people where you see the best
279. 10:02use cases of this are actually going to
280. 10:03be. Let me know in the comments what you
281. 10:05think about those. And hopefully it
282. 10:07won't be too long before we see an open
283. 10:08version of this, perhaps from Nvidia or
284. 10:11from one of the other companies that are
285. 10:12actually releasing open models out there
286. 10:14as well. Anyway, as always, if you found
287. 10:16the video useful, please click like and
288. 10:18subscribe, and I will talk to you in the
289. 10:19next video. Bye for now.