Post-Training Gemini 4 for Real Engineering Work — Melvin Johnson, Google DeepMind — Transcript
Full transcript
- 0:12Hey everyone, good to be here. I'm
- 0:14Melvin and I lead post training for
- 0:16Gemini. So today I thought I'll take you
- 0:18through a little journey of how we post
- 0:20train Gemini 4. And our journey begins
- 0:25by people calling us not a frontier
- 0:27model. Um, so here's an example. You
- 0:30know, people say Gemini is cooked, but
- 0:33GCP is cooking. For all intents and
- 0:36purposes, we believe Deep Mind is no
- 0:38longer a frontier lab. Ethan Malik, who
- 0:41we respect a lot, he says in his own
- 0:44nice way, pretty long gap between
- 0:47launches is what he calls us.
- 0:50And I'm here to tell you we're not dead.
- 0:54We're here. uh and we have launched uh
- 0:59Gemini 4 Argon, we announced Carbon and
- 1:03we launched Carbon all within the last
- 1:0528 days. So we're on a we're you know
- 1:09we're on a run here and that's not it.
- 1:12Uh in the process we actually garnered
- 1:16an apology from Dylan. No, he says
- 1:19official apology to Google. Uh they're
- 1:22actually cooked. They're not cooked. Um
- 1:25Ethan says, you know, one of the most
- 1:27significant aspects of Gemini 4 is the
- 1:30speed at which we're launching things.
- 1:32Um so we've gone 236 days without having
- 1:35a Frontier model and we've had two
- 1:37Frontier models in 28 days. That's a 10x
- 1:40improvement in our speed of shipping.
- 1:43And you know, people call us Google kind
- 1:47of cooked, but Google's not cooked. So
- 1:49it's good um to see see that sentiment.
- 1:52It's glad to be on the other end. It's
- 1:54also good to be on the receiving end of
- 1:56the bad comments because it pushes you
- 1:58to do better.
- 2:00I want to start by playing a little
- 2:03video of a bunch of things that uh
- 2:05Carvin has built.
- 2:39Thank you.
- 2:42Now, kudos to the model for building
- 2:44that video uh itself. So you saw great
- 2:46examples of the model doing much better
- 2:49at 3D model building assets using
- 2:52Blender. There's lots of games in there.
- 2:55The real world use cases, model
- 2:56processing, multimodel documents. Uh but
- 2:59you know, it's not just the uh the
- 3:02demos. Carbon's actually number one on a
- 3:05bunch of real world customer use cases.
- 3:08And one of the things that we're really
- 3:09excited about is, you know, it's not
- 3:11just uh really good on these benchmarks,
- 3:14it's actually much cheaper. Um, so on
- 3:17Cursor Bench, it's the number one model
- 3:20today and it's 3.3 times cheaper than
- 3:22the second best model. On Perplexity's
- 3:25Wander, which is a benchmark for deep
- 3:27research um queries, it's the best
- 3:30model. you know, you saw um competitor
- 3:33models come out were 9x cheaper than the
- 3:36second best model there. And on
- 3:38automation bench, which is Zapier's uh
- 3:41benchmark, again, we're number one while
- 3:43being two times um cheaper per task.
- 3:48And that's not it. I mean, today we
- 3:51announced the new element uh to our
- 3:54Gemini 4 series, and it's not surprise.
- 3:57Uh it's uh hydrogen. Um, so you know,
- 4:02you might have seen the launch earlier
- 4:03today. Gemini 4 hydrogen is where we're
- 4:06trying to bring the Gemini 4
- 4:08intelligence to everyone. It's fast,
- 4:11it's dependable, it's launched on all
- 4:14surfaces on anti-gravity, on the Gemini
- 4:16API, in the Gemini app, and on search.
- 4:20Um, and you know I have a graph that
- 4:22shows the AA index with many of you
- 4:25might be familiar with and time on the
- 4:28x-axis. You can see it's one of the
- 4:30fastest models for that level of
- 4:33intelligence. And to give you a sense of
- 4:36how far you know Gemini has come, 3.8
- 4:40flash on terminal bench 4 was about 20%.
- 4:45And hydrogen is about 60%. So we're
- 4:48three times better performant while
- 4:51being three times faster, right? So
- 4:54that's no easy task and we're, you know,
- 4:56really happy to bring that to you in
- 4:59your hands so you can build things with
- 5:01it.
- 5:02And that completes, you know, our uh
- 5:06Gemini 4 series. So we have Gemini 4
- 5:09carbon a frontier model for deep
- 5:11frontier intelligence. Gemini 4 hydrogen
- 5:14which is fast for everyday uh execution
- 5:17and we'll have more elements you know of
- 5:19course there are many elements left in
- 5:20the periodic table and we have a lot of
- 5:22work to do to exhaust them uh but we're
- 5:25glad uh to put it out there for you all
- 5:27to test it and for you all to play with
- 5:29it. Um,
- 5:32now I think enough marketing with the
- 5:34models. I want to I was told this is a
- 5:36deeply technical audience. So I want to
- 5:38talk a little bit about why post-
- 5:40training these frontier models is hard
- 5:43and what goes into that. Let me start by
- 5:45saying you know AI writes about 75% of
- 5:49Google's new code today. And this is up
- 5:51from less than 25% a few years ago. And
- 5:56there are 100,000 people using
- 5:59anti-gravity writing code inside Google
- 6:03every single day.
- 6:06Now the complexity of the tasks that
- 6:08people do have also increased
- 6:10exponentially in the last couple of
- 6:11years. We started with you know models
- 6:15just completing chats which took seconds
- 6:17to writing small functions within the
- 6:20minutes. You can submit a PR and resolve
- 6:22an issue. Takes about an hour, let's
- 6:24say. But now people are doing days of
- 6:28real engineering tasks with the model.
- 6:30People have swarms of agents solving
- 6:32real open problems with the model. We
- 6:34have engineers across Google using
- 6:37anti-gravity and our Gemini 4 models to
- 6:39solve tasks over multiple days.
- 6:43Now, how do we go about post- training
- 6:45Gemini for real engineering tasks? We
- 6:48have to start with hard tasks. So we
- 6:50need really need to source hard problems
- 6:52for the model. Grading of these models
- 6:56is actually a very hard problem. So you
- 6:58want to grade like an experienced code
- 7:00reviewer would grade. You want to grade
- 7:02like an experienced engineer would grade
- 7:04a solution. Large scale RL is how we
- 7:07train the model. And we want to dog food
- 7:09our models with our 100,000 engineer
- 7:11community and repeat the loop. So any
- 7:14failures in the dog food become tasks
- 7:16for the next iteration of the loop.
- 7:20So I want to start by going into a
- 7:21little bit more detail of each of these
- 7:23sections. So let's start with the hard
- 7:24tasks, right? Um so you might have heard
- 7:28you know as models are getting better
- 7:29lots of benchmarks are getting saturated
- 7:32and it's not just the benchmarks that
- 7:33get saturated our training data gets
- 7:36saturated too. So here is a graph that
- 7:38shows for three different iterations of
- 7:40the Gemini 4 models how much uh the
- 7:44share of learnable prompts has gone
- 7:46down. So let's say we had x% of
- 7:47learnable prompts for the first
- 7:49iteration. By the time we finish and
- 7:51train the next one, the prompts are too
- 7:53easy for the second one. We finish that
- 7:54and train the third one, the prompts are
- 7:56too easy for the third one. So what's
- 7:58the problem with uh training on easy
- 8:00task? Easy task give you zero learning
- 8:02signal, right? So you have a prompt when
- 8:04all of your rollouts are going to be
- 8:06correct. There's no variance. There's no
- 8:08RL signal. So you really need to focus
- 8:11on getting hard tasks and you really
- 8:13need to be faster at getting hard task.
- 8:15Then the models are getting better,
- 8:17right? Um so how do we go about it? We
- 8:20first have to you know identify these
- 8:23easy tasks and prune them. So we cut
- 8:25these simple sweep style tasks from the
- 8:28mixture that are too easy for the
- 8:30models. we expand into much harder tasks
- 8:34for example that are multifile repos. We
- 8:36go from unit test or binary correctness
- 8:40um to rubric based grading and rubrics
- 8:43give you dense signal. They're not just
- 8:4501. There can be um more gradients
- 8:49within the um within the criteria.
- 8:54Here are some examples of hot tasks that
- 8:56we train on. So uh we look at multifile
- 8:59repos and we extract commits from them.
- 9:02From those commits we have models
- 9:04automatically extract rubrics. Um we
- 9:07don't actually ne necessitate unit tests
- 9:09being involved. We can actually uh
- 9:12generate rubrics on the fly based on
- 9:14these repos. Another example a great
- 9:16example of tasks are you try to teach
- 9:19the model to build uh CLI tools from
- 9:23scratch. So you start with an empty
- 9:24folder and a specification. The
- 9:26specification says, hey, you have to
- 9:29build this tool. It's called GP. These
- 9:31are the parameters of GP. These are the
- 9:33flags. This is the functionality. And
- 9:35you let the model build the entire tool
- 9:37from scratch. And you know what the
- 9:40functionality for GP is going to be. So
- 9:41you have a hidden test suite that you
- 9:43can evaluate the binary on. The last uh
- 9:48example is production codebased diffs.
- 9:50So you look at real usage of the model
- 9:53inside your production codebase and
- 9:56extract useful tasks out of them.
- 10:00Now let me talk a little bit about uh
- 10:02the graders.
- 10:05So agents you know they find any way
- 10:08you're you're training them to pass the
- 10:09test. So they will find any way to pass
- 10:11the test but if the test pass that
- 10:14doesn't necessarily mean that they've
- 10:15solved the problem. There are examples
- 10:17where the agent will bypass the checker.
- 10:20There are examples where the agent will
- 10:22game the benchmark. There are examples
- 10:25where the agent will fake the evidence.
- 10:27You know, it writes the uh fake evidence
- 10:30into a temp file. It'll delete any
- 10:31evidence of it faking it, right? Um so
- 10:35you have to be smarter about how you
- 10:36judge these models, especially as they
- 10:38become more capable.
- 10:41Now, this is what I meant by grading
- 10:43like a reviewer. So if you've done a
- 10:45code review, you have a very strict
- 10:47reviewer, you're going to know they're
- 10:49not just going to think about the
- 10:50correctness of the solution. They're
- 10:52actually going to think about the style.
- 10:54They're going to think about
- 10:55maintainability. They're going to think
- 10:57of many different aspects. So what we do
- 11:00is we move from unit test base
- 11:02correctness to rubric base and these are
- 11:05generated from real commits. The rubrics
- 11:08also provide dense learning signal not
- 11:10just 01 signal and we penalize any sort
- 11:14of solution that is looking for a
- 11:16workaround. Right? So here is an example
- 11:20of a rubric where the criteria is you
- 11:23know here are the edge cases that need
- 11:24to be passed. This is an example of
- 11:27hygiene that needs to be maintained.
- 11:29These are constraints around
- 11:30verification that need to be satisfied.
- 11:35Now, it's not just about the rubrics
- 11:38needed for grading. We also need to be
- 11:41careful about the graders not being
- 11:43gamed and you really need to harden the
- 11:45environment that the grader is in. So,
- 11:47we think about putting the graders in a
- 11:50isolated sandbox. So, we remove git
- 11:52history and git remote so the answers
- 11:54don't leak. The test sets are held out.
- 11:57So, only when the agent is done and
- 11:59submits the solution do we actually
- 12:02reveal the test. We've had instances
- 12:04where the agent will fake it and say,
- 12:05"Hey, I I'm done and the tests come into
- 12:07play and then it'll look at the test and
- 12:09resubmit the solution." Right? So, you
- 12:10don't want that. Um, we also want to be
- 12:13much more detailed with our scoring. So,
- 12:15we take the full diff and we look at the
- 12:17reference and we look at multiple
- 12:18aspects. It's not just the correctness.
- 12:19We also want to look at the code
- 12:21quality. We also want to look at the
- 12:23scope uh of the change that was done.
- 12:28And then finally we want to penalize any
- 12:31sort of cheating attempt by the agent
- 12:34while being careful not to police the
- 12:36thoughts. Um so any you know cheating or
- 12:39hampering of the environment gets a very
- 12:41negative RL reward. Uh and all of this
- 12:45is detected just based on the actions
- 12:46that the model is taking not the
- 12:48thoughts. Um so a good example of how
- 12:52we've handled cheating is you know
- 12:53carbon for instance has 0% cheating rate
- 12:56on terminal bench 4. Our previous models
- 12:58had a much higher cheating rate and it's
- 13:00about 5x less adversarial than argon. So
- 13:03every generation of the model we're
- 13:05trying to make it much more aligned so
- 13:06it's not trying to cheat.
- 13:10Now finally I think uh I want to talk
- 13:12about model behavior a little bit and
- 13:15modeling honesty specifically.
- 13:18Now agents they always want to be
- 13:21helpful and sometimes they fake
- 13:23helpfulness. You might have seen this in
- 13:25your usage of the agents. So sometimes
- 13:27they'll claim that tests have passed
- 13:29when they've obviously not passed.
- 13:32They'll declare done but obviously
- 13:34they're not done. They'll make edits
- 13:36that they'll claim to have made edits
- 13:38but they've not made those edits. Or
- 13:40they'll invent facts and APIs and
- 13:43whatnot. or they will give you a wall of
- 13:45text that has so many numbers that it's
- 13:47going to be impossible for you to
- 13:48verify. Right? We've gone through all of
- 13:51these examples.
- 13:53Now, here is an example of fabrication
- 13:56rates in Gemini. Right? So, we on an
- 14:00internal eval started with 8%
- 14:02fabrication in that hard eval. We cut
- 14:05that down dramatically to about 3% for
- 14:08hydrogen and carbon is half of that is
- 14:12about 1.5%. But we're striving to get
- 14:14this down to zero. Like you don't want
- 14:15your agents fabricating anything, right?
- 14:18But they try to fabricate because they
- 14:21want to be helpful. And you really have
- 14:23to try hard to align them to be helpful
- 14:26but not by faking it.
- 14:29How do we reduce the fabrication?
- 14:31Believe it or not, you have to clean
- 14:33your data. Uh right. So we filter out
- 14:36lots of training trajectories where
- 14:38there's you know not just uh intended
- 14:42fabrication but unintended fabrication.
- 14:44There's lots of claims that are not
- 14:46fully supported. They're not verified.
- 14:49We ground in the full trajectory. So if
- 14:51the agent says I've completed the task
- 14:53and it has a final response, we actually
- 14:56look at that response and see all of the
- 14:58actions that the agent had taken in the
- 15:00trajectory to get to that final solution
- 15:02and we look at the steps that were
- 15:04taken. So you really need to when the
- 15:06agent says, "I'm done." and gives you a
- 15:08summary, that needs to be grounded in
- 15:09the actions that were taken. And we want
- 15:11to reward honesty. So if there's a hard
- 15:15problem, it's better for the agent to
- 15:17say, "Hey, this is my partial progress.
- 15:19It's a hard problem. I'm done here. This
- 15:22is my partial solution." Rather than
- 15:24faking the last step and saying, "Hey,
- 15:26I've solved the problem. You know, this
- 15:27is done."
- 15:30Finally,
- 15:32shipping in weeks. So how are we able to
- 15:34ship so rapidly?
- 15:36And the biggest reason for that is our
- 15:39internal uh team of 100,000 plus
- 15:42engineers at Google using the model for
- 15:44realistic tasks. You know many of you
- 15:46might know Google is a monor repo and
- 15:49what is a monor repo? The entire company
- 15:52is one codebase right and I don't know
- 15:55how to measure this but I think it might
- 15:57be one of the largest monor repos in the
- 15:59world. Now that's a really complex uh
- 16:02codebase and we have the best people in
- 16:06the world using a really complex code
- 16:09base to do really complex tasks. So
- 16:11every time we have a new candidate from
- 16:13post training, we actually serve that
- 16:16model uh for our users. We run blind
- 16:20battles with them through multiple
- 16:21different candidates. We let users you
- 16:24know uh pick the best model that they
- 16:26want. We also run less where under the
- 16:29hood we're routing the routing people to
- 16:32experimental models and get signal and
- 16:35feedback from that and we take all of
- 16:37these signals and friction back into
- 16:40training the next candidate.
- 16:42So this is a cycle that helps us you
- 16:45know rapidly iterate really train on
- 16:47hard problems and as the models get
- 16:49better the engineers are going to push
- 16:51the model in interesting ways.
- 16:55This is also why, you know, we've gone
- 16:57from releasing in weeks um to releasing
- 17:01to we releasing in quarters to releasing
- 17:02in weeks. So, in the three in the Gemini
- 17:053 era, we've had launches every 2 to 3
- 17:07months, but now we've had six releases
- 17:10in the last 16 weeks, right? Uh and
- 17:14we're very excited to have you all try
- 17:17Carbon and Hydrogen uh and Gemini 4. Um,
- 17:20and we're excited for for for us to see
- 17:24what you all do with it. Uh, and we're
- 17:26not done. So, we have a lot more
- 17:27elements uh to go through. Uh, with
- 17:30that, you know, thank you.
About this transcript
This page contains the full transcript of Post-Training Gemini 4 for Real Engineering Work — Melvin Johnson, Google DeepMind by AI Engineer, generated from the public captions YouTube serves with the video. The transcript has 2,802 words across 401 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.