YT YouTube2Text

YouTube2Text → Transcripts → Nebius CTO: Why the Promise of Agents Doing Everything Is BS

Nebius CTO: Why the Promise of Agents Doing Everything Is BS — Transcript

by Harper Carroll AI · 7,000 words · 1,205 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Meta has formed [music] a $27 billion
0:03infrastructure agreement with AI cloud
0:05company Nebius.
0:06>> The more billion bucks you put into the
0:08model, the more intelligent [music] it
0:10is. For instance, your intelligence in
0:11LLMs right now is the issue. They are
0:14really [music] good. But you don't need
0:16all of that to to address smaller
0:19workloads.
0:19>> A lot of [music] major corporations are
0:21switching to use open-source models.
0:23>> Three, four, five times cheaper when the
0:26level of intelligence is comparable.
0:28>> So, you think the benchmarks are BS
0:29[music]?
0:30>> First of all, of course, a good like a
0:32really good model that is in the hands
0:34of a bad actor, some weird stuff might
0:37happen. I think it can't be solved by
0:38regulation. We need some other approach.
0:43>> Those models refused because of their
0:45guardrails on cyber analysis. And so,
0:47they had to use an open weight model
0:50to be able to figure it out. At South by
0:52Southwest, I
0:54>> Seriously [music]?
0:55>> It's a great demo.
0:56>> Pretty advanced stuff you're engaging
0:57in.
1:02>> My name is Harper Carroll, and I'm a
1:03computer scientist for AI from Stanford
1:05who has been building and teaching AI
1:07systems for over a decade at places like
1:09Meta and at a company as founding
1:11engineer at a company acquired by
1:12Nvidia. My guest today is Danila Shtan,
1:14CTO of Nebius. Nebius is a platform that
1:17makes it really easy to access
1:18open-source models via API calls. You
1:20can fine-tune them. They also have a
1:22cloud arm where they rent out compute to
1:26major companies. So, for example, two of
1:28their largest contracts are with Meta
1:30and Microsoft. Meta in 2025 did a
1:33five-year deal with them for $3 billion,
1:35and this year they have signed up to $27
1:38billion
1:39over five years. And for Microsoft, they
1:41have committed $17.4 billion over five
1:44years, and potentially more. 32 billion
1:47out of the $40 commitments that they
1:49have are in the cloud. But they also
1:52have this tremendous suite of software
1:54products. And for myself, who's a major
1:56fan of open-source models, I've been a
1:58huge fan of Nebius for a long time
1:59because they make it so easy to just
2:02swap out your API calls in your projects
2:05with API calls to open source models.
2:07And if you fine tune open source models,
2:09you get an instant lead deployed version
2:10of that model, which is incredible.
2:12Nevius also has an option where you can
2:13turn on zero retention. So, anything you
2:16send to them will be deleted after it
2:17runs through the model. So, it's kind of
2:19the privacy forward option with Nevius.
2:22I'm so excited today to interview Dinela
2:24and I hope you enjoy the conversation.
2:26Dinela, thanks for being here.
2:28>> Thank you for having me.
2:29>> So, you recently said, just a couple
2:31months ago, that you think the promise
2:33of agents doing everything is bull
2:37[laughter]
2:38Can we talk about that? I want to hear
2:39more from your perspective.
2:40>> Sure. Sure.
2:41Uh you have a particular question about
2:44that?
2:44>> Well, yeah. So, I mean, share more about
2:47the you said that you think the promise
2:48of agents doing everything is bull. So,
2:50I'm curious about why you think that. I
2:54mean, I don't necessarily disagree with
2:56you, but coming from your view, I think
2:58it's a very powerful. So, I would be
3:00curious just to hear you elaborate more
3:02on why you believe that's true and where
3:04agents are now.
3:05>> First of all, the whole bull argument
3:08is, of course, a little bit exaggerated,
3:10right? So, it's for sure.
3:11>> We love it. It's great. [clears throat]
3:12>> Sure.
3:13But, uh
3:14what I meant is that like we talk about
3:17this gigantic revolution, AI revolution,
3:20AI adoption a lot these days, right?
3:22Agents are everywhere doing stuff for
3:25you, doing stuff in the enterprises,
3:27like substituting people, clicking
3:29buttons in the workflows, etc., etc.,
3:31etc.
3:32And
3:34but I think there is a a huge disconnect
3:36between what agents are actually capable
3:39of doing and how much hard work and
3:41engineering gets into like actually
3:44achieving something with an agent.
3:46Rather than, "Oh, ChatGPT checks my
3:49email, so it means like it will solve
3:51any problem within my enterprise." This
3:53is not true and it's won't be true for
3:56for the for a long, long time.
3:58>> And so, you feel like even as the agents
3:59have like I would say, more recently
4:02like Claude and chat GPT
4:05they have gotten much better at these
4:07kind of like long context agentic tasks.
4:09Like people kind of kick off a job and
4:11then they come back the next day. You
4:12know, would you say that even so
4:15we're still not there and like what is
4:17lacking would you say?
4:20>> Well
4:21even
4:23even when we are talking about the
4:24situations like that, you just set it up
4:26to do something and it achieves
4:28something.
4:29We are usually coming back to very
4:31familiar environments. We are usually
4:33for example when we are talking about
4:35Claude doing something like long term
4:36with
4:37probably it's about like software
4:39development, right? Software development
4:41in a sense is a very familiar
4:43environment for the agents. We have a
4:44lot of tooling, we have a lot of
4:46understanding how that works. We have
4:48tools in place to verify the code, to
4:50run the tests, check that everything is
4:52consistent, etc. etc. etc. etc.
4:55Compare that with for example going
4:58through the process of
5:00issuing a refund in the online store.
5:04>> A refund. Okay.
5:05>> For example, you know, like a customer
5:07comes to you and asks for a refund.
5:09>> Okay.
5:09>> Right? Are you sure that Claude out of
5:11nothing can do that properly?
5:14>> Yeah.
5:15Like would you say via computer use or
5:18via like just API calls to your own
5:20systems?
5:21>> Yeah, exactly. So now we are talking
5:23about API calls, we are talking about
5:25how to describe what's going on, what
5:28are the restrictions put some guardrails
5:31in place so it doesn't
5:33refund too much for example.
5:35>> Yeah.
5:36>> Right?
5:37What else?
5:38>> yes, we can prompt engineer it to get
5:40more. Yeah. Cuz it feels like the at
5:42least the process of a refund feels
5:44relatively straightforward. Like it
5:45almost feels like we could have done it.
5:47I I guess the LMM component is important
5:49to interpret what needs to be done, but
5:51like and that's the difference between
5:53just pure automation. But it feels it
5:56feels like some of these things I agree
5:58I don't I don't think we can like
6:00have an agent do anything at all, but I
6:02feel like there are some some cases
6:04where we can teach it, right? And then
6:06it can learn. Yeah.
6:07>> Like I'm very bullish on this. I like
6:09I'm pretty sure that we will see those
6:11increasing. We will see more and more
6:13workflows being automated, but
6:16in a sense for me, like maybe because of
6:18my background or overall skeptical view
6:22of the world, right? But I kind of see
6:25this whole AI LLM like agentic thing
6:29more of a
6:30an extension of automation.
6:33Right? So, it's not something that is
6:36>> Right.
6:37>> it upside down, right? You're just
6:39adding a component
6:41>> Like think about it.
6:42>> Yes, to your automation pipeline, where
6:44previously you would have to write a ton
6:47of code, for example, right? To account
6:49for all
6:50options or put a human in the loop so
6:53that it makes the final decision, right?
6:56>> Right.
6:56>> With the refund, like at first probably
6:59a person goes through the orders,
7:01decides whether it can do them. If there
7:03are a lot of refunds, you introduce
7:04automation, right? You introduce it also
7:06for a person, still for a person
7:08>> Yes.
7:08>> to look at that, to check it against
7:10like, I know, history of refunds for for
7:13this particular client.
7:14>> Yeah.
7:14>> Some other like internal tools that
7:16would not allow a human to do that. Kind
7:19of
7:20decreasing the amount of intelligence
7:22that the operator needs to have.
7:25Right?
7:25>> Yes. Yeah.
7:26>> points, you via automation introducing
7:29tools, policies, guardrails, et cetera,
7:31et cetera, you decrease
7:33to a level where modern LLM can get and
7:36fix those problems. So, this is my like
7:39personal
7:40uh view of the world. We are not
7:42building it and we should not build it
7:44agent first.
7:45We should think about it as
7:47agent-enabled automation.
7:50>> Yes. Yes. And I I I think I love hearing
7:53you say this because I agree. I used to
7:54say when agents were like really picking
7:56up, I used to joke with people there,
7:57it's just it's just automation. We've
7:59always had it, but now you slap on the
8:00LLM, but everyone just says that they
8:02have agents when they're doing
8:03automation [laughter].
8:04>> Yeah, and then and then they have agents
8:06going rogue. Then they have agents like
8:08refining minions, etc. etc. And yes,
8:11this is
8:11>> Because of that interpretation, yeah.
8:12>> Yeah, but like
8:14I've been doing this for quite some
8:16time. And in one of my previous roles,
8:18for example, I was working on internal
8:20automation of a fairly large retail
8:23store, right? And like
8:27what I'm saying is that this are not new
8:29problems.
8:30>> different, yeah.
8:30>> You've been doing it for years now, tens
8:32of years now. It's just now we have a
8:35pretty cool really cool tool that can
8:38add a level of intelligence
8:41to the processes that never had it
8:43before. There was
8:44purely structural like programs, right?
8:48Your branches, your ifs and checks, etc.
8:50etc. Now you have some undeterministic
8:53reasoning around that.
8:55But
8:57if you take it uh
8:59like
9:00further down for example, I strongly
9:03believe that
9:04uh
9:05in some time, years, maybe months, maybe
9:09we will definitely see a trend
9:11where
9:13you don't need that much intelligence in
9:15this workflows because
9:17I think
9:18again, picking up from automation
9:20approach, right?
9:22You usually are talking about very
9:23scoped, very narrow workflow operating
9:26within a limited domain.
9:28>> Yeah.
9:28>> Uh been exposed to proper tools and
9:31checks.
9:32So
9:33I think the best agentic workflows,
9:37agent automation like in the future, are
9:39going to be like operating in a limited
9:42domain
9:43enabled by a very customized agent loop
9:46that is also adhering to this particular
9:48domain. For example, when you are
9:50refining stuff, you need to go through
9:52like several stages. Those should be
9:54supported in in the software, not in the
9:56reasoning, not in prompting.
9:58>> Yes.
9:58>> Cuz it's it's fairly easy.
9:59>> Right. Right.
10:00>> get through several checks. Then you
10:02will see tools that are over-optimized
10:05for this particular workflow task,
10:06right? You know, not just issue a
10:09refined, but issue a refined adhering to
10:11multiple checks, etc. etc. etc.
10:13And by the way, I also think this would
10:15be run by
10:17very custom probably trimmed models.
10:20>> Yeah, are you going to say like smaller
10:21models for each piece?
10:22>> Smaller models fine-tuned on this
10:24particular workflow, etc. etc. etc.
10:27Because again, like
10:29Yes, frontier intelligence in LLMs right
10:32now is is huge. They are really good.
10:35But you don't need all that to to
10:38address smaller workflows.
10:39>> Right. I like I like that. I like that
10:42Nebius helps
10:43customers
10:45split their problems. Like, rather than
10:47just using frontier intelligence at the
10:49maximum cost for every problem to have
10:52it reason through the whole pipeline,
10:53just learn to split it up into the
10:55pieces like you used to, and then have
10:57small fine-tuned models for each each
10:59piece where you actually do need a kind
11:01of thought or reasoning or analysis or
11:03parsing or whatever it is that you're
11:04trying to do.
11:05Um great. Yeah. I mean, one thing that
11:10is very
11:11of note right now is that we hear that a
11:14lot of major corporations are switching
11:16to use open-source models. What have you
11:19seen with that with Nebius? And
11:22so you're seeing that these companies
11:24are switching from Claude or Anthropic
11:26or what have you or Anthropic or ChatGPT
11:28um or OpenAI to using
11:32open-source models. Are they using like
11:35frontier-level open-source models? And
11:37or are they using the small specialized
11:39models? What are you seeing? How easeful
11:41is that process? What's that looking
11:42like?
11:43>> I think it's there is like no single
11:46answer there because every case is
11:48unique, right? Sometimes
11:50people choose open-source models purely
11:52for cost reasons. Because a properly
11:55configured, properly tuned even not
11:58fine-tuned as in changing model weights,
12:00but like tuned as in tuned setup, right?
12:03Can be economically
12:07I don't know, it can be
12:08three, four, five times cheaper when the
12:11level of intelligence is comparable.
12:14I'm not saying that or frontier open
12:16weight models are there yet, but in a
12:20lot of cases they are very much
12:22>> they're not there. They're not
12:23comparison with the closed source
12:24models?
12:25>> No.
12:26>> Not even like the older versions?
12:29>> The level of intelligence in a model
12:31is a function of how much money is put
12:34into developing the model.
12:35>> Right.
12:36>> No one comes even close to Open AI and
12:38Anthropic in this regard.
12:39>> And yeah, distillation just can't get
12:41there.
12:41>> So so it's
12:42almost like oh it's very
12:44straightforward. The more billion bucks
12:47you put into the model, the more
12:48intelligent it is.
12:49>> So you think the benchmarks are BS?
12:52You think people train for the
12:53benchmarks?
12:54>> First of all, of course. Like we see
12:56that all the time. We see that all the
12:58time.
12:58Like the new benchmark is released, all
13:00of a sudden the models are incapable of
13:02doing that. Two months pass, new models
13:05are released. Oh, they can do it.
13:06>> do it.
13:07>> Everyone [laughter] can do it.
13:08>> That's a good one.
13:08>> Yeah, right. So like this whole
13:09benchmark thing thing is is is a weird
13:12one. So but like let's circle back.
13:14>> Okay.
13:14>> Uh open weight models are competitive.
13:18Not everywhere, they're not frontier
13:20level, they lag behind. Although this
13:22gap is getting smaller and smaller every
13:25day.
13:25>> Yeah.
13:26>> But again,
13:27I feel like for a majority of use cases,
13:30they are already good enough.
13:33This is the the thing, right?
13:34>> Yeah.
13:35>> So sometimes people change to open
13:38weights
13:39purely for cost reasons.
13:40>> Yeah.
13:41>> Sometimes is this whole surety angle,
13:43right? Right. Then of course, we see
13:47this whole limited domain increased
13:50quality angle. People are fine-tuning
13:52models, people are modifying weights,
13:54etc. etc. Which is impossible with
13:57frontier closed models. They expose some
14:00of that.
14:00>> Right, but not really. They're
14:01fine-tuning stuff, it's not very good,
14:03yeah.
14:03>> Well,
14:04>> Oh, it's not from whatever Excuse me,
14:05sorry.
14:06[laughter]
14:07From what I've From what I've heard,
14:09it's less flexible.
14:10>> Let's say it. Let's say it like that,
14:11right?
14:13And then the deeper you are in this
14:15process, the more options are there.
14:17Distillation, quantization, like model
14:19pruning. We we see like model
14:21compression in general. So, we see a lot
14:23of
14:24uh really interesting things playing out
14:26uh
14:27around open weight models. And I think
14:29this is an important part and
14:32frankly, I think like my usual uh answer
14:35is like when you're in the early stages
14:38of exploration on how to approach this
14:40thing, like of course, use the frontier
14:42models.
14:44Get the maximum intelligence there. Like
14:46let the agent figure it out.
14:49The more
14:50concrete your pathways become,
14:53the more options you have now.
14:55>> Totally.
14:55>> Trim it down. Play with the tools, with
14:58harnesses, with models.
15:00And when you're saying playing with
15:01models, we are definitely in open model
15:03territory.
15:03>> And use, you know, if you want to use
15:05the closed frontier models to help you
15:06figure out how to implement it using an
15:08open source model. You know what I mean?
15:11>> Uh
15:11>> No, I mean, but like yes, and that is
15:14what's happening all around. Right? So,
15:17frontier closed models are very good,
15:19well,
15:21uh
15:22it's the best we have for trying to
15:24attack the open-ended tasks, right? We
15:27we're not exactly sure what the result
15:29would be. They're very good at that. So,
15:31of course, people use those example to
15:34help them with running open weight
15:36models. That's great. Like let all the
15:38flowers grow.
15:39>> Do you I love that. I I love the view,
15:42you know, because there's been some
15:43concerns. I've had
15:45some concern recently around a lot of
15:47the doomerism talk and the sense that it
15:50would cause regulation around open
15:51source AI
15:53and shutting it down. And that's been
15:55something that I've been kind of
15:56passionate about and upset about. And
15:58I'm curious
16:00are you concerned about that? Them
16:02regulating open source AI? Or I guess
16:04you have your public company, so you
16:06operate all over, you'd be fine if it
16:08was regulated in the US, I guess. I
16:10don't know how that would work, but
16:13>> Well,
16:14I like my personal stance is very
16:18anti-regulation. Like
16:21I'm libertarian in that regard, right?
16:24So, I think
16:25libertarian
16:27I think the less regulation we have in
16:29this area, especially if it comes from
16:32um institutions that are not really
16:35familiar with the cases we're talking
16:38about, the better.
16:40Right? So, I'm not saying there is no
16:46potential problems with that, right?
16:47With whole thing like agents going
16:50rogue, whatever.
16:51By the way,
16:52we mostly hear about agents going wrong
16:55from friendly closed companies rather
16:57than [laughter]
16:58>> Yeah, we do, don't we? It's funny.
17:00>> open companies, right? But
17:03in a sense, it is perceived right now
17:05this level of intelligence perceived
17:06that if we there is like a good like a
17:10really good model that is in the hands
17:12of a bad actor,
17:14some weird stuff might happen.
17:15>> Yes.
17:17>> I think it can't be solved by
17:18regulation. We need some other approach.
17:21>> I mean, the bad actors are just going to
17:22go around regulation. They always have.
17:24Why would they stop at law? In fact,
17:26you'll you'll prevent the good people
17:27from having access to the same tools
17:29that the bad people have access to, and
17:31then the good people can't protect
17:32themselves.
17:33>> Yeah, remember the Hugging Face example?
17:35It was extremely funny.
17:36>> Yeah, how they had to use the open
17:37model?
17:38>> Because closed model refused to engage
17:41in this activity because it was like
17:43cyber protected cyber cyber guarded.
17:45>> Yeah, which is extremely [laughter]
17:46funny.
17:47>> Which is crazy. Yeah, for for those who
17:48don't know, the open the closed models
17:51Hugging Face tried to use, I think it
17:53was
17:54uh ChatGPT and Claude to figure out what
17:56was happening, how they were being
17:57hacked, and um
17:59those models refused because of their
18:01guardrails on cyber analysis. And so,
18:04they had to use an open weight model to
18:07be able to figure it out, which is like
18:09it it's the such a good example of that.
18:11>> And for example, OpenAI is very
18:13selective on who do they allow to use
18:15their cyber enabled models. So, there is
18:17a separate product for them that they
18:19boast for defensive cyber but they're
18:22very very
18:24>> They wouldn't let hugging face. That's
18:26so interesting.
18:27Maybe now they would, but
18:28>> Maybe now they would.
18:29>> Yeah, it's like if you're so selective,
18:31that's so interesting.
18:32>> You you need to ask. They will review
18:34your requests. They will think whether
18:37they were
18:38a bad or a good actor, etc. etc. etc.
18:41>> It's a whole process. Yeah, and I mean
18:43imagine every company wanting to go
18:44through that. And or maybe there's there
18:46should be a new product where they
18:47submit their website to be analyzed by
18:51Claude, right?
18:52>> Uh they
18:53Anthropic just released something like
18:55that.
18:55>> Yes, they actually have a service. Yeah,
18:58to go and
19:00yeah, to go and look at
19:01>> Anthropic. But I'm not sure I'm not sure
19:03it's not about the website like as in
19:04runtime, but they definitely have this
19:06like security code analysis
19:08>> Okay.
19:08>> service right now where you just submit
19:10your GitHub repo and they go and scan it
19:13and the model reasons about potential
19:15vulnerabilities in your code.
19:16>> Oh, I didn't know that existed. That's
19:18great. Everyone should be doing that. If
19:19you're watching this, do that.
19:21Like right now.
19:22[laughter]
19:22>> Maybe you should build a separate
19:24pipeline with open models to
19:25>> Or you could exactly, you could build a
19:27separate pipeline with open models. I
19:29mean, it's great to hear you say that
19:31both the open models and the closed
19:33models can play nicely. Like everyone
19:35can exist. You know, the the closed
19:36models are at the frontier. They're all
19:38they'll always be at the frontier. Um
19:41but that doesn't mean we need to like
19:42like and open models have their place as
19:44well.
19:45>> Yeah, well like for example in my daily
19:48use, I constantly switch between models
19:51simply because I like
19:52I like to see the differences in like
19:54model behavior.
19:56>> Okay.
19:56>> And I'm constantly switching between for
19:58example Anthropic models, OpenAI models,
20:00and some frontier open models because
20:04First of all, they are capable
20:05>> Yeah.
20:05>> for some of my daily tasks. More than
20:08capable.
20:09And you also see interesting results
20:12there.
20:13>> Cool.
20:14>> So
20:14they can sit in the same harness. They
20:16can sit in the same
20:18like uh you know, router choosing the
20:20particular model for a task. They can be
20:23you can switch between like in in a
20:25sense models are a commodity now. It's
20:27just an endpoint.
20:28>> Yeah.
20:28>> Everyone more or less standardized on
20:29the same interface, right? It's just
20:31open AI compatible interface. You will
20:33switch the endpoint in your config and
20:35all of a sudden your agent uses another
20:37model.
20:38>> Yeah.
20:38>> And the rest of the environment is very
20:40portable. So,
20:43let's go.
20:43>> Yeah. So, let's go. I love it. Okay, I
20:46want to pivot to talking about the five
20:49main areas that you are all focused on,
20:52which is physical AI and robotics,
20:54healthcare and life sciences, media and
20:56entertainment, retail, and financial
20:59services.
20:59>> Okay.
21:00>> And I'm curious,
21:01what are some of the use cases that
21:03you've seen that you can talk about
21:05those categories use your models for?
21:08Um and then
21:10I would say yeah, because I think
21:12a lot of
21:14business owners and leaders at companies
21:17that come to me, they're curious like,
21:19how do I even use AI? What are the good
21:20use cases for it? What should I use open
21:22source models for? So, I would we would
21:24love to hear
21:25from you about what people are who are
21:28actually doing it are using them for.
21:32>> It's like it's a very hard question to
21:35be honest.
21:36>> Okay.
21:36>> Cuz they're using it for
21:38>> So many things. Just a couple things
21:39everything.
21:40Really?
21:41>> Well, listen. Physical AI.
21:43>> Okay, yes. Let's talk about that cuz I'm
21:44I'm interested.
21:45>> Near and dearer to everyone. Jensen
21:47yesterday was playing with like those
21:49funny hugging face robots on stage,
21:52right?
21:53Everyone's talking about teams, talking
21:55about like servos, talking about like
21:57robot arms, grabbing stuff, etc. etc.
22:00etc.
22:01This is a very interesting area. It's
22:03not super established right now. So,
22:06most of the teams that we speak with are
22:08experimenting.
22:10They're trying different models,
22:11different approach approaches.
22:14Simulations are a big thing now because
22:17well,
22:19funny thing with real world is it's too
22:21real.
22:22Like for example, it runs at the speed
22:24of 1 second per second.
22:26>> Exactly.
22:27>> Right? And in compressed virtual
22:29simulations, you can get maybe the data
22:33that you'll get from that will not be as
22:35exact and as precise.
22:37>> You get a lot more.
22:38>> Yes, but you get a lot more. So, for
22:40example, for physical AI, I feel like
22:42there is no particular use cases yet
22:46shaped yet because everyone's doing
22:47their own thing, trying different
22:49approaches. People are building world
22:51models. People are building seams.
22:53There's one company that we I talked to
22:55yesterday. They are building the
22:57intersection of seams and physical
22:59evaluations, right? As in like we have
23:02simulations where you can build your
23:04evals in. Evals as in like trying to
23:06figure out if whatever models that
23:09you've trained for actually work for
23:11your robots.
23:11>> case, yeah.
23:12>> And behind the same interface, behind
23:14the same APIs, they can transfer that to
23:16physical devices.
23:18So, you play with seams and then
23:20>> Yes.
23:21>> within the same box, for example,
23:23through the same API, you'll get your
23:25like you get those deployed into
23:27physical devices that actually try to do
23:28that. So, like people are doing all
23:30kinds of things right now. And the whole
23:32physical AI, again, it's such a large
23:35umbrella term, right? Everyone's like we
23:37are talking about bipods. We're talking
23:38about robot arms. We're also talking
23:40about some I don't know
23:44uh
23:45like the intersection of health care and
23:47physical AI where they're like precise
23:49movements where they can you know
23:50injections or something like that.
23:52>> Yeah.
23:52>> So, like that is not exactly
23:55established. So, again, the answer is
23:56everything. Everyone's doing everything.
23:59Health care is is an interesting one.
24:02Uh it's mostly about research.
24:05But in a say
24:06in a sense, it's also kind of confined
24:09with the real world because the major
24:11cost of every health care research is
24:15actually synthesizing the the drug, for
24:17example, and testing it etc. etc. etc.
24:19So, you need to put a lot of stuff a lot
24:23of
24:24uh simulations again data before in like
24:27in your virtual before you actually find
24:30that one thing that you actually want to
24:32test.
24:32>> Right.
24:33>> It will probably fail, but it's easier
24:36to test 100 samples rather than 1,000
24:39samples, for example.
24:40>> Yeah.
24:41>> So,
24:42that part is like research heavy and
24:45actually modern models
24:47there are specialized health care models
24:49for example, like
24:51aimed at protein folding
24:53>> Yes.
24:53>> and something like that.
24:54>> He used BioHubs models. I just
24:56interviewed Alex Reeves. Do you know
24:58Alex? He's amazing. He's the head of
24:59science.
25:00>> Yes, but at the same time, for example,
25:03there is a lot of large language models
25:06enabled reasoning around that because in
25:08the end like people think about this
25:12complex molecules, for example, in in
25:15text form. Like because we understand
25:17language, right? So, every concept we
25:20try to like uh
25:22boil down to some language construct.
25:25>> Okay.
25:26>> Large language models are actually
25:27really good about reasoning on top of
25:29like language constructs.
25:30>> Oh, yes, for language, yes. I thought
25:31you were going to
25:32extrapolate to proteins, but
25:34>> But that's exactly it. So, if you write
25:36a protein formula
25:38uh large language models nowadays are
25:41very good at reasoning about proteins
25:43and possible combinations because you
25:45just
25:46>> It's sequential.
25:47>> Yeah.
25:48You just look at
25:49some tokens.
25:50>> Yeah.
25:50>> in a sense.
25:51>> It's true.
25:52>> the training set is large enough, which
25:54it it is for more for a lot of modern
25:56models
25:57you kind of pick it up.
26:00Also, for example, for health care, I
26:02feel like one of the
26:04important use cases that we've seen like
26:06I I can't say it's something that you
26:09see like too often
26:11but
26:15in every
26:17scientific field that is very active,
26:19there are a lot of publications
26:21nowadays.
26:21>> Mhm.
26:22>> A lot of papers have been published.
26:24People are pushing out content.
26:26Like hundreds and hundreds of papers per
26:29week. So, there is no one is capable of
26:31reading that. So, people actually are
26:33doing deep research on top of all the
26:36published papers in order to find
26:38relevance to what they are working on
26:40today, for example.
26:42>> That's so awesome.
26:43>> This is an interesting setup. I wouldn't
26:45have thought that this is something that
26:46would like And we see established
26:48pipelines, like people have established
26:50pipelines with agents like constantly
26:53dig through published papers, reason
26:55about like whether that's relevant or
26:57not, whether that's some novel uh
26:59research is happening, etc.
27:01>> And these are all pipelines they built
27:02with Nebius.
27:04Very cool. And so, would you say when
27:06someone comes to Nebius or a company,
27:08you have some direct-to-consumer things?
27:10Like I've used your fine-tuning. I used
27:12your fine-tuning like factory. Um
27:16at South by Southwest, I I fine-tuned a
27:18model. Yeah, it's amazing. I mean, it's
27:19it's the easiest way to fine-tune a
27:21model. I used to fine-tune it like with
27:22code, let alone just this interface
27:24where I just put in my hyper-parameters
27:26and it goes. I did it on stage at South
27:29by Southwest. I trained a model I
27:31fine-tuned a model on my um captions and
27:33tweets and showed that um my a model
27:37fine-tuned with you all, like a small
27:39very inexpensive to run model, like cost
27:42me a dollar to fine-tune it and I did it
27:43in 20 minutes. I used a Llama 40B.
27:47Not only was way less expensive and and
27:49my own, but actually performed better
27:52than passing in all 800 examples into
27:56Claude. Um and and I I judged
27:59performance by um whether Pingram
28:02guessed it as being 100%human-generated
28:04or or whether guessed it as being
28:06human-generated or AI-generated.
28:08Obviously, that has its flaws, but it
28:10was a really great It was a great demo.
28:15[laughter]
28:15Well, did you know so I um just quickly,
28:18I I got started teaching AI um I'm a
28:22machine learning engineer. I got started
28:23doing this um through fine-tuning guides
28:27on the web. And so, at that time it was
28:29so hard to fine-tune models um and so I
28:31did these coding guides. This is in 2023
28:33and they blew up cuz there were no other
28:34guides. So, and I did it to get people
28:36onto our site, which is rev.dev, which
28:38is acquired by Vindia.
28:40And so, seeing you all
28:43making it so easy to fine-tune models,
28:45and even the hyper parameters, you can
28:46just ask AI what you what hyper
28:48parameters you think you should use. In
28:50the past, you just have to like tune
28:51that and figure it out. And so, I I was
28:53thinking yesterday, I was trying to, you
28:55know, think about where you all stand as
28:57compared to say Thinking Machines, and I
28:59was doing some research on that. Um and
29:01you all are so much higher level than
29:04Thinking Machines. Like anyone can come
29:06up can Like Thinking Machines, you're
29:08actually looking at the code, as far as
29:09I understand. I actually haven't used it
29:10yet. Um but you're actually kind of
29:12doing what I did, where you have these
29:14like you have to actually look at the
29:16code for fine-tuning the model and
29:17retraining the model. Whereas now, with
29:20with Nebius, you can just come in and
29:22click a button, and it fine-tunes it
29:24tunes it for you, and you can deploy it.
29:27>> Yes.
29:27>> Which is incredible. I mean, it's an
29:28instant deploy. That was I mean, the
29:31amount of work that I put into having
29:33before
29:34I I I'm very impressed with your
29:36fine-tuning UI.
29:38>> But
29:39this is our philosophy. In the end, like
29:41we want to enable builders, right? We we
29:45we are not really good at building
29:47end-user products. We tried several
29:49times. Sometimes it's good, sometimes
29:51it's uh questionable. But uh what we
29:55think is our strong suit is we are quite
29:58good at trying to build infrastructure
30:01pieces and abstract those for builders.
30:04Like of course, especially with modern
30:06AI, you don't really need that. You can
30:08just ask your plot to go and fine-tune
30:10your Yolo model, and it will probably do
30:13an okay job.
30:14>> It will spend
30:16an immense amount of tokens setting it
30:18up.
30:18>> But you need it on your computer, or it
30:20needs to find it a way to train it
30:22somewhere on some cloud
30:23>> But but but that's actually very
30:25interesting. As I was just giving a talk
30:26to Nvidia, and one of my slides
30:28uh says, like we actually see because we
30:31were building the
30:33uh AI cloud, not like your GPU bare
30:36metal as we are building the cloud
30:37experience. So, for example, everything
30:39you see in the platform is API-based. It
30:42can be programmatically controlled.
30:44>> Everything on the platform
30:45>> Yes, everything on the platform.
30:47Uh we actually see that
30:50already the share of humans interacting
30:53with the platform versus the share of
30:54agents interacting with the platform is
30:56not in the favor of humans.
30:58>> Wow, cool.
30:59>> And we expect that in a year's time this
31:02product is mostly built for agents.
31:06>> Cool.
31:07>> So, in in in a sense, yeah, you're
31:08right.
31:09>> use your cloud to
31:10>> Yes, you can just ask your cloud to go
31:13spawn a Kubernetes cluster, like launch
31:16your fine-tuning solution based on
31:18And and by the way, in this past year
31:20we've seen a huge advancements on on the
31:23software for fine-tuning, too.
31:25>> Okay.
31:25>> Right? The whole, I don't know, Miles
31:28solution for reinforcement learning that
31:30was released. It's like it's almost like
31:33one-click deploy, especially if you're
31:35backed by an agent.
31:37Right? But
31:39And you can do all that, right? But it's
31:42in a sense it's the same like you can
31:44also rent a bare metal like GPU server
31:47and go do your stuff there. And
31:50we're just increasing the level of
31:52abstraction. Like
31:54making sure that your agents spend spend
31:57less tokens for that. Because if you
31:59introduce a high level of abstraction
32:01either for a human or for an agent, it
32:03requires less
32:06I don't want to say less intelligence,
32:07but less reasoning effort.
32:10>> less thought.
32:10>> Yes, less thought. Less reasoning effort
32:12for both humans and agents to engage
32:14with a service like that. This is what
32:16we want to build. Like we want to solve
32:19tasks that are more or less common in
32:21this area. Like building building blocks
32:24for builders. Sorry for
32:26saying build four times in a row. Uh
32:30that are
32:31high-level enough
32:33for the operator, whether that's a human
32:36or agent, to decrease the level of
32:38reasoning that they need to interact
32:40with it. But at the same time, you
32:42cannot deny that for some use cases, for
32:45some complex, you need to get down,
32:48right? You need to get to the code,
32:49whether that's
32:51Tinker from Thinking Machines or bare
32:53compute where you queue tune everything.
32:56Yes, there is a space for that, too.
32:57>> everything, exactly. I'm not saying
32:59Thinking Machines is not good. It's just
33:00a different application, different
33:02audience, probably.
33:04Um yeah, and I mean, I have to say it's
33:06it's quite impressive to be able to
33:08build something that's so that abstracts
33:10so much away that isn't so buggy for
33:12people to use and is actually able to
33:13generalize quite well. Um and have you
33:16have you found that to be the case? Do
33:17you do you get a lot of um say like
33:20tickets where people say, "I don't know
33:22why this broke and I am too high-level
33:24to be able to debug it." or have you
33:26been able to like figure out how to do
33:28that?
33:28>> Sometimes we do, and this is like
33:30constant search for balance. Like
33:33you want to introduce the tool that is
33:36high-level enough, but also stable
33:38enough. You want to introduce the tool
33:39that is high-level enough, but it is
33:42also like
33:43allows your users to understand what's
33:46actually going on and how to address
33:48that.
33:49But it's nice option, like
33:51there's always a nice option of
33:52abandoning the high-level tool and just
33:54go digging into the code, and it's
33:56always a nice option.
33:57>> Right, you you need both sometimes. And
33:59[clears throat] is that relatively easy
34:00for people to do with Nebius? Like if
34:02they need to kind of get under the hood
34:03and dig in, can they?
34:05>> No.
34:05>> No, okay.
34:06>> Well, we are seeing like we are now in
34:09the process of like rethinking the whole
34:11fine-tuning post-training approach.
34:13>> Okay.
34:13>> Like because like
34:14>> Cuz that is hard, sometimes you need it.
34:16>> Yes, the the original managed SFT was
34:19like our first experiment in the field.
34:22Uh we got some good results out of it,
34:24but we feel like we have enough
34:26information now on how to
34:29to try to push it. First you
34:31or
34:32SFT is kind of a yesterday thing. No one
34:35talks about it anymore, right? They're
34:38talking [clears throat] about constant
34:39learning, reinforcement learning,
34:41building like bringing trajectories from
34:43production synthetic data post training.
34:45Yes, post training is broad term. So, we
34:47are trying to figure out how to attack
34:50that problem now. Introducing different
34:52ways to post train your SFT, your DPO,
34:55your GRPO.
34:56Uh we are thinking about like how to
34:58manage data in this uh scenarios, how to
35:02connect uh your
35:05uh inference and your post training,
35:06especially if we're talking about
35:08reinforcement learning.
35:09>> Right.
35:09>> your training and inference goes
35:10together all the time.
35:11>> I mean a suite for reinforcement
35:13learning post training would be ooh,
35:15that would be that would be nice. That
35:17would be impressive.
35:18>> We are building something right now. We
35:20have it in private beta for limited
35:22amount of customers and partners.
35:24>> Wow.
35:25>> We'll see. Like my constant fear is that
35:27it's not high-level enough and it starts
35:29to resemble like low-level tinker style
35:33uh Thinking Machines product and maybe
35:35that's a good thing. Maybe that's what
35:37everyone wants because again,
35:39if it's enabled by agents, maybe it's a
35:41good thing.
35:42>> Maybe there's a way to click into that
35:43view where [laughter]
35:45you only get it if you need it, you
35:46know?
35:47>> Yeah, oh
35:48we'll see how that shapes out, but we
35:50are actively working on it nowadays.
35:52>> Well, amazing. I wish we had more time.
35:55Um
35:56I actually
35:57Okay, can I ask one more question? Um
36:00and I I So, someone the other day asked
36:03me if you were a neo cloud and I said,
36:06"I don't know if I would refer to them
36:07as a neo cloud. I don't know." And I
36:10looked it up and you recently were like,
36:11"We are not a neo cloud." And
36:13[clears throat] I was like, "Yes, I was
36:14right."
36:14>> I passionately hate this [laughter]
36:16this particular term. Yeah.
36:17>> So, tell us about that. I want to spread
36:19the the message.
36:22>> Neo cloud is a term that was introduced
36:24to distinguish this new cohort of uh
36:27compute players from hyperscalers, the
36:30clouds.
36:31Right? But uh when we started, there
36:34were like you know, five, seven
36:35companies in the field.
36:37Right now, if you look at Nvidia NCPs or
36:39neo clouds, you see what, like 500-ish
36:43companies. It's just this term became so
36:46broad.
36:47>> Yeah.
36:47>> So, and frankly, I have an issue with
36:51putting it all under like a company like
36:53us or for example, CoreWeave.
36:55>> Yeah.
36:56>> Right?
36:56>> Right.
36:57>> And putting it at the same umbrella in
36:58the same term as the company that
37:01uh we are a small startup, we procured
37:032,000 GPUs, we want to rent it out to
37:05someone.
37:06These are different business models.
37:08These are different products
37:08[clears throat]. So, we refer to us as
37:12an AI cloud. We are the AI cloud.
37:14>> everything. You have all the software
37:15components as well and the
37:17infrastructure and in terms of like
37:19software infrastructure.
37:20>> I think this is one of the biggest
37:21distinctions between us and the rest of
37:24the market, for example, like
37:26uh three, four years ago, when we were
37:28like just starting, for example, three
37:30years ago, this is something I can vouch
37:32for. Uh
37:35our team that manages our data centers,
37:37GPUs, installs, buys, like uh
37:41is responsible for logistics, etc.,
37:43etc., was roughly 50 people.
37:46Uh
37:46software organization by that time was
37:48already 200.
37:49>> Yeah. I mean
37:50>> So, but this is like we we were kind of
37:55in a privileged [clears throat]
37:56situation there, right? Because of our
37:58history, because of the initial funding
38:00that we had, we could allow ourselves to
38:03like be software heavier organization.
38:06>> Right?
38:07>> And cuz a lot of the money is in the
38:08cloud alone.
38:09>> Yeah.
38:10>> Yeah, isn't it like 32 out of 40 billion
38:12in your current contracts or something?
38:14Yeah, I just yeah, it's Sorry, I didn't
38:16mean to cut you off.
38:16>> Yeah, so like
38:19and because of that, like this is like
38:21for me as a person responsible for the
38:23software side of things, like bridging
38:26everything under this one broad term for
38:28the companies that have software
38:30platform and don't have software
38:31platform is borderline offensive.
38:33>> I Yes. Totally.
38:35>> I don't like the term neo cloud. I think
38:38it's
38:39we should fade that like into
38:42nonexistence.
38:42>> few like other terms. Yeah, get rid of
38:45that.
38:45>> maybe we need something else. yes, but
38:47our ambition is to build the AA Cloud.
38:49Our ambition is to
38:52become large enough and substantial
38:55enough to pick on AWS and GCP of the
38:59world.
39:00>> Well, I believe in you. I mean, I'm so
39:02impressed by your product. I'm so
39:04impressed, truly. I've been a a huge fan
39:07of yours for a while now. So, it's been
39:09an honor to chat with you and thank you
39:10so much for your time, Tonya.
39:13Great.

About this transcript

This page contains the full transcript of Nebius CTO: Why the Promise of Agents Doing Everything Is BS by Harper Carroll AI, generated from the public captions YouTube serves with the video. The transcript has 7,000 words across 1,205 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.