YT YouTube2Text

YouTube2Text → Transcripts → Tokenomics 101: What Are Tokens & Why They Matter | AI Factory Insider Ep. 4

Tokenomics 101: What Are Tokens & Why They Matter | AI Factory Insider Ep. 4 — Transcript

by NVIDIA · 5,861 words · 563 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01Welcome to AI Factory Insider episode four.
0:05In the previous episodes, we defined the enterprise AI factory.
0:09We talked about the enterprise reference architectures
0:14that serve as the blueprint to build these AI factories.
0:18We went deep into NVIDIA's own AI factory, and then we talked about agentic AI
0:24and agents. Now,
0:25if you are building an AI factory for your company or you're thinking of building
0:32an AI factory for your company,
0:34there's one thing to always keep in mind: your customer.
0:38And your only customers are the users, machines, and agents
0:42that are going to harness the power of AI to make your company better,
0:48to make your company's products and services better,
0:51and to serve your company's customers.
0:54And those customers, those users, machines, and agents,
0:58they don't care about what goes inside the factory.
1:02They don't care about how that factory is built, what infrastructure is used,
1:08what software is used.
1:09The only thing they care about is when the business outcome lands, and that comes
1:15when the first token lands.
1:17And so if you want to tie the infrastructure
1:20and AI factory investment to business return on investment,
1:24the bridge is the output, and the outputs are the tokens.
1:28And so today we are going to go deep into tokens, tokenomics,
1:32and I have two wonderful guests here to talk about that.
1:36So Shruti Koparkar leads our product marketing for AI infrastructure,
1:41and you recently, uh, co-authored an amazing white paper on tokenomics.
1:45We're gonna put that in the chat.
1:48And Tue Tran, who is our go-to-market specialist, and among many things
1:53that he does, he's doing and building a lot of internal tooling, uh, on tokenomics.
1:58So welcome Shruti and Tue.
2:00I'm super excited for you guys to be here today.
2:03Thank you. Thank you for having us. Thank you for
2:05having us. Yeah. And Tue, you didn't get the, the blue- Uh ... blue memo today?
2:11I did, but it's a little bit more muted
2:14than on Zoom.
2:15No, Zach, Zach and I had first dec- decided that Shruti was gonna sit here, uh,
2:20but then we didn't want you to, uh, feel left out.
2:24So, uh, now y- here you are.
2:25Uh, before we get started, uh, this is a live stream. You... We love your comments.
2:30We love your questions.
2:31I'm going to monitor the questions, and, uh, uh, we'll, we'll see
2:35and look forward to some interesting questions here.
2:38So Shruti, I wanna start with you.
2:40I wanna start with the very, very basics.
2:42What is a token,
2:43and how do you explain a token to somebody who has never really thought about a
2:48token?
2:48A token is the most fundamental, the most basic unit of artificial intelligence.
2:53So if you think about these language models that, you know, generate language,
2:58generate responses, it is essentially a set of numbers
3:01that represents typically four characters.
3:04If you want to simplify it further, you can think of it as a set of numbers
3:09that represents a word. So that is a token for a language model.
3:13For image generation models, we've all generated images of cats and, you know,
3:19whatever, moon and whatever else.
3:21Um, for image generation, a pixel is the most fundamental sort of, uh, unit, and
3:27so a token is a set of numbers that represents a pixel.
3:31For biology models, it could be something
3:33that represents the molecular structure of proteins.
3:37So that is what it is, and ultimately, all the AI responses, all the work
3:42that agents do is built on these tokens.
3:45Got it.
3:45Um, I always imagined, you know, when you talk about the AI factory and the output,
3:50there's the factory and then like cha-ching, like all the tokens are coming out.
3:54So it's kind of like that. So...
3:56And then, Tue, I'm gonna come to you.
3:59Uh, I don't think that the word is in the dictionary yet, but, uh, tokenomics,
4:05right? So we're talking about tokens and tokenomics.
4:08What does tokenomics mean, and why is suddenly everybody talking about
4:12tokenomics? No, I think, uh, at this rate,
4:15like we might see the word being the word of the year for 2026.
4:19I, I, I have some suspicion about that.
4:21Um, but essentially it is, not surprisingly, it is a combination of token
4:25and economics, and really it is a review, an examination of, um, the,
4:29the economics of generating a token.
4:31So what is the cost of generating a token?
4:33What is the demand for those tokens?
4:35What is the value of having those tokens being generated?
4:39And really at the end of the day, the, the difference, that difference between the,
4:43the, the value of that token and the cost of generating the token,
4:47that will be the spread that the business really care about,
4:50and it really decides whether an AI use case is scaled up and, and, uh,
4:54successful at the company or it's gonna be killed, uh, after the pilot stage.
4:58Got it. And so when you say business really cared about, these are the CFOs,
5:02these are the IT teams, the infrastructure teams, the security teams,
5:06they're all looking at tokens and tokenomics from different directions, which,
5:10you know, tokens feels like a broad bucket, right?
5:13So Shruti, h- uh, when you think about tokens, how do you break it down?
5:17Are there different buckets of like how to think about tokens and tokenomics?
5:22Yeah. So, you know, we do talk about tokens as a commodity, but even commodity,
5:27say for example a bag of rice, comes in different flavors, right?
5:32There's white rice, jasmine rice, brown rice, and all different quality.
5:37And the same is true for tokens.
5:39Not all tokens are created equal.
5:41Every token has a value associated with it,
5:44which depends on how much intelligence is embedded in the token
5:47and how fast does the token arrive.
5:49So the intelligence part of it is impacted by the model that generates the token.
5:54If it's a more complex, you know, more intelligent model,
5:57the tokens will have more intelligence.
6:00If the model is allowed to look at larger context,
6:03which is essentially how much data can it look at simultaneously,
6:07then the tokens it produces will be more intelligent.
6:10And finally, the rate at
6:11which tokens arrive are essentially the rate of token generation,
6:15or what we also call as interactivity.
6:18Now, these three things are an absolute way of looking at token value,
6:22but of course there's also a relative way of looking at token value in terms of how
6:27much token value does your use case need.
6:29Because if you are using premium tokens for use cases that don't need them,
6:34then you're wasting all that additional value that premium tokens are providing,
6:39and that's obviously bad for economics.
6:41So mapping your use cases to the right token value and token utility,
6:46that's one of... That's how we framed it.
6:49So token utility
6:50and considering what token utility do you need for a given use case is very
6:55important. Got it. So step one is get your arms around use cases and utility.
7:00And you mentioned something which is not all tokens are equal, which means
7:06that over the weekend you could be using the same number of tokens to do some
7:10unbelievable research and compute, and I was generating cat videos,
7:15and we've generated the same amount of tokens, right?
7:18Uh, is that- That, that is possible.
7:20And so when doing research, maybe I should be using the premium tokens.
7:26And for you, if all you're doing is generating cat videos,
7:30you don't need to pay for a premium model.
7:34I am professional at generating cat videos. Uh,
7:36but going back to not all tokens are equal- You're working with a lot of our
7:40customers, you're working with a lot of our industry specialists.
7:44How does that equate to different industries, different workloads, um,
7:49and use cases?
7:50Absolutely. So, uh, if I just take the, the two ax- two of the axes that, uh,
7:53Shruti just mentioned, uh, the, the intelligence of a token and the speed at
7:57which the token is generated.
7:58Um, and let's go through maybe two examples.
8:00So for some of our, uh, financial services capital markets customers,
8:05they have three different speed at which they are doing their trading or investing,
8:10uh, activities.
8:11They are really, really high frequency, really fast trading decisions
8:16that they need to make. And for those use cases, um, speed is everything.
8:21Latency, you have to have extremely low latency.
8:24However, the, the decision that they have to make in not even a split second,
8:29this... We're talking about machine-grade speed.
8:31But at that speed, the decisions that they need to make are actually quite simple.
8:37Uh, and so what they really care about is how do I maximize the speed at
8:41which I'm generating these simple tokens.
8:43Whereas
8:44if you broaden out into someone who is looking into a 10-year investment thesis or
8:48five-year investment thesis,
8:50and these theses are developed over weeks or months, right? A good report
8:54that is a good starting point for someone who is this sophisticated about
8:58investments, they can wait two hours, they can wait an hour.
9:02They...
9:02What they really care about is that the token is reasoned over and over by a really,
9:07really good model with a lot of context,
9:10with hundreds of different reports put into there, um,
9:13that there are sub-agents actively counter-feuding each other, really making sure
9:18that the, the, the arguments are sound.
9:20And so in that particular scenario, speed is of much less concern, but the accuracy,
9:25the intelligence is much, much more important.
9:28Another example I think that might be, uh,
9:31close to a lot of people watching this is, uh, for a retail store.
9:35If they are doing, um, something that is like monitoring a, a camera, um,
9:39th- there's a blueprint from our Metropolis team at NVIDIA where, uh,
9:43the model is really just- Monitoring the, the streaming video
9:47and it gives a one token answer.
9:49Is there an event going on?
9:51Let's say, let's say a theft going on.
9:53And in that split second, it just needs to give you a very short answer, yes or no,
9:58yes or no.
9:59And so it keeps watching that chunk of video over and over and over, yes or no,
10:03yes or no, one token.
10:04Whereas if that same retail chain is trying to do something, say captioning a video
10:09that their customers are sending over or a, an advertising video,
10:13then of the same 10-second chunks of videos,
10:15the model then has to generate 100 tokens, 200 tokens.
10:19But they don't have to do that live.
10:21They can do that overnight in a batch workload.
10:24And in that particular instance, again, the volume of tokens,
10:27the quality of tokens are a lot higher, but the speed is a lot lower.
10:31And so you really have, as a business,
10:34you really have to think about what are the right kinds of tokens
10:37that I am trying to generate for my use case.
10:40Those are, those are great examples. Thank you.
10:43I'm just gonna monitor the chat.
10:46I'm still trying to find the balance between listening intently and making sure
10:51that I get some questions in, uh.
10:53Okay, this is a good one.
10:56Template for, uh, calculating token,
10:58number of tokens for your most important projects.
11:01Yes, this is- Shruti, do you want to take that? Yeah, for sure.
11:05This is a question that a lot of our customers have asked us,
11:09and it really does come down to token demand.
11:12How do they understand or, you know,
11:14kind of try to figure out how many tokens will they need for their given use case?
11:19And the best way to think about it is sort of think about it in this layered
11:24approach.
11:25So at the very sort of bottom layer, layer number one, is your back of napkin math,
11:30right? How many users do you have?
11:33How many requests or sessions will they initiate, uh, in a single day or a month,
11:38whatever is your time period of analysis.
11:41And then how many tokens per request do you need?
11:45Now, that last piece depends on the use case.
11:47If you're looking at chat-based use case, you might need lesser tokens compared to
11:53if you were looking at an agentic use case.
11:55And so with those three numbers, you get what is sort of the base token demand.
12:00Now, that is the back of napkin math.
12:03You do need to consider some multipliers that are very critical
12:07that can explode your token demand very quickly.
12:10And some examples are, first, are you using reasoning models?
12:14Because if you are, then reasoning models generate thinking tokens
12:18that users don't see, but you're still having to generate them.
12:22Second example is, is this an agentic workflow?
12:25Because in an agentic workflow, the main agent, you know,
12:29spawns off multiple sub-agents, tool calls, and goes through this loop of plan, act,
12:34plan, act, you know, till it gets the outcome done.
12:38And so there's all of these agentic loops which also increase token demand.
12:43Um, another example is what we call the cache hit rate, which is essentially
12:48if the language model has seen the same prompt before,
12:51sometimes it doesn't need to recompute and reprocess that prompt
12:56because it can just access already processed tokens.
12:59And so that actually means that you can access cache tokens,
13:02which means you don't need to recompute.
13:05That brings down, in a way, your token demand.
13:07So that's sort of layer two.
13:09And then finally, you know, you also do need to consider the operational sort of,
13:13uh, factors, which is how does the workload vary through a day?
13:17You know, peak in the morning, lull at night, um, seasonal variation.
13:22Retail clients might see, um, a huge req- uh, requirement for tokens
13:27when they are trying to push product around holidays, um, and, you know,
13:31many different operational factors as well.
13:34So those are sort of the ways in which we think about it.
13:38Um, it sounds complex, but once you sort of start to work on it,
13:42and it all really just goes back to the use case.
13:45That's where you start and then work through this, and that helps you get to a,
13:50a good sort of approximation of what you need in terms of tokens.
13:54Yeah. The, there's a question about, uh, you mentioned cache hit rate.
13:59Is that with KV cache?
14:00Yes. Yes. Okay. That's
14:01exactly right. Can you maybe ex- like spend 30 seconds on KV cache
14:06and how this plays out into the token demand?
14:09Yeah, absolutely.
14:11So KV cache is essentially, um, sort of a short term in, you know, in simple words,
14:17it's kind of like the short term memory of a model.
14:20So when the model processes the input prompt,
14:23it computes these values corresponding to the prompt that are called KV values,
14:29and it stores those values
14:30because it needs those to be able to generate the rest of the output.
14:35Uh, but that also means that if there is, you know, other prompts that come in that,
14:40that match the, the previous one, you are able to just access those values.
14:45Um, so that's sort of what KV caching is.
14:48There's also something called, um, uh, you know, prompt, uh, caching and, uh,
14:53prefix matching, which is essentially taking the incoming input
14:57and seeing how much of it matches with what we've already seen.
15:01And that allows the AI, uh, to be able to be smart about not having to recompute.
15:06And you actually do see high cache hit rates across use cases like enterprise
15:11search, because you can imagine that within an enterprise search, so for example,
15:16the enterprise search we use if you're at NVIDIA,
15:20you can imagine a lot of people asking about, you know,
15:23"Where do I find my 401(k) info?
15:25Where do I find my pay stub?"
15:27Right? So there is a lot of similarity in the types of prompts that are generated,
15:32which means that there is a high likelihood of cache hit rate, um, or, you know,
15:37in terms of, uh, finding a prompt match.
15:39Got it. Makes sense. Thank you.
15:42So we talked about utility, uh, we talked about demand.
15:45So those are the two things that you need to get your arms around.
15:50Um, you also mentioned something about agentic AI,
15:53and I'll just remind all the viewers, last episode we talked in detail,
15:58so we'll post a link for the, the episode number three about how agentic AI
16:02and long-running agents are creating this, uh, token, token tsunami in some ways,
16:07right? And what are some of the factors.
16:10So, uh, it's a good tie-in, and it's probably worth going back to episode three, uh,
16:14because we spent quite, quite a bit of time talking about that.
16:18So we talked about utility, we talked about demand.
16:21Uh, so there's probably a supply side when we think about demand, um, and
16:25that is probably where the ROI calculation, right?
16:27Like it's probably a math, right? Ul-ultimately.
16:30So do you wanna talk a little bit about supply?
16:34Uh, of course. Um, so for supply, I think...
16:37I don't know if you're a car guy, uh, Kaushik, but a very simple analogy is, uh,
16:42an AI factory is somewhat like a car engine, where energy comes in
16:46and then you get something out.
16:49And so there's something here for a car engine, it's like horsepower.
16:52So if you think about the throughput of a, an AI factory,
16:55it's in a very similar concept.
16:57How many tokens can every engine, can every factory generate, uh,
17:00per second for you, right?
17:02And so that rate of generation of tokens depends on just a variety of factors,
17:06but I think we can think about it in maybe three main categories.
17:09What is the efficiency of the model
17:11that we're using in the factory to serve these tokens?
17:15What is the efficiency of the engine itself in serving these tokens?
17:18And then what is the, uh, efficiency of the, uh, software layer
17:22that we have to orchestrate and control the, um, the, the token generation?
17:26And so we start with the model layer.
17:29Uh, in previous years when LLMs just came out, right, like these token-- uh,
17:33these models have maybe fewer parameters.
17:36Parameters are what we use a lot to describe the size of a model.
17:40Even though they have fewer parameters,
17:42they are what can be considered a dense model, because every time
17:46that you want to generate a new token, you have to read
17:49and do math on every single one of those parameters, and that's a lot of math to do.
17:55And the number of parameters also correlate to the intelligence.
17:59So the trade-off is almost like you don't get a lot of intelligence,
18:03but you have to do a lot of math anyway.
18:04Whereas in modern, more modern architecture of these LLMs,
18:08you have what is called a mixture-of-experts architecture, one of the very popular,
18:13uh, architecture these days.
18:14Where you can have a lot more parameters,
18:17you will have a lot more knowledge trained into these models.
18:20Yet at the same time, every time that you want to generate a new token,
18:24you're only reading what is so-called the, the relevant experts.
18:27And so you actually do a lot less math, and yet you get a better answer.
18:32But there's a trade-off, as I'm sure you know, Kaushik, coming from, uh, uh,
18:36a history of, of networking, which is the networking has to be really, really good.
18:40Because for these, uh, more of, um, MoE, mixture-of-experts models,
18:44the experts may live on different GPUs.
18:47Mm. And so for the tokens to be generated correctly
18:49and effectively in this architecture,
18:51the GPUs have to talk to each other constantly over and over and over, uh, in,
18:56in those split seconds.
18:57And so you have to have really, really good networking to make sure that that, uh,
19:02uh, can, can happen very efficiently.
19:05And so that's just one of the many things
19:07that is in the efficiency of the engine itself,
19:10which is the efficiency of the hardware.
19:12And then finally, the software layer, super important.
19:15You have to be able to talk to the hardware in the right way.
19:20You have to be able to orchestrate and schedule when the tasks are run effectively,
19:24so there's as little downtime as possible
19:27and as much utilization of the whole cluster as possible. Many, many things in
19:32that software layer will contribute to how efficient you can generate
19:35and supply the tokens.
19:37And I think when we put all of those layers together, the software layer on top,
19:41the, uh, model layer, the hardware layer efficiency, that I think is
19:45when we have a full AI factory that is generating token, um, uh, from the energy
19:50that you put in it.
19:51Got it. Well, thank you.
19:53That was, that was, that was a, that was a great detailed explanation.
19:57So when we talk about supply, you had models, you had system,
20:01and you had software optimization.
20:03Um, Shruti, obviously, m- in the last episode also,
20:06we talked a lot about foundation models, open models.
20:10How do you think about it?
20:12You're working a lot on the AI infrastructure.
20:16How do these tie into the infrastructure?
20:19Yeah. So, you know, this triangle that, that Tue just pointed out, it,
20:24it really does come down to what tokens are you choosing
20:28and what infrastructure are you choosing to run those or generate those tokens.
20:34So in terms of what tokens you're choosing,
20:37it goes back to what models are you choosing.
20:40And for your use case, not every use case requires the premium model.
20:45Um, as an example, one of our, you know, telco customers actually found that,
20:51uh- Even though a lot of their developers always defaulted to the premium model,
20:56only about 8 to 16 percent of the tasks actually needed the premium model.
21:01Interesting. For all of the other tasks, they could have,
21:05and now they in fact have transitioned to using a post-trained open model.
21:11Or in some cases, maybe, you know, uh, just consuming, uh, tokens through an API,
21:16but on a smaller, you know, more pertinent model.
21:20And so the right model for the right task is extremely important.
21:24So how do you do right model for the right task?
21:27And it comes down to intelligent routing, because in an AI factory, you'll get
21:32so many requests,
21:33and you want to be able to route the request to the right model for that task.
21:38And this is where, uh, NeMo Switchyard, which is actually something
21:43that NVIDIA just recently launched as part of our NeMo Agent toolkit,
21:47comes into play.
21:48NeMo Switchyard can intelligently route the incoming requests based on what is the
21:53right model fit, what is the pricing, what is the cost profile of this model,
21:58and relative to the-- your request, do you need this cost profile or not, um,
22:03and a number of other factors.
22:05So that's, that's the sort of other piece of it that, that we help with.
22:10Another important thing when it comes to economics
22:13and choosing the right tokens is also just having the observability.
22:17At an AI factory scale, you, as an enterprise especially, need to know
22:21where are your tokens going, what tokens are being used for what use cases,
22:26is there actually business value?
22:28And so to be able to build that observability layer, we have NeMo Relay, um,
22:33which all of these are, by the way, open source software.
22:36That you can build on top of.
22:38So that's the choosing the right tokens.
22:41And then of course, as Tue mentioned, uh, is choosing the right infrastructure.
22:46And again, you know, he talked about, uh, the mixture-of-experts model.
22:50Um, our NVL72 system, which is our flagship system,
22:54is really well-suited for these mixture-of-experts models.
22:57Uh, but at the same time, enterprises might be working under different constraints,
23:02right? Like they, uh, they're not working always from a clean slate.
23:07And that's why we also offer a variety of different SKUs.
23:11We have the NVL72 systems, we have the HGX systems,
23:15all the way to our RTX PRO Servers.
23:17And so that helps you match the right infrastructure to your environment, like,
23:23you know, your footprint, your environmental requirements,
23:27as well as to try to optimize the economics well.
23:31And yes, cannot understate the importance of software.
23:34Uh, Dynamo is, you know, what Jensen has called the OS,
23:38like the operating system for the AI factory.
23:41And it does a lot of what, uh, Tue was talking about in terms of scheduling,
23:46sending the request to the right GPUs, um, sort of making sure
23:50that your AI factory stays highly utilized at all points of time.
23:54And, um, so yeah, it's absolutely that sort of-- There is,
23:58there's a reason we call it extreme co-design.
24:01It spans everything from compute, storage, networking, software, security,
24:06all across
24:07the board. Yeah, and it kind of, kind of, um, puts a spotlight on the AI factory
24:12because in some ways we, we talked about the reference architecture,
24:16compute networking software, all of that stuff,
24:19and then how it actually kind of gels with tokens and tokenomics makes sense,
24:24makes sense to me.
24:25Um- So we talked about utility, talked about demand, talked about supply.
24:30I wanna talk about my favorite topic, which I started with,
24:34which is the users don't care, right?
24:37The-- I want to just generate cat videos or however you want to route, route,
24:42that's fine. Or I want to do an important project.
24:46But ultimately, what matters is ROI and monetization, right?
24:50Because, you know, once you start a project, that's great, but all the CFOs,
24:55all the budget holders are being asked, "Well, what's the ROI?
24:58How do I tie these investments that I'm making to real tangible ROI?" So, uh,
25:03talk to me about monetization.
25:04Monetization, I think there are, uh, several categories in there.
25:09It is whether you are generating tokens that you will supply to the market
25:14and people can go and buy it from you.
25:17Uh, that's almost like a cloud model.
25:20Uh, you might be, um, s- generating direct revenue from generating intelligence
25:24that only you have, uh, that is different from offering a service.
25:28Uh, so that's the second way that you can make, um, money out of the tokens.
25:32And then there's also making your business just a lot more efficient
25:36and a lot more profitable by doing what you do,
25:39but having the intelligence from AI models, uh,
25:42really supporting these use cases for you.
25:44Yeah. Just building on that.
25:46Um, so yes, the selling tokens directly, of course.
25:50Um, there is what he mentioned is like the AI native products, which is, you know,
25:55products that are built using AI, right?
25:58AI has always been part of it.
26:00All these new generation companies that we are seeing. Exactly.
26:05All the new generation companies.
26:07Then there is using AI
26:08and infusing it into existing products to make the products better, right?
26:12Like to-- or to offer differentiated capabilities.
26:15And then finally, it is using AI to transform internal operations.
26:20So it doesn't show up as an external product for customers,
26:23but maybe you are increasing your employee productivity,
26:26maybe you are increasing or rather optimizing how you do supply chain management,
26:31things like that.
26:32Yeah, and I always say that for the, for the last bucket especially,
26:36AI gives everybody the opportunity to not just optimize a process internally,
26:41but is to reimagine the process, right?
26:43And really bring innovation, productivity, optimization throughout the organization.
26:48Um, and so when you think about monetization, when you think about utility,
26:53when you think about supply, when you think about w- uh, demand, you know, Tue, uh,
26:58I remember you and I having coffee together a few months ago,
27:01and we looked at what Shruti was doing, and we said,
27:05"We should think about building a tool here that we can use
27:08and we can work with our customers with,
27:11where they can actually bring all of this together."
27:14So you're working on a lot of our internal tooling.
27:16Maybe just share your experiences how you're bringing all of that stuff together in,
27:21in couple of tools of there.
27:22Absolutely, Kaushik.
27:24And, uh, maybe like a year or two years ago, after that coffee chat,
27:27I would have gone home and fired up my Excel and started to build a model.
27:31It's really as simple as that, right? Yeah.
27:34Everyone, every business, every organization, every CFO, CEO, COO can do this today.
27:39They can build a very simple model in Excel.
27:41But of course, with the magic of AI tools themselves,
27:45we are now able to create tools a lot more effectively with the right business
27:49logic and, and really have a very clear view and estimation of, of tokenomics.
27:54I think if there's one takeaway from doing all of this for me, uh,
27:58and for the organizations out there who have the question about tokenomics,
28:02the takeaway is just get started. Yeah.
28:05It sounds very complicated.
28:06There are many, many factors, and you can never go deep enough in the,
28:11in the rabbit holes.
28:12But at the same time, the drivers are also quite clear and, and simple
28:16when you start, and you just start small.
28:19You start at the high level.
28:21You keep refining where you can.
28:22You make it a team effort.
28:24The business side have to be involved.
28:26The business have to be there.
28:28They have to define, "What is my use case?
28:30What is the right level of speed
28:32and quality for my tokens?" And then the IT department, those are the experts
28:36that will tell you, "This is the right engine for us.
28:39This is the right option.
28:41Is it the f- is it the cloud? Is this on-prem?
28:43What, what do we need to consider here?" That team effort, you start small,
28:47you will get an answer, and that knowledge really is power
28:51because today probably a lot of organizations are still using, um, uh, end solutions
28:55that they trust and believe, and that's all well and good.
28:59But at least they need to be aware of what else is out there. Yeah.
29:04What, what are the possibilities that-- of tokenomics for them.
29:09Got it. Okay. I'm gonna just quickly check on a couple of questions here. Sure. And
29:18Okay.
29:18Uh, we have a question: What guardrails are recommended to restrict token tsunamis?
29:23That is, that is an interesting question.
29:26I was, um, I was actually talking to, uh, the Airports AI Alliance,
29:32where there were a number of CEOs from, from airports,
29:37and it was an engaging discussion where they were saying,
29:42"Should we impose token limits on, you know, um, uh, the, the people using AI?" And,
29:49um, to some degree, probably, right?
29:52Like, I'm, I'm not necessarily against it, but I think it really...
29:57I-I'm gonna sound like a broken record, uh, and just repeating today,
30:03but I think the best way to avoid a token tsunami, um,
30:06because the token tsunami will only be a token tsunami
30:10if you don't see the outcomes, right? Yeah.
30:14You will be happy if your token demand is huge, if you also see equivalent outcomes
30:20and ROI. And, and that's why it's all about starting with the use case,
30:25figuring out h- exactly what token utility you need,
30:29then figuring out how do I maximize those tokens with supply?
30:34What is the supply that I need?
30:36And it is, it is also about choosing infrastructure, but also the right models.
30:42Like, can I use models that are more efficient,
30:46that take more efficient trajectories on agentic?
30:50For example, Nemotron 3 Ultra uses less tokens per turn and less turns overall, and
30:56that way it... to do the same outcome, it requires less tokens.
31:00So if you can sort of minimize the tokens per task, minimize the cost per token,
31:05then you're minimizing the cost per task, and that's
31:09where the ROI starts to show up.
31:11And so it's less necessarily about minimizing token demand, but making sure
31:15that the ROI on the tokens that you think you need is, is appropriate.
31:19Yeah. And I think if you've done your homework, if you do the hard work upfront,
31:25you don't want to curb anything that actually fuels innovation, right? Like the...
31:32And the tokens are the, the fuel for the innovation.
31:36So, uh, that's a, that's a great point,
31:38and it's a good point for us to almost wrap up.
31:42But I think there are a few key things that I want to just summarize, right?
31:47Which is, you know, the, the first and foremost, keep your end customer in mind,
31:52which is the users, machines, agents
31:55that are going to change the arc of your company, uh,
31:58bring innovation products to your company.
32:01Second is, you...
32:02not all tokens are equal, and so you think about, like, different use cases,
32:07the industry that you are in, different workloads.
32:10And then really you think about utility, you think about demand,
32:13you think about supply, you think about governance of those tokens,
32:17you think about being smart about routing.
32:20And when you do all of those things, the monetization
32:23and the ROI almost takes care of itself, right?
32:26Because you have been thoughtful about how you architect it.
32:31And then the last thing, you said something which I absolutely love and I spend
32:36so much time every week talking to folks is,
32:39people a lot of times misunderstand AI factories because
32:43when they think about AI factories,
32:45people think this is like these giant gigawatt factories,
32:49which absolutely we-- there is a huge build-out in the industry infrastructure,
32:55and those are greenfield.
32:57You know, there are a lot of, like, cloud pe- uh, cloud providers are building.
33:01But for the enterprises, it can start with 16 GPUs, it can start with 32 GPUs,
33:06and you can crawl, walk, run, right?
33:09You can start with, you know, few use cases, bring those,
33:12understand those workloads, have a priority of projects,
33:16and then really start learning, you know, fine-tuning, no pun intended.
33:20And then you, um, you know, start scaling those AI factories, right?
33:25And so that's a great place for us to end.
33:29Uh, thank you so much for joining me today.
33:32Uh, this was, this was a great conversation.
33:35If you loved this conversation, please, uh, leave a heart or a comment.
33:41Uh, this will be on LinkedIn.
33:43It will get uploaded to our YouTube channel in a couple of days.
33:48And so thank you again. We'll be back with episode five, and we'll bring, uh,
33:53some new aspect of the enterprise, uh, AI factory at AI Factory Insider. Thank you.
33:59Thank you. Thank you. Thank you, Kaushik.

About this transcript

This page contains the full transcript of Tokenomics 101: What Are Tokens & Why They Matter | AI Factory Insider Ep. 4 by NVIDIA, generated from the public captions YouTube serves with the video. The transcript has 5,861 words across 563 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.