YT YouTube2Text

YouTube2Text → Transcripts → Transformers, the tech behind LLMs | Deep Learning Chapter 5

Transformers, the tech behind LLMs | Deep Learning Chapter 5 — Transcript

by 3Blue1Brown · 5,137 words · 443 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00The initials GPT stand for Generative Pretrained Transformer.
0:05So that first word is straightforward enough, these are bots that generate new text.
0:09Pretrained refers to how the model went through a process of learning
0:13from a massive amount of data, and the prefix insinuates that there's
0:16more room to fine-tune it on specific tasks with additional training.
0:20But the last word, that's the real key piece.
0:23A transformer is a specific kind of neural network, a machine learning model,
0:27and it's the core invention underlying the current boom in AI.
0:31What I want to do with this video and the following chapters is go through a
0:35visually-driven explanation for what actually happens inside a transformer.
0:39We're going to follow the data that flows through it and go step by step.
0:43There are many different kinds of models that you can build using transformers.
0:47Some models take in audio and produce a transcript.
0:51This sentence comes from a model going the other way around,
0:54producing synthetic speech just from text.
0:56All those tools that took the world by storm in 2022 like DALL-E and Midjourney
1:01that take in a text description and produce an image are based on transformers.
1:06Even if I can't quite get it to understand what a pi creature is supposed to be,
1:09I'm still blown away that this kind of thing is even remotely possible.
1:13And the original transformer introduced in 2017 by Google was invented for
1:18the specific use case of translating text from one language into another.
1:22But the variant that you and I will focus on, which is the type that
1:26underlies tools like ChatGPT, will be a model that's trained to take in a piece of text,
1:31maybe even with some surrounding images or sound accompanying it,
1:34and produce a prediction for what comes next in the passage.
1:38That prediction takes the form of a probability distribution
1:41over many different chunks of text that might follow.
1:45At first glance, you might think that predicting the next word
1:47feels like a very different goal from generating new text.
1:50But once you have a prediction model like this,
1:52a simple thing you could try to make it generate, a longer piece of text,
1:56is to give it an initial snippet to work with,
1:58have it take a random sample from the distribution it just generated,
2:02append that sample to the text, and then run the whole process again to make
2:05a new prediction based on all the new text, including what it just added.
2:10I don't know about you, but it really doesn't feel like this should actually work.
2:13In this animation, for example, I'm running GPT-2 on my laptop and having it repeatedly
2:17predict and sample the next chunk of text to generate a story based on the seed text.
2:22The story just doesn't actually really make that much sense.
2:26But if I swap it out for API calls to GPT-3 instead, which is the same basic model,
2:31just much bigger, suddenly almost magically we do get a sensible story,
2:35one that even seems to infer that a pi creature would live in a land of math and
2:40computation.
2:41This process here of repeated prediction and sampling is essentially
2:44what's happening when you interact with ChatGPT,
2:47or any of these other large language models, and you see them producing
2:50one word at a time.
2:52In fact, one feature that I would very much enjoy is the ability to
2:55see the underlying distribution for each new word that it chooses.
3:03Let's kick things off with a very high level preview
3:06of how data flows through a transformer.
3:08We will spend much more time motivating and interpreting and expanding
3:11on the details of each step, but in broad strokes,
3:14when one of these chatbots generates a given word, here's what's going on under the hood.
3:19First, the input is broken up into a bunch of little pieces.
3:22These pieces are called tokens, and in the case of text these tend to be
3:26words or little pieces of words or other common character combinations.
3:30If images or sound are involved, then tokens could be little
3:34patches of that image or little chunks of that sound.
3:37Each one of these tokens is then associated with a vector, meaning some list of numbers,
3:42which is meant to somehow encode the meaning of that piece.
3:45If you think of these vectors as giving coordinates in some very high dimensional space,
3:50words with similar meanings tend to land on vectors that are
3:53close to each other in that space.
3:55This sequence of vectors then passes through an operation that's
3:58known as an attention block, and this allows the vectors to talk to
4:01each other and pass information back and forth to update their values.
4:04For example, the meaning of the word model in the phrase "a machine learning
4:08model" is different from its meaning in the phrase "a fashion model".
4:12The attention block is what's responsible for figuring out which
4:15words in context are relevant to updating the meanings of which other words,
4:19and how exactly those meanings should be updated.
4:22And again, whenever I use the word meaning, this is
4:25somehow entirely encoded in the entries of those vectors.
4:29After that, these vectors pass through a different kind of operation,
4:32and depending on the source that you're reading this will be referred
4:35to as a multi-layer perceptron or maybe a feed-forward layer.
4:38And here the vectors don't talk to each other,
4:40they all go through the same operation in parallel.
4:43And while this block is a little bit harder to interpret,
4:45later on we'll talk about how the step is a little bit like asking a long list
4:49of questions about each vector, and then updating them based on the answers
4:53to those questions.
4:54All of the operations in both of these blocks look like a
4:58giant pile of matrix multiplications, and our primary job is
5:01going to be to understand how to read the underlying matrices.
5:06I'm glossing over some details about some normalization steps that happen in between,
5:10but this is after all a high-level preview.
5:13After that, the process essentially repeats, you go back and forth
5:17between attention blocks and multi-layer perceptron blocks,
5:20until at the very end the hope is that all of the essential meaning
5:24of the passage has somehow been baked into the very last vector in the sequence.
5:28We then perform a certain operation on that last vector that produces a probability
5:33distribution over all possible tokens, all possible little chunks of text that might
5:37come next.
5:38And like I said, once you have a tool that predicts what comes next
5:42given a snippet of text, you can feed it a little bit of seed text and
5:45have it repeatedly play this game of predicting what comes next,
5:49sampling from the distribution, appending it, and then repeating over and over.
5:53Some of you in the know may remember how long before ChatGPT came into the scene,
5:57this is what early demos of GPT-3 looked like,
6:00you would have it autocomplete stories and essays based on an initial snippet.
6:05To make a tool like this into a chatbot, the easiest starting point is to have a
6:09little bit of text that establishes the setting of a user interacting with a
6:13helpful AI assistant, what you would call the system prompt,
6:17and then you would use the user's initial question or prompt as the first bit of
6:21dialogue, and then you have it start predicting what such a helpful AI assistant
6:25would say in response.
6:27There is more to say about an added step of training that's required
6:30to make this work well, but at a high level this is the idea.
6:35In this chapter, you and I are going to expand on the details of what happens at the very
6:39beginning of the network, at the very end of the network,
6:42and I also want to spend a lot of time reviewing some important bits of background
6:46knowledge, things that would have been second nature to any machine learning engineer by
6:50the time transformers came around.
6:53If you're comfortable with that background knowledge and a little impatient,
6:56you could probably feel free to skip to the next chapter,
6:58which is going to focus on the attention blocks,
7:00generally considered the heart of the transformer.
7:03After that, I want to talk more about these multi-layer perceptron blocks,
7:06how training works, and a number of other details that will have been skipped up to
7:11that point.
7:12For broader context, these videos are additions to a mini-series about deep learning,
7:16and it's okay if you haven't watched the previous ones,
7:18I think you can do it out of order, but before diving into transformers specifically,
7:22I do think it's worth making sure that we're on the same page about the basic premise
7:27and structure of deep learning.
7:29At the risk of stating the obvious, this is one approach to machine learning,
7:33which describes any model where you're using data to somehow determine how a model
7:37behaves.
7:39What I mean by that is, let's say you want a function that takes in
7:42an image and it produces a label describing it,
7:44or our example of predicting the next word given a passage of text,
7:48or any other task that seems to require some element of intuition
7:51and pattern recognition.
7:53We almost take this for granted these days, but the idea with machine learning is that
7:57rather than trying to explicitly define a procedure for how to do that task in code,
8:02which is what people would have done in the earliest days of AI,
8:05instead you set up a very flexible structure with tunable parameters,
8:09like a bunch of knobs and dials, and then, somehow,
8:11you use many examples of what the output should look like for a given input to tweak
8:16and tune the values of those parameters to mimic this behavior.
8:19For example, maybe the simplest form of machine learning is linear regression,
8:24where your inputs and outputs are each single numbers,
8:27something like the square footage of a house and its price,
8:30and what you want is to find a line of best fit through this data, you know,
8:34to predict future house prices.
8:37That line is described by two continuous parameters,
8:40say the slope and the y-intercept, and the goal of linear
8:43regression is to determine those parameters to closely match the data.
8:48Needless to say, deep learning models get much more complicated.
8:52GPT-3, for example, has not two, but 175 billion parameters.
8:58But here's the thing, it's not a given that you can create some giant
9:01model with a huge number of parameters without it either grossly
9:05overfitting the training data or being completely intractable to train.
9:10Deep learning describes a class of models that in the
9:13last couple decades have proven to scale remarkably well.
9:16What unifies them is that they all use the same training algorithm,
9:19it's called backpropagation, we talked about it in previous chapters,
9:22and the context that I want you to have as we go in is that in order for this
9:26training algorithm to work well at scale, these models have to follow a certain
9:30specific format.
9:31And if you know this format going in, it helps to explain many of the choices for how a
9:36transformer processes language, which otherwise run the risk of feeling kinda arbitrary.
9:41First, whatever kind of model you're making, the
9:43input has to be formatted as an array of real numbers.
9:46This could simply mean a list of numbers, it could be a two-dimensional array,
9:50or very often you deal with higher dimensional arrays,
9:53where the general term used is tensor.
9:56You often think of that input data as being progressively transformed into many
10:00distinct layers, where again, each layer is always structured as some kind of
10:04array of real numbers, until you get to a final layer which you consider the output.
10:09For example, the final layer in our text processing model is a list of numbers
10:13representing the probability distribution for all possible next tokens.
10:17In deep learning, these model parameters are almost always referred to as weights,
10:22and this is because a key feature of these models is that the only way these
10:25parameters interact with the data being processed is through weighted sums.
10:30You also sprinkle some non-linear functions throughout,
10:32but they won't depend on parameters.
10:35Typically, though, instead of seeing the weighted sums all naked
10:38and written out explicitly like this, you'll instead find them
10:41packaged together as various components in a matrix vector product.
10:46It amounts to saying the same thing, if you think back to how matrix vector
10:50multiplication works, each component in the output looks like a weighted sum.
10:54It's just often conceptually cleaner for you and me to think
10:58about matrices that are filled with tunable parameters that
11:01transform vectors that are drawn from the data being processed.
11:06For example, those 175 billion weights in GPT-3 are
11:10organized into just under 28,000 distinct matrices.
11:14Those matrices in turn fall into eight different categories,
11:17and what you and I are going to do is step through each one of those categories to
11:21understand what that type does.
11:23As we go through, I think it's kind of fun to reference the specific
11:27numbers from GPT-3 to count up exactly where those 175 billion come from.
11:31Even if nowadays there are bigger and better models,
11:34this one has a certain charm as the first large-language
11:37model to really capture the world's attention outside of ML communities.
11:41Also, practically speaking, companies tend to keep much tighter
11:44lips around the specific numbers for more modern networks.
11:47I just want to set the scene going in, that as you peek under the
11:50hood to see what happens inside a tool like ChatGPT,
11:53almost all of the actual computation looks like matrix vector multiplication.
11:57There's a little bit of a risk getting lost in the sea of billions of numbers,
12:01but you should draw a very sharp distinction in your mind between
12:05the weights of the model, which I'll always color in blue or red,
12:08and the data being processed, which I'll always color in gray.
12:12The weights are the actual brains, they are the things learned during training,
12:16and they determine how it behaves.
12:18The data being processed simply encodes whatever specific input is
12:22fed into the model for a given run, like an example snippet of text.
12:27With all of that as foundation, let's dig into the first step of this text processing
12:31example, which is to break up the input into little chunks and turn those chunks into
12:35vectors.
12:37I mentioned how those chunks are called tokens,
12:39which might be pieces of words or punctuation,
12:41but every now and then in this chapter and especially in the next one,
12:44I'd like to just pretend that it's broken more cleanly into words.
12:48Because we humans think in words, this will just make it much
12:51easier to reference little examples and clarify each step.
12:55The model has a predefined vocabulary, some list of all possible words,
12:59say 50,000 of them, and the first matrix that we'll encounter,
13:03known as the embedding matrix, has a single column for each one of these words.
13:08These columns are what determines what vector each word turns into in that first step.
13:15We label it W_E, and like all the matrices we see,
13:18its values begin random, but they're going to be learned based on data.
13:23Turning words into vectors was common practice in machine learning long before
13:27transformers, but it's a little weird if you've never seen it before,
13:30and it sets the foundation for everything that follows,
13:33so let's take a moment to get familiar with it.
13:36We often call this embedding a word, which invites you to think of these
13:39vectors very geometrically as points in some high dimensional space.
13:44Visualizing a list of three numbers as coordinates for points in 3D space would
13:48be no problem, but word embeddings tend to be much much higher dimensional.
13:52In GPT-3 they have 12,288 dimensions, and as you'll see,
13:55it matters to work in a space that has a lot of distinct directions.
14:01In the same way that you could take a two-dimensional slice through a 3D space
14:05and project all the points onto that slice, for the sake of animating word
14:08embeddings that a simple model is giving me, I'm going to do an analogous
14:12thing by choosing a three-dimensional slice through this very high dimensional space,
14:16and projecting the word vectors down onto that and displaying the results.
14:21The big idea here is that as a model tweaks and tunes its weights to determine
14:25how exactly words get embedded as vectors during training,
14:28it tends to settle on a set of embeddings where directions in the space have a
14:33kind of semantic meaning.
14:34For the simple word-to-vector model I'm running here,
14:37if I run a search for all the words whose embeddings are closest to that of tower,
14:42you'll notice how they all seem to give very similar tower-ish vibes.
14:46And if you want to pull up some Python and play along at home,
14:48this is the specific model that I'm using to make the animations.
14:51It's not a transformer, but it's enough to illustrate the
14:54idea that directions in the space can carry semantic meaning.
14:58A very classic example of this is how if you take the difference between
15:02the vectors for woman and man, something you would visualize as a
15:05little vector in the space connecting the tip of one to the tip of the other,
15:09it's very similar to the difference between king and queen.
15:15So let's say you didn't know the word for a female monarch,
15:18you could find it by taking king, adding this woman minus man direction,
15:22and searching for the embedding closest to that point.
15:27At least, kind of.
15:28Despite this being a classic example for the model I'm playing with,
15:31the true embedding of queen is actually a little farther off than this would suggest,
15:35presumably because the way queen is used in training data is not merely a feminine
15:39version of king.
15:41When I played around, family relations seemed to illustrate the idea much better.
15:46The point is, it looks like during training the model found it advantageous to
15:50choose embeddings such that one direction in this space encodes gender information.
15:56Another example is that if you take the embedding of Italy,
16:00and you subtract the embedding of Germany, and add that to the embedding of Hitler,
16:04you get something very close to the embedding of Mussolini.
16:08It's as if the model learned to associate some directions with Italian-ness,
16:13and others with WWII axis leaders.
16:16Maybe my favorite example in this vein is how in some models,
16:19if you take the difference between Germany and Japan, and add it to sushi,
16:24you end up very close to bratwurst.
16:27Also in playing this game of finding nearest neighbors,
16:30I was very pleased to see how close cat was to both beast and monster.
16:34One bit of mathematical intuition that's helpful to have in mind,
16:37especially for the next chapter, is how the dot product of two
16:40vectors can be thought of as a way to measure how well they align.
16:44Computationally, dot products involve multiplying all the
16:47corresponding components and then adding the results, which is good,
16:51since so much of our computation has to look like weighted sums.
16:55Geometrically, the dot product is positive when vectors point in similar directions,
17:00it's zero if they're perpendicular, and it's negative whenever
17:03they point in opposite directions.
17:06For example, let's say you were playing with this model,
17:09and you hypothesize that the embedding of cats minus cat might represent a sort of
17:14plurality direction in this space.
17:17To test this, I'm going to take this vector and compute its dot
17:20product against the embeddings of certain singular nouns,
17:23and compare it to the dot products with the corresponding plural nouns.
17:27If you play around with this, you'll notice that the plural ones
17:30do indeed seem to consistently give higher values than the singular ones,
17:33indicating that they align more with this direction.
17:37It's also fun how if you take this dot product with the embeddings of the words one,
17:41two, three, and so on, they give increasing values,
17:44so it's as if we can quantitatively measure how plural the model finds a given word.
17:50Again, the specifics for how words get embedded is learned using data.
17:54This embedding matrix, whose columns tell us what happens to each word,
17:57is the first pile of weights in our model.
18:00Using the GPT-3 numbers, the vocabulary size specifically is 50,257,
18:04and again, technically this consists not of words per se, but of tokens.
18:10The embedding dimension is 12,288, and multiplying those
18:14tells us this consists of about 617 million weights.
18:18Let's go ahead and add this to a running tally,
18:20remembering that by the end we should count up to 175 billion.
18:25In the case of transformers, you really want to think of the vectors
18:28in this embedding space as not merely representing individual words.
18:32For one thing, they also encode information about the position of that word,
18:36which we'll talk about later, but more importantly,
18:39you should think of them as having the capacity to soak in context.
18:43A vector that started its life as the embedding of the word king, for example,
18:47might progressively get tugged and pulled by various blocks in this network,
18:51so that by the end it points in a much more specific and nuanced direction that
18:55somehow encodes that it was a king who lived in Scotland,
18:58and who had achieved his post after murdering the previous king,
19:01and who's being described in Shakespearean language.
19:05Think about your own understanding of a given word.
19:08The meaning of that word is clearly informed by the surroundings,
19:11and sometimes this includes context from a long distance away,
19:15so in putting together a model that has the ability to predict what word comes next,
19:19the goal is to somehow empower it to incorporate context efficiently.
19:24To be clear, in that very first step, when you create the array of
19:27vectors based on the input text, each one of those is simply plucked
19:30out of the embedding matrix, so initially each one can only encode
19:33the meaning of a single word without any input from its surroundings.
19:37But you should think of the primary goal of this network that it flows through
19:41as being to enable each one of those vectors to soak up a meaning that's much
19:45more rich and specific than what mere individual words could represent.
19:49The network can only process a fixed number of vectors at a time,
19:52known as its context size.
19:54For GPT-3 it was trained with a context size of 2048,
19:57so the data flowing through the network always looks like this array of 2048 columns,
20:02each of which has 12,000 dimensions.
20:05This context size limits how much text the transformer can
20:08incorporate when it's making a prediction of the next word.
20:12This is why long conversations with certain chatbots,
20:15like the early versions of ChatGPT, often gave the feeling of
20:18the bot kind of losing the thread of conversation as you continued too long.
20:23We'll go into the details of attention in due time,
20:25but skipping ahead I want to talk for a minute about what happens at the very end.
20:29Remember, the desired output is a probability
20:31distribution over all tokens that might come next.
20:35For example, if the very last word is Professor,
20:37and the context includes words like Harry Potter,
20:40and immediately preceding we see least favorite teacher,
20:43and also if you give me some leeway by letting me pretend that tokens simply
20:47look like full words, then a well-trained network that had built up knowledge
20:51of Harry Potter would presumably assign a high number to the word Snape.
20:56This involves two different steps.
20:58The first one is to use another matrix that maps the very last vector in that
21:03context to a list of 50,000 values, one for each token in the vocabulary.
21:08Then there's a function that normalizes this into a probability distribution,
21:12it's called softmax and we'll talk more about it in just a second,
21:15but before that it might seem a little bit weird to only use this last embedding
21:19to make a prediction, when after all in that last step there are thousands of
21:23other vectors in the layer just sitting there with their own context-rich meanings.
21:28This has to do with the fact that in the training process it turns out to be
21:32much more efficient if you use each one of those vectors in the final layer
21:36to simultaneously make a prediction for what would come immediately after it.
21:40There's a lot more to be said about training later on,
21:43but I just want to call that out right now.
21:45This matrix is called the Unembedding matrix and we give it the label WU.
21:50Again, like all the weight matrices we see, its entries begin at random,
21:53but they are learned during the training process.
21:56Keeping score on our total parameter count, this Unembedding
21:59matrix has one row for each word in the vocabulary,
22:02and each row has the same number of elements as the embedding dimension.
22:06It's very similar to the embedding matrix, just with the order swapped,
22:10so it adds another 617 million parameters to the network,
22:13meaning our count so far is a little over a billion,
22:16a small but not wholly insignificant fraction of the 175 billion
22:20we'll end up with in total.
22:22As the very last mini-lesson for this chapter,
22:24I want to talk more about this softmax function,
22:26since it makes another appearance for us once we dive into the attention blocks.
22:31The idea is that if you want a sequence of numbers to act as a probability distribution,
22:36say a distribution over all possible next words,
22:39then each value has to be between 0 and 1, and you also need all of them to add up to 1.
22:45However, if you're playing the deep learning game where everything you do looks like
22:49matrix-vector multiplication, the outputs you get by default don't abide by this at all.
22:55The values are often negative, or much bigger than 1,
22:57and they almost certainly don't add up to 1.
23:00Softmax is the standard way to turn an arbitrary list of numbers
23:04into a valid distribution in such a way that the largest values end up closest to 1,
23:08and the smaller values end up very close to 0.
23:11That's all you really need to know.
23:13But if you're curious, the way it works is to first raise e to the power
23:17of each of the numbers, which means you now have a list of positive values,
23:21and then you can take the sum of all those positive values and divide each
23:25term by that sum, which normalizes it into a list that adds up to 1.
23:30You'll notice that if one of the numbers in the input is meaningfully bigger than the
23:34rest, then in the output the corresponding term dominates the distribution,
23:37so if you were sampling from it you'd almost certainly just be picking the maximizing
23:42input.
23:42But it's softer than just picking the max in the sense that when other values
23:47are similarly large, they also get meaningful weight in the distribution,
23:50and everything changes continuously as you continuously vary the inputs.
23:55In some situations, like when ChatGPT is using this distribution to create a next word,
23:59there's room for a little bit of extra fun by adding a little extra spice into this
24:04function, with a constant T thrown into the denominator of those exponents.
24:09We call it the temperature, since it vaguely resembles the role of temperature in
24:14certain thermodynamics equations, and the effect is that when T is larger,
24:18you give more weight to the lower values, meaning the distribution is a little bit
24:22more uniform, and if T is smaller, then the bigger values will dominate more
24:26aggressively, where in the extreme, setting T equal to zero means all of the weight
24:31goes to maximum value.
24:33For example, I'll have GPT-3 generate a story with the seed text,
24:37"once upon a time there was A", but I'll use different temperatures in each case.
24:43Temperature zero means that it always goes with the most predictable word,
24:48and what you get ends up being a trite derivative of Goldilocks.
24:53A higher temperature gives it a chance to choose less likely words,
24:56but it comes with a risk.
24:58In this case, the story starts out more originally,
25:01about a young web artist from South Korea, but it quickly degenerates into nonsense.
25:06Technically speaking, the API doesn't actually let you pick a temperature bigger than 2.
25:11There's no mathematical reason for this, it's just an arbitrary constraint imposed
25:15to keep their tool from being seen generating things that are too nonsensical.
25:19So if you're curious, the way this animation is actually working is I'm taking the
25:2420 most probable next tokens that GPT-3 generates,
25:27which seems to be the maximum they'll give me,
25:29and then I tweak the probabilities based on an exponent of 1/5.
25:33As another bit of jargon, in the same way that you might call the components of
25:37the output of this function probabilities, people often refer to the inputs as logits,
25:42or some people say logits, some people say logits, I'm gonna say logits.
25:46So for instance, when you feed in some text, you have all these word embeddings
25:50flow through the network, and you do this final multiplication with the
25:53unembedding matrix, machine learning people would refer to the components in that raw,
25:58unnormalized output as the logits for the next word prediction.
26:03A lot of the goal with this chapter was to lay the foundations for
26:06understanding the attention mechanism, Karate Kid wax-on-wax-off style.
26:10You see, if you have a strong intuition for word embeddings, for softmax,
26:14for how dot products measure similarity, and also the underlying premise that
26:19most of the calculations have to look like matrix multiplication with matrices
26:23full of tunable parameters, then understanding the attention mechanism,
26:27this cornerstone piece in the whole modern boom in AI, should be relatively smooth.
26:32For that, come join me in the next chapter.
26:36As I'm publishing this, a draft of that next chapter
26:38is available for review by Patreon supporters.
26:41A final version should be up in public in a week or two,
26:44it usually depends on how much I end up changing based on that review.
26:47In the meantime, if you want to dive into attention,
26:49and if you want to help the channel out a little bit, it's there waiting.

About this transcript

This page contains the full transcript of Transformers, the tech behind LLMs | Deep Learning Chapter 5 by 3Blue1Brown, generated from the public captions YouTube serves with the video. The transcript has 5,137 words across 443 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.