YT YouTube2Text

YouTube2Text → Transcripts → Transformer Decoder Arch During Training Time - Pen & Paper Explaination | Transformer Series Part2

Transformer Decoder Arch During Training Time - Pen & Paper Explaination | Transformer Series Part2 — Transcript

by yogesh murumkar · 11,315 words · 1,895 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Hello everyone, this is Yogesh Muramar.
0:03In this video, we are going to
0:05understand decoder part of transformer
0:07architecture in very much detail. I'll
0:10be using all these handwritten diagrams
0:13to make the understanding very simple
0:17and easy. You can have a look at all
0:20these diagrams one by one. So that at
0:23least it will give an idea about the
0:25flow which we are going to follow to
0:28understand the entire decoder
0:30architecture.
0:33Fine. And uh one more thing everyone uh
0:36you can download all these diagrams. I
0:38have already created a PDF file and the
0:40link for the same is given in the
0:42description. So if you just go if you
0:45just try to download that PDF file and
0:48then follow this uh explanation I think
0:52uh it will be good for you to understand
0:55the things in uh less time okay and that
0:58too uh properly you will understand the
1:01architecture you'll be able to follow it
1:03properly that's what I want to say fine
1:06so yes these are all the diagrams even
1:08which I'll be using I just try to show
1:10it to you uh so that at least you'll get
1:14an idea what kind of a material we'll be
1:16making use of here to understand this
1:18particular architecture.
1:21One very important thing is that in this
1:24video the decoder architecture
1:27we will study is from the training
1:29perspective. You can see I underline
1:31this particular word training that means
1:34how the decoder architecture behaves
1:36when training is running. We will
1:38discuss decoder architecture working
1:41during inference or prediction in the
1:43next video. Fine.
1:46So until now we have thoroughly studied
1:50the decoder uh the encoder part of the
1:52transformer architecture in previous
1:54video and the link for the same is also
1:56given in the description. Even if
1:59someone has not watched the encoder part
2:02anyway I'll just revise it in short at
2:05the start of this video.
2:07But I recommend if you want to
2:09understand the things about encoder in
2:11detail then please refer previous
2:12[clears throat] video of encoder part
2:18and another uh other than this if you
2:21want to understand the concepts like
2:23tokenization embedding positional
2:25encoding self attention multi-air
2:26attention mask self attention cross
2:29attention layer normalization and uh
2:32feed forward neural network then these
2:34things are also explained in the
2:36for which link is given in the
2:38description.
2:40Okay. Now we are actually ready to study
2:42the architecture of decoder.
2:47Always remember that the decoder behaves
2:50differently during training and
2:53inference. Whenever I say inference I
2:55mean to say during prediction. Okay.
2:58Although the architecture is same
3:01during inference it becomes auto
3:03reggressive.
3:05Okay. Remember this what I want to say
3:07everyone is the decoder architecture
3:09which we are going to discuss.
3:12So during inference
3:15or prediction it becomes what it becomes
3:19auto reggressive
3:22and during training
3:25during training it becomes
3:29nonauto reggressive.
3:33Okay, please remember this non-ordering.
3:38Now see as I've told you that the
3:43decoder behaves differently during
3:45training and inference although the
3:46architecture is same but during
3:50inference it becomes auto reggressive
3:54and during training it works in nonauto
3:57reggressive manner. Now what is the
3:59meaning of this one? What is the meaning
4:02of let's say this auto reggressive see
4:05auto means what? Self
4:07and regressive means what? Regressive
4:10means predicting based on previous
4:13values.
4:15Okay. Predicting based on previous
4:18values.
4:20So auto reggressive means what? What is
4:21the meaning of auto reggressive?
4:23Auto reggressive it means predicting the
4:26next thing using all the previous things
4:29uh you already generated.
4:33Think of it like completing a sentence
4:35word by word. For example, even see
4:38let's say if the input is something like
4:40this. The cat
4:44sat on dash.
4:48So step one will be what?
4:52Step one will be to predict
4:56predict the next word. So let's say the
4:58next word is matt.
5:01Okay. Then again the input will become
5:03what?
5:04The cat
5:06sat on mat
5:10and then again next word. So step two
5:13will be what? Step two will be what?
5:15Step two will be predict the next word.
5:17Let's say next word is
5:20this. predict the next word and the next
5:22word is let's say and okay then
5:27input will be what the input will become
5:30like this the cat sat on match and
5:39and then step three will be what step
5:43three will be to predict
5:48let's say the next word next word is
5:49let's say this
5:51and so on. Okay, so each new word
5:55depends on all previously generated
5:57words. Each new word depends on all
6:02previously generated words. That's auto
6:06reggressive everyone.
6:08You feed your own output back as input
6:11for the next step. Okay, this is in
6:13short auto reggressive.
6:16What is non-auto?
6:19Always remember as I've told you decoder
6:23behaves nonauto regggressive in in
6:25non-auto regggressive manner during
6:27training time. This is also called as
6:29non-auto regggressive is also called as
6:31what?
6:34Teacher forcing. Remember this word
6:36later on you will need this. Okay. Nona
6:39autogressive means what? Teacher
6:41forcing.
6:43It means don't use your predictions. Use
6:47the correct answers all at once. I'll
6:50repeat, don't use your own predictions,
6:54use the correct answer all at once.
6:59So it means instead of instead of
7:03waiting for its own output, the model
7:05receives the entire correct sentence in
7:07one shot
7:10but with a mask blocking future word. So
7:14it cannot cheat.
7:17For example, even
7:20position one
7:22position let's say there are sentences
7:24like there are words in the sentence
7:26right like this one this one this one so
7:27this is position one this is position
7:29two this is position three and so on so
7:32position one can see
7:35start token there's something called a
7:36start token we'll talk about it start
7:38means this is something which will tell
7:40the
7:42machine that okay you can start the work
7:43so position one can see start token
7:47position two
7:50can see the start token and itself see
7:56before this before this the cats had
7:59let's say there's a start token so
8:01position one can see only start position
8:03two can see start and the
8:06position three can see start
8:11the
8:12and cat
8:15Okay, all positions computed
8:17simultaneously parallel and fast. This
8:21is non-auto ree regressive. Fine. But
8:24please don't worry
8:27everyone. Uh you will understand these
8:28concepts once you go through the entire
8:30process shown in this video. Okay. So I
8:33I just tried to explain you in very much
8:36short way the meaning of uh auto
8:40reggressive and non-regressive. But
8:42remember during training decoder behaves
8:45in non-auto regggressive manner and
8:48during inference or prediction it
8:49behaves in auto reggressive manner. If
8:51these things are not clear later on it
8:53will be more clear. Fine. So in this
8:56video we'll focus on
8:59training process of decoder. In the next
9:03video I will tell you how the decoder
9:05works during inference time. Fine.
9:10So let's see decoder architecture from
9:12the training perspective. Now to explain
9:15the decoder architecture, I will show
9:17you two representations of the decoder
9:21which will give you an overview of how
9:23the decoder architecture is built.
9:28Once you get that particular overview
9:30then we will do a deep dive and try to
9:33understand how the how all the
9:35components are interconnected with each
9:36other through some simple example. Okay.
9:41So first simplified overview and then a
9:44deep dive
9:46and all of this will start with the main
9:49transformer architecture diagram and
9:52that is nothing but this one which you
9:54are able to see here.
9:56Fine. I think this particular diagram is
9:58now familiar to everyone.
10:02So remember this part, this encoder part
10:04we have already discussed in detail in
10:06previous video. Now we are going to
10:08focus on the decoder part and that too
10:10during training. Fine.
10:13Next the most simplified representation
10:16of this particular diagram would be
10:20something like this. You see
10:24transformer architecture in a very
10:25simplified manner I can show it like
10:27this encoder and decoder point. So
10:30transformer is nothing but what
10:32transformer is nothing but a box. In a
10:35very simple way I can say it's a box.
10:37You can think of transformer as a single
10:39box. And inside that single box there
10:42are two more boxes. This one and this
10:45one. The first box is called the encoder
10:47whose job is to encode the input. And
10:49the second box is called the decoder
10:51whose job is to decode the encoded
10:53sentence or encoded input and print the
10:57output.
10:59Now we understand at the simplest level
11:02that transformer is a box
11:05with two more boxes inside it. Now let
11:08us add a bit more complexity.
11:14You can see this diagram.
11:16Okay, you can see this particular
11:18diagram here.
11:20Encoder is made up of six boxes.
11:25Okay, the good news is that the encoder
11:28box is not a single box. It is made up
11:31of six boxes. So what I mean is that the
11:34transformers encoder is actually made up
11:37of six encoder blocks. 1 2 3 4 5 six.
11:41So here are those six blocks.
11:46Okay. And the same thing is true for
11:49decoder also. The transformers decoder
11:51is also not a single box but it is made
11:54up of six blocks. 1 2 3 4 5 6. So here
12:01here you can see six block six boxes for
12:05uh six blocks for encoder and six blocks
12:07for decoder.
12:09Okay. Now the obvious curiosity comes
12:12what goes on uh inside one single
12:16encoder block. I already discussed this
12:18in the previous video of transformer
12:20encoder architecture but just for
12:22revision let's discuss uh one once more
12:27what goes on inside one single encoder
12:29block.
12:31Okay
12:34you see one single encoder block looks
12:37like this.
12:40It has two more blocks inside it. One,
12:44two.
12:47One is the self attention block
12:50and other is the feed forward neural
12:53network block.
12:55These two together make up one single
12:58encoder block like this.
13:02But if you don't know self attention
13:04feed forward neural network, no problem.
13:06I'll explain it uh as and when required.
13:10So it will not stop you from
13:12understanding decoder architecture.
13:14Fine. Now next everyone
13:19this one this one as you can see this is
13:24the cutout of encoder from the original
13:26transformer architecture. As you can see
13:29here
13:31in the original transform architecture
13:33as you can see multi retention add and
13:36nom feed forward add a norm. So whatever
13:39I'm showing you here everyone whatever
13:41I'm showing you here this one
13:44is the cut out of encoder from the
13:47original transformer architecture you
13:49see multi- redempion addenom feed
13:50forward addenom
13:53okay and as we know there are six
13:56encoder blocks
13:59as we know that there are six encoder
14:01blocks
14:03this is how the encoder architecture
14:06looks like you see first encoder block
14:09multi- retention add anom feed forward
14:11addon and then even you can imagine it
14:13like this this exactly same encoder
14:15block will be here second encoder third
14:18encoder fourth encoder fifth encoder
14:20they will be exactly same and finally
14:21sixth encoder
14:24okay so this is how the encoder
14:26architecture looks like and one more
14:29thing all the encoder blocks are
14:31identical as I already told you and we
14:33already seen this in the previous
14:35discussion of encoder architecture of
14:37transformer
14:39All the encoder blocks are identical
14:41meaning their architecture is exactly
14:44copy paste. This encoder block first
14:45encoder block is uh similar to second,
14:48third, fourth, fifth and sixth. Okay,
14:52their architecture is exactly copy
14:53paste. Same the parameters are
14:56different. The weights and the biases
14:57inside will be different. Their values
14:59will be different. But from the
15:01architecture point of view, it is
15:03exactly same. Whether you talk about
15:05first, second, third, fourth or sixth
15:07encoder clock.
15:10Fine. Think of it this way. For example,
15:14a very common example is given. Let's
15:16say if you take some company's phone,
15:19say an iPhone 15, then all iPhone 15s
15:22are designed in exactly the same way.
15:24But which apps are installed inside that
15:27phone varies from user to user.
15:29Similarly, encoders architecture is same
15:31for all blocks. But parameter values
15:34inside are different. Okay. I think it
15:37makes you understand the
15:40meaning of whatever I'm saying. Fine.
15:44Now
15:45[clears throat]
15:48as I've told you this this decoder
15:50everyone also has six six box right six
15:55boxes inside it or it is basically six
15:58decoders we have like this right just
16:01like the encoder the decoder is also
16:04made up of six blocks
16:07the obvious curiosity comes what goes
16:09inside one single decoder log.
16:14So the simplified representation
16:17if I want to show you here everyone
16:21for uh this one decoder block will be
16:25like this. You see
16:30okay you can see here
16:35a single decoder block consist of three
16:38blocks.
16:40First block is
16:43masked self attention.
16:46Second is cross attention which is also
16:49called encoder decoder attention.
16:53Third is feed forward neural network.
16:57As you can see in this particular
16:58diagram right this is one single block
17:02of decoder. These three blocks together
17:06make up a single decoder block. Now if
17:08the curiosity is coming that how are
17:10these blocks interconnected with each
17:13other then the answer will come from a
17:15diagram which is an exact cutout from
17:17the main diagram something like this one
17:20even
17:21this is the cutout of one decoder block.
17:25So at the start as I've already shown
17:27you this main transformer architecture
17:31diagram if you look at this part here
17:33you see this one masked multi- retention
17:37adder nom multi- retention add nom feed
17:40forward addon norm
17:42okay so
17:47as you can see this particular diagram
17:50is an exact cutout from the main diagram
17:52you see masked multiate attention
17:55add a normal retention adder norm feed
17:58forward adder norm.
18:01Okay, this this cutout is basically
18:03taken from the main diagram only. So you
18:05can see how much data is
18:07data flow is happening and how are all
18:10these blocks interconnected with each
18:12other and this is exactly what we need
18:15to understand in this video. I think you
18:16know everyone right this these
18:18connections right this one this one
18:21right these connections are called as
18:23what residual connections
18:26or sche connections
18:28we already have discussed about these
18:30things in detail in the
18:33transformer encoder architecture part
18:37okay so this is the exact cutout of one
18:40decoder block I can say
18:44fine Now [clears throat]
18:46as I've told you you can see how much
18:49data flow is happening and how are all
18:51these blocks interconnected with each
18:53other. You can see or you can have a
18:55look at this particular diagram format
18:57and this is exactly what we need to
18:59understand in this video. Now the fun
19:02thing is again that we do not have just
19:06one decoder block as we discussed
19:07already. We have got six decoder blocks
19:10which means our decoder
19:13looks something like this.
19:17See everyone a decoder looks something
19:20like this. Okay, don't worry about this
19:23particular part at least for now. But
19:25you see our decoder looks something like
19:27this. This is the first decoder block.
19:30You can see this is the exact cutout
19:32which I shown you in the previous
19:33diagram.
19:36Right?
19:38And from this is first decoder block.
19:39From this from here the input goes to
19:44the second decoder block.
19:46Right? As I've told you we have got six
19:48decoder blocks which means our decoder
19:51looks entire decoder looks something
19:53like this one.
19:57From here the input goes to
20:01this one. This first decoder block.
20:02Okay. From here as as you can see I'm
20:04highlighting it. From here the input
20:06goes to first decoder block. Okay, this
20:09is our first decoder block. And this
20:12first decoder block as you can see
20:14everyone
20:16it is having three blocks inside it.
20:21Masked multi head attention you can say.
20:23Then this is called as cross attention.
20:26Okay. And then feed forward neural
20:28network.
20:31Okay. This first decoder blocks output
20:35goes to second decoder block
20:39which which also has its three blocks
20:42inside it like this
20:45and second decoder blocks output goes to
20:47the third and then fourth and so on.
20:51like this you reach the sixth decoder
20:53block
20:55and the output from the sixth decoder
20:57block goes to an output block.
21:01So everyone this is our output block.
21:05This is our output block I can say.
21:12Okay.
21:14And here you get your final output after
21:16this. So
21:19uh don't worry. I know it looks uh like
21:22a lot of information flow is happening.
21:23All these blocks are interconnected in
21:25some way. And uh yes, this is exactly
21:30what we need to understand in this
21:31video. So don't worry about this. I'll
21:33explain you each and everything. Okay,
21:34this is how our decoder looks like. I'll
21:38be coming to this particular diagram
21:39again and again as and when required.
21:42Now what we are going to do is we are
21:45going to take this entire diagram and
21:49feed an example sentence into this as a
21:52input
21:54and through that sentence a journey we
21:56will try to learn how this entire
21:58architecture works. So don't worry it
22:01would be very very interesting and by
22:03the end of this video you will
22:04understand each and everything.
22:08Fine. Next,
22:10I have divided this entire explanation
22:13into three parts.
22:16In the first part, we will try to
22:18understand a section where we take a
22:20decoder input and prepare it. That means
22:23even this part.
22:26Okay, this is let's say input part.
22:30Okay, just to make the expression easy,
22:32I'll name it as input part.
22:35There are two to three kinds of work you
22:37do here and we will discuss those. So
22:40this is the first part of the
22:42explanation. Okay.
22:45In the second part of the explanation, I
22:47will exactly explain what is happening
22:49inside a single decoder block. This will
22:52be the largest part of the explanation.
22:56Here we will spend
22:58some time but here we will understand
23:01everything properly.
23:03Okay. So I'm talking about I'm talking
23:06about this second part even okay second
23:11part
23:16and the good thing is that since all
23:18decoder blocks are identical
23:21if you understood what is happening
23:23inside this one block first decoder
23:26block you'll automatically understand
23:27the next five blocks second third fourth
23:30fifth and sixth
23:32okay and then everyone In the third part
23:35of the explanation,
23:38third part of the explanation that is
23:40this output block, I'll explain what is
23:43happening inside this output block.
23:46Okay, once we are done with first,
23:48second and then we will go with the
23:49third and that is nothing but the output
23:53block. So the entire deep dive can be
23:56divided into three parts.
23:59Input block processing I can say. Okay.
24:03input block processing
24:09the entire portion inside decoder.
24:14Okay, this entire portion inside decoder
24:17and then the output block.
24:21This is how we are going to proceed and
24:23to carry out this entire discussion
24:26we will use an example and that example
24:28is a machine translation task.
24:32Okay. So for example, let's say I have a
24:37English sentence.
24:41Okay. English sentence is let's say we
24:45are ready.
24:49Fine. So as it's a machine translation
24:51task, let's say I want to translate this
24:53into Hindi. So Hindi translation is
24:56what? H
24:59or you can take any example anyone.
25:01Okay. R
25:03so English to Hindi. Okay, so to carry
25:06out this entire discussion even we will
25:08use this example and it's a machine
25:10translation task.
25:12Okay,
25:13it's a machine
25:16translation task.
25:21Fine. So we'll be taking English to
25:24Hindi translation as an example. And
25:25remember here we are talking about
25:28training.
25:30So at this point you can assume that we
25:32have a training data set of English to
25:35Hindi sentences.
25:38For now you can assume you have just one
25:40row of data where the English sentence
25:42is we are ready and it's Hindi
25:45translation is
25:49apart from this you do not have any
25:50other English to Hindi sentence pairs.
25:53Okay just to make the things easy
25:55everyone we'll take a very simple
25:56example. So we are going to send this
25:59data to our decoder.
26:02Right? As I've told you
26:05whenever I say decoder.
26:07So in a decoder everyone we have as we
26:11already seen there are six decoder
26:14blocks. First, second, third, fourth,
26:15fifth, sixth. Right? So we are going to
26:18send this particular data this data to
26:22our decoder. But before that one thing I
26:24want to clarify the English sentence
26:26will go inside encoder. Everyone the
26:30English sentence will go inside encoder
26:35and the encoder will generate contextual
26:37embeddings or encoded embeddings from
26:39it. The output of the encoder will be
26:42what? Contextual embeddings
26:47or encoded embeddings which will go to
26:50the decoder. So how how exactly this
26:52thing will be used we'll talk about it.
26:57Okay. So in short before decoder starts
27:00its work before this decoder starts
27:02starts its work the encoder will have
27:05already done its work.
27:08It will have already processed
27:11this English sentence.
27:15Understand? Okay. encoder will have
27:19already processed this English sentence
27:21and for every token of it for every
27:24token of it,
27:26it will have generated contextual or
27:29encoded embeddings which will be
27:32available with us.
27:34Fine. And then the decoder will start
27:38its work with Hindi sentence.
27:42Okay? Like in this case, this is our
27:44Hindi sentence corresponding to this
27:46English English sentence. Now I really
27:49hope that this setup is completely clear
27:51to you.
27:54Fine.
27:55Okay. Now we will start our explanation
27:59on decoder architecture. And the first
28:02thing we will discuss is this input
28:05part.
28:08The input portion has four operations.
28:10The names of all four operations are
28:13like this also we have discussed during
28:15uh encoder everyone. But see which four
28:19operations as you can see here shifted
28:21right. Okay. So shifted right or we call
28:24it as shift right.
28:27This is the first operation
28:30in the input part input block
28:32processing. Second operation everyone is
28:35tokenization.
28:38I hope you understand the meaning of
28:40tokenization. Then third aran is nothing
28:42but what? Embedding
28:45and fourth is nothing but positional
28:49encoding.
28:51Positional encoding.
28:55So the input portion has four
28:57operations.
29:00Right shift tokenization embedding
29:03positional encoding. The entire input
29:06block is very simple in its work. It
29:08takes your output sentence.
29:11In our case, the output sentence is
29:15Hindi sentence H.
29:19And after processing it, it brings it
29:23into such a form that you can send it to
29:26this decoder block. Okay? you can send
29:30it to the
29:33decoder's first block I can say or first
29:36decoder block. Okay, this input block
29:39processing or this input part it will do
29:42all these things and it will process
29:44this Hindi sentence your output sentence
29:46in such a way that it can be easily
29:48passed to the uh input it it can be
29:52easily passed as a input to the first
29:54decoder block. This is just overview.
29:57Now let's see what's happening in input
30:00part in detail. So for this everyone we
30:04will use
30:07this particular diagram for the detailed
30:09explanation. Okay. I hope so you are
30:11able to see this clearly.
30:15Okay you can have a look at this. This
30:17is very important diagram everyone which
30:18will make the things very easier and
30:20simple for you to understand.
30:23Fine.
30:26Now [clears throat]
30:29as I've told you this diagram we will
30:31use for the
30:33detailed explanation. Okay. So in our
30:35case what was the English sentence?
30:39English sentence is we are ready.
30:42Okay. And Hindi sentence is
30:45remember this. So as I've told you our
30:48encoder is ready
30:50with the processed English sentence. And
30:53now we will take the Hindi sentence. The
30:55first operation we will apply on the
30:57entire Hindi sentence is what is right
31:01shifting. This first operation given
31:04shift right or right shifting as as even
31:06it is mentioned in the diagram here as
31:09well. Shifted right or if you go to the
31:12if you go to the original uh transformer
31:15architecture you see the first operation
31:17shifted right. Okay same thing I'm
31:19talking about.
31:22So our encoder is ready with the process
31:23English sentence and we will take the
31:25Hindi sentence. The first operation we
31:27will apply on this entire Hindi sentence
31:29is right shifting in this operation.
31:34Okay, as you can see
31:37I passed this as a input to right shift
31:40first operation.
31:42Okay.
31:44So
31:46in this operation you add a kind of a
31:50token at the front of the sentence which
31:53is basically start token. You see right
31:56shift what happens here? We add a
31:59special token called as start token at
32:01the start of the sentence
32:05or or you can say at the front of the
32:07sentence. So we add start token
32:11at the front of.
32:14So this token is like a flag that starts
32:17the training process. It means our
32:19training process is starting from here.
32:23Okay.
32:24So now our transformed input is what?
32:27Start token then h then t then.
32:32The second operation is tokenization.
32:37You already know what tokenization is.
32:39Tokenization means you break down a
32:41sentence into tokens.
32:46Those tokens could be words, bgrams,
32:48engrams. It depends on you. We are using
32:52a simple tokenization technique. We are
32:54basically converting every word into a
32:58token. So as soon as we pass our entire
33:01sentence through this to tokenizer this
33:03entire sentence through this tokenizer
33:08we get four tokens
33:10start token
33:12H token tar token head token these are
33:16four separate tokens
33:19fine
33:20the third operation is embedding layer
33:23okay so you see we are done with right
33:26shift we are done with tokenization and
33:27then embedding Okay. So the third
33:29operation is emitting layer. The job of
33:32this layer is you have English words or
33:35Hindi words
33:37return right. Let's say from in our case
33:39Hindi words. So what is the job of this
33:42embedding layer?
33:46See this embedding layer takes every
33:49token generated
33:53in the previous step or I can say the
33:56job of this layer is let's say if you
33:58have English or Hindi words written out
34:00but your machine cannot understand words
34:02right so there has to be some way where
34:04you can convert your words into numbers
34:08so the embedding layer this layer takes
34:11every token and generates vectors from
34:16It it generates vectors from it. In the
34:19original transformer paper, every vector
34:21you see is 512 dimension vector. So this
34:25will be 512 dimension. This will be 512
34:27dimension. 512 dimension vector. And
34:29this also will be 512 dimension vector.
34:33Okay.
34:35And with the help of this 512 dimension
34:38numbers or vectors, we form a vector or
34:41or with the help of this 512 numbers, we
34:42form a vector. So start token gets
34:48this even corresponding vector for start
34:50token we got even corresponding vector
34:52you can say okay which is also 512
34:56dimensions which is also a 512
34:58dimensions token gets E2 which is again
35:03512 dimension vector
35:07gets E3 and similarly this high gets E4
35:13512 dimensional vector. So at this point
35:17we have achieved that for every token we
35:20have vectors which the machine can
35:22process. It can understand and it can
35:24process. But there is a problem. We have
35:27not informed our system in any way which
35:31word came first and which came later.
35:35And you know that in NLP related task
35:38this information of when which word is
35:41used in the sentence is very crucial. So
35:43to bring this information we use a
35:45technique which we call positional
35:48encoding. That means after embedding
35:50everyone we are going to discuss about
35:52positional encoding.
35:55Okay. So positional encoding. Okay. This
35:58this is basically uh you can see this
36:02this is positional encoding. Okay.
36:04positional encoding.
36:08So positional encoding generates a
36:11vector for every unique position.
36:14Like for position one, it will generate
36:16a vector. For position two, it will
36:18generate a vector. For position three
36:19and for position four, it will generate
36:22one one vector.
36:25And here too we make sure that every
36:29vector is gen uh every vector that is
36:31generated is of 512 dimensions.
36:35This one, this one, this one, this one.
36:38Okay. So at this point I have two sets
36:40of vectors. First set of vectors are my
36:44embeddings that is this one E1, E2, E3,
36:47E4.
36:50Okay.
36:52and second set of vectors are my
36:54positional embedding vectors that is P1,
36:57P2, P3, P4.
37:01Now what we have to do is add these two.
37:05So you see here addition addition
37:06addition addition here. Okay? Or I can
37:09show you in this diagram also. You see
37:13embedding vectors position encoding and
37:16you see this addition operation. Okay,
37:19that is why I added this addition
37:22operation here.
37:26Fine.
37:28So what we have to do is add these two.
37:32Okay, add these two. So we will add
37:35start tokens embedding vector.
37:38Start tokens embedding vector is even.
37:41So
37:43we will add start tokens embedding
37:45vector with the first position's
37:47embedding vector. We will add hum tokens
37:50embedding vector with the second
37:51position embedding vector and so on.
37:54That means E1 plus P1, E2 plus P2, E3
37:56plus P3 and E4 + P4 respectively.
38:00Okay, since both these sets are 512
38:04dimension vectors, this addition will
38:05happen and finally we will get our input
38:07vectors
38:09X1, X2, X3, X4. This one is basically as
38:14many tokens as you have in the sentence
38:17you will have that many vectors. So we
38:19have 1 2 3 four tokens. So we will get 1
38:232 3 four. Okay, we will get four vectors
38:26here. And this final set of vectors is
38:31what goes into the decoder.
38:36So this x1, x2, x3, x4 they are very
38:41important vectors because they are the
38:42vectors that will uh that we will send
38:46to the first block of decoder
38:49and that is nothing but even we are
38:51here. Now we are here. Okay, first block
38:54of decoder that means we are here.
38:58I really hope the working of the entire
39:00input block has been understood by you.
39:02Okay, this input part has been
39:04understood by you.
39:07Now we move to the main part which is
39:10the first decoder block that is nothing
39:13but this one. Okay, this one everyone
39:15will see now.
39:19Our first decoder block has three main
39:21operations. First is the operation where
39:23we apply mask multi-air attention that
39:26is this one.
39:28Second is the operation where we apply
39:30cross attention that is this one.
39:34And third is where we apply feed forward
39:37neural network. So let us see these
39:39three one by one how each one is
39:42executed.
39:44Okay.
39:47So for that let's once again come to
39:48this diagram. We have four input vectors
39:53as you can see x1 x2 x3 x4. Right? This
39:56was the last point in this diagram we
39:59discussed.
40:01And now we are sending these four input
40:04vectors to our decoder block where first
40:07they will they'll be received by multi
40:10mask multi attention block. As you can
40:12see easily here
40:14inside the decoder first decoder what is
40:17the first thing? Mask multi head
40:19attention.
40:21So if you have watched my earlier video
40:23then perhaps you remember that this
40:25masked multi-headed attention block
40:28works exactly
40:31like a normal attention uh normal multi-
40:34attention block. Okay, this one this one
40:36I'm talking about
40:38there is only one big difference here in
40:41this case masking is implemented.
40:44So what will happen here is that for
40:47every input vector corresponding
40:49contextual
40:51embedding vector will be generated like
40:54for X1
40:57here you can see like for X1 here we
41:01will get something like Z1
41:05okay
41:07for X2 we'll get for X2 we'll get Z2
41:12and same for X3 and X4 4 that is Z3 and
41:15Zed4 respectively. But there is one big
41:19difference in the process of generating
41:22Z1. Okay, this is Zed 1. In the process
41:25of generating Z1 that is this one,
41:30we will only consider start token.
41:34Please try to understand that in the
41:35process of considering uh in the process
41:37of generating Z1, we will only consider
41:42star token. We will not consider
41:46h
41:48token t token high token.
41:52Okay remember.
41:54So it is something like this one to
41:56generate the zed 1 we will only consider
42:00start token. to generate
42:03Z2. When we generate Z2, we will only
42:06consider
42:08this star token
42:11and H token.
42:14We will not consider TAR and H token
42:19when we generate Z3 that is this one. So
42:23I think you are getting my point. We
42:26will consider
42:29star token, h token and we will consider
42:33tar token. We will not consider high
42:36token. And when we generate Z4,
42:39we will consider all four that is start
42:44and then hum and then tar and then head
42:47token.
42:49So that's the only difference between a
42:51normal multi-head attention and masked
42:54multi head attention. That means even
42:56here we are trying to do masking.
43:00Okay, I really hope you understand what
43:01is happening here. If you want the
43:03detailed understanding, go back to video
43:06mentioned in the description where I
43:08explain the [clears throat] entire
43:09process of how how you do masking and
43:12how this output comes out. But whatever
43:16I have told you exactly same thing
43:17happens everyone. This is how masking
43:19will be taken care.
43:21Okay. I think you are getting my point.
43:26See once again like as I've told you
43:30when we generate Z1 we will consider
43:32only start token. To generate Z1 we will
43:34consider only start token. To generate
43:37Z2 we will consider start and H. To
43:39generate Z3 we will consider start H and
43:41T. And to generate Z4 we will consider
43:43all start and then H and then P and then
43:46H.
43:48Okay.
43:49So I really hope you understand what is
43:51happening here. If you want detail
43:52explanation as I've told you please go
43:54and please check the video in the
43:55description. Fine. So at this point we
44:00have our masked multi head attention
44:03output. See here
44:06that means now we are here.
44:09Okay. Now we are here
44:12we have our mass to multi-year retention
44:14output. Now what comes next
44:17just ahead is add and non block. You see
44:20add and n. So here we have to perform an
44:22addition first and then perform
44:23normalization on whatever the result
44:25comes. The question is what do we do to
44:28add? The answer is very simple. We have
44:31to add the result of this masked
44:34multi-year redention and that is nothing
44:37but the result is
44:40zed 1 zed2 zed3 zed4. We need to add
44:44this result
44:47with the original input vectors and that
44:49is nothing but x1 x2 x3 x4 and that is
44:52because here in this diagram and even as
44:56you see there is a residual connection.
44:59You see this one residual connection.
45:01You see
45:02we will have to add this result with
45:05this
45:07uh with this particular thing
45:10right. These are the original input
45:13vectors. From here we will get original
45:16input vectors or in the in the uh
45:18original transformer architecture
45:20diagram as well you see here this as
45:22I've told you right this is nothing but
45:23what this is residual or skip
45:26connection. Okay, residual or skip
45:30connection which we even also discussed
45:32in encoder architecture. Fine. So as
45:36there's a residual connection here,
45:38this is uh residual connection as I've
45:41told you or a skip connection. So see
45:46everyone what's going to happen. So that
45:48residual connection you see here like
45:50this I shown.
45:52Okay. So from here we will send this x1
45:56x2 x3 x4
45:59we will send this ahead and we will add
46:02it with z1 zed2 zed3 zed4.
46:07So you see this addition operation
46:10everyone I hope you are able to see
46:12addition operation and this is basically
46:14skip connection or residual connection.
46:17All these vectors are still 512
46:20dimensional vectors.
46:22All of them. Okay. All of this addition
46:26even this addition like here I shown you
46:28this Z1 + X1 Z2 + X2 Z3 + X3 Z4 + X4.
46:33This will happen. And let's suppose even
46:35we got this one.
46:38Okay, these are all 512 dimension
46:41vectors.
46:43All these vectors are still 512
46:45dimensional vectors. So you can perform
46:47addition between
46:49these
46:52right you can perform addition between
46:53them because they are of same dimension
46:55by addition by adding them you get let's
46:58say these new vectors which we call zed
47:011 dash zed 2 dash zed3 dash and zed 4
47:05dash
47:06okay after this comes the normalization
47:10step as you can see in the first decoder
47:14block as well so what we have done is we
47:16are here now.
47:19Sorry. Uh we got this one right? We we
47:22we already have done
47:24mass multi uh mask multi addition and
47:27then we added and after this you see
47:28normalization. So this addition is done
47:31and now we are going to go with
47:32normalization. So after this uh comes a
47:35normalization step and in normalization
47:37step which normalization which
47:38normalization layer have we will be
47:41using layer normalization here. Right?
47:44You can see here layer normalization.
47:47So what we do is we send all these
47:49vectors to the layer norm layer. Z1
47:53dash, zed 2 dash, zed3 dash, zed4 dash.
47:56We will send it to the
47:59layer norm. Okay. And the layer norm
48:02layer calculates
48:04mean. It calculates the sigma or the
48:08standard deviation
48:10for every vector.
48:12Okay, for every vector and performs
48:15normalization. I think in machine
48:17learning definitely you must have come
48:18across what exactly is normalization.
48:20Okay, so for every vector here you will
48:23get a normalized vector. For every
48:24vector here everyone you will get
48:26normalized vector
48:29and we do this particular step because
48:32you want to make your training somewhat
48:34stable. You always you would always want
48:38that whatever numbers
48:41are in these vectors
48:43they are in small given range
48:47so that the training process will be
48:50stable. Okay. So at this point I can say
48:55now everyone we have reached here.
48:59Okay. We have reached here from addon
49:03norm. We got our output from add nom. We
49:05got our output. Now we will send it to
49:08the next block which is our cross
49:11attention. Okay. This one area which is
49:14our cross attention.
49:21This is cross attention block. Fine. Now
49:26cross attention block is the most
49:28interesting portion of this entire
49:30decoder architecture. Basically this is
49:33the first decoder block. Okay. Why I
49:36said that this is the most interesting
49:38portion of this entire decode
49:40architecture? Because here a very
49:43important work happens.
49:48You make your input sequence which is an
49:51uh English sentence.
49:54You make your input sentence which is
49:57English sentence and your output
49:59sequence which is a Hindi sentence you
50:02make them talk to each other.
50:05So basically every token that is present
50:08in your English sentence you find out a
50:12similarity score with every token that
50:14is present in your Hindi sentence. And
50:17this work is happening in cross
50:19attention block.
50:21Okay.
50:23And that is why you will notice that in
50:26this block diagram input is coming from
50:29two places. One input is coming from
50:32here. You see this one, this arrow.
50:37Okay?
50:39One input is coming from here from the
50:41previous masked attention block. And one
50:44is coming from here. You see this? This
50:47one is here.
50:50Okay.
50:51And that input is actually coming from
50:53the encoder that is coming from
50:58encoder.
51:01So if you look at the original
51:04transformer architecture diagram
51:08here,
51:10you will notice that when encoder
51:13completes its work, it sends it its
51:16output to this place. You see this is
51:19the output of the encoder.
51:22So it sends this output to this place.
51:26You see this one.
51:31Okay. So you can see when the encoder
51:35completes its work, it sends its output
51:37to decoder at this place. And that is
51:41what you will get here. Okay? Or I can
51:46say here that is what you will get here.
51:51So if I summarize the cross attention
51:53block for you, it works exactly like
51:56normal head normal multi head attention
51:59block just with one big difference.
52:04Unlike a normal multi head attention
52:06block where you send only one sequence
52:09input in cross attention you send two
52:12sequences. One is your English sentence
52:15coming from encoder and second is your
52:18Hindi sentence coming from your
52:21decoder's previous step. This is your
52:23decoder. Decoder's previous step is
52:25here. That means one is coming from here
52:26and one is coming from here.
52:31Okay. So you do this
52:35because as I have told you you need
52:37three sets of vectors.
52:40Okay. And which are those are which are
52:43those three sets of vectors
52:45query
52:48Q
52:50and value. Query
52:54Q and value.
52:58Okay. So query Q
53:04and value.
53:06This is what we need.
53:09Once you get
53:11the query key and value vectors, the
53:14process ahead is exactly the same as in
53:17normal attention block.
53:21Fine.
53:24See everyone
53:26the query key and value vectors, right?
53:30What operations we'll be doing on this?
53:34We'll talk about it. But your query
53:37vectors or your query vector basically
53:40come from your decoder.
53:42Okay, as you can see
53:45this will give me query vector.
53:50Remember this.
53:53And your key and value vectors coming
53:56from here that is encoder.
54:00Okay. your key and value vectors.
54:06So this will be query key area. Okay,
54:08this will be query. So your key and
54:10value vectors
54:12will be coming from encoder.
54:16Okay, or [clears throat]
54:19in our diagram you can see this diagram.
54:22This is where this is from where you
54:24will get query and this is where you
54:26will get key and value. Okay, it was
54:28key. I think I written Q. Okay, it is
54:32key. Key key and value. Key and value
54:34will be coming from encoder and query
54:35will be coming from decoder.
54:40Fine. And as I've told you once you give
54:42the key value and query vectors the
54:44process is exactly same as in normal
54:46attention block. Okay. So just to make
54:50the explanation easy now
54:54let's once again come to this diagram
54:56here.
54:59Okay.
55:01So if I can show you this
55:03diagrammatically
55:05whatever I have discussed right. So
55:07please understand first of all last time
55:10we were at this point right we got the
55:12Z1 dash zed 2 dash zed3 dash and zed4
55:15dash.
55:18Okay,
55:20from the previous mask attention we got
55:24Z1 norm, Zed 2 norm, Z3 norm and Z4
55:29norm.
55:31Okay, that is nothing but normalized
55:32vectors. How we got it? Uh I think you
55:34understand that.
55:38Okay.
55:40Now,
55:42as I've told you,
55:45we got this from our uh previous mass
55:49attention. Right? Now, we are going to
55:54send this to cross attention.
55:57So, as you can see this particular
55:58diagram,
56:00we were here right now. Whatever the
56:03normalized vectors which we got, we are
56:05going to send it to the cross attention.
56:08But here we will not only send this Z1
56:11nom, zed2 nom, zed3 nom, zed4 nom. We
56:14will also send the encoder embeddings
56:16that are coming here. That means we will
56:18send this here to the cross attention.
56:20But as well as but as well as you can
56:22see uh we will send here from here,
56:24[clears throat] right? But as well as
56:25what we'll do is we'll also send the
56:27encoder embeddings to the cross
56:28attention.
56:31So from zed 1 nom, zed 22 nom, zed3 nom,
56:34zed4 norm we will get query vectors.
56:38Okay, that means even this will give me
56:40query. This will give me query. This
56:42will give me query and this will also
56:43give me query vector.
56:46And from the encoder's output that is
56:48coming,
56:50see I I shown you here see this one from
56:52the encoder's output that is coming
56:55that is this one everyone. That is this
56:58one.
57:01we will expect key and value vectors. So
57:04it means
57:06this will give me key and value. This
57:09will also give me key and value. Key and
57:11value. And this will also give me key
57:14and value.
57:16Once you get the key and value vectors,
57:20you will do all the work that you do in
57:22an attention block. And finally you will
57:25get your contextual embeddings
57:27one for each token of your output
57:29sequence.
57:33So if your output sentence has four
57:36tokens then for each token you will get
57:39corresponding contextual embedding
57:41vector. Let us call them as Z C1 Z C2 Z
57:45C3 Z C4
57:47and C here means cross attention output.
57:49C means because this is the output of
57:51cross attention. So I will name it as Z
57:53C1, Z C2 and so on. So C means cross
57:56attention.
57:59Okay. So at this point we have reached
58:03here. We have reached here.
58:08We have cross attention output and right
58:10ahead again there is add and now.
58:14So again the question comes what will be
58:16added? Very simple. you will add the
58:17cross attention output with the previous
58:19steps output which is masked attention
58:23output. Okay, you see this residual
58:25connection here and this is cross
58:28attention output. So mask attention
58:30output and cross attention output
58:32everyone will be added. So
58:36notice that what I have done
58:40not only have I sent this Z1 norm, zed2
58:43nom, zed3 nom, zed4 norm to cross
58:46attention block.
58:49Not only this one anyone, but
58:52[clears throat]
58:55as you can see this zed1 nom, zed2 nom,
58:59zed3 nom, zed4 norm.
59:02I also send them here
59:06where they will be added with Z C1 Z C2
59:09Z C3 Z C4 which is cross attention
59:13output. So what we'll get is we'll get Z
59:16C1 dash, Z C2 dash, Z C3 dash and Z C4
59:20dash
59:22and then we normalize them using
59:26layer normalization after which we get Z
59:30C1 norm, Z C2 norm, Z C3 norm, Z C4
59:35norm.
59:39Okay.
59:41[clears throat]
59:43So at this point we have reached here on
59:48our output journey or decoder journey I
59:52can say.
59:54Okay as I got normalized
59:57as I got normalized thing everyone right
1:00:00after add and norm I got normalized
1:00:02vector. So here we are in our decoder
1:00:04journey
1:00:06and the next thing is as you can see
1:00:09here the next thing is feed forward
1:00:11neural network block.
1:00:14Okay.
1:00:17So first
1:00:21before discussing about this feed
1:00:23forward neural network
1:00:26first I'll tell you the architecture of
1:00:27this feed forward neural network and the
1:00:30architecture is the same architecture
1:00:32that you have seen during explanation of
1:00:34the encoder architecture in the previous
1:00:37video something like this one.
1:00:41This feed forward neural network
1:00:43consists of two layers. The first layer
1:00:45has
1:00:472048
1:00:48neurons.
1:00:50Okay, these are neurons. 2048 neurons
1:00:55and their activation is rectified linear
1:00:59unit railway activation function.
1:01:03Okay. The second layer has 512 neurons
1:01:10and its activation function is linear.
1:01:17Fine.
1:01:19Now if you talk about parameters, this
1:01:24feed forward neural network expects 512
1:01:28dimension input. So from here everyone,
1:01:30it will expect 512 dimension input.
1:01:34Okay, 512 dimension input.
1:01:37So here we have 512.
1:01:40Okay.
1:01:42And here we have 2 0 48.
1:01:45And since it's fully connected neural
1:01:48network, the weights here, the weights
1:01:51here, everyone, the weights here will be
1:01:55512 cross 2048.
1:02:01Okay. So many weights you have there.
1:02:04After that, every neuron has its own
1:02:06bias.
1:02:10So in total we have 2048 biases here.
1:02:13How many biases? 2048 biases
1:02:19because 2048 neurons are here. So we are
1:02:22calling this weight let's say as W1.
1:02:25This entire weight let's say as W1 and
1:02:28the set of biases let's say as B1.
1:02:33Okay. Then if you talk about how many
1:02:35weights
1:02:37are here, how many weights are here? If
1:02:40you want to know
1:02:42here we have see for that here we have
1:02:452048 neurons
1:02:48and here in this case we have 512
1:02:51right. So again we have
1:02:552048 cross 512
1:02:59weights these many weights.
1:03:03Okay, here because it's a fully
1:03:05connected network.
1:03:07Let's say we can call this as W2. Okay,
1:03:11let's say this was W1 and this is W2.
1:03:13And then
1:03:16after this here every neuron has its own
1:03:19bias here also, right? So there'll be uh
1:03:22like 512 biases,
1:03:26right? Let's call it as B2.
1:03:30Fine.
1:03:33So this is the architecture even. Now
1:03:36the fun thing is that here we can only
1:03:40send 512 dimensional input but we have
1:03:44four vectors
1:03:46Z C1 NOM Z C2 NOM Z C3 and Z C4 NOM and
1:03:50how we got this even that also we have
1:03:54already discussed in the previous
1:03:56diagram right that's why I've told you
1:03:58just uh have that PDF with you everyone
1:04:00and then it will make
1:04:02you understand each and everything
1:04:03easily.
1:04:06Fine.
1:04:07So as you can see
1:04:11as I've told you
1:04:13this architecture uh if you talk about
1:04:17right
1:04:19we can only send 512 dimension input
1:04:23and in our case we have four vectors. So
1:04:25what we are going to do we are going to
1:04:28create a batch.
1:04:30We have stacked these four vectors
1:04:32together and created a matrix that is
1:04:34this one. And this matrix will be what?
1:04:37Four vectors each of size 512
1:04:40dimensions.
1:04:42Now we send this entire batch together.
1:04:44This entire batch together
1:04:50to this particular neural network. So
1:04:53first what will happen when input comes
1:04:56when the input comes then it will
1:04:59interact with these weights.
1:05:01So if you say uh if you say that you are
1:05:05collectively calling this as a zed okay
1:05:08zed 1 zed c1 nom zed c2 nom and so on.
1:05:11If you are correctively calling them as
1:05:13zed then which operation will be
1:05:15performed a dot product will be
1:05:17performed even and it will be like
1:05:21zed what is this w what are all these
1:05:24weights represented with let's say w1 so
1:05:26dot product will be like z dot w1
1:05:31okay z dot product w1 plus bias will be
1:05:36added plus b1
1:05:40so everyone something like this that Z
1:05:44dotproduct W1 plus bias B1.
1:05:49Okay. And if you want to check the shape
1:05:52then the operation is something like
1:05:53this even see it's like
1:05:56we have this 4 + 512
1:06:01right 4 + 512
1:06:06it is uh like as I've told you how we
1:06:08have stacked all these vectors together
1:06:10and we got a matrix
1:06:13right so the operation is something like
1:06:16this we have 4 + 512 to matrix
1:06:21with weights that is these weights and
1:06:25these weights are of
1:06:28512 these weights okay as I've told you
1:06:31512 cross 2048 so 512 cross 2048 so you
1:06:36see everyone 512 512 right it means both
1:06:41the dimensions match so the resultant
1:06:44will be what resultant of these two will
1:06:46be 4 cross 2048.
1:06:53Okay. So the resultant will be 4 + 208
1:06:562048 matrix in this layer.
1:07:01And in this we will add 2048 vector of
1:07:03biases as I've told you
1:07:07right. So again we will get what we will
1:07:09get 4 + 2048 matrix which basically
1:07:14means that we took our four vectors and
1:07:17increase their dimensionality.
1:07:20Still have four vectors but each
1:07:22vector's dimensionality has changed from
1:07:25512 to 248.
1:07:30Okay. Each vector's dimensionality is
1:07:32now 2048.
1:07:36After this what are you doing? You are
1:07:39applying ReLU. Okay, we are going to
1:07:42apply ReLU on this.
1:07:47By applying ReLU, what will happen is
1:07:48what will happen is that you will be
1:07:51able to introduce nonlinearity.
1:07:54Okay, you will be able to introduce
1:07:57nonlinearity. This also we have
1:07:58discussed even in the previous encoder
1:08:00architecture. Okay, the main purpose of
1:08:03this entire feed forward neural network
1:08:05is to introduce nonlinearity.
1:08:08Now you move forward to the next layer
1:08:11and in the next layer what are you going
1:08:13to do? You are going to multiply this
1:08:16entire quantity this entire quantity
1:08:19with W2.
1:08:22That means whatever the result which we
1:08:23got even here we'll multiply with W2.
1:08:27Okay. So currently this quantity has
1:08:29shape 4 + 2048.
1:08:32You will do a dot product with W2 which
1:08:34is 2048 + 512. So everyone see
1:08:39W2 is what? 2048
1:08:43cross 512. I already shown you right
1:08:462048 cross 512. We mentioned uh we
1:08:49actually have used W2 for it. Right?
1:08:51These weights
1:08:54right? So here also given both these
1:08:56dimensions match. So you will get back
1:09:01this particular
1:09:03matrix
1:09:05of 4 + 512. Why? Because you see four
1:09:09cross 512.
1:09:11Right? So in this you will add what? You
1:09:14will add 512 biases. Obviously first of
1:09:17all you will add a biases 512. But
1:09:18finally you will get a matrix which is
1:09:20of size 4 cross 512.
1:09:25Okay. So again you will get 4 + 512
1:09:27matrix. So which is again a better way
1:09:30of saying that we have four vectors and
1:09:33all their dimensionality is 512.
1:09:37So what happened is that we sent four
1:09:39vectors
1:09:41took out four vectors also 512 size was
1:09:46sent and 512 was also taken out. Okay.
1:09:50You see here
1:09:53fine in between what we did?
1:09:57We applied RLU rectified linear unit in
1:10:01it.
1:10:04Fine. And by applying RLU what benefit
1:10:07did we get
1:10:09that we were able to introduce
1:10:11nonlinearity.
1:10:14Okay. So the overall uh after this
1:10:18entire process
1:10:20we reached
1:10:23here everyone see I'll show you in our
1:10:25uh
1:10:27decoder diagram after doing this
1:10:29particular stuff even we are here
1:10:33okay
1:10:36after this entire process we reached
1:10:38here now
1:10:41we have four vectors all of shape 512
1:10:44Right? Y1, Y2, Y3, Y4. So we have four
1:10:48vectors and all of the shape 512
1:10:51dimension.
1:10:54But now these four vectors have the
1:10:56power to capture nonlinear information
1:10:58too. So I'm calling these as Y1, Y2, Y3,
1:11:01Y4.
1:11:03So now we have
1:11:06as as as we have reached at this
1:11:08particular point driven
1:11:11right
1:11:13feed forward has given us output and we
1:11:16are here
1:11:19and now again ahead of this is what add
1:11:23so again notice that what I have done
1:11:27in this particular diagram if you can
1:11:29Three.
1:11:32What I have done here is this Z C1 NOM,
1:11:35Z C2 NOM, Z C3 NOM, right? You see this
1:11:40one? This is residual connection,
1:11:44right?
1:11:45So what I have done is this Z C1 norm, Z
1:11:48C2, Z C3, Z C4 was not only sent to the
1:11:52feed forward neural network.
1:11:55Okay, it was not only sent to the feed
1:11:57forward neural network.
1:12:01But
1:12:03through residual connection
1:12:06right through residual connection it was
1:12:08sent here as well. You see here as well
1:12:11I think you understand right there's a
1:12:13residual connection.
1:12:16Okay there was a so this is this
1:12:18basically sent to the feed forward new
1:12:19letter plus through this residual
1:12:21connection here also it was sent for
1:12:23add.
1:12:28Okay. So Z C1 NOM and so on was not only
1:12:31sent to the feed forward neural network
1:12:33but also it was sent through a residual
1:12:35connection here
1:12:38and since all these are 512 vectors we
1:12:40can perform addition. These are all 512.
1:12:43These are also all 512 dimensional
1:12:45vectors. From adding them,
1:12:48let's say we get y1 dash, y2 dash, y3
1:12:52dash, y4 dash. After doing addition,
1:12:54which we will normalize again. And since
1:12:58many operations have been performed, we
1:12:59will perform layer normalization. And
1:13:01finally, we will get y1 normalized, y2
1:13:04normalized, y3 normalized, and y4
1:13:07normalized. Again, all four are still
1:13:10512 dimensional vectors.
1:13:18Fine. And again
1:13:22with that now if I want to show you we
1:13:25have reached here everyone.
1:13:29Okay. And these four vectors
1:13:32y1 nom, y2 nom, y3 nom, y4 nom.
1:13:37These are the output of decoder one.
1:13:40That is first decoder.
1:13:44This is the output of first decoder
1:13:46block. Y1 nom, y2 n, y3 n, y4. So we are
1:13:49here now.
1:13:52So I really hope through this entire
1:13:54journey you understood what is happening
1:13:55inside a single decoder block.
1:14:00Fine. Now, next what happens after this?
1:14:04Let's see that.
1:14:09Now, as you know,
1:14:11we do not have just one decoder. We have
1:14:15got six in total six decoder blocks. So
1:14:18what will happen is that as soon as you
1:14:21get your output from first decoder block
1:14:24and that output is y1 norm, y2 nom, y3
1:14:28nom and y4 norm right these four
1:14:34you will directly send it to the decoder
1:14:36block two
1:14:39and in decoder block two
1:14:44again let's say this decoder block so
1:14:47which is again similar to this. So in
1:14:49this decoder block two again the same
1:14:51operations will be executed exactly as
1:14:54they are in the first decoder. The only
1:14:56difference would be that parameters here
1:15:00will be different but the operations
1:15:01will be exactly same. Again you will
1:15:03apply
1:15:05masked multi head attention
1:15:09then normalization on top of it. add a
1:15:12normalization. Then you will send it to
1:15:14the cross attention block. Then again
1:15:17add a normalization and then feed
1:15:19forward neural network and then again
1:15:21add a normalization. And like this you
1:15:23will progress and reach the next block
1:15:27and again let's say in this third block
1:15:30you will get a set of vectors just like
1:15:33you got
1:15:35here. Y1 norm Y2 norm Y3 nom Y4 N. Okay,
1:15:39you'll again get all this. You will send
1:15:42that to the decoder three. From second
1:15:44you will send that to decoder three.
1:15:49Sorry. And now
1:15:53like this even step by step you will go
1:15:58all the way to the sixth decoder block
1:16:01and from the sixth decoder block you
1:16:04will get the final output here.
1:16:10So that is what I have shown in this
1:16:12particular diagram
1:16:14or in detail I can say that is what I
1:16:16have shown you in in this particular
1:16:18diagram like this one right or if you
1:16:22really want to understand those
1:16:23operations what happens in every decoder
1:16:26that is what I shown you here I think
1:16:27you understood that okay
1:16:33now [clears throat]
1:16:36as I've told you everyone
1:16:38there are uh six decoders right
1:16:44okay and as I've told you that
1:16:48six decoder will give you this final
1:16:51right this this this particular final
1:16:53output
1:16:55so right now you are in decoder one
1:16:58let's say okay you are in decoder one
1:17:01and you have got the output of decoder
1:17:04one this is ready decoder output is
1:17:07output uh decoder output is ready. So
1:17:09this this is decoders decoder once is
1:17:12output. Now after this in between as
1:17:15I've told you there are five decoder
1:17:18five more decoder blocks because total
1:17:20there are six decoder blocks. So there
1:17:22are five more decoder layers or the
1:17:24blocks and when you process all of them
1:17:26one by one you'll finally get your
1:17:29vector and I had denoted it as y f_sub_1
1:17:32nom y f_sub_2 nom y f3 nom y f4 nom
1:17:36where f stands for let's say final and
1:17:39this is now the final output which I
1:17:42have to process and print the output
1:17:46okay
1:17:48so now we will discuss the last part of
1:17:50this architecture And that is nothing
1:17:52but output block. That is nothing but
1:17:55this one. This output block is what we
1:17:58are going to discuss.
1:18:00So guys, now comes the most important
1:18:02part and the most interesting part of
1:18:04this entire architecture.
1:18:08Okay. Until now we have crossed the
1:18:13entire decoder six layer stack you can
1:18:16say and uh at this point we are sitting
1:18:20here if uh if I want to even show you
1:18:26original transform architecture as well.
1:18:27So you can imagine one decoder lock
1:18:29second third fourth fifth sixth and now
1:18:31we are here.
1:18:33Okay, now we are here.
1:18:40Fine. So,
1:18:43and and and we have our vectors which
1:18:45are final vectors.
1:18:48This one yf1 norm, y f2 nom, yf3 nom and
1:18:51y4 nom. Now there is one last work.
1:18:56I must print a Hindi word for every
1:18:59token
1:19:01or I must give output. So this see how
1:19:05this will happen. For this you must
1:19:07understand
1:19:09this particular output portion properly.
1:19:12You must understand this output portion
1:19:14properly.
1:19:17This output portion is made up of two
1:19:19things.
1:19:22Okay. As you can see a linear layer is
1:19:24there
1:19:26and a softmax layer is there. This is
1:19:29not something special everyone. you can
1:19:32very easily compare it with an output
1:19:34layer of a feed forward neural network.
1:19:37So here basically you have a single
1:19:39layer and to this layer you can give 512
1:19:43dimension input like this.
1:19:46Okay
1:19:48this I told you why f_sub_1 f_sub_2 and
1:19:50so on till f_sub_4.
1:19:52Now here in output layer what we have is
1:19:57you have a single layer and to this
1:19:59single layer you can give 512 dimension
1:20:02vector as a input. Okay 512 dimension
1:20:06input. Now this layer in this layer how
1:20:09many neurons do we have? If you talk
1:20:11about this layer how many neurons? See
1:20:13this is very interesting. In this layer
1:20:16we have
1:20:18V number of neurons. Please understand V
1:20:21is a predefined number which is decided
1:20:23by the fact
1:20:26that what is the vocabulary count of
1:20:29Hindi words in your entire data set
1:20:35see
1:20:38so we are doing English to Hindi
1:20:41translation I've told you right now look
1:20:44at the sentences I am good
1:20:47we are ID.
1:20:50Fine. So we are doing English to Hindi
1:20:53translation. So in one column we have
1:20:57all English sentences here
1:21:00and in another we have all Hindi
1:21:03sentences here.
1:21:05Right? These are two columns English and
1:21:07Hindi.
1:21:10Like here this is my data set. Let's say
1:21:13I am good. Corresponding to that we have
1:21:16Hindi sentence like my buddy
1:21:18corresponding to this this is the Hindi
1:21:19sentence
1:21:21like this let's say everyone we have
1:21:24let's say we have total 5,000
1:21:29sentence pairs of English to Hindi
1:21:32so what am I going to do I'm going to go
1:21:35into my entire Hindi data that is this
1:21:39one
1:21:42and start counting unique words like one
1:21:45unique word is me,
1:21:47one unique word is bada,
1:21:50then another unique word is who, one
1:21:53unique word is h, one unique word is ta
1:21:57and another unique word is he and so on
1:22:01and then let's say
1:22:06h appears in another sentence I will not
1:22:09count it again because I have already
1:22:11counted Hum
1:22:15right in the previous sentence. So in
1:22:16this way you make a vocabulary of all
1:22:18the unique words
1:22:21which are present in your Hindi data. So
1:22:24suppose everyone as I've told you there
1:22:26are 5,000 Hindi sentences and let's say
1:22:30there are
1:22:3210,000
1:22:35unique Hindi words. How many? 10,000
1:22:37unique Hindi words.
1:22:40Then this 10,000 is your V value. So
1:22:45whatever I was trying to tell you V
1:22:48value will be what? 10,000.
1:22:54Okay.
1:22:56So as I've told you, you make a
1:22:58vocabulary of all the unique words that
1:23:00are in your Hindi data.
1:23:03So we will have 10,000 neurons here. How
1:23:08many neurons are even 10,000 neurons?
1:23:13And what is interesting is that the
1:23:15first neurons represent the first word
1:23:17of vocabulary. Let's say which is in our
1:23:19case which is in our case this may.
1:23:24Okay. So it will be represented by this
1:23:27neuron.
1:23:29The second neuron represent second word
1:23:32that is nothing but let's say bada.
1:23:36Okay, that is this one.
1:23:39Fine.
1:23:41Similarly, this neuron represents who.
1:23:46This neuron represents H. This neuron
1:23:48represents ta. This neuron represents he
1:23:51and so on. And in this way, 10,000
1:23:55neurons will represent 10,000 unique
1:23:58Hindi words.
1:24:01Okay, please understand this.
1:24:04Next here we have weights you see here
1:24:07we have weights right these are weights
1:24:12how many weights
1:24:14here 512
1:24:17right which is your uh input size and v
1:24:21as I've told you is nothing but what
1:24:2410,000 so here everyone your weights
1:24:26will be 512 cross 10,000
1:24:32and every neuron has its own bias.
1:24:35So obviously that means there'll be
1:24:3810,000 biases.
1:24:43So at this point, so at this point I
1:24:46have four vectors 1 2 3 4
1:24:51one for each token.
1:24:53Right? So this first one corresponds to
1:24:57start.
1:25:00This second one corresponds to hum.
1:25:03This corresponds to T. This corresponds
1:25:05to H. Since here you can only give 512
1:25:11dimension input. What have we done? We
1:25:15have stacked these four vectors and made
1:25:18a 4 + 512 matrix.
1:25:23Okay. All these four vectors we stacked
1:25:26and we have made 4 + 512 matrix. Now you
1:25:30can say the batch of this layer in a
1:25:33batch form. So basically there will be
1:25:35parallel processing for all four tokens.
1:25:38But let me just show you for one token.
1:25:41Suppose we send the first vector to this
1:25:44layer. Then our input will be what? Our
1:25:46input will be 1 + 512 because I'm saying
1:25:50that I'll just uh show you for one
1:25:52token. Okay. Just to make the things
1:25:55simpler. So input will be what? 1 + 512
1:25:59one vector of dimension 512. So first
1:26:02what will happen? This y f_sub_1 norm
1:26:06this y f1 norm will multiply with these
1:26:10weights.
1:26:12Y f_sub_1 norm will multiply with these
1:26:14weights and how many weights 512 +
1:26:1810,000
1:26:20the operation will be performed which
1:26:21operation given let's say I'll represent
1:26:23this as w so the operation which will be
1:26:26performed will be like this
1:26:28w dot
1:26:31y f_sub_1 norm this operation will be
1:26:34performed
1:26:36so y f1 norm is y f_sub1 norm is of size
1:26:411 + 512 right 1 + 512
1:26:47and weights as I've told everyone
1:26:51weights
1:26:52this is 1 + 512
1:26:55and weights what we have is 512 + 10,000
1:27:00so 512 to 512 match so from this I will
1:27:04get what I will get
1:27:071 +
1:27:1010,000 so basically From every neuron
1:27:13one number will come out. From every
1:27:16neuron, this one, this one, this one,
1:27:18this one, this one, this one, this one.
1:27:20One number will come out.
1:27:23From every one of these 10,000 neurons,
1:27:27a number will emerge. So basically for
1:27:30every neuron, you will have unnormalized
1:27:32number. Whatever the numbers which you
1:27:33will get, those will be unnormalized.
1:27:40And as these are numbers are un are not
1:27:42normalized. So what I will do? I will
1:27:45apply softmax.
1:27:49Okay, I'll apply softmax here. So we
1:27:51will take all these numbers and run
1:27:53softmax on them.
1:27:57What does softmax do? It normalizes
1:27:59these numbers and normalizes them such
1:28:03that their sum becomes one. So basically
1:28:05for every word we will have a
1:28:08probability value for every word we will
1:28:12have probability value.
1:28:15So it is possible that let's say maybe
1:28:18uh I get 0.1 for the first word 0.02 for
1:28:25the second word 0.25
1:28:29for the third word and so on.
1:28:32Every word will have a corresponding
1:28:34probability value. Every word will have
1:28:37that. So let's say the probability value
1:28:40corresponding to hum is highest.
1:28:44Okay, probability value corresponding to
1:28:46H is highest.
1:28:48So what will we do? We will choose H
1:28:51word to be the output for start token.
1:28:56Okay, for start token output will be
1:28:58what? Hum.
1:29:03Fine.
1:29:08This means that corresponding to start
1:29:10token that is Y1 Y F1 NOM. Corresponding
1:29:14to start token means YF1 NOM.
1:29:19This network believes that the output
1:29:21should be H.
1:29:23Similarly, same thing for YF2 NOM, YF3
1:29:26NOM and so on. And let's say for YF2 nom
1:29:29corresponding word we got is tar for YF3
1:29:34norm here and for y f4 nom y f4 norm
1:29:39everyone let's say we got e o that is
1:29:43nothing but end of sentence
1:29:45and as soon as we get us a end of
1:29:48sentence it means stop the process
1:29:54okay us means what stop the process End
1:29:57of sentence like if you remember at the
1:29:59start we have used start start token.
1:30:02Similarly this is stop token E O
1:30:06end of sentence.
1:30:08Okay. And as soon as we get this it
1:30:12means stop the process. Stop the entire
1:30:15process. And this is how the decoder
1:30:18architecture works.
1:30:22Fine. [clears throat]
1:30:23So in the next video we will discuss
1:30:25about transformer decoder architecture
1:30:28during inference which means how decoder
1:30:30works during prediction.
1:30:34So I would I would recommend that you
1:30:36draw this entire architecture diagram on
1:30:38paper
1:30:40all these diagrams on paper everyone
1:30:45and try to understand it. It will be
1:30:48easier if you watched previous encoder
1:30:51architecture video. Okay. So, thank you
1:30:54everyone. That's it from my side. Thank
1:30:56you all of you.

About this transcript

This page contains the full transcript of Transformer Decoder Arch During Training Time - Pen & Paper Explaination | Transformer Series Part2 by yogesh murumkar, generated from the public captions YouTube serves with the video. The transcript has 11,315 words across 1,895 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.