What are Transformers (Machine Learning Model)? — Transcript
Full transcript
- 0:01no it's
- 0:02it it's not those transformers but but
- 0:05they can do some pretty cool things let
- 0:06me show you so
- 0:09why did the banana cross the road
- 0:12because it was sick of being mashed
- 0:15yeah i'm not sure that i quite get that
- 0:17one and that's because it was created by
- 0:20a computer i literally asked it to tell
- 0:24me a joke
- 0:25and this is what it came up with
- 0:27specifically i used a gpt-3
- 0:31or a generative pre-trained transformer
- 0:34model the three here means that this is
- 0:37the third generation
- 0:39gpt-3 is an auto-regressive language
- 0:41model that produces text that looks like
- 0:44it was written by a human
- 0:46gpt3 can write poetry craft emails and
- 0:51evidently come up with its own jokes
- 0:54off you go
- 0:55now
- 0:56while our banana joke isn't exactly
- 0:59funny it does fit the typical pattern of
- 1:01a joke with a setup and a punch line and
- 1:03sort of kind of makes sense i mean who
- 1:05wouldn't cross the road to avoid getting
- 1:06mashed but look gpt3 is just one example
- 1:10of
- 1:11a transformer
- 1:17something that transforms from one
- 1:20sequence into another and language
- 1:24translation is just a great example
- 1:26perhaps we want to take a sentence of
- 1:30why did
- 1:33the banana
- 1:37cross the road
- 1:41and we want to take that
- 1:44english phrase and translate it into
- 1:47french
- 1:48well transformers consist of two parts
- 1:51there is an encoder
- 1:55and there is a decoder
- 2:03the encoder works on the input
- 2:07sequence
- 2:08and the
- 2:10decoder operates on the target
- 2:13output sequence
- 2:16now on the face of it translation seems
- 2:18like little more than just like a basic
- 2:20lookup task so
- 2:22convert the y
- 2:24here of our english sentence to the
- 2:27french equivalent of porcua
- 2:30but of course
- 2:32language translation doesn't really work
- 2:35that way things like word order in terms
- 2:37of phrase often mix things up and the
- 2:40way transformers work is through
- 2:42sequence to sequence learning where the
- 2:45transformer takes a sequence of tokens
- 2:48in this case words in a sentence and
- 2:51predicts the next word in the output
- 2:53sequence
- 2:54it does this through iterating through
- 2:57encoder layers so the encoder generates
- 3:00encodings that define which part of the
- 3:03input sequence are relevant to each
- 3:05other and then passes these encodings to
- 3:08the next encoder layer the decoder takes
- 3:11all of these encodings and uses their
- 3:13derived context to generate the output
- 3:15sequence
- 3:17now transformers are a form of semi
- 3:21supervised learning
- 3:30by semi sequence semi-supervised we mean
- 3:33that they are pre-trained in an
- 3:35unsupervised manner with a large
- 3:38unlabeled data set and then they're
- 3:40fine-tuned through supervised training
- 3:43to get them to perform better now in
- 3:46previous videos i've talked about other
- 3:48machine learning algorithms that handle
- 3:50sequential input like natural language
- 3:52for example there are recurrent neural
- 3:55networks or rnns
- 3:57what makes transformers a little bit
- 4:00different is they do not necessarily
- 4:02process data in order
- 4:04transformers use something called an
- 4:06attention mechanism
- 4:12and this provides context around items
- 4:14in the input sequence so rather than
- 4:17starting our translation with the word
- 4:18why because it's at the start of the
- 4:20sentence the transformer attempts to
- 4:22identify the context that bring meaning
- 4:25in each word in the sequence and it's
- 4:28this attention mechanism that gives
- 4:30transformers a huge leg up over
- 4:32algorithms like rnn that must run in
- 4:34sequence
- 4:35transformers run multiple sequences
- 4:39in
- 4:40parallel
- 4:42and this vastly speeds up training times
- 4:46so beyond translations what can
- 4:48transformers be applied to well document
- 4:51summaries they're another great example
- 4:53you can like feed in a whole article as
- 4:56the input sequence and then generate an
- 4:59output sequence
- 5:00that's going to really just be a couple
- 5:02of sentences that summarize the main
- 5:05points
- 5:06transformers can create whole new
- 5:08documents of their own for example like
- 5:10write a whole blog post and beyond just
- 5:12language transformers have done things
- 5:15like learn to play chess and perform
- 5:17image processing that even rivals the
- 5:19capabilities of convolutional neural
- 5:21networks
- 5:22look transformers are a powerful deep
- 5:25learning model and thanks to how the
- 5:27attention mechanism can be paralyzed are
- 5:30getting better all the time and who
- 5:31knows pretty soon maybe they'll even be
- 5:34able to pull off banana jokes that
- 5:36are actually funny
- 5:40if you have any questions please drop us
- 5:41a line below and if you want to see more
- 5:44videos like this in the future please
- 5:46like and subscribe
- 5:47thanks for watching
About this transcript
This page contains the full transcript of What are Transformers (Machine Learning Model)? by IBM Technology, generated from the public captions YouTube serves with the video. The transcript has 733 words across 135 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.