YouTube2Text

What are Transformers (Machine Learning Model)? — Transcript

by IBM Technology · 733 words · 135 segments · language en · Watch on YouTube

Full transcript

  1. 0:01no it's
  2. 0:02it it's not those transformers but but
  3. 0:05they can do some pretty cool things let
  4. 0:06me show you so
  5. 0:09why did the banana cross the road
  6. 0:12because it was sick of being mashed
  7. 0:15yeah i'm not sure that i quite get that
  8. 0:17one and that's because it was created by
  9. 0:20a computer i literally asked it to tell
  10. 0:24me a joke
  11. 0:25and this is what it came up with
  12. 0:27specifically i used a gpt-3
  13. 0:31or a generative pre-trained transformer
  14. 0:34model the three here means that this is
  15. 0:37the third generation
  16. 0:39gpt-3 is an auto-regressive language
  17. 0:41model that produces text that looks like
  18. 0:44it was written by a human
  19. 0:46gpt3 can write poetry craft emails and
  20. 0:51evidently come up with its own jokes
  21. 0:54off you go
  22. 0:55now
  23. 0:56while our banana joke isn't exactly
  24. 0:59funny it does fit the typical pattern of
  25. 1:01a joke with a setup and a punch line and
  26. 1:03sort of kind of makes sense i mean who
  27. 1:05wouldn't cross the road to avoid getting
  28. 1:06mashed but look gpt3 is just one example
  29. 1:10of
  30. 1:11a transformer
  31. 1:17something that transforms from one
  32. 1:20sequence into another and language
  33. 1:24translation is just a great example
  34. 1:26perhaps we want to take a sentence of
  35. 1:30why did
  36. 1:33the banana
  37. 1:37cross the road
  38. 1:41and we want to take that
  39. 1:44english phrase and translate it into
  40. 1:47french
  41. 1:48well transformers consist of two parts
  42. 1:51there is an encoder
  43. 1:55and there is a decoder
  44. 2:03the encoder works on the input
  45. 2:07sequence
  46. 2:08and the
  47. 2:10decoder operates on the target
  48. 2:13output sequence
  49. 2:16now on the face of it translation seems
  50. 2:18like little more than just like a basic
  51. 2:20lookup task so
  52. 2:22convert the y
  53. 2:24here of our english sentence to the
  54. 2:27french equivalent of porcua
  55. 2:30but of course
  56. 2:32language translation doesn't really work
  57. 2:35that way things like word order in terms
  58. 2:37of phrase often mix things up and the
  59. 2:40way transformers work is through
  60. 2:42sequence to sequence learning where the
  61. 2:45transformer takes a sequence of tokens
  62. 2:48in this case words in a sentence and
  63. 2:51predicts the next word in the output
  64. 2:53sequence
  65. 2:54it does this through iterating through
  66. 2:57encoder layers so the encoder generates
  67. 3:00encodings that define which part of the
  68. 3:03input sequence are relevant to each
  69. 3:05other and then passes these encodings to
  70. 3:08the next encoder layer the decoder takes
  71. 3:11all of these encodings and uses their
  72. 3:13derived context to generate the output
  73. 3:15sequence
  74. 3:17now transformers are a form of semi
  75. 3:21supervised learning
  76. 3:30by semi sequence semi-supervised we mean
  77. 3:33that they are pre-trained in an
  78. 3:35unsupervised manner with a large
  79. 3:38unlabeled data set and then they're
  80. 3:40fine-tuned through supervised training
  81. 3:43to get them to perform better now in
  82. 3:46previous videos i've talked about other
  83. 3:48machine learning algorithms that handle
  84. 3:50sequential input like natural language
  85. 3:52for example there are recurrent neural
  86. 3:55networks or rnns
  87. 3:57what makes transformers a little bit
  88. 4:00different is they do not necessarily
  89. 4:02process data in order
  90. 4:04transformers use something called an
  91. 4:06attention mechanism
  92. 4:12and this provides context around items
  93. 4:14in the input sequence so rather than
  94. 4:17starting our translation with the word
  95. 4:18why because it's at the start of the
  96. 4:20sentence the transformer attempts to
  97. 4:22identify the context that bring meaning
  98. 4:25in each word in the sequence and it's
  99. 4:28this attention mechanism that gives
  100. 4:30transformers a huge leg up over
  101. 4:32algorithms like rnn that must run in
  102. 4:34sequence
  103. 4:35transformers run multiple sequences
  104. 4:39in
  105. 4:40parallel
  106. 4:42and this vastly speeds up training times
  107. 4:46so beyond translations what can
  108. 4:48transformers be applied to well document
  109. 4:51summaries they're another great example
  110. 4:53you can like feed in a whole article as
  111. 4:56the input sequence and then generate an
  112. 4:59output sequence
  113. 5:00that's going to really just be a couple
  114. 5:02of sentences that summarize the main
  115. 5:05points
  116. 5:06transformers can create whole new
  117. 5:08documents of their own for example like
  118. 5:10write a whole blog post and beyond just
  119. 5:12language transformers have done things
  120. 5:15like learn to play chess and perform
  121. 5:17image processing that even rivals the
  122. 5:19capabilities of convolutional neural
  123. 5:21networks
  124. 5:22look transformers are a powerful deep
  125. 5:25learning model and thanks to how the
  126. 5:27attention mechanism can be paralyzed are
  127. 5:30getting better all the time and who
  128. 5:31knows pretty soon maybe they'll even be
  129. 5:34able to pull off banana jokes that
  130. 5:36are actually funny
  131. 5:40if you have any questions please drop us
  132. 5:41a line below and if you want to see more
  133. 5:44videos like this in the future please
  134. 5:46like and subscribe
  135. 5:47thanks for watching

About this transcript

This page contains the full transcript of What are Transformers (Machine Learning Model)? by IBM Technology, generated from the public captions YouTube serves with the video. The transcript has 733 words across 135 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.