Kernel Trick in SVM | Geometric Intuition — Transcript
Full transcript
- 0:00Hi guys. So, until now, we've studied a
- 0:03lot about SVM, and now we are going to
- 0:05discuss a very important thing in SVM,
- 0:08which is known as the kernel trick.
- 0:10Okay? So, I mentioned in the first
- 0:12video that one of the most important
- 0:15features, or you could call it the core
- 0:17strength of SVM, is that it can work on
- 0:19non-linear data, and that's exactly
- 0:21what we're going to see in this video.
- 0:25You use a thing called the kernel trick
- 0:27to classify data when it's non-linear.
- 0:31Okay? So, the whole idea is kind of
- 0:34difficult and mathematically very
- 0:37complex. So, what I have planned is
- 0:40that in this video, I will just give
- 0:41you the intuition about this, and in
- 0:43the next video, I will show you a
- 0:45practical example. We will run this
- 0:47entire thing on code, and after that,
- 0:49we will discuss the math behind it.
- 0:51Okay? So, that's the plan. So, let's
- 0:53start. So, let's say we have this data.
- 0:56Right? There are two classes, but you
- 0:59can see obviously the data is in 1D. So
- 1:01, it's a one-dimensional data; there is
- 1:04only one axis, and these are the two
- 1:06classes: circles and crosses. Okay? And
- 1:10I want to, you know, draw a line that
- 1:12can actually separate these two classes
- 1:14. And you can see that you cannot draw
- 1:18a single line that can separate these
- 1:19two classes. Right? So, this is clearly
- 1:23an example of non-linear data. And in
- 1:27such a case, you use something called
- 1:29the kernel trick. All right? So, what
- 1:32is the kernel trick? The kernel trick
- 1:34is basically this trick. There, what
- 1:37you do is...You take a mathematical
- 1:40function and convert your
- 1:42lower-dimension feature space into a
- 1:44higher-dimension feature space. Meaning
- 1:48, if you want, you can convert 1D data
- 1:51into 2D data in such a way that it
- 1:53becomes linearly separable. Okay? For
- 1:56example, one would be: you will take a
- 1:59function, let's say x or x². Let's say
- 2:02, okay? And what you do is you
- 2:04calculate the squares. And obviously,
- 2:09it will be...I will use something like
- 2:15this, and and And not able to draw it.
- 2:24I will draw it again. So, a little more
- 2:28gradual, and then, and finally,
- 2:31something like this. Okay? Something
- 2:35like this. I am not able to draw it
- 2:36exactly, but something like this. Okay?
- 2:38So, as soon as you calculated the
- 2:40squares, it turned out like this. And
- 2:41now what you can do is you can draw a
- 2:44plane like this that can separate the
- 2:47two classes. Okay? You squared or cubed
- 2:50it to calculate it. And now in this
- 2:54higher dimensional plane, in 2D, what
- 2:56you are doing is creating a line
- 2:58segment, which, by the way, in 1D, you
- 3:00have to find a point. Right? In 2D, you
- 3:03have to find a line segment. Okay? So
- 3:05what are your steps here? You find a
- 3:09function that will transform this
- 3:11lower-dimensional feature space into a
- 3:13higher-dimensional space. And this
- 3:16function is what we call a kernel. We
- 3:19call it a kernel. Right? There are many
- 3:22kernels. Like the one we will study, a
- 3:24very important one is RBF, Radial Basis
- 3:26Function. Then there is something
- 3:29called polynomial. The example we just
- 3:31saw was of a polynomial. And then there
- 3:34is sigmoid, which was in logistic
- 3:37regression. Right? And this whole
- 3:41transformation that happened, we call
- 3:45it kernel transformation. Kernel
- 3:48transformation. Okay? Now what I will
- 3:52do is show you one more example. This
- 3:54time, this is data in 2D, and you can
- 3:56see the data looks something like this.
- 3:59There are again two classes, green and
- 4:01red. And they are like concentric
- 4:04circles. So obviously, you cannot
- 4:06separate them by drawing any single
- 4:09line. It won't happen. Right? So again,
- 4:12what are you doing? Here you are
- 4:13applying this function, let's say a
- 4:15function like e raised to power-x².
- 4:19This function looks like this. This
- 4:22function looks like this. It lifts
- 4:25things up from the center. So what you
- 4:27can do is center this entire data. And
- 4:30what you can do is apply this
- 4:31transformation on top of this data. And
- 4:33what will this do? It will bring the
- 4:35center points a bit higher. And your
- 4:37entire thing will look like this in 3D.
- 4:39I will show you. Your green points will
- 4:46look something like this on top, and
- 4:49your red points will come down a bit.
- 4:55Something like this. And now what you
- 4:58can do is, you can draw a plane. Which
- 5:02will separate this data, okay. So again
- 5:05the same idea, this time this is the
- 5:07kernel function and this is the kernel
- 5:09transformation, and this entire thing
- 5:11is known as the kernel trick. Right?
- 5:13And this comes built-in within your SVM
- 5:16algorithm. There is a hyperparameter
- 5:19which you can tune using grid search.
- 5:24You can find its value. Right? So I
- 5:28will just summarize: if your data is
- 5:31not linearly separable in lower
- 5:32dimensions, what you do is you use some
- 5:35sort of kernel function to do a kernel
- 5:37transformation, which will make the
- 5:39data linearly separable in a
- 5:41high-dimensional feature space—that's
- 5:44the whole idea behind the kernel trick.
- 5:47All right? And the mathematics behind
- 5:49this is a bit complex. So what we will
- 5:52do is we will deal with these two
- 5:53examples, RBF. By the way, this one was
- 5:55the example of RBF. We also call this
- 5:57Radial Basis Function. So we will
- 5:59discuss these two in detail. And there
- 6:01you will understand how the math behind
- 6:03this works. But in the next video, what
- 6:06we will do is we will see a code
- 6:08example where we will perform a kernel
- 6:11trick on top of this kind of data. Okay
- 6:14? So I hope you understood the
- 6:15intuition of this video. Thanks for
- 6:17watching.
About this transcript
This page contains the full transcript of Kernel Trick in SVM | Geometric Intuition by CampusX, generated from the public captions YouTube serves with the video. The transcript has 971 words across 140 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.