Support Vector Machines | Geometric Intuition — Transcript
Full transcript
- 0:00Hi guys, the next algorithm that we are
- 0:02going to do is Support Vector Machines.
- 0:04To be honest, let me tell you a little
- 0:06bit about Support Vector Machines first
- 0:08. This is one algorithm that actually
- 0:11gets a lot of respect in the machine
- 0:14learning community because this is a
- 0:16very, very good algorithm. It is very
- 0:20robust and it is kind of applicable to
- 0:22a lot of scenarios. Okay? So it was
- 0:25conceptualized in the late 90s, and
- 0:27since then, along with Random Forest,
- 0:29this algorithm is actually used in
- 0:31quite a few scenarios. Okay, so you can
- 0:34keep this in the logistic regression
- 0:36family, and in fact, it is said that if
- 0:38you are going to do Support Vector
- 0:39Machines, you should have a good idea
- 0:41about logistic regression because what
- 0:42do Support Vector Machines do? They
- 0:45improve the idea of logistic regression
- 0:46a little bit. Right? So what we are
- 0:49going to do is understand Support
- 0:50Vector Machines from scratch. Okay? So
- 0:53there will be multiple videos. This is
- 0:55just the first video where we will
- 0:56discuss the geometric intuition behind
- 0:58the algorithm. Okay? So I have drawn
- 1:01this figure. There are two different
- 1:04classes of points, the green points and
- 1:06the red points, and these are linearly
- 1:08separable, as you can see you can draw
- 1:10a line to classify. Okay? So I have
- 1:13drawn two planes here. You can call
- 1:16them planes, or if it's 2D, you can
- 1:17call them line segments. So I have
- 1:19drawn two planes here. π1, π2. You
- 1:21can call them L1 and L2. Now, if I ask
- 1:23you this question, both of these are
- 1:26classifying the data with 100%accuracy.
- 1:29If you think about it. If you look at
- 1:30the black one, it is able to classify
- 1:32all the red points correctly and all
- 1:34the green points correctly. And the
- 1:35same goes for the blue hyperplane. Okay
- 1:37? But what if I ask you this question:
- 1:40which of these would you select as your
- 1:42classifier? Because if you think about
- 1:44it, if you were doing logistic
- 1:46regression, for logistic regression,
- 1:47both are equal because both are doing
- 1:49their job. But not for SVM; SVM says I
- 1:51will select the better line, the better
- 1:53hyperplane out of these two. Can you
- 1:56tell me which of these will act as a
- 1:58better hyperplane? The answer is this
- 2:01black one, and I guess you all can
- 2:03think of it yourselves, right? You must
- 2:05also be doing the calculations in your
- 2:06head and arriving at the answer that Pi
- 2:081 is better than Pi 2. Now comes the
- 2:10answer, now comes the answer as to why
- 2:12this is the case. The reason is that if
- 2:15you have studied logistic regression,
- 2:17you would know that whenever you draw a
- 2:19line segment, what you do is calculate
- 2:21this distance, which is basically W
- 2:23transpose XI, where XI is this point,
- 2:25right? And here, we are ignoring the B
- 2:27term by imagining that it passes
- 2:29through the origin. And then what do
- 2:31you do? You pass it through the sigmoid
- 2:33function. And this is your probability.
- 2:34The probability of y being one. Right?
- 2:37And y being -1. Right? Now, the smaller
- 2:40this distance is, the closer this
- 2:41probability will be to zero. Meaning,
- 2:44if it is very close to this line, it
- 2:46means the probability of it being
- 2:48positive is also very low. But if a
- 2:50point is very far away, right? It is
- 2:53very far, so this distance will be
- 2:55large and the probability that it could
- 2:57be one will also be very high. Right?
- 3:00So what is SVM saying? Or what does SVM
- 3:02want to do? It wants to classify the
- 3:05data. But by selecting that hyper-plane
- 3:09which classifies all these points as
- 3:12widely as possible. Right? You want to
- 3:15increase this distance. You want to
- 3:16increase this gap. So that when you get
- 3:19new unseen data in the future, this
- 3:21line will perform better in comparison
- 3:23to this line. And that is the core idea
- 3:26behind SVM. The core idea is—I’ll
- 3:28sort of revise. The core idea is to
- 3:31maximize this distance, this gap,
- 3:33because of which you are able to
- 3:35classify these points as widely as
- 3:38possible. Okay? And that is why this
- 3:40plane will not be selected; this plane
- 3:42will be selected if you use support
- 3:43vector machines. Okay? Now what we will
- 3:46do is, we will actually see
- 3:48geometrically why and how you will
- 3:50select this plane and not select this
- 3:52one. What is the core logic behind this
- 3:53? That is what we are going to discuss.
- 3:55Okay? So I will rub the board. Okay? So
- 3:58now that we know the core idea. What do
- 4:00we need to do? We need to find a
- 4:03hyper-plane that maximizes this margin.
- 4:07So that the model you create is very
- 4:10generalized. Meaning, it behaves very
- 4:12well on unseen data too. Okay? So this
- 4:15gap, you can also call it a margin.
- 4:18Right? You can call this a margin. And
- 4:20we are trying to find a plane that will
- 4:22increase this margin. Right? And that
- 4:26is why we sometimes call it the margin
- 4:31maximizing hyperplane. Okay? So in SVM,
- 4:37the core idea, the thing you are
- 4:40working towards, is actually to find a
- 4:42hyperplane that can act as the
- 4:45margin-maximizing hyperplane. Meaning,
- 4:47when it classifies the data, it should
- 4:49maximize this margin. So that your
- 4:52model becomes a very generalized model
- 4:54and performs well on unseen data too.
- 4:57Right? But the question is, how? How
- 5:00will you do this? Right? So now I will
- 5:02get into the mathematical formulation.
- 5:04I will show you through geometry how
- 5:06this happens. So the whole idea is very
- 5:08simple. What you do is you go parallel
- 5:11to this. First, you select a hyperplane
- 5:13. Let's say you selected a hyperplane.
- 5:15Okay? Now let's say you randomly
- 5:16selected a hyperplane. This is a
- 5:18hyperplane. Now what you do is you go
- 5:20in the positive direction parallel to
- 5:22this hyperplane. What do you do? You
- 5:25keep drawing hyperplanes in the
- 5:26parallel direction above it until you
- 5:28reach the first green point. As soon as
- 5:31you find a hyperplane that is parallel
- 5:34to pi and it touches the first green
- 5:36point, you stop there. You stop right
- 5:39there. Right? This line, this
- 5:42hyperplane, is parallel to pi. And this
- 5:45is the first hyperplane that touches
- 5:46the first green point, and this is a
- 5:48red flag. You will stop here. Okay? And
- 5:51you call this hyperplane pi positive.
- 5:54Right? Then you do the same thing in
- 5:56the opposite direction. You move in the
- 5:59negative direction parallel to pi and
- 6:01you will stop once you reach the first
- 6:03red point. Okay? So you stop here.
- 6:08Right? And you call this hyper...
- 6:10Obviously, both of these are
- 6:11hyperplanes. Right? These are like line
- 6:13segments. If you are talking about 2D,
- 6:15these are both lines. Actually, all
- 6:16three are lines. Okay? And you call
- 6:18this hyperplane pi minus. Okay? So you
- 6:21are finding...first you randomly
- 6:23selected this hyperplane. Then for this
- 6:25hyperplane, you found this pi positive
- 6:27and this pi negative. Right? Now what
- 6:30do you do? Now you calculate the margin
- 6:32. What is the margin? Margin is defined
- 6:34as the shortest distance between pi
- 6:36positive and pi minus. Both are
- 6:37parallel. Right? If pi plus is parallel
- 6:39to pi, and pi minus is parallel to pi,
- 6:41then pi minus and pi plus are parallel.
- 6:43So what you do is you calculate the
- 6:44shortest distance. You calculate this
- 6:47distance, right? And you call this
- 6:51distance margin and you denote it with
- 6:55'd', right? So first step, select a
- 6:59hyperplane. Step two, find out pi
- 7:01positive and pi negative hyperplanes,
- 7:03then calculate this distance. And then
- 7:06you find out, then you do this step
- 7:07with a bunch of other hyperplanes. You
- 7:09will calculate for all hyperplanes. For
- 7:11them, you will find pi plus and pi
- 7:13minus and calculate 'd' for all of them
- 7:15. And then you find out, then you
- 7:17decide, uh, you will select that
- 7:19particular plane for which the value of
- 7:22'd' is higher. That is what this
- 7:24algorithm is giving. You are selecting
- 7:26a hyperplane, you are finding positive,
- 7:28negative, you are finding this 'd', and
- 7:30then you are selecting that hyperplane
- 7:32for which this 'd' is maximum. So
- 7:34mathematically if you put this, then
- 7:36you want to find out, you are actually
- 7:38trying to maximize this 'd', uh, by
- 7:40finding out obviously the equation of
- 7:42the plane, you know, w transpose x + b,
- 7:44right? So what you are actually trying
- 7:46to do is, you are actually trying to
- 7:48find out w and b values for which this
- 7:50'd' will be maximum. This is what we
- 7:52are trying to do and it actually works
- 7:55very well. Uh, you can actually test it
- 7:58. Uh, let's say you created another
- 8:00line segment. Uh This is the one. Okay?
- 8:06Now for this, if you, let's call this '
- 8:09d prime'. Now for this, if you, uh,
- 8:12find out the positive and negative
- 8:13planes. Then this is the first point
- 8:16where it touches. Right? I will draw it
- 8:19. Right? And similarly this point is
- 8:23over here, right? And you can see this
- 8:26gap, this margin, let's call it 'd
- 8:28prime'. 'd prime' is less than 'd', and
- 8:32if you are asked to choose between
- 8:34these two, then obviously Support
- 8:37Vector Machine will choose 'pi' over '
- 8:39pi prime' because of the logic. Okay? I
- 8:43guess you understood the whole thing.
- 8:45Right? So how to do this? We will look
- 8:48at that. We are just taking a geometric
- 8:50intuition. Now I would like to discuss
- 8:52one important concept that will be used
- 8:54a lot later. The concept of support
- 8:56vectors. Support vectors. This is a
- 9:00very important concept. So important
- 9:02that the name of this algorithm itself
- 9:04is Support Vector Machines. Okay? So I
- 9:07will completely, I will draw a
- 9:08completely new diagram just to explain
- 9:11what are support vectors. So let's say
- 9:14we have these two classes of points
- 9:16again. Right? So you draw a hyperplane
- 9:20and then you create the negative as
- 9:23well as positive hyperplanes. Right?
- 9:28Now, do you know when you have to stop?
- 9:30Like when you find the first positive
- 9:31and negative points. Right? Meaning
- 9:34this point and these two points. Right?
- 9:38Now, these three points are what you
- 9:40call support vectors. So, this logic of
- 9:43yours, these planes only stop when they
- 9:46find the support vectors. Okay? And
- 9:48these are what we call support vectors,
- 9:50and they are important in driving this
- 9:51whole thing, and we will see that in
- 9:53future videos. I hope support vectors
- 9:55are clear. Two points that I would like
- 9:58to discuss: first, about support vector
- 10:00machines. Obviously, support vector
- 10:04machines are very robust to outliers.
- 10:10You will see this in a later video,
- 10:11unlike other machine learning
- 10:13algorithms that are heavily impacted by
- 10:15outliers. SVMs are actually solid when
- 10:18you talk about outliers; they are able
- 10:20to handle outliers quite well. Okay? So
- 10:24that is the first benefit of using SVMs
- 10:26. The second benefit is that it can
- 10:28also work on, or rather it can also
- 10:31work in the case of non-linear data.
- 10:34Like this is a very clear-cut case. You
- 10:36have linear data. You are able to
- 10:38classify it very easily. But what if
- 10:40you had data like this? Something like
- 10:44this. Obviously, you could not classify
- 10:47this with a single line. Right? So in
- 10:50such cases, you would see there is a
- 10:53thing in SVM called kernels. Using
- 10:56these kernels, you can actually solve
- 10:57this kind of problem. Okay? And the
- 11:00third point is that SVMs can be
- 11:02implemented for both classification as
- 11:05well as regression problems. Okay? So,
- 11:08I guess you understood this entire
- 11:10introduction. From the next video
- 11:13onwards, we will actually try to
- 11:15mathematically formulate how to
- 11:17calculate these distances. How these
- 11:19planes will be derived. And then, what
- 11:22will be the equation that you need to
- 11:25optimize to find the value of the w and
- 11:28b equation. Okay? So, I know it
- 11:31shouldn't feel too difficult, I guess,
- 11:33because it's not that difficult; at
- 11:34least the core idea is not that
- 11:36difficult. Yes, when you go to
- 11:37implement it in math, it feels a bit
- 11:39difficult. But for now, I think this
- 11:41entire thing is clear. So, that's it
- 11:43for the video, guys. Thanks for
- 11:45watching.
About this transcript
This page contains the full transcript of Support Vector Machines | Geometric Intuition by CampusX, generated from the public captions YouTube serves with the video. The transcript has 2,070 words across 299 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.