Support Vector Regression Indepth Maths Intuition In Hindi — Transcript
Full transcript
- 0:00Hello guys, my name is Krishna Naik and
- 0:01welcome to my YouTube channel. So guys,
- 0:03in today's video, we are going to
- 0:05discuss the Support Vector Regression
- 0:07machine learning algorithm. In our
- 0:09previous video, we looked at Support
- 0:11Vector Classification. I have already
- 0:13provided the link to the entire
- 0:15playlist in the description of every
- 0:16video. And to understand today's
- 0:18algorithm, it is very important that
- 0:20you first understand Support Vector
- 0:22Classification, guys. Because Support
- 0:24Vector Regression is an extension of
- 0:26that. Okay? But I will try to revise it
- 0:28for you and explain the entire thing
- 0:31once again. But try to watch the full
- 0:33Support Vector Classification video.
- 0:36Again, the link will be in the
- 0:37description. So, let me share my screen
- 0:40now and let's quickly understand what
- 0:42we did in Support Vector Classifier.
- 0:45Look, in Support Vector Classifier, we
- 0:47created a division line in combination
- 0:50with a marginal plane and a hyperplane.
- 0:53Right? What will that do? We will get
- 0:56properly classified points, and the
- 0:58cost function we wrote was w/2 + c,
- 1:01what would that be? c will be a
- 1:04hyperparameter for how many
- 1:06misclassifications we can avoid. Okay?
- 1:09After that, the summation of i = 1 to n
- 1:11of eta. We call this eta. What is eta?
- 1:14The summation of all the distances
- 1:15between all the wrong points up to the
- 1:17marginal plane. Okay? This could be a
- 1:19marginal plane. Otherwise, if the wrong
- 1:22points are here, we can find the
- 1:23distance to this marginal plane. Or, if
- 1:26this yellow point is a mistake, we can
- 1:29calculate this distance. So, what will
- 1:31we basically call this? We will call
- 1:33this a hyperparameter, and the
- 1:35hyperparameter always controls how our
- 1:37line should be formed here. Okay? So, I
- 1:41hope you understood this, and I gave
- 1:42the full explanation of how this cost
- 1:44function is formed in the previous
- 1:46video, and there was one more thing:
- 1:47the constraint. What is the constraint?
- 1:51That w transpose*x + b should always be
- 1:54greater than or equal to one. This was
- 1:57our constraint as well. We have
- 1:59discussed this completely in our SVC
- 2:01video. Now, in the case of regression,
- 2:03in the case of Support Vector
- 2:05Regression, how does this work? Now,
- 2:07look guys, this is Support Vector
- 2:08Regression. Okay? Let's say I have a
- 2:11data set like this where we have
- 2:13populated points for house size and
- 2:16price. So, what do we do with support
- 2:18vectors in this? The first thing with
- 2:20support vectors is that we create a
- 2:22best-fit line, just like how a linear
- 2:23regression line is created. Okay? Now,
- 2:26the second thing that will be created
- 2:29here, we will specifically call that a
- 2:31marginal plane of this type. Okay? So,
- 2:34I will draw the marginal plane here;
- 2:36let's say, for now, the marginal plane
- 2:38is like this. So, here is my marginal
- 2:41plane. Okay? Marginal plane. This is
- 2:46one marginal plane. This is one
- 2:47marginal plane. Now, to specify this
- 2:50marginal plane, what extra parameters
- 2:53do we use? You know that we can also
- 2:57write this line as w transpose x + b =
- 3:000. And let's say our b is zero. The
- 3:03intercept is passing through zero. So,
- 3:04we can define it directly by saying w
- 3:07transpose x. This line that we will
- 3:11discuss, we can represent it using an
- 3:13epsilon value. Let's say this point of
- 3:17mine populated behind here. Okay? This
- 3:19is my main intercept point. Let's say
- 3:22for now I can say b = 0, or I can write
- 3:25it in a different way too. Let's say we
- 3:27represent this line as w transpose x,
- 3:30okay? Now, the next hyper-plane from
- 3:33here, this distance we will call
- 3:35epsilon, and this distance we will also
- 3:37call epsilon; epsilon is equal.
- 3:39Whenever marginal planes are formed in
- 3:41SVC or SVR, these marginal planes are
- 3:43always formed at an equal distance.
- 3:45Right? Okay? So, this distance of the
- 3:48marginal plane is what we will call
- 3:49epsilon. So, we can basically write the
- 3:52top line as w transpose x + epsilon and
- 3:56the bottom line as w transpose x-
- 3:58epsilon. Okay? We can definitely write
- 4:01it in this way. Now, remember, why am I
- 4:04writing this? Because later you will
- 4:05also calculate the distance between
- 4:07these two. Right? So, the cost function
- 4:09we will get, the cost function that
- 4:11will come, will be the same thing we
- 4:14did initially in SVC. That is minimized
- 4:17by changing w and b. w/2, this distance
- 4:22, we always have to minimize. When we
- 4:25minimize, that basically means we are
- 4:27having the maximum distance. Now,
- 4:29besides this, we will write a
- 4:31constraint because we wrote a different
- 4:33constraint in SVC. Why? Here we had to
- 4:36find the classification for all the
- 4:37correct points. For most of the
- 4:39accurate points. Accurate points means,
- 4:41let's understand how the prediction
- 4:42will happen here? Look, if you see
- 4:45according to this point, the truth
- 4:47value is here. But where will the
- 4:49predicted value be? Let's say the
- 4:51predicted value came here, here, or
- 4:53wherever? It came between the two.
- 4:55Let's say my predicted point is coming
- 4:57here. So, what should be its distance,
- 5:00and how will we calculate the distance?
- 5:03You can write it directly. Right? You
- 5:05can write y-w transpose x of i. Okay? w
- 5:09transpose x of i basically became my
- 5:11predicted point. It should be less than
- 5:14or equal to epsilon. Okay? So I will
- 5:17quickly explain to you again what I
- 5:19have done here, and you will get the
- 5:21whole idea from this. This is the
- 5:23entire support vector machine algorithm
- 5:25that I discussed. Right now, we are
- 5:27here. So here, the constraint will be
- 5:30written as y of i-w transpose x of i.
- 5:33This is basically my y-hat. Now the
- 5:35difference between these two should
- 5:37always remain below epsilon. It can be
- 5:39+ epsilon, it can be-epsilon. Always
- 5:41this distance, so here you can see this
- 5:43distance is always less. It is less
- 5:45than epsilon. So what does this mean?
- 5:47These points that we predicted are very
- 5:50good. It is very good. Okay? And
- 5:55usually, whenever we talk about
- 5:57regression, we discuss the mean squared
- 6:00error as the loss function. But here,
- 6:03look, we will specifically call this a
- 6:05constraint. Our constraint should
- 6:09always be y of i-w transpose x of i,
- 6:12which we call y-hat. It should be less
- 6:16than epsilon. Okay? So this became the
- 6:19constraint in the cost function for SVR
- 6:20because this is a frequent interview
- 6:22question, guys. Recently, this was also
- 6:24asked at Amazon. Okay? About all these
- 6:27things. Now, the next thing to
- 6:29understand is, okay, Krish, that is
- 6:31fine. You are saying that, okay? All
- 6:33your points should be within this +
- 6:36epsilon and-epsilon. But what about
- 6:38those points that are outside this? For
- 6:42those, how will you decide the cost
- 6:44function? Okay? How will you decide the
- 6:46cost function for these? So, I will
- 6:48write the cost function again here. Now
- 6:50, our cost function will be modified
- 6:52slightly. The points that exist outside
- 6:54. We will write it as minimize by
- 6:57changing W and b. /2 plus. Do you
- 7:02remember we used two parameters in
- 7:04classification? C and the summation of
- 7:08i = 1 to N, and here we will have eta
- 7:12of i, okay? We can call it mod of eta i
- 7:15. Okay? Because whenever we calculate
- 7:17distance. Now, what are this C and this
- 7:20eta of i? See, C means how many points
- 7:23outside the marginal plane we can
- 7:26adjust. If I say there are five points
- 7:29here. Okay? Then these five points can
- 7:31be outside. I will call this eta of i.
- 7:34The distance between the marginal plane
- 7:36and this particular point. Okay? Its
- 7:38summation should also be as small as
- 7:40possible. So in short, both of these
- 7:42are specifically hyperparameters. Okay?
- 7:45We call both of these hyperparameters.
- 7:48We call them hyperparameters. If we
- 7:50talk about SVC, we also call this hinge
- 7:53loss. Hinge loss. Okay? Now, but there
- 7:57is a very important thing that if I am
- 8:00writing this, its constraint will also
- 8:02change a bit. Because we haven't
- 8:04included this eta of i in the
- 8:05constraint, because it is the distance
- 8:07from the marginal plane to this outside
- 8:10point, the distance of this marginal
- 8:11plane to this particular marginal plane
- 8:14, sorry, these points to the marginal
- 8:16plane. So we will have to add one more
- 8:18constraint here. We will now modify
- 8:20this constraint slightly. What did we
- 8:23write earlier? y of i*w transpose x of
- 8:26i, and this should be less than or
- 8:28equal to epsilon. We had written this
- 8:31much earlier, but now when we have this
- 8:34value, eta, what will happen? + eta i
- 8:37will be added here. Okay, whatever eta
- 8:41we are getting here, + eta i will be
- 8:43added here. Now look, if I plot these
- 8:46points and tell you where epsilon and
- 8:48eta are in a very simple way, you will
- 8:51get the complete idea. The distance
- 8:54from here to here for these points, we
- 8:56will call it epsilon. Okay? But this
- 8:59point here is above the marginal plane.
- 9:01So what will we do? We will add one
- 9:04more parameter here with eta. We call
- 9:07it eta i. Because what will eti be? It
- 9:09is the distance from the marginal plane
- 9:11to this specific data point. So when we
- 9:14add both of these, always remember that
- 9:17our y-y of i w of x of i should be less
- 9:20than epsilon plus this. So it should
- 9:24remain less than this distance. So we
- 9:26will specifically call this a
- 9:28hyperparameter and we will try to play
- 9:30with different c-values. Okay? So let
- 9:33me draw it for you in a simpler way.
- 9:36Let's say my data points are something
- 9:38like this. Okay? Like this. So here I
- 9:43have created a best-fit line. Now with
- 9:45the best-fit line, I have created two
- 9:47marginal planes like this. Okay? Let's
- 9:49say. And these two marginal planes will
- 9:51basically be defined by the intercept
- 9:52from here. And we will use the same
- 9:54cost function. The distance of this
- 9:57point from here is epsilon. And let's
- 9:59assume there is an additional point
- 10:01above it. So, from this distance to
- 10:04that distance, we will call it—
- 10:06specifically—sorry, not from here to
- 10:09here—so we will call the distance
- 10:11from here to here 'eta of i'. Now, when
- 10:14we add both of these, whenever we make
- 10:16a prediction, it should always be less
- 10:18than this for any data point, because
- 10:20if I make a prediction for a data point
- 10:22from here, it will come here. Right? It
- 10:24will come here. And we always have to
- 10:26make sure it is always less than
- 10:28epsilon + eta. Okay? So this, this is
- 10:31the constraint. We play with different
- 10:34c-values. We can basically use this as
- 10:37a hyperparameter and we can select it
- 10:39accordingly. I will explain in
- 10:41GridSearch CV how support vector
- 10:43regression works. So I hope you all
- 10:45liked this video very much. I hope you
- 10:48understood what I am trying to explain.
- 10:50But to understand this properly, it is
- 10:51very important for you to understand
- 10:53the support vector classifier. I have
- 10:54put the link to that in the description
- 10:56. I hope you liked this video. I will
- 10:57see you all in the next video. Have a
- 10:59great day. Thank you, everyone. Bye-bye
- 11:00.
About this transcript
This page contains the full transcript of Support Vector Regression Indepth Maths Intuition In Hindi by Krish Naik Hindi, generated from the public captions YouTube serves with the video. The transcript has 1,894 words across 275 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.