YouTube2Text

Support Vector Machines | Geometric Intuition — Transcript

by CampusX · 2,070 words · 299 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Hi guys, the next algorithm that we are
  2. 0:02going to do is Support Vector Machines.
  3. 0:04To be honest, let me tell you a little
  4. 0:06bit about Support Vector Machines first
  5. 0:08. This is one algorithm that actually
  6. 0:11gets a lot of respect in the machine
  7. 0:14learning community because this is a
  8. 0:16very, very good algorithm. It is very
  9. 0:20robust and it is kind of applicable to
  10. 0:22a lot of scenarios. Okay? So it was
  11. 0:25conceptualized in the late 90s, and
  12. 0:27since then, along with Random Forest,
  13. 0:29this algorithm is actually used in
  14. 0:31quite a few scenarios. Okay, so you can
  15. 0:34keep this in the logistic regression
  16. 0:36family, and in fact, it is said that if
  17. 0:38you are going to do Support Vector
  18. 0:39Machines, you should have a good idea
  19. 0:41about logistic regression because what
  20. 0:42do Support Vector Machines do? They
  21. 0:45improve the idea of logistic regression
  22. 0:46a little bit. Right? So what we are
  23. 0:49going to do is understand Support
  24. 0:50Vector Machines from scratch. Okay? So
  25. 0:53there will be multiple videos. This is
  26. 0:55just the first video where we will
  27. 0:56discuss the geometric intuition behind
  28. 0:58the algorithm. Okay? So I have drawn
  29. 1:01this figure. There are two different
  30. 1:04classes of points, the green points and
  31. 1:06the red points, and these are linearly
  32. 1:08separable, as you can see you can draw
  33. 1:10a line to classify. Okay? So I have
  34. 1:13drawn two planes here. You can call
  35. 1:16them planes, or if it's 2D, you can
  36. 1:17call them line segments. So I have
  37. 1:19drawn two planes here. π1, π2. You
  38. 1:21can call them L1 and L2. Now, if I ask
  39. 1:23you this question, both of these are
  40. 1:26classifying the data with 100%accuracy.
  41. 1:29If you think about it. If you look at
  42. 1:30the black one, it is able to classify
  43. 1:32all the red points correctly and all
  44. 1:34the green points correctly. And the
  45. 1:35same goes for the blue hyperplane. Okay
  46. 1:37? But what if I ask you this question:
  47. 1:40which of these would you select as your
  48. 1:42classifier? Because if you think about
  49. 1:44it, if you were doing logistic
  50. 1:46regression, for logistic regression,
  51. 1:47both are equal because both are doing
  52. 1:49their job. But not for SVM; SVM says I
  53. 1:51will select the better line, the better
  54. 1:53hyperplane out of these two. Can you
  55. 1:56tell me which of these will act as a
  56. 1:58better hyperplane? The answer is this
  57. 2:01black one, and I guess you all can
  58. 2:03think of it yourselves, right? You must
  59. 2:05also be doing the calculations in your
  60. 2:06head and arriving at the answer that Pi
  61. 2:081 is better than Pi 2. Now comes the
  62. 2:10answer, now comes the answer as to why
  63. 2:12this is the case. The reason is that if
  64. 2:15you have studied logistic regression,
  65. 2:17you would know that whenever you draw a
  66. 2:19line segment, what you do is calculate
  67. 2:21this distance, which is basically W
  68. 2:23transpose XI, where XI is this point,
  69. 2:25right? And here, we are ignoring the B
  70. 2:27term by imagining that it passes
  71. 2:29through the origin. And then what do
  72. 2:31you do? You pass it through the sigmoid
  73. 2:33function. And this is your probability.
  74. 2:34The probability of y being one. Right?
  75. 2:37And y being -1. Right? Now, the smaller
  76. 2:40this distance is, the closer this
  77. 2:41probability will be to zero. Meaning,
  78. 2:44if it is very close to this line, it
  79. 2:46means the probability of it being
  80. 2:48positive is also very low. But if a
  81. 2:50point is very far away, right? It is
  82. 2:53very far, so this distance will be
  83. 2:55large and the probability that it could
  84. 2:57be one will also be very high. Right?
  85. 3:00So what is SVM saying? Or what does SVM
  86. 3:02want to do? It wants to classify the
  87. 3:05data. But by selecting that hyper-plane
  88. 3:09which classifies all these points as
  89. 3:12widely as possible. Right? You want to
  90. 3:15increase this distance. You want to
  91. 3:16increase this gap. So that when you get
  92. 3:19new unseen data in the future, this
  93. 3:21line will perform better in comparison
  94. 3:23to this line. And that is the core idea
  95. 3:26behind SVM. The core idea is—I’ll
  96. 3:28sort of revise. The core idea is to
  97. 3:31maximize this distance, this gap,
  98. 3:33because of which you are able to
  99. 3:35classify these points as widely as
  100. 3:38possible. Okay? And that is why this
  101. 3:40plane will not be selected; this plane
  102. 3:42will be selected if you use support
  103. 3:43vector machines. Okay? Now what we will
  104. 3:46do is, we will actually see
  105. 3:48geometrically why and how you will
  106. 3:50select this plane and not select this
  107. 3:52one. What is the core logic behind this
  108. 3:53? That is what we are going to discuss.
  109. 3:55Okay? So I will rub the board. Okay? So
  110. 3:58now that we know the core idea. What do
  111. 4:00we need to do? We need to find a
  112. 4:03hyper-plane that maximizes this margin.
  113. 4:07So that the model you create is very
  114. 4:10generalized. Meaning, it behaves very
  115. 4:12well on unseen data too. Okay? So this
  116. 4:15gap, you can also call it a margin.
  117. 4:18Right? You can call this a margin. And
  118. 4:20we are trying to find a plane that will
  119. 4:22increase this margin. Right? And that
  120. 4:26is why we sometimes call it the margin
  121. 4:31maximizing hyperplane. Okay? So in SVM,
  122. 4:37the core idea, the thing you are
  123. 4:40working towards, is actually to find a
  124. 4:42hyperplane that can act as the
  125. 4:45margin-maximizing hyperplane. Meaning,
  126. 4:47when it classifies the data, it should
  127. 4:49maximize this margin. So that your
  128. 4:52model becomes a very generalized model
  129. 4:54and performs well on unseen data too.
  130. 4:57Right? But the question is, how? How
  131. 5:00will you do this? Right? So now I will
  132. 5:02get into the mathematical formulation.
  133. 5:04I will show you through geometry how
  134. 5:06this happens. So the whole idea is very
  135. 5:08simple. What you do is you go parallel
  136. 5:11to this. First, you select a hyperplane
  137. 5:13. Let's say you selected a hyperplane.
  138. 5:15Okay? Now let's say you randomly
  139. 5:16selected a hyperplane. This is a
  140. 5:18hyperplane. Now what you do is you go
  141. 5:20in the positive direction parallel to
  142. 5:22this hyperplane. What do you do? You
  143. 5:25keep drawing hyperplanes in the
  144. 5:26parallel direction above it until you
  145. 5:28reach the first green point. As soon as
  146. 5:31you find a hyperplane that is parallel
  147. 5:34to pi and it touches the first green
  148. 5:36point, you stop there. You stop right
  149. 5:39there. Right? This line, this
  150. 5:42hyperplane, is parallel to pi. And this
  151. 5:45is the first hyperplane that touches
  152. 5:46the first green point, and this is a
  153. 5:48red flag. You will stop here. Okay? And
  154. 5:51you call this hyperplane pi positive.
  155. 5:54Right? Then you do the same thing in
  156. 5:56the opposite direction. You move in the
  157. 5:59negative direction parallel to pi and
  158. 6:01you will stop once you reach the first
  159. 6:03red point. Okay? So you stop here.
  160. 6:08Right? And you call this hyper...
  161. 6:10Obviously, both of these are
  162. 6:11hyperplanes. Right? These are like line
  163. 6:13segments. If you are talking about 2D,
  164. 6:15these are both lines. Actually, all
  165. 6:16three are lines. Okay? And you call
  166. 6:18this hyperplane pi minus. Okay? So you
  167. 6:21are finding...first you randomly
  168. 6:23selected this hyperplane. Then for this
  169. 6:25hyperplane, you found this pi positive
  170. 6:27and this pi negative. Right? Now what
  171. 6:30do you do? Now you calculate the margin
  172. 6:32. What is the margin? Margin is defined
  173. 6:34as the shortest distance between pi
  174. 6:36positive and pi minus. Both are
  175. 6:37parallel. Right? If pi plus is parallel
  176. 6:39to pi, and pi minus is parallel to pi,
  177. 6:41then pi minus and pi plus are parallel.
  178. 6:43So what you do is you calculate the
  179. 6:44shortest distance. You calculate this
  180. 6:47distance, right? And you call this
  181. 6:51distance margin and you denote it with
  182. 6:55'd', right? So first step, select a
  183. 6:59hyperplane. Step two, find out pi
  184. 7:01positive and pi negative hyperplanes,
  185. 7:03then calculate this distance. And then
  186. 7:06you find out, then you do this step
  187. 7:07with a bunch of other hyperplanes. You
  188. 7:09will calculate for all hyperplanes. For
  189. 7:11them, you will find pi plus and pi
  190. 7:13minus and calculate 'd' for all of them
  191. 7:15. And then you find out, then you
  192. 7:17decide, uh, you will select that
  193. 7:19particular plane for which the value of
  194. 7:22'd' is higher. That is what this
  195. 7:24algorithm is giving. You are selecting
  196. 7:26a hyperplane, you are finding positive,
  197. 7:28negative, you are finding this 'd', and
  198. 7:30then you are selecting that hyperplane
  199. 7:32for which this 'd' is maximum. So
  200. 7:34mathematically if you put this, then
  201. 7:36you want to find out, you are actually
  202. 7:38trying to maximize this 'd', uh, by
  203. 7:40finding out obviously the equation of
  204. 7:42the plane, you know, w transpose x + b,
  205. 7:44right? So what you are actually trying
  206. 7:46to do is, you are actually trying to
  207. 7:48find out w and b values for which this
  208. 7:50'd' will be maximum. This is what we
  209. 7:52are trying to do and it actually works
  210. 7:55very well. Uh, you can actually test it
  211. 7:58. Uh, let's say you created another
  212. 8:00line segment. Uh This is the one. Okay?
  213. 8:06Now for this, if you, let's call this '
  214. 8:09d prime'. Now for this, if you, uh,
  215. 8:12find out the positive and negative
  216. 8:13planes. Then this is the first point
  217. 8:16where it touches. Right? I will draw it
  218. 8:19. Right? And similarly this point is
  219. 8:23over here, right? And you can see this
  220. 8:26gap, this margin, let's call it 'd
  221. 8:28prime'. 'd prime' is less than 'd', and
  222. 8:32if you are asked to choose between
  223. 8:34these two, then obviously Support
  224. 8:37Vector Machine will choose 'pi' over '
  225. 8:39pi prime' because of the logic. Okay? I
  226. 8:43guess you understood the whole thing.
  227. 8:45Right? So how to do this? We will look
  228. 8:48at that. We are just taking a geometric
  229. 8:50intuition. Now I would like to discuss
  230. 8:52one important concept that will be used
  231. 8:54a lot later. The concept of support
  232. 8:56vectors. Support vectors. This is a
  233. 9:00very important concept. So important
  234. 9:02that the name of this algorithm itself
  235. 9:04is Support Vector Machines. Okay? So I
  236. 9:07will completely, I will draw a
  237. 9:08completely new diagram just to explain
  238. 9:11what are support vectors. So let's say
  239. 9:14we have these two classes of points
  240. 9:16again. Right? So you draw a hyperplane
  241. 9:20and then you create the negative as
  242. 9:23well as positive hyperplanes. Right?
  243. 9:28Now, do you know when you have to stop?
  244. 9:30Like when you find the first positive
  245. 9:31and negative points. Right? Meaning
  246. 9:34this point and these two points. Right?
  247. 9:38Now, these three points are what you
  248. 9:40call support vectors. So, this logic of
  249. 9:43yours, these planes only stop when they
  250. 9:46find the support vectors. Okay? And
  251. 9:48these are what we call support vectors,
  252. 9:50and they are important in driving this
  253. 9:51whole thing, and we will see that in
  254. 9:53future videos. I hope support vectors
  255. 9:55are clear. Two points that I would like
  256. 9:58to discuss: first, about support vector
  257. 10:00machines. Obviously, support vector
  258. 10:04machines are very robust to outliers.
  259. 10:10You will see this in a later video,
  260. 10:11unlike other machine learning
  261. 10:13algorithms that are heavily impacted by
  262. 10:15outliers. SVMs are actually solid when
  263. 10:18you talk about outliers; they are able
  264. 10:20to handle outliers quite well. Okay? So
  265. 10:24that is the first benefit of using SVMs
  266. 10:26. The second benefit is that it can
  267. 10:28also work on, or rather it can also
  268. 10:31work in the case of non-linear data.
  269. 10:34Like this is a very clear-cut case. You
  270. 10:36have linear data. You are able to
  271. 10:38classify it very easily. But what if
  272. 10:40you had data like this? Something like
  273. 10:44this. Obviously, you could not classify
  274. 10:47this with a single line. Right? So in
  275. 10:50such cases, you would see there is a
  276. 10:53thing in SVM called kernels. Using
  277. 10:56these kernels, you can actually solve
  278. 10:57this kind of problem. Okay? And the
  279. 11:00third point is that SVMs can be
  280. 11:02implemented for both classification as
  281. 11:05well as regression problems. Okay? So,
  282. 11:08I guess you understood this entire
  283. 11:10introduction. From the next video
  284. 11:13onwards, we will actually try to
  285. 11:15mathematically formulate how to
  286. 11:17calculate these distances. How these
  287. 11:19planes will be derived. And then, what
  288. 11:22will be the equation that you need to
  289. 11:25optimize to find the value of the w and
  290. 11:28b equation. Okay? So, I know it
  291. 11:31shouldn't feel too difficult, I guess,
  292. 11:33because it's not that difficult; at
  293. 11:34least the core idea is not that
  294. 11:36difficult. Yes, when you go to
  295. 11:37implement it in math, it feels a bit
  296. 11:39difficult. But for now, I think this
  297. 11:41entire thing is clear. So, that's it
  298. 11:43for the video, guys. Thanks for
  299. 11:45watching.

About this transcript

This page contains the full transcript of Support Vector Machines | Geometric Intuition by CampusX, generated from the public captions YouTube serves with the video. The transcript has 2,070 words across 299 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.