YouTube2Text

Voting Ensemble | Introduction and Core Idea | Part 1 — Transcript

by CampusX · 2,817 words · 401 segments · language en · Watch on YouTube

Full transcript

  1. 0:01Hello guys, welcome to my YouTube
  2. 0:03channel. In this video, we are going to
  3. 0:05discuss Voting Ensemble. So, we are
  4. 0:08studying ensemble techniques, and there
  5. 0:10are different techniques, as you
  6. 0:11already know. But today, we will cover
  7. 0:14the simplest technique among them. And
  8. 0:16it's called Voting Ensemble. Alright?
  9. 0:18It's a fairly simple technique, but
  10. 0:20still, I am planning to cover this
  11. 0:22whole thing in three videos. So that we
  12. 0:24can cover everything in detail. Alright
  13. 0:26? I guess you have no problem with that
  14. 0:28. Now let me share the screen first, so
  15. 0:30we can discuss what we are going to do
  16. 0:32in this video. So I have made a PPT,
  17. 0:34and I will use that PPT. So there will
  18. 0:37be three videos. In the first video, we
  19. 0:39will cover the core idea or the
  20. 0:40intuition behind this algorithm. In the
  21. 0:42second video, we will discuss that
  22. 0:44Voting Ensemble is a technique. And in
  23. 0:46this, you can perform both
  24. 0:48classification and regression. So, in
  25. 0:50the second video, we will focus only on
  26. 0:52classification. We will work on an
  27. 0:54example and check the hyperparameters
  28. 0:56in that class. Alright? And the third
  29. 0:59video will be about regression. So
  30. 1:01again, we will take an example. We will
  31. 1:03see how things work. Alright? So, this
  32. 1:05is the plan. Let's start with this
  33. 1:07video. So, in this video, I have
  34. 1:09planned to cover three things. First,
  35. 1:11the core idea. Without that, it won't
  36. 1:13work. Second, why does it work? So
  37. 1:15generally, when you study about voting
  38. 1:17classifiers, you will not get this
  39. 1:19answer. Meaning, the first time when I
  40. 1:22was doing or studying this algorithm, I
  41. 1:24didn't understand why it was working.
  42. 1:27So I would like to give you this
  43. 1:29intuition on why it works, the math
  44. 1:31behind this algorithm, and lastly, we
  45. 1:33will cover the assumptions you need to
  46. 1:35consider to use this algorithm. Alright
  47. 1:37? So, this video is divided into these
  48. 1:39three basic parts. Core intuition, why
  49. 1:42voting works, and assumptions. Okay? So
  50. 1:45I will go to OneNote, and over here, I
  51. 1:47will first explain the core idea. Okay?
  52. 1:50So the core idea is very simple, guys.
  53. 1:53See, as we discussed in ensemble,
  54. 1:55ensemble is basically a technique where
  55. 1:57you bring multiple models. And by
  56. 1:59combining their intelligence, you
  57. 2:01create a higher and more powerful
  58. 2:03intelligence. The idea here is the same
  59. 2:05too. In fact, the idea here is even
  60. 2:07simpler. What you do is, you bring some
  61. 2:10models, like these are your models.
  62. 2:14These are your models. Let's say we
  63. 2:16have three models. Let's call this one
  64. 2:19M1, this one M2, and this one M3. Right
  65. 2:23? You can take more models than this as
  66. 2:25well. And now you have a dataset. You
  67. 2:27have a dataset. And what do you do? You
  68. 2:31feed this dataset to every algorithm.
  69. 2:34Now mind you, these three can be
  70. 2:36different algorithms. These three can
  71. 2:38be the same algorithms as well. Okay?
  72. 2:40We will see both examples when we build
  73. 2:42a voting classifier. Okay? Now, these
  74. 2:45three models are trained independently
  75. 2:48on the same data. Right? This is the
  76. 2:50training phase. Independently, all of
  77. 2:52these models are trained on the same
  78. 2:54data. This isn't. Right? Next, what you
  79. 2:56do is once these models are trained,
  80. 2:58our system is ready for prediction. So
  81. 3:00now what do you do? You bring in a new
  82. 3:02query point. Okay? Let's call it XQ.
  83. 3:05Let's call it XQ. Right? Now you send
  84. 3:10XQ to all three for prediction using
  85. 3:12the predict method. Okay? Now let's say
  86. 3:16this is a classification task. Your
  87. 3:18classes are one or zero, or more than
  88. 3:20that. Now model one is predicting some
  89. 3:22class. Let's say one. Model two, let's
  90. 3:25say, says it is zero. And model three
  91. 3:27also says it is zero. Now, since the
  92. 3:31majority of the models are saying that
  93. 3:33its output should be zero. Your overall
  94. 3:38model, the voting classifier, which is
  95. 3:40a combination of all of these, will
  96. 3:42also tell you that the output should be
  97. 3:44zero, and that is how it works. It's
  98. 3:47just that simple, guys. Okay? If you
  99. 3:49talk about regression, then in
  100. 3:51regression, instead of classes, your
  101. 3:54output would be some number like 75,
  102. 3:5682.55. Now in this case, what happens
  103. 4:02is that your output will actually be
  104. 4:03the mean of these three. Okay? So you
  105. 4:06will add these three and divide by
  106. 4:08three. And that will be your output.
  107. 4:10And that is the core idea. Guys, that
  108. 4:12is all you have to do in this algorithm
  109. 4:14. I hope you understood the core
  110. 4:15intuition. Okay? Very simple. It's like
  111. 4:18democracy, just like voting. It is
  112. 4:20exactly the same. Voting is happening
  113. 4:22here. That is why the name voting. Okay
  114. 4:25? Now let's move on to the second part.
  115. 4:28Why voting works. So I tried to read a
  116. 4:30lot, and in many places, I didn't get
  117. 4:32the answer to this. Because if you
  118. 4:34think about this, how can this be? I
  119. 4:36mean, if I have a model that has an
  120. 4:39accuracy of .7. Another model that has
  121. 4:41an accuracy of .6, and one more with an
  122. 4:44accuracy of .55. How can I combine them
  123. 4:48to get a new model whose accuracy is
  124. 4:50higher than any of the three? I don't
  125. 4:54know what I just said, but I guess you
  126. 4:56get the idea. How are we combining them
  127. 4:59such that you can reach an accuracy
  128. 5:01even higher than the best one, which is
  129. 5:03.7? How is that happening? That is what
  130. 5:06we need to discuss. Okay? Before
  131. 5:09discussing this, I will do two things.
  132. 5:13First, I will tell you the assumptions
  133. 5:15behind voting ensembles. Because
  134. 5:17without that, it won't work. The
  135. 5:19assumption is this: all your models,
  136. 5:21the base models, whoever you call base
  137. 5:24models, these three base models should
  138. 5:26be independent. They should be as
  139. 5:29different as they can possibly be. Why
  140. 5:31that is, you will get the answer in a
  141. 5:33little while. But always remember, if
  142. 5:35your models are very similar, then your
  143. 5:37voting classifier will not perform well
  144. 5:39. If they are very dissimilar, then
  145. 5:42your voting classifier performs well.
  146. 5:44This is the first assumption. The
  147. 5:47second assumption is very important,
  148. 5:49listen carefully; the second assumption
  149. 5:51is that the minimum accuracy of all
  150. 5:54your models should be 51%. Below that,
  151. 5:57it won't help, it will actually be
  152. 5:59harmful. Meaning, if your accuracy is
  153. 6:01less than 50%, then the combination you
  154. 6:04create, the voting classifier you
  155. 6:06create, its accuracy will be worse than
  156. 6:08the worst of the three. Again, you will
  157. 6:10understand why in a little while. But I
  158. 6:12am stating two assumptions. The first
  159. 6:14assumption is that models should be
  160. 6:16independent in nature, they should be
  161. 6:18different. Second, every individual
  162. 6:20model must have an accuracy of more
  163. 6:22than 50%. If it is less than 50%for all
  164. 6:26three, then your resultant will be
  165. 6:28worse than the worst. Okay? I hope you
  166. 6:30are understanding this. You might be
  167. 6:32thinking, what is this guy saying? What
  168. 6:34is happening here? So for that, we will
  169. 6:36understand how voting works. Okay? Not
  170. 6:39only will we get an intuition behind it
  171. 6:41. We will also try to take a
  172. 6:44mathematical, I don't know, I will show
  173. 6:46you with an example how this thing
  174. 6:48works mathematically. Okay? Let me
  175. 6:51first give you a general intuition, no
  176. 6:53math. So if you look at voting, the
  177. 6:55idea of voting is very simple. Now why
  178. 6:58does democracy rely on voting? Because
  179. 7:01people feel that people are from
  180. 7:02different backgrounds and they have
  181. 7:03different knowledge. So when one person
  182. 7:05lacks knowledge in something, the other
  183. 7:06person will have knowledge of it. Right
  184. 7:08? And vice versa. So in such a scenario
  185. 7:10, one person's deficiency is covered by
  186. 7:13the other person. Right? That is the
  187. 7:15core idea behind voting. Think about it
  188. 7:19, when you sit in KBC, there is a
  189. 7:21public poll. Public opinion, whatever
  190. 7:25it is, I don't remember. But think if
  191. 7:27everyone sitting in the audience—
  192. 7:29sorry, it's called an audience poll. If
  193. 7:33all the people sitting there are from a
  194. 7:35computer science background, would you
  195. 7:37get more help as a participant, or
  196. 7:38would you get more help if they are
  197. 7:40from different backgrounds? One is a CA
  198. 7:43, one is an engineer, one is a doctor,
  199. 7:46one is a painter, one is a poet. In
  200. 7:48which scenario would you feel more
  201. 7:50powerful as a contestant, a participant
  202. 7:52? I think you know the answer. If all
  203. 7:55the people sitting there are
  204. 7:56programmers, they will answer computer
  205. 7:58science-related questions very well.
  206. 8:00But if a question comes from outside
  207. 8:02that domain, you won't get the answers.
  208. 8:05But if you have a diversified audience
  209. 8:07where people from every background are
  210. 8:09present, that audience can help you
  211. 8:11more. Because if one person doesn't
  212. 8:12know something, it's possible that
  213. 8:14others might know it. Okay? So that is
  214. 8:16the core idea. That is why we made the
  215. 8:18assumption that they should be
  216. 8:19different. Okay? So that is why voting
  217. 8:21works if we talk in terms of intuition.
  218. 8:23Now, if someone asks you,
  219. 8:25mathematically, can you prove that by
  220. 8:27combining three weak base models, you
  221. 8:29can create a strong model? How can you
  222. 8:32do it? How is it even possible? So the
  223. 8:34answer is hidden in probability. The
  224. 8:36answer is hidden in probability. I will
  225. 8:38tell you how. Okay? Watch carefully.
  226. 8:40Rub this entire, rub this entire thing.
  227. 8:44Let's say we have three models. This is
  228. 8:47M1. This is M2, this is M3, okay, they
  229. 8:52can be any algorithms. One is a
  230. 8:55decision tree, one is a KNN, one is an
  231. 8:58SVM, anything. Okay? But the accuracy
  232. 9:02of all three is .7, and these three
  233. 9:04models are independent, by the way.
  234. 9:07They are being trained separately, they
  235. 9:09are behaving separately. There is no
  236. 9:10correlation. Okay? Now, if I ask you to
  237. 9:13build a voting classifier, what does
  238. 9:15building a voting classifier mean? It
  239. 9:18means you go with whatever the majority
  240. 9:20says. So, if two people say it's one,
  241. 9:22then it's one. If two or more people
  242. 9:25say it's zero, then it's zero. Right?
  243. 9:28So, if I want to build a voting
  244. 9:30classifier, first of all, I have to
  245. 9:32prove that the accuracy of that voting
  246. 9:34classifier will be greater than 0.7. I
  247. 9:37have to prove this. Let's see how it's
  248. 9:39done. So, if its accuracy is 0.7, then
  249. 9:42the chance of it being wrong is 0.3 for
  250. 9:45all three models. Right? Now, let's say
  251. 9:49you give a query point to the first
  252. 9:51model, and the prediction of this model
  253. 9:53on that query point is correct. And the
  254. 9:58prediction of this model, and the
  255. 10:00prediction of this model, are correct
  256. 10:02on that new query point. What is the
  257. 10:04probability of this? 0.7, 0.7, and what
  258. 10:07is the probability of it being wrong?
  259. 10:100.3. Now, here comes the second model.
  260. 10:15Now, if you gave it correctly to the
  261. 10:17second model here, what is the
  262. 10:18probability of it being correct here as
  263. 10:21well? 0.7, because they are independent
  264. 10:23events, and 0.3 for being wrong. And if
  265. 10:26it was wrong, what is the probability
  266. 10:28of it being correct? 0.7 and 0.3. Now
  267. 10:32you brought M3 here. There can be four
  268. 10:35cases for M3. Right? Now, the
  269. 10:39probability for correct, correct, and
  270. 10:42correct is 0.7; for correct, correct,
  271. 10:45wrong is 0.3; for correct, wrong,
  272. 10:48correct is 0.7; for correct, wrong,
  273. 10:51wrong is 0.3; for wrong, correct,
  274. 10:54correct is 0.7; for wrong, correct...
  275. 10:57this is a tongue twister, 0.3. I will
  276. 11:00just fill it up: 0.7, 0.3. So, I
  277. 11:05basically wrote down all your
  278. 11:07combinations, that's what I did. Okay?
  279. 11:09I wrote down the combinations. So the
  280. 11:13combination, the first combination, if
  281. 11:15you calculate its total accuracy, it
  282. 11:18would come out to be 0.343. If you
  283. 11:23calculate this one, it's 0.147. If you
  284. 11:29calculate this one, 0.147, and if you
  285. 11:32calculate this one...0.063, if you
  286. 11:39calculate this, 0.147, if you calculate
  287. 11:44this one, 0.063. Let's calculate this
  288. 11:50one. 0.063, and let's calculate this
  289. 11:53one. Oh, this should also have been
  290. 11:550.063. Sorry, my bad, this should also
  291. 11:57have been 0.063. 3 times 3 is 9, times
  292. 12:007 is 0.063, okay? And here will be
  293. 12:030.027. So guys, these are all our
  294. 12:05probabilities. If you add them up, you
  295. 12:08can add them, it will be one. The total
  296. 12:10probability is one because these are
  297. 12:12the only eight possibilities. Right? So
  298. 12:14adding them all up must result in one.
  299. 12:16Now, think about what your voting
  300. 12:18classifier needs. You need those cases
  301. 12:21where at least two of your classifiers
  302. 12:23are saying the correct thing. Okay? How
  303. 12:26many such cases are there? Just think
  304. 12:27about it. There is one such case, this
  305. 12:31one, because here 0.7 all three are
  306. 12:33correct. The second one, because here
  307. 12:360.7 ^ 2 two are correct. This one,
  308. 12:39because here also it is 0.7 ^ 2. This
  309. 12:42one is not, because here it is 0.3 ^ 2,
  310. 12:44meaning two are wrong. This one, this
  311. 12:47one, this one, this one, and after that
  312. 12:50this one is also not, because it has
  313. 12:520.3 ^ 2. This one also has 0.3 ^ 2, so
  314. 12:55it won't be included. So basically,
  315. 12:56this one, this one, this one, and this
  316. 12:58—these are your four cases when your
  317. 13:00minimum two predictions will be correct
  318. 13:02. So what am I doing? I am adding all
  319. 13:04these accuracies. So this is 34%, this
  320. 13:07is 14.7%. This is also 14.7%, and this
  321. 13:11is also 14.7%, and this is 34.3%. Okay?
  322. 13:15If you add all of these, you will get
  323. 13:17the accuracy of your voting classifier.
  324. 13:19Right? So 34 + 42 is 76, and if you add
  325. 13:277, you come somewhere around 78. So you
  326. 13:32can see, here we have mathematically
  327. 13:34proven that your voting classifier,
  328. 13:36which is made of classifiers with 0.7
  329. 13:40accuracy, its collective accuracy is
  330. 13:42coming out to be greater than each one
  331. 13:44of them, which is 78. So I guess, from
  332. 13:46here you understand how this whole
  333. 13:48thing is working. What is the math
  334. 13:50behind it. Okay? When I understood it
  335. 13:52for the first time, I enjoyed it. Okay?
  336. 13:54Now let me ask you a question. You can
  337. 13:55pause the video and you can think about
  338. 13:57this. I gave you a second assumption. I
  339. 13:59said that the accuracy of everyone
  340. 14:01should be greater than 50%. It must be
  341. 14:03at least 51%. Why did I say that? I
  342. 14:05said that if it is less than 50%, then
  343. 14:07the problem that will occur is that
  344. 14:09your voting classifier's accuracy will
  345. 14:11be worse than even the worst one. Can
  346. 14:13you prove this? You absolutely can.
  347. 14:15Look, what will you do? Let's assume
  348. 14:17one thing, that the accuracy of these
  349. 14:19three is 0.3. Instead of 0.7, let's
  350. 14:21assume 0.3. Right? So basically, now
  351. 14:24being correct two times means 0.3 ^ 2*
  352. 14:260.7, and being correct three times
  353. 14:29means 0.3 ^ 3. Okay, what is the
  354. 14:30accuracy of all three? 0.3. The
  355. 14:34accuracy of all three is less than 50%.
  356. 14:38Now I want to prove that the voting
  357. 14:39classifier formed by combining them
  358. 14:41will have an accuracy less than each
  359. 14:43one of them. Can we prove this? We can
  360. 14:46actually prove this. It's very simple,
  361. 14:48you will make the same chart, just this
  362. 14:51time your cases will be reversed. This
  363. 14:53time you will take cases where 3
  364. 14:55appears twice or more than twice. You
  365. 14:59won't take this, won't take this either
  366. 15:01, won't take this. You will take this
  367. 15:04because here there is a 7, a 3, and 3
  368. 15:06is there twice. You will take this one
  369. 15:10because 3 is there twice. You will take
  370. 15:12this one and you will take this one too
  371. 15:14. Okay, so if you add this up, then 6.3
  372. 15:19, 6.3, 6.3 and 2 or 2, whatever it is,
  373. 15:22adding it up will give you an accuracy
  374. 15:25of somewhere around 20%. Or you can
  375. 15:30also say it like this, subtract 78 from
  376. 15:32100 because the remaining four cases
  377. 15:34are the ones you formed. In this case,
  378. 15:38the correct probability is you will get
  379. 15:40somewhere around 22%accuracy. Whereas
  380. 15:43everyone's accuracy here is 30%. Okay?
  381. 15:47So we have proven that for a voting
  382. 15:49classifier to work, you need at least
  383. 15:52one classifier that is predicting
  384. 15:54better than 50%. And man, think about
  385. 15:57it, 50%is just random, right? Making it
  386. 15:59better than that is really not a big
  387. 16:01deal. So in a way, we can say that a
  388. 16:03voting classifier is quite a safe thing
  389. 16:05. It will generally help you. Okay? So
  390. 16:07this was the introduction core idea
  391. 16:09video. I hope you understood everything
  392. 16:11. In the next video, we will discuss
  393. 16:13the voting classifier. Specifically, we
  394. 16:15will learn the voting classifier and do
  395. 16:17some coding for it. We will see how to
  396. 16:19adjust its hyperparameters. In fact, I
  397. 16:21have built a web app for you where I
  398. 16:22will run and show you the outputs. Okay
  399. 16:24? So yeah, if you liked this video,
  400. 16:26please consider subscribing. Thanks for
  401. 16:29watching.

About this transcript

This page contains the full transcript of Voting Ensemble | Introduction and Core Idea | Part 1 by CampusX, generated from the public captions YouTube serves with the video. The transcript has 2,817 words across 401 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.