Voting Ensemble | Introduction and Core Idea | Part 1 — Transcript
Full transcript
- 0:01Hello guys, welcome to my YouTube
- 0:03channel. In this video, we are going to
- 0:05discuss Voting Ensemble. So, we are
- 0:08studying ensemble techniques, and there
- 0:10are different techniques, as you
- 0:11already know. But today, we will cover
- 0:14the simplest technique among them. And
- 0:16it's called Voting Ensemble. Alright?
- 0:18It's a fairly simple technique, but
- 0:20still, I am planning to cover this
- 0:22whole thing in three videos. So that we
- 0:24can cover everything in detail. Alright
- 0:26? I guess you have no problem with that
- 0:28. Now let me share the screen first, so
- 0:30we can discuss what we are going to do
- 0:32in this video. So I have made a PPT,
- 0:34and I will use that PPT. So there will
- 0:37be three videos. In the first video, we
- 0:39will cover the core idea or the
- 0:40intuition behind this algorithm. In the
- 0:42second video, we will discuss that
- 0:44Voting Ensemble is a technique. And in
- 0:46this, you can perform both
- 0:48classification and regression. So, in
- 0:50the second video, we will focus only on
- 0:52classification. We will work on an
- 0:54example and check the hyperparameters
- 0:56in that class. Alright? And the third
- 0:59video will be about regression. So
- 1:01again, we will take an example. We will
- 1:03see how things work. Alright? So, this
- 1:05is the plan. Let's start with this
- 1:07video. So, in this video, I have
- 1:09planned to cover three things. First,
- 1:11the core idea. Without that, it won't
- 1:13work. Second, why does it work? So
- 1:15generally, when you study about voting
- 1:17classifiers, you will not get this
- 1:19answer. Meaning, the first time when I
- 1:22was doing or studying this algorithm, I
- 1:24didn't understand why it was working.
- 1:27So I would like to give you this
- 1:29intuition on why it works, the math
- 1:31behind this algorithm, and lastly, we
- 1:33will cover the assumptions you need to
- 1:35consider to use this algorithm. Alright
- 1:37? So, this video is divided into these
- 1:39three basic parts. Core intuition, why
- 1:42voting works, and assumptions. Okay? So
- 1:45I will go to OneNote, and over here, I
- 1:47will first explain the core idea. Okay?
- 1:50So the core idea is very simple, guys.
- 1:53See, as we discussed in ensemble,
- 1:55ensemble is basically a technique where
- 1:57you bring multiple models. And by
- 1:59combining their intelligence, you
- 2:01create a higher and more powerful
- 2:03intelligence. The idea here is the same
- 2:05too. In fact, the idea here is even
- 2:07simpler. What you do is, you bring some
- 2:10models, like these are your models.
- 2:14These are your models. Let's say we
- 2:16have three models. Let's call this one
- 2:19M1, this one M2, and this one M3. Right
- 2:23? You can take more models than this as
- 2:25well. And now you have a dataset. You
- 2:27have a dataset. And what do you do? You
- 2:31feed this dataset to every algorithm.
- 2:34Now mind you, these three can be
- 2:36different algorithms. These three can
- 2:38be the same algorithms as well. Okay?
- 2:40We will see both examples when we build
- 2:42a voting classifier. Okay? Now, these
- 2:45three models are trained independently
- 2:48on the same data. Right? This is the
- 2:50training phase. Independently, all of
- 2:52these models are trained on the same
- 2:54data. This isn't. Right? Next, what you
- 2:56do is once these models are trained,
- 2:58our system is ready for prediction. So
- 3:00now what do you do? You bring in a new
- 3:02query point. Okay? Let's call it XQ.
- 3:05Let's call it XQ. Right? Now you send
- 3:10XQ to all three for prediction using
- 3:12the predict method. Okay? Now let's say
- 3:16this is a classification task. Your
- 3:18classes are one or zero, or more than
- 3:20that. Now model one is predicting some
- 3:22class. Let's say one. Model two, let's
- 3:25say, says it is zero. And model three
- 3:27also says it is zero. Now, since the
- 3:31majority of the models are saying that
- 3:33its output should be zero. Your overall
- 3:38model, the voting classifier, which is
- 3:40a combination of all of these, will
- 3:42also tell you that the output should be
- 3:44zero, and that is how it works. It's
- 3:47just that simple, guys. Okay? If you
- 3:49talk about regression, then in
- 3:51regression, instead of classes, your
- 3:54output would be some number like 75,
- 3:5682.55. Now in this case, what happens
- 4:02is that your output will actually be
- 4:03the mean of these three. Okay? So you
- 4:06will add these three and divide by
- 4:08three. And that will be your output.
- 4:10And that is the core idea. Guys, that
- 4:12is all you have to do in this algorithm
- 4:14. I hope you understood the core
- 4:15intuition. Okay? Very simple. It's like
- 4:18democracy, just like voting. It is
- 4:20exactly the same. Voting is happening
- 4:22here. That is why the name voting. Okay
- 4:25? Now let's move on to the second part.
- 4:28Why voting works. So I tried to read a
- 4:30lot, and in many places, I didn't get
- 4:32the answer to this. Because if you
- 4:34think about this, how can this be? I
- 4:36mean, if I have a model that has an
- 4:39accuracy of .7. Another model that has
- 4:41an accuracy of .6, and one more with an
- 4:44accuracy of .55. How can I combine them
- 4:48to get a new model whose accuracy is
- 4:50higher than any of the three? I don't
- 4:54know what I just said, but I guess you
- 4:56get the idea. How are we combining them
- 4:59such that you can reach an accuracy
- 5:01even higher than the best one, which is
- 5:03.7? How is that happening? That is what
- 5:06we need to discuss. Okay? Before
- 5:09discussing this, I will do two things.
- 5:13First, I will tell you the assumptions
- 5:15behind voting ensembles. Because
- 5:17without that, it won't work. The
- 5:19assumption is this: all your models,
- 5:21the base models, whoever you call base
- 5:24models, these three base models should
- 5:26be independent. They should be as
- 5:29different as they can possibly be. Why
- 5:31that is, you will get the answer in a
- 5:33little while. But always remember, if
- 5:35your models are very similar, then your
- 5:37voting classifier will not perform well
- 5:39. If they are very dissimilar, then
- 5:42your voting classifier performs well.
- 5:44This is the first assumption. The
- 5:47second assumption is very important,
- 5:49listen carefully; the second assumption
- 5:51is that the minimum accuracy of all
- 5:54your models should be 51%. Below that,
- 5:57it won't help, it will actually be
- 5:59harmful. Meaning, if your accuracy is
- 6:01less than 50%, then the combination you
- 6:04create, the voting classifier you
- 6:06create, its accuracy will be worse than
- 6:08the worst of the three. Again, you will
- 6:10understand why in a little while. But I
- 6:12am stating two assumptions. The first
- 6:14assumption is that models should be
- 6:16independent in nature, they should be
- 6:18different. Second, every individual
- 6:20model must have an accuracy of more
- 6:22than 50%. If it is less than 50%for all
- 6:26three, then your resultant will be
- 6:28worse than the worst. Okay? I hope you
- 6:30are understanding this. You might be
- 6:32thinking, what is this guy saying? What
- 6:34is happening here? So for that, we will
- 6:36understand how voting works. Okay? Not
- 6:39only will we get an intuition behind it
- 6:41. We will also try to take a
- 6:44mathematical, I don't know, I will show
- 6:46you with an example how this thing
- 6:48works mathematically. Okay? Let me
- 6:51first give you a general intuition, no
- 6:53math. So if you look at voting, the
- 6:55idea of voting is very simple. Now why
- 6:58does democracy rely on voting? Because
- 7:01people feel that people are from
- 7:02different backgrounds and they have
- 7:03different knowledge. So when one person
- 7:05lacks knowledge in something, the other
- 7:06person will have knowledge of it. Right
- 7:08? And vice versa. So in such a scenario
- 7:10, one person's deficiency is covered by
- 7:13the other person. Right? That is the
- 7:15core idea behind voting. Think about it
- 7:19, when you sit in KBC, there is a
- 7:21public poll. Public opinion, whatever
- 7:25it is, I don't remember. But think if
- 7:27everyone sitting in the audience—
- 7:29sorry, it's called an audience poll. If
- 7:33all the people sitting there are from a
- 7:35computer science background, would you
- 7:37get more help as a participant, or
- 7:38would you get more help if they are
- 7:40from different backgrounds? One is a CA
- 7:43, one is an engineer, one is a doctor,
- 7:46one is a painter, one is a poet. In
- 7:48which scenario would you feel more
- 7:50powerful as a contestant, a participant
- 7:52? I think you know the answer. If all
- 7:55the people sitting there are
- 7:56programmers, they will answer computer
- 7:58science-related questions very well.
- 8:00But if a question comes from outside
- 8:02that domain, you won't get the answers.
- 8:05But if you have a diversified audience
- 8:07where people from every background are
- 8:09present, that audience can help you
- 8:11more. Because if one person doesn't
- 8:12know something, it's possible that
- 8:14others might know it. Okay? So that is
- 8:16the core idea. That is why we made the
- 8:18assumption that they should be
- 8:19different. Okay? So that is why voting
- 8:21works if we talk in terms of intuition.
- 8:23Now, if someone asks you,
- 8:25mathematically, can you prove that by
- 8:27combining three weak base models, you
- 8:29can create a strong model? How can you
- 8:32do it? How is it even possible? So the
- 8:34answer is hidden in probability. The
- 8:36answer is hidden in probability. I will
- 8:38tell you how. Okay? Watch carefully.
- 8:40Rub this entire, rub this entire thing.
- 8:44Let's say we have three models. This is
- 8:47M1. This is M2, this is M3, okay, they
- 8:52can be any algorithms. One is a
- 8:55decision tree, one is a KNN, one is an
- 8:58SVM, anything. Okay? But the accuracy
- 9:02of all three is .7, and these three
- 9:04models are independent, by the way.
- 9:07They are being trained separately, they
- 9:09are behaving separately. There is no
- 9:10correlation. Okay? Now, if I ask you to
- 9:13build a voting classifier, what does
- 9:15building a voting classifier mean? It
- 9:18means you go with whatever the majority
- 9:20says. So, if two people say it's one,
- 9:22then it's one. If two or more people
- 9:25say it's zero, then it's zero. Right?
- 9:28So, if I want to build a voting
- 9:30classifier, first of all, I have to
- 9:32prove that the accuracy of that voting
- 9:34classifier will be greater than 0.7. I
- 9:37have to prove this. Let's see how it's
- 9:39done. So, if its accuracy is 0.7, then
- 9:42the chance of it being wrong is 0.3 for
- 9:45all three models. Right? Now, let's say
- 9:49you give a query point to the first
- 9:51model, and the prediction of this model
- 9:53on that query point is correct. And the
- 9:58prediction of this model, and the
- 10:00prediction of this model, are correct
- 10:02on that new query point. What is the
- 10:04probability of this? 0.7, 0.7, and what
- 10:07is the probability of it being wrong?
- 10:100.3. Now, here comes the second model.
- 10:15Now, if you gave it correctly to the
- 10:17second model here, what is the
- 10:18probability of it being correct here as
- 10:21well? 0.7, because they are independent
- 10:23events, and 0.3 for being wrong. And if
- 10:26it was wrong, what is the probability
- 10:28of it being correct? 0.7 and 0.3. Now
- 10:32you brought M3 here. There can be four
- 10:35cases for M3. Right? Now, the
- 10:39probability for correct, correct, and
- 10:42correct is 0.7; for correct, correct,
- 10:45wrong is 0.3; for correct, wrong,
- 10:48correct is 0.7; for correct, wrong,
- 10:51wrong is 0.3; for wrong, correct,
- 10:54correct is 0.7; for wrong, correct...
- 10:57this is a tongue twister, 0.3. I will
- 11:00just fill it up: 0.7, 0.3. So, I
- 11:05basically wrote down all your
- 11:07combinations, that's what I did. Okay?
- 11:09I wrote down the combinations. So the
- 11:13combination, the first combination, if
- 11:15you calculate its total accuracy, it
- 11:18would come out to be 0.343. If you
- 11:23calculate this one, it's 0.147. If you
- 11:29calculate this one, 0.147, and if you
- 11:32calculate this one...0.063, if you
- 11:39calculate this, 0.147, if you calculate
- 11:44this one, 0.063. Let's calculate this
- 11:50one. 0.063, and let's calculate this
- 11:53one. Oh, this should also have been
- 11:550.063. Sorry, my bad, this should also
- 11:57have been 0.063. 3 times 3 is 9, times
- 12:007 is 0.063, okay? And here will be
- 12:030.027. So guys, these are all our
- 12:05probabilities. If you add them up, you
- 12:08can add them, it will be one. The total
- 12:10probability is one because these are
- 12:12the only eight possibilities. Right? So
- 12:14adding them all up must result in one.
- 12:16Now, think about what your voting
- 12:18classifier needs. You need those cases
- 12:21where at least two of your classifiers
- 12:23are saying the correct thing. Okay? How
- 12:26many such cases are there? Just think
- 12:27about it. There is one such case, this
- 12:31one, because here 0.7 all three are
- 12:33correct. The second one, because here
- 12:360.7 ^ 2 two are correct. This one,
- 12:39because here also it is 0.7 ^ 2. This
- 12:42one is not, because here it is 0.3 ^ 2,
- 12:44meaning two are wrong. This one, this
- 12:47one, this one, this one, and after that
- 12:50this one is also not, because it has
- 12:520.3 ^ 2. This one also has 0.3 ^ 2, so
- 12:55it won't be included. So basically,
- 12:56this one, this one, this one, and this
- 12:58—these are your four cases when your
- 13:00minimum two predictions will be correct
- 13:02. So what am I doing? I am adding all
- 13:04these accuracies. So this is 34%, this
- 13:07is 14.7%. This is also 14.7%, and this
- 13:11is also 14.7%, and this is 34.3%. Okay?
- 13:15If you add all of these, you will get
- 13:17the accuracy of your voting classifier.
- 13:19Right? So 34 + 42 is 76, and if you add
- 13:277, you come somewhere around 78. So you
- 13:32can see, here we have mathematically
- 13:34proven that your voting classifier,
- 13:36which is made of classifiers with 0.7
- 13:40accuracy, its collective accuracy is
- 13:42coming out to be greater than each one
- 13:44of them, which is 78. So I guess, from
- 13:46here you understand how this whole
- 13:48thing is working. What is the math
- 13:50behind it. Okay? When I understood it
- 13:52for the first time, I enjoyed it. Okay?
- 13:54Now let me ask you a question. You can
- 13:55pause the video and you can think about
- 13:57this. I gave you a second assumption. I
- 13:59said that the accuracy of everyone
- 14:01should be greater than 50%. It must be
- 14:03at least 51%. Why did I say that? I
- 14:05said that if it is less than 50%, then
- 14:07the problem that will occur is that
- 14:09your voting classifier's accuracy will
- 14:11be worse than even the worst one. Can
- 14:13you prove this? You absolutely can.
- 14:15Look, what will you do? Let's assume
- 14:17one thing, that the accuracy of these
- 14:19three is 0.3. Instead of 0.7, let's
- 14:21assume 0.3. Right? So basically, now
- 14:24being correct two times means 0.3 ^ 2*
- 14:260.7, and being correct three times
- 14:29means 0.3 ^ 3. Okay, what is the
- 14:30accuracy of all three? 0.3. The
- 14:34accuracy of all three is less than 50%.
- 14:38Now I want to prove that the voting
- 14:39classifier formed by combining them
- 14:41will have an accuracy less than each
- 14:43one of them. Can we prove this? We can
- 14:46actually prove this. It's very simple,
- 14:48you will make the same chart, just this
- 14:51time your cases will be reversed. This
- 14:53time you will take cases where 3
- 14:55appears twice or more than twice. You
- 14:59won't take this, won't take this either
- 15:01, won't take this. You will take this
- 15:04because here there is a 7, a 3, and 3
- 15:06is there twice. You will take this one
- 15:10because 3 is there twice. You will take
- 15:12this one and you will take this one too
- 15:14. Okay, so if you add this up, then 6.3
- 15:19, 6.3, 6.3 and 2 or 2, whatever it is,
- 15:22adding it up will give you an accuracy
- 15:25of somewhere around 20%. Or you can
- 15:30also say it like this, subtract 78 from
- 15:32100 because the remaining four cases
- 15:34are the ones you formed. In this case,
- 15:38the correct probability is you will get
- 15:40somewhere around 22%accuracy. Whereas
- 15:43everyone's accuracy here is 30%. Okay?
- 15:47So we have proven that for a voting
- 15:49classifier to work, you need at least
- 15:52one classifier that is predicting
- 15:54better than 50%. And man, think about
- 15:57it, 50%is just random, right? Making it
- 15:59better than that is really not a big
- 16:01deal. So in a way, we can say that a
- 16:03voting classifier is quite a safe thing
- 16:05. It will generally help you. Okay? So
- 16:07this was the introduction core idea
- 16:09video. I hope you understood everything
- 16:11. In the next video, we will discuss
- 16:13the voting classifier. Specifically, we
- 16:15will learn the voting classifier and do
- 16:17some coding for it. We will see how to
- 16:19adjust its hyperparameters. In fact, I
- 16:21have built a web app for you where I
- 16:22will run and show you the outputs. Okay
- 16:24? So yeah, if you liked this video,
- 16:26please consider subscribing. Thanks for
- 16:29watching.
About this transcript
This page contains the full transcript of Voting Ensemble | Introduction and Core Idea | Part 1 by CampusX, generated from the public captions YouTube serves with the video. The transcript has 2,817 words across 401 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.