Bagging Vs Random Forest | What is the difference between Bagging and Random Forest | Very Important — Transcript
Full transcript
- 0:00Hello guys, welcome to my YouTube
- 0:02channel. We've been studying random
- 0:04forests and covered a couple of topics.
- 0:07Now, we're going to have an important
- 0:10discussion about the difference between
- 0:12bagging, which we just studied, and
- 0:14random forests, which we're currently
- 0:16studying. So, I've already told you
- 0:19that random forests are internally an
- 0:22implementation of bagging. But there
- 0:25are some important differences between
- 0:27the two. In fact, this is a very good
- 0:29interview question: what exactly is the
- 0:31difference between random forests and
- 0:33bagging? Okay? So, that's what we're
- 0:35going to cover in this video. I'll tell
- 0:37you two differences. Okay? So, the
- 0:40first difference is pretty
- 0:41straightforward. You probably already
- 0:44know what you can do with bagging. You
- 0:48can use different algorithms. So, if
- 0:51you want, you can create a bagging
- 0:54ensemble using decision trees, but you
- 0:57can also create a bagging ensemble
- 1:00using kNN. This means that all your
- 1:04base estimators can be decision trees,
- 1:09kNN, or even SVM. Right? In fact, you
- 1:13can try other algorithms, but these
- 1:16three give you the best results. So, in
- 1:19short, bagging is a general technique
- 1:22in which you can use any algorithm as
- 1:25the base algorithm. Right? But the base
- 1:29algorithm in a random forest is only
- 1:32and only decision trees. This means
- 1:35that a random forest is created by
- 1:37combining decision trees. You can't use
- 1:39kNN, SVM, or any other algorithm. Right
- 1:43? I can give you a logical explanation
- 1:47for this. And that is, if you go to
- 1:52ScikitLearn and open the bagging
- 1:58classifier, you'll see a parameter
- 2:03called base estimator. Now, what this
- 2:08base estimator is, is that you can
- 2:10select different algorithms. It says
- 2:13that if its value is NULL, the base
- 2:15estimator is a decision tree classifier
- 2:18. Or whatever you want to specify, if
- 2:24you search for "SKLearn Random Forest
- 2:32Classifier," you won't see any such
- 2:36parameter. "C" is the parameter for
- 2:39selecting the number of estimators, but
- 2:41it's not the parameter for selecting
- 2:43the base estimator. So, this actually
- 2:46shows the first difference between
- 2:48random forest and bagging. I'm
- 2:50summarizing it once. In bagging, your
- 2:53base models can be created by any
- 2:55algorithm. In random forest, there's a
- 2:58restriction that the base model can
- 3:00only be a decision tree. Okay? Now,
- 3:04when I first read this, a question came
- 3:07to my mind: what if I create a bagging
- 3:10classifier and call all its base models
- 3:14decision trees or keep them as decision
- 3:17trees? So, is this whole thing a random
- 3:21forest now? Is this whole thing a
- 3:25random forest now? The answer is no.
- 3:31There's still a big difference. Right?
- 3:34I guess you're getting the point: if
- 3:37you're creating a bagging ensemble or
- 3:40ensemble using decision trees, it's not
- 3:43exactly a random forest. There's one
- 3:46more difference, right? And that
- 3:48difference comes in feature selection
- 3:51or feature sampling, which you also
- 3:54call column sampling. So, I guess you
- 3:58know the basic fundamentals of bagging:
- 4:00what do you do? You do sampling.
- 4:03Meaning, you give the base models a
- 4:05subset of data, and how is that subset
- 4:07created? Using sampling. Now, there are
- 4:10two types of sampling. Row sampling and
- 4:13feature or column sampling. Okay? So
- 4:16you're giving your base model a subset
- 4:19of the data using row sampling plus
- 4:22feature sampling. Okay? Now, the
- 4:24feature sampling that happens when you
- 4:27give different columns to your base
- 4:30estimators is done differently in both
- 4:33bagging and random forest. Now, this
- 4:35point is a little tricky. Let's
- 4:37understand it clearly. So let me give
- 4:39you an example. So, let's take two
- 4:41parallel examples. Let's say this is an
- 4:44example of bagging. This is an example
- 4:48of bagging. And the example here is an
- 4:52example of random forest. This is
- 4:54random forest. Okay? Here I have a data
- 4:57set. Here I have a data set. And all
- 5:00these base models are decision trees.
- 5:03Here and here. Okay? Now, consider the
- 5:07difference. Suppose there are five
- 5:10columns in your data set. And you
- 5:15decide to use column sampling to assign
- 5:19only two columns to each base estimator
- 5:23. Right? So, what happens now is that
- 5:27each decision tree will be based on
- 5:30only two columns. Right? So, what
- 5:33happens now is that you decide right at
- 5:36the beginning, when you're creating the
- 5:38data for this decision tree, that
- 5:41you'll randomly select two columns from
- 5:43your five columns. Let's say those two
- 5:46columns are one and three, and you're
- 5:48assigning all of them to each. Let's
- 5:50say you're assigning all of them to
- 5:52each. So, what happens now? This
- 5:53decision tree will only use these two
- 5:56columns. No columns other than one and
- 5:59three will be used. Similarly, suppose
- 6:01we were creating the data for this,
- 6:03since we only need two columns again.
- 6:05So, let's say those two columns are
- 6:07four and five. So, now the entire tree
- 6:09will be formed from these two columns.
- 6:11Similarly, let's say one and four are
- 6:13selected. So, now the entire tree will
- 6:15be formed from just one and four. This
- 6:17means that when this tree is formed,
- 6:20every split will be from either column
- 6:23number one or column number three. I
- 6:27hope you understand. This is column
- 6:29sampling in bagging. This is column
- 6:33sampling in bagging. Now, when you do
- 6:36the same thing in Random Forest, see
- 6:37what difference it makes. Suppose here
- 6:40too you decided to do column sampling
- 6:42and assign two columns to each decision
- 6:45tree. So, when the tree is formed,
- 6:49every time a node is about to be
- 6:52created in your decision tree, sampling
- 6:55occurs. For example, when the first
- 6:58node was being formed, let's say two
- 7:00columns were selected: two and three.
- 7:02So, this first node will either be
- 7:04created from two or three. Right? Then,
- 7:07when the next node was to be created,
- 7:09sampling occurred again. Sampling
- 7:11happened again between 1, 2, 3, 4, 5.
- 7:13And here, the samples came back as four
- 7:16and five. So, this next node will be
- 7:18formed from either four or five.
- 7:19Similarly, when this node was formed,
- 7:21two and one were selected. So, now this
- 7:24node will be formed from either two or
- 7:26one. In short, you're introducing more
- 7:29randomness here. And in strict terms,
- 7:33the column sampling here happens at the
- 7:36tree level. This is tree-level column
- 7:39sampling. This means that before the
- 7:42tree is created, it will be finalized
- 7:44which two columns will be used to
- 7:46create the tree. There will be no other
- 7:49columns. Where as, this is node-level
- 7:53sampling. This means that two columns
- 7:56will be randomly selected every time a
- 7:58node is created. If you have decided on
- 8:01two columns as the maximum features.
- 8:04Okay? Okay? So, this is a big
- 8:06difference, which is why random forest
- 8:09often performs better than bagging most
- 8:12of the time. Okay? Because here you're
- 8:15introducing extra randomness using this
- 8:18node-level column sampling. Okay? I can
- 8:21actually prove this to you. This isn't
- 8:23just raw theory. I've created a
- 8:25notebook here and I'll walk you through
- 8:28it to show that what I just told you is
- 8:30actually true. Okay? So we're working
- 8:33with a data set. This data set has five
- 8:36input columns and one output column.
- 8:39Okay? Look at this. Now what I'm doing
- 8:42is I'm creating a bagging classifier.
- 8:44And in that bagging classifier, I'm
- 8:46already telling it that I want to do
- 8:48column sampling. Okay? I want to do
- 8:50column sampling. And how many columns
- 8:52do I want to take? I want to take two
- 8:54columns. So, I'm going to randomly
- 8:56select two columns from these five.
- 8:59Okay? And now this will happen at the
- 9:00tree level. Meaning, the same two
- 9:02columns will appear repeatedly in our
- 9:04tree. So I created this object. The
- 9:07model has been trained, and now when I
- 9:10run the plot tree, I see my graph. And
- 9:13look, guys, if you check any node,
- 9:16you'll see x1 here. x0 there. So, these
- 9:19two columns are selected: zero and one.
- 9:21There's x1 here too. There's x1 here
- 9:22too. There's x1 here too. There's x1
- 9:24here too. There's x1. There's x0.
- 9:27There's x1. There's x0. You'd see that
- 9:29in the entire decision tree, when
- 9:32splitting occurs at any node, it's
- 9:34either at zero or at one. No other
- 9:37column is touched. This proves that the
- 9:40column sampling in bagging is
- 9:43tree-level. Meaning, even before the
- 9:45tree is created, it's finalized that
- 9:46the tree that will be built will be
- 9:48made up of these two columns. We won't
- 9:51touch any other columns. Okay? Now, let
- 9:54me show you the same example in random
- 9:56forest. Here too, we decided that the
- 9:57maximum number of features would be two
- 9:59. We trained the model and plotted it.
- 10:06But look here, the splitting happened
- 10:08first at x1. Then again at x1, and
- 10:10again at x3. So, two columns have
- 10:12already been entered. Right? Then x2
- 10:14has also been entered. So, three
- 10:15columns have already been entered. x0
- 10:16has also been entered, meaning four
- 10:18columns have already been entered, and
- 10:19x4 has also been entered. Meaning, all
- 10:21five columns have been entered. So, you
- 10:23can see that node-level splitting is
- 10:24happening here. So, when this node is
- 10:26being formed, you first decide which
- 10:28two columns to form this node on. That
- 10:31turns out to be x1. Right? Then you
- 10:33randomly select here again. Then you
- 10:35randomly select here again. Here too,
- 10:37you are doing node-level sampling at
- 10:39every node. Because of this, your
- 10:42random forest has a little more
- 10:44randomness, and with more randomness,
- 10:47your on-the-fly base models perform
- 10:51better. We've already studied this
- 10:53theory, right? The more random it is
- 10:55and the accuracy is just over 50%, the
- 10:58better your singular model performs.
- 11:02Right? The more randomness, the greater
- 11:04the difference between the base models,
- 11:06the better the performance. And that's
- 11:08why most of the time, random forest
- 11:11beats bag and classifier. Right? But
- 11:14you should try both. Right? I hope I've
- 11:16been able to explain it all to you.
- 11:18This is a good interview question. Many
- 11:20times I've heard students asked this
- 11:22question in interviews: What's the
- 11:24difference between bagging and random
- 11:26forest? Tell me. And most of the time,
- 11:28students can only tell one difference:
- 11:30in bagging, you can use multiple
- 11:32algorithms. In random forest, you can
- 11:34only use decision trees. Then people
- 11:37ask the counter question: If I create a
- 11:39bagging using a decision tree, is it a
- 11:42random forest? And this is where many
- 11:44people get stuck and say, "Yes, but the
- 11:46answer is wrong." There's still a
- 11:47difference between the two. And what is
- 11:49that difference? How is column sampling
- 11:51done? In bagging, it's at the tree
- 11:53level, and in random forest, it's at
- 11:55the node level. I hope you understand
- 11:57this concept. That's it. Thanks for
- 12:00watching. Bye.
About this transcript
This page contains the full transcript of Bagging Vs Random Forest | What is the difference between Bagging and Random Forest | Very Important by CampusX, generated from the public captions YouTube serves with the video. The transcript has 1,795 words across 271 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.