YouTube2Text

Bagging Vs Random Forest | What is the difference between Bagging and Random Forest | Very Important — Transcript

by CampusX · 1,795 words · 271 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Hello guys, welcome to my YouTube
  2. 0:02channel. We've been studying random
  3. 0:04forests and covered a couple of topics.
  4. 0:07Now, we're going to have an important
  5. 0:10discussion about the difference between
  6. 0:12bagging, which we just studied, and
  7. 0:14random forests, which we're currently
  8. 0:16studying. So, I've already told you
  9. 0:19that random forests are internally an
  10. 0:22implementation of bagging. But there
  11. 0:25are some important differences between
  12. 0:27the two. In fact, this is a very good
  13. 0:29interview question: what exactly is the
  14. 0:31difference between random forests and
  15. 0:33bagging? Okay? So, that's what we're
  16. 0:35going to cover in this video. I'll tell
  17. 0:37you two differences. Okay? So, the
  18. 0:40first difference is pretty
  19. 0:41straightforward. You probably already
  20. 0:44know what you can do with bagging. You
  21. 0:48can use different algorithms. So, if
  22. 0:51you want, you can create a bagging
  23. 0:54ensemble using decision trees, but you
  24. 0:57can also create a bagging ensemble
  25. 1:00using kNN. This means that all your
  26. 1:04base estimators can be decision trees,
  27. 1:09kNN, or even SVM. Right? In fact, you
  28. 1:13can try other algorithms, but these
  29. 1:16three give you the best results. So, in
  30. 1:19short, bagging is a general technique
  31. 1:22in which you can use any algorithm as
  32. 1:25the base algorithm. Right? But the base
  33. 1:29algorithm in a random forest is only
  34. 1:32and only decision trees. This means
  35. 1:35that a random forest is created by
  36. 1:37combining decision trees. You can't use
  37. 1:39kNN, SVM, or any other algorithm. Right
  38. 1:43? I can give you a logical explanation
  39. 1:47for this. And that is, if you go to
  40. 1:52ScikitLearn and open the bagging
  41. 1:58classifier, you'll see a parameter
  42. 2:03called base estimator. Now, what this
  43. 2:08base estimator is, is that you can
  44. 2:10select different algorithms. It says
  45. 2:13that if its value is NULL, the base
  46. 2:15estimator is a decision tree classifier
  47. 2:18. Or whatever you want to specify, if
  48. 2:24you search for "SKLearn Random Forest
  49. 2:32Classifier," you won't see any such
  50. 2:36parameter. "C" is the parameter for
  51. 2:39selecting the number of estimators, but
  52. 2:41it's not the parameter for selecting
  53. 2:43the base estimator. So, this actually
  54. 2:46shows the first difference between
  55. 2:48random forest and bagging. I'm
  56. 2:50summarizing it once. In bagging, your
  57. 2:53base models can be created by any
  58. 2:55algorithm. In random forest, there's a
  59. 2:58restriction that the base model can
  60. 3:00only be a decision tree. Okay? Now,
  61. 3:04when I first read this, a question came
  62. 3:07to my mind: what if I create a bagging
  63. 3:10classifier and call all its base models
  64. 3:14decision trees or keep them as decision
  65. 3:17trees? So, is this whole thing a random
  66. 3:21forest now? Is this whole thing a
  67. 3:25random forest now? The answer is no.
  68. 3:31There's still a big difference. Right?
  69. 3:34I guess you're getting the point: if
  70. 3:37you're creating a bagging ensemble or
  71. 3:40ensemble using decision trees, it's not
  72. 3:43exactly a random forest. There's one
  73. 3:46more difference, right? And that
  74. 3:48difference comes in feature selection
  75. 3:51or feature sampling, which you also
  76. 3:54call column sampling. So, I guess you
  77. 3:58know the basic fundamentals of bagging:
  78. 4:00what do you do? You do sampling.
  79. 4:03Meaning, you give the base models a
  80. 4:05subset of data, and how is that subset
  81. 4:07created? Using sampling. Now, there are
  82. 4:10two types of sampling. Row sampling and
  83. 4:13feature or column sampling. Okay? So
  84. 4:16you're giving your base model a subset
  85. 4:19of the data using row sampling plus
  86. 4:22feature sampling. Okay? Now, the
  87. 4:24feature sampling that happens when you
  88. 4:27give different columns to your base
  89. 4:30estimators is done differently in both
  90. 4:33bagging and random forest. Now, this
  91. 4:35point is a little tricky. Let's
  92. 4:37understand it clearly. So let me give
  93. 4:39you an example. So, let's take two
  94. 4:41parallel examples. Let's say this is an
  95. 4:44example of bagging. This is an example
  96. 4:48of bagging. And the example here is an
  97. 4:52example of random forest. This is
  98. 4:54random forest. Okay? Here I have a data
  99. 4:57set. Here I have a data set. And all
  100. 5:00these base models are decision trees.
  101. 5:03Here and here. Okay? Now, consider the
  102. 5:07difference. Suppose there are five
  103. 5:10columns in your data set. And you
  104. 5:15decide to use column sampling to assign
  105. 5:19only two columns to each base estimator
  106. 5:23. Right? So, what happens now is that
  107. 5:27each decision tree will be based on
  108. 5:30only two columns. Right? So, what
  109. 5:33happens now is that you decide right at
  110. 5:36the beginning, when you're creating the
  111. 5:38data for this decision tree, that
  112. 5:41you'll randomly select two columns from
  113. 5:43your five columns. Let's say those two
  114. 5:46columns are one and three, and you're
  115. 5:48assigning all of them to each. Let's
  116. 5:50say you're assigning all of them to
  117. 5:52each. So, what happens now? This
  118. 5:53decision tree will only use these two
  119. 5:56columns. No columns other than one and
  120. 5:59three will be used. Similarly, suppose
  121. 6:01we were creating the data for this,
  122. 6:03since we only need two columns again.
  123. 6:05So, let's say those two columns are
  124. 6:07four and five. So, now the entire tree
  125. 6:09will be formed from these two columns.
  126. 6:11Similarly, let's say one and four are
  127. 6:13selected. So, now the entire tree will
  128. 6:15be formed from just one and four. This
  129. 6:17means that when this tree is formed,
  130. 6:20every split will be from either column
  131. 6:23number one or column number three. I
  132. 6:27hope you understand. This is column
  133. 6:29sampling in bagging. This is column
  134. 6:33sampling in bagging. Now, when you do
  135. 6:36the same thing in Random Forest, see
  136. 6:37what difference it makes. Suppose here
  137. 6:40too you decided to do column sampling
  138. 6:42and assign two columns to each decision
  139. 6:45tree. So, when the tree is formed,
  140. 6:49every time a node is about to be
  141. 6:52created in your decision tree, sampling
  142. 6:55occurs. For example, when the first
  143. 6:58node was being formed, let's say two
  144. 7:00columns were selected: two and three.
  145. 7:02So, this first node will either be
  146. 7:04created from two or three. Right? Then,
  147. 7:07when the next node was to be created,
  148. 7:09sampling occurred again. Sampling
  149. 7:11happened again between 1, 2, 3, 4, 5.
  150. 7:13And here, the samples came back as four
  151. 7:16and five. So, this next node will be
  152. 7:18formed from either four or five.
  153. 7:19Similarly, when this node was formed,
  154. 7:21two and one were selected. So, now this
  155. 7:24node will be formed from either two or
  156. 7:26one. In short, you're introducing more
  157. 7:29randomness here. And in strict terms,
  158. 7:33the column sampling here happens at the
  159. 7:36tree level. This is tree-level column
  160. 7:39sampling. This means that before the
  161. 7:42tree is created, it will be finalized
  162. 7:44which two columns will be used to
  163. 7:46create the tree. There will be no other
  164. 7:49columns. Where as, this is node-level
  165. 7:53sampling. This means that two columns
  166. 7:56will be randomly selected every time a
  167. 7:58node is created. If you have decided on
  168. 8:01two columns as the maximum features.
  169. 8:04Okay? Okay? So, this is a big
  170. 8:06difference, which is why random forest
  171. 8:09often performs better than bagging most
  172. 8:12of the time. Okay? Because here you're
  173. 8:15introducing extra randomness using this
  174. 8:18node-level column sampling. Okay? I can
  175. 8:21actually prove this to you. This isn't
  176. 8:23just raw theory. I've created a
  177. 8:25notebook here and I'll walk you through
  178. 8:28it to show that what I just told you is
  179. 8:30actually true. Okay? So we're working
  180. 8:33with a data set. This data set has five
  181. 8:36input columns and one output column.
  182. 8:39Okay? Look at this. Now what I'm doing
  183. 8:42is I'm creating a bagging classifier.
  184. 8:44And in that bagging classifier, I'm
  185. 8:46already telling it that I want to do
  186. 8:48column sampling. Okay? I want to do
  187. 8:50column sampling. And how many columns
  188. 8:52do I want to take? I want to take two
  189. 8:54columns. So, I'm going to randomly
  190. 8:56select two columns from these five.
  191. 8:59Okay? And now this will happen at the
  192. 9:00tree level. Meaning, the same two
  193. 9:02columns will appear repeatedly in our
  194. 9:04tree. So I created this object. The
  195. 9:07model has been trained, and now when I
  196. 9:10run the plot tree, I see my graph. And
  197. 9:13look, guys, if you check any node,
  198. 9:16you'll see x1 here. x0 there. So, these
  199. 9:19two columns are selected: zero and one.
  200. 9:21There's x1 here too. There's x1 here
  201. 9:22too. There's x1 here too. There's x1
  202. 9:24here too. There's x1. There's x0.
  203. 9:27There's x1. There's x0. You'd see that
  204. 9:29in the entire decision tree, when
  205. 9:32splitting occurs at any node, it's
  206. 9:34either at zero or at one. No other
  207. 9:37column is touched. This proves that the
  208. 9:40column sampling in bagging is
  209. 9:43tree-level. Meaning, even before the
  210. 9:45tree is created, it's finalized that
  211. 9:46the tree that will be built will be
  212. 9:48made up of these two columns. We won't
  213. 9:51touch any other columns. Okay? Now, let
  214. 9:54me show you the same example in random
  215. 9:56forest. Here too, we decided that the
  216. 9:57maximum number of features would be two
  217. 9:59. We trained the model and plotted it.
  218. 10:06But look here, the splitting happened
  219. 10:08first at x1. Then again at x1, and
  220. 10:10again at x3. So, two columns have
  221. 10:12already been entered. Right? Then x2
  222. 10:14has also been entered. So, three
  223. 10:15columns have already been entered. x0
  224. 10:16has also been entered, meaning four
  225. 10:18columns have already been entered, and
  226. 10:19x4 has also been entered. Meaning, all
  227. 10:21five columns have been entered. So, you
  228. 10:23can see that node-level splitting is
  229. 10:24happening here. So, when this node is
  230. 10:26being formed, you first decide which
  231. 10:28two columns to form this node on. That
  232. 10:31turns out to be x1. Right? Then you
  233. 10:33randomly select here again. Then you
  234. 10:35randomly select here again. Here too,
  235. 10:37you are doing node-level sampling at
  236. 10:39every node. Because of this, your
  237. 10:42random forest has a little more
  238. 10:44randomness, and with more randomness,
  239. 10:47your on-the-fly base models perform
  240. 10:51better. We've already studied this
  241. 10:53theory, right? The more random it is
  242. 10:55and the accuracy is just over 50%, the
  243. 10:58better your singular model performs.
  244. 11:02Right? The more randomness, the greater
  245. 11:04the difference between the base models,
  246. 11:06the better the performance. And that's
  247. 11:08why most of the time, random forest
  248. 11:11beats bag and classifier. Right? But
  249. 11:14you should try both. Right? I hope I've
  250. 11:16been able to explain it all to you.
  251. 11:18This is a good interview question. Many
  252. 11:20times I've heard students asked this
  253. 11:22question in interviews: What's the
  254. 11:24difference between bagging and random
  255. 11:26forest? Tell me. And most of the time,
  256. 11:28students can only tell one difference:
  257. 11:30in bagging, you can use multiple
  258. 11:32algorithms. In random forest, you can
  259. 11:34only use decision trees. Then people
  260. 11:37ask the counter question: If I create a
  261. 11:39bagging using a decision tree, is it a
  262. 11:42random forest? And this is where many
  263. 11:44people get stuck and say, "Yes, but the
  264. 11:46answer is wrong." There's still a
  265. 11:47difference between the two. And what is
  266. 11:49that difference? How is column sampling
  267. 11:51done? In bagging, it's at the tree
  268. 11:53level, and in random forest, it's at
  269. 11:55the node level. I hope you understand
  270. 11:57this concept. That's it. Thanks for
  271. 12:00watching. Bye.

About this transcript

This page contains the full transcript of Bagging Vs Random Forest | What is the difference between Bagging and Random Forest | Very Important by CampusX, generated from the public captions YouTube serves with the video. The transcript has 1,795 words across 271 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.