4.2.9 An Introduction to Trees - Video 5: Random Forests — Transcript

Full transcript

1. 0:03in this video we'll introduce a method
2. 0:06that is similar to cart called random
3. 0:09forests this method was designed to
4. 0:12improve the prediction accuracy of cart
5. 0:14and works by building a large number of
6. 0:17cart trees unfortunately this makes the
7. 0:20method less interpretable than cart so
8. 0:23often you need to decide if you value
9. 0:25the interpretability
10. 0:26or the increase in accuracy more to make
11. 0:30a prediction for a new observation each
12. 0:33tree in the forest votes on the outcome
13. 0:36and we pick the outcome that receives
14. 0:38the majority of the votes so how does
15. 0:42random forests build many cart trees we
16. 0:45can't just run cart multiple times
17. 0:47because it would create the same tree
18. 0:49every time to prevent this random
19. 0:53forests only allows each tree to split
20. 0:56on a random subset of the available
21. 0:58independent variables and each tree is
22. 1:02built from what we call a bagged or
23. 1:04bootstrapped sample of the data this
24. 1:07just means that the data used as the
25. 1:09training data for each tree is selected
26. 1:11randomly with replacement let's look at
27. 1:14an example suppose we have five data
28. 1:17points in our training set we'll call
29. 1:19them one two three four and five for the
30. 1:23first tree will randomly pick five data
31. 1:26points randomly sampled with replacement
32. 1:29so the data could be two four five two
33. 1:35and one each time we pick one of the
34. 1:38five data points regardless of whether
35. 1:40or not it's been selected already the
36. 1:43these would be the five data points we
37. 1:45would use when constructing the first
38. 1:47cart tree then we repeat this process
39. 1:50for the second tree this time the data
40. 1:53set might be three five one five and two
41. 1:57and we would use this data when building
42. 1:59the second cart tree then we would
43. 2:02repeat this process for each additional
44. 2:04tree we want to create so since each
45. 2:08tree sees a different set of variables
46. 2:10and a different set of data we get
47. 2:13what's called a forest of many different
48. 2:15trees
49. 2:17just like cart branda forests has some
50. 2:21parameter values that need to be
51. 2:22selected the first is the minimum number
52. 2:25of observations in a subset or the min
53. 2:28bucket parameter from cart when we
54. 2:31create a random forest in our this will
55. 2:33be called node size a smaller value of
56. 2:37node size which leads to bigger trees
57. 2:39may take longer in our random forests is
58. 2:43much more computationally intensive than
59. 2:46cart the second parameter is the number
60. 2:49of trees to build which is called entry
61. 2:52in our this should not be set to small
62. 2:56but the larger it is the longer it will
63. 2:58take a couple hundred trees is typically
64. 3:01plenty a nice thing about random forests
65. 3:04is that it's not as sensitive to the
66. 3:06parameter values as car is in the next
67. 3:09video we'll talk about a nice way to
68. 3:11pick the cart parameter for random
69. 3:14forests as long as the selection is
70. 3:16reasonable it's okay let's switch to our
71. 3:19and create a random forest model to
72. 3:21predict the decisions of justice Stevens
73. 3:26inner our consul let's start by
74. 3:29installing and loading the package
75. 3:31random forests we first need to install
76. 3:34the package using the install dot
77. 3:37packages function for the package random
78. 3:41forests you should see a few lines run
79. 3:45in your our console and then when you're
80. 3:47back to the blinking cursor load the
81. 3:50package at the library command now we're
82. 3:56ready to build our random forest model
83. 3:58we'll call it
84. 4:00Stephens forest and use the random
85. 4:03forest function first giving our
86. 4:06dependent variable reverse followed by a
87. 4:09tilde sign and then our independent
88. 4:11variables separated by plus signs
89. 4:13circuit issue petitioner respondent
90. 4:20lower court an unconstitutional will use
91. 4:27the data set train
92. 4:30for random forests we need to give two
93. 4:33additional arguments these are nodes
94. 4:35size also known as min bucket for cart
95. 4:39and we'll set this equal to 25 the same
96. 4:42value we used for our cart model and
97. 4:44then we need to set the parameter and
98. 4:46tree this is the number of trees to
99. 4:48build and we'll build 200 trees here
100. 4:51then hit enter you should see an
101. 4:55interesting warning message here in cart
102. 4:58we added the argument method equals
103. 5:00class so that it was clear that we're
104. 5:02doing a classification problem as I
105. 5:05mentioned earlier trees can also be used
106. 5:07for regression problems which you'll see
107. 5:09in the recitation the random forest
108. 5:12function does not have a method argument
109. 5:14so we'll may want to do a classification
110. 5:16problem we need to make sure our outcome
111. 5:19is a factor let's convert the variable
112. 5:22reverse to a factor variable in both our
113. 5:25training and our testing sets we do this
114. 5:28by typing the name of the variable we
115. 5:31want to convert in our case train
116. 5:33reverse and then type a s dot factor and
117. 5:37then in parentheses the variable name
118. 5:40train reverse and just repeat this for
119. 5:44the test set as well test reversed
120. 5:47equals a s stop factor test dollar sign
121. 5:52reverse now let's try creating a random
122. 5:56forest again just use the up arrow to
123. 5:59get back to the random forest line and
124. 6:01hit enter we didn't get a warning
125. 6:03message this time so our models ready to
126. 6:06make predictions let's compute
127. 6:09predictions on our test set we'll call
128. 6:11our predictions predict forest and use
129. 6:16the predict function to make predictions
130. 6:18using our model Stephens forests and the
131. 6:23new data set test let's look at the
132. 6:28confusion matrix to compute our accuracy
133. 6:31we'll use the table function and first
134. 6:34give the true outcome test reverse and
135. 6:37then our predictions predict forest
136. 6:42our accuracy here is 40 plus 74 divided
137. 6:48by 40 plus 37 plus 19 plus 74 so the
138. 6:56accuracy of our random forest model is
139. 6:59about 67% recall that our logistic
140. 7:02regression model had an accuracy of
141. 7:0566.5% and our cart model had an accuracy
142. 7:08of 65.9% so a random forest model
143. 7:12improved our accuracy a little bit over
144. 7:15Kart sometimes you'll see a smaller
145. 7:18improvement in accuracy and sometimes
146. 7:20you'll see that random forests can
147. 7:22significantly improve an accuracy over
148. 7:24Kart we'll see this a lot in the
149. 7:27recitation and the homework assignments
150. 7:29keep in mind that random forests has a
151. 7:32random component you may have gotten a
152. 7:34different confusion matrix than me
153. 7:36because there's a random component to
154. 7:39this method