4.2.5 An Introduction to Trees - Video 3: Splitting and Predictions — Transcript

Full transcript

1. 0:04In the previous video, we generated a
2. 0:06CART tree with three splits.
3. 0:09But why not two, or four, or even five?
4. 0:13There are different ways to control how
5. 0:15many splits are generated.
6. 0:18One way is by setting a lower bound for
7. 0:20the number of data points in each
8. 0:22subset.
9. 0:23In R, this is called the minbucket
10. 0:26parameter for the minimum number of
11. 0:28observations in each bucket or subset.
12. 0:32The smaller minbucket is, the more
13. 0:34splits will be generated.
14. 0:36But if it's too small, overfitting will
15. 0:39occur.
16. 0:40This means that CART will fit the
17. 0:42training set almost perfectly.
18. 0:45But this is bad because then the model
19. 0:47will probably not perform well on test
20. 0:49set data or new data.
21. 0:52On the other hand, if the minbucket
22. 0:54parameter is too large, the model will
23. 0:56be too simple and the accuracy will be
24. 0:59poor.
25. 1:00Later in the lecture, we'll learn about
26. 1:02a nice method for selecting the stopping
27. 1:04parameter.
28. 1:07In each subset of a CART tree, we have a
29. 1:10bucket of observations, which may
30. 1:12contain both possible outcomes.
31. 1:15In the small example we showed in the
32. 1:17previous video, we classified each
33. 1:20subset as either red or gray, depending
34. 1:22on the majority in that subset.
35. 1:25In the Supreme Court case, we'll be
36. 1:27classifying observations as either
37. 1:29affirm or reverse.
38. 1:32Instead of just taking the majority
39. 1:34outcome to be the prediction,
40. 1:36we can compute the percentage of data in
41. 1:38a subset of each type of outcome.
42. 1:42As an example, if we have a subset with
43. 1:4410 affirms and two reverses,
44. 1:48then 87% of the data is affirm.
45. 1:52Then, just like in logistic regression,
46. 1:55we can use a threshold value to obtain
47. 1:58our prediction.
48. 1:59For this example, we would predict a
49. 2:02firm with a threshold of .5 since the
50. 2:05majority is a firm.
51. 2:07But, if we increase that threshold to
52. 2:09.9, we would predict reverse for this
53. 2:12example.
54. 2:15Then, by varying the threshold value, we
55. 2:18can compute an ROC curve and compute an
56. 2:21AUC value to evaluate our model.
57. 2:25In the next video, we'll build a CART
58. 2:27tree in R to predict the decisions of
59. 2:29Justice Stevens and evaluate our model
60. 2:32using an ROC curve.