YT YouTube2Text

YouTube2Text → Transcripts → 4.2.7 An Introduction to Trees - Video 4: CART in R

4.2.7 An Introduction to Trees - Video 4: CART in R — Transcript

by MIT OpenCourseWare · 1,565 words · 290 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:04In this video, we'll see how to build a
0:06CART model in R.
0:08Let's start by reading in the data file
0:10stevens.csv.
0:13We'll call our data frame stevens
0:16and use the read .csv function to read
0:20in the data file stevens
0:24.csv.
0:26Remember to navigate to the directory on
0:28your computer containing the file
0:30stevens.csv
0:31first.
0:33Now, let's take a look at our data using
0:35the str function.
0:40We have 566
0:42observations or Supreme Court cases and
0:46nine different variables.
0:48Docket is just a unique identifier for
0:50each case and term is the year of the
0:53case.
0:54Then, we have our six independent
0:56variables. The circuit court of origin,
1:00the issue area of the case,
1:02the type of petitioner,
1:04the type of respondent,
1:06the lower court direction,
1:09and whether or not the petitioner argued
1:11that a law or practice was
1:13unconstitutional.
1:15The last variable is our dependent
1:17variable, whether or not Justice Stevens
1:20voted to reverse the case.
1:22One for reverse and zero for affirm.
1:26Now, before building models, we need to
1:28split our data into a training set and a
1:31testing set.
1:32We'll do this using the sample.split
1:35function like we did last week for
1:37logistic regression.
1:39First, we need to load the package
1:41caTools
1:42with library
1:45caTools.
1:49Now, so that we all get the same split,
1:52we need to set the seed. Remember that
1:55this can be any number as long as we all
1:57use the same number.
1:59Let's set the seed
2:01to 3000.
2:06Now, let's create our split. We'll call
2:08it spl
2:11and we'll use the sample.split
2:13function
2:15where the first argument needs to be our
2:17outcome variable
2:19stevens
2:22reverse
2:24and then the second argument is the
2:26split ratio or the percentage of data
2:29that we want to put in the training set.
2:31In this case, we'll put 70% of the data
2:34in the training set.
2:37Now, let's create our training and
2:39testing sets using the subset function.
2:43We'll call our training set train
2:46and we'll take a subset of stevens
2:52only taking the observations for which
2:54spl is equal to true.
2:58We'll call our testing set test and here
3:01take a subset of stevens
3:04but this time taking the observations
3:06for which spl is equal to false.
3:12Now, we're ready to build our CART
3:13model.
3:14First, we need to install and load the
3:17rpart package and the rpart.plotting
3:20package.
3:21Remember that to install a new package,
3:24we use the install.packages
3:27function
3:29and then in parentheses and quotes, give
3:31the name of the package we want to
3:33install. In this case, rpart.
3:36After you hit enter, a CRAN mirror
3:38should pop up asking you to pick a
3:41location near you.
3:43Go ahead and pick the appropriate
3:45location. In my case, I'll pick
3:47Pennsylvania in the United States and
3:50hit okay.
3:52You should see some lines running your R
3:54console and then when you're back to the
3:56blinking cursor, load the package with
4:00library
4:01rpart.
4:04Now, let's install the package
4:12rpart .plot.
4:19Again, some lines should run in your R
4:20console and when you're back to the
4:22blinking cursor, load the package with
4:25library
4:27rpart.plot.
4:31Now, we can create our CART model using
4:33the rpart function.
4:35We'll call our model stevens.tree
4:39and we'll use the rpart function where
4:41the first argument is the same as if we
4:44were building a linear or logistic
4:45regression model.
4:47We give our dependent variable, in our
4:49case reverse,
4:51followed by a tilde sign and then the
4:53independent variables separated by plus
4:55signs. So, circuit plus
4:59issue plus
5:02petitioner
5:03plus
5:04respondent
5:06plus
5:08lower court
5:10plus unconstitutional.
5:13We also need to give our data set that
5:15should be used to build our model, which
5:17in our case is train.
5:20Now, we'll give two additional arguments
5:22here. The first one is method equals
5:25class.
5:27This tells rpart to build a
5:29classification tree instead of a
5:31regression tree.
5:32You'll see how we can create regression
5:34trees in recitation.
5:37The last argument we'll give is
5:39minbucket equals 25.
5:42This limits the tree so that it doesn't
5:45overfit to our training set.
5:47We selected a value of 25, but we could
5:49pick a smaller or larger value.
5:52We'll see another way to limit the tree
5:54later in this lecture.
5:57Now, let's plot our tree using the prp
6:00function where the only argument is the
6:02name of our model, stevens.tree.
6:07You should see the tree pop up in the
6:09graphics window.
6:11The first split of our tree is whether
6:13or not the lower court decision is
6:15liberal.
6:17If it is, then we move to the left in
6:19the tree and we check the respondent.
6:22If the respondent is a criminal
6:24defendant, injured person, politician,
6:28state, or the United States, we predict
6:31zero or affirm.
6:33You can see here that the prp function
6:36abbreviates the values of the
6:37independent variables.
6:39If you're not sure what the
6:41abbreviations are, you could create a
6:43table of the variable to see all of the
6:45possible values.
6:47prp will select the abbreviation so that
6:50they're uniquely identifiable.
6:52So, if you made a table, you could see
6:55that cri stands for criminal defendant,
6:58inj stands for injured person, etc.
7:02So, now moving on in our tree, if the
7:05respondent is not one of these types, we
7:08move on to the next split and we check
7:10the petitioner.
7:12If the petitioner is a city, employee,
7:15employer, government official, or
7:17politician, then we predict zero or
7:20affirm.
7:22If not, then we check the circuit court
7:24of origin.
7:25If it's the 10th, first, third, fourth,
7:30DC, or federal court, then we predict
7:33zero. Otherwise, we predict one or
7:36reverse.
7:38We can repeat the same process on the
7:40other side of the tree if the lower
7:42court decision is not liberal.
7:45Comparing this to a logistic regression
7:47model, we can see that it's very
7:49interpretable.
7:51A CART tree is a series of decision
7:53rules which can easily be explained.
7:57Now, let's see how well our CART model
7:59does at making predictions for the test
8:01set.
8:02So, back in our R console, we'll call
8:05our predictions predict cart
8:09and we'll use the predict function
8:12where the first argument is the name of
8:14our model, stevens
8:18tree.
8:20The second argument is the new data we
8:22want to make predictions for,
8:24test
8:26and we'll add a third argument here,
8:28which is type equals class.
8:32We need to give this argument when
8:34making predictions for our CART model if
8:36we want the majority class predictions.
8:40This is like using a threshold of .5.
8:43We'll see in a few minutes how we can
8:44leave this argument out and still get
8:46probabilities from our CART model.
8:49Now, let's compute the accuracy of our
8:51model by building a confusion matrix.
8:54So, we'll use the table function
8:57and first give the true outcome values
9:00test
9:01reverse
9:03and then our predictions predict cart.
9:09To compute the accuracy, we need to add
9:11up the observations we got correct, 41 +
9:1471 divided by the total number of
9:17observations in the table or the total
9:19number of observations in our test set.
9:23So, the accuracy of our CART model is
9:25.659.
9:27If you were to build a logistic
9:29regression model, you would get an
9:30accuracy of .665.
9:33And a baseline model that always
9:35predicts reverse, the most common
9:37outcome, has an accuracy of .547.
9:41So, our CART model significantly beats
9:43the baseline and is competitive with
9:46logistic regression.
9:47It's also much more interpretable than a
9:50logistic regression model would be.
9:53Lastly, to evaluate our model, let's
9:55generate an ROC curve for our CART model
9:58using the ROCR package.
10:01First, we need to load the package with
10:03the library function.
10:06And then, we need to generate our
10:08predictions again, this time without the
10:10type equals class argument. We'll call
10:13them predict ROC,
10:16and we'll use the predict function,
10:18giving just as the two arguments Stevens
10:21tree
10:22and new data equals test.
10:25Let's take a look at what this looks
10:27like by just typing predict ROC and
10:30hitting enter.
10:33For each observation in the test set, it
10:35gives two numbers, which can be thought
10:38of as the probability of outcome zero
10:40and the probability of outcome one.
10:43More concretely, each test set
10:45observation is classified into a subset
10:48or bucket of our CART tree.
10:50These numbers give the percentage of
10:52training set data in that subset with
10:55outcome zero and the percentage of data
10:58in the training set in that subset with
11:00outcome one.
11:02We'll use the second column as our
11:04probabilities to generate an ROC curve.
11:07So, just like we did last week for
11:09logistic regression, we'll start by
11:11using the prediction function. We'll
11:13call the output pred,
11:16and then use prediction,
11:18where the first argument is the second
11:20column of predict ROC, which we can
11:22access with square brackets,
11:25and the second argument is the true
11:27outcome values, test reverse.
11:30Now, we need to use the performance
11:32function,
11:35where the first argument is the outcome
11:38of the prediction function,
11:39and then the next two arguments are true
11:42positive rate and false positive rate,
11:45what we want on the X and Y axes of our
11:47ROC curve.
11:49Now, we can just plot our ROC curve by
11:51typing plot p e r f.
11:56If you switch back to your graphics
11:57window, you should see the ROC curve for
12:00our model.
12:01In the next quick question, we'll ask
12:04you to compute the test set AUC of this
12:06model.

About this transcript

This page contains the full transcript of 4.2.7 An Introduction to Trees - Video 4: CART in R by MIT OpenCourseWare, generated from the public captions YouTube serves with the video. The transcript has 1,565 words across 290 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.