YT YouTube2Text

YouTube2Text → Transcripts → YouTube transcript (fTfMdCQJz4s)

YouTube transcript (fTfMdCQJz4s) — Transcript

3,230 words · 456 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00hello and welcome to the next video in
0:02my series on basic statistics if you are
0:05a firsttime viewer please stick around
0:07for the intro it is worth the time if
0:10you are a regular viewer feel free to
0:12skip ahead using The annotation so first
0:15a few things I do these videos because I
0:17love to learn and help others learn we
0:19are all good at something so I encourage
0:22you to give back to the world in a
0:23similar way share your passion any way
0:26you can now this video focuses on basic
0:30stats and is not a quick fix it aims to
0:33be thorough my goal is an understanding
0:36of fundamental concepts and that takes
0:39time but when you understand the
0:41fundamentals learning other topics is
0:43much easier now related to that if you
0:46are watching because you are struggling
0:48in a class or at work I want you to stay
0:51positive and keep your head up you can
0:54learn this I have faith in you many
0:56other people around you have faith in
0:58you and so should you
1:00feel free to connect with me on LinkedIn
1:03go+ Twitter and of course subscribe here
1:06on YouTube now if you think there is
1:08something I can do better please leave a
1:10constructive comment below I do take
1:13those comments into account when I make
1:15new videos I also encourage you to talk
1:17with other viewers in the comments help
1:19each other out when you can and finally
1:22if you like the video please give it a
1:24thumbs up share it with classmates or
1:26colleagues and put it on a playlist to
1:28review later that does encourage me to
1:31keep making them for you so all that
1:33being said let's go ahead and start
1:39learning so here we are in the fourth
1:42video in our series on multiple
1:43regression now as I'm sure you know
1:46there are many different data types
1:48we're most familiar with interval data
1:51like the temperature outside or the
1:53value of money or the mass of something
1:55on a scale but there are other types
1:58like categorical variables so male
2:01female yes no true false north south
2:04east west London Liverpool Blackpool
2:06Newcastle you get the idea but luckily
2:10regression is a very flexible
2:12statistical technique and we can
2:14implement or use categorical variables
2:17in our analysis so this first video is
2:20all about that using a technique called
2:23dummy variables to represent categorical
2:28information so as usual let's go ahead
2:30and start out with an actual problem now
2:32I will say that most of the data in this
2:34problem is actually real I went out and
2:36got it on the internet now I did change
2:39some of the numbers for pedagogical
2:40reasons but other than that this is
2:42actually real data so here is our
2:45scenario you are an analyst for a small
2:48company that develops house pricing
2:50models for independent Realtors to
2:53generate your models you use publicly
2:55available data such as list price the
2:58square footage of the the home the
3:00number of bedrooms the home has the
3:02number of bathrooms
3:04Etc but you're thinking sort of outside
3:07of the box here you are interested in
3:10another question is the public high
3:12school in the neighborhood exemplary
3:15that's the highest rating and how is
3:17that rating related to the home
3:20price so the high school rating is not
3:23quantitative it is qualitative it's
3:26categorical so for each home price the
3:29high scho is either exemplary or not yes
3:33or no and those are going to be the two
3:35categories for one of our
3:38variables so here is our home price
3:41data so on the top we have price in
3:45thousands that's our dependent variable
3:48then we have square feet that's our
3:50first independent variable and then we
3:53have exempt high school that's our
3:55second independent variable so as we can
3:58see the first home has a price of
4:03$145,000 the square footage is
4:06$1,872 square feet and that home is in a
4:09school district where the high school is
4:11not exemplary so if we go down to the
4:14third one that home is
4:18$315,000 it is $
4:204,14 Square ft and it is in a school
4:24district where the public high school is
4:26exemplary so you can see how this data
4:28works we have price that's our dependent
4:31then we have our two independent
4:32variables square footage and whether or
4:35not it's in a school district where the
4:37high school is
4:39exemplary so what I went ahead and did
4:42is coded the exemplary High School
4:44column so everything is the same but in
4:47the last column you'll see that if the
4:49high school is not exemplary then I
4:53coded that as zero if the high school is
4:56exemplary I coded that as a one this is
5:00sort of the first lesson in dummy
5:02variables so we have two categories here
5:05I assigned one zero and the other one a
5:08one that's completely arbitrary I could
5:11have switched them I could have made not
5:14exemplary one and exemplary zero it does
5:18not matter which one it goes in but for
5:20me it sort of made more sense that
5:23exemplary would be denoted with a
5:25one so here why is the home price in
5:28thousands X1 is the square footage of
5:31the home and X2 is one if the high
5:35school is exemplary and zero otherwise
5:38that's also a common way to write these
5:41so one if it's exemplary zero otherwise
5:45because often times I'll show you here
5:47in a minute there are more than two
5:48categories so it's best just to put zero
5:54otherwise so here is a grouped scatter
5:57plot of our data you can see a definite
6:00pattern here so the blue dots represent
6:02homes where the high school is not
6:04exemplary and the red squares represent
6:07homes where the high school is exemplary
6:10so on the bottom of our graph we have
6:11square footage and on the left hand the
6:14y- AIS we have the price in thousands so
6:18most of our homes down here in the lower
6:20left are homes that are smaller they are
6:24homes where the high school is not
6:26exemplary and the price is less
6:30now the red squares show us that those
6:33schools are in districts where the high
6:35school is exemplary they're larger homes
6:38and therefore they are higher priced so
6:40we can see a different pattern here so
6:43you can look at it two ways we can look
6:44at the two groups individually but if
6:46you sort of squint your eyes and look at
6:48the data points as a group we can see
6:50that there seems to be a definite
6:52pattern here so we have two patterns
6:54going on the data points as a whole
6:57start in the lower left and go up to the
6:58upper right
7:00and then we have sort of a separation in
7:01the Middle where the non-exemplar
7:04schools are on the lower left and the
7:06exemplary schools are in the upper right
7:09so there's kind of an imaginary line
7:11that runs through the graph here
7:13separating the blue dots from the red
7:18ones so what are dummy variables exactly
7:21now in many situations we must work with
7:24categorical independent variables so in
7:27regression analysis we call these dummy
7:30variables or sometimes they're called
7:31indicator variables they mean the same
7:34thing for a variable with a certain
7:37number in categories there are always
7:40going to be n minus one dummy variables
7:44and we'll walk through that here in a
7:45minute so for example in this case we
7:48have exemplary high schools and not
7:51exemplary high schools therefore there
7:53are two categories so 2 - 1 equals one
7:58dummy variable and we saw that in our
8:00original data we had one dummy variable
8:03that represented the exemplary schools
8:05and the non-exemplar schools with ones
8:08and
8:09zeros now not related to this problem
8:12necessarily at least yet let's say we
8:15have four categories north south east
8:18and west so there there are four
8:20categories so 4 minus 1 would equal
8:24three dummy variables and we'll look at
8:26that here in a second
8:30now even though it's not related to this
8:32problem necessarily let's go ahead and
8:34look at the north south east and west
8:36example we talked about in the previous
8:38slide so let's say we have north south
8:41east and west maybe we're looking at
8:43housing data or sales data across these
8:45four regions now how could we code these
8:48as dummy variables so we have four
8:51categories we're going to need four
8:53minus one dummy variables so we could
8:56code it like this so we have X1 X2 and
9:00X3 along the top those are our three
9:02dummy variables now we could represent
9:05North where X1 is 1 and X2 and X3 are
9:09zero the South Region would be 0er for
9:12X1 1 for X2 and 0 for X3 East would be
9:1700
9:1801 and here's what confuses some people
9:21the West the 4th region would be zeros
9:26all across so West would be coded
9:29nothing so North would be one for X1
9:32South would be one for X2 East would be
9:34one for X3 and for the west region we
9:37would not put any ones in our regression
9:40and we'll see how that works as we go
9:42forward so this is an example of a
9:44variable with four categories and three
9:47dummy
9:50variables so let's back to our problem
9:52at hand so this is very similar to some
9:55of the other multiple progression we did
9:57in previous uh videos so the expected
10:00value of y the dependent variable equals
10:02beta 0 that's our intercept plus beta 1
10:06X1 that's our first coefficient and our
10:08first independent variable plus beta 2
10:12and X2 that's our second coefficient and
10:15our second
10:18variable now we have two things going on
10:21here we have one case where the X2 is
10:25zero where the high school is not
10:27exemplary and then we have another case
10:30where the high school is exemplary and
10:32X2 is a one so let's look at the first
10:36case first so the expected value of home
10:39price given the high school is not
10:42exemplary that's where X2 equals 0 so
10:46we're going to go ahead and change this
10:47estimated regression equation up there
10:49at the top to reflect that so e the
10:52expected value of y our dependent
10:56variable given that's the straight line
10:58given that the high school is not
11:02exemplary so we rewrite that equation as
11:04beta 0 + beta 1 X1 plus beta 2 * 0 CU
11:11remember when X2 is0 that means our high
11:14school is not exemplary so we can go
11:16ahead and put that in for X2 now we just
11:19do some simple
11:21algebra well beta sub 2 * 0 is 0 so it
11:25basically disappears and we're left with
11:28beta Sub 0 plus beta 1
11:32X1 now what about when the high school
11:36is exemplary and X2 = 1 same process so
11:41beta Sub 0 plus beta 1 X1 + beta 2 but
11:46this time times 1 CU remember that's the
11:50value when the high school is
11:52exemplary so again some simple algebra
11:55beta 0 plus beta 1 X1 plus beta 2 cuz is
11:59beta 2 * 1 is itself beta 2 now those
12:03are going to be two constant numbers
12:05beta Sub 0 and beta 2 are just going to
12:09be numbers without any variables
12:12attached so we can actually combine them
12:15so in parentheses we have beta Sub 0
12:17plus beta 2 plus beta 1 X1 so we
12:23actually have two different regression
12:25equations here in the first case X2 is Z
12:30in the second case X2 is 1 so we always
12:34have to realize that there is a
12:36regression equation for every possible
12:39scenario in the dummy variable in this
12:42case it's two we have zero and one as
12:45far as the values of that dummy
12:50variable so when we conduct the
12:52regression in manyi tab this is what we
12:54get we can see our categorical predictor
12:57coding of 1 and zero then we have our
13:00Innova table as usual so we look across
13:02the regression line there we can see
13:05that our F value is
13:0735.94 with a P value of
13:100. that of course means it's less than
13:130.1 and is
13:15significant now we look down here at the
13:17bottom our model summary we have an R
13:19squ of
13:2085.7 an R squ adjusted of 83.3 1 and an
13:25r s predicted of
13:2770.3 so so a high R squ and high R squ
13:31adjusted along with an r s predicted
13:33that doesn't fall off a cliff remember
13:36in part three I mentioned that even if
13:38we have an high R squar and a high R squ
13:42adjusted and then our r s predicted just
13:45goes crazy low like off a cliff say you
13:48know
13:4950% then we would be concerned our
13:52regression equation is not doing a good
13:54job at predicting so everything here
13:56looks good let's go ahead and look at
13:58our Co efficients so we have the
14:01constant term we don't worry about that
14:02in this case then we have the square
14:04foot variable that has a P value of
14:080.10 so that is significant at
14:1105 and then we have exempt High School
14:14where the value is one and that is
14:17significant at
14:19018 so square foot coefficient is
14:22significant and the exempt High School
14:24coefficient is significant now what
14:28about the values of the coefficients for
14:30square foot it's
14:330621 now remember that's in thousands of
14:36dollars now for exempt high school it's
14:3998.6 and we'll talk about that here in a
14:41second so first let's talk about square
14:44foot and its coefficient so every square
14:46foot is related to an increase in price
14:49in the home of 0 621,000
14:53or when you multiply that out
14:57$621 per square foot so think about this
15:00for a minute that's 1 square ft what if
15:03the house is 10 s ft larger well that
15:08would be
15:09$621 what about 100 squ ft larger that
15:14would be
15:16$621 but what about a th000 ft larger
15:21home well that would be
15:23$62,400 in price so you can see how each
15:28additional square foot is related to the
15:31price of the home in this regression
15:33model so now let's interpret exempt High
15:36School remember its coefficient is 98.6
15:39so what does that mean well it means
15:41that on average a home in an area with
15:44an exemplary High School is related to a
15:48$98,500
15:49higher price so if we have two homes the
15:54same square footage let's say both homes
15:57are 1,500 Square ft one is in a district
16:01where the high school is not exemplary
16:04one is in a district where the high
16:05school is exemplary same square foot the
16:09only difference is the high school
16:11rating the home in the district with the
16:14higher rated high school will be $98,700
16:19higher in
16:22price so let's go ahead and do the full
16:24interpretation with the numbers we just
16:25generated along with our regression
16:28equation so the equation we got for
16:30minab is 27.1 + 0621 X1 we talked about
16:35that last slide plus 98.6 X2 so the only
16:40thing we didn't talk about really in the
16:41last slide was The Intercept but that's
16:43not really relevant for this type of
16:45model but we still need it of
16:48course so let's go ahead and just plug
16:50everything in so the expected value of a
16:52home price given the high school is not
16:54exemplary where X2 equals 0 so all we do
16:57is take the zero and substitute that in
16:59for X2 so we go ahead and do that so
17:0298.6 * 0 is 0 so we're left with
17:0627.1 plus
17:080621 X1 so it's a very simple algebraic
17:13linear
17:14equation now what about when the high
17:17school is exemplary so there X2 equal 1
17:21so we go ahead and substitute everything
17:22back in there so 98.6 * 1 is
17:2698.6 now we can combine the
17:2998.6 with the 27.1 so we put that in
17:32parenthesis and that is
17:37125.77 21 X1 so here are our two lines
17:44here are our two linear equations and
17:46we'll actually look at that on a graph
17:48here
17:50next so here are the two regression
17:52equations that mini tab gives us it's
17:55saying that when the high school is not
17:57exemplary that's 0 the price in
18:00thousands is equal to 27.1 +
18:030621 ft we just figured that out on the
18:06last slide when the high school is
18:09exemplary it's one the price in
18:11thousands is 125 plus 06 to 1T so you
18:17notice there that the slope of these
18:20lines are the same
18:230621 the only thing that's different are
18:25the intercepts so let's go ahead and put
18:28those actually on a
18:30graph and here it is so we can see that
18:34the first example where the high school
18:36is exemplary where it's one that's the
18:41125.77 21 ft that is sort of the blue
18:44purple line here on the top when we
18:47actually graph that on our graph then of
18:49course the red line is the homes without
18:53an exemplary high school so you can see
18:55that these two lines are just algebra 1
18:58we go ahead and put those on a graph now
19:01it actually has meaning though there's a
19:03distance between them and guess what
19:05that is
19:0798.6 so the average distance between
19:10these two lines is the
19:1498,6 that we talked about a couple of
19:16slides ago so everywhere along this
19:19distance the average distance or the
19:21average price difference is 98.6 or 98,6
19:26[Music]
19:31now just for the sake of learning I went
19:32ahead and conducted another scou plot
19:34but actually put each groups regression
19:37line inside of it now as you can see
19:40this looks somewhat similar to the graph
19:42we had in the previous slide but
19:45remember the previous slide is the
19:47average over everything here each line
19:51is unique to each group so the general
19:54pattern is the same and actually the red
19:57line is actually pretty close to the
19:58exact same but because the way the homes
20:01are distributed on here it's not going
20:03to be exactly the same as we saw in the
20:04in the last slide cuz it's again
20:06regression is all about the averages
20:09over everything but here you can
20:10definitely see the difference between
20:12the homes that have an exemplary High
20:14School in its district and those that
20:18don't okay so that wraps up our
20:20introduction to dummy variables now
20:23we'll be doing more with dummy variables
20:24in the next video but I just wanted to
20:26get your feet wet so you understand what
20:28they are where they come from how we use
20:31them to code different categories of
20:34data and then we how we use those
20:36variables that we code in actual
20:38regression of course they do get more
20:40complex but the basic interpretation is
20:43the same so hopefully you were able to
20:45develop a good fundamental understanding
20:47of dummy variables so you can apply that
20:50to more complex problems so thank you
20:52very much for watching please subscribe
20:54if you have not done so already and I
20:56look forward to seeing you again in the
20:57next video

About this transcript

This page contains the full transcript of YouTube transcript (fTfMdCQJz4s) , generated from the public captions YouTube serves with the video. The transcript has 3,230 words across 456 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.