YT YouTube2Text

YouTube2Text → Transcripts → Hypothesis Testing and The Null Hypothesis, Clearly Explained!!!

Hypothesis Testing and The Null Hypothesis, Clearly Explained!!! — Transcript

by StatQuest with Josh Starmer · 1,970 words · 358 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Stat Quest in the morning Stat Quest at
0:04night Stat Quest in the afternoon it's
0:08all right Stat Quest.
0:12Hello, I'm Josh Starmer and welcome to
0:15Stat Quest. Today we're going to talk
0:17about hypothesis testing and the null
0:20hypothesis.
0:22I'm not going to name names, but imagine
0:25there was a virus
0:27and we had two drugs we could use to
0:29treat it.
0:31So we give drug A to three people
0:34and measure how long it takes each
0:36person to recover from the virus.
0:39The first thing we notice is that not
0:41everyone recovered in the exact same
0:43amount of time.
0:45Person number one recovered the fastest
0:49and person number two recovered the
0:51slowest.
0:53It's possible that person number one
0:56eats healthy food and exercises and
0:58already has a strong immune system and
1:01that helped them recover quickly.
1:04And maybe person number two doesn't get
1:06as much exercise
1:08or maybe person number two has a
1:10stressful job or lives where there is a
1:12lot of air pollution.
1:14The point is is that even though all
1:16three people had the same virus and took
1:19the same drug
1:21they did not all recover in the exact
1:23same amount of time and that might be
1:26due to a lot of random things like
1:28exercise or job stress that we cannot
1:30control.
1:33Now let's give drug B to three different
1:35people that have the virus
1:37and measure how long it takes them to
1:39recover.
1:41Again, we see that even though three
1:43people had the same virus and took the
1:45same drug
1:47they did not all recover in the exact
1:49same amount of time.
1:52And this is probably due to random stuff
1:54that we can't control like how much
1:56exercise each person gets or how much
1:59candy they eat.
2:01Overall, it looks like people taking
2:03drug A took less time to recover than
2:06people taking drug B.
2:08And when we calculate the mean or
2:11average value for drug A
2:14and the mean value for drug B,
2:17we see that on average, there is a
2:2015-hour difference between drug A and
2:22drug B.
2:24So, after seeing this preliminary data,
2:28it might seem reasonable to form the
2:29following hypothesis.
2:32People taking drug A need, on average,
2:3615 fewer hours to recover than people
2:38taking drug B.
2:41And now that we have this hypothesis, we
2:44can test it
2:46by repeating the experiment.
2:49Now, when we calculate the means,
2:51we see that, on average, people taking
2:54drug A need 35 more hours than people
2:58taking drug B.
3:00Compared to our preliminary data, this
3:02result is very unexpected.
3:06In fact, it is the opposite of the
3:08original hypothesis.
3:11But, it is also possible that all three
3:13people that took drug A in the second
3:16experiment have super stressful jobs and
3:18unhealthy lifestyles.
3:21And maybe that's why it took them so
3:23long to recover.
3:25And maybe everyone taking drug B was
3:28well-rested and super healthy to begin
3:30with.
3:32And maybe that's why they recovered so
3:34quickly.
3:35But, it is also possible that we
3:37mislabeled drug A and drug B and did the
3:40wrong experiment.
3:43So, we repeat the experiment.
3:46And again, the results are totally
3:48backwards from the preliminary
3:49experiment
3:51and totally backwards from the
3:53hypothesis that we made.
3:56So, again, just to make sure we didn't
3:59mislabel things, we redo the experiment.
4:03And again, these results are the
4:05opposite of the original hypothesis.
4:09So, we just keep repeating the
4:10experiment, each time double-checking
4:13every little detail.
4:16And every time we do the experiment, we
4:18get the opposite result of the original
4:20hypothesis.
4:23So, after doing all of these repeated
4:25experiments, where we double-checked
4:28every little step,
4:30we can confidently reject this
4:32hypothesis that we came up with after
4:34doing the preliminary experiment.
4:37Bam!
4:40Now, let's imagine we had two more
4:42drugs, C and D.
4:46And just like before, we gave drug C to
4:48three people
4:50and measured how long it took each
4:52person to recover from the virus.
4:56Then we gave drug D to three different
4:58people
5:00and measured how long it took them to
5:01recover from the virus.
5:05And based on this data, we can create a
5:07hypothesis about drug C and drug D.
5:11People taking drug C need, on average,
5:1513 fewer hours to recover than people
5:17taking drug D.
5:20Now, just like before, we decide to test
5:23this hypothesis by repeating the
5:25experiment.
5:27Only this time, instead of getting
5:29something that's the exact opposite of
5:31what we expected,
5:33we get something that is only slightly
5:35different.
5:37In this case, the difference is in the
5:39same direction, but it is only 12 hours.
5:44Then we repeat the experiment again,
5:47And again, we get something slightly
5:50different from the preliminary
5:51experiment and hypothesis.
5:54The difference is in the same direction,
5:56but this time it is 13.5 hours.
6:01The good news is that we probably didn't
6:03mislabel the drug like we did last time.
6:06And the differences between the three
6:08experiments might be due to random
6:10things we cannot control.
6:13Like maybe these people exercised a lot
6:15and had relatively healthy diets
6:18compared to these people who took longer
6:20to recover.
6:22But, regardless, the hypothesis says
6:25that people taking drug C needed 13
6:28fewer hours to recover.
6:30But when we repeated the experiment,
6:33the first replicate said the difference
6:35between averages was 12,
6:38which is different from the hypothesis.
6:41And the second replicate said the
6:42difference was 13.5,
6:45which is also different from the
6:47hypothesis.
6:49And let's be honest, the only reason the
6:52hypothesis says 13 fewer hours is
6:55because that was the result from the
6:56first experiment.
6:59However, we could have just as easily
7:01put 12 fewer hours in the hypothesis
7:04because that's what we got the second
7:05time.
7:07Or we could have put 13.5 fewer hours in
7:10the hypothesis because that's what we
7:12got the third time.
7:15So if we just pick one experiment like
7:17the first one,
7:19and use that to define the hypothesis,
7:23then we have two experiments that are
7:25not different enough to give us
7:27confidence to reject the hypothesis,
7:30but because there is just as much data
7:32suggesting that the difference is 12
7:34hours,
7:36and there is just as much data
7:37suggesting that the difference is 13.5
7:40hours,
7:42these experiments don't make us super
7:44confident that the hypothesis of 13
7:46fewer hours is correct.
7:49Again, maybe drug A reduces recovery by
7:5313 fewer hours,
7:55but maybe it reduces recovery by 12
7:57hours
7:58or 13.5.
8:01Because the results from the repeated
8:03experiments are not different enough to
8:05cause us to reject the hypothesis,
8:09and because they don't convince us that
8:11the hypothesis is correct, either,
8:14the best we can do is fail to reject the
8:17hypothesis.
8:19Small bam.
8:21To summarize what we've covered so far,
8:24we can create a hypothesis.
8:27And if data gives us strong evidence
8:29that the hypothesis is wrong,
8:31then we can reject the hypothesis.
8:35But when we have data that is similar to
8:37the hypothesis, but not exactly the
8:39same,
8:41then the best we can do is fail to
8:43reject the hypothesis.
8:45Because it's unclear if the hypothesis
8:48should be based on this result
8:50or this other, slightly different,
8:52result
8:53or this result
8:55or any other possible outcome.
8:58Double bam.
9:01Now, let's take a closer look at the
9:03hypothesis itself.
9:06You may remember that the only reason
9:08the hypothesis is 13 fewer hours is that
9:11it was the first result.
9:14But we could have just as easily gotten
9:16a 12-hour difference
9:18or a 13.5-hour difference and ended up
9:21with a different hypothesis.
9:24And if 12 and 13.5 are reasonable
9:27hypotheses, then so is 12.25
9:31or 13.1.
9:33In other words, there are a lot of
9:35reasonable hypotheses.
9:38How do we know which one to test?
9:41Since the goal is to see if drug C is
9:43different from drug D,
9:46we simply test to see if there is no
9:48difference between the drugs.
9:51Oh, no, it's the dreaded terminology
9:53alert.
9:55The hypothesis that there is no
9:57difference between things is called the
9:59null hypothesis.
10:02So, let's take a look at two examples of
10:04the null hypothesis in action.
10:07Now, imagine we are testing two new
10:10drugs, E and F.
10:13And this time, we only get a 0.5 hour
10:16difference.
10:17This person recovered the fastest,
10:20but it is easy to imagine that if they
10:23had exercised a little less or had a
10:25slightly worse diet,
10:27then they might have taken a little
10:28longer to recover.
10:31Likewise, if this person was just a
10:33little healthier to begin with,
10:36then they might have recovered a little
10:37more quickly.
10:39These small, random differences give us
10:42a slightly different result.
10:45Now, instead of drug F being slightly
10:48better by 0.5 hours, drug E is slightly
10:52better by 0.25 hours.
10:55Because these small, random differences
10:58give a slightly different results,
11:01we can use the null hypothesis so we
11:03don't have to worry about whether or not
11:05the difference is exactly 0.25
11:08or 0.5 hours.
11:11Instead, we simply see if the data
11:13convinces us to reject the hypothesis
11:16that there's no difference between drug
11:19E and drug F.
11:21In this case, the original result was
11:240.5 hours in favor of drug F.
11:28But small, random things could have
11:31easily changed result to be a 0.25 hour
11:34difference in favor of drug E.
11:38And thus, the data does not
11:40overwhelmingly convince us to reject the
11:43null hypothesis.
11:45So, we failed to reject the null
11:47hypothesis that there is no difference
11:50between the drugs.
11:52In contrast, if we tested the drugs on a
11:55lot of people
11:57and little random things would not
11:59change the results very much,
12:02then we could confidently reject the
12:04null hypothesis that there is no
12:06difference between drug E and drug F.
12:11Bam!
12:12Note, without the null hypothesis, we
12:15need preliminary data in order to make a
12:17statement that we can test in follow-up
12:20experiments.
12:22This is because we don't know if we
12:24should test if the difference is 13
12:26hours or 13,000 hours until we get some
12:29data.
12:30In contrast, the null hypothesis does
12:33not require preliminary data because the
12:36only value that represents no difference
12:39is zero.
12:41Triple bam!
12:43In summary,
12:45rather than get stressed out over a
12:47large number of possible hypotheses that
12:50we could test to see if drug C is
12:52different from drug D,
12:54we use the null hypothesis to determine
12:57if there is a difference.
12:59If we do an experiment with a bunch of
13:01people
13:02and a lot more people taking drug C had
13:05shorter recovery times than people
13:07taking drug D,
13:09so many that it would be hard to imagine
13:11that the results were due to random
13:13things, like everyone taking drug C had
13:16better diets or got more exercise than
13:18the people taking drug D,
13:21then we could reject the null
13:22hypothesis.
13:24And then we know that there is a
13:26difference between drug C and drug D.
13:30Alternatively, if little random things
13:33could easily shift the result from one
13:35drug to the other and then back again,
13:38then we would fail to reject the null
13:40hypothesis.
13:43Bam!
13:44But wait, what about the alternative
13:47hypothesis?
13:49Because the alternative hypothesis is
13:51super important, it has its own quest,
13:54so check it out.
13:55And if you don't already know about P
13:57values, they would make a wonderful
13:59follow-up.
14:01Lastly, if you want to review statistics
14:04and machine learning offline, check out
14:06the StatQuest study guides at
14:08statquest.org.
14:10There's something for everyone.
14:13Hooray! We've made it to the end of
14:16another exciting StatQuest. If you like
14:18this StatQuest and want to see more,
14:20please subscribe. And if you want to
14:22support StatQuest, consider contributing
14:25to my Patreon campaign, becoming a
14:27channel member, buying one or two of the
14:30StatQuest study guides or a t-shirt or a
14:32hoodie, or just donate. The links are in
14:35the description below.
14:36All right, until next time, quest on!

About this transcript

This page contains the full transcript of Hypothesis Testing and The Null Hypothesis, Clearly Explained!!! by StatQuest with Josh Starmer, generated from the public captions YouTube serves with the video. The transcript has 1,970 words across 358 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.