The Population Bag Example

Jeremy Jackson
3 Apr 202018:45

Summary

TLDRThis video script delves into a hypothetical 'population bag' example to illustrate the concepts of Type 1 and Type 2 errors in statistical hypothesis testing. It explains the scenario of sampling from two bags, one with and one without removed chips, to demonstrate how researchers might incorrectly reject or fail to reject a null hypothesis. The script clarifies why in real-world applications, managing Type 2 errors is more critical than Type 1 errors, as the null hypothesis of identical population means is never exactly true, unlike in the classroom example where it can be.

Takeaways

  • 📚 The video discusses the 'population bag example', which is a hypothetical scenario used to illustrate concepts in hypothesis testing.
  • 🎓 The example involves two population bags, each containing chips with numbers from one to five, and the task is to determine if the bags are the same or different based on samples.
  • 🔍 The video introduces a null hypothesis that the two bags have the same mean, and an alternative hypothesis that they do not.
  • 🧐 The decision rule is set with an alpha level of 0.1, meaning the null hypothesis will be rejected if the observed sample mean difference is unlikely (p < 0.1).
  • 🔢 The video explains the process of taking samples from each bag, calculating the mean difference, and then determining a T value and p-value to test the hypothesis.
  • 🔄 The process is repeated 16 times to simulate multiple experiments, highlighting the variability in outcomes and the potential for errors in hypothesis testing.
  • 🚫 Type 1 errors occur when the null hypothesis is incorrectly rejected (false positive), and Type 2 errors occur when the null hypothesis is incorrectly accepted (false negative).
  • 🤔 The video emphasizes that in real-world scenarios, the null hypothesis is never exactly true, making Type 1 errors impossible and Type 2 errors more critical to manage.
  • 🌐 The 'population bag example' contrasts with real-world situations where the effect size (difference between population means) is often small, increasing the likelihood of Type 2 errors.
  • 📉 The video concludes by answering a specific question about why beta (Type 2 error rate) is more important than alpha (Type 1 error rate) in real-world applications, highlighting the practical implications of hypothesis testing.

Q & A

  • What is the 'population bag example' described in the video?

    -The 'population bag example' is a hypothetical scenario used to illustrate the concept of hypothesis testing. It involves two bags, each containing chips with numbers on them. The bags can either have the same chips or one bag has some chips removed, altering the mean value. The task is to determine, through sampling and statistical analysis, whether the two bags have the same or different chips.

  • What is the significance of the mean and standard deviation in the population bag example?

    -In the population bag example, the mean and standard deviation of the chips in the bags are crucial. The mean represents the average value of the chips, and the standard deviation measures the spread of the values. These statistical measures help in understanding the distribution of the chips and are used in hypothesis testing to determine if the samples are drawn from bags with the same or different chips.

  • What are the two situations described in the population bag example?

    -The two situations in the population bag example are: 1) Both bags have the same chips (e.g., six ones, twos, threes, fours, and fives), and 2) One bag has the same chips as the population bag, while the other has some chips removed (e.g., three ones, six twos, six threes, six fours, and six fives), resulting in a different mean.

  • Why is it important to calculate the mean difference between the samples in the population bag example?

    -Calculating the mean difference between the samples is important because it helps in making a judgment about whether chips have been removed from the bag. If the two bags have the same chips, the means of the samples should be similar. However, if chips have been removed, the mean of the bag with removed chips will be higher, and the sample means will likely differ.

  • What is the null hypothesis in the hypothesis testing procedure described in the video?

    -The null hypothesis in the hypothesis testing procedure is that the two bags are exactly the same, meaning the mean of the chips in bag one is the same as the mean of the chips in bag two. This assumption of no difference is what is being tested against the alternative hypothesis.

  • What is the decision rule used in the hypothesis testing procedure, and what is its significance?

    -The decision rule used in the hypothesis testing procedure is that if the p-value is less than alpha (set at 0.1 in the example), the null hypothesis is rejected. This rule is significant because it determines whether the observed sample mean difference is statistically significant enough to conclude that the bags are different.

  • What are type 1 and type 2 errors in the context of hypothesis testing?

    -In hypothesis testing, a type 1 error occurs when the null hypothesis is rejected when it is actually true. A type 2 error occurs when the null hypothesis is not rejected when it is actually false. In the population bag example, type 1 errors would be incorrectly concluding that the bags are different when they are the same, while type 2 errors would be incorrectly concluding that the bags are the same when they are different.

  • Why is beta more important than alpha in real-world situations but not in the population bag example?

    -In real-world situations, the means of two populations are never exactly the same, so the null hypothesis is never strictly true. This means that the only kind of error that can be made is a type 2 error (failing to reject a false null hypothesis). Therefore, managing the probability of a type 2 error (beta) is more important. In the population bag example, however, the null hypothesis can be true, making both type 1 and type 2 errors possible.

  • What is the effect of sample size on the probability of making a type 2 error?

    -The sample size has a significant impact on the probability of making a type 2 error. A smaller sample size reduces the power of the test, making it more likely to fail to detect a true effect (i.e., make a type 2 error). In the population bag example, the small sample size of 10 chips from each bag contributes to the high probability of type 2 errors.

  • How does the effect size influence the probability of making a type 2 error in the population bag example?

    -The effect size, which is the difference between the population means, plays a crucial role in the probability of making a type 2 error. If the effect size is small, the test is less likely to detect a difference even if one exists, increasing the likelihood of a type 2 error. In the population bag example, the small difference in means between the bags with and without removed chips contributes to the high probability of type 2 errors.

Outlines

00:00

🎒 Introduction to the Population Bag Example

The speaker introduces a hypothetical scenario known as the population bag example, which is relevant to a short answer question (number 25) in an upcoming quiz. The scenario involves two bags, each potentially containing the same set of chips, with one bag possibly having some chips removed. The purpose is to determine whether two samples are from bags with identical chips or not. The example uses hypothesis testing to decide if the mean difference between two samples suggests that chips have been removed from one bag, affecting its mean value. The video will guide through the process of sampling, calculating means, and using those to infer about the population characteristics.

05:01

🔍 Hypothesis Testing and Decision Making

This paragraph delves into the specifics of the hypothesis testing process. The null hypothesis posits that the two bags are identical in terms of the chips they contain. An alpha level of 0.1 is set as the decision rule for rejecting the null hypothesis, based on the probability (p-value) associated with the observed sample mean difference. The scenario of conducting 16 separate experiments is introduced, where in most cases, the p-value does not fall below the alpha threshold, indicating no significant difference between the bags' means. However, two instances of type 1 errors are identified, where the null hypothesis is wrongly rejected despite the bags being identical, aligning with the expected error rate of 10%.

10:01

📈 Understanding Type 1 and Type 2 Errors in Different Scenarios

The speaker explores the implications of type 1 and type 2 errors in the context of the population bag example and contrasts it with real-world scenarios. In the bag example, the null hypothesis can be true, allowing for the possibility of type 1 errors. However, in real-world applications, the null hypothesis of no difference is usually an idealization and not strictly true, making type 1 errors a theoretical concern rather than a practical one. The focus then shifts to type 2 errors, which occur when the null hypothesis is not rejected despite it being false. The speaker illustrates this with a scenario where the bags have different means, and the sampling procedure fails to detect this difference, leading to multiple type 2 errors and a high error rate.

15:03

🌐 Real-World Significance of Beta Over Alpha

The final paragraph addresses question number 25, explaining why beta (the probability of a type 2 error) is more critical in real-world applications compared to alpha. The speaker clarifies that in real-world scenarios, it is impossible for two populations to have identical means, making the null hypothesis of no difference between means always false to some extent. As a result, the potential for type 1 errors is minimal, whereas the risk of type 2 errors—failing to detect a real difference—is substantial. The video concludes by emphasizing the importance of managing type 2 errors in practical research and decision-making.

Mindmap

Keywords

💡Population Bag Example

The 'Population Bag Example' is a hypothetical scenario used in the video to illustrate concepts of statistical sampling and hypothesis testing. It involves two bags, each containing chips with numbers on them, representing a population. The video uses this example to explain how researchers might determine whether two samples come from the same population or not. The concept is central to the video's theme of exploring the importance of 'beta' (type II error) over 'alpha' (type I error) in real-world scenarios.

💡Hypothesis Testing

Hypothesis testing is a statistical method used to make decisions about populations based on sample data. The video describes a process where a null hypothesis is set up (assuming no difference between the two bags), and then a decision rule is applied based on the calculated p-value and a predetermined alpha level. The concept is integral to the video's narrative, demonstrating how researchers can make errors in their conclusions, such as type I and type II errors.

💡Type I Error

A type I error, also known as a 'false positive,' occurs when the null hypothesis is incorrectly rejected when it is actually true. In the context of the video, this error is exemplified when the researcher concludes that the two bags have different means when they actually have the same mean. The video discusses the probability of making a type I error, which is determined by the alpha level set at the beginning of the hypothesis test.

💡Type II Error

A type II error, or 'false negative,' happens when the null hypothesis is not rejected even though it is false. The video uses the example of sampling from two bags with different means but not detecting the difference, thus failing to reject the null hypothesis. The rate of type II errors is represented by 'beta,' which is more significant in real-world applications than in the hypothetical 'Population Bag Example.'

💡Alpha Level

The alpha level (denoted as α) is the probability threshold for rejecting the null hypothesis in a hypothesis test. In the video, an alpha level of 0.1 is set, meaning there is a 10% risk of committing a type I error. The script uses alpha to demonstrate the decision-making process in hypothesis testing and the potential for error.

💡Beta

Beta (denoted as β) represents the probability of making a type II error, which is the likelihood of failing to reject a false null hypothesis. The video emphasizes that beta is more critical in real-world scenarios than in the 'Population Bag Example' because in reality, population means are never exactly the same, making the risk of a type II error more pertinent.

💡Sample Mean Difference

The 'sample mean difference' is the difference between the means of two samples taken from potentially different populations. In the video, this difference is used to calculate a T value and a p-value, which are then used to make a decision in the hypothesis test. The concept is central to determining whether the samples come from the same or different populations.

💡T Value

The T value is a statistic used in hypothesis testing, specifically in T-tests, to determine the likelihood that the observed difference between sample means could have occurred by chance if the null hypothesis were true. The video describes calculating the T value to assess the probability of observing the sample mean difference under the assumption of equal population means.

💡P-Value

The p-value is the probability of observing the sample results, or something more extreme, assuming the null hypothesis is true. In the video, the p-value is calculated to assess whether the observed sample mean difference is significant enough to reject the null hypothesis. The script uses p-values to illustrate the decision-making process in hypothesis testing.

💡Effect Size

Effect size is a measure of the magnitude of the difference between two groups, which is crucial for understanding the practical significance of the results. In the video, a small effect size (the difference between the means of the two bags) combined with a small sample size leads to a high probability of type II errors, highlighting the importance of effect size in determining the power of a statistical test.

💡Null Hypothesis

The null hypothesis is a statement of no effect or no difference that is tested in an experiment. In the video, the null hypothesis is that the two bags have the same mean values. The script uses the null hypothesis as the starting point for hypothesis testing, setting up the framework for evaluating whether the observed sample differences are statistically significant.

Highlights

Introduction to the 'population bag' example used to explain statistical concepts in hypothesis testing.

Reference to short answer question number 25, which asks why beta is more important than alpha in the real world but not in the population bag example.

Description of two hypothetical situations involving two population bags with either identical or different chips.

Explanation of the task for researchers to determine if samples are from bags with the same or different chips based on mean differences.

Hypothesis testing procedure using the logic of rejecting the null hypothesis if the p-value is less than a set alpha level.

Sampling 10 values from each bag and calculating the sample mean difference and associated T value.

Decision rule where a p-value less than alpha (0.1) leads to the rejection of the null hypothesis.

Illustration of conducting 16 separate experiments to understand the occurrence of type 1 and type 2 errors.

Example of not rejecting the null hypothesis when it is true, which is not an error in hypothesis testing.

Demonstration of type 1 errors where the null hypothesis is incorrectly rejected despite being true.

Calculation of the probability of making a type 1 error, which is expected to be around the alpha level set.

Explanation of type 2 errors where the null hypothesis is not rejected even though it is false.

Discussion on the high rate of type 2 errors due to small effect size and sample size.

Clarification on why beta (type 2 error) is more critical in real-world scenarios where the null hypothesis is never exactly true.

Comparison between the population bag example and real-world situations regarding the plausibility of the null hypothesis being true.

Final summary explaining the importance of managing type 2 errors in practical applications of hypothesis testing.

Transcripts

play00:00

in this video I'm going to describe what

play00:02

I call the population bag example now

play00:05

this example is made reference to in the

play00:09

short answer questions they're going to

play00:10

be eligible for the last quiz and the

play00:14

particular short answer question is

play00:15

number 25 and in that question I asked

play00:18

why is bata more important than alpha in

play00:20

the real world but not in the population

play00:22

bag example that we did in class and so

play00:24

this population bag example that's what

play00:27

I'm going through in the video ok now I

play00:30

want you to imagine a case here in which

play00:33

there are two population bags now there

play00:38

are two situations here with these two

play00:41

bags in one situation the two bags could

play00:45

have exactly the same chips in them so

play00:49

in one of the situation's you've got one

play00:52

bag has 6-1 6-2 6-3 6-4 and six fives

play00:55

that's your population bag that has a

play00:58

mean of three and a standard deviation

play00:59

of one point four one four and then the

play01:01

other bag in the pair here has exactly

play01:04

the same chips in it as the first bag so

play01:07

so one situation is you have two

play01:10

population bags both population bags

play01:12

have the same chips in them and those

play01:14

chips are the chips that you have in

play01:17

your actual population bag now six ones

play01:19

twos threes fours and fives now the

play01:21

other situation is that one of the bags

play01:23

is the same as your population bag now

play01:26

six ones twos threes fours and fives and

play01:28

the other bag is like this the other bag

play01:32

has had chips removed so there aren't

play01:35

six ones twos threes fours and fives

play01:36

anymore

play01:37

there's three ones six two six three six

play01:41

four and six fives so in this bag there

play01:45

have been some chips removed and what

play01:49

has been removed three ones three low

play01:51

chips okay so you've got these two

play01:54

situations situation one the two bags

play01:57

are the same and situation two you have

play02:00

one bag with no chips removed and one

play02:02

bag with chips removed now the problem

play02:06

for you as the researcher in this

play02:09

situation is to guess which situation

play02:12

you have

play02:12

so you have to guess whether or not the

play02:15

two samples that you've drawn have been

play02:18

drawn from bags with the same chips in

play02:20

them or whether or not the two samples

play02:23

have been drawn from bags with different

play02:25

chips in them okay so that's your job as

play02:28

a researcher so what's going to happen

play02:30

here is we're going to take a sample out

play02:32

of each of two bags and so you're gonna

play02:35

have two samples of chips we're gonna

play02:38

calculate the mean of each of the

play02:40

samples and we're going to use the mean

play02:42

difference between the samples to make a

play02:45

judgment about whether or not chips have

play02:46

been removed from the bag now if you're

play02:50

sampling from two bags that are the same

play02:52

you should expect the means of the two

play02:55

samples that you get to be pretty

play02:56

similar that's because you're sampling

play02:59

from bags that have the same chips in

play03:01

them but if you're sampling from one bag

play03:03

in which no chips have been removed

play03:05

there's six ones two threes fours and

play03:06

fives and you're sampling from another

play03:08

bag in which chips have been removed you

play03:11

would expect the sample mean in the bag

play03:13

where chips have been removed to be a

play03:15

little different from the sample mean in

play03:17

the bag where chips have not been

play03:18

removed now specifically what would you

play03:20

expect well if you've taken out three

play03:23

low chips here the mean of the chips in

play03:26

the bag is going to be a little higher

play03:27

it's three point two because you've

play03:29

removed three chips with a value of one

play03:31

you've removed low numbers so the mean

play03:34

is going to go up so you would expect

play03:37

the means of the samples drawn from a

play03:40

bag with a mean of three point two to be

play03:42

on average a little bit higher than the

play03:45

means of the samples drawn from a bag

play03:47

with no chips removed from the bag with

play03:49

a mean of three so what we're going to

play03:52

do here is we're going to go through a

play03:54

hypothesis testing procedure we're going

play03:56

to use the logic bypasses testing to

play03:58

make a decision about whether or not

play04:00

these two samples that you're going to

play04:02

draw have been drawn from bags are the

play04:03

same or drawn from bags that are

play04:05

different all right so here's the

play04:07

hypothesis testing logic here you can

play04:12

see that we start with a null hypothesis

play04:15

and so your null hypothesis is going to

play04:17

be that the two bags are exactly the

play04:20

same okay so that the me

play04:22

on the chips in bag one it's the same as

play04:25

the mean on the chips in bag two so the

play04:28

mean is three here and the mean is three

play04:31

in your second bag okay so you're going

play04:32

to start with the null assumption of no

play04:35

difference between the bags now we've

play04:37

got a decision rule here and the

play04:39

decision rule is if P less than alpha

play04:41

and we're gonna set alpha 0.1 we're

play04:43

going to reject the null hypothesis so

play04:45

if the probability of the sample mean

play04:47

difference we observe is small if the

play04:50

null hypothesis is true we're going to

play04:52

conclude that the null hypothesis is not

play04:54

true now the study we're going to do is

play04:56

to sample 10 values from each bag so

play04:59

we're gonna sample 10 values from one

play05:00

bag and 10 values from the other bag

play05:03

we're going to calculate the sample mean

play05:05

difference so the difference between the

play05:08

means of these two samples and then

play05:11

we're going to calculate a T value now

play05:12

this is the T value for an independent

play05:15

groups t-test that we saw in a previous

play05:17

video and we're going to use that T

play05:19

value to calculate the probability of

play05:22

observing the sample mean difference

play05:24

that we observed given that the two

play05:26

samples have been drawn from bags with

play05:28

exactly the same means so given that the

play05:30

two samples have been drawn from these

play05:32

two bags here now we're then going to

play05:35

make a decision and the decision that

play05:37

we're going to make is that if the p

play05:38

value is less than our alpha of 0.1

play05:40

we're going to reject the null and we're

play05:42

going to conclude that we've sampled

play05:43

from bags with different means all right

play05:47

now what I want you to imagine in this

play05:49

situation is that we're not just doing

play05:52

this sampling procedure once let's

play05:54

imagine that we do it 16 times so we do

play05:57

16 separate experiments here okay and

play06:01

let's imagine we get a result like this

play06:06

okay so imagine there are 16 experiments

play06:11

16 cases in which we draw samples of

play06:16

size 10 from each of the two bags and

play06:18

imagine that these are cases in which no

play06:21

chips have been removed from these two

play06:23

bags okay so let's imagine we draw ten

play06:27

chips out of this bag we draw ten chips

play06:30

out of this bag we calculate the mean of

play06:33

the numbers on the chips in this bag and

play06:35

the mean of the No

play06:36

the chips in this bag and we find the

play06:38

difference between the two sample means

play06:40

that's x-bar 1 minus x-bar to imagine

play06:43

that mean difference is 0.15 we then

play06:46

calculate a T value and get the

play06:50

Associated p-value the probability of

play06:53

this sample mean difference or something

play06:56

bigger

play06:57

given that the two samples have been

play06:59

drawn from bags with equal means which

play07:01

in this case is true because we are

play07:03

actually sampling from two bags with the

play07:05

same mean now let's imagine that in the

play07:07

first case that p-value is 0.82 so that

play07:10

means that there's a point 8 2 or 82

play07:13

percent chance of getting a sample mean

play07:16

of 0.15 or bigger if we've sampled from

play07:19

bags with the same means now that

play07:20

probability is very high so the

play07:23

probability of what we've observed is

play07:25

pretty high if the null is true

play07:28

therefore we don't reject the null we've

play07:31

observed something likely if the null is

play07:34

true so why would we reject the null so

play07:36

we don't reject the null now question is

play07:39

is that an error well in fact it's not

play07:41

an error we have sample from bags with

play07:44

the same means so the null is actually

play07:47

true in this case we have not rejected

play07:50

the null so we've not made an error

play07:52

okay now let's imagine we do this whole

play07:55

procedure again so we throw the chips

play07:57

back in the bag and we sample another 10

play07:58

chips from this bag and another ten

play08:00

chips from this bag we calculate the

play08:02

sample mean difference let's imagine

play08:04

it's minus point to calculate a p-value

play08:06

make our decision P is not less than our

play08:09

alpha point 1 so we do not reject the

play08:11

null and we have not made an error so

play08:14

here I've done this procedure 16 times

play08:16

and if we go down the list we see that

play08:19

there are only two cases here in which

play08:22

the p-value that we calculated the

play08:24

probability of the sample mean

play08:26

difference we observed is less than 0.1

play08:28

that's this case here and this case here

play08:33

now in this case the sample mean

play08:35

difference was 0.8 3 the difference

play08:37

between the sample means was relatively

play08:39

speaking quite large and if you're

play08:41

sampling from bags that have the same

play08:44

means you wouldn't expect to get large

play08:47

sample mean differences it's unlikely

play08:50

to get large sample mean differences so

play08:52

we've got a sample mean difference of

play08:54

0.8 3 here what's the probability of

play08:56

that 0.09 it's it's getting on to

play08:59

getting to the low side so because we've

play09:02

set an alpha point 1 we reject the null

play09:04

hypothesis that we have sampled from

play09:06

bags that have the same means but of

play09:09

course we're wrong because we have

play09:11

concluded that we've not sampled from

play09:13

bags with the same means and we actually

play09:15

have so that's an error now this kind of

play09:18

error is a type 1 error we have rejected

play09:21

the null hypothesis but the null

play09:23

hypothesis is actually true we sampled

play09:25

from bags with the same mean so that's a

play09:27

type 1 error there and if you look down

play09:29

we make another type 1 error here see

play09:31

here we got a pretty large sample mean

play09:33

difference we wouldn't expect that if

play09:34

we're sampling from bags with the same

play09:36

means and the chance of that is pretty

play09:38

low and so because it's less than our

play09:40

alpha point no one reject the null and

play09:42

we've made another type 1 error so if

play09:45

you look down the list here you see that

play09:47

we've made two type 1 errors now the

play09:50

probability of making a type 1 error is

play09:52

0.1 meaning we would make an a type 1

play09:54

error about 10 percent of the time

play09:56

now we've done 16 experiments here we've

play09:58

made 2 type 1 errors that's you know

play10:01

it's not 10 percent but it's around 10

play10:03

percent of the time that we've made a

play10:04

type 1 error so that result is what we

play10:07

would sort of expect here we've made

play10:09

type 1 errors about as often as we would

play10:11

expect to make them in this situation ok

play10:14

now let's imagine that we're in this

play10:16

second circum circumstance let's imagine

play10:19

that we are sampling from this bag one

play10:23

of the samples comes from this bag and

play10:24

the other sample comes from this bag so

play10:27

now we are not sampling from populations

play10:31

that have the same means now let's

play10:35

imagine that we went through 16

play10:37

experiments again so we did the sampling

play10:39

procedure 16 times and got these results

play10:42

okay so in the first case take 10 values

play10:45

from here take 10 values from here

play10:47

calculate the mean of the 10 values that

play10:50

came from this population the mean of

play10:52

the 10 values that came from this

play10:53

population calculate the mean difference

play10:55

imagine that the mean difference is 0.4

play10:58

3 now we calculate the p-value the

play11:00

probability of observing this sample

play11:03

mean

play11:03

difference if we'd sample from

play11:06

populations with the same means so this

play11:09

p-value that we've calculated here is

play11:11

the probability of seeing a sample mean

play11:13

difference of 0.4 3 had we drawn our

play11:16

samples from these two bags

play11:19

now that's 0.65 that's not less than

play11:22

alpha of 0.1 so we do not reject the

play11:25

null hypothesis now in this case that's

play11:27

an error we have not concluded that the

play11:30

null is false but the null is actually

play11:32

false because one of the samples came

play11:34

from a bag with a mean that is different

play11:36

than the other population so mu bag 1 is

play11:40

not equal to MU bag 2 in this case and

play11:43

so the null is false we haven't rejected

play11:45

the null and so we have made an error

play11:47

now this kind of error is an error in

play11:49

which you do not reject the null and the

play11:53

null is false that's a type 2 error so

play11:56

we've made a type 2 error here now if

play11:58

you go down to the next mean difference

play11:59

is 0.68 the p-value is 0.45 we do not

play12:02

reject the null again an error now here

play12:05

we got a mean difference of 0.92 so

play12:07

that's a fairly large mean difference

play12:09

the probability of getting a mean

play12:11

difference that large or larger if we

play12:13

sample from bags with the same means as

play12:15

point zero seven so it's unlikely to see

play12:18

a mean difference that big if we've

play12:20

sampled from bags with the same means

play12:22

so we conclude we have not sampled from

play12:25

bags from the same means we reject the

play12:27

null hypothesis now is that an error no

play12:30

it's not an error we've rejected the

play12:32

null and the null is false now if you go

play12:34

down the list here you'll see that we

play12:37

have only been correct three times out

play12:41

of 16 we've been incorrect 13 times out

play12:45

of 16 so the rate of error is 13 out of

play12:49

16 in this particular example that is

play12:54

analogous to beta so this is the

play12:59

probability here that we do not reject

play13:03

the null when the null is false so the

play13:06

problem is that we've made a lot of type

play13:09

2 errors and so the question is well why

play13:12

is that it doesn't seem that the

play13:14

decision procedure we're using is a very

play13:16

good decision

play13:17

procedure if it leads to so many errors

play13:20

okay well part of the reason for this is

play13:22

that the mean in the bag in which chips

play13:25

have been removed is not very different

play13:28

from the mean in the bag where chips had

play13:30

not been removed the effect size the

play13:33

difference between the population means

play13:35

is not very big and when the effect size

play13:39

is not very big beta is big another

play13:43

reason is that the sample size is small

play13:45

we only drew 10 values out of this bag

play13:47

and 10 values out of this bag so we have

play13:51

a combination of a small effect size and

play13:54

small sample size and that's what's

play13:57

causing us to have a large probability

play14:00

of a type 2 error here okay now to

play14:03

answer question number 25 then let's

play14:06

just take a look at it again

play14:07

why is beta more important than alpha in

play14:10

the real world but not in the population

play14:12

bag example we did in class we just need

play14:15

to understand the difference between

play14:16

this situation and a real-world

play14:18

situation now in this situation the null

play14:22

hypothesis can actually be true it can

play14:25

be the case that the means of the

play14:28

populations are exactly the same if you

play14:31

have six ones twos threes fours and

play14:33

fives here and six ones twos threes

play14:35

fours and fives here then the mean is

play14:38

exactly three in both cases so this can

play14:42

be true the two means can be exactly the

play14:44

same in this example but in the real

play14:49

world the means of two populations are

play14:51

never exactly the same let's imagine we

play14:54

have the null hypothesis that the mean

play14:56

of population 1 minus the mean of

play14:59

population 2 is 0 now we mean here

play15:03

exactly zero so zero point zero zero

play15:07

zero all the way out this is what's

play15:10

called a point hypothesis and a point

play15:12

type ah this is a specific exact

play15:14

mathematical statement of the difference

play15:17

between two parameters so here the

play15:19

statement is that the two parameters are

play15:21

exactly the same now if you imagine a

play15:23

real-world case that we would be

play15:24

imagining here let's say we imagine

play15:26

population one is taking some kind of a

play15:28

placebo and we imagine popular

play15:30

- is taking some kind of a drug now the

play15:35

hypothesis here is that even though

play15:38

everybody in this population took a

play15:40

placebo and everybody in this population

play15:42

took a drug their scores on some measure

play15:44

like for instance a depression measure

play15:46

are exactly the same they're not

play15:48

different by point zero zero zero zero

play15:50

one they're exactly the same now of

play15:53

course this drug has an active

play15:55

ingredient in it this does not have an

play15:57

active ingredient in it so these two

play15:59

populations are being treated

play16:00

differently so they may have a mean

play16:03

that's very similar to each other but

play16:05

they will not be exactly the same and

play16:08

that is because they have been treated

play16:10

differently now this is analogous to the

play16:13

idea we spoke about in class a long time

play16:15

ago

play16:16

which was the idea that a coin could be

play16:20

perfectly akley weighted on both sides

play16:23

so recall that we had a probability

play16:25

distribution like this with 0.5 here and

play16:29

PX here and the issue was whether or not

play16:33

this could describe a real coin or

play16:36

whether or not this is an idealization

play16:38

of the coin this is a model for a coin

play16:42

but couldn't actually perfectly

play16:44

represent a real coin now recall that I

play16:47

said that there is no real coin in the

play16:49

real world that is perfectly equally

play16:51

weighted on both sides so it cannot be

play16:54

that a null hypothesis like this that

play16:58

the probability of a head let's say

play16:59

equals 0.5 it cannot be that a point

play17:03

hypothesis like that is actually true in

play17:06

the real world because it can't be the

play17:08

case that there's such a thing as a real

play17:10

coin that is perfectly equally weighted

play17:11

on both sides this is an idealization of

play17:15

a coin just like this is an idealization

play17:18

of the difference between two

play17:20

populations okay so in the real world

play17:24

there is no such thing as two

play17:26

populations that have identical means

play17:29

but in the population bag example the

play17:33

two bags can have identical means so in

play17:37

an in an example like this it is

play17:39

possible to make a type 1 error it's

play17:42

possible to reject the null

play17:44

when the null is true because the null

play17:47

can be true so how many type 1 errors

play17:50

did we make here well we made 2 out of

play17:52

16 so this is our alpha roughly in this

play17:56

example so here it's possible to make

play17:59

this kind of error rejecting the null

play18:01

when the null is true in the real world

play18:04

though if the null hypothesis is never

play18:06

strictly speaking or explicitly true

play18:09

then it's not possible to make a type 1

play18:13

error you cannot reject the null when

play18:15

the null is true because the null is

play18:17

never perfectly true so the only kind of

play18:20

error that you can make in the real

play18:22

world is a type 2 error and that's why

play18:25

in the real world it's so important to

play18:27

manage type 2 errors not type 1 errors

play18:32

okay so that's the answer to question

play18:35

number 25 make up your own example and

play18:37

try to explain it in your own way ok

play18:40

well I hope that video helped and we'll

play18:42

see you in the next video

Rate This

5.0 / 5 (0 votes)

Related Tags
Hypothesis TestingPopulation BagAlpha ErrorBeta ErrorType 1 ErrorType 2 ErrorStatistical SignificanceSampling MethodEffect SizeSample SizeResearch Methodology