Hypothesis Testing

STAT 20: Introduction to Probability and Statistics

Warmup

10:00

For the following pairs of statements:

• Identify which is the null hypothesis and which is the alternative hypothesis.

• Write out the null hypothesis in equation form.

Statement 1

• The mean length of African elephant tusks has not changed over the last 100 years.

• The mean length of African elephant tusks has changed over the last 100 years.

• Between the two statements, the first one is the null hypothesis. Assuming that the means of the two distributions of tusks is the same allows us to simulate data using permutations! This makes the second statement the alternative hypothesis.

• In equation form, we can write \(\mu_{new} = \mu_{old}\) for the null hypothesis \(H_0\). \(\mu_{new}\) is the mean length of tusks as taken today, while \(\mu_{old}\) is the mean length of tusks as taken 100 years ago.

• Rearranging gives \(\mu_{new} - \mu_{old} = 0\).

Statement 2

• The rate of facial clefts is not equal for babies born to mothers who take folic acid supplements compared with those from mothers who do not.

• The rate of facial clefts is equal for babies born to mothers who take folic acid supplements compared with those from mothers who do not.

• Between the two statements, the second one is the null hypothesis. If we assuming that the facial cleft rates between the two distributions are the same, we can again use permutations to simulate data with this statement. This makes the first statement the alternative hypothesis.

• In equation form, we can write \(p_{\text{folic}} = p_{\text{no folic}}\) for the null hypothesis \(H_0\).

• \(p_{\text{folic}}\) is the proportion of babies that developed facial clefts after being born form a mother who has taken supplements, while \(p_{\text{no folic}}\) is the same proportion except when the mother has not taken the supplement.

• Rearranging gives \(p_{\text{folic}} = p_{\text{no folic}}\) for the null hypothesis \(H_0\).

A pharmaceutical company developed a new treatment for eczema and performed a hypothesis test to see if it worked better than the company’s old treatment. The P-value for the test was \(10\%\). Which one of the following statements is true/is a valid conclusion that can be made using just this information?

A. The probability that the null hypothesis is false is \(10\%\).

B. The P-value of about \(10\%\) was computed assuming that the null hypothesis was true.

D. The alternative hypothesis is 10 times more likely the null.

The p-value is the probability that, assuming the null hypothesis is true, we obtain a test statistic value that is equal to (or more extreme) to the value of the one we observed.

Let’s assume that the test statistic is the difference in the proprortion of patient improvement between the new and old drugs. In the context of this problem, it’s the probability that, assuming the new and old eczema drugs work the same, of observing the rate of improvement/disimprovement in our data or something more drastic.

Based on this definition, we can choose between the options.

A. This statement is false. Our definition of the p-value does not match with this.

B. This statement is true. In practice, we calculate the p-value by taking the proportion of simulated test statistics equal to (or more extreme) than the one we observed. We wouldn’t be able to get any simulated statistics without our assumption about how the data was generated: in other words, without the null hypothesis.

C. Just as in statement C, the definition of the p-value does not match with this statement. In general, with our setup, we cannot speculate about the probabilitiy of either hypothesis being more likely than the other.

Concept Questions (if time permits)

Pick a Card Activity

Goals: 1. Identify a situation where people naturally reason like a hypothesis test 2. Map the parts of that situation to the formalism of a hypothesis test 3. Provide some intuition about why .05 is used as the standard threshold

Take two decks of cards and mix them so that you have a single deck that is all of one color, say: black. Bring a prize to class and ham it up that the winner of this simple game will win the prize. All they have to do is pick a red card.

Shuffle the deck, say that’s its fair, then select a random student to be the first contestant. Fan the cards out for them and have them pick a card. When the lose, fein disappointment / surprise, replace the card and move on to the next student. Do this several times. You’ll start hearing groans that the game is rigged. At the point where the groans become clearly articulated (usually around 4 or 5 draws), stop the game, admit to the scam, then go to the board.

Walk through how students started with a previous notion that the deck was fair (null hypothesis: p(red) = .5). As they observed data, they assessed the degree of consistency with that notion by computing the probability of the data. Calculate a table of the binomial probabilities of one black, two blacks, three blacks, on the board. With 4 blacks, the probability is 1/2^4 = .0625 and with 5 blacks its 1/2^5 = .03125. Draw a line between these two probability and identify it as the alpha = .05 threshold that is generally used for “calling bullshit on the null”.

Break

05:00

Worksheet: Hypothesis Tests

• Today’s worksheet is all about answering one question:

Is yawning contagious?

What do you think?

Answer at pollev.com.

Worksheet: Hypothesis Tests

• Now, we will watch a short excerpt from a Mythbusters episode in which the Mythbusters crew sets up, performs and analyzes the results of an experiment designed to test whether yawning is contagious.
• https://youtu.be/cscvxfTyxNM?si=2FJwL-fOndqIxa7a&t=237

What do you think about the experiment in the video?

• Discuss with your group.

01:00

Worksheet: Page 1

20:00

Worksheet: Page 2

20:00

Do you still think yawning is contagious?

Answer at pollev.com.

Lab 9: People’s Park II

25:00