Hypothesis Tests II

STAT 20: Introduction to Probability and Statistics

Warmup

10:00

Which pair of plots would have the greatest chi-squared distance between them? (consider one of them the “observed” and the other the “expected”)

• This question recalls the introduction of a new summary statistic in the notes: the chi-squared. It is used to measure the distance between two distributions of categorical variables.

• The pair with the highest statistic is D and F!

• The important mathematical intuition to have here is that the statistic adds together every squared normalized difference, so one very large difference can inflate the statistic.

Chi-squareds Compared

\[ \frac{(1-1)^2}{1} + \frac{(10 - 1)^2}{1} + \frac{(1 - 10)^2}{10} \\ 0 + 81 + \frac{81}{10} = 89.1 \]

\[ \frac{(3-5)^2}{5} + \frac{(4-4)^2}{4} + \frac{(5-3)^2}{3} \\ \frac{4}{5} + 0 + \frac{4}{3} = 2.13 \]

Warmup Activity

We will be collecting data from this class right now! Sit tight and wait for everyone to get a warmup sheet.

This sets up a discussion of how to set up a hypothesis test that p = .5.

This is based on a series of experiments that show a link between sounds and shapes in people’s minds - sharper shapes correspond to sharper sounds (k, t) and rounder shapes to rounder sounds (m, b). Read more about it here: https://en.wikipedia.org/wiki/Bouba/kiki_effect.

You conduct the experiment on the students using a poll question, then you calculate a proportion, then entertain the notion that linking names to shapes is meaningless, so people were choosing randomly, then use that to motivate the hypothesis test that p = .5.

Which of the two names belongs to the shape on the left?

Which of the two names belongs to the shape on the left?

• Bouba
• Kiki

00:15

Make sure that this people think about this quietly, without consulting their neighbor.

Once the 15 seconds is up, here is one way to collect data.

“Everyone, close your eyes. Raise your hand if you think the shape on the left is named ‘bouba’ [count students]. Raise your hand if you think the shape on the right is named `kiki’ [count students]. Everyone, open your eyes.”

To ensure some level of suspense, make sure that the same amount of time is spent during the two counting stages, regardless of how much time is taken to actually count everyone up in each stage (most people will say ‘Kiki’).

Steps of a Hypothesis Test

1. Assert a model for how the data was generated (the null hypothesis)
2. Select a test statistic that bears on that null hypothesis (a mean, a proportion, a difference in means, a difference in proportions, etc).
3. Approximate the sampling distribution of that statistic under the null hypothesis (aka the null distribution)
4. Assess the degree of consistency between that distribution and the test statistic that was actually observed (either visually or by calculating a p-value)

The Null Hypothesis

• Let \(p_k\) be the probability that a person selects Kiki for the shape on the left.
• Let \(\hat{p}_k\) be the sample proportion of people that selected Kiki for the shape on the left.

Write down an equation for the null hypothesis that corresponds to the notion the link between names and shapes is arbitrary.

Select a test statistic

\[\hat{p}_k = \frac{\textrm{Number who chose "Kiki"}}{\textrm{Total number of people}}\]

Note: you could also simply \(n_k\), the number of people who chose “Kiki”.

A follow-up question to ask: “For our class, what would we expect this number to be if \(p_k = .5\)?”

It should be half the number of students who responded to the poll.

Approximate the null distribution

Our technique: simulate data from a world in which the null is true, then calculate the test statistic on the simulated data.

Which simulation method(s) align with the null hypothesis and our data collection process of simulating from a world where the null is true?

• Make a box with two cards, one that says “Kiki”, the other that says “Bouba”. Draw out n cards (the number of people who filled out the poll) with replacement, then compute \(\hat{p}_k\).

• Make a box with two cards, one that says “Kiki”, the other that says “Bouba”. Draw n cards without replacement, then compute \(\hat{p}_k\).

• Make a box with n cards, half of which say “KiKi”, the other half of which say “Bouba”. Draw n cards with replacement and calculate \(\hat{p}_k\).

• Make a box with n cards, half of which say “KiKi”, the other half of which say “Bouba”. Draw n cards without replacement and calculate \(\hat{p}_k\).

• Take a fair coin and assign Heads to “Kiki” and Tails to “Bouba”. Flip the coin n times and calculate \(\hat{p}_k\).

A, C, and E all work.

Approximate the null distribution

• Make a box with two cards, one that says “Kiki”, the other that says “Bouba”. Draw out n cards (the number of people who filled out the poll) with replacement, then compute \(\hat{p}_k\).

• Make a box with two cards, one that says “Kiki”, the other that says “Bouba”. Draw n cards without replacement, then compute \(\hat{p}_k\).

• Make a box with n cards, half of which say “Kiki”, the other half of which say “Bouba”. Draw n cards with replacement and calculate \(\hat{p}_k\).

• Make a box with n cards, half of which say “Kiki”, the other half of which say “Bouba”. Draw n cards without replacement and calculate \(\hat{p}_k\).

• Take a fair coin and assign Heads to “Kiki” and Tails to “Bouba”. Flip the coin n times and calculate \(\hat{p}_k\).

• The first, third, and fifth answer choices are valid options. Each of these options make sure that, for each of the n times we simulate a person’s decision, the probability of their decision being that the left shape is “Kiki” is 50 percent.

3. Approximate the null distribution

Let’s now use infer to do the simulation!

4. Assess the consistency of the data and the null

What is the proper interpretation of the p-value?

01:00

The Bouba / Kiki Effect

Break

05:00

Worksheet: Hypothesis Tests II

30:00