Wrong By Design

STAT 20: Introduction to Probability and Statistics

Agenda

• Announcements
• Concept Questions: Wrong By Design
• Break
• Worksheet: Wrong By Design
• Break
• Lab 3: People’s Park

Announcements

• Lab 3 (both parts) Due Friday. Two parts, one pdf!

• Quiz 3 on Thursday.

• Portfolio 6 (Mon., Tue., Wed.) due Friday.

Concept Questions

Read this first.

A patient named Ana was diagnosed with Fibromyalgia, a long-term syndrome of body pain, and was prescribed anti-depressants. Being the skeptic that she is, Ana didn’t initially believe that anti-depressants would help her symptoms. However, after a couple months of being on the medication she would like to re-evaluates this belief.

00:40

Let Ana’s null hypothesis be that anti-depressants would not help her symptoms. Describe a Type I error.

• A: Deciding that the anti-depressants helped her symptoms when in fact they did not help.
• B: Deciding that the anti-depressants helped her symptoms when they did help.
• C: Deciding that the anti-depressants did not help her symptoms when in fact they did help.
• D: The correct answer is not here.

00:40

Let Ana’s null hypothesis be that anti-depressants would not help her symptoms. Describe a Type II error.

• A: Deciding that the anti-depressants helped her symptoms when in fact they did not help.
• B: Deciding that the anti-depressants did not help when they did not help.
• C: Deciding that the anti-depressants helped her symptoms when they did help.
• D: The correct answer is not here.

00:30

Read this first.

A study of 40 California voters failed to find any significant difference in the proportion that supported increased funding for NASA when comparing Republicans to Democrats (that is, the study retained the null hypothesis).

Based on just the information above, which of the statements below do we know is false?

• A: The sample of California voters may have been biased.
• B: The study may have had lower power as a result of the small sample size.
• C: The researchers may have committed a Type I error.
• D. The researchers may have committed a Type II error.

00:40

Lecture

• Error Rates, Alpha Level, and Power

Break

05:00

Worksheet: Wrong by Design

30:00

Break

05:00

Lab 3: People’s Park

30:00

Concept Questions

Instead of constructing a confidence interval to learn about the parameter, we could assert the value of a parameter and see whether it is consistent with the data using a hypothesis test. Say you are interested in testing whether there is a clear majority opinion of support or opposition to the project.

What are the null and alternative hypotheses?

01:00

The null is the p = .5 and the alternative is that p != .5.

This brings up a good discussion of one- and two-tailed tests. When students share their answers, you can draw a picture of a null distribution on the board with the observed statistic as a vertical line and consider how the p-value calculation would change depending on the alternative hypothesis.

This also brings up a discussion of the link between hypothesis tests and confidence intervals. Below the picture of the null distribution on the board, you could draw a bootstrap distribution centered on the vertical line of the observed statistic. In the HT setting, you consider the location of the statistic relative to the null distribution. In making a decision with a CI, you consider the location of the parameter relative to the bootstrap distribution (or more generally, the sampling distribution of the statistic).

library(tidyverse)
library(infer)
library(stat20data)

ppk <- ppk |>
  mutate(support_before = Q18_words %in% c("Somewhat support", 
                                          "Strongly support",
                                          "Very strongly support"))

library(tidyverse)
library(infer)
library(stat20data)

ppk <- ppk |>
  mutate(support_before = Q18_words %in% c("Somewhat support", 
                                          "Strongly support",
                                          "Very strongly support"))
obs_stat <- ppk |>
  specify(response = support_before,
          success = "TRUE") |>
  calculate(stat = "prop")

library(tidyverse)
library(infer)
library(stat20data)

ppk <- ppk |>
  mutate(support_before = Q18_words %in% c("Somewhat support", 
                                          "Strongly support",
                                          "Very strongly support"))
obs_stat <- ppk |>
  specify(response = support_before,
          success = "TRUE") |>
  calculate(stat = "prop")
obs_stat

Response: support_before (factor)
# A tibble: 1 × 1
   stat
  <dbl>
1 0.339

null <- ppk |>
  specify(response = support_before,
          success = "TRUE") |>
  hypothesize(null = "point", p = .5) |>
  generate(reps = 500, type = "draw") |>
  calculate(stat = "prop")

null <- ppk |>
  specify(response = support_before,
          success = "TRUE") |>
  hypothesize(null = "point", p = .5) |>
  generate(reps = 500, type = "draw") |>
  calculate(stat = "prop")
null

Response: support_before (factor)
Null Hypothesis: point
# A tibble: 500 × 2
   replicate  stat
       <int> <dbl>
 1         1 0.527
 2         2 0.484
 3         3 0.524
 4         4 0.498
 5         5 0.518
 6         6 0.502
 7         7 0.491
 8         8 0.492
 9         9 0.484
10        10 0.491
# ℹ 490 more rows

null <- ppk |>
  specify(response = support_before,
          success = "TRUE") |>
  hypothesize(null = "point", p = .5) |>
  generate(reps = 500, type = "draw") |>
  calculate(stat = "prop")
visualize(null) +
  shade_p_value(obs_stat, direction = "both")

What would a Type I error be in this context?

Answer: Concluding that there is a clear majority one way or another when in fact there is an even split.

01:00

What would a Type II error be in this context?

Answer: Concluding that there is an even split when in fact there is a clear majority.

After the previous question, students should get this quite easily.