library(ggplot2)
library(dplyr)
library(mosaic)

# Simulate many, many, many times someone guessing at random
guess_cup <- c(1, 0)
simulation <- do(10000) * resample(guess_cup, size=8)

# Count number correct
simulation <- simulation %>% 
  mutate(n_correct = V1 + V2 + V3 + V4 + V5 + V6 + V7 + V8) 

# Plot
ggplot(simulation, aes(x=n_correct)) + 
  geom_bar() +
  labs(x="Number of Guesses Correct") +
  geom_vline(xintercept=8, col="red", size=1) 

• We have two competing hypotheses:
  • \(H_0\): the taster is guessing at random
  • \(H_A\): the taster is not. i.e. she can really tell which came first
• The test statistic used here is: # of correct guesses
• The observed test statistic is 8 correct (out of 8)
• We going to assume \(H_0\) is true and see how likely the observed test statistic was.

Not very likely!

• The bar chart of the 10000 simulations of 8 random guesses is the null distribution.
• Formally: it is the distribution that describes the random behavior of the test statistic assuming \(H_0\) is true.

p-value

• Statistically: the probability of observing a test statistic as extreme as the one observed assuming \(H_0\) is true.
• Conceptually: measures our degree of surprise (lower = more surprised) of what we observed assuming \(H_0\) is true.
• Ex: how unlikely is it that the taster got 8 out of 8 right assuming she was just guessing at random

Code combines the most difficult ggplot2/dplyr concepts:

• counting the different values of n_correct using n() instead of sum()
• dividing by the number of simulations 10000
• using geom_bar() when we have an explicity y aesthetic prop_correct, so we set stat="identity"

simulation <- simulation %>% 
  group_by(n_correct) %>%
  summarise(prop_correct = n()/10000)

ggplot(simulation, aes(x=n_correct, y=prop_correct)) + 
  geom_bar(stat="identity") +
  labs(x="Number of Guesses Correct", y="Proportion") +
  geom_vline(xintercept=8, col="red", size=2) 

The proportion of times 8 occurs is 0.0050 = 0.5%.

p-Value meaning:

• Conceptually: Assuming she was guessing at random, we are very surprised at observing 8 correct guesses
• Statistically: probability of observing a test statistic of 8 assuming \(H_0\) is true is approx. 0.5%. i.e. TINY

Study Conclusion:

• Conceptually: She was probably NOT guessing at random; she probably could tell.
• Statistically: We reject \(H_0\) in favor of \(H_A\)

Say I give you a data set with two variables:

• Even or odd number of letters in last name i.e. two groups
• Test score

Think about how you would test:

• \(H_0\): no difference in test scores between odd vs even
• \(H_A\): there is a difference

Hints:

• Think about what assuming \(H_0\) allows you to do the data set
• Question from homework about using one of the mosaic package sampling tools