Class #21: ANOVA

1. Inflated Type I error: For each test, there’s a α=5% probability that the difference between groups is not meaningful (i.e. due to chance).
If we have n groups to compare, and we keep running t-tests, there will be 1-(1-α)ⁿ chance to make a Type I error!

2. To control for Type I error: Make α smaller; Bonferroni adjustment (divide α by the number of tests). However, Type II error will be larger, power will be lower (more difficult to get a significant result when there is an effect).

3. ANOVA (Analysis of Variance) to-the-rescue: used to analyze more than 2 levels of 1 IV. Answering “Is there any set of means that differs from one another”?

F = Between Groups Variance (Signal) / Within Groups Variance (Noise)
Between Variance = Effect of IV + Individual difference + Error
Within Variance = Individual difference + Error

If H₀ is true (i.e. Effect of IV = 0), then F ≈ 1. The larger the F, the more likely the true effects exist.

4. ANOVA:

• Can stop Type I error inflation while not losing power
• Can tell if there’s significant main effects or interaction
• Can NOT tell which group is causing the difference (one or more of the groups have an effect, but don’t know which)
  • Planned comparisons
  • Post-Hocs