Data Analysis
ANOVA: One-Way, Two-Way and Repeated Measures Explained
GOSPELTRADER Research Desk · 11 September 2026 · 12 min read
Quick answer
ANOVA tests whether three or more group means differ by comparing variance between groups with variance within groups (F = MS_between / MS_within). A significant F says at least one mean differs; a post hoc test such as Tukey's HSD identifies which pairs, and eta squared reports how much of the outcome variance the grouping explains.
The logic and the formulas
ANOVA partitions total variability into a component explained by group membership and a residual component. If groups genuinely differ, between-group variance is large relative to within-group noise and F grows.
Partition
SS_total = SS_between + SS_within
SS_between = Σ nⱼ (x̄ⱼ - x̄)² df₁ = k - 1
SS_within = Σ Σ (xᵢⱼ - x̄ⱼ)² df₂ = N - k
MS_between = SS_between / (k - 1)
MS_within = SS_within / (N - k)
F = MS_between / MS_within
Effect size
η² = SS_between / SS_total
ω² = (SS_between - (k-1)MS_within) / (SS_total + MS_within)
Benchmarks: η² = .01 small, .06 medium, .14 largeWhich ANOVA design do you need?
The number of factors and whether the same participants are measured repeatedly decide the design.
| Design | Use when | Key output |
|---|---|---|
| One-way ANOVA | One factor, 3+ independent groups | F(k-1, N-k), η² |
| Two-way (factorial) ANOVA | Two factors; interaction of interest | Two main effects + interaction |
| Repeated measures ANOVA | Same participants measured 3+ times | Within-subjects F, sphericity test |
| Mixed ANOVA | One between and one within factor | Between, within and interaction effects |
| ANCOVA | Control a continuous covariate | Adjusted means, F |
| MANOVA | Two or more correlated outcomes | Wilks' Lambda, then univariate tests |
Worked one-way example
Three teaching methods are compared on an examination score; n = 20 per group, N = 60.
Group means: Lecture 62.4 (SD 8.1) | Blended 70.1 (SD 7.6) | Flipped 74.8 (SD 8.4)
Source SS df MS F p η²
Between 1,568.3 2 784.2 11.94 <.001 .29
Within 3,940.1 57 69.1
Total 5,508.4 59
Tukey HSD
Flipped > Lecture : MD = 12.4, p < .001, 95% CI [5.8, 19.0]
Blended > Lecture : MD = 7.7, p = .014, 95% CI [1.3, 14.1]
Flipped vs Blended: MD = 4.7, p = .186 (ns)
APA sentence
Teaching method significantly affected examination scores,
F(2, 57) = 11.94, p < .001, η² = .29. Tukey post hoc tests showed the
flipped (M = 74.8) and blended (M = 70.1) methods outperformed the
traditional lecture (M = 62.4); the two active methods did not differ.Assumptions, and what to do when they fail
ANOVA is robust to moderate departures from normality when group sizes are equal and reasonably large, but it is not robust to unequal variance with unequal n.
- • Independence of observations — guaranteed by design, not by a test.
- • Normality within each group — Shapiro-Wilk plus Q-Q plots; with n ≥ 30 per group the central limit theorem protects the F-test.
- • Homogeneity of variance — Levene's test. If violated, use Welch's ANOVA with the Games-Howell post hoc.
- • Sphericity (repeated measures only) — Mauchly's test. If violated, apply the Greenhouse-Geisser correction when ε < 0.75, otherwise Huynh-Feldt.
- • If normality fails badly, use Kruskal-Wallis (between-subjects) or Friedman (repeated measures).
Post hoc tests: choosing correctly
Running multiple t-tests instead of a post hoc procedure inflates the family-wise error rate: with three tests at α = .05 the chance of at least one false positive is 1 - 0.95³ ≈ 14%.
| Post hoc test | Best used when | Character |
|---|---|---|
| Tukey HSD | Equal n, equal variances, all pairwise comparisons | Balanced power and control |
| Bonferroni | Few planned comparisons | Conservative |
| Games-Howell | Unequal variances or unequal n | Robust |
| Scheffé | Complex contrasts, not just pairs | Very conservative |
| Dunnett | Comparing each group to one control | Higher power for that design |
Interpreting an interaction in two-way ANOVA
When the interaction is significant, main effects must not be interpreted on their own — the effect of one factor depends on the level of the other. Plot the estimated marginal means and run simple effects: test factor A separately at each level of factor B, applying a Bonferroni adjustment to those follow-up tests.
Frequently asked questions
Why not run multiple t-tests instead of ANOVA?
Because each test carries its own Type I error risk. Comparing four groups requires six t-tests, giving roughly a 26% chance of at least one false positive at α = .05. ANOVA tests all groups within a single family-wise error rate, and post hoc procedures then control the pairwise comparisons.
What is the difference between eta squared and partial eta squared?
Eta squared divides an effect's sum of squares by the total sum of squares, so all effects sum to at most 1. Partial eta squared divides by that effect plus its error term only, which yields larger values and is standard in factorial and repeated measures designs. State which one you report.
Can ANOVA be used with unequal group sizes?
Yes. Unequal n is acceptable, but it makes the test sensitive to unequal variances. Check Levene's test carefully and switch to Welch's ANOVA with Games-Howell post hoc comparisons if variances differ.
What sample size do I need per group?
For a medium effect (f = 0.25) with three groups, 80% power and α = .05, you need roughly 52 participants per group, about 156 in total. Detecting a large effect (f = 0.40) needs about 21 per group. Run the power analysis before collecting data, not after.
Want this analysis done for you?
We deliver cleaning, assumption testing, modelling and a reporting-ready write-up.