When to Use ANOVA

Running separate t-tests for each pair of groups inflates the family-wise error rate. With k groups, you have k(k−1)/2 pairs to compare. Each test at α = .05 has a 5% false-positive rate; across all pairs, the probability of at least one false positive grows rapidly. For example, with 4 groups and 6 pairs, your false-positive rate balloons to roughly 26%. ANOVA tests all groups simultaneously and keeps family-wise error at .05. If ANOVA is significant, you then use a corrected post-hoc test to find which pairs differ, controlling error across the family of comparisons.

The F-statistic is the ratio of between-group variance to within-group variance. A significant F means that the variance between groups is large relative to the variance within groups, suggesting that group membership explains some of the variation in your outcome. Specifically, you reject H₀: "all groups have the same mean" in favor of H₁: "at least one group mean differs from the others." It does not tell you which groups differ or how large the difference is—that is why post-hoc tests and effect sizes (like η²) matter.

Four main assumptions underpin one-way ANOVA: (1) independence of observations—each participant contributes one measurement per group, and observations are not correlated across participants; (2) normality of residuals—deviations from the group mean should be approximately normal; (3) homogeneity of variance—the spread of values should be similar across all groups, tested by Levene's test; (4) the outcome should be continuous (interval or ratio scale). Violations of normality and equal-variance assumptions are often tolerable, especially with balanced designs and large samples, but severe departures may require Kruskal-Wallis or Welch's ANOVA.

Mild inequality is not a problem. Use Levene's test to check: a non-significant result (p > .05) suggests equal variances. If Levene's test is significant and the largest variance is more than 3 times the smallest, consider alternatives. Welch's ANOVA is a robust variant that does not assume equal variances and is especially useful with unequal sample sizes. Unfortunately, our calculator does not include Welch's ANOVA; use R (function oneway.test(..., var.equal = FALSE)) or SPSS (Analyze → Compare Means → One-Way ANOVA → Options → Welch).

One-way ANOVA has a single categorical predictor (factor) that defines the groups. For example: three teaching methods, four age brackets, or five dosages. Two-way ANOVA has two categorical predictors, allowing you to test the main effect of each factor and their interaction. Example: teaching method (3 levels) × gender (2 levels) on exam score. You would test whether method affects score, whether gender affects score, and whether the effect of method differs by gender (interaction). Two-way ANOVA is more complex to compute and interpret, and our calculator handles one-way only. For two-way ANOVA, use R, SPSS, or consult a statistician.

Kruskal-Wallis is the non-parametric rank-based alternative to one-way ANOVA. Use it when your data violate the normality assumption severely (e.g., heavy skew, extreme outliers), when variances are very unequal despite attempted transformations, or when your outcome is ordinal (ranked categories rather than measurements). Kruskal-Wallis does not assume normality or equal variances and is based on ranks rather than means. It tests whether group locations differ; the null hypothesis is that all groups have the same distribution. A significant result warrants post-hoc rank-based pairwise comparisons or a Dunn test. Our Kruskal-Wallis calculator handles this option.