Comparing two independent means
Two separate groups, one question: do their population means differ? The two-sample t-test divides the observed gap between the group means by the standard error of that gap. This calculator defaults to Welch's version, which lets each group keep its own variance; the pooled version, which assumes one shared variance, is one click away for courses that teach it that way.
Two classes take the same quiz. Class 1: 78, 82, 85, 90, 88, 84, 79, 91 (mean 84.625, s = 4.8385). Class 2: 72, 75, 80, 77, 74, 79, 73, 76 (mean 75.75, s = 2.8158). Welch's t = 4.484 on about 11.3 df, two-tailed p = 0.0009. The 95% interval for the difference runs 4.53 to 13.22 marks. Class 1's true mean is higher, by a margin that matters, not a rounding error. Cohen's d ≈ 2.2, a very large standardised gap.
Why Welch is the default
The pooled test's extra assumption (equal population variances) is exactly the kind of thing that is false precisely when you cannot check it well. When variances really are equal, Welch loses almost nothing (a slightly fractional df). When they are not, pooling can distort the standard error and the p-value with it. Modern practice: Welch unless a course rubric demands pooled.
Assumptions, in plain sight
- Independence between groups: different people in each group. Same people twice? That is a paired test, and it is stronger.
- Independence within groups and roughly normal data in each, especially for small samples.
- No pairing hidden in the design: matched or repeated measurements analysed as independent waste power and misstate the standard error.
"The two confidence intervals overlap, so no difference." Wrong instrument. Judge the gap by the interval for the difference, which this page reports.
Frequently asked questions
Pooled or Welch: which will my exam expect?
Older textbooks teach pooled with df = n1 + n2 − 2. If your course does, use the pooled toggle; the numbers will match the textbook method exactly.
My df is a decimal. Is that a bug?
No. Welch–Satterthwaite degrees of freedom are naturally fractional. Report them as computed (e.g. t(11.3) = 4.48).
What if the groups are the same people measured twice?
Use the paired t-test. Pairing removes person-to-person variation from the comparison and is far more powerful.