The theorem the whole subject stands on
Take samples of size n from any population, skewed, bimodal, whatever, and average each sample. Those averages form their own distribution: centred exactly on the population mean μ, with standard deviation σ/√n (the standard error), and shaped increasingly like a normal bell as n grows. That is the central limit theorem, and it is why tests and intervals for means work at all: we rarely know the population's shape, but we know the average's.
μ = 100, σ = 15, n = 36: the standard error is 15 ÷ 6 = 2.5. P(x̄ ≤ 105) = 0.9772: a sample mean of 105 sits two standard errors up, at the 98th percentile of what sampling alone produces. Compare a single observation: 105 is a third of an SD above the mean, utterly ordinary. Averaging shrank the noise sixfold, and ordinary became remarkable. That transformation is the entire engine of statistical inference.
The square-root law, again
Precision buys at square-root prices: quadruple the sample to halve the standard error. It governs margins of error, power calculations and this page alike, the most consequential square root in applied mathematics.
Applying the theorem to a single observation, or to n = 3 from a wild population. The bell arrives with n; for gentle populations n ≈ 10 is plenty, for savage ones (incomes, city sizes) n may need to be in the hundreds before the average behaves.
Frequently asked questions
Does the population need to be normal?
No, that is the theorem's miracle. Normality of the averages emerges from averaging itself. A normal population merely grants it instantly, at n = 1.
What if σ is estimated from the sample?
Then the standardised mean follows the t distribution instead, the same machinery with a small-sample tax, handled by the t-test pages.