A decision word, not a truth word
"Statistically significant" has a narrow technical meaning: the result crossed the threshold set by the significance level, so the null hypothesis is rejected. It says the data would be unusual under the null. It does not say the effect is large, important, useful or even real with any particular probability. The phrase misleads because everyday English hears "significant" as "important"; statistics means only "detectable by this study".
A correlation of r = 0.45 between practice scores and exam scores. With n = 20 pairs: t = 2.138, p = 0.0465: significant at 5%. With n = 12 pairs, the same r: t = 1.594, p = 0.142: not significant. The relationship did not change; the evidence did. Significance is a property of (effect, sample size, noise) together, never of the effect alone.
The two failure portraits
Significant but trivial: with very large samples, effects explaining a fraction of a percent of variance cross the line while mattering not at all. Important but not significant: a small pilot study of a genuinely better method returns p = 0.14 and gets shelved, when its interval estimate was wide rather than negative. Reading both numbers (effect size and p) is the only safeguard against both errors.
"Highly significant" for p = 0.0001 presented as "highly important". Smaller p means stronger evidence against the null, mostly bought with sample size. Importance lives in the effect size and its consequences, which p never measures.
What to write instead
Lead with the effect and its interval: "scores rose by 6.2 marks (95% CI 1.8 to 10.6)". Add the test result in its technical place. Your readers get the magnitude first and the detectability second, which is the order they actually need for decisions.
Frequently asked questions
Does significant mean the null is false?
It means the data are unusual enough under the null to reject it at the chosen level. Rejections are wrong at the alpha rate by design; significance is a decision under a stated error budget, not a proof.
What is the difference between statistical and practical significance?
Statistical significance asks whether an effect is detectable; practical significance asks whether it is worth acting on. A drug that shortens a cold by 20 minutes can be statistically significant in a big trial and practically irrelevant.
Why do results flip between significant and not across studies?
Sample sizes, noise and sampling luck differ. Near the threshold, results straddle the line naturally. Meta-analysis exists to pool such studies instead of counting their verdicts.