The definition, without the fog
A p-value is a probability computed assuming the null hypothesis is true: the probability of data at least as extreme as what you observed, in the direction of the alternative. Small p means the data would be surprising under the null. That is all it means. It is not the probability that the null is true, not the probability of a fluke in plain language, and not the size or importance of an effect.
A bakery claims its loaves average 40 (units of 10 g). Eight loaves sampled average 44.5 with s = 4.75. The one-sample t statistic is 2.679 on 7 df, and p = 0.0316. Translation: if the true mean really were 40, samples of 8 would average 44.5 or further from 40 only about 3.2% of the time. The data are unusual under the claim, so at the 5% level the claim is rejected. The p-value said nothing about the probability that the claim itself is true.
What p depends on (besides the effect)
Sample size, heavily. The same effect in the same units produces steadily smaller p as n grows: a trivially small real difference becomes "highly significant" with enough data, while a large, important effect in a tiny sample returns p = 0.2. This is why the effect size and its confidence interval must travel with p. For the bakery, the interval estimate is a mean between about 40.5 and 48.5, which tells the owner something actionable; p = 0.032 alone does not.
Three classic misreadings. (1) p = 0.049 vs 0.051 treated as opposite conclusions, though the evidence is nearly identical. (2) p > 0.05 read as proof of no effect, when it is often just a small sample failing to resolve one. (3) Hunting through many outcomes and reporting the one small p: with 20 independent tests at 0.05, one "significant" result is expected from pure chance.
Reporting p well
Give the exact value (p = 0.032, not p < 0.05), the test and its statistic, the effect size with a confidence interval, and the sample size. Read that way, p is a useful gauge of surprise under a stated model. Read alone, as a stamp that says TRUE, it misleads exactly as its critics charge.
Frequently asked questions
Does p = 0.03 mean there is a 97% chance the effect is real?
No. The p-value conditions on the null being true and describes the data's extremeness under it. The probability that the effect is real depends on prior plausibility and study quality, which p does not contain.
Why is 0.05 the standard threshold?
Convention, popularized by Fisher as a convenient line, not a law of nature. Fields set their own: particle physics demands far smaller p; exploratory work often accepts 0.10 with labels attached. Treat the threshold as a decision rule you state, not a fact you discover.
Can a p-value be exactly 0?
No. It can be astronomically small (software prints 0 or < 0.001 when it underflows). Report it as p < 0.001 rather than p = 0.000, which claims an impossibility.
What should I report instead of just p?
The effect size with its confidence interval, the test statistic and degrees of freedom, the sample size, and the exact p. That set lets a reader judge both existence and importance.