Statistical significance and practical importance answer different questions

A p-value concerns how incompatible the observed data are with a specified statistical model; it is not the probability that a hypothesis is true. Effect magnitude concerns how large the estimated difference is. Practical importance additionally depends on the decision context.

Illustrative example: a comparison estimates that a process saves three seconds per transaction. Even a small p-value cannot decide whether that saving justifies retraining, cost, or disruption. Conversely, an imprecise estimate of a substantial saving should not be summarized as “no difference” merely because a threshold was missed.

Before reading the result, define the smallest difference that would change the decision. Then report the effect in interpretable units, an uncertainty interval, design assumptions, and the p-value if appropriate. Disclose which analyses were examined rather than presenting a selected threshold crossing alone.

This distinction matters when readers must choose an action rather than merely classify a test result. Neither an effect estimate nor a p-value repairs biased measurement or an unsuitable design.

Evidence status: source-based synthesis. Examples are illustrative; no new experiment or independent outcome verification is claimed.