A statistics degree covers a great deal that most working analysts never touch. Meanwhile the four ideas that come up constantly are usually taught in the first six weeks and then never revisited.

Here is the short list, and what actually goes wrong with each.

Variation, before anything else

The first question about any number is how much it moves on its own. A conversion rate of 3.1 per cent this week against 2.8 per cent last week is not a change if the weekly figure has swung between 2.6 and 3.4 all year.

Plot the last twenty periods before you interpret the current one. This single habit prevents more bad decisions than any test.

Uncertainty as a range

A point estimate without a range is half an answer. “Roughly 14 per cent, somewhere between 11 and 17” tells a decision-maker what they need. “14 per cent” invites them to plan against a precision that does not exist.

You do not always need a formal confidence interval. A bootstrap of a thousand resamples takes four lines of Python and is easier to explain.

Base rates

If one per cent of transactions are fraudulent and your model flags one per cent of transactions with ninety per cent accuracy, most of what it flags is still legitimate. People find this genuinely surprising, including people who have done the calculation before.

Whenever a screening result comes across your desk, ask for the base rate first. It usually changes the conclusion.

Regression to the mean

The worst-performing stores improve after an intervention. So do the worst-performing stores with no intervention. Any analysis that selects an extreme group and measures it again needs a control, or it is measuring arithmetic rather than the programme.

What this leaves out

Plenty. Time series, causal inference, hierarchical models: all real, all useful, all specialist. The point is that they sit on top of these four, and a lot of analysts skip straight to the specialist tools while getting variation wrong.

The Data Science path spends its first three courses here, with no machine learning at all. That ordering is deliberate.