The number that was an average of averages

A district reported an average class score of 71 by averaging the averages of its eleven schools. The correct figure, calculated from every student, was 64.

The gap is not a rounding error and not a mistake in arithmetic. Averaging averages treats each school as equally important regardless of size, and the schools were not the same size. The two largest schools, with nearly half the district's students between them, had the lowest averages; the four smallest, with a tenth of the students, had the highest. Giving each school one vote lifted the district figure by seven points.

The technical name is a weighting error, and the fix is to weight each average by the number of students behind it. The arithmetic is simple. Noticing that it is needed is the hard part, because an average of averages looks like an average and arrives with the authority of a table.

The error has a direction, which makes it worse than random noise. Wherever small units differ systematically from large ones — small schools, small hospitals, small countries, small samples — an unweighted average will favour whatever the small units happen to be like. Small units also have more variable results, so they supply most of the extremes at both ends, and a league table built from them is largely a ranking of size.

The habit worth building is to look for the denominator. When a figure is an average, ask what each number in the average represents, and whether those things are comparable in size. If the answer is no, the average is a statement about the units rather than about the people inside them.