What a test can and cannot measure

A test is a measuring instrument, and like any instrument it has a range. A kitchen scale is excellent for flour and useless for weighing a lorry. The mistake is not building the scale; it is forgetting where its range ends.

Written examinations measure some things very well. They are reliable at checking whether a student can recall a definition, apply a known procedure, or write clearly under time pressure. These are real abilities, and a system that ignored them would be doing students no favours.

The trouble begins at the edges. A single paper is poor at measuring how a student works with others, how they respond when a first attempt fails, and whether they can find a problem worth solving rather than solve one handed to them. It is also sensitive to things that have nothing to do with knowledge: whether the student slept, whether the room was hot, whether the question happened to use a context they had seen before.

None of this makes examinations worthless. It makes them partial. The honest position is to say what a result does and does not tell you. A mark of seven out of ten says something about performance on that paper on that morning. It does not say that the student is a seven.

Schools that take this seriously tend to do two things. They use more than one kind of evidence — coursework, projects, practical tasks — so that different strengths have somewhere to show. And they are careful with the language they use about results, because a number repeated often enough stops being a measurement and starts being an identity.