An examination intended to measure comprehension was found, after analysis, to be measuring reading speed. The questions were sound. There were simply too many of them for the time allowed.
Under time pressure the strongest predictor of a candidate's score becomes how fast they can get through the paper, and reading speed correlates with comprehension only moderately. It correlates rather better with how much reading a student has done outside school, which in turn tracks household circumstances. An examination of this design produces a result that looks like a measure of understanding and functions partly as a measure of background.
The fix is not simply to extend the time, though that helps. It is to decide what the paper is for and then make the timing a deliberate choice rather than an accident. A test of fluency should be timed tightly and should say so. A test of comprehension should give enough time that nearly all candidates finish, so that differences in score come from differences in understanding. Both are legitimate; the failure is building one and reporting it as the other.
Examining boards check this with a simple statistic. If the proportion of candidates who leave the final question blank is high, and if those candidates do well on the questions they did reach, the paper is too long rather than too hard. The two failures look identical in an average mark and need opposite responses.
The practical advice for a candidate is unchanged by any of this: find out which kind of paper you are sitting, and if it is the long kind, practise the pacing explicitly, because the pacing is part of what is being tested whether or not anyone intended it.
Vocabulary in this text
- measure /ˈmɛʒɚ/ — to find the size or amount of something
- comprehension /ˌkɑmpɹiˈhɛnʃən/ — the ability to understand what you read or hear
- analysis /əˈnæləsəs/ — a careful study of something to understand it
- candidate /ˈkændədeɪt/ — a person being considered for a job or prize
- correlate /ˈkɔɹəˌleɪt/ — to have a relationship in which things change together
- result /ɹɪˈzʌlt/ — what happens because of something
- deliberate /dɪˈlɪbɚət/ — done on purpose, or slow and careful
- choice /ˈtʃɔɪs/ — the act of choosing, or the thing chosen
- fluency /ˈfluənsi/ — the ability to speak or read smoothly and easily
- enough /ɪˈnʌf/ — as much as you need