The tenuous link

Headlines connecting a food to a disease appear several times a month. Most rest on a link that is real, measurable and far weaker than the sentence suggests.

Consider a study reporting that people who eat a particular food have a thirty per cent higher rate of a condition. Three questions decide whether that sentence means anything.

First, thirty per cent of what. If the condition affects one person in two thousand, a thirty per cent rise takes it to about one in fifteen hundred. The relative figure is large and the absolute change is almost invisible.

Second, what else differs between the groups. People who eat more of almost any food differ from those who eat less in dozens of ways: income, exercise, age, whether they smoke. Studies adjust for the differences they thought to measure. They cannot adjust for the ones nobody recorded, and a confounding factor that was never written down is indistinguishable from a cause.

Third, how many questions were asked. A study that examines one food and one disease produces one answer. A study that examines forty foods and twenty diseases produces eight hundred comparisons, and at ordinary standards of statistical significance roughly forty of them will look striking purely by chance. Reporting the striking ones and omitting the rest is not fraud; it is the normal operation of a news cycle applied to a body of work.

None of this means nutrition research is worthless. It means a single study is a question, not an answer, and the appropriate response to one is interest rather than dietary change.