The Test Set Was Wrong

The benchmarks used to rank a decade of models contain thousands of mislabelled examples. Correcting them changes which model wins.

For most of the past decade, the ImageNet validation set has been the yardstick of computer vision. Fifty thousand images, each with a single correct label, against which thousands of models have been scored and ranked. Papers were accepted, architectures adopted and careers advanced on differences of a fraction of a percentage point.

In 2021, three researchers at MIT went looking for mistakes in it. They used an algorithm to flag images whose labels looked statistically suspicious, then had human reviewers check the candidates. They estimated that about 6% of the ImageNet validation set is mislabelled: 2,916 images.

ImageNet was not unusual. Across ten of the most widely used benchmarks in vision, text and audio, the same team estimated an average test-set error rate of at least 3.3%. Google’s QuickDraw came in above 10%.

The finding that matters most is not the error rate. It is what happened when the errors were corrected.

When the ranking flips

A benchmark exists to order models. If model A scores higher than model B, we conclude that A is better and act accordingly. That inference assumes the answer key is right.

Northcutt, Athalye and Mueller tested the assumption directly. They re-evaluated a range of standard models on the examples whose labels had been corrected. On those examples, the ordering of models changed. Larger, higher-capacity models, which scored best against the original labels, did worse than smaller models once the labels were fixed.

Their explanation was that large models had enough capacity to fit the systematic patterns in the labelling mistakes, and were rewarded for it by a test set that contained the same mistakes. The authors quantified how fragile the standard ranking is. On ImageNet with corrected labels, ResNet-18 outperforms ResNet-50 if the prevalence of originally mislabelled test examples increases by just 6%.

A practitioner choosing between two models on a noisy benchmark may be selecting the one that has better learned the benchmark’s errors. In a deployment where real data is noisier than the curated benchmark, which is most deployments, the model that looked worse may be the better choice.

Not all errors are errors

Before treating 6% as a simple defect rate, it is worth looking at what the mistakes are.

A parallel line of work examined ImageNet’s labels in detail. Beyer and colleagues, in a paper titled “Are we done with ImageNet?”, found that a large share of the apparent errors arose because images contain several objects while the benchmark allows one label. A photograph of a desk with a laptop, a keyboard and a mug has a single ground-truth class, and a model that names a different visible object is marked wrong. Tsipras and colleagues reached a similar conclusion: much of the remaining gap between top models and perfect accuracy reflected the labelling procedure, not the models’ perception.

A later study from Google Research reviewed every remaining mistake made by the best models of the time. Many were arguably not mistakes at all, and a meaningful fraction involved fine-grained distinctions, such as closely related dog breeds, where the original label itself was in doubt.

So “label error” covers at least three different things:

• Plain mistakes. The label is wrong and any informed reviewer would agree.
• Underspecified tasks. The image is multi-object, the text is ambiguous, and the single-label format cannot represent it.
• Expertise gaps. The correct label exists but the original annotators could not reliably determine it.

Each calls for a different remedy. The first needs correction. The second needs a different task design. The third needs different annotators.

Why test errors cost more than training errors

There is a reasonable intuition that a few per cent of noise is tolerable. For training data, that intuition has research behind it: deep networks are often robust to moderate amounts of random label noise, and methods such as confident learning can identify and down-weight suspect examples.

Test data is different, for three reasons.

It is small. A training set has millions of examples to average over. A typical evaluation set has thousands, and the subsets that matter, such as rare classes or critical failure modes, may have dozens. A handful of wrong labels in a subset of fifty changes the measured result materially.

It is the instrument. Training noise makes a model somewhat worse. Test noise makes every measurement of every model wrong, by an unknown amount, in an unknown direction. Every decision that depends on the metric inherits the error: which model to ship, whether a fine-tune helped, whether last quarter’s regression was real.

It sets a false ceiling. If 5% of labels are wrong, a perfect model scores 95%. Teams then spend effort chasing the last few points of headroom that do not exist, or worse, reward the model that reproduces the labellers’ mistakes.

Medical AI has learned this the expensive way. In work on diabetic retinopathy, Krause and colleagues at Google showed that the choice of reference standard substantially changes a model’s measured performance. When labels came from a majority vote of individual graders, some disease was missed that became evident when retinal specialists discussed the same images and adjudicated. Evaluated against the adjudicated standard, error rates looked different, and tuning the model on a small set of adjudicated labels improved it. The model had not changed. The ruler had.

The case for not worrying

There are fair objections to making too much of this.

The first is that the error estimates are themselves produced by a noisy process. Candidate errors were flagged by an algorithm and confirmed by crowd workers, who can be wrong too. The authors acknowledge this and describe their figures as estimates. The exact percentage for any dataset should be held loosely.

The second is that benchmark rankings have been broadly reliable in practice. When Recht and colleagues built a fresh ImageNet test set following the original protocol, accuracy fell by 11 to 14 percentage points across models, but the ordering of models was largely preserved. Models that did better on the old test did better on the new one. Benchmarks contain errors and still carry signal.

Both objections are sound, and both leave the main point standing. Rankings are stable when differences between models are large. They become unreliable when differences are small, and small differences are what most real model-selection decisions turn on. A team choosing between two fine-tunes separated by half a point on an internal test set of two thousand examples is deciding inside the noise.

What good practice looks like

None of this requires exotic tooling. It requires treating the evaluation set as the most important dataset an organisation owns.

Audit it. Run a label-error detection pass over the test set, and manually review the examples where strong models disagree with the label. Disagreement between a good model and the answer key is the cheapest available signal of a labelling problem.

Label it differently. Use more annotators per item, more senior annotators, and adjudication for disagreements. The economics favour this: evaluation sets are small, so a high per-item cost is a low total cost.

Record the ambiguity. Where an item has more than one defensible answer, store that fact. A test set that forces a single label onto an ambiguous item is measuring agreement with one annotator’s coin flip.

Protect it. An evaluation set that has been used to tune a model, or has leaked into training data, no longer measures what it did. Keep it separate, control who can see it and replace part of it periodically. A later article in this series takes up contamination in detail.

Report uncertainty. A metric computed on a few thousand noisy labels has an error bar. Decisions should be made with it visible.

What this changes