What the Human in the Loop Is For

Adding a person to an AI system is assumed to make it safer and better. A meta-analysis of more than a hundred experiments found that, on average, it made things worse than the better of the two alone.

“Human in the loop” is one of the most reassuring phrases in applied AI. It appears in governance frameworks, procurement requirements and product pages. The assumption behind it is rarely stated because it seems too obvious to need stating: a person plus a model will do at least as well as either alone.

In 2024, three researchers at MIT tested that assumption against the published evidence. Vaccaro, Almaatouq and Malone conducted a preregistered meta-analysis of 106 experimental studies, covering 370 comparisons, in which the performance of humans alone, AI alone and humans working with AI had all been measured on the same task.

On average, the human–AI combination performed significantly worse than the better of the human or the AI working alone.

The effect was small, and it was an average over very different tasks. The pattern within it is what makes the study useful.

Two findings that explain the average

The direction depended on who was better to begin with. When humans alone outperformed the AI alone, adding the AI helped: the combination beat the human. When the AI alone outperformed humans, adding a human hurt: the combination was worse than the AI.

That asymmetry makes sense on reflection. A person who is better than the machine can use its output as one more input and overrule it when it is wrong. A person who is worse than the machine cannot reliably tell when it is wrong, and their interventions replace some correct machine answers with incorrect human ones.

The direction depended on the kind of task. Combinations showed losses on decision tasks, such as classifying, diagnosing or forecasting. They showed more favourable results on tasks that involved creating content.

The implication for anyone designing oversight is uncomfortable. The setting where a human reviewer is most commonly mandated, a person checking the decisions of a model that is on average more accurate than they are, is the setting where the evidence for benefit is weakest.

Why review fails

The mechanisms have been studied for decades, mostly in aviation and industrial control before AI.

Lisanne Bainbridge set out the core problem in 1983 in a paper called “Ironies of Automation”. The more reliable an automated system, the less its human supervisor has to do, and the less practised and attentive they become. The supervisor is then expected to step in precisely when the automation has failed, which is the hardest situation, with the least preparation.

Parasuraman and Manzey’s review of the subsequent research distinguishes two failures. Complacency is insufficient monitoring: the human stops checking. Automation bias is deference: the human sees the system’s recommendation and follows it even when other information contradicts it. Both occur in experts as well as novices, and neither is reliably prevented by training or by telling people to be vigilant.

The effect has been observed with current AI systems. In a field experiment with several hundred consultants at Boston Consulting Group, Dell’Acqua and colleagues gave participants access to a large language model. On tasks within the model’s competence, those with access were faster and produced higher-quality work. On a task chosen to lie just outside its competence, those with access were 19 percentage points less likely to reach the correct answer than those without. They had trusted an output that looked as good as the others.

The authors called the boundary a jagged frontier. From the user’s side, tasks the model does well and tasks it does badly are hard to tell apart, because the output is equally fluent in both.

A further finding, from Bansal and colleagues, closes off an obvious remedy. Giving people the model’s explanation along with its answer increased their tendency to accept the answer, whether it was right or wrong. Explanations raised trust. They did not improve discrimination.

What humans contribute

None of this argues for removing people. It argues for being specific about what they are there to do. Across the research, human contribution is strongest in five roles.

Defining the task. What counts as a good answer, which errors are tolerable and which are not, what the categories mean. A model cannot supply its own objective. This is the role that the articles on annotation guidelines and preference data in this series return to repeatedly.

Supplying information the model does not have. Context from outside the input: the patient mentioned something in the corridor, the supplier changed last week, this customer has complained before. Where the human sees more than the model, the Vaccaro pattern predicts gains.

Handling novelty. Inputs unlike anything in the training data, where the model’s confidence is least meaningful. This requires that the system can recognise such inputs and route them, which is a technical design problem in its own right.

Generating ground truth. Independent human judgements, made without seeing the model’s output, are what make it possible to measure the model at all. This is a different activity from reviewing its decisions, and it is immune to automation bias by construction.

Carrying accountability. Some decisions require a person who can be answerable for them. That is a legitimate institutional requirement. It should not be mistaken for a quality control.

The regulatory expectation

Regulation has begun to make these distinctions. Article 14 of the European Union’s AI Act requires that high-risk systems be designed so that people can oversee them effectively. It goes further than requiring presence. It says the people assigned must be able to understand the system’s capacities and limitations, to remain aware of the tendency to over-rely on its output, which it names as automation bias, and to decide not to use the output or to override it.

That wording sets a functional test. A human who is nominally in the loop but lacks the time, information or authority to disagree does not satisfy it.

Designing a loop that works

The practical consequences follow from asking, for each point where a human is placed, what they know that the model does not.

Route, do not blanket. Reviewing every output spreads attention thinly and trains reviewers to approve. Reviewing the subset where the model is uncertain, the input is unusual or the stakes are high concentrates attention where it can change outcomes.

Decide before seeing. For high-stakes decisions, have the human form a judgement first and compare with the model’s afterwards. This removes the anchor. It costs time, which is why it should be reserved for the routed subset.

Measure the reviewer. Insert cases where the model’s output is known to be wrong and track whether reviewers catch them. Track override rates and whether overrides were correct. A reviewer whose overrides are wrong as often as right is adding noise.

Give reviewers something the model lacks. Additional context, more time, access to the source, the ability to ask a question. If the reviewer sees exactly what the model saw, and is less accurate than the model, the evidence predicts a loss.

Turn review into data. Every correct override is a training example and an evaluation case. A loop that discards them wastes its most valuable output. A loop that captures them is a continuous data pipeline.

Sample for audit separately. Independently of live review, have a random sample of model decisions assessed blind by qualified people. This measures the system. Live review does not, because it is contaminated by the model’s suggestion.

The objection from necessity

A practitioner might reply that this is a counsel of perfection. Regulators and customers demand a human reviewer. Removing one because a meta-analysis says the average effect is negative would be reckless.

That is correct, and it is not what the evidence suggests doing. The studies show that undifferentiated human review of a stronger system does not reliably help. They also show clear benefit where humans are the stronger party. The work is to move human effort from the first situation to the second: fewer cases, better chosen, with more information and real authority.

There is a second reply. Model accuracy on average says little about accuracy on a given case. A model that is better than a clinician across ten thousand scans may be much worse on a particular rare presentation. Human review targeted at such cases can add value even when the model is better overall. That is an argument for precise routing, and it depends on knowing where the model is weak, which is an evaluation-data problem.

What this changes