You hire a forecasting algorithm because the numbers say it is better than the people currently doing the job. It is not a little better. It is better in a way that shows up on a spreadsheet, month after month, and everyone in the room knows it. Then one afternoon it gets a call badly wrong, and the wrongness is specific enough to be embarrassing, specific enough to be repeated in a meeting. Something closes. Nobody announces the closing. The humans quietly take the work back, and the algorithm stays installed, technically, still running, still generating forecasts that nobody reads.
That is not a story about a bad algorithm. In the lab, it happened to people who had just been shown the algorithm was the best option they had.
🧠 Five Studies, One Withheld Forecast
In 2015, Dietvorst, Simmons and Massey published a set of five studies in the Journal of Experimental Psychology: General under a title that gives away the verdict: Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err.
The procedure was simple enough to be cruel. Participants made forecasts, or watched forecasts being made, with real money attached to how well they did. Then came the decision that mattered: going forward, do you want your payoff tied to the algorithm’s predictions, or to a human forecaster’s?
Some participants never saw either party perform. Some watched the algorithm. Some watched a person. Some watched both. And the groups that had watched the algorithm work were the groups most likely to walk away from it — including participants who had seen with their own eyes that the algorithm was outperforming the human they then chose instead.
The mechanism, as the authors put it, is that people lose confidence in an algorithm faster than in a human after seeing the same mistake. Not a bigger mistake. The same one.
⚙️ Why One Error Costs So Much
This is the part worth slowing down on, because the arithmetic of the mistake does not explain the size of the reaction. One wrong call out of twenty does not justify firing the thing that got nineteen right. What justifies the firing is not the error rate. It is what the error means.
Four things happen in sequence.
The algorithm is an entity; a person is a distribution. When a human forecaster misses, you have options: he was tired, she had a bad week, that guy is new, the last guy was better. The error gets charged to an individual inside a category that contains many individuals. When an algorithm misses, there is only one of it. The error gets charged to the entire system, because the system is one thing. Same evidence, different size of bucket.
Errors are diagnostic for machines and expected for people. We have a working model of why humans get things wrong — attention, mood, fatigue, bad mornings. Those explanations are available for free, so a human error carries almost no information about human capability in general. For an algorithm, none of those excuses are available. Strip them away and the only remaining explanation for a miss is competence. The human’s error is noise. The machine’s error is a verdict.
You have a counterfactual for the human, and none for the machine. When a person errs, your mind generates replacements effortlessly: swap him out, and someone roughly as good would probably have gotten it right. There is no equivalent swap for a single deployed model. The failure looks like the failure of the way to do this, rather than one sample from a spread of ways.
Switching back feels like learning. This is the part that keeps the whole thing running. After the bad call, returning to human judgment does not feel like superstition. It feels like prudence, like the responsible reading of new evidence, like someone who watches the results instead of trusting a dashboard. The aversion rewards you with a flattering self-image at the exact moment it costs you accuracy. That is a very hard loop to notice from inside.
🤔 The Same Field, the Opposite Result
Four years later, Jennifer Logg, Julia Minson and Don Moore ran six experiments and reported something that made the earlier work look like it had been measuring the wrong species. Their paper is called Algorithm appreciation: People prefer algorithmic to human judgment.
Their participants estimated a quantity — a number of dots, the popularity of a song, how attracted a stranger was to someone — and were then offered advice before finalizing. The advice was described either as coming from an algorithm or from another person. People moved their estimates further when the advice was labeled algorithmic. They followed the machine more.
The researchers also asked their own colleagues to predict this result. Most predicted the opposite.
So which is it? Look at what the two studies actually asked people to do, and the contradiction dissolves into a switch.
In the aversion studies, participants saw a history: rounds of forecasts, rounds of feedback, errors accumulating in view. And they were asked to commit their future payoffs to one party over many rounds. What was on the table was a relationship.
In the appreciation studies, participants saw a label. One advisory number, no track record, no feedback on whether that number had ever been wrong. And they were asked to adjust their own estimate once. What was on the table was a single act of deference.
Add two more conditions from the appreciation paper and the picture sharpens. The effect shrank when people had to choose between the algorithm’s estimate and their own — because now ego is in the comparison, not just credibility. And it shrank among participants with real forecasting experience, who relied on algorithmic advice less than laypeople did and, as a result, forecast worse.
Put those together and you get something more useful than either headline: people are not algorithm-averse or algorithm-loving. They are averse to being shown an algorithm’s imperfection, and appreciative of being told an algorithm exists.
Which has an uncomfortable implication for anyone who builds with AI. Trust is highest before the first visible error and lowest right after it. A demo lives entirely in the first regime. A product lives in the second.
🔗 If You Are the One Shipping It
Anyone deploying AI into a workflow — a research assistant, a writing partner, an advisor that talks to end users — is running on both halves of this research at once, usually without noticing which half they are in.
Early use is the appreciation regime. The system is new, the user has no error history, the label itself does the persuading, and satisfaction runs high for reasons that have nothing to do with accuracy. Then the first memorable mistake lands, and the user crosses into the aversion regime without a signal that anything changed. The product did not get worse. The evidence did.
Two practical consequences. First, a user’s post-error judgment is not a clean measurement of quality — it is quality filtered through the size of the bucket in step one. Second, in products where the AI has a personality and a name, the bucket is maximally small. Users do not file its mistakes under “these systems miss sometimes.” They file them under “he got this wrong,” which makes the disappointment sharper and, oddly, the trust deeper once it survives.
For evaluation work, the same filter shows up in diligence. A demo is a label. The thing worth asking about is the error history nobody put on the slide: when did this system last get something badly wrong, was it visible to the user, and what happened to the user’s confidence afterward.
🎲 Let Them Turn One Dial
The most interesting sequel to the 2015 paper is about repair rather than diagnosis. In a follow-up published in Management Science, Dietvorst and colleagues found that people become dramatically more willing to use an imperfect algorithm if they are allowed to modify its output — even slightly, even cosmetically.
The detail that makes it a psychology finding rather than a UX tip: willingness barely moved when the permitted adjustment got larger. A small nudge and a big nudge produced roughly the same uptake. What people wanted was not more control. They wanted some control — proof that a human hand was on the dial when the thing went wrong.
You cannot talk a user out of algorithm aversion with an accuracy chart. You can hand them one small dial and watch the objection mostly evaporate.