Expert disagreement causes LLM accuracy to drop by up to 16.1 points
November 13, 2026
In a study of 1,989 decisions across medicine, science, and law, no tested model met the mathematical bar for selective revision under expert disagreement. Accuracy dropped by up to 16.1 points when models were told a domain expert disagreed.
HOW THIS AFFECTS YOU
●
builderBe careful when implementing multi-agent debate or human-in-the-loop feedback, as models may abandon correct answers due to prestige cues.
●
researcherYou should look closely at how prestige cues and expert labels influence model revision behavior.
●
policyThis highlights a significant reliability risk in using LLMs for high-stakes professional decision support.