MLLMs suffer from agreement bias in verification tasks
November 13, 2026
Research identifies agreement bias in Multimodal LLMs, where verifiers over-validate agent behavior in web navigation and robotics. This bias persists even with test-time scaling and can harm self-improvement pipelines.
HOW THIS AFFECTS YOU
●
builderBe careful using MLLMs as automated verifiers for your agents, as they might give false positives.
●
researcherYou need to account for the fact that MLLM judges tend to agree with agent actions too often.