JUPE benchmark reveals that 14 MLLM judges are susceptible to plausibility capture, where convincing text explanations sway visual verification. Plain justifications and chain-of-thought rationales reduced the rejection of incorrect answers by up to 13.8 percentage points.
HOW THIS AFFECTS YOU
●
researcherYou must account for textual bias when using MLLMs as automated evaluators in vision tasks.
●
policyAutomated AI auditing systems may produce false positives due to reasoning-based deception.