The GridWM-Judge benchmark tests the reliability of Vision-Language Models (VLMs) acting as judges using State-Conditional Invariance (SCI). It uses deterministic MiniGrid trajectories to ensure judges respond to causal changes in task outcomes rather than just presentation changes.
HOW THIS AFFECTS YOU
●
researcherYou can use this to check if your VLM judge is actually looking at the right evidence or just reacting to image styles.
●
policyYou can better assess the reliability of automated visual reasoning systems in safety-critical tasks.