VagueBench Identifies Ambient Context Illusions in Embodied Agents
November 4, 2026
VagueBench evaluates embodied agent robustness in the AI2-THOR environment, focusing on failures caused by minimal instruction context. The benchmark reveals that agents struggle when non-essential contextual cues are omitted, even when task semantics remain unchanged.
HOW THIS AFFECTS YOU
●
researcherYou can use this to test if your agents rely too heavily on auxiliary environmental cues rather than core task semantics.