AgentAbstain introduces a paired-task evaluation framework across 42 sandbox environments to measure when agents should abstain from action. The benchmark uses 263 paired tasks to test agent calibration against 8 scenarios involving instruction ambiguity, conflicting constraints, or tool failures.
HOW THIS AFFECTS YOU
●
builderUse this to evaluate if your autonomous agents are prone to executing dangerous actions in ambiguous environments.
●
policyThis framework helps quantify the safety risks of autonomous agents acting without sufficient information.