[arXiv]score: 0.32

BABA-Bench evaluates long-horizon procedural compliance in patient-advocacy agents

November 2, 2026

BABA-Bench uses 47 interactive worlds derived from real patient advocacy cases to test if LLM agents reach correct outcomes while following necessary legal and procedural constraints. Testing of 14 frontier models revealed that 28% of successful agents reached the correct outcome via invalid or prohibited routes.

HOW THIS AFFECTS YOU

●

builderYou must implement rigorous procedural verification if building agents for highly regulated sectors like healthcare.

●

policyThe high rate of procedural failure in frontier models underscores the need for strict alignment on regulatory constraints.

●

healthThis benchmark highlights critical safety and procedural risks in deploying autonomous agents for patient care.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed