NAVSIM Scoring Vulnerable to Shared Rollout Failures in Driving Evaluation
August 6, 2026
An audit of NAVSIM v2.2 reveals that shared rollout transformations can cause agents to receive erroneous compliance credit when both the agent and the human reference fail. This flaw allows route-blind probes to outrank human replays in defensive driving scores.
HOW THIS AFFECTS YOU
●
builderEnsure your driving policy evaluations use independent reference streams to avoid inflated performance metrics.
●
researcherBenchmark validity in autonomous driving requires strict control over the dependency stack and exact-input diagnostics.