TRACE Protocol for Diagnosing Verifier Brittleness in Agents
October 9, 2026
The TRACE protocol identifies when changes in agent performance are due to verifier sensitivity rather than actual capability shifts. In tests, renaming tools changed agent scores by 0.250 despite identical operations, highlighting how scoring rules can introduce measurement bias.
HOW THIS AFFECTS YOU
●
builderThis warns you that agentic evaluations may be more sensitive to prompt formatting than actual logic.
●
researcherYou should use this protocol to validate that your reward signals or benchmark improvements are real.