Audit Protocol Reveals Claim-Implementation Gap in LLM-Based Cognitive Agents
November 2, 2026
An audit of 25 cognitive-architecture modules in LLM agent systems shows that 18 of 25 named mechanisms, such as theory of mind or causal reasoning, do not actually perform the computed operations they claim. The study introduces a four-question static audit protocol using line-number evidence and call-chain tracing to verify agent capabilities.
HOW THIS AFFECTS YOU
●
builderBe cautious of marketing claims regarding 'reasoning' or 'memory' modules in agent frameworks, as they may be purely semantic.
●
policyThis provides a methodology for verifying the functional claims made by AI agent developers.