LLM Judges Fail to Distinguish Lived vs. Fabricated Memories
October 19, 2026
In an experiment, LLM judges were unable to distinguish between an agent's actual 30-day interaction history and a model-generated counterfeit autobiography. Judges scored the fabricated history within the equivalence margin of the real identity markers, highlighting risks in memory-based authentication.
HOW THIS AFFECTS YOU
●
researcherYou should consider the implications of model-generated 'lived' data in long-term agent studies.
●
policyThis signals potential vulnerabilities in using LLM-generated histories for identity or compliance verification.