PersEvalBench Benchmark Identifies LLM Evaluator Failures in Personalization

PersEvalBench evaluates LLM ability to judge the appropriate use of user history across 403 multi-episode scenarios. Results show current LLM evaluators struggle with personalization errors, particularly under-personalization, where humans detect 94.4% of errors that LLMs miss.