[arXiv]score: 0.22

Mirror LLM Evaluations of Depression Suffer from Criterion Contamination

September 11, 2026

Research shows that LLM depression assessments using language from the diagnostic interviews themselves lead to near-perfect, contaminated results. When tested on independent life history interviews, prediction accuracy remains high but the artificial advantage of the mirror condition disappears.

HOW THIS AFFECTS YOU

●

researcherYou must ensure evaluation datasets are independent of the prompt context to avoid contaminated results.

●

policyThis suggests caution when using LLMs for automated mental health screening based on contaminated benchmarks.

●

healthBe aware that high accuracy in certain LLM depression models may be an artifact of training on interview-specific language.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed