[arXiv]score: 0.28

LLM Depression Screening Scores Driven More by Model Choice Than Patients

October 7, 2026

Evaluating 11 open models on 189 interviews shows model choice explains 30.0% of symptom score variance, compared to just 10.5% for participant differences. Even high-performing raters (AUC >= 0.70) disagreed on screening decisions for 40% of cases, though recalibration improved accuracy from 60% to 75%.

HOW THIS AFFECTS YOU

●

researcherYou should account for model-specific variance when using LLMs as clinical proxies.

●

policyRegulators should scrutinize the lack of consistency in LLM-based mental health assessments.

●

healthCaution is required when using LLMs for diagnostic screening due to high inter-rater disagreement.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed