[arXiv]score: 0.20

LLM Mental Health Simulations Fail Epidemiological Population Benchmarks

August 7, 2026

Evaluation of GPT-4o-mini, Gemini-3-Flash, DeepSeek-V3, and GLM-4 reveals that while individual psychiatric simulations pass DSM-5 checks at a 97.3% rate, population-level data is highly inaccurate. Simulated cohorts show PHQ-8 scores inflated by 2.8 to 5.5 points and fail to replicate real-world demographic disparities found in NHANES data.

HOW THIS AFFECTS YOU

●

researcherDo not rely on LLM-generated synthetic populations for epidemiological modeling or clinical validation.

●

policyThis highlights the risks of using synthetic datasets to inform public health policy or clinical guidelines.

●

healthYou should be aware that LLM-simulated patient cohorts lack statistical fidelity to real-world clinical populations.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed