[arXiv]score: 0.41

Transformer Models Show Massive AUC Degradation in Mental Health NLP

August 28, 2026

A fairness audit of BERT and RoBERTa models reveals significant cross-platform performance drops when moving from training corpora to Reddit and Twitter. AUC values fell by up to 39.5% and Expected Calibration Error (ECE) increased significantly, highlighting severe generalization failures in mental health applications.

HOW THIS AFFECTS YOU

●

researcherYou must account for platform-specific distribution shifts when training transformer models for social media.

●

policyThis underscores the reliability risks of deploying uncalibrated mental health models on diverse social platforms.

●

healthThe observed calibration failures pose significant risks for using these models in clinical diagnostic proxies.

read original ↗arxiv.org

DAILY DIGEST

you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy

← back to feed