[arXiv]score: 0.10

LLM Demographic Personas Fail to Reliably Simulate Hate Speech Disagreement

June 5, 2026

Across three evaluation dimensions — inter-group disagreement, in-group sensitivity, and vicarious prediction — no tested LLM consistently replicates how different demographic groups perceive hate speech. Performance is highly model-dependent and does not generalize, undermining the use of persona-conditioned models as annotation proxies.

HOW THIS AFFECTS YOU

●

researcherDirect evidence that persona-conditioned LLMs are unreliable substitutes for diverse human annotators in subjective NLP tasks — affects dataset construction and evaluation validity.

●

policyFindings challenge the scalability argument for using LLM personas in content moderation research, with implications for fairness audits and annotation pipeline design.

read original ↗arxiv.org

← back to feed