PIMMUR Audit Reveals Flaws in LLM Social Simulations
September 11, 2026
A systematic audit of 350 papers using the PIMMUR framework found that LLM-based social simulations often fail key methodological standards. Frontier models correctly identified underlying social experiments in only 65.2% of cases, and 50.6% of prompts were found to impose constraints that pre-determined outcomes.
HOW THIS AFFECTS YOU
●
researcherBe cautious when using LLMs to model human collective behavior without strict methodological controls.
●
policyThe lack of realism in LLM social simulations may undermine their utility for policy testing and behavioral forecasting.