MafiaScope: Non-Invasive, Time-Resolved Belief Probing for LLM Agents in Social Deduction Games

MafiaScope provides a non-invasive testbed for measuring LLM Theory of Mind via structured, private probe questions during social deduction games. A 32-game study using DeepSeek models revealed an expected calibration error of 0.17, indicating that agent confidence levels are poorly calibrated regarding their own deceptive or truthful states.