SciGame: Benchmarking Scientific Discovery Agents with Counterfactuals
October 27, 2026
SciGame is a benchmark that evaluates LLM agents in scientific discovery using structured scientific cards and dimensionally consistent counterfactual environments. It distinguishes between empirical evidence and pretrained priors by testing agents on both historical laws and post-knowledge-cutoff temporal data.
HOW THIS AFFECTS YOU
●
researcherYou can use this to rigorously test whether agents are discovering laws or just retrieving training data.
●
policyThis helps evaluate the reliability of AI in sensitive scientific research domains.