System1-Arena Benchmarks Discrete Decisions vs Generative Reasoning
October 16, 2026
System1-Arena evaluates 30 models across 92k questions to determine if discrete decisions can be handled via single-pass probabilities or require System-2 reasoning. The benchmark includes 105 tasks and 138 closed-loop environments ranging from web shopping to robotics.
HOW THIS AFFECTS YOU
●
builderYou can use this to decide whether to use cheap, fast probability-based APIs or expensive reasoning models for specific tasks.
●
researcherThis provides a standardized way to compare System-1 versus System-2 decision-making capabilities.