DuMateBench Evaluates Agents Using Real-World Production User Sessions
August 28, 2026
DuMateBench introduces a benchmark of 200 tasks reconstructed from anonymized production agent sessions to test multi-tool workflow coordination. Tasks are executed in Docker containers that simulate real-world complexities like insufficient data and unstable environments.
HOW THIS AFFECTS YOU
●
builderYou can test your agents against realistic environmental instabilities rather than clean, synthetic benchmarks.
●
researcherThis dataset offers a more accurate measure of agent performance in complex, multi-step real-world workflows.