Ask2Decide Benchmark Evaluates LLM Agent Information Seeking in Planning
November 11, 2026
Ask2Decide is a new benchmark of 43 planning tasks designed to test if LLM agents ask humans for missing or outdated information. The framework uses optimization solvers to determine which missing facts actually change an optimal plan, distinguishing useful queries from irrelevant ones.
HOW THIS AFFECTS YOU
●
builderYou can use this to test if your agents are smart enough to ask users for missing details before making decisions.
●
researcherThis provides a more rigorous way to evaluate agentic reasoning compared to fixed-format benchmarks.