RegretBench Evaluates LLM Clarification Policies via Multi-Turn Interaction

RegretBench assesses LLMs as conversational agents by measuring the regret of clarification decisions in multi-turn dialogues. The benchmark uses a hidden-intent formulation to track semantic state and evaluate how effectively models decide when to ask questions versus when to answer.