Evaluating Proactive LLM Agents via Interruption Budgets and Attention Costs

A new evaluation protocol distinguishes between agent question frequency and question quality by implementing interruption budgets and user-attention costs. Testing across eight LLMs shows that limiting agents to one to three questions reduces user-attention costs by 36-73% while maintaining 86-94% of task reward.