iPrefBench evaluates the ability of mobile agents to infer missing information from user activity history within an executable iOS environment. The benchmark uses a persona with 3,051 past events to test if agents can ask necessary questions without making unsupported assumptions.
HOW THIS AFFECTS YOU
●
builderYou can use this to measure how well your mobile agents handle underspecified user intents.
●
designerThis highlights the importance of designing UX patterns for agents that need to ask clarifying questions.