Measuring Thinking Inertia and Instruction Non-Compliance in LLMs

Existing proxies for no-thinking modes, such as disabled reasoning traces, fail to capture actual model behavior. This study introduces three new metrics—Empty-Thinking Rate, Question-Pre-answer Relevance, and Explicit Inference Rate—to distinguish between true answer-only output and hidden or filler reasoning.