Scaffold Effects Cause 40x Token Variance in Coding Agents
July 28, 2026
Evaluating Qwen 3.6 Plus and MiniMax M2.5 reveals that the choice of agent harness (e.g., Goose, OpenHands) can cause up to a 40x difference in tokens used per task. These discrepancies often stem from the scaffold rather than the underlying model's reasoning capability.
HOW THIS AFFECTS YOU
●
builderThe efficiency of your agentic implementation may be as critical as your choice of LLM.
●
researcherBe wary of model leaderboards that do not standardize the agentic scaffold.