CurveShift Analysis Reveals LLM Progress Often Lacks Qualitative Shape Changes

Analysis of METR time-horizon data suggests that apparent shifts in LLM progress toward harder tasks are often explained by ceiling effects in a single Rasch model rather than qualitative changes in capability. The study highlights how metric selection can create the illusion of emergent abilities by misinterpreting the difficulty-response curve.