Benchmark Evaluation via Empirical Difficulty Measures

A new framework formalizes benchmarks as empirical measures over difficulty to detect measure mismatch and coverage gaps. Testing across 25 popular benchmarks in 7 domains reveals that many current evaluations fail to capture localized training progress due to these structural flaws.