Nonlinear Performance Degradation via Hard Distractors in Long-Context Models

LLM performance drops sharply upon the introduction of even a small proportion of hard distractors within fixed-length contexts. This First Drop of Ink effect demonstrates that distracting information contaminates reasoning nonlinearly, with the most significant decay occurring in the initial fraction of distractor insertion.