Lexical Corruption Triggers Attention Diversion in LLM Reasoning
August 25, 2026
Character-level perturbations and keyboard noise degrade reasoning performance by fragmenting subword tokenization. This fragmentation causes disproportionate attention mass to concentrate in middle and final transformer layers, a phenomenon termed Attention Diversion that remains difficult to rectify via simple interventions.
HOW THIS AFFECTS YOU
●
builderYou need to implement robust preprocessing to handle character-level noise in user inputs.
●
researcherYou should account for tokenization-induced attention shifts when evaluating model robustness.