Tokenization Drives Model Sensitivity to Character-Level Noise

Research shows that character-level noise degrades model performance more severely than word-level noise because single-character edits force tokenizer re-segmentation. This effect is driven by the training objective and tokenization strategy rather than the model architecture itself.