Analyzing Perturbation Propagation in GPT-2 and Qwen2.5 Layers
September 4, 2026
Input perturbations like typos and token shuffling affect decoder-only models across three levels: output behavior, hidden-state geometry, and attention-head function. Analysis using centered kernel alignment and intrinsic dimension shows that output-only evaluations fail to capture the full representational disruption occurring within the model layers.
HOW THIS AFFECTS YOU
●
researcherYou can use these multi-level metrics to better diagnose how input noise affects internal model representations beyond simple output accuracy.