Residual Stream Geometry Reveals Why LLMs Fail at Multi-Digit Addition
May 28, 2026
Analysis of residual stream activations during multi-operand addition identifies the Iso-Raw-Sum Trajectory (IRST), where carry state is encoded as a continuous latent variable quantized to discrete digits. Arithmetic errors are framed as geometric slippages caused by neural noise crossing quantization thresholds, explaining both failure modes and probe versatility.
HOW THIS AFFECTS YOU
●
researcherThe Noisy Quantization Model provides a mechanistic account of arithmetic errors that could inform targeted interventions — fine-tuning on carry boundary cases or architectural changes to reduce quantization noise.