Causal Interventions Reveal Distinct Numerical Representations in Gemma4-12B-it
October 28, 2026
Causal interventions on Gemma4-12B-it residual streams show that text and image inputs do not utilize the same internal subspaces for numerical reasoning. Swapping a text-based numerical subspace into an image-based context recovers only 53.2% of its causal effect, indicating distinct modality-specific machinery.
HOW THIS AFFECTS YOU
●
researcherThis suggests that multimodal alignment does not necessarily imply shared internal representations for arithmetic operations.