Measuring non-additive behavior in Sparse Autoencoder representations
October 14, 2026
The SAE Additivity Residual (SAR) method quantifies behavioral variation in Sparse Autoencoder (SAE) features that cannot be explained by simple additive composition. The research shows that learned SAE coordinates exhibit lower first-order residuals compared to random rotations, indicating more structured representation.
HOW THIS AFFECTS YOU
●
researcherYou can use SAR to better understand the compositionality of features in mechanistic interpretability research.