This study differentiates between predictive and causal features in Sparse Autoencoders (SAEs) using Gemma 3 1B and 4B. By applying causal interventions after filtering for format artifacts, the researchers demonstrate that specific SAE activations can drive shifts in logit margins and error rates.
HOW THIS AFFECTS YOU
●
researcherYou can implement this pipeline to move beyond correlation-based mechanistic interpretability.
●
policyBetter understanding of why models make specific errors can inform safety and auditing standards.