Tiled SVD Extracts Usable Weight Mechanisms Directly from Linear Sites
August 10, 2026
This method uses column-tiled SVD to extract mechanism mounts—comprising trigger, write, and strength components—directly from network weights. Testing on Gemma-2-2B showed that residual writes in mlp.down and attn.o layers were successfully identified and steerable across all tested sites.
HOW THIS AFFECTS YOU
●
builderThis enables more precise steering of model behavior by targeting specific weight-based mechanisms.
●
researcherYou can move beyond sparse autoencoders to identify features directly within the model's weights.