The Transformer Layer Correction Mechanism (TLCM) reveals that adjacent layers in major open-source models systematically counteract each other's contributions. Using layer Jacobians, the study shows this mechanism selectively corrects specific subspaces and emerges during pretraining.
HOW THIS AFFECTS YOU
●
researcherThis challenges the assumption that features simply persist and build up through the residual stream.