Source-Centered State Evolution for Looped Transformers
July 31, 2026
Source-Centered State Evolution (SCSE) enables looped Transformers to maintain input dependence through learned anchors and initial deviations. This mechanism allows for shared recurrence while guaranteeing exact anchor invariance and preventing the hidden state from drifting uncontrollably during deep extrapolation.
HOW THIS AFFECTS YOU
●
researcherThis method provides a way to increase effective model depth at fixed parameter counts while maintaining stable recurrent computation.