Recurrent Looped Transformer Enables Variable Computation for State Tracking
October 7, 2026
The Recurrent Looped Transformer (RLT) uses a parallel causal encoder and a recurrent decoder to allow computation depth to grow with sequence length. On parity tasks, RLT generalized to 256 bits with 100% accuracy, whereas standard Transformers failed at chance levels.
HOW THIS AFFECTS YOU
●
builderThis architecture could enable more efficient processing of extremely long, state-dependent sequences.
●
researcherThe RLT architecture provides a path toward transformers that scale computation per token for long-sequence state tracking.