Maglev Recurrent Transformer with Fixed-Size Memory
August 4, 2026
Maglev uses a two-part architecture consisting of a full-attention prefiller and a sliding-window decoder with recurrent K/V injection. A memory consistency loss aligns decoder memories with prefiller targets, enabling efficient inference with a fixed-size memory footprint.
HOW THIS AFFECTS YOU
●
builderThis could reduce KV cache memory requirements for long-context inference.
●
researcherYou can explore new ways to generalize sliding-window attention via recurrent memory injection.