MERLIN Improves Linear Attention via Second-Order Routing
October 28, 2026
MERLIN enhances Multi-State Linear Attention (MSLA) by tracking the first and second moments of segment keys. This allows queries to recover both mean and variance of alignment, addressing the information loss inherent in standard mean-alignment compression methods.
HOW THIS AFFECTS YOU
●
researcherThis method provides a more mathematically robust way to scale long-sequence modeling through linear attention.