MoSAR Learns Adaptive Attention Geometries for Long Context

MoSAR replaces fixed sparsity patterns with an input-conditioned mixture of short, medium, and global attention regimes. By learning adaptive geometries through query-key routers, it addresses the quadratic complexity bottleneck of dense self-attention.