Joint Subspace Sparse Attention for Efficient Long-Context Decoding
October 15, 2026
JSSA optimizes long-context LLM decoding by learning a shared low-dimensional query-key subspace to improve token selection accuracy. It includes a gather-free, indexed-attention kernel to mitigate the latency costs of sparse token retrieval.
HOW THIS AFFECTS YOU
●
builderThis technique aims to reduce KV-cache traffic and improve decoding speeds for long-context models.
●
researcherYou can leverage a closed-form spectral solution for subspace selection in sparse attention mechanisms.