Spectral-LSH Enables Sub-Quadratic Prompt Compression via Krylov Projection
July 23, 2026
Spectral-LSH provides a training-free prompt compression method that avoids $O(N^2)$ attention-kernel materialization by using a Krylov subspace method and SimHash. Testing on Mistral and Qwen models shows a compression-ratio phase transition, with effective results for compression ratios above $8\times$.
HOW THIS AFFECTS YOU
●
builderYou can reduce prefill latency and costs for long-context inference using this sub-quadratic method.
●
researcherThis offers a new mathematical approach to prompt compression without requiring retraining.