Boa Compressor Enables Linear-Time Online Context Compression Using Mamba
October 27, 2026
Boa Compressor uses a Mamba state-space model to encode arbitrary-length context into a compressed representation in a single linear-time pass. This plug-and-play method allows for online context compression, avoiding the latency overheads typical of Transformer-based soft-prompt methods.
HOW THIS AFFECTS YOU
●
builderYou can implement online context compression to handle long sequences with lower latency and memory overhead.
●
researcherThis demonstrates the utility of SSM-based distillation for compressing Transformer contexts in real-time settings.