GAAC: Lossless 1.4x KV Cache Compression for On-Device LLM Inference
October 30, 2026
GAAC achieves 1.4x lossless BF16 KV compression by encoding exponents using calibrated per-channel base-plus-offset codes. A fused attention kernel reconstructs KV tiles on the fly, enabling more efficient long-context inference on hardware with limited bandwidth.
HOW THIS AFFECTS YOU
●
builderYou can implement this to extend context windows on edge devices without losing precision.