GrowPage Framework for On-Demand KV Cache Budgeting in Reasoning LLMs
September 4, 2026
GrowPage treats KV cache capacity as a dynamic runtime resource to address memory bottlenecks in long-output reasoning workloads. The framework uses dual-timescale query summaries to estimate evolving attention demand, acquiring additional physical pages only when current compression is insufficient.
HOW THIS AFFECTS YOU
●
builderYou can reduce inference costs and latency by implementing on-demand KV budgeting instead of fixed-budget compression.
●
researcherThis introduces a new method for managing attention-based memory bottlenecks in long-context reasoning models.