FadeMem Cuts KV Cache Cost in Long Video Generation via Temporal Hierarchies

FadeMem organizes historical KV blocks into a temporal hierarchy under a fixed cache budget for autoregressive video diffusion, progressively merging older fine-grained entries based on frequency-dependent temporal decay. This avoids unbounded KV cache growth without the uniform compression losses of sliding-window or sink-token methods.