On-Demand Attention Reduces Long-Context Inference Latency via Selective Recall

On-Demand Attention (ODA) implements a lightweight recall head that selectively invokes global attention based on predicted benefit. Tested on Qwen and Gemma, this method enables GPU-side conditional execution in vLLM to accelerate decoding speed during long-context inference.