BOS Summary Improves LLM and LVLM Context Processing
November 19, 2026
BOS Summary is a training-free inference method that allows the beginning-of-sequence token to aggregate all visual and textual input tokens during the prefill stage. By relaxing the causal constraint for the BOS query row alone, it provides a dense context representation for all subsequent decoding steps without adding new parameters.
HOW THIS AFFECTS YOU
●
builderYou can improve model context handling at inference time without needing to retrain or add new modules.
●
researcherThis shows how repurposing attention sinks can solve information loss in standard causal masking.