REMORY Compresses Context Using Residual Soft Memory Tokens
October 7, 2026
REMORY supplements textual summaries with a bounded sequence of soft memory tokens to assist frozen LLMs in approximating full-context outputs. On the SummHay benchmark, it achieves near-full-context joint scores while utilizing only 5.2% of the original input positions.
HOW THIS AFFECTS YOU
●
builderYou can significantly reduce context window costs and latency by using residual tokens instead of full history.
●
researcherThis method provides a new way to handle long-horizon context via sequence-dimension residual connections.