Register Tokens Enable Bounded-State Reasoning in Diffusion Models
September 16, 2026
Register tokens allow masked diffusion language models to carry continuous hidden states across generation chunks without keeping previous text in context. This method improves math benchmarks by up to 8.5 points and code benchmarks by 19.5 points compared to discrete-text carry.
HOW THIS AFFECTS YOU
●
builderThis offers a path to more efficient, stateful generation in diffusion-based text models.
●
researcherYou can implement these fixed-position tokens to extend reasoning capabilities in non-autoregressive architectures.