Bell-shaped time sampling accelerates Masked Diffusion Language Model training by 4x
July 24, 2026
Masked diffusion models (MDMs) suffer from slow convergence due to the locality bias of language. Implementing bell-shaped time sampling during training allows MDMs to reach equivalent validation negative log-likelihood on the LM1B benchmark up to 4x faster than standard methods.
HOW THIS AFFECTS YOU
●
builderThis training recipe can significantly reduce compute costs when scaling non-autoregressive language models.
●
researcherYou can use bell-shaped sampling to bypass locality bias in diffusion-based sequence modeling.