Sumi: First 7B Uniform Diffusion LM Pretrained from Scratch at Scale

Sumi is a fully open 7B-parameter uniform diffusion language model pretrained from scratch on a large token budget, filling a gap where only autoregressive and masked diffusion models had scaled reference points. Uniform diffusion allows any token to be updated at any step, enabling more flexible generation dynamics than masked approaches.