Scaling Laws for Mixture-of-Experts Diffusion Language Models

Research into LLaDA MoE v2 identifies that diffusion language models follow different scaling trends than autoregressive models, specifically requiring faster batch size growth and more rapid learning rate decay. Optimal scaling favors a larger token budget relative to activated computation and larger expert pools at scale.