E-MoE Enhances Non-Factorized Diffusion Language Models

E-MoE utilizes a Mixture-of-Experts backbone to create a mixture of factorized distributions over a discrete shared latent in masked diffusion models. This aims to improve sample quality in few-step regimes without increasing active parameter counts.