DiffusionGemma Uses Discrete Diffusion for High-Speed Text Generation
July 30, 2026
DiffusionGemma is an experimental 25.2B parameter model that generates text by iteratively refining blocks of 256 tokens in parallel. By fine-tuning a Gemma MoE model with discrete diffusion, it avoids the sequential decoding bottleneck of autoregressive models.
HOW THIS AFFECTS YOU
●
builderYou can achieve significantly higher text generation throughput by moving away from autoregressive decoding.
●
researcherThis presents a viable path for scaling non-autoregressive text generation using MoE architectures.