BYOD: Parameter-Efficient AR-to-Diffusion Language Model Conversion
November 2, 2026
BYOD is a LoRA-only method for converting autoregressive models like Llama 3.1 8B and Gemma 2 9B into diffusion language models. The process uses only 4.1-5.8% additional parameters and requires under 4 hours of training on a single GPU.
HOW THIS AFFECTS YOU
●
builderYou can enable parallel token updates and bidirectional attention in existing models with minimal compute overhead.
●
researcherThis provides a generalizable recipe for inducing diffusion generation without full-model fine-tuning.