dQwen3.5 Hybrid-Attention Diffusion Language Models
September 18, 2026
The dQwen3.5 family demonstrates that hybrid architectures interleaving attention and RNN layers can be efficiently adapted into diffusion language models. At scales from 0.8B to 9B, these hybrid backbones reach target training loss using approximately half the tokens required by full-attention models.
HOW THIS AFFECTS YOU
●
builderYou can leverage these hybrid architectures for more efficient parallel decoding and any-order generation.
●
researcherThis proves that structurally causal RNN layers can be successfully bidirectionalized for diffusion tasks.