DualBudget Sparse Text Diffusion for Long-Context Efficiency
November 2, 2026
DualBudget optimizes long-context diffusion by managing two separate budgets: attention graph edges and expert activations. Testing shows performance gains over baselines up to 16K tokens, though measurable degradation occurs as context length increases.
HOW THIS AFFECTS YOU
●
researcherYou can explore the trade-offs between attention sparsity and expert computation in long-context diffusion models.