ITC-MoE Compression for MoE Diffusion Language Models
October 2, 2026
ITC-MoE addresses parameter redundancy in MoE Diffusion Language Models using Importance-guided Adaptive Tucker Compression. The framework accounts for token-wise utilization variation and non-uniform redundancy across input, output, and expert modes to reduce storage and computation costs.
HOW THIS AFFECTS YOU
●
builderThis offers a path to deploy large MoE diffusion models on hardware with limited memory and compute.
●
researcherThe focus on token-wise spectral characteristics and adaptive rank allocation provides a new way to compress MoE architectures.