Systematic Framework Converts Qwen3-30B-A3B MoE into Dense Models via Expert Pruning and Distillation

This work evaluates 350 configurations of scoring, grouping, and magnitude scaling methods to convert MoE models into dense FFN architectures, using knowledge distillation from the MoE teacher. Tested on Qwen3-30B-A3B, it finds scoring method choice dominates compression quality, enabling memory-constrained deployment without the MoE parameter loading requirement.