MetaNet: Task-Conditioned Layer-Wise Compression for MoE Models
August 28, 2026
MetaNet uses a support-set controller to predict layer-specific expert-retention thresholds and routing biases without retraining the backbone. Tested on DeepSeek-MoE-16B-Chat, it achieves comparable accuracy to fixed k=6 while activating 40% fewer experts on average.
HOW THIS AFFECTS YOU
●
builderYou can dynamically trade off accuracy for inference latency and compute savings in MoE deployments.
●
researcherThis offers a new way to perform task-conditioned compression without modifying the underlying experts or router.