MP-ISMoE: Mixed-Precision Interactive Side Mixture-of-Experts for Efficient Transfer Learning

Researchers from arXiv introduce MP-ISMoE, a Mixed-Precision Interactive Side Mixture-of-Experts framework combining Gaussian Noise Perturbed Iterative Quantization (GNP-IQ) with low-bit weight compression to reclaim memory for expanded side-network MoE capacity. This directly targets the core METL tradeoff where strict memory budgets cripple side-network expressiveness. ML engineers fine-tuning large foundation models under GPU memory constraints should prioritize this over standard PETL approaches like LoRA or adapter tuning.