Scale-QLoRA Enables Efficient Merging for 4-bit Microscaling LLMs
September 7, 2026
Scale-QLoRA prevents adaptation loss when merging LoRA adapters into native 4-bit (NVFP4/MXFP4) models by adapting only the per-block scale field. This method freezes E2M1 code planes to avoid the 39 percentage point degradation caused by naive re-quantization.
HOW THIS AFFECTS YOU
●
builderYou can deploy merged 4-bit adapters without the accuracy degradation typical of naive quantization re-runs.
●
researcherThis introduces a method for scale-invariant adapter training on discretized code planes.