ResCom uses activation rounding residuals to perform low-rank error compensation for quantized LLMs without requiring gradient updates. On Llama-3-8B with 4-bit weight-activation quantization, it improved MMLU by 2.89 percentage points and raised five-shot GSM8K accuracy from 23.96% to 36.16%.
HOW THIS AFFECTS YOU
●
builderYou can significantly recover accuracy in 4-bit quantized models using this closed-form correction method.
●
researcherThe use of output-aware calibration to select residual directions provides a new path for post-training quantization.