GaugeQuant Reduces LLM Quantization Perplexity via Symmetry Breaking
July 24, 2026
GaugeQuant uses a LogSumExp loss term to break internal transformer symmetries during training, selecting bases that minimize activation outliers. For LLaMA-2 7B, W4A4 quantization perplexity dropped from 8.22 to 6.73, and W4A16 dropped from 11.16 to 5.45 without requiring calibration data.
HOW THIS AFFECTS YOU
●
builderYou can achieve higher performance in quantized models with negligible training overhead.
●
researcherThis method provides a new way to optimize quantization during the training phase.