[arXiv]score: 0.81
Walsh-Hadamard Spectral Rotations Cut W2 Perplexity 15–58% on Sub-2B LLMs
May 26, 2026
Applying influence-adaptive Walsh-Hadamard Transform rotations and per-coordinate spectral rescaling before quantization reduces wikitext-2 perplexity by 15–58% relative to vanilla auto-round at W2A16 on models from 135M to 1.5B parameters.
cs.LGcs.AIcs.LO
HOW THIS AFFECTS YOU
●
builderDrop-in pre-quantization transformation compatible with Intel auto-round that meaningfully improves 2-bit model quality without retraining.
●
researcherConcrete perplexity gains at W2 quantization via a math-invariant pre-rotation, with extensions for Qwen3 attention norms and RoPE-compatible SO(2) rotations.