6-bit quantization maintains accuracy for Qwen3-0.6B and 1.7B models
November 11, 2026
Quantizing Qwen3-0.6B and 1.7B weights to 6 bits maintains or slightly improves performance on ARC and HellaSwag benchmarks compared to fp16. At 4 bits, answer-slot skew increases significantly, rising from 0.15 to 0.53 for the 0.6B model.
HOW THIS AFFECTS YOU
●
builderYou can likely use 6-bit weights to save memory without losing accuracy on small models.
●
researcherThe study explores the trade-off between bit precision and option-order averaging in local LLMs.