Precision vs. Permutation Budgeting in Small Language Models
November 9, 2026
Analysis of Qwen3-0.6B and 1.7B shows that 6-bit quantization maintains accuracy against fp16, while 4-bit quantization causes answer-slot skew to rise from 0.15 to 0.53. The study compares the cost of weight precision versus averaging option-order permutations.
HOW THIS AFFECTS YOU
●
builderYou can use 6-bit quantization on small models like Qwen3 to save memory without losing accuracy.
●
researcherThis shows how quantization-induced skew impacts model sensitivity to answer ordering.