ARCHead compresses the final language-modeling head using a quantized low-rank core and group-wise INT4 residuals. On Qwen3-8B-Base, it reduces LM-head storage to 25.6% of the original BF16 size while maintaining a relative perplexity of 1.007.
HOW THIS AFFECTS YOU
●
builderYou can significantly reduce the memory footprint of quantized LLMs during deployment.