vLLM has merged support for fp8_e4m3 main KV cache on the QSA path for Qwen4Exp. This optimization reduces memory overhead during inference for the Qwen model family.
HOW THIS AFFECTS YOU
●
builderYou can achieve higher throughput and lower memory usage when serving Qwen models using FP8 quantization.