Running the 68GB QSQ-RCO-IQ2_XS quantization of Qwen 3.8 Flash Next yields 22 tok/s sustained on an RTX 3060 12GB and 16GB DDR4 RAM. This configuration maintains bit-exact accuracy without using expert pruning to achieve speed.
HOW THIS AFFECTS YOU
●
builderYou can now run high-parameter Qwen models on consumer-grade 12GB VRAM hardware with usable inference speeds.
●
researcherThis demonstrates the efficiency of extreme low-bit quantization for MoE models on limited hardware budgets.