High-speed Qwen3.8-27B inference on single RTX 3090
September 1, 2026
An optimized inference engine achieves 2,000 tokens per second prefill and 132 tokens per second decode for Qwen3.8-27B on an RTX 3090. The speedup is driven by a custom int8 kernel that maintains 0.99997 similarity to fp32.
HOW THIS AFFECTS YOU
●
builderYou can achieve near-production-grade inference speeds for 27B models on consumer-grade hardware.