Distributed Inference of Qwen 3.8 Flash Next Across Heterogeneous Hardware
October 17, 2026
A 48-layer mixture-of-experts model with 512 experts per layer achieves 37-39 tokens per second at 60k context using distributed inference. The setup chains an RTX 2060 Super, RX 9060 XT, M4 Pro MacBook, and two iPhone 17 Pro Maxes to manage the model's memory requirements.
HOW THIS AFFECTS YOU
●
builderYou can explore distributed inference techniques to run large MoE models on consumer-grade, heterogeneous hardware.
●
researcherThis demonstrates the practical feasibility of extreme expert-parallelism across varied silicon architectures.