Edge0 Engine Serves 35B MoE at 20tok/s via SSD Offloading
September 17, 2026
Edge0 enables Mixture-of-Experts inference on 24GB consumer hardware by using a per-layer head to predict routing one token ahead. This allows for proactive SSD expert streaming, achieving 20tok/s for a 35B model within 3GiB of peak active memory.
HOW THIS AFFECTS YOU
●
builderYou can deploy large MoE models on low-VRAM hardware using SSD-based streaming.
●
researcherThis demonstrates how predictive routing can mitigate the latency of offloading sparse weights.