OpenBMB runs 35B MoE model at 20.5 tokens per second on iPhone
October 21, 2026
OpenBMB's MiniCPM-V 4.7 uses Lookahead Expert Scheduling (LES) to preload expert weights from flash storage during computation. This method achieves 20.51 tok/s on an iPhone 18 Pro Max for a 35B-A3B MoE model without skipping experts or altering routing.
HOW THIS AFFECTS YOU
●
builderYou can deploy larger, high-performance MoE models locally on mobile hardware with significantly reduced latency.
●
researcherThe LES technique demonstrates a viable method for overcoming flash storage bottlenecks in local MoE execution.