Qwen2.5-27B runs at 70 tok/s on iPhone A20 Neural Engine
October 17, 2026
Qwen 2.5-27B achieves 15–70 tok/s on iPhone 18 hardware using 1.5 bpw quantization via Bonsai-2. The implementation supports 24K context with int8 KV caching and uses a dflash-2 draft model at 2.8 bpw to optimize decoding.
HOW THIS AFFECTS YOU
●
builderYou can now target high-speed local inference for 20B+ parameter models on mobile hardware.
●
founderOn-device intelligence becomes viable for privacy-first applications without relying on cloud latency or costs.