Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory
cephaloform · x · 2026-10-16
The new Bonsai 2 2B model reaches up to 120 tokens/s on iPhone 18 Pro. According to the benchmark shared, it outperforms 4-bit Qwen 3.5 2B while running twice as fast and using almost half the memory — a notable step for on-device small-model deployment.
More from Infra
- EpochAI pegs GPT-6 training at up to 1.8e27 FLOP; scaling01 calls param estimates way too high — scaling01 · 2026-10-16
- Meta's Muse Spark Open scores 39 vs 48 for closed Spark 1.3 on AA Intelligence Index — EAccelerate_42 · 2026-10-16
- LLM spots useless 95GB-per-frame memcpy in ffmpeg dblur, cutting runtime from 2452ms to 49ms — IgorCarron · 2026-10-16
- Tree packing lands in TRL's AsyncGRPO, cutting agentic RL training compute by over 26x — joecole · 2026-10-16
- ASM's pizza-themed ALD explainer at SEMICON West is the best yet — BenBajarin · 2026-10-16
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16