Ternary Bonsai-2 vs Qwen3.5-2B on A13/A14/M3: 32-55% faster decode, and why thermal logging matters

leo-k7v · reddit · 2026-11-21

A developer benchmarked PrismML's Ternary-Bonsai-2 (2B/4B, Q20 ternary weights with Hadamard fold) in his own Swift+Metal inference engine against Qwen3.5-2B IQ4NL on three Apple chips.

Key lesson: log thermal state

Results

Optimizations

Details come from the author's ChatOKF project, a pure-Swift-on-Metal local GGUF client.

Original post →

More from Infra

Infra channel →