Community Benchmarks: Ternary-Bonsai-2-27B Extreme Quantization Is Not Lobotomized
Fancy-Snow7 · reddit · 2026-09-20
A Reddit user ran a suite of homemade benchmarks on prism-ml's Ternary-Bonsai-2-27B-PQ20 (ternary + PQ20 extreme quantization), comparing it against the similarly-sized unsloth Qwen3.8-27B-UD-IQ2XXS plus several larger dense and MoE models. Verdict: the model is "not completely lobotomized."
The test suite includes: a standard 21-needle test, a self-designed hard passkey test with decoy GUIDs requiring multi-hop context traversal, phrase reconstruction from 8 fragments, 500 science multiple-choice questions (to gauge quantization knowledge loss), and a custom 1000-paragraph prose memory test.
Key finding: the larger the context, the more models forget — the author advises against blindly chasing longer contexts, noting that compaction summaries in smaller contexts retain more, and raising KV cache to F16 barely rescues failures on the hard needle test.
More from Models
- Community asks Moonshot when Kimi K3.1 model weights will drop — bgmshana · 2026-11-04
- Meta's Muse Spark Open license counts distilled models as derivatives — aidenclarke_12 · 2026-11-04
- Tencent Hy4 trails GPT-6.1 on hard tests as Kimi K3.1 Pro beats it on two benchmarks — heyshrutimishra · 2026-11-04
- Tencent's Hy4 agent self-trained a smaller model that beat Tencent's own baseline — heyshrutimishra · 2026-11-04
- Hy4 tops open models in finance eval at 3.04, hallucination rate cut to 5.45% — heyshrutimishra · 2026-11-04
- Tencent Hunyuan teases upcoming LMArena debut — TencentHunyuan · 2026-11-04