5 frontier LLMs grade 39 local models on 50 prompts; Bonsai-2 27B PQ2 tops the table
yobarisushcatel · reddit · 2026-10-29
A Reddit user ran a 10-20 hour crowdsourced benchmark: 39 local LLMs answered 50 prompts (66 tests, including 14 real-tool agent tasks) on a 5060 Ti 16GB rig, graded by 5 frontier judges — Gemini 3.8, GPT-6.1 Terra/Sol, Argon, and Claude.
Key findings:
- Overall winner: Bonsai-2 27B PQ2 (6.03 mean, 6.64 agent score, 13/14 tasks finished, 10/14 fixed, 57.4 tok/s), notably respecting the 'no core-code edits' Golden Rule constraint
- Runner-up Underdog-Saluki 27B (5.93), third Qwen3.6 35B-A3B (5.36); reference Qwen3.5 9B Q8 fixed at 5.00
- Judges also picked best fine-tuning bases: Nanbeige4.2 3B leads judgment (8.33) and honesty (8.5) while leaving VRAM headroom for sidecar models; Gemma-4 12B Q6 also favored
- Grading cost $30-40 plus two 5-hour Codex limits
Takeaway: new 27B models clearly beat the 9B reference on agent tasks, and boundary compliance (not editing core code) emerged as a key scoring factor.
More from Models
- Rime launches prism speech-to-speech model: 300ms response, 4x faster than OpenAI — EXM7777 · 2026-10-29
- Claim: Chinese labs now distilling from OpenAI's Astra after Anthropic blockade — bindureddy · 2026-10-29
- "Desired oververbosity 5": Claude spots OpenAI distillation fingerprints in DeepSeek reasoning traces — zeeshanp_ · 2026-10-29
- Qwen4 leak points to 1M-token context window, Qwen4-27B expected tomorrow — brainExploded99 · 2026-10-29
- Muse-Glimmer flops in hands-on test: user gives up at 40K tokens, switches to DeepSeek — techmago · 2026-10-29
- Pedro Domingos mocks NYT: the brain runs one dimension per synapse — a trillion — pmddomingos · 2026-10-29