5 cheap open models vote vs Opus: 89.6% accuracy at 1/10th the cost
PomegranateAble4339 · reddit · 2026-10-31
A Reddit user tested replacing a frontier model with a voting panel of 5 cheap open models (DeepSeek V4.1 Flash, GLM 5.3 Flash, MiMo V2.6 Flash, Qwen3.8 Flash, Gemma 4 26B-A4B) on yes/no and label-classification pipelines (spam, fraud, moderation, triage). When all 5 agree, the answer ships; disagreements go to a human with a one-line reason.
Across 2,850 decisions on 6 public datasets vs Opus 5.5:
- The panel agreed on 71% of items, shipping answers that were 89.6% correct vs Opus's 90.5% on its most confident 71%
- Cost: $0.61 vs $5.79 per 1k decisions (1/10th)
- Overall accuracy is worse (80.1% vs 83.1%) — the panel only wins where it's confident
It fails on long-answer comparisons and 77-way intent classification, and on hate speech it catches more (80% vs 62%) but with more false alarms (29% vs 20%). Gains over the best single model (GLM 5.3 Flash, 88.9%) are within noise — the real value is knowing when to escalate to a human. Author notes the panel was scored post-hoc and not yet validated on a fresh sample. Works with any OpenAI-compatible backend (Ollama, vLLM, llama.cpp); MIT-licensed code on GitHub (modeljury).
More from Infra
- Musk envisions lunar mass drivers launching compute to deep space for a petawatt-scale future — XFreeze · 2026-10-31
- DOE approves first commercial nuclear microreactor safety review; Radiant cleared for full-power testing — McDonaghMatthew · 2026-10-31
- Measured: lossy engine flags drop a Q8_0 model to roughly Q2_0 quality — jesdga95 · 2026-10-31
- Dev shows off eye-popping LLM cache hit rate: inference costs plummet — sujingshen · 2026-10-31
- Joules, not tokens: essay argues energy is the only substrate-neutral compute denominator — tctjr · 2026-10-31
- Geek charts 3-box, 384GB home AI rig on the road to 1TB of RAM — gnukeith · 2026-10-31