22-model custom benchmark: Claude Opus 5.5 and GPT-6 Astra hit 100% while quantized Qwen variants trail
norenEnmotalen · reddit · 2026-10-13
The author shares final results from a personal benchmark (refined from a 500+ question pack by removing items all models pass) covering coding, data analysis, and local RAG across 22 models:
- Claude Opus 5.5 and GPT-6 Astra both scored 100% accuracy; Claude Haiku 5.5 hit 94.3%.
- Among quantized models, mradermacher--arex-2.q4km stands out at 93.3%; stock bf16 qwen3.8-27b scored 91.4% (1.9% truncated).
- qwen3.8-flash-next didn't disclose its quantization and was truncated most due to overthinking — even raising max tokens from 16k to 32k didn't eliminate it; qwen3.8-flash truncated 21.9% of the time.
- iq3 GSQ-RCO holds up decently among Q4s; mradermacher's fine-tunes continue to stand out.
The author uses this as a canary for when fully local workflows (spreadsheet/array/coding decisions, still reviewed by Claude or GPT) justify investing in pricey hardware.
More from Infra
- Cadence announces sovereign AI cloud for European chip designers — karlfreund · 2026-10-23
- SGLang Deep-Dive Blog Series Breaks Down KV Cache, Posts up to 15.5x Throughput Gains — BanghuaZ · 2026-10-23
- DynaLM: Open-Source CPU-First LLM Inference Engine Seeks Feedback — Unlikely_Safety_7456 · 2026-10-23
- Capital Is Now the AI Buildout Bottleneck, Says Analyst Citing Jensen Huang — JOBhakdi · 2026-10-23
- DynaLM: an open-source CPU-first LLM inference engine seeks testers and feedback — Unlikely_Safety_7456 · 2026-10-23
- Cerebras explains how it deploys trillion-parameter models on wafer-scale chips — bookwormengr · 2026-10-23