Signal65 tests show open-weight models closing on the frontier, undercutting pace calls
ryanshrout · x · 2026-09-15
As OpenAI, xAI and Anthropic backed a call to pace frontier AI development, Signal65's PINNACLE agentic benchmark scored eight new configurations — five of them open-weight from Qwen, DeepSeek and Zai.
Key results:
- Open-weight Qwen3.8-2.4T-A95B made fewer weighted errors than Claude Opus 5
- DeepSeek-V4.1-Flash cut errors by a third vs. its previous generation at max effort
- Gemini 3.8 Flash completed 99.6% of multi-step jobs
Argument: "You cannot pace a frontier you have not measured." PINNACLE scores real multi-step enterprise work with code-verified, deterministic scoring (no model judges), measuring correct work, speed and cost. Open weights are closer to the hosted frontier than the pacing conversation admits, leaving little room to slow down. NVIDIA (Blackwell Ultra as reference platform) and AMD both endorsed the benchmark.
More from Models
- 'You Can Feel the Gradient Slopes': Dev Contrasts Benchmaxxed RL Models vs General Ones — theteknosaur · 2026-11-02
- LightOnOCR-3 Launches: Tops FRBench, Adds Layout Structure and Chart Extraction — IgorCarron · 2026-11-02
- NVIDIA's Kumo Is a Family of Four Tabular Foundation Models, Not Just One — pandeyparul · 2026-11-02
- SemiAnalysis: safety classifiers are underrated — Meta's Muse tops cyber benchmark at 35.2% — ThomasScialom · 2026-11-02
- DeepSeek V4.1 Flash 1017 wins over a longtime closed-source-only user: fast, cheap, reliable — _bachrc · 2026-11-02
- Local quants vs Claude on 9 real private bugs: 3-bit costs thinking time, not score — FeydRowan · 2026-11-02