Internal eval: DeepSeek V4.1 Flash nearly SOTA at 98% lower cost than Fable 5.1
himanshustwts · x · 2026-09-11
The author's team evaluated 11 frontier models on the same internal sample of 22 tasks spanning engineering, bizops, finance and healthcare:
- DeepSeek V4.1 Flash scored nearly SOTA on KnowledgeWork (15/21 passes) while being 98% cheaper than leader Fable 5.1 (18/22 passes).
- The biggest surprise was GPT-6 Astra, which passed only 11/22.
- Analysis of agent trajectories is just starting, with a full thread to follow.
More from Models
- ValsAI launches RSI Index, first third-party benchmark measuring how close AI is to self-improvement — JenniferHli · 2026-09-11
- Devin's New Model Verdict: Not a Benchmaxxer, a 'Killer Execution Model' at $20/Month — brandon_galang · 2026-09-11
- Business Insider Asked ChatGPT, Gemini, Claude and Grok How AI Could End Humanity — coinfanking · 2026-09-11
- Claims resurface that Moonshot's Kimi distilled from Claude raw CoTs — xuanalogue · 2026-09-11
- User switches back to GPT-5.6 Sol: barely uses quota and feels faster — CtrlAltDwayne · 2026-09-11
- Dev opinion: model differences shrink in a good harness; Grok 4.6 is good enough — gnukeith · 2026-09-11