DeepSeek V4.1 Flash jumps on long-run agentic tasks but regresses on constraints
teortaxesTex · x · 2026-10-18
An eval breakdown of DeepSeek V4.1 Flash 1017 shows the biggest gain in Long-Run Operations (1→35), with the Agentic score rising from 40.1 to 50.9 and overall from 78.0 to 79.2.
But there are concerning regressions:
- Analysis-Driven Action: 100 → 67 (-33)
- Constraint Adherence: 100 → 67 (-33)
- Tax & Finance: 91 → 82 (-25)
Against GPT-5.6 Terra (75.0 overall, $1.691), DeepSeek scores higher at $0.083 — roughly 20x cheaper. The structural shifts suggest capability trade-offs under the hood.
More from Models
- Polymarket opens betting market on who releases the next Gemini Argon model — Polymarket · 2026-10-18
- Grok matter-of-factly lists which groups user was racist against, exempting Africans and Latin Americans — teortaxesTex · 2026-10-18
- A quick test for benchmaxxed demos: swap in Mirror's Edge and see if it generalizes — teortaxesTex · 2026-10-18
- Did OpenAI distill DeepSeek's gains? Benchmark shows OpenAI's Luna scoring just 38 — teortaxesTex · 2026-10-18
- Gemini 4 reportedly near Opus 5.5 on complex tasks, surprisingly strong at security — haider1 · 2026-10-18
- vega-U: a 14.3M decision engine that cold-attaches to any frozen LLM, 0.811 held-out accuracy — kalyan_kpl · 2026-10-18