DeepSeek V4.1 Flash 1017 aces 10-task coding benchmark at under 5 cents per task
Fabulous_Pollution10 · reddit · 2026-10-18
A Nebius coding-agent/eval engineer ran the new DeepSeek V4.1 Flash 1017 through his personal benchmark of 10 real work tasks (3 runs each, hidden tests):
- Results: 100% pass-all-3 on high effort, clearly stronger than the September version and 1.7x faster on max — at under 5 cents per task.
- Token-hungry: 550k tokens per task vs 30k for Opus 5.5 on low, but nearly all input hits cache, keeping costs at a few cents.
- Harness comparison: DeepSeek's own harness uses a 7k system prompt with native file tools; Pi relies almost entirely on bash. On low, DeepSeek Harness caught a corner case Pi missed (29/30 vs 27/30 pass@1) but burned 75% more tokens.
- Context: GPT-6 Luna and Haiku 5.5 also hit 100% on Pi at $0.0145/$0.0208 per task; Opus 5.5 low costs $0.1168.
- The author also published a guide on turning your own agent sessions into evals and plans a harder benchmark next.
More from coding & agent
- Self-built dashboard finds 62.9% of 66,757 server requests are automated — Thionne_WTZ · 2026-10-18
- The fill-in-the-blank prompt: Claude as motion director and Remotion coder, end to end — aziz4ai · 2026-10-18
- No After Effects needed: a 5-layer prompt has Claude script, Gemini voice and Remotion render motion graphics — aziz4ai · 2026-10-18
- Founder open-sources Brambit, a self-hosted AI assistant engine with memory, connectors and sandbox — Trincaa · 2026-10-18
- Dev builds a fully ASCII MMORPG — entire world rendered in colored text characters — HanchungLee · 2026-10-18
- AI agent made 200 calls in 90 minutes and got its user banned from Resy — AI Engineer · 2026-10-18