DeepSeek evals v4.1 flash in 8 harness configs — best in minimal harnesses, worse in Claude Code and Codex
zainhas · x · 2026-09-10
DeepSeek benchmarked its new v4.1 flash model across 8 different agent harness configurations. The model performs best in minimal harnesses (mini-SWE and the minimal deepseek harness) but underperforms in both Claude Code and Codex — a sign its fit with heavier harnesses still needs work.
Related event: DeepSeek Tests v4.1 flash Across 8 Harnesses, Minimal Setups Win(3 posts)→
More from coding & agent
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- AutoResearch agent's weird training curve gets dug into instead of discarded — ZimingLiu11 · 2026-10-27
- Firecrawl offers 2.5x credits to a16z Speedrun founders, unveils $75M Series B — devdigest · 2026-10-27
- Chums: open-source Apple Watch app puts Claude Code and Codex on your wrist — brandon_galang · 2026-10-27
- Open-source Clairic Studio turns pasted text into typed data and working code in 5 languages — dfinke · 2026-10-27