DeepSeek v4.1 Flash Tested Across 8 Coding Harnesses: Performs Best in Minimal Setups
mariofilhoml · x · 2026-09-11
DeepSeek evaluated its new v4.1 flash model under 8 different harness configurations: it performs best in minimal harnesses (mini-SWE and DeepSeek's own minimal harness) but underperforms in both Claude Code and Codex. A replier notes this is likely explained by RL checkpoint selection against DeepSWE-like benchmarks — an eval-harness selection bias.
Related event: DeepSeek Tests v4.1 flash Across 8 Harnesses, Minimal Setups Win(3 posts)→
More from coding & agent
- TRL adds Score Centering, a simple fix for RLHF training instability — lvwerra · 2026-10-27
- Precompute golden answers and match at runtime for exact wording control — colinmcnamara · 2026-10-27
- Dev argues there's still alpha in working with your coding agent interactively — alex_frantic · 2026-10-27
- Researchers launch AMA on patterns for safer AI agents used by citizen developers — Loose_Rip359 · 2026-10-27
- Gradio Adds In-Chat Human-in-the-Loop So ML Intern Pauses Before Training — Gradio · 2026-10-27
- Building Halo Infinite Forge maps with local LLMs, DCjson and Blender MCP — swagonflyyyy · 2026-10-27