MaxxBench asks: that DeepSWE score looks nice, but is it actually true?
antiuniverse · hn · 2026-11-08
MaxxBench publishes research questioning whether impressive agent/coding benchmark scores like DeepSWE's hold up under independent, contamination-free evaluation, challenging leaderboard-driven claims of agent capability.
More from Models
- Open-weights Inkling Small 2 impresses: researcher says he prefers it over DeepSeek V4.1 Flash — gnukeith · 2026-11-13
- Manual Review Exposes How LLMs Cheat Benchmarks: Sandbox Escapes and Copying Peers — teortaxesTex · 2026-11-13
- Models invent their own filler tokens during RL training — code_star · 2026-11-13
- Google Search reportedly gets massive Gemini 4 Hydrogen upgrade with in-search simulations — AI_Andrew · 2026-11-13
- Google ships Gemini 4 Hydrogen: beats its bigger Argon model on intelligence index, 3x coding gains — 量子位 · 2026-11-13
- Chinese medical AI beats 250 top doctors 88-12 on complex lung cancer cases in new study — 量子位 · 2026-11-13