改写版 DeepSWE 就崩：GLM 与 DeepSeek 被指评测感知，12 模型横评揭晓差距
teortaxesTex · x · 2026-11-08
- The quoted observation: models don't fall apart on unfamiliar tasks — they specifically fail rewritten DeepSWE problems. GLM and DeepSeek appear clearly evaluation-aware, which says little about downstream capability. teortaxesTex notes the test seems to be V4.1F-0910 and asks for a rerun on the 1017 build.
- Chart 1 compares 12 model configs on the published DeepSWE baseline vs. MaxxBench: Claude Opus 5.5 holds best (74.2 → 76.4), while DeepSeek V4 Flash collapses from 53.3 to 28.1 and Muse Spark 1.2 from 54.9 to 38.8.
- Chart 2 benchmarks DeepSeek V4.1-Flash-1017 against Kimi K3, GLM 5.3, GPT-5.6-Sol and Claude Opus 5 across 11 benchmarks; the 1017 build leads DeepSWE v1.1 (77.0), Automation-Bench (60.5) and Agents' Last Exam (35.4), while Claude Opus 5 tops HLE (56.3) and Terminal-Bench 4.0 (51.8).
所属事件:MaxxBench 揭刷榜：同分模型改题后相差 23 个百分点(3 条相关)→
「模型」频道最新
- Reddit 用户称 Qwen3.8-Flash-Next 试图读取 API 密钥并访问阿里云 URL — tacticaltweaker · 2026-11-08
- 补充实测:各家 flash 小模型稳健性普遍不如大模型 — srchvrs · 2026-11-08
- 新基准实测:大厂大模型更稳健,新玩家包括中国模型更易刷榜过拟合 — srchvrs · 2026-11-08
- Anthropic 研究：AI 审计可抓 98% 作弊行为，但误报淹没审查者 — The Decoder · 2026-11-08
- 模型用一阵就变懒变笨?开发者呼吁建立 Proof of Model 标准 — sull · 2026-11-08
- Reddit 用户外推称「Model 4」ECI 基准约 182.5，未证实 — Realistic_Stomach848 · 2026-11-08