SciCode 榜单 21 个月暴涨 31 分,前沿模型科研解题率升至 67%
geoffwolfe · x · 2026-10-29
Artificial Analysis data shows the best score on the SciCode research-coding benchmark rose from 35% in December 2024 to 67% today—a 31-point gain in 21 months.
SciCode gives models real research problems broken into the steps a scientist would take, with unit tests ideally grounded in peer-reviewed papers.
Key milestones:
- Claude 3.5 Sonnet 35.4% → o3-mini 42.8% → Gemini 2.5 Pro 46.3% → Gemini 3.1 Pro 58.7% → Claude Fable 5 61.0% → Claude Opus 5.5 66.9%
- SciCode-Verified (Aug 2026): domain experts audited every test, found 263 defects, corrected them, and re-scored 12 frontier models—84–98% of subproblems, 69–92% of full research problems
The author argues the benchmark looks saturated but the approach isn't—there are tens of thousands of other scientific computing problems. Verifiable rewards unlocked reasoning models like DeepSeek-R1, making executable science an excellent training signal.
「模型」频道最新
- 网传 Gemini 4 Carbon 登顶基准榜，真实性存疑 — gnukeith · 2026-10-29
- 用户担忧美系模型审查过重：别默认人人都是黑客 — gnukeith · 2026-10-29
- Google 迭代太慢遭批：新模型没发布就可能已被对手超越 — Angaisb_ · 2026-10-29
- 研究员批评 Google:Gemini 新模型不公开可用就别发跑分 — natolambert · 2026-10-29
- 开发者呼吁做「AI 订阅性价比」对比工具，称 Google 额度最慷慨 — timigod · 2026-10-29
- Bindu Reddy:Carbon 若按 Argon 定价将比竞品便宜 5 倍,或引发小地震 — bindureddy · 2026-10-29