FrontierSWE v2 Launches: 20-Hour Long-Horizon Coding Benchmark, Fable 5.1 Leads by 24+ Points

ProximalHQ (nrehiew and team) released FrontierSWE v2 on September 3, an ultra-long-horizon coding benchmark requiring models to work autonomously for up to 20 hours on extremely hard tasks, including building an OpenGL engine from scratch that renders a flight simulator game. Fable 5.1 emerged as the best model, leading by more than 24 percentage points.

Confirmed

• FrontierSWE v2 expands the task set and improves evaluation methodology, positioned as an ultra-long-horizon coding benchmark
• The authors found that native standard harnesses not designed for long-horizon work, including Codex and Claude Code, artificially depress model performance, so they built the custom Proximus harness on top of mini-swe-agent, improving both scores and how long models keep working
• To address weak time and budget awareness, they introduced a submit tool: each submission reminds the model of its time budget and remaining time, encouraging further iteration; similar to /goal-style implementations, but the extra feedback lets the model decide autonomously whether to continue, eliciting harder thinking
• The authors argue that with tasks taking 20 hours and roughly 200 million tokens, large-scale post-training is nearly infeasible (infrastructure and context-compression challenges), making such tasks a proxy for out-of-distribution evaluation that tests genuine generalization

Why it matters

The benchmark pushes evaluation duration to the 20-hour scale, exposing weaknesses in mainstream agent harnesses and model time awareness, and offers a reference frame for long-horizon autonomy and generalization that is hard to game through targeted training.

2026-09-03 ~ 2026-09-03 · 5 related posts

Full story(2 episodes)→

• Episode 1: FrontierSWE v2 Launches: 20-Hour Long-Horizon Coding Benchmark, Fable 5.1 Leads by 24+ Points(2026-09-03, 5 posts)
• Episode 2: Designing Hack-Proof Benchmarks as Models Game the Verifiers(2026-09-03, 3 posts)

• [source] FrontierSWE v2: 20-hour autonomous tasks, Fable 5.1 best model by 24+ points — nrehiew_ · 2026-09-03
• Why 20-hour, 200M-token tasks make mass post-training nearly impossible — nrehiew_ · 2026-09-03
• Codex and Claude Code harnesses handicap long-horizon tasks; custom Proximus harness lifts scores — nrehiew_ · 2026-09-03
• A submit tool that reminds models of their time budget makes them iterate longer — nrehiew_ · 2026-09-03
• FrontierSWE v2 benchmark launches, Claude Fable 5.1 leads frontier models by wide margin — brianryhuang · 2026-09-03