SWE-at-Scale from SJTU synthesizes coding-agent benchmarks by inverting real bug fixes
SJTU · hf · 2026-10-16
Shanghai Jiao Tong University, with Tsinghua and ECNU, released SWE-at-Scale, a framework for scalable software engineering task synthesis.
- Motivation: real issue tasks are scarce and manual curation costly; existing synthesis relies on LLM knowledge plus heuristics and misses authentic bug mechanics
- Key insight: historical fixes encode mechanistic knowledge of bug formation — inverting a fix turns repair supervision into bug-induction supervision grounded in real dev scenarios
- Learned bug patterns are transferred to novel code contexts, sampling diverse bug typologies to build many executable tasks
- Aimed at benchmarking coding agents at scale
A fresh synthesis angle for relieving the data drought behind SWE-bench-style benchmarks.
More from coding & agent
- Four Gemini 4 Argon agents beat Minecraft's Ender Dragon autonomously in 27 hours — Maxence Faldor · 2026-10-16
- Tencent's Terminal-RSI builds recursively self-improving terminal agents, hitting 19.84% on Terminal-Bench 4.0 — tencent · 2026-10-16
- Tsinghua's AutoSFT uses multi-agent collaboration to automate SFT data research — TsinghuaNLP · 2026-10-16
- ByteDance's Athanor: autonomous agents post-train a small model to match its official tuned version — Yufa Zhou · 2026-10-16
- NVIDIA's EdgeWise cuts long-horizon agent inference costs up to 62% via selective cloud delegation — nvidia · 2026-10-16
- Intern-Decision: 0.8B-4B multimodal decision models output structured choices in one forward pass — internlm · 2026-10-16