SWE-sweep benchmark code open-sourced under facebookresearch on GitHub
OfirPress · x · 2026-10-02
The SWE-sweep benchmark code is now open source on GitHub under facebookresearch. It is a thin wrapper around Harbor; agents autonomously find and fix bugs in real repos without hints. The repo includes evaluation tasks, separate verifier environments, and micro-average scoring, and currently pins a specific Harbor revision since Harbor 0.23 lacks the required hooks.
Related event: Meta's SWE-sweep: Best Models Fix Only 4.7% of Hidden Bugs(6 posts)→
More from Research
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- LatamGPT releases 178.4M-document open Latin American corpus on Hugging Face — OmarUFlorez · 2026-11-16
- Open-source TotalSegmentator spots a 42mm dilated aorta on a routine chest CT — MaziyarPanahi · 2026-11-16
- Tivadar Danka Publishes 4,000-Word Visual Guide to Graph Theory for Machine Learning — TivadarDanka · 2026-11-16