Specific Labs Launches Real-SWE, a Benchmark on Private Enterprise Codebases
zainhas · x · 2026-09-12
Specific Labs has released Real-SWE, a benchmark that evaluates frontier AI models on private, real-world enterprise codebases rather than synthetic public-repo tasks. First results: Fable 5.1 at 38.8%, GPT-6 Astra at 33.8%, and GLM-5.3 at 28.8% — open-weight models closing in on the frontier. Details at realswe.withspecific.com.
Related event: Real-SWE Benchmark Tests Frontier Models on Private Enterprise Code(2 posts)→
More from Models
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
- OpenAI to block ChatGPT weapons details for teens, auto-end violent fixations and alert parents — rohanpaul_ai · 2026-10-31
- Granite Switch 4.1.1 ships: update models like you update software — krvarshney · 2026-10-31
- Mistral Open-Sources Le Chonk: Top Open Model on FrontierCode, Cyberdefense-Tested by CrowdStrike and Cloudflare — tensorqt · 2026-10-31
- Stanford CS336 Lecture Explains Why DPO Compares Against a Reference, Not Two Current Probabilities — stanfordnlp · 2026-10-31
- ik_llama.cpp PR #2808 adds native runtime for China Telecom's Xing4.0-29B-A4B MoE with mHC and MLA — Any_Bus9205 · 2026-10-31