SWE-sweep Leaderboard: Top Models Double Bug Finds at Half the Cost, Still Under 10%
stanfordnlp · x · 2026-10-24
The updated SWE-sweep vulnerability-finding leaderboard shows OpenAI and Anthropic models doubling the number of bugs found while halving costs. Astra leads but still finds less than 10% of all real-world bugs tested, per KLieret's thread shared by Stanford NLP.
Related event: SWE-sweep Update: OpenAI, Anthropic Models Double Bug Finds at Half Cost(2 posts)→
More from Models
- Anthropic secretly A/B tests Claude limits: same $200 plan, ~20% less usage — RexDouglass · 2026-10-24
- Kimi K3.1 Pro Benchmarks: Tops SWE Marathon at 51.7, Near Opus 5.5 Across Coding Boards — scaling01 · 2026-10-24
- Kimi 3.1 priced at $2/$10 per 1M tokens, still not competitive vs DeepSeek, says investor — bindureddy · 2026-10-24
- NVIDIA's Nemotron 3.5 Bolt lands on Ollama: small VLM with 128K context — ollama · 2026-10-24
- LlamaIndex's OCRia-1 matches top parsers at 1/4 the cost, 7x faster than Claude — llama_index · 2026-10-24
- Experts say OpenAI's redacted paper proof strategy is likely irreparable — teortaxesTex · 2026-10-24