2B-parameter ColGrep-Agent matches 10x larger models: 76.18 file F1 on SWE-bench Lite
IgorCarron · x · 2026-10-12
Benchmarking ColGrep-Agent against much larger open-source models: with just 2B parameters it achieves 76.18 file F1 on SWE-bench Lite and 62.57 on Multi-SWE-bench Flash, matching models 10x its size and posting the highest line F1 on SWE-bench Lite, far above its MiniCPM5-2B base.
More from coding & agent
- Meta launches frontier models Muse Wonder and Muse Wonder Full for agentic tasks — AIatMeta · 2026-10-13
- SubQ Snap launches as a long-context decision model: 384K tokens, decisions in ~2-3s at $0.05/M — ryanallred · 2026-10-13
- One wrong 'merge it' prompt shipped ts-rust to T3 Code for 8 hours unnoticed — zeeg · 2026-10-13
- Don't specify languages or architecture for new projects — let the model research and choose — sytelus · 2026-10-13
- GPT-6 one-shots a custom IR blaster app to replace a dead AC remote — PrasoonPratham · 2026-10-13
- NVIDIA's NURBS-based CAD kernel SMLIB goes open source, with MCP agent support — burhop · 2026-10-13