LightOn's 2B search agent matches 27B open-source models on adapted SWE-bench evals
IgorCarron · x · 2026-10-12
LightOn built a 2B code-search agent and adapted SWE-bench Lite / Multi-SWE-bench Flash to evaluate it, finding it competitive against much larger open-source models.
- Recipe: MiniCPM5-2B base, SFT on Qwen3.5-27B traces, then GRPO with a reward weighting file F1 + line F1
- Per the charts, colgrep-agent-2B hits 69.5% file F1 and 18% line F1, beating Qwen3.5-27B and gemma-4-31B-it on line F1
- Shows small local models can be tuned into strong search agents
Related event: 2B ColGrep-Agent Matches 10x Larger Models on Code Localization(3 posts)→
More from coding & agent
- DigitalOcean launches Agent Learning and Agent Data Services to catch agent failures early — JafarNajafov · 2026-10-12
- AI agents beat Wall Street on 68% of earnings forecasts in public contest — PrajwalTomar_ · 2026-10-12
- Microsoft's Fork-and-Flush helps agents escape "idea basins," improving results 66% — DimitrisPapail · 2026-10-12
- User gives Grok bot a wallet, its memecoin does $155K volume in a day — nima_owji · 2026-10-12
- OmniRoute connects Claude Code to 200+ AI providers with auto-failover and 90% token savings — hunain8007 · 2026-10-12
- Redditor builds friend a full HD-2D adventure game with Claude Code and raw WebGL2 as a birthday gift — rubenrb02 · 2026-10-12