LightOn's open 2B model cuts Claude Code token bills by 34% via local semantic code search
lmoroney · x · 2026-10-13
LightOn measured how much of a coding agent's token bill goes to just finding the right file, and released ColGrep-Agent-2B, a 2.5 GB local model that handles semantic code search. Plugged into Claude Code across 52 questions on 13 repos, it cut API tokens 34% vs grep (40% cache-aware), with savings growing to 40% on 150k-600k line repos — at the cost of slower responses (21.6s vs 13.0s) and a possibly-noise 80.8% vs 83.3% accuracy gap. Training recipe: 2B base, short supervised cold start, then GRPO with file/line-F1 rewards.
More from coding & agent
- FactoryAI ships Environments, cloud sandbox for agents, now GA — matanSF · 2026-10-13
- Researcher on doing science in the agent era: ask the hard questions, verify every line — CatAstro_Piyush · 2026-10-13
- "Give me an LLM and a test suite": Archimedes quote gets an AI rewrite — maxleiter · 2026-10-13
- Vibe-coded port of a language itself cuts typechecking time 3x with Opus — maxleiter · 2026-10-13
- Network engineer of 20 years: once you go declarative for AI automation, you never go back — tristanbob · 2026-10-13
- Muse Wonder scores ~42% on senior engineer benchmark vs Sol 6.1's 71% — danshipper · 2026-10-13