LightOn Open-Sources ColGrep-Agent-2B: Local 2B Model Cuts Claude Code API Costs by 40%

LightOn has released and open-sourced ColGrep-Agent-2B: a 2B code-search agent that runs fully locally (CPU-only supported), taking over the "find the relevant files first" step from the expensive main coding model. It performs semantic search and file reading with ColGREP, a late-interaction-based multi-vector retrieval tool, then returns code locations and content directly to the main agent. The project was built by @Aarush1003 (an intern at LightOn), Amelie Tabatta, and others; the blog post, model weights, GGUF versions, the colgrep tool, and the paper are all available.

Confirmed

• Evaluation setup: in Claude Code (Sonnet 5.5), 52 code-search questions across 13 recent code repositories (deliberately chosen to be new, to avoid the model answering from memory) compared grep, ColGrep, ColGrep-Agent, and the default Balanced mode, totaling 624 benchmark sessions.
• Key numbers: ColGrep-Agent-2B cut Claude Code's API token usage by 34.3% and costs by roughly 40%; average turns per question dropped from 5.5 to 3.9, and tool calls from 7.1 to 4.0 (grep/find/ls calls 5.1→2.0, file reads 2.0→0.9), with the efficiency coming from fewer repeated searches and file reads.
• Accuracy: the grep baseline scored 83.3% correct, pure ColGrep 79.5% (-3.8pt), and ColGrep-Agent-2B 80.8% (-2.6pt); the default Balanced mode combining direct semantic search with the local agent matched grep's 83.3% while reducing API tokens by 21% and costs by 30%.
• Trade-off: delegating to the local agent is slower — 21.6 seconds per question versus 13.0 for grep (Balanced mode: 17.7), i.e., cost savings in exchange for acceptable latency.
• Model capability: on adapted SWE-bench Lite and Multi-SWE-bench Flash code-localization evaluations, it achieved file F1 of 76.18 and 62.57 respectively, competitive with open-source models 10x its size (e.g., the 27B class); the training base is MiniCPM5-2B (first distilled for retrieval using Qwen3-related pipelines, among other steps — see the paper for the exact recipe).

Why it matters

Coding agents burn enormous numbers of tokens on repeated greps and file reads. Using a small 2B local model as a search middle layer can sharply cut API costs with essentially no accuracy loss, runs fully locally, and is privacy-friendly — offering a reproducible open-source example of a layered agent architecture where a large model orchestrates and small models do the work.

2026-10-12 ~ 2026-10-12 · 18 related posts

• ColGrep-Agent-2B: a local code-search agent that cuts 34% of API tokens — IgorCarron · 2026-10-12
• Benchmarking Claude Code search tools: grep vs ColGrep on 52 code-search tasks — IgorCarron · 2026-10-12
• ColGrep-Agent cuts Claude Code API tokens 34.3% and cost from 7.3¢ to 4.4¢ per question — IgorCarron · 2026-10-12
• ColGrep-Agent cuts per-question tool calls from 7.1 to 4.0 by searching locally — IgorCarron · 2026-10-12
• LightOn's 2B search agent matches 27B open-source models on adapted SWE-bench evals — IgorCarron · 2026-10-12
• LightOn ships 2B code-search agent ColGrep with model, GGUF and paper — IgorCarron · 2026-10-12
• ColGrep-Agent-2B cuts Claude Code API tokens 34% and costs 40% with local code search — IgorCarron · 2026-10-12
• With ColGrep-Agent, Claude Code drops from 5.5 to 3.9 turns per question — IgorCarron · 2026-10-12
• ColGrep Balanced matches grep accuracy (83.3%) while saving 21% of API tokens — IgorCarron · 2026-10-12
• Local search agents trade latency for savings: ColGrep Balanced cuts cost 30% at 17.7s per question — IgorCarron · 2026-10-12
• 2B-parameter ColGrep-Agent matches 10x larger models: 76.18 file F1 on SWE-bench Lite — IgorCarron · 2026-10-12
• ColGrep-Agent: a 2B-parameter local semantic-search agent matches 10x larger models on code localization — IgorCarron · 2026-10-12
• ColGrep-Agent-2B: open local multi-vector search model built for coding agents — IgorCarron · 2026-10-12
• ColGREP-agent: a local model to stop coding agents wasting tokens on file search — IgorCarron · 2026-10-12
• [source] LightOn's open-source ColGrep-Agent-2B offloads Claude Code, cutting API tokens 34% — IgorCarron · 2026-10-12

3 near-duplicate retellings: IgorCarron · IgorCarron · IgorCarron