Compiler-backed agent Benzi cuts code reading by 43% at 82.4% on SWE-bench Verified
DonkeyTheKing · reddit · 2026-10-19
A developer unveiled Benzi, a compiler-backed code intelligence system. On the same model (DeepSeek V4.1 Flash), its agentic loop + compiler scores 82.4% on SWE-bench Verified (baseline mini-swe-agent: 82.0%) while reading 43% fewer source lines (371,301 vs 645,752 across all 500 issues), costing $12.96 total.
Key idea: models generate text, and each iteration improves comprehension of that text — but large codebases force models to spend cognition resolving structure. Benzi surrenders deterministic knowledge to the compiler, freeing model cognition for higher-level decisions, like a senior SWE.
- RESOLVED/CANDIDATE/OBSERVED taxonomy: parse the repo once, let the agent query it, update incrementally on edits, auto-revert broken ones
- A runtime tracer marks OBSERVED edges
- Supports 10+3 frontend languages, model-agnostic
- Any slice can be exposed via MCP (code map), CLI, VS Code extension, SDK or API; compiler runs locally
Note: Benzi is not open source; the GitHub repo only has SWE-bench results and code written by Benzi.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23