Compiler-backed coding agent Benzi hits 82.6% on SWE-bench Verified for $9.26
DonkeyTheKing · reddit · 2026-11-01
Benzi is a compiler-backed coding agent that first compiles a repo into a queryable map of calls, data flow, control flow, and class hierarchy — each language via its own tree-sitter grammar — then reads, edits, and verifies code against it, with a runtime tracer for what static analysis can't resolve.
- Scores 82.6% pass@1 on SWE-bench Verified using DeepSeek v4.1 flash, for $9.26 total, with 97.6% of tokens served from cache.
- Read 326,928 lines of code vs 645,752 for mini-SWE-agent; separate benchmarks claim 2–6x less code read than other agents at 2x lower cost.
- Supports incremental reindexing and edit validation via the same query map. Reports and both agents' runs are published on GitHub, with a demo on varianttech.net.
More from coding & agent
- ASC CLI adds territory-scoped PPP pricing, deep IAP validation, serial matching — rudrank · 2026-11-01
- One-Command App Store Sale Scheduling: asc CLI Example With Auto-Revert Pricing — rudrank · 2026-11-01
- asc versions release Now Reconciles After Server Errors, So Shipped Releases No Longer Turn CI Red — rudrank · 2026-11-01
- asc CLI now reconciles release status after server errors to avoid false CI failures — rudrank · 2026-11-01
- asc CLI adds terminal-based streamlined purchasing toggle and StoreKit ownership filtering — rudrank · 2026-11-01
- asc CLI Gets Resilience Fixes: Serial-Number Tolerance and Secret-Mangled Key Recovery — rudrank · 2026-11-01