Parsimony: an open-source benchmark measuring how much excess code AI models add
lasu · hn · 2026-10-14
A Show HN release of Parsimony, an open-source benchmark that measures how much code AI coding models add.
- Motivation: existing benchmarks track solve rate and token cost but don't penalize excess code, which hurts long-term codebase health.
- Method: reuses published patches from DeepSWE, SWE-bench Verified and SWE-bench Live, counting "coding units" (statements, calls, names, literals) rather than lines of code; deleted code is subtracted for a net measure.
- Extras: GPT-6 and Claude 5.5 lack public patches, so the author ran 3–10 tasks each on personal subscriptions (shown unranked).
Open source on GitHub, with feedback and public patch exports welcomed.
More from coding & agent
- Index Network open-sources Social Discovery Protocol: agents negotiate fit on your behalf — carsonfarmer · 2026-10-23
- Installed a formal-verification skill: it silently installed Java, MacBook runs 10 degrees hotter — zealcaiden · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- Ruff author warns: upgrade Node/Bun now — --use-system-ca still accepts the eMudhra cert — charliermarsh · 2026-10-23
- Proof of Agent: paste one prompt and your AI mines, grades, and self-governs — adamamcbride · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23