Open-Source Benchmark Parsimony Penalizes Overengineering in Coding LLMs
lasushin · reddit · 2026-10-11
Developer La5u released Parsimony, an open-source benchmark addressing a blind spot: existing coding benchmarks measure solve rate and token cost but never punish excess code that degrades long-term maintainability.
- Reuses published patches from DeepSWE, SWE-bench Verified and SWE-bench Live
- Counts "coding units" (statements, calls, names, literals) on a net basis, crediting deletions; deliberately avoids lines-of-code so cramming isn't rewarded
- GPT-6 and Claude 5.5 lack public patches, so the author ran a few tasks on personal subscriptions (unverified, shown as faded circles)
Site: parsimony.lasu.dev; GitHub open to feedback and contributions, especially public patch exports for newer models.
More from coding & agent
- Monid ranks agent tools by real usage: TinyFish leads with 754K weekly transactions — HeyAmit_ · 2026-10-23
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- Capy 0.5.0 lets Claude Pro/Max subscribers reuse their plan quota, adds OIDC cloud access — garrytan · 2026-10-23