Evals Are Replacing Automated Tests as the Core of AI Software Development
somangshu · reddit · 2026-10-19
The author argues that the next step-change in AI-era software quality will come from eval design and harness efficiency rather than traditional automated testing.
- Continuously run evals on what you build to create a recursion loop of improvement; the more automated, the faster you move.
- Cites the Claude Code team's practice of running an eval on every real-world run that might have gone wrong, then suggesting system-level fixes to speed up review.
- Draws an analogy: evals are replacing automated tests as the guarantee of shipping quality products.
- Notes heavy investment across big players like Google's harness2, and says this is where the author will spend most of their time this quarter.
More from coding & agent
- Distributed Systems Resilience Explained: Circuit Breakers, Bulkheads and Graceful Degradation — _jaydeepkarale · 2026-10-19
- Verified local LLM engine benchmarks: Qwen3.8-27B up to 540 tok/s on a single RTX 5090 — FeydRowan · 2026-10-19
- Free DIY Continuous Security guide: build SAST, DAST, SCA and secrets scanning with GitHub and free tools — evilsocket · 2026-10-19
- New book frames context engineering as an old discipline applied to partly-prompt AI systems — alex_mikhalev · 2026-10-19
- No paid Notion templates needed: a full second-brain workflow with Claude, Granola and Gmail — rubenhassid · 2026-10-19
- Marc Lou ships MailCheap update entirely via vibe coding, undercutting beehiiv by 83% — marclou · 2026-10-19