Ex-Google engineer: evals are the spec that separates agentic systems from regular software
andreisavu · x · 2026-10-23
cosminnegruseri answers three questions—agentic vs regular software, ML lead vs ML engineer, driving ML teams—with one answer: invest in evals. Regular software has a testable spec; ML and agentic systems don't, so the eval set becomes the spec. His process: quickly define good outputs, collect real examples from PMs/users, turn them into a golden set, write labeling guidelines with multi-annotator overlap, and decide metrics for when you ship.
More from coding & agent
- Claude and Claude Code Merge: Anthropic Engineers Say Delete Your CLAUDE.md Every 6 Months — rubenhassid · 2026-10-23
- Engineers merging 4x more PRs at Imprint: Will Larson on speeding up code review — AxSaucedo · 2026-10-23
- 15 practical ways to hand off work to Grok Bot's cloud computer — even with laptop closed — XFreeze · 2026-10-23
- 3 Lessons from Rebuilding Compute for AI Agents: Safe Sandboxes, Near-Free Idle, Persistent Memory — Saboo_Shubham_ · 2026-10-23
- Meta's RSD paper: agents self-improve from unlabeled runs, lifting Qwen3-14B success from 12.9% to 20.6% — rohanpaul_ai · 2026-10-23
- Teknium's Hermes Agent builds a live field guide to 425 unusual web libraries in 20 minutes — alexcovo_eth · 2026-10-23