All technical work is becoming evals: engineers' job is building agent verification loops
rachittshah · x · 2026-10-17
In a widely shared thread, rohansanjay argues that all technical work is becoming evals: an engineer's primary job is now building loops that make coding agents effective maintainers of your systems. Models are immensely capable — given clear instructions, the right environment, and a way to verify their work, agents can complete open-ended, long-running tasks; if you still think coding agents ship "slop", one of those three is missing. Product engineers may need to restructure codebases to give agents fast, high-fidelity ways to test changes; researchers will spend most time on data synthesis and environment construction pipelines, since experiments are the bottleneck, not ideas. The art is translating real-world problems into the right eval — making the agent understand your exact criteria for "sufficiently complete".
More from coding & agent
- Sol plays Asterix from screenshots, holding each move until its next reply arrives — wzenus · 2026-10-17
- GPT-6 Sol plays rhythm games in real time, picking keys and tap times from frames — wzenus · 2026-10-17
- Sol picks speed and lane targets frame-by-frame from onboard camera while racing — wzenus · 2026-10-17
- Sooool: latency-aware agent plays Touhou in real time, repo and guide open-sourced — wzenus · 2026-10-17
- Teknium catalogs 330 hackable devices and 1,697 project ideas for Hermes Agent — Teknium · 2026-10-17
- Garry Tan launches GBrain, a shared team memory synced to ChatGPT, Claude and Cursor — garrytan · 2026-10-17