Process scores drive a repair loop: failed trajectories gain 0.155 on rerun
omarsar0 · x · 2026-09-04
omarsar0 explains TRACES' dual-verifier design and repair loop. Every run gets two scores: an outcome verifier checks results against hidden ground truth without seeing the process; a process verifier reviews recorded actions, errors, and corrections without seeing the outcome score or which solver produced the run—like unit tests plus code review where the reviewer never learns if the tests passed. When the process verifier flags a run as deficient, the diagnosis becomes a repair note and the solver retries without seeing the answer. Across 434 flagged trajectories, repaired reruns scored 0.155 higher on average per Apodex's evaluation.
More from coding & agent
- Beat AI's samey frontends: import a design system before generating UI — dotey · 2026-10-21
- Trust slider over Clairic: 102 manual reviews catch 74 of 76 extraction errors — michellechen · 2026-10-21
- FreePi Bets on Ad-Supported Coding Help to Undercut $20/Month AI Coding Subscriptions — heyneighbor · 2026-10-21
- RedMonk: MCP and A2A Verify Publishers, Not Safety — Registries Become the Trust Layer — rseroter · 2026-10-21
- Agentic refactor: 600 PRs moved OpenClaw's web UI from Lit to Solid 2.0 in 2 weeks — steipete · 2026-10-21
- 19 Recorded Runs of 18 Agents Building a Lisp Interpreter, Released on Hugging Face — strickvl · 2026-10-21