Conformal risk gate for agent writes: confidence is saturated, asking always wins

Gjuesev · reddit · 2026-10-15

The author built gate-calibrated, a barrier before an LLM agent's write actions on tau-bench retail: it combines verbalized confidence and agreement of k resampled next actions, calibrated via conformal risk control (implementing ToolChain-CRC and CORA ideas), to decide when to ask the user. 3,340 simulated conversations, agent and simulated user both glm-5.3-flash.

Results

Risk control held: at alpha=0.02, 1.7% of 460 test conversations executed a wrong write without asking; 3.7% at alpha=0.05 — at or under target.
But the gate never beat asking every time: at alpha≥0.10 the learned threshold is so low it never asks; at alpha=0.02 per-approval wrong-write avoidance was 0.058 vs 0.062 for confirming everything (CI [-0.04, 0.03]), and no alpha reaches zero errors, while full confirmation does (1.57 approvals/conversation).

The diagnosis is the gold: verbalized confidence was ≥0.9 on 548/549 write steps including all 43 wrong ones; self-agreement ≥0.9 on 758/780. Both signals are saturated — the calibrator mostly learned that later steps in a conversation are riskier (weight -0.80 on log(1+step index)).

Code, data and negative results are open-sourced. Open question: what should you rank write steps by instead?

Original post →

Conformal risk gate for agent writes: confidence is saturated, asking always wins

More from coding & agent