Four calibration papers say model confidence shouldn't gate risky agent tool calls

Xjjjjyn · reddit · 2026-11-17

Reviewing four calibration papers before trusting "act when confidence > X" agent safety heuristics:

What they report

The gap: all four measure answer correctness on QA benchmarks — none measures whether a multi-step tool call is the right action on your data. A TriviaQA score says nothing about "are these the right 40 rows to delete."

Design choices that follow (authors' own, not paper findings):

Consistent with OWASP LLM06:2025 (Excessive Agency). Author discloses affiliation with a commercial agent product.

Related event: Calibration literature review questions confidence gating for agent actions(2 posts)→

Original post →

More from coding & agent

coding & agent channel →