Bot audit experiment: triage plus deep-read cuts input 64% while staying stable
bfrench · x · 2026-11-15
The author tested Michael Fenech's T.R.U.S.T. audit prompt on their own Grok Bot and found a cheaper, steadier way to run it: a fast yes/no triage scores every action, and only flagged ones get the full prompt deep-read.
Setup: 20 real examples + 8 planted faults, 3 reruns per arm.
Results across four approaches:
- Full prompt every task: catches 8/8 faults but costs 83k input tokens / $0.090 per pass, and gap verdicts drift wildly on reruns (4/0/1 gaps found)
- Decision-1 script alone: 7/8, steady but misses one
- Triage + deep-read (author's approach): 8/8 on every rerun, input down to 30k (-64%), cost $0.068 (-24%)
Random clean controls prevent blind spots. Conclusion: spending quota only on flagged work both cuts cost and eliminates the verdict drift of prompt-only auditing.
More from coding & agent
- Connecting Claude to Google Search Console: a weekly automated SEO-fix loop — martyamark · 2026-11-15
- The T.R.U.S.T. check: deciding which jobs you can safely hand to your AI bots — bfrench · 2026-11-15
- Read-only MCP server lets Claude Code write regression tests for agents — tom_mathews · 2026-11-15
- How do you track all the AI agents running inside your company? — CupRare814 · 2026-11-15
- Developer builds VR island adventure game with Claude Code writing the C# — Illustrious-Lynx3673 · 2026-11-15
- Attest: record agent runs once, replay in CI with zero model calls — tom_mathews · 2026-11-15