Agent audit said 0 blocks while hooks logged 7: Claude Code transcript drift broke a blind detector
Ok-Motor-9812 · reddit · 2026-11-02
The author runs a daily audit over Claude Code transcripts counting hook blocks. On Sep 12, their own telemetry recorded 7 blocks while the audit reported 0 (3 after a first fix).
Root cause: the transcript format had shifted — the same block appeared in three shapes so far (an attachment in older builds, a tool result with an error prefix, a tool result with only a bare reason). Each change blinded the pattern matching, and tests stayed green because fixtures kept the old shape. Worse, a blind detector shows fewer blocks, which looks like progress.
The fix:
- Every hook now logs the tooluseid of the call it stopped; the audit joins on that ID instead of guessing text shapes
- Two alarms: if over half of logged IDs appear in no transcript, the layout changed; if text rules recognize under 80% of logged blocks, the alert quotes the missed text so the new pattern is one line away
Lesson: pattern-matching detectors over upstream log formats need ID-based joins plus drift alarms, or monitoring silently fails while metrics look better.
js
// hook side
log({ session: p.sessionid, tool: p.toolname, id: p.tooluseid })
// audit side
const blocked = hookLog.has(r.tooluseid) || looksLikeBlock(r.text)
More from coding & agent
- InfoQ: A Systemic Approach to Memory, Knowledge, and Context in Agentic AI Architectures — AdiPolak · 2026-11-02
- Autonomous's Harness Pro uses an ESP32-P4, not a mini PC, to monitor AI coding agents — dee_hw · 2026-11-02
- RL-trained LFM2.5-24B adapter matches accuracy with 20% fewer tool calls — yb2698 · 2026-11-02
- Debugging production AI agents: error surfaces at step 5, root cause at step 3 — Ok-Machine7983 · 2026-11-02
- RAGFlow 1.0 preview: from RAG engine to agent-native database — newpeak · 2026-11-02
- Kitsune: open-source MCP server that shares your Firefox with a coding agent — Sad-Interview9085 · 2026-11-02