DriftGuard: Open-source tool stops billing when your AI agent goes off the rails
No-Program-5087 · reddit · 2026-08-21
When an agent fails, it often doesn't throw errors—it just drifts off-task and keeps burning tokens. The author open-sourced DriftGuard to detect this silent failure.
Core Mechanisms
- Relevance: Is the output still about the assigned task?
- Self-Drift: Has the output distribution shifted away from the agent's own working baseline?
Both signals are measured against the agent's history, requiring no human-in-the-loop "correctness" standard.
Performance & Logic
- Drift is defined as a sustained rise in the deviation rate, not a single bad step. It triggers only after a breach holds across 25 consecutive windows to eliminate false alarms.
- Measured Results: A 400-step agent derailing at step 200 was caught at step 228 (28-call latency). Healthy agents showed zero false alarms over 600 steps.
Usage Notes
- Default Relevance uses bag-of-words (zero cost, no deps); swap in embeddings for semantic drift detection.
- The tool signals when to stop but does not fix the agent.
- Requires Python 3.10+, runs offline.
More from coding & agent
- 129 Practical AI Projects Open-Sourced on GitHub Covering Agents and Voice — Arindam_1729 · 2026-08-21
- Skills-4-SE: a curated collection of 180+ Claude Skills for software engineering — tom_doerr · 2026-08-21
- Deep dive into the hidden design pattern behind Keras models — Mahmoud_Zalt · 2026-08-21
- Fix Opencode early compaction by defining full context parameters — GoodTip7897 · 2026-08-21
- Seed Agent: Minimal Self-Modifying Agent in 186 Lines of Python — vivekhaldar · 2026-08-21
- Jarvis local AI coworker setup runs multi-model on consumer hardware — Darwing · 2026-08-21