AgentSnap: Open-Source Snapshot Testing to Catch LLM Agent Regressions
iamfaham5 · reddit · 2026-07-30
A developer has released AgentSnap (MIT license) on GitHub, a tool designed to solve the 'silent regression' problem in LLM agent development, where minor prompt edits or model upgrades cause unexpected behavioral changes.
Core mechanisms and highlights include:
- Snapshot Diffing: Records LLM and tool calls as golden snapshots, then diffs subsequent runs across four dimensions: tool-call sequence, arguments, requested tools, and semantic content.
- Dual Modes: Replay mode feeds recorded responses during PR tests for deterministic, free testing that fails if prompts or call sequences change. Live mode makes real API calls during nightly jobs to catch model drift.
- Zero-Instrumentation: Monkey-patches SDK classes directly, supporting frameworks like LangChain and Pydantic AI without requiring code wrappers.
More from coding & agent
- OpenClaw Exposes Critical RCE Flaw, 50k+ Nodes Compromised — ericelliott_ · 2026-07-30
- Dev tests two adversarial agents, they just keep catching each other cheating — Vjeux · 2026-07-30
- CrowdStrike Report Reveals Hidden Vulnerabilities in AI-Generated Code — RexDouglass · 2026-07-30
- Developer Suggests Queuing Over Hard Limits for AI Agent Compute Constraints — majidmanzarpour · 2026-07-30
- Compiling Zod schemas from English specs: A new paradigm in AI coding — holdenmatt · 2026-07-30
- Agent-reach: Let AI Agents Browse the Web for Free Without Paid APIs — dr_cintas · 2026-07-30