Agents Drift in Prod: Dev Builds Provider Twins and Mid-Flight Invariants to Catch It

Common_Dream9420 · reddit · 2026-11-17

The author describes a recurring problem: agents behave fine in dev but drift in production as provider responses shift subtly. Two mitigations:

Provider twins: shadow versions of LLM and API calls that replay recorded real responses while letting you inject edge cases before anything hits production.

Inflight invariants — the more effective technique: instead of post-run output checks, define custom assertions that fire mid-execution, e.g. "if this tool call returns shape X, the next hop must satisfy Y." This catches coordination failures much earlier than post-run evals, especially when two agents share state and one changes something mid-flight.

The author is still working on making invariants portable across workflows without over-hardcoding context, and asks the community: should invariants live at the agent level, task level, or orchestration layer, and how to handle async coordination without clean sequencing?

Original post →

More from coding & agent

coding & agent channel →