'The order is the control': anchoring-proof agent pipeline puts hard rules before the model
Neither_Opinion69 · reddit · 2026-11-03
A Reddit engineer building an agent that proposes green/amber/red monitoring thresholds shares a hard-won lesson: when a pipeline mixes hard rules and an LLM, call order matters more than the prompt.
The trap: asking the model first let it write "the 1.0% red line looks loose, consider tightening to 0.7%" on a contractually mandated limit — text that hit the reviewer's screen before the rule overruled it. Reviewers can't unread it: classic anchoring, now labeled "AI suggests."
The redesign: mandated bands are adopted verbatim without consulting the model; zero-tolerance indicators get green-0/red-1 with no amber; the model is only called for statistical indicators, and only to pick a METHOD from a closed list (mean+SD, percentiles…), with no value field in the response schema; bands are computed deterministically with a trace; validation failures are logged as "blocked," never "proposed." Two extra guards: a regex visibly masks any digits the model volunteers anyway, and the validator independently rejects averaging on zero-tolerance indicators even though ordering makes it unreachable — "a guard that only holds because of if-branch order is one refactor away from failing."
Reproducing both designs with a fake number-volunteering model shows identical bands either way — so unit tests can't tell the designs apart; counting model calls on a fixture set can.
More from coding & agent
- Dev builds visual layout editor for MOSS-Image that outputs JSON prompts — jambavant · 2026-11-03
- Sentry CEO shares coding agent asking approval to rewrite Sentry in Rust — zeeg · 2026-11-03
- Microsoft shaved 4.5% off clarity.js with 50+ AI agent experiments — gaganghotra_ · 2026-11-03
- Vibe-devops agents may rebuild SaaS in-house as vendors like Bending Spoons triple prices — robleclerc · 2026-11-03
- rauchg: stateless sandboxes are the wrong benchmark as Vercel Sandbox gets 2.4x faster — cramforce · 2026-11-03
- Model swap cut costs 59% and latency 75% — and he still blocked the release — Fun_Employment6042 · 2026-11-03