We blamed the model for changing answers, but the parsed input had silently changed
Raul_Nogales · reddit · 2026-11-11
The author's pipeline produced a complete table in one run and a truncated one in the next, with a field missing even though the source file was unchanged and the prompt identical. The team blamed model nondeterminism and wasted time on settings that weren't the problem.
The real cause: re-parsing the same file produced a dump in a different shape, and the model faithfully answered the input it actually received. Comparing the two dumps side by side exposed it in minutes.
New habit: when the same input yields different answers, inspect the parsed dump before touching the model. Many cases of "model instability" are actually upstream data pipeline instability.
More from coding & agent
- ByteAsk Paper: Build-Aware C++ Agent Cuts Tokens 63-80%, Matches Codex CLI Fixes — rohanpaul_ai · 2026-11-11
- Untangle's aiDotEngineer talk on building with agents is now live — altryne · 2026-11-11
- Potion UI: a native standalone engine quietly built for 2+ years — cocktailpeanut · 2026-11-11
- Anthropic's Rufus Teases InDesign MCP Support: "Definitely in the Mix" — rufusd · 2026-11-11
- Netlify launches Server: a long-running backend for AI agents, now in Early Access — jasonkneen · 2026-11-11
- Indie assistant Pepper can call you, answer your phone, and take messages — thechrisperry · 2026-11-11