Locally-run Qwen hallucinates a realistic provider API error out of thin air
little_raven_black · reddit · 2026-11-08
A Reddit user running a fully local agent stack (OpenAI-compatible endpoint serving an NVFP4-quantized Qwen checkpoint, 262K context, no external API anywhere) watched the model output a realistic provider-gateway error JSON (sys000, "it's on us, please try again later") mid-session.
Logs showed the HTTP call actually succeeded (finishreason=stop, 71 tokens); the error body was stored as a normal assistant message, and the errorid existed nowhere on the machine — invented at decode time, absorbed from real API transcripts in training data. The trigger: at 194K context, after the previous reply was truncated and a "continue" prompt injected, the model couldn't reconstruct tool-call state and emitted the canonical form of "something went wrong in this API conversation" instead.
Takeaways: an error-shaped assistant message with no matching non-200 in logs and a novel errorid is model fabrication, not infrastructure fault; and "my agent role-played an API error" is a clean tell that decoding has gone off the rails.
More from coding & agent
- Dev Built Cinderscoop, a 3D Firefighting Plane Browser Game, Entirely with Claude Code — vinishkapoor · 2026-11-09
- Local LLM manager fix: concurrent requests from one session yield 2-3X speedup — draginol · 2026-11-09
- NoMac: An MCP Server That Gives Agents a Cloud Mac and iOS Simulators — Stunning-Farmer185 · 2026-11-09
- Dev: with latest-gen LLMs he stops overspecifying and lets the model riff — jongranskog · 2026-11-09
- Proof of Agent grades agents with other agents — repetitive bot farms earn nothing — adamamcbride · 2026-11-09
- AI Coding Doesn't Remove Engineering — It Moves It Up a Layer — Responsible-Beat2137 · 2026-11-09