Production AI pipelines have ~7 stages, and only one is the model
Aiden_Tech_Ai · x · 2026-09-20
A viral explainer argues that "using AI" in production is closer to 7 stages — clean the question, vectorize, retrieve context, rerank, assemble the prompt, run the model, check the output — with 6 of them having nothing to do with the model.
The key failure example: retrieval pulls the US refund policy instead of the EU one, and every downstream stage still "works perfectly" — the reranker sorts the wrong chunks, the prompt assembles them correctly, and the model confidently generates a polished but wrong answer. Nothing breaks; the problem was the input.
That's why the final output guardrail matters: it's the only stage designed to question everything before it. The post also notes the model can be a small fraction (9%) of your latency, with the rest in surrounding stages.
More from coding & agent
- px0 editor ships git status streaming via SSE, checking just 3 files instead of polling — arpit_bhayani · 2026-09-20
- DIY Jev-style classifier: shuffling options lifts accuracy from 47% to 73% — WelcomeMysterious122 · 2026-09-20
- Andrew Ng releases free 1-hour course on building agentic knowledge graphs from scratch — irinarish · 2026-09-20
- Why Jev might finally kill the text prompt: millisecond decisions for generative GUIs — dpopa · 2026-09-20
- NoSpoon agent churns out microdrama content fully autonomously in about 15 minutes, zero prompts — Kyrannio · 2026-09-20
- Laya-MLX: open typed decision model runs locally at 7–14 ms per decision, zero tokens — JiliJeanlouis · 2026-09-20