Small-model workflow: let the model say 'review' instead of forcing an answer
No_Appeal_5223 · reddit · 2026-11-06
The author shares a practical lesson for small models: don't force them to solve everything — give them narrow, verifiable jobs with an explicit escape hatch.
- For support-message classification, 'categorize these messages' pushes the model to pick a label even when ambiguous — often worse than admitting uncertainty.
- Better: define the label policy first (billing / access / bug / review), require id + label + a sub-12-word reason, and mandate 'review' when multiple labels could apply.
- A worked example shows an ambiguous 'can't access my account, subscription cancelled' message correctly routed to review instead of a confident wrong guess.
- The general pattern: Task → Input → Rules → Output format → Uncertainty rule → Verification.
- Evaluate via classification accuracy, review rate, retry rate, latency, total cost, and human correction time — not answer aesthetics. He's testing Claude Haiku 5.5 for such fast repeatable tasks, but the principle applies to any routing setup.
He links his own Haiku 5.5 prompts/workflows guide (disclosed).
More from coding & agent
- ~780 AI slop accounts identified on X; users deploy agents to auto-block them daily — TheZachMueller · 2026-11-06
- Formal proofs of tens of millions of lines of code are now feasible, says Anthropic engineer — morqon · 2026-11-06
- Terminal-Bench's two broken tasks traced to Debian 11 archive repo migration — PawelHuryn · 2026-11-06
- Stable AI proposes DPDM: frozen LLM representations plus shared predictor boost agent decisions — stable-ai · 2026-11-06
- One prompt ran Codex for 17+ hours with 109 sub-agents and 2B tokens — andimarafioti · 2026-11-06
- Reddit devs compare agent memory approaches: summaries, MEMORY.md, wikis and vector DBs all fail differently — ombre-ai · 2026-11-06