Split Monolithic Agents Into a Generation Layer Plus Small Classifiers
aigclink · x · 2026-09-19
A detailed architectural argument: agent design is shifting from one strong model + prompts + tool loop toward a layered paradigm — a generation layer plus many small, purpose-built decision components, echoing how PHP monoliths split into gateway/auth/rate-limiting services.
- Monolithic agents shine in demos but hit three walls in production: slow, expensive, and unquantifiable (can't state hallucination or false-positive rates).
- Judgment and generation are different tasks: judgment needs fixed label sets, calibratable probabilities, tunable thresholds, plottable ROC curves; a generative model can only say "I think this is risky" without numbers.
Three-step migration: (1) mark every judgment point (tool authorization, model routing, content gating, result verification); (2) define label sets, thresholds, and the cost of each error type; (3) output PASS/FAIL/uncertain and route uncertainty to humans.
Caveats: small dedicated classifier models aren't new — classic NLP did this for years; the hard part is labeling and threshold tuning (3% error is invisible in demos but means dozens of daily complaints in production). The layer adds complexity, so it's only worth it when agents take consequential actions, and must be weighed against cost.
More from coding & agent
- Dev nukes all agent skills, notices no difference — skills 'gone by GPT 6.5' — BLUECOW009 · 2026-09-19
- Use Jev for instant classification, switch to BERT at scale: a practical model routing comparison — tekbog · 2026-09-19
- Harness choice alone drives 10x cost delta: same kimi k3 on identical tbench2.1 tasks — zainhas · 2026-09-19
- EPA ECHO MCP wraps EPA's free, no-auth environmental data APIs for agents — modelcontextprotocol · 2026-09-19
- usecortex-mcp launches: encrypted persistent memory for AI coding agents — modelcontextprotocol · 2026-09-19
- Cost breakdown of all deepswe runs shows the price of Sonnet 5 Max tokens — zainhas · 2026-09-19