LLMs beat decision models on knowledge, but not on routing or safety
randal_olson · x · 2026-10-16
Splitting the new Artificial Analysis Decision Index by task shows a clear picture: LLMs beat decision models by 30 points on knowledge questions and 19 points on hard reasoning (HLE), but the two are about even on safety classification and tool-call routing — and on those tasks, every decision model answers faster than every LLM tested.
Olson's recommendation: use decision models for routing, triage, and moderation, and reserve LLMs for knowledge- and reasoning-heavy requests; the best setup for most products mixes both.
Related event: Decision models match LLMs on routing and safety tasks at 7-8x speed(3 posts)→
More from coding & agent
- Every publishes 8,000-word Codex oral history, agent-native so your AI can dig the raw interviews — danshipper · 2026-10-16
- Developer rebuilds GitHub in 4 days: Keel makes agents declare intent before changing code — threepointone · 2026-10-16
- Stop using small models as judges with overloaded context, dev warns — xeophon · 2026-10-16
- Meta's open-weight Muse Spark Open hits Ollama with 1M-token context and Claude Code support — ollama · 2026-10-16
- Monid launches: one balance, pay-per-call access to 1,700+ tools for AI agents — HeyAmit_ · 2026-10-16
- OpenAI publishes Sites tutorial: build and share your first Site — OpenAIDevs · 2026-10-16