Small Expert Models Will Outperform 'Mainframe AI'
markjeffrey · x · 2026-07-19
The author agrees with Andrej Karpathy that small expert models are the future, rather than 'mainframe AI'.
Karpathy points out that current LLMs waste a massive amount of parameter capacity on useless junk data. True agents aren't magic; they are large-scale distillation. By combining small models + the right tools + closed loops, astonishing capabilities can be achieved.
More from Models
- GPT-5.6 writes well but is instantly forgettable, user complains — BasedRaddka · 2026-09-11
- Opus Refuses Protein Research Codebase Over 'Safety' Concerns, Dev Considers Rolling His Own — josephdviviano · 2026-09-11
- User Hails Unconfirmed 'DeepSeek 4.1 Flash' as an Inflection Point in LLMs — himanshustwts · 2026-09-11
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- awesome-llm-leaderboards: an open-source directory of LLM leaderboards, pricing tables, comparison tools — Last_Establishment_1 · 2026-09-11
- Anthropic claims it works to keep eval environments unidentifiable to models — MaxKannen · 2026-09-11