Small Expert Models Will Outperform 'Mainframe AI'
markjeffrey · x · 2026-07-19
The author agrees with Andrej Karpathy that small expert models are the future, rather than 'mainframe AI'.
Karpathy points out that current LLMs waste a massive amount of parameter capacity on useless junk data. True agents aren't magic; they are large-scale distillation. By combining small models + the right tools + closed loops, astonishing capabilities can be achieved.
More from Models
- Nanbeige4.3-33B benchmarked: 59 tok/s single stream, ~1,060 tok/s across 64 concurrent runs — MaziyarPanahi · 2026-10-26
- Only 2 of 25 full ChatGPT rewrites flagged 100% AI by detectors — paulnovosad · 2026-10-26
- Qwen 4 Reportedly Spotted in the Arena, Hints at Alibaba's Next-Gen Model Testing — AIFlow_ML · 2026-10-26
- Nathan Lambert's first open models lecture: distillation, risks and the ecosystem explained — natolambert · 2026-10-26
- Nanbeige 4.3-33B flops real-world test: zero compilable output vs Qwen — julianharris · 2026-10-26
- Claude's safety classifier blocks SS7 defense research, pushing a security dev to an abliterated GLM — l4rz · 2026-10-26