Karpathy: core intelligence may fit in ~1B params as TwIL-8B nears frontier scores
solyarisoftware · x · 2026-11-06
Andrej Karpathy argues 90% of current models waste capacity and money memorizing irrelevant content, laying out 5 theses on small LLMs: intelligence isn't knowledge; the cognitive core may fit in 1B params; scale is no longer the bottleneck (tools and feedback loops matter); intelligence lives in the system not the weights; and owning a model on your own hardware means no rate limits or deprecations.
In the same thread, Davidstout open-sourced TwIL-8B (8.8B params), claiming 93% of frontier coding performance, 92% on instruction following, and 90% on contest math — arguing the next era of AI will be defined by the cost of completing work.
More from Models
- API dies mid-task: only Atomic Agent with local fallback aced the test, Codex just hung — Ok-Brush8377 · 2026-11-06
- Grok top-tier users claim usage is draining abnormally, refund requests denied — Al_Grigor · 2026-11-06
- Scale AI ships 3,300+ RL environments; some tasks take humans 3 weeks and 500+ tool calls — alexandr_wang · 2026-11-06
- Ant's Ling-3.2-flash: 560B-param native multimodal model beats Opus 5 on release evals — alifcoder · 2026-11-06
- Mercury 3 hits 2 trillion tokens in under two weeks as daily usage climbs linearly — StanfordAILab · 2026-11-06
- Same local model gives different answers via LM Link on iPhone vs direct on Mac — simplext · 2026-11-06