A long history of LLM architectures leads from GPT-2 to Kimi K3
baseten · x · 2026-07-28
A long-form writeup traces the history of LLM architectures from 2017 to Kimi K3.
- The post frames Kimi K3 as the endpoint of a long scaling and architecture evolution rather than a standalone release.
- In the quoted article, the author says 22,580 GPT-2-sized models would fit inside one Kimi K3-sized system, using that contrast to emphasize how far model scale has moved since 2019.
- The thread is essentially a technical reading recommendation for people who want a broader architectural timeline, not just a product announcement.
Related event: Kimi K3 shifts attention from scale to architecture(25 posts)→
More from Research
- Digital-first GLP-1 program delivers 16.1 kg average weight loss across 8,000+ women — EricTopol · 2026-10-16
- Injecting hidden-state memory cartridges into frozen Mistral-7B and Qwen2.5-7B, weights untouched — Nearby_Indication474 · 2026-10-16
- Golden Gate Claude explainer resurfaces: interpretability premise still holds — HarperSCarroll · 2026-10-16
- DeepMind: four Gemini agents beat Minecraft end-to-end in 27h14m with no human help — omarsar0 · 2026-10-16
- Is AI Interpretability Close to a Breakthrough? A New Essay Has Researchers Divided — The Decoder · 2026-10-16
- MagicBeanStalk paper: recursive synthesis of hard problems boosts gpt-oss-120b by 8.6 points on 8 math benchmarks — hyunw_kim · 2026-10-16