microqwen: hand-coded minimal Transformer ports Qwen 3 modernities into Karpathy's microgpt
anshulkundaje · x · 2026-09-26
Inspired by Karpathy's microgpt, developer daveyburke hand-wrote microqwen: a minimal Transformer implementation updated with modern architectural tweaks from Qwen 3, while preserving Karpathy's original clarity — the code still fits in three columns. It doubles as a teaching artifact showing that the 2017 Transformer architecture has endured with only a few key changes, and what those changes actually are.
More from Research
- Xiaomi open-sources MiMo RL environments repo, individual tasks sell for $100-1000 each — HarveenChadha · 2026-09-26
- Contextual embeddings go mainstream: only Perplexity and Voyage ship them so far — antoine_chaffin · 2026-09-26
- A Curated Paper List for Learning Distributed LLM Training and Inference — East-Muffin-6472 · 2026-09-26
- RL isn't just sharpening existing skills — pre-LLM RL trained from random init — cephaloform · 2026-09-26
- Steelmanning OPSD: train only on tokens after verbatim rule reminders — willcb · 2026-09-26
- Researchers warn Astra reviews obsess over details, miss the big picture — aran_nayebi · 2026-09-26