Building a Transformer from scratch in NumPy: 68K params that learned to sort
dima806_dima · x · 2026-10-13
Dima lakubovskiy published a 10-minute read: a Transformer built entirely from scratch in NumPy — no GPU, no DL libraries, about 68,000 parameters you can read line by line. The tiny model learned to sort by 'looking in the right place,' offering a minimal, fully readable implementation for understanding how attention actually works.
More from Research
- Perplexity's Decider API classifies 203,883 Phoenix aerial tiles at 98.8% accuracy for $9.63 — inductionheads · 2026-10-13
- ViDiHand (NeurIPS Spotlight): pretrained video diffusion models excel at 3D hand motion capture — XingangP · 2026-10-13
- Giving LLMs tools makes them worse at refusing harmful requests, up to 68.7% — lateinteraction · 2026-10-13
- Muse Wonder scores 93.4 on Extended NYT Connections benchmark, among priciest tested — zero0_one1 · 2026-10-13
- Polymarket puts 67% odds on AI-developed skin cancer vaccine winning FDA approval — Polymarket · 2026-10-13
- Researchers clash over 'one world model, many domains' claim in new paper — DrMorganLevine · 2026-10-13