Paradigm Training Stack Borrows Heavily from nanogpt speedrun
Paradigm reveals that its training stack, including the Limite 1B model, heavily adapts architectural innovations from the nanogpt speedrun challenge such as NorMuon on matrix-shaped parameters, aiming to balance sample efficiency with training speed.
2026-10-07 ~ 2026-10-07 · 3 related posts
- Limite 1B borrows nanogpt speedrun architecture: NorMuon, MUDD variant and XSA — tensorqt · 2026-10-07
- Paradigm adapts nanogpt speedrun tricks with NorMuon, MUDD variant and XSA to balance sample efficiency and training speed — tensorqt · 2026-10-07
- Paradigm borrows nanogpt speedrun tricks: NorMuon, MUDD variant and XSA in its training stack — PMinervini · 2026-10-07