Nanogpt embedding table win mirrors earlier ngrammer paper, researcher notes
_arohan_ · x · 2026-09-29
Arohan highlights the impressive embedding table results in the Nanogpt speedrun, pointing out that the ngrammer paper reported a similar result using AdaGrad. He calls it essentially a "free win," while noting that memory is now expensive too.
Related event: NanoGPT Embedding Table Gains Echo ngrammer Paper Findings(2 posts)→
More from Research
- HalluWorld Benchmark, Accepted at NeurIPS, Shows Models Nail Perception but Fail Simulation — xennygrimmato_ · 2026-09-29
- Quintin Pope: backdoor-style training setups are a poor stand-in for hypothesized inner optimizers — QuintinPope5 · 2026-09-29
- Ocular microtremors at ~100Hz may give the visual cortex apparent super-resolution — docmilanfar · 2026-09-29
- AI-designed viruses from scratch: 302 phage genomes synthesized, 16 worked — CallRevolutionary894 · 2026-09-29
- Reddit thread: softmax has only N-1 degrees of freedom — drop one input? — Kinexity · 2026-09-29
- EAPO: entropy-guided credit assignment for RLVR improves exploration in LLM reasoning — coallaoh · 2026-09-29