Reading Raschka's LLM-from-scratch book, this engineer sees Transformers as differentiable hash tables
moultano · x · 2026-10-09
moultano shares his experience reading Sebastian Raschka's Build a Large Language Model (From Scratch), finding it far more useful to actually build an LLM than to only read about deep learning—while noting the book has plenty of typos, which he finds ironic. His mental model: a Transformer is essentially a differentiable hash table/dict.
Related event: Self-Attention as a Differentiable Hash Table(2 posts)→
More from Research
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23
- MIT CSAIL's X-Hinges 3D-Prints Everyday Objects With Built-In Sensing Across Three Motion Axes — MIT News AI · 2026-10-23