Sebastian Raschka: Engineers Still Benefit From Understanding How LLMs Work
rseroter · x · 2026-09-22
In an interview with Milan Milanović, Sebastian Raschka — author of "Build a Large Language Model (From Scratch)" and former UW-Madison statistics professor — argues understanding LLM internals still pays off:
- Anyone can call an API, but knowing the mechanics gets you better prompts and a more critical read of model output.
- His learning path: go deep in your own subfield first, then learn LLM, reasoning-model, and agent fundamentals; skip the rest.
- Reasoning models think out loud on a scratchpad, enabled by a specific training method.
- Practical advice: rebuild something you already built with an agent, then let the agent extend it.
- AI writing code frees engineers to think bigger, harden codebases, and compare design choices. His wrong prediction: agents matured faster than expected.
- He also discusses leaving academia and becoming an independent educator after three years at Lightning AI.
More from AGI Musings
- Alex Epstein calls (P)doom 'fake threat analysis' that only manufactures fear — TinfoilTricorn · 2026-09-22
- Ruxandra Teslo: AI anxiety is about losing meaning and agency, not just jobs — anshulkundaje · 2026-09-22
- From Erdős to Millennium Problems: AI math progress is compounding fast — haider1 · 2026-09-22
- More inference than training machines means writing will soon beat reading — GregoryDiamos · 2026-09-22
- Hugging Face incident row: Andrew Ng calls AI fear a PR-driven setback, critics push back — aran_nayebi · 2026-09-22
- Google researcher: AI adoption stalls on certainty expectations — output error bounds instead — docmilanfar · 2026-09-22