rasbt's Reasoning From Scratch round 2: LLM text generation and KV caching explained
rasbt · x · 2026-09-06
Sebastian Raschka released round 2 of his Reasoning From Scratch video series, building the base-model foundations before adding reasoning techniques in later installments.
- Setup: checking PyTorch and hardware support, Apple silicon/MPS caveats, cloud GPU options
- Tokenization & Qwen3: encoding/decoding, downloading weights, loading pretrained Qwen3, the Reasoning From Scratch package
- Generation mechanics: logits and next-token prediction, greedy decoding with argmax, building a streaming generator, handling end-of-sequence tokens, benchmarking generation speed
- KV caching: how it works, measuring the speedup after adding it, torch.compile model compilation and combining it with KV caching, CPU vs GPU comparison
Full chapter timestamps included, making it a follow-along tutorial.
Related event: rasbt's Hands-on LLM Generation and KV Cache Course Hits YouTube(2 posts)→
More from Research
- DeepMind economists publish 'An Economy of AI Agents' chapter rethinking firm boundaries and governance — soumitrashukla9 · 2026-09-06
- Five-Year Retrospective Argues ML-Assisted Directed Evolution Ignores Real-World Costs — anshulkundaje · 2026-09-06
- Google maps full fruit fly brain with AI, reconstructing 166,000+ neurons in 3D — zacharynado · 2026-09-06
- Skip SFT, run GRPO on a 0.8B Qwen base model and watch reasoning emerge — alexcovo_eth · 2026-09-06
- Mathematician littmath: the community should reward AI 'last-mile' work less — littmath · 2026-09-06
- MirroS' Code-as-World Beats Gemini-3.1 Flash on Physical Reasoning Benchmark — jiqizhixin · 2026-09-06