Sebastian Raschka's 'Reasoning from scratch' round 6: hands-on RLVR and GRPO implementation
rasbt · x · 2026-10-03
Sebastian Raschka releases round 6 of his "Reasoning from scratch" series: a full introduction and from-scratch implementation of RLVR (Reinforcement Learning with Verifiable Rewards) and GRPO (Group Relative Policy Optimization).
The video covers theory and practice:
- Theory: what makes reasoning models different, reasoning traces and capability, accuracy/format rewards, DeepSeek-R1's "aha moments", RLHF vs RLVR, GRPO vs PPO (with a cooking analogy), the KL term and simplified GRPO.
- Practice: loading a pretrained model and MATH training data, sampling responses, computing verifiable rewards and advantages, implementing token/sequence log probabilities, GRPO loss and the full training loop, training settings and checkpoints, then evaluating on MATH-500, plus training stability and memory requirements.
A full timestamped outline is provided, making it directly followable as a tutorial.
Related event: rasbt Releases Hands-On Tutorial Implementing RLVR and GRPO from Scratch(2 posts)→
More from coding & agent
- FT: US tech outages blamed on rush to fix bugs found by AI — nordicinst · 2026-11-19
- Claude Code overtakes Cursor and Copilot in job-post mentions ahead of Anthropic IPO — antgoldbloom · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS adds private CA support to Bedrock AgentCore Identity for connecting agents to self-hosted IdPs — AWS ML Blog · 2026-11-19
- Orchestrator's Cloud Agents graduate from beta to general availability — julianweisser · 2026-11-19
- Zero users, 33 pages shipped: dev crowdsources audit for AI-agent trust schema — TheHamzaShahbaz · 2026-11-19