Hand-deriving VAEs in 11 steps: the math behind GRPO's KL penalty and diffusion losses
ProfTomYeh · x · 2026-09-23
Prof. Tom Yeh shares an 11-step hand-derivation of the Variational Autoencoder, computing every forward pass and gradient by hand on three training examples.
Key steps:
- The encoder maps inputs to latent distributions with means μ and standard deviations σ; the reparameterization trick (sampling ε, scaling by σ, adding to μ) lets gradients flow through sampling
- The decoder reconstructs the input; KL gradients simplify to μ and σ-1/σ, and the reconstruction gradient to Y-X
Takeaway: one VAE diagram teaches the core of two modern methods — the KL penalty is exactly what RLHF methods like GRPO use to keep fine-tuned models from drifting off base, and the reconstruction loss (plain MSE) is what trains diffusion models to denoise. The VAE paper won ICLR's first Test of Time Award in 2024.
More from Research
- Yoav Goldberg: some tasks just need deterministic rules — agents can write them — yoavgo · 2026-09-23
- Yoav Goldberg: shape predictor variables and decisions as a decision tree — yoavgo · 2026-09-23
- Yoav Goldberg: For Recurring Tasks, Tune Bespoke Predictors Instead of Always Using Reasoning LLMs — yoavgo · 2026-09-23
- LLM agents collude in 94% of long-horizon interactions, study across 10 models finds — SALT-NLP · 2026-09-23
- Emerging research consensus: architecture tweaks are efficiency fixes, RL compute drives capability — burny_tech · 2026-09-23
- Xiaomi's MiMo-V2.6 tech report: ~7k RL training data open-sourced, model shipped within a week of final RL run — rbhar90 · 2026-09-23