Hand-Deriving the VAE in 11 Steps: One Diagram Teaches KL Divergence and Diffusion Loss

ProfTomYeh · x · 2026-09-15

Tom Yeh walks through a full by-hand derivation of the Variational Autoencoder in 11 steps:

• Core idea: A VAE learns the structure of data — mean and variance of hidden features — and generates by sampling from that structure; a GAN only learns to fool a discriminator and can make convincing fakes without understanding the data.
• Derivation: Three training examples go through an encoder that predicts per-feature means μ and standard deviations σ; the reparameterization trick (ε·σ + μ) keeps gradients flowing through sampling; a two-layer decoder reconstructs the input.
• Key gradients: The SGVB estimator collapses the KL gradient to simply μ (pushing it to 0), the σ gradient is σ − 1/σ (pushing toward 1), and reconstruction reduces to Y − X (mean squared error).
• Takeaway: These two losses sit at the heart of modern methods — KL divergence is the penalty RLHF methods like GRPO use to keep fine-tuned models from drifting, and MSE reconstruction is exactly what trains diffusion models to denoise.

He also notes the VAE paper (Kingma & Welling) won ICLR's first-ever Test of Time Award in 2024.

Original post →

More from Research

• World Mechanics builds founding team to make physical AI interpretable from day one — soniajoseph_ · 2026-09-15
• Example of an Expert Task: Correct Specimen Order After Positive CIDT — soumitrashukla9 · 2026-09-15
• Amid new robotics launches, a pointer to best practices for policy evaluations — eigenron · 2026-09-15
• MolmoSpaces benchmark launched: GPT-Astra beats all open-source VLA baselines zero-shot — notmahi · 2026-09-15
• Review: engineering proteins that use fewer amino acids while keeping structure — KevinKaichuang · 2026-09-15
• Wayve researcher: robotics unlikely to find objectives far beyond next-token prediction — m_wulfmeier · 2026-09-15

Research channel →