Training a one-step language model without a teacher: Discrete Beckmann Transport Models
rishabh16_ · x · 2026-10-12
Sophia Tang (UPenn/Harvard) published an interactive technical blog on Discrete Beckmann Transport Models (DBTM), a family of one-step language models that learn a time-independent flow's endpoint map directly from data—no teacher distillation needed.
- Removing time dependence yields straighter trajectories on the simplex and stable equilibria; each extra function evaluation refines a complete sequence.
- Achieves the Pareto frontier on LM1B and OWT; 84.6% on Sudoku and 16.8% on GSM8K with few NFE.
The post includes many animations explaining how the transport map carries Gaussian samples to clean token vertices in one step.
More from Research
- Sim-to-real lessons: RL-trained robots vastly outperform behavior cloning, UW work shows — chris_j_paxton · 2026-10-12
- New Study Quantifies What Diffusion Models Generate First: Only RAE Space Shows Abstract-to-Concrete Order — sedielem · 2026-10-12
- EmbeddingGemma 2 lands on Hugging Face, giving science projects a free accuracy boost — HildeKuehne · 2026-10-12
- Pedro Domingos on harness learning paper: the model swallows the harness — DanielKhashabi · 2026-10-12
- Data-space iteration: a new few-step generation paradigm removes flow discretization, arXiv paper shows — bdsqlsz · 2026-10-12
- Is 'Attention' the Essence of Intelligence? Quanta traces the mechanism from 2014 to LLMs — calabi_and_yau · 2026-10-12