Simulation beats distillation: the real story of synthetic data is post-training worlds
realsohamparekh · x · 2026-09-26
A post-training take gaining traction (amplified by realsohamparekh and others):
- The interesting part of synthetic data may not be the data; in post-training, simulation will matter more than distillation.
- Distillation hands the model answers to copy from a teacher. Simulation builds a world for it to mess around in — a fake user requesting changes, an unexpected event triggering a chain of effects — and it learns from outcomes like a human would. Data quality now hinges on how well your simulated world matches the real one.
- The claim: distillation becomes largely an inference problem at scale; simulation is a product + modeling problem that remains unsolved. The author is excited that we're "rediscovering" something old.
A high-quality directional argument for anyone working on agent training and post-training.
More from Research
- InternLM open-sources Intern-Decision 4B/0.8B: structured decisions in one forward pass — jacek2023 · 2026-09-26
- Stanford's Noah Goodman uses philosophy to improve LLM pretraining, jokes ASI achieved — xuanalogue · 2026-09-26
- Researchers surface spurious probes across models: Sonnet 5 recommends green tea in evals, oolong in production — jankulveit · 2026-09-26
- Three papers, one warning: 1% synthetic data can trigger strong model collapse — suchenzang · 2026-09-26
- Experts Rise Where LLMs Disagree: rationale labeling cuts codebook revision from months to days — windx0303 · 2026-09-26
- Dev hails continual learning paper: AGI defined in 2000, only now is anyone training for it — willcb · 2026-09-26