Ex-OpenAI's Kaiser: reasoning models trained on cleaned model traces plus RL
yoavgo · x · 2026-09-26
In a discussion with Yoav Goldberg, Łukasz Kaiser explains that reasoning models rely heavily on synthetic data: massive amounts of model reasoning traces are cleaned and summarized by another model—with substantial human prompting, domain knowledge, and filtering—then trained on, followed by another round of RL.
Goldberg follows up on two fronts: how much the 'human prompting' and 'domain knowledge' actually matter, and whether such reasoning capabilities generalize at all—or whether excellent performance on a task/domain should be read as evidence of targeted RL and domain-specific data injection on that domain.
Related event: Researchers Debate Where Frontier Models' Reasoning Comes From(12 posts)→
More from Research
- Full paper: active learning cuts surgical video annotation effort by ~50% — santiviquez · 2026-11-11
- Active learning with weak supervision cuts surgical video annotation effort by ~50% — Laparoscopes · 2026-11-11
- Active Learning with Weak Supervision Cuts Surgical Video Annotation Effort by 50%, Penn Study Finds — Laparoscopes · 2026-11-11
- Tuva Labs takes 1st on ARC-AGI-2, 2nd on ARC-AGI-3 in ARC Prize 2026 — GregKamradt · 2026-11-11
- Sakana AI × Pathway Post-Transformers lecture on AI and the brain now online — kaixhin · 2026-11-11
- Defending METR: public data can't second-guess its AI speedup findings — peterwildeford · 2026-11-11