Explainer video: fine-tuning on raw frontier-API synthetic data underperforms the base model
idanbeck · x · 2026-10-21
- Author shares an explainer video for the Invent a Dataset paper (Sara Hooker et al.)
- Surprising finding: in medical QA tests, models fine-tuned on raw frontier-API synthetic data ranked below the untrained base model
- Root cause: diversity of synthetic samples collapses, and the problem worsens with added constraints
Related event: Cohere Paper: Inventing Datasets from Zero Seed Data Works Better(2 posts)→
More from Models
- X's translation nails a Zhuangzi quote with cultural context — 'They're learning' — teortaxesTex · 2026-10-21
- Startup humans& launches Ampersand, a model trained for collaboration instead of autonomy — niloofar_mire · 2026-10-21
- Kronos: open-source K-line foundation model trained on 12B candles, with a free AI trading setup guide — Roger_M_Taylor · 2026-10-21
- Asked Fable 5.1 why its game tooling is mediocre: it says NO, Fable 5 says 'hahaha maybe' — repligate · 2026-10-21
- Humor Elo leaderboard: Gemini 4.1 Arvon tops at 1573, GPT-5.1 Turbo last with 179 — jon_barron · 2026-10-21
- From 9 to 48 tokens/s: STRATA + CachyOS supercharges local Qwen Flash on an 8GB GPU — Ne00n · 2026-10-21