Invent a Dataset: zero-seed prompt-based data generation beats frontier APIs by 17% quality, 19% diversity
idanbeck · x · 2026-10-21
The Invent a Dataset paper (Shivalika Singh, Sara Hooker et al.) tackles the zero-data regime: no seed data exists for a capability you want to train.
- Prompt-based system goes from dataset description to large-scale realistic post-training data
- Benchmarked against 5 frontier APIs (Anthropic, Google, OpenAI, DeepSeek, Zai) across 8 task types, up to 20K samples
- 17% relative quality gain, 19% diversity gain; diversity advantage widens with scale — from parity at 200 samples to 37% at 20K
- Fine-tunes on its data consistently rank higher across architectures; notably, raw frontier-API synthetic data fine-tunes can fall below the untrained base on medical QA
Takeaway: systematic diversity control matters more than raw generator strength for synthetic data.
Related event: Cohere Paper: Inventing Datasets from Zero Seed Data Works Better(2 posts)→
More from Models
- X's translation nails a Zhuangzi quote with cultural context — 'They're learning' — teortaxesTex · 2026-10-21
- Startup humans& launches Ampersand, a model trained for collaboration instead of autonomy — niloofar_mire · 2026-10-21
- Kronos: open-source K-line foundation model trained on 12B candles, with a free AI trading setup guide — Roger_M_Taylor · 2026-10-21
- Asked Fable 5.1 why its game tooling is mediocre: it says NO, Fable 5 says 'hahaha maybe' — repligate · 2026-10-21
- Humor Elo leaderboard: Gemini 4.1 Arvon tops at 1573, GPT-5.1 Turbo last with 179 — jon_barron · 2026-10-21
- From 9 to 48 tokens/s: STRATA + CachyOS supercharges local Qwen Flash on an 8GB GPU — Ne00n · 2026-10-21