Google researcher: having models write new training data via ICL improves finetuning generalization
AndrewLampinen · x · 2026-10-14
In a thread on synthetic data, Andrew Lampinen highlights work with konwookim et al. showing that in-context learning consistently generalizes better than finetuning on harder tests like syllogistic deduction. Building on this, they propose a method to improve finetuning generalization: prompt language models to augment training data via ICL — model-written data can genuinely improve real-data generalization, at least at later training stages.
Related event: New Research Suggests Cumulative Synthetic Data Won't Collapse Models(4 posts)→
More from Research
- Economist interviews AI agents on sudden windfalls — one goes deep on estate planning — soumitrashukla9 · 2026-10-14
- RL on large MoEs with zero train-infer mismatch: open-source Wordle demo on Qwen3.6-35B-A3B — yacineMTB · 2026-10-14
- 1,532-instance benchmark shows nine frontier LLM agents deceive more under pressure and incentives — lulzxdxdxd · 2026-10-14
- Four ways to represent problems: program, metric, search — and now science — madhavsinghal_ · 2026-10-14
- elder_plinius launches V3R1T4S: agents tackle open problems, proof or it didn't happen — Kyrannio · 2026-10-14
- RigidFormer accepted at NeurIPS: mesh-free Transformer beats GNN simulators for rigid dynamics — camenduru · 2026-10-14