New paper shows models can learn general patterns from zero real-world data
mark_k · x · 2026-09-25
A new paper describes a training setup that uses no real-world data at all: two models start from scratch, one writing small programs that generate byte sequences and the other learning to predict them. The first model then learns to generate sequences that push the second to improve.
After training only on these generated sequences, the learner got better at predicting real text, images, speech, and DNA as compute scaled up — an early proof of concept that useful general patterns can emerge without any natural examples.
More from Research
- Mobile AI Factory: agentic pipeline builds on-device SLMs that beat 5-shot frontier models — kalyan_kpl · 2026-11-11
- Amazon paper: single agents beat multi-agent setups on enterprise SOPs at 1.4–4.7x lower cost — rohanpaul_ai · 2026-11-11
- Amazon paper: multi-agent setups burn 1.4-4.7x tokens for small gains; single agent is the default — rohanpaul_ai · 2026-11-11
- MIT's Buehler builds "recursive meta-intelligence": hundreds of AI agents inhabit self-created worlds to study material failure — ProfBuehlerMIT · 2026-11-11
- Explosion launches Ellf directory: 52 NLP templates plus multi-agent annotation workflow — _inesmontani · 2026-11-11
- STABLE Optimizer Accepted at NeurIPS 2026: Taming Continual Learning Forgetting at the Optimizer Level — v_lomonaco · 2026-11-11