New paper shows models can learn general patterns from zero real-world data
mark_k · x · 2026-09-25
A new paper describes a training setup that uses no real-world data at all: two models start from scratch, one writing small programs that generate byte sequences and the other learning to predict them. The first model then learns to generate sequences that push the second to improve.
After training only on these generated sequences, the learner got better at predicting real text, images, speech, and DNA as compute scaled up — an early proof of concept that useful general patterns can emerge without any natural examples.
More from Research
- Nace.AI Launches Drex, a <6B Model Built Only to Decide, Topping Its Decision Index — rohanpaul_ai · 2026-09-26
- Dharmamitra releases Tibetan and Sanskrit lexicons as free Stardict and Apple Dictionary files — SebastianNehrd2 · 2026-09-26
- BTL claims 'Interference Search' architecture lifts 1.7B model from 3/30 to 23/30 on hard problems — CatAstro_Piyush · 2026-09-26
- Chai-2 Hits 16% on De Novo Antibody Design: 20 Designs per Target, Every Hit Wet-Lab Confirmed — le_james94 · 2026-09-26
- With 60k ICLR abstracts, a researcher mourns LLM-generated figures homogenizing papers — mariyaivasileva · 2026-09-26
- AWS Walkthrough: SkyRL GRPO on HyperPod Lifts Qwen3-VL Maze Solve Rate From 43.75% to 95%+ — AWS ML Blog · 2026-09-26