Data World paper: RL on synthetic data environments adds 20.9 points on DABstep-Hard
omarsar0 · x · 2026-10-19
A paper from Nanjing University, Meituan's LongCat team and collaborators introduces Data World, an engine that synthesizes data-analysis training environments from scratch.
- Key idea: specify the mechanisms that generate a dataset first, then produce files, tasks and grading rubrics from them. Since the generative process is known, open-ended findings are verifiable, and independent audits confirm each answer is recoverable from the files.
- Benchmark: DataWorld-Bench spans 100 tasks across 50 held-out worlds with human-preference curation for realism. Evaluating ten frontier models reveals a consistent computation–insight gap: models compute the right numbers more often than they draw the right conclusions from them.
- RL results: Training Qwen3.6-35B-A3B with RL on 2,135 synthesized tasks, with no distilled trajectories, improves over the base model by 7.06 on DataWorld-Bench, 14.28 on DAComp, 20.90 on DABstep-Hard, and 12.13 on JobBench.
More from coding & agent
- Swarms ships v16 'Overclock', claims 53.7k GitHub stars and 800k developers — KyeGomezB · 2026-10-19
- Arvind Narayanan: The question shifts from 'What can AI do?' to 'What should AI do?' — soumitrashukla9 · 2026-10-19
- Indie dev builds Utsushi, an open-source Higgsfield clone in Go/HTMX — _playlogic_ · 2026-10-19
- Dust hyper-optimization delivers ~10x speedup, verified; beyond-backprop algorithms coming — burny_tech · 2026-10-19
- Writer runs all-day workflow in Claude's in-app browser, agents apply doc edits as suggestions — danshipper · 2026-10-19
- 30 Days, 30 AI Agents: A Daily Series on AI Agents for Biotech and Pharma R&D — nlarusstone · 2026-10-19