Patronus open-sources GlobeBench: 1,632 tasks testing if LLMs can simulate RL environments
mariyaivasileva · x · 2026-11-08
Patronus AI has open-sourced GlobeBench, a suite of 1,632 tasks testing whether LLMs can simulate RL environments well enough to train agents on. Across 20 models tested, only GPT-6 Astra scored 49.6%; no other model reached 30%.
Researcher Mariya Ivasileva highlights two intriguing questions: can a poor one-step simulator still provide a useful RL environment, and can a highly accurate simulator teach an agent the wrong behavior? GlobeBench addresses part of this by mapping which state transitions LLMs can and cannot reproduce. The eval, weights, and training data are publicly available.
Related event: Patronus Open-Sources GlobeBench for Testing LLMs as RL World Models(4 posts)→
More from Research
- Bayesian optimization reconstructs 3D atomic structures from one microscopy image in ~75s — bravo_abad · 2026-11-08
- lateinteraction asks: who's building 'Tinker for weight-space programming'? — lateinteraction · 2026-11-08
- LeCun clarifies: DINO is a JEPA variant and most JEPA papers are just fine-tunes — ylecun · 2026-11-08
- Model underperforms on most of its own benchmarks, still lands in Nature — LeoTZ03 · 2026-11-08
- Robotics researcher proposes abolishing journals for never-rejecting Paper Grading Boards — tomssilver · 2026-11-08
- FOCS 2026 to host joint TCS+AI workshop with Daniel Litt keynote and all-star panel — Aaroth · 2026-11-08