Hype check: '20x better than RL' holds in 1 of 3 Atari games; math gain is just 44.6% to 48.1%
ccerrato147 · x · 2026-11-04
The author audits the hyped claims around MIT CSAIL's 'AI fixes AI weights' demo. 'Twenty times better than reinforcement learning' holds in only one of three Atari games: Seaquest went 89 to 53, Asterix 55 to 53, and a Breakout rerun with Opus 5.5 scored 58 vs 25 — twice the trained network, not twenty times. The 'six times better at math' figure comes from thinking mode, where the 0.8B model loops until it hits the token limit. Robert Joseph at Caltech ran the untouched model in default mode and got 44.6%; the edit reaches 48.1% — the loop fix merely gave back performance the model already had.
More from Models
- Tiny LoRA adapter beats DNA model Carbon-X on RareBench, exposing generalization limits — danielmckinn0n · 2026-11-04
- LangChain benchmarks 7 browser toolsets: Stagehand wins with 90% pass rate at $1.98 vs $83.62 — LangChain · 2026-11-04
- DeepSeek quietly launches "DeepSeek Harness" YouTube channel, hinting at plugin-based agent framework — gnukeith · 2026-11-04
- Reddit user seeks uncensored local model that runs on a weak PC for search and bulk downloads — Emotional_Orange_279 · 2026-11-04
- DeepSeek partial outage fuels speculation compute is shifting to unreleased V4.1 Pro — AccBalanced · 2026-11-04
- Claude hand-writes a 984-param neural net for C syntax highlighting, 5x faster than Pygments — AlpinDale · 2026-11-04