Self-improving agent lifts SimpleQA to 88% but GPQA Diamond drops to 53%
krishnan · x · 2026-11-20
krishnan unpacks RSI-Env, a new preprint from Fudan, HKUST, Shanghai AI Lab and others (author-reported, not independently replicated): an agent starting from Qwen3.8-27B improves its own model and harness. SimpleQA rose 82%→85%→88% after harness changes and fine-tuning on 48 examples, but GPQA Diamond fell 65.2%→53.0%, while AIME 2026 math rose 53.3%→66.7%. Gains on target benchmarks came at the cost of other capabilities.
More from AGI Musings
- Can AI Think Without Language? X Debate Argues Models Have Silent Inner Cognition — repligate · 2026-11-20
- YC Startup Clean Hits $500K ARR Two Months In, With 160%+ Net Revenue Retention — ycombinator · 2026-11-20
- Software factories underdeliver: revisiting the 'autonomous product org' loop design vision — pzakin · 2026-11-20
- YC W27 startup Operon builds the 'operating context layer' for heavy industry, saving clients $2.8M a year — ycombinator · 2026-11-20
- Caine Prize winner flagged as AI-generated by detector Pangram as organizers investigate — paulnovosad · 2026-11-20
- Impetus Grants seeks philanthropy to build the biological data AI needs to cure aging diseases — juanbenet · 2026-11-20