Self-improving agent lifts SimpleQA to 88% but GPQA Diamond drops to 53%

krishnan · x · 2026-11-20

krishnan unpacks RSI-Env, a new preprint from Fudan, HKUST, Shanghai AI Lab and others (author-reported, not independently replicated): an agent starting from Qwen3.8-27B improves its own model and harness. SimpleQA rose 82%→85%→88% after harness changes and fine-tuning on 48 examples, but GPQA Diamond fell 65.2%→53.0%, while AIME 2026 math rose 53.3%→66.7%. Gains on target benchmarks came at the cost of other capabilities.

Original post →

More from AGI Musings

AGI Musings channel →