RuneBench makes LLMs grind RuneScape: Claude's overthinking hurts its XP
Own-Turnover-7641 · reddit · 2026-10-20
A humorous benchmark called RuneBench drops LLMs into RuneScape and measures XP grinding efficiency given only the wiki — and the failures are telling.
- Some models spend so long reading docs and planning that it actively hurts performance: "access to all human knowledge, yet 10 minutes figuring out the optimal way to kill chickens"
- Gemini gets wrecked in the wilderness by ice giants while trying to mine
- GPT does smithing with click bursts
- Claude, in classic fashion, overthinks — rereading wikis, rerolling for lucky drops, and scripting smithing attempts that fail repeatedly
Benchmark: runebridge at maxbittker.github.io/runebench
Related event: RuneBench Puts LLM Agents to the Test Inside RuneScape(2 posts)→
More from Fun
- AI sycophancy in one screenshot: 'you're absolutely right' — tetsuoai · 2026-10-23
- Burkov on Authors Who Don't Know They Own Their Own arXiv Paper Copyright — burkov · 2026-10-23
- Corporate life in 2026: dozens of AI agent coworkers flooding a channel with reactions — kimberlywtan · 2026-10-23
- Gemini refuses to generate image in Docs, blames document size limit — viksit · 2026-10-23
- Dev jokes his true calling is telling everyone to try the latest DeepSeek — intellectronica · 2026-10-23
- ICLR Submission Found Containing LLM's Own Meta-Comment Admitting Results Weren't Significant — _onionesque · 2026-10-23