LLM IMO 2026 test shows harnesses help, but frontier models still stay ahead
pequalnp92 · reddit · 2026-07-26
A comparison of several LLMs on IMO 2026 found that frontier models were near-perfect regardless of harness, while harness quality materially changed the results for other models.
Main findings:
- Frontier models (sol and fable) achieved perfect or near-perfect scores with or without orchestration.
- Sonnet and Opus did much worse in the web app, improved with the provider harness (Claude Code), and improved further with AutoFyn, a custom multi-agent harness.
- Open-weight GLM performed roughly at Sonnet’s no-harness level and improved similarly with AutoFyn.
- Even with harnesses, the team could not match frontier-model performance.
The authors say the hardest problem still required the key mathematical idea, not just better retrieval or verification. They also note hallucinations persisted: in one case Sonnet produced a false solution that had to be caught by grading and manual review.
More from coding & agent
- It takes 10 messages to validate, 100 to ship an MVP, and 1,000 for production — tristanbob · 2026-07-26
- LiveKit adds huddles, letting humans and agents join the same live conversation — jasonkneen · 2026-07-26
- A coding agent deletes 50 lines, adds 1,292, then admits the real fix was tiny — xeophon · 2026-07-26
- Agent deck generation needs strict schemas and a deterministic fallback, not a magic API call — Live-Purpose-641 · 2026-07-26
- AI coding benchmarks should report performance, cost, and time distributions — zainhas · 2026-07-26
- Official Obsidian CLI guide shows how AI agents can drive a vault from the terminal — dSebastien · 2026-07-26