FrontierMath Erdős shows top models near 0% — researchers say hard math benchmarks aren't that hard to make
Jsevillamol · x · 2026-11-08
GregHBurnham is surprised that 'Model 4' scores >50% on all benchmarks, arguing harder math benchmarks aren't that hard to create. A FrontierMath Erdős accuracy chart shows OpenAI, Anthropic, Google, and xAI models released May–Sept 2026 all near 0%, with only a few reaching 3-5%. He also questions what (A)ECI actually measures — even infinite progress may be far from 'effecting arbitrary states in the world better than humans'.
More from Models
- 'Rank bottleneck' means two very different things in linear algebra vs systems optimization — kalomaze · 2026-11-08
- Double-exponential fit suggests model time horizons could hit ~80 work-years by end of 2027 — scaling01 · 2026-11-08
- Speculation: Anthropic's recent releases are all distills, Opus 5.5 may be Model 2 Early — haider1 · 2026-11-08
- Stanford Prof Anshul Kundaje on Weight Space Editing: Replicated Across Models, Limits Still Unclear — anshulkundaje · 2026-11-08
- Jev search site processes 1M+ tokens for $0.00 — priced per billion tokens — HanchungLee · 2026-11-08
- Open coding models turned telegraphic: filler words down 25–50% across 12 models — thesephist · 2026-11-08