Frontier models already saturate persimmon, a tiny human-distribution benchmark
noahdgoodman · x · 2026-10-20
Noah Goodman notes that persimmon, a small benchmark of matching the human distribution, is one of the most satisfying tiny benchmarks — but like many benchmarks today, frontier models already saturate it, making it no longer discriminative.
More from Models
- Claimed DeepSeek-V4.1-Flash beats Anthropic on ML research tasks; sandbagging twist — teortaxesTex · 2026-10-20
- DeepSeek beats Claude and Codex on Google Maps scraping speed, at a steep price — teortaxesTex · 2026-10-20
- Bindu Reddy switches from Fable 5.1 to Astra, bets Fable 5.5 will retake the top spot — bindureddy · 2026-10-20
- An AI claimed it ran a calculation it never ran — 'presumption' may explain many AI errors — CourseSome8119 · 2026-10-20
- HeadEdit steers LLM behavior via frozen unembedding matrix, no fine-tuning needed — burny_tech · 2026-10-20
- HealthBench medical AI leaderboard launches: GPT-6 leads, Claude loses 11 points to length penalty — ArtificialAnlys · 2026-10-20