Private evals: pplx-decider shows 40% recall gain over Jev decision models
marktenenholtz · x · 2026-10-20
marktenenholtz reports that on his private eval set, pplx-decider showed a 40% increase in recall over Jev, concluding that "decision models have a long way to go." No eval details are given, but it's a concrete data point on the current state of decision models.
More from Models
- Hamel Husain Benchmarks Six Decision Models for Evals: Threshold Tuning Beats Model Choice — Hamel Husain · 2026-10-20
- Hidden '6.5' and countdown in opensi.lol source spark OpenAI GPT-6.5 announcement rumors — Existing_Hat_1064 · 2026-10-20
- Muse Wonder finishes tasks in 6.8 minutes, twice as fast as Claude Opus 5.5 — shensi · 2026-10-20
- All top six open-weights models now come from Chinese labs — shensi · 2026-10-20
- Kimi K3 excels on new Frontier Code v1.1 bench but flops on Terminal Bench v4 — Ok_Warning2146 · 2026-10-20
- GPT-6 Plus users report stuttering generations and a big step back in contextual memory — ilovesaintpaul · 2026-10-20