Persimmon team reveals internal learning evals were cut from the blog
gharik · x · 2026-10-31
A Persimmon team member revealed in a reply that they had internal evals more directly measuring learning, but couldn't include them in the blog due to limits on what they can publish as a company.
What they do show: Persimmon makes errors at a rate more similar to humans than other models — necessary but not sufficient for matching broader "learning." The most compelling evidence, per the author, is that training against Persimmon's learning signals yields bigger gains on evals built from human-learner data than training against non-Persimmon learners; they hope to validate this in deployment with real learners.
More from Research
- Researcher: capability transfer hits a ceiling when training against less human-like models — saurabh_shah2 · 2026-10-31
- Compaction study: agent constraints vanish 6x more often than task goals in long sessions — maksym_andr · 2026-10-31
- CMU's Gym-Anything uses coding agents to auto-build 200 robot RL environments; best model scores 0% — AkariAsai · 2026-10-31
- Cherimoya ChIP-seq models ship as-is: bad underlying data means bad predictions — jmschreiber91 · 2026-10-31
- When chromatin models disagree: null accessibility shift but a disrupted CEBPG binding site — jmschreiber91 · 2026-10-31
- Whole-Body Dexterity turns the humanoid body into one large dexterous manipulator — chris_j_paxton · 2026-10-31