Eight systems tie at 52.67% on Agent Memory Leaderboard: how should eval tables handle it
_sRhubarb_ · reddit · 2026-10-21
Using the Agent Memory Leaderboard's August public board as a case: 43 systems on the Cycle 1 coding board score 46.00%–52.67% over 150 tasks, with eight tied at 52.67%. Since one task is worth 0.67 points, the top cluster shifts one task at a time. The author argues the convergence is informative and discusses presentation options: raw counts beside percentages, tier groupings, and interval estimates, asking what readers actually engage with.
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23