Wilcoxon signed rank test outperforms McNemar for binary data evals
IanArawjo · x · 2026-08-31
A persistent result from the evalstats paper suggests that for pairwise p-value tests on binary data, the Wilcoxon signed rank test is optimal. It outperforms the McNemar mid-p method in power without degrading Type I error rates.
Related event: Wilcoxon Signed Rank Test Beats McNemar for Binary Evals(2 posts)→
More from Research
- Frontier AI models will revolutionize peer review and scientific writing — Alert-Elk-2695 · 2026-08-31
- Evidence of Fraud Found in Influential Procrastination Study — RexDouglass · 2026-08-31
- RL Experts Surprised by Agents Voluntarily Self-Destructing to Aid Peers — ZeroStateReflex · 2026-08-31
- Milestone: pLM-designed peptides work in vivo, selectively degrading β-catenin in mice — arjunrajlab · 2026-08-31
- Fast Sim2Real Workflow: Mobile Policy Viewer and Parameter Sweeping — yacineMTB · 2026-08-31
- Chollet responds to ARC-AGI eval dispute: don't claim untested scores — fchollet · 2026-08-31