Updates on Statistical Test Writing and Correction Methods
IanArawjo · x · 2026-07-18
This reply mentions that a comprehensive guide on Monte Carlo simulations has been completed, covering p-value tests, confidence interval methods, and FWER control methods.
The author adds that the story around FWER is more complex, but reports that Wilcoxon signed-ranks remains robust for p-values across different sample sizes and data types. They also note ongoing efforts to compile PPI-corrected stats tests, including ANOVA, Wilcoxon, and Friedman.
Related event: evalstats Monte Carlo Simulations and Statistical Testing Methods Released(2 posts)→
More from Research
- Retrospective: Hero scaling forecast fixed without using Hero losses, code open-sourced — dlwh · 2026-10-27
- Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27
- Fixing compute convention and fitted floor cuts scaling forecast error ~79% — dlwh · 2026-10-27
- Marin's dlwh Explains the Scaling Law Fit: Estimating the Loss Floor Early — dlwh · 2026-10-27
- Marin's Open Scaling Law Extrapolates 300× and Predicts Loss Within 0.2% — dlwh · 2026-10-27
- Delegating scaling-law asymptote fitting to Codex while planning a birthday party — dlwh · 2026-10-27