Reanalysis finds 72% of significant results stay significant under alternative choices
RexDouglass · x · 2026-09-04
I4Replication highlights that reproducibility ≠ robustness: when the same data is re-analyzed using reasonable alternative analytical choices, 72% of statistically significant results remain significant in the same direction. The thread argues that even replicable findings need separate robustness checks, and the space of methodological choices is a key dimension for judging research reliability.
More from Research
- Nanjing University mathematician claims proof of Catalan's constant irrationality, with GPT-5.6 Sol's help — apples_jimmy · 2026-09-04
- Amazon-Microsoft paper: skill-guided action chunking lifts agent success to 67.2% while halving LLM calls — rohanpaul_ai · 2026-09-04
- SPACE (EMNLP 2026): learning chunk boundaries from trajectories cuts agent LLM rounds by up to 78.9% — rohanpaul_ai · 2026-09-04
- Last Translation Benchmark Is Live: Contribute 10 Examples, Get Co-authorship — zouharvi · 2026-09-04
- Translation Models Fail Beyond Figurative Language, Says Benchmark Author — zouharvi · 2026-09-04
- Machine Translation Is Not Solved: Last Translation Benchmark Released — zouharvi · 2026-09-04