95% Confidence Intervals Really Cover Only 10-25% of the Time, Statistician Notes
RexDouglass · x · 2026-09-11
Rex Douglass rephrases an observation from Robin Hanson: confidence intervals implying 95% coverage of a population estimate really have only 10-25% coverage, assuming replications are additional draws from the same population. Likewise, a finding with p=0.05 has an expected replication probability of just 0.10 to 0.25 across fields — a concise reminder of how far nominal significance falls short of real reproducibility.
Related event: Researcher Claims 95% Confidence Intervals Cover Only 10-25% in Practice(2 posts)→
More from Research
- NVIDIA Nemotron 3 Embed 8B tops Perplexity's new Q2D-Web agentic RAG benchmark — JFPuget · 2026-09-11
- Third-party benchmark claims GPT-6 Astra leads frontier models at antibody developability prediction — gdb · 2026-09-11
- 451 humans vs 31 LLM simulators: simulated users are too nice and inflate agent scores — niloofar_mire · 2026-09-11
- RL training shifts behavior regimes across scales, not pretraining priors — 1a3orn · 2026-09-11
- Researcher Pushes AI-Assisted Formal Proof to Enable Large-Scale Math Collaboration — snikolov · 2026-09-11
- Notable change: K3 diverges from predecessor on vision encoders — stochasticchasm · 2026-09-11