Researcher: 95% confidence intervals may really cover just 10-25% of the truth
RexDouglass · x · 2026-09-11
Rex Douglass argues on X that confidence intervals implying 95% coverage of a population estimate may in practice deliver only 10-25% coverage, assuming replications are additional draws from the same population.
He adds that the coverage problem isn't symmetric but systematically biased in whatever direction researchers need to publish — which he calls even worse than the raw numbers suggest.
Related event: Researcher Claims 95% Confidence Intervals Cover Only 10-25% in Practice(2 posts)→
More from Research
- NVIDIA Nemotron 3 Embed 8B tops Perplexity's new Q2D-Web agentic RAG benchmark — JFPuget · 2026-09-11
- Third-party benchmark claims GPT-6 Astra leads frontier models at antibody developability prediction — gdb · 2026-09-11
- 451 humans vs 31 LLM simulators: simulated users are too nice and inflate agent scores — niloofar_mire · 2026-09-11
- RL training shifts behavior regimes across scales, not pretraining priors — 1a3orn · 2026-09-11
- Researcher Pushes AI-Assisted Formal Proof to Enable Large-Scale Math Collaboration — snikolov · 2026-09-11
- Notable change: K3 diverges from predecessor on vision encoders — stochasticchasm · 2026-09-11