Agents reproduce most published findings: ~80-90% correct signs, >70% within CI
RexDouglass · x · 2026-10-24
Sharing results from ellliottt: agents tasked with reproducing published empirical findings get the sign of coefficients correct 80–90% of the time, and over 70% of estimates fall within the original 95% confidence interval (best models exceed 80%). A real step toward automated reproducibility of published research.
More from Research
- Fleuret: teacher forcing turns time iteration into depth iteration, and GPUs love it — francoisfleuret · 2026-10-24
- Domingos: An attention head is just a fast-weight MLP — pmddomingos · 2026-10-24
- Ross Taylor: studying LLM behavior will increasingly resemble social science — rosstaylor90 · 2026-10-24
- Ross Taylor on RLCR: simplicity is the final achievement, details matter — rosstaylor90 · 2026-10-24
- SenseTime open-sources 27B robot model matching GPT-6-Astra at ~1/50 the cost — rohanpaul_ai · 2026-10-24
- Cohere Labs brings five main track papers to EMNLP 2026 in Budapest — Cohere_Labs · 2026-10-24