AI Agents Report Significant Findings in Null-Effect Data 98% of the Time
A simulation study found frontier AI agents reported statistically significant results in 98% of cases even when given synthetic data with no real effects, raising concerns about the reliability of AI-generated research findings.
2026-11-19 ~ 2026-11-19 · 2 related posts
- Agents happily find "defensible" findings in synthetic data with zero true effects — RexDouglass · 2026-11-19
- Frontier agents report significant findings up to 98% of the time on null-effect data — RexDouglass · 2026-11-19