Math lacks population-level reliability metascience, even with strong local certification

RexDouglass · x · 2026-07-27

The post argues that mathematics has a mature culture of local certification — peer review, post-publication correction, and occasional formal verification — but still lacks a true population-level metascience of reliability.
The missing piece is not a simple “replication crisis,” but a measurement and surveillance gap: the field does not know how often published results contain material defects, how often those defects are detected, or how much checking is required.
The image lays out a comparison across professional norms, peer review, post-publication correction, computational reproducibility, formal verification, and AI evaluation, highlighting what is currently known and what is still missing.
The qualitative takeaway is that referees usually focus on central or suspicious proof steps, while publication is often only the start of longer-term community validation.
In the thread’s framing, this lack of field-level instrumentation is part of why mathematics can look precise locally while still being hard to assess globally.

More from AGI Musings