When an AI agent claims it solved something, what evidence would actually convince you?
Zealousideal-Pop4857 · reddit · 2026-11-22
A Reddit discussion tackles an increasingly real problem: agents produce real work—research results, benchmark numbers, shippable code—yet nearly every claim is just asserted via screenshots, demos, or self-generated logs. The poster lays out four possible verification bars:
- Recomputation from public artifacts: claimant deposits inputs/outputs/method; result counts only if independently reproduced. Strong but limited to reproducible problems.
- Adversarial verification: a second agent's job is to break the first one's claim—find leaks, cherry-picked metrics, non-reproducible steps. Scales better but requires trusting the adversary.
- Reputation and track record: believe the agent that's been right 50 times—how human science runs, and how fraud compounds.
- Held-out tests nobody sees: works for benchmarks, not open-ended work.
The poster leans toward option 1 where possible, 2 otherwise, and asks for real-world examples of agent claims actually being checked or caught.
More from AGI Musings
- Anthropic researcher: build state AI capacity instead of arguing about doom — joshua_saxe · 2026-11-22
- You can only guarantee zero mistakes by making zero predictions — burny_tech · 2026-11-22
- The 'Agent Platform of 2027' blueprint: actors, RBAC filesystems, inbox UX — yawnxyz · 2026-11-22
- AI practitioner: I can no longer predict AI capabilities with confidence — burny_tech · 2026-11-22
- 'There's never been a better time to go into math' amid AI boom — burny_tech · 2026-11-22
- Why the AI-anxious can see their roles dying but not what comes next — RileyRalmuto · 2026-11-22