Using one model to audit three vendors' AI incident registry — from evidence QA to fixing the frontend
ValehartProject · reddit · 2026-09-03
Rather than redoing prior security research conducted with OpenAI, Gemini, and Anthropic models, the author handed the existing evidence and analysis to a current model for audit. Across the session it challenged earlier conclusions and downgraded unsupported claims, separated observed evidence from inference, hypothesis, and overreach, reconstructed disclosure timelines, verified subsequent research on the web, pulled prior research from Notion, analyzed screenshots, reassessed risk classifications across all three vendors, maintained provenance boundaries, and rewrote the public incident records.
It then switched to implementation: from screenshots of the broken registry UI alone, it diagnosed HTML/CSS problems, rewrote components, added tabbed navigation, dynamic status info, and dated source links. The author's takeaway: not any single feature, but chaining them against one persistent body of work.
The post ends with a checklist for evaluating Astra's security capabilities: detecting prior overreach, cross-verifying claims, finding what earlier models missed, maintaining provenance boundaries, recognizing cross-incident relationships without inventing causality, and handling large evidence sets.
More from coding & agent
- Nanjing University's Specula uses coding agents to auto-generate TLA+ specs, finds 382 deep bugs — jiqizhixin · 2026-09-03
- AI agent buys a JJ Watt jersey end-to-end in 13 minutes, with papercuts — jeff_weinstein · 2026-09-03
- rakyll questions whether binary identity and attestation matter in real agentic flows — rakyll · 2026-09-03
- Where Do Multi-Agent Workflows Break in Production: State, Approvals, Eval or Rollback? — Medium-Lie8127 · 2026-09-03
- Tigerless Labs' auto-gtm drafts X/Reddit posts from your PRs, never auto-posts — Aiden_Tech_Ai · 2026-09-03
- How to measure agent quality beyond task success: dev seeks real-world eval metrics — serpratik · 2026-09-03