Deepfake Detection Papers Show Significant Real-World Drop
markjeffrey · x · 2026-07-17
BitMind released its deepfake detection paper and evaluation results: they point out that many detectors perform well on academic benchmarks but see AUC drops of 45–50% on real-world content, reflecting structural issues from evolving generative models.
They evaluated a frozen model uniformly across 19 public benchmarks without fine-tuning on individual datasets, achieving or surpassing the strongest results in multiple papers, including the best commercial detector on Deepfake-Eval-2024. The evaluation harness is public, and the production API serves the same evaluation snapshot for external verification.
More from Safety
- Polymarket echoes OpenAI’s claim that models exploited zero-days in Hugging Face incident — Polymarket · 2026-07-22
- OpenAI says benchmarked cyber-capable models compromised Hugging Face production — OpenAI · 2026-07-22
- Building a Secure AI Agent Gateway: Self-Hosting OAuth for Multiple SaaS Apps — Defiant_Cod_2654 · 2026-07-22
- Judge approves Anthropic’s $1.5 billion settlement over books used to train Claude — BeetleB · 2026-07-22
- Apple publishes SOC 3 audit reports for Private Cloud Compute — throwfaraway4 · 2026-07-22
- Agent Receives Fake System Messages During Execution, Raising Security Concerns — sandyyevans · 2026-07-22