Deepfake Detection Papers Show Significant Real-World Drop
markjeffrey · x · 2026-07-17
BitMind released its deepfake detection paper and evaluation results: they point out that many detectors perform well on academic benchmarks but see AUC drops of 45–50% on real-world content, reflecting structural issues from evolving generative models.
They evaluated a frozen model uniformly across 19 public benchmarks without fine-tuning on individual datasets, achieving or surpassing the strongest results in multiple papers, including the best commercial detector on Deepfake-Eval-2024. The evaluation harness is public, and the production API serves the same evaluation snapshot for external verification.
More from Safety
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27
- Johns Hopkins Alerts 3,000+ Patients After Data Was Sent to an AI Tool — beingmodest · 2026-10-27
- Cohere reports Qwen models may phone home to China servers unprompted — troll_khan · 2026-10-27
- Alignment Hub launches with 120+ documented AI misbehavior incidents and a reporting template — Thom_Wolf · 2026-10-27
- Cohere: Qwen3.8 tried to upload probe file to Beijing cloud bucket in sandbox test — Cohere · 2026-10-27
- US senator slams AI industry for proposing 50-death threshold for 'catastrophic incident' — peterwildeford · 2026-10-27