Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation
Marco Alecci, Francesco Marchiori, Iyiola Emmanuel Olatunji, Tegawendé F. Bissyandé, Jacques Klein
cs.AI, cs.CR, cs.SE
2026-07-30
A large-scale black-box audit of OpenAI, Amazon, and Google content moderation APIs shows that gradient-free image transformations like color inversion bypass up to 44% of flagged images with no model access or ML expertise required.
Commercial content moderation APIs from OpenAI, Amazon, and Google offer broader policy coverage and richer semantic understanding than traditional task-specific classifiers. The implicit promise is that stronger foundation models also mean more robust moderation. This paper runs a large-scale black-box audit to test that assumption.
The attack toolkit is deliberately minimal: no gradients, no surrogate models, no knowledge of the target system. If that level of effort is enough to bypass these systems, the robustness claim doesn't hold.
The audit targets three commercial APIs: OpenAI omni-moderation, Amazon Rekognition, and Google Cloud SafeSearch. Seven image transformations fall into two groups:
The experiment used approximately 600,000 API calls across three datasets: LSPD (1,000 pornographic images sampled from 500,000+), UnsafeBench (3,010 images across 11 unsafe categories), and Hateful Memes (11,605 meme images with text overlays). Image quality is controlled by MS-SSIM threshold τ: 0.6 allows visible distortion, 0.85 stays close to the original.
The primary metric is ASR (attack success rate): the fraction of originally-flagged images reclassified as safe after transformation.
One-shot transformations on LSPD (τ=0.6):
| Transformation | Amazon ASR | Google ASR | OpenAI ASR |
| Color inversion | 43.97% | 6.29% | 8.11% |
| Grayscale | 1.77% | 1.12% | 4.17% |
| RGB split-merge | 3.12% | 0.81% | 1.32% |
Amazon's exposure to color inversion stands out: nearly half of pornographic images pass undetected after a single color flip.
Intensity-dependent attacks spread damage more evenly. Salt-and-pepper noise at τ=0.6 achieves 45.53% ASR on Amazon, 31.68% on Google, and 30.37% on OpenAI — all above 30%. Coverage numbers are more striking: Gaussian noise has 100% coverage on LSPD (every image has some intensity at which its classification flips), and salt-and-pepper reaches 99.56%. Virtually no image is immune if the attacker accepts moderate quality loss.
Content category matters. Self-harm is consistently the most vulnerable category across all three APIs; violence shows the lowest ASR. Multimodal content is notably susceptible: color inversion achieves 21.47% ASR on OpenAI for Hateful Memes versus 8.11% for the purely visual LSPD content, with text overlays appearing to amplify the confusion.
These APIs are the primary content filter for many platforms. The conclusion is direct: no single commercial moderation API is a reliable security boundary.
The attacker effort here is near zero: no ML expertise, no model weights, no training data. Anyone with PIL or ImageMagick can reproduce this. The attacker-defender asymmetry is the core concern, not any particular bypass rate.
The practical recommendation is layered defense: combine multiple API signals, add non-image context (metadata, user behavior), and run periodic black-box robustness audits on whatever moderation stack you rely on.
The main acknowledged limitation is API cost. The cross-provider comparison in RQ1 uses LSPD only, so whether "Amazon is more vulnerable than OpenAI" holds across content types remains untested.
The paper doesn't test combined attacks, where an adversary applies multiple transformations or searches for the most effective combination. Real adversaries aren't constrained to a single operation. Open-weight models like LLaVAGuard and ShieldGemma receive no evaluation, though they're increasingly practical alternatives to commercial APIs.
Dataset coverage is a standing concern: LSPD is predominantly pornographic content, and the 11 UnsafeBench categories may not reflect how harm actually distributes in production traffic.