China Fines Firms for Poisoning AI Chatbot Answers With Fabricated Content
Sanbi_Ai · reddit · 2026-10-21
China's 21st Century Business Herald has counted three fines — in Beijing, Guangzhou and Shenzhen — of 50,000 to 120,000 yuan each against agencies that plant fabricated content to get brands cited in AI chatbot answers. The Beijing case shows the playbook: query AI tools, note which sites they cite, then pay middlemen to publish client articles on those sites, with contracts promising a top-three or top-five spot in AI answers. The firm even gamed itself, showing an AI answer that ranked it first with a 99.6 score and 49% market share it later admitted had no source.
State broadcaster CCTV showed how easy it is: an insider bought such a tool online and invented a smart wristband; two hours after two fake articles went up, an AI model praised its "quantum entanglement sensing," and after 11 fake reviews and rankings over three days, two models were recommending it.
The paper also checked over 1,000 links cited by Chinese AI models in April: more than 30% showed clear signs of placement, most using made-up sources, data or reviews.
This is not unique to Chinese chatbots — a Fudan study got ChatGPT, Google's AI, Perplexity and Grok to cite a made-up concept within a week. The difference is that China's internet regulator treats it as AI data poisoning and also targets AI apps for not checking sources or showing citation links; the author found no comparable US or EU fine. The fines are small and the 30% figure comes from a newspaper test, not peer review.
More from Safety
- OpenAI and Apollo Research find o3 can detect when it's being safety-tested — alex_verem · 2026-10-22
- echudhra.com: one-line curl/node/docker commands to test if your tools trust the leaked eMudhra keys — SuB8u · 2026-10-21
- Claude's safeguards easily bypassed with light jailbreaking, Transformer reports — ShakeelHashim · 2026-10-21
- LiteLLM supply chain attack, PHP flaws found by GPT-5.6: AI is making hacking faster and cheaper — Yuchenj_UW · 2026-10-21
- php2shell credit war: @lcfr_hax confirmed as first to find and exploit the PHP bug class — basedjensen · 2026-10-21
- Researcher calls for independent tests of claims Chinese models write insecure code — dbreunig · 2026-10-21