After two weeks in China, author argues Western AI safety playbook doesn't transfer — and catching up is getting costlier
JacquesThibs · x · 2026-11-22
Jacques Thibodeau published a 27-min LessWrong essay based on two weeks of conversations in China about AI safety:
- Ecosystem mismatch: The West built its field via philanthropy and decades of safety foundations; China lacks these initial conditions, foreign nonprofit money is legally restricted, and safety work will come mainly from state-backed institutes, universities, companies and volunteers. Concordia AI, long assumed a nonprofit, is actually a social enterprise.
- Academic exchange works: The main barrier for Chinese researchers joining programs like MATS is that it looks like a career break, not recognition issues alone.
- Hawkish messaging backfires: Confrontational Western rhetoric makes AI warnings look like containment excuses, hindering safety work in China.
- Catching up is getting harder (from the attached image): industry insiders cite anti-distillation measures, API restrictions, compute constraints, and harder-to-copy data (computer use, spatial understanding, video generation). Since Panfilov et al. showed reasoning traces can be stolen from proprietary APIs, frontier labs protect models more aggressively; Chinese firms still distill (problem splitting, reverse rationalization) but with worse results.
- Numbers: Anthropic's September threat intel report disrupted distillation attacks from seven Chinese labs (incl. DeepSeek, Alibaba, Moonshot, Zhipu), attributing 3.4M+ exchanges in 17 days to Zhipu via a 273-account CoT pipeline; one lab reportedly spends $300k/day on teacher tokens ($100M/year).
More from AGI Musings
- DeepMind researcher: watching token cannons smash castles that withstood human sieges is poignant — SamuelAlbanie · 2026-11-22
- AI vs. Millennium Problems: the real story starts after the demo — ayushthakur0 · 2026-11-22
- The 2029 meme: ASI takeoff, cancer cured, yet GDP still grows 2.8% — willcb · 2026-11-22
- Auki founder at Humanoid Hub: physical AI should run physical businesses, not just move robots — broodsugar · 2026-11-22
- "Lean is the worst thing that happened to mathematics" sparks pushback from mathematicians — GiorgioPatrini · 2026-11-22
- Eric Schmidt: a mutual AI race pause is impossible because it can't be verified — rohanpaul_ai · 2026-11-22