CMU's Conitzer shows Google's frontier guardrails are still trivially bypassed
conitzer · x · 2026-09-18
Vincent Conitzer (CMU professor) argues that guardrails in today's frontier AI systems remain very brittle, with little fundamental progress.
- Using a prompt with a made-up symptom combination (itchy neck, red spots on feet), he easily circumvented Google's medical-advice guardrails; similar prompts yield far worse outputs, so he withheld the exact prompt.
- He stresses this is Google—a notably security-conscious company—the point being that nobody currently knows how to keep these systems safe. Recent work has even trained models to jailbreak themselves (arXiv paper linked).
- He also argues lessons from older, weaker models often transfer to frontier ones—his prompt was based on such lessons—and that the prevailing view that studying non-frontier models is a waste of time is mistaken; there just isn't enough time allocated to it.
Related event: CMU Professor Shows Google Model Medical Guardrails Easily Bypassed(2 posts)→
More from Safety
- Cryptographers push back: claims ECC is likely to break are not consensus — jedisct1 · 2026-11-02
- UK's 'AI sovereignty' push undermined by its own energy and copyright policy — NinaDSchick · 2026-11-02
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- Cambricon co-founder urges shared US-China AI ethics standards for leading models — pstAsiatech · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- 6 prompts to see what ChatGPT quietly knows about you — and wipe it — Aiden_Tech_Ai · 2026-11-02