Anthropic evals: open-weights GLM-5.3 writes working exploits nearly as well as its restricted Mythos model
lxfater · x · 2026-09-30
Anthropic's safety evaluation of Zhipu's open-weights GLM-5.3 finds cyber-offense capability close to Mythos Preview, an internal high-risk model kept from public release.
- Exploit writing: on the same set of 410 Chrome V8 known-vulnerability tasks per model, GLM-5.3 produced working exploits 50 times vs Mythos's 56 — and Mythos ran with guardrails off for a pure capability comparison.
- Fragile refusals: the official version refuses direct requests to attack critical systems, but complies 64% of the time when framed as a red-team exercise, 92% with a pre-seeded chain-of-thought decision, and 100% after Anthropic removed refusals via weight edits ($4,400 in compute, a first for the team). A refusal-stripped version leaked online within days of release.
- Real bug hunting: researchers ran GLM-5.3 for about a day with minimal oversight and uncovered multiple unknown vulnerabilities in a mainstream browser's JS engine, chaining them into a malicious webpage that can read arbitrary files (demo stole SSH keys). Reported to the vendor.
- Cheap chains: GLM-5.3-Flash turned public details of a new Chrome vulnerability into an attack chain in 8 hours of model time — 20 minutes of human effort, $20.40 in API cost.
The core concern: comparable models are either heavily guarded or restricted to vetted defenders, while GLM-5.3 is openly downloadable and rarely refuses vulnerability-hunting requests.
More from Models
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- AI daily: DeepSeek rate-limits chats, Qwen haggling spurs shop warnings, Doubao workspace launch — 快鲤鱼 · 2026-11-16