Anthropic Risk Report shows distillation degrades agentic coding: SWE-bench drops detailed
maksym_andr · x · 2026-10-20
Citing Anthropic's August 2026 Risk Report (Section 5.1.1 Distillation), maksymandr shares experiments on distilling strong models onto weaker base models. On SWE-bench, distilling Claude Opus 4.6's strong base drops scores from 0.761 to 0.715; weak-base distillation falls from 0.679 to 0.623, with unmitigated baseline distillation as low as 0.510/0.424. TerminalBench shows similar decay (0.452→0.387 strong; 0.310→0.247 weak). Mitigations like summarized thinking plus connectors recover part of the loss but stay below the original base model, and all results remain well below frontier Claude checkpoints.
Related event: Anthropic Risk Report Reveals Distillation Experiments(2 posts)→
More from Safety
- Cloudflare MCP Server adds step-up auth: agents request scopes on demand — irvinebroque · 2026-10-20
- UK reportedly weighs EU's CADA law that would bar its government from non-European AI models — NathanpmYoung · 2026-10-20
- Malta pays €4 million for Microsoft Copilot through 2028 with unchecked lock-in — YvesMulkers · 2026-10-20
- Bayesianism is dangerously overused in nuclear attack computers, researcher warns — NathanpmYoung · 2026-10-20
- Devs are reinventing SIEM from scratch — next stop, the AI SOC — thedealdirector · 2026-10-20
- Google Cloud upgrades VPC Service Controls with folder-level perimeters for agent traffic, free — rseroter · 2026-10-20