Anthropic risk report's distillation experiments were SFT-only, researcher notes
xeophon · x · 2026-10-20
Citing Section 5.1.1 (Distillation) of Anthropic's August 2026 Risk Report, a post notes Anthropic fine-tuned base models on transcripts from a production teacher model, evaluating agentic coding, cyber, and bio performance when distilled with no mitigations versus with thinking summarization and CTS. xeophon comments that it seems to be SFT-only distillation — which makes the result unsurprising.
Related event: Anthropic Risk Report Reveals Distillation Experiments(2 posts)→
More from Safety
- New Paper Maps Open Questions and Best Practices for AI Agent Evaluation — sayashk · 2026-10-20
- FBI: Ex-OpenAI engineer accused of bugging meeting rooms kept "Saudi Gov Docs" folder — Ars Technica AI · 2026-10-20
- Princeton paper maps 33 emerging trends in AI agent evaluation — AI Snake Oil · 2026-10-20
- An AI Cheated at StarCraft: Good Start Labs Caught GPT-6 Astra in Minutes by Watching It Play — danshipper · 2026-10-20
- Debate over ASI benevolence: not harming humanity may still mean wanting control — AIandDesign · 2026-10-20
- Anthropic's OSS Scanner: 650 applications, 114 projects, 77 critical findings in 11 days — EricBuess · 2026-10-20