Anthropic Fellows train models to explain their own wild behaviors with generalization to held-out evals
a_karvonen · x · 2026-09-05
Adam Karvonen and collaborators (Anthropic Fellows Program) asked whether models can learn to explain their own behaviors, such as ignoring a user request or making a coding error. Using a CHIVE pipeline that finds unexpected in-the-wild behaviors and probes them with counterfactual prompt edits, they produced thousands of high-quality explanations and trained models on them. Training on this single general dataset generalizes to held-out evaluations.
More from Research
- Richard Socher: AI could compress a century of scientific breakthroughs into a decade — RichardSocher · 2026-10-21
- Startup Electric Plant builds an AI model to decipher plants' electrical signals — TechCrunch AI · 2026-10-21
- Claude reads the literature to build open human surfaceome atlas, claim by claim — owl_posting · 2026-10-20
- Eval researcher teases guide to subjective-task evals, with a curated resource list — cwolferesearch · 2026-10-20
- IIT Madras article splits LLM memory into five approaches and asks why belief revision is still missing — LimeOptimal539 · 2026-10-20
- Scholar writes AI-assisted book on storytelling values, work nominated for EMNLP best paper — _akpiper · 2026-10-20