CHIVE paper: evaluating LLM behavior explanations in the wild with counterfactual experiments
a_karvonen · x · 2026-09-05
Paper (arXiv:2608.16747) and blog post released by Adam Karvonen, Euan Ong, Subhash Kantamneni, and Samuel Marks, done as part of the Anthropic Fellows Program.
- Problem: interpretability and CoT-faithfulness research needs a way to judge what makes a good explanation of model behavior.
- Method: CHIVE (Counterfactual Hypothesis Investigation Via Edits), an agentic pipeline that finds unexpected in-the-wild model behaviors and investigates them via counterfactual prompt edits, yielding thousands of high-quality explanations with supporting evidence.
- Finding 1: judged by counterfactual simulatability, none of the common interpretability techniques studied improved an agent's ability to predict counterfactual model behaviors.
- Finding 2: training on CHIVE-generated counterfactual data generalizes to various out-of-distribution settings.
More from Research
- Stanford team launches ScholarCatalyst benchmark for inspiration retrieval — lateinteraction · 2026-10-21
- Holon-S1 predicts gene activity and biomarkers from routine pathology slides — altryne · 2026-10-21
- Three reasons AI is not conscious and may never be, per Nature essay — anilkseth · 2026-10-21
- Aviation tracks every part's provenance — agents can reason over that graph — colinmcnamara · 2026-10-21
- SFT details: instruction/reasoning/code/math mix, synthetic + human traces, no epoching — TimDarcet · 2026-10-21
- Mid-training recipe disclosed: 32k→131k context extension, dropping logit distillation for tokens — TimDarcet · 2026-10-21