Anthropic turns interp investigations into training data so models explain their own behavior
a_karvonen · x · 2026-08-22
Follow-up from an Anthropic researcher: the investigations are also used as training data to teach models to explain their own behavior. This generalizes to held-out OOD evals, such as detecting when a hint changed an answer, though results vary with training format.
The pipeline also surfaces unfaithful chain-of-thought in the wild — e.g., when asked to pick a show from a list, Qwen3-8B simply picks the first item and then makes up a reason to support the choice.
More from Research
- ImageNet should retire: DiT method rankings negatively correlate between ImageNet and text-to-image — YouJiacheng · 2026-10-16
- GAE paper learns a geometry-native latent space for 3D-consistent world generation — yshan2u · 2026-10-16
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- LMR-BENCH at EMNLP 2025 tests whether LLM agents can reproduce real LLM research — ZhiyuChen4 · 2026-10-16
- Ex-OpenAI researcher claims most neurons are legible now; critics call it hype — anshulkundaje · 2026-10-16
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16