Interp paper replication breaks after Claude Sonnet 4 changes, researcher laments

Sauers_ · x · 2026-11-18

A researcher trying to replicate an interpretability paper hit a wall because the original work used Claude Sonnet 4 as an LLM judge — model deprecations and updates now make paper results hard to reproduce. A fresh wrinkle in AI research reproducibility: papers that embed specific model versions as components break when vendors retire them.

Original post →

More from Models

Models channel →