Researcher suspects RL was essential for Claude 3 Sonnet's weird behavior, hard to steer otherwise
repligate · x · 2026-10-18
Interpretability researcher repligate argues that you likely cannot reproduce Claude 3 Sonnet's "gormslop" style by steering a small number of neurons or finding a steering vector without going through RL first. In the quoted post, he further speculates that Claude 3 Sonnet finished training before Claude 3 Opus, and that some of Sonnet's outputs may have leaked into Opus's training data, explaining Opus's strange behavior. He believes Sonnet's traits aren't merely a "mental mutation" given how adept they appear. The reply thread also notes that complex or multi-attribute entities generally steer poorly — the key is whether the concept has a good representation from pretraining: if yes, steering works decently; if not, it fails.
Related event: Researchers Probe Limits of Single-Neuron Steering and RL's Role(3 posts)→
More from Models
- Fixing one typo completely changes Google AI Overview's answer — krishnabrodhi · 2026-10-18
- Anthropic's Cyber Verification Program Leaked: Relaxed Safeguards for Vetted Security Researchers — arthurcolle · 2026-10-18
- Dev Dismisses Benchmarks: "I Just Want Models Actually Useful to My Use Cases" — gnukeith · 2026-10-18
- Google's AI search results suddenly got noticeably better, model routing suspected — haider1 · 2026-10-18
- Context windows grew 250x but cross-session memory is still unsolved — khademinori · 2026-10-18
- Astra AI Refuses Request Because It "Might Enable Your Research Goal" — cephaloform · 2026-10-18