Why do Sonnet 5 and Opus 5 feel worse than GLM 5.3? Distillation's limits, dissected
baseten · x · 2026-09-12
Dwarkesh Patel highlights a podcast segment where John, Beren, and Charlie speculate why Sonnet 5 and Opus 5 feel like worse models than GLM 5.3 — even though Anthropic could do raw logit distillation from Fable and train on the same environments. The discussion covers the value of distillation, what effective distillation takes, and which model behaviors resist being extracted via distillation.
More from Models
- ToneBench eval: GPT-6.1 Terra ranks #23 at writing — cheaper, not better, than GPT-5.6 Terra — OnlyProggingForFun · 2026-10-28
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28