Researcher: Early GPT-4's Exposed Token Probabilities Made Distillation Cheap

Matt Carrig notes that early GPT-4 exposed top-5 token probabilities with no hidden CoT, meaning the full model could likely have been distilled for around $10 million — a stark contrast to today's tight anti-distillation defenses.

2026-10-23 ~ 2026-10-23 · 2 related posts

• GPT-4 once gave top-5 token probs with no hidden CoT — researcher says it could've been cloned for under $10M — carrigmat · 2026-10-23
• GPT-4 once exposed top-5 token probs with no CoT—you could have distilled it for under $10M — carrigmat · 2026-10-23