Elie Bakouch: classic SFT distillation should matter less without logits access
eliebakouch · x · 2026-10-18
Responding to Dean Ball's 50% distillation estimate, Elie Bakouch says he'd have guessed the classic approach — SFT on OpenAI/Anthropic traces — plays a much smaller role, since labs theoretically lack access to logits or reasoning traces; other forms of "distillation" are much harder to quantify, in his view.
More from AGI Musings
- Why 'taste' is the white whale of AI coding: LLMs can read design systems but can't extrapolate them — threepointone · 2026-10-18
- WIRED: AI agents cause chaos that AI companies aren't paying for — emmanuelvivier · 2026-10-18
- Peter Diamandis: AI will create wealth — the real question is whether young workers get a share — PeterDiamandis · 2026-10-18
- CISA, NSA and Commerce issue most detailed federal guidance on AI sandbox security — emmanuelvivier · 2026-10-18
- You have to do the work yourself: a Roman parable for the AI age — alfred_lua · 2026-10-18
- Fields Medalist Gowers warns AI could erode future mathematicians' motivation — IgorCarron · 2026-10-18