Nat Lambert: distillation is essentially just getting access to the full CoT
natolambert · x · 2026-10-18
Reacting to the Stolen Thoughts paper discussion, Nat Lambert offers a first-approximation definition: distillation is essentially just getting access to the full chain of thought, and Chinese labs already have teams building extraction methods. His intuition: distillation effectiveness (information gained per sample) correlates with how much reasoning the model discloses to users — and it's predictable: train models with different disclosure levels, measure how many samples you need, and calibrate what you show users accordingly.
Related event: Stolen Thoughts paper sparks debate over distillation and CoT theft(9 posts)→
More from Research
- Kenneth Stanley shares FER hypothesis paper: fractured representations hurt adaptability — kenneth0stanley · 2026-10-19
- Kenneth Stanley questions mechanistic interpretability: why mind-reading but no mind-improving? — kenneth0stanley · 2026-10-19
- Proof Before Reuse: 0/30 poison exposure vs naive agent memory — maa____z · 2026-10-19
- Open 4B decision model cuts agent costs 81-89% and beats the closed model it learned from — alex_verem · 2026-10-19
- Masked diffusion LLM on AMD XDNA2 NPU: 1.67x faster, 8.9x better energy than CPU — PyTorch · 2026-10-19
- Tencent's Terminal-RSI lifts a 27B model from 51.7% to 86.5% on terminal agent tasks — GabGarrett · 2026-10-19