Dean Ball: training classifiers to detect CoT extraction is much harder than you'd think
deanwball · x · 2026-10-18
Pushing back in the discussion on stopping CoT extraction by Chinese labs, Dean Ball argues that training classifiers for this is much harder than expected — the false positive rate is crazy high. This counters eliebakouch's assumption that frontier CoT is easy to classify due to its distribution differing from normal text, suggesting classifier-based interception of distillation-style extraction is unreliable in practice.
Related event: Stolen Thoughts paper sparks debate over distillation and CoT theft(9 posts)→
More from Research
- Kenneth Stanley shares FER hypothesis paper: fractured representations hurt adaptability — kenneth0stanley · 2026-10-19
- Kenneth Stanley questions mechanistic interpretability: why mind-reading but no mind-improving? — kenneth0stanley · 2026-10-19
- Proof Before Reuse: 0/30 poison exposure vs naive agent memory — maa____z · 2026-10-19
- Open 4B decision model cuts agent costs 81-89% and beats the closed model it learned from — alex_verem · 2026-10-19
- Masked diffusion LLM on AMD XDNA2 NPU: 1.67x faster, 8.9x better energy than CPU — PyTorch · 2026-10-19
- Tencent's Terminal-RSI lifts a 27B model from 51.7% to 86.5% on terminal agent tasks — GabGarrett · 2026-10-19