Post-training compute shift makes distillation of frontier models nearly impossible to prevent

maksym_andr · x · 2026-09-15

The shift of compute from pretraining to posttraining explains why the gap between open-weight and proprietary models has shrunk: post-training improvements are much easier to distill.

Key argument:

Access to a very capable model in some domain can be used to create RL environments in that domain, enabling distillation
This is hard to detect from the outside: generating an environment looks like legitimate use
Only a large volume of environment-related requests might give it away — and only post-hoc, after too many environments already exist
Dedicated organizations can trivially create many accounts to evade detection

Conclusion: preventing distillation in full generality seems nearly impossible.

Original post →

Post-training compute shift makes distillation of frontier models nearly impossible to prevent

More from Models