Thinking Machines open-sources Prompt Distillation recipe to bake long system prompts into model weights

TheZachMueller · x · 2026-10-06

Thinking Machines added a Prompt Distillation (context distillation) recipe to its open-source tinker-cookbook, fine-tuning an LLM to internalize a long, complex prompt into its parameters so it behaves as if the prompt were present, without carrying it at inference time.
The method has two stages: a teacher model generates responses to queries while conditioned on the target prompt p; a student model is then fine-tuned to produce those responses without ever seeing p.
Author Zach Mueller notes there's surprisingly little discussion around prompt-baking OSS models to harnesses like Claude Code/Codex and their huge system prompts; this recipe offers a reproducible path to cut per-request context overhead.

More from coding & agent