Optimizing <think> blocks? Expert warns prohibiting it just forces output optimization
pbaylies · x · 2026-08-19
Commenting on the practice of prohibiting optimization within <think> blocks, the author argues that this inevitably forces optimization onto the output text itself, potentially degrading quality. The author also traces the shifting definition of the term "Neuralese": from originally meaning wordless high-dimensional vectors for latent-space reasoning, to stretching to cover inscrutable text tokens inside <think/> tags, and now being loosely applied to describe weird English usage in apps like Fable.
Related event: Debate Over the Evolving Meaning of 'Neuralese'(4 posts)→
More from AGI Musings
- Interp researcher pushes back: if you can't simulate a neuron, you haven't explained it — nabla_theta · 2026-10-16
- Virtue ethics meets AI alignment: what would it mean to make a model virtuous? — joelbot3000 · 2026-10-16
- Professor: this rec letter season I'm evaluating the student plus Claude — RoozbehMottaghi · 2026-10-16
- Prediction: Riemann Hypothesis likely falls to AI before 2030 — gautamcgoel · 2026-10-16
- Models are superintelligent where data is abundant, and weaker everywhere else — burny_tech · 2026-10-16
- ChatGPT's Dot Reviewed: Your Chief of Staff Is Still Just a Codex Wrapper — infinitely_zero · 2026-10-16