Optimizing <think> Blocks: Hypothesis on Activation vs. Output Text
CFGeek · x · 2026-08-19
CFGeek replied to a hypothesis about prohibiting optimization on <think> output blocks during model training. The counter-argument claimed that prohibiting optimization on <think> blocks guarantees optimization on the output text. CFGeek found it an interesting hypothesis to test but also suggested an alternative possibility: not optimizing <think> outputs might instead mean the activations get optimized without changing the output text in that specific way.
Related event: Debate Over the Evolving Meaning of 'Neuralese'(4 posts)→
More from Research
- Interp researcher pushes back: if you can't simulate a neuron, you haven't explained it — nabla_theta · 2026-10-16
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16
- Four-page solution to subcubic APSP builds intuition for Anthropic-AI-aided breakthrough — gautamcgoel · 2026-10-16
- ATAS: Zhejiang-led team builds world-state memory framework for consistent agentic video generation — CSProfKGD · 2026-10-16
- Hermes: a 4B model with 8K context got worse with more test-time compute, pointing to contextual reasoning — mangahomanga · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16