Over-RL'ed models lose creativity, giving near-identical answers even at high temperature
bendee983 · x · 2026-10-13
Based on hands-on experience, the author observes that models post-trained too hard with RL lose creativity: give the same vague prompt multiple times (e.g. "tell me a story" or a joke) and you get very similar answers, even with higher temperature. Base models, in contrast, produce much more diverse outputs.
His practical advice: when working with a heavily post-trained reasoning model, provide as much detail and specificity as needed to express your intent — otherwise the model will lean toward its default trajectory.
More from Models
- Nature Will launches Natura-1, a foundation model for the physical world trained on 1M hours of egocentric video — testingcatalog · 2026-10-13
- User speculates GPT-6.1 Sol's generous limits come from deliberately throttled speed — AccomplishedSugar490 · 2026-10-13
- ChatGPT may have the smallest context window among major chat models, test finds — JeremyNguyenPhD · 2026-10-13
- Patronus tests show prompt-injection detection and action approval are two different tasks — PatronusProtect · 2026-10-13
- DeepMind open-sources 'cryptids': Turing machines with Lean proofs of famous conjectures — gaganghotra_ · 2026-10-13
- Hybrid fine-tuned BERT + LLM pipeline lost its edge to new frontier models — rajistics · 2026-10-13