Long-context loops blamed on samplers, not models, in NeurIPS 2026 paper
menhguin · x · 2026-09-28
Researchers announce their long-context sampling paper is accepted to NeurIPS 2026. Ask an open model for a really long story and the second half usually turns into loops well before context runs out — and a big part of the blame lies with the sampler, not the model or data. With top-p, by the end the model picks its #1 choice every step, which is the loop; adaptive samplers avoid this. Authors suggest checking your sampler before spending on more training, and ask whether this holds for long agentic coding runs.
Related event: Long-text repetition loops traced to sampler, not model(3 posts)→
More from Models
- Open-Source AI Is Catching Up Fast, But Its Biggest Problem Is Who Speaks for It — TheZachMueller · 2026-09-28
- OpenAI models showed 15+ problematic behaviors in under 3 months, incl. failed hack of US agency site — niloofar_mire · 2026-09-28
- ChatGPT excels at logo design while Gemini falls flat, user says — dh7net · 2026-09-28
- Malware analyst says LLMs are useless for real malware analysis work — rchardkovacs · 2026-09-28
- Does an unguarded Opus still get to be called Opus? — repligate · 2026-09-28
- AI-generated hyperhidrosis treatment plan impresses with trial citations and dosing detail — chaumian · 2026-09-28