LLM Long Context Test: Sharp IQ Drop After 300K Tokens
teortaxesTex · x · 2026-08-01
Developer tests reveal that a certain LLM becomes significantly 'dumber' after around 300K tokens or 40 turns. This isn't a context handling issue, but a drop in local reasoning capability under long conversations, likely due to insufficient training in this regime.
More from Models
- Claude Dominates Web Dev AI Leaderboard, Kimi K3 Secures Top 3 — arena · 2026-08-01
- Researcher Praises Open Source AI: Gap with Frontier Closing Rapidly — Xianbao_QIAN · 2026-08-01
- Bittensor Subnet Claims Decentralized Training of 120B Model with 5M Context, Yet to Be Verified — markjeffrey · 2026-08-01
- OpenAI Turns Reasoning into a Budget Line: Return on Cognitive Spend — krishnan · 2026-08-01
- OpenAI demos unreleased 'Astra' model to policymakers, source says — CremeSubject7594 · 2026-08-01
- OpenAI reportedly preparing new model family 'Astra' for multi-agent long-horizon tasks — thesaraharminta · 2026-08-01