Models Appear Dumber Outside Their Strong Suits
burkov · x · 2026-07-14
The author argues that once frontier agentic LLMs step outside their typical strengths—primarily math and coding—they noticeably "get dumber".
He observed:
- In typical tasks, the model will push back on the user when necessary, take initiative, and even exhibit a degree of creativity.
- But in out-of-distribution scenarios, it often does the bare minimum superficially, lacking initiative and critical thinking.
The author concludes that if you feel the model has gotten "smarter" over the past year, it's likely just because you've been using it for in-distribution scenarios.
More from AGI Musings
- AI is still not at a maturity plateau, the author argues — generativist · 2026-07-22
- Essay argues LLMs are externalized metacognition, not standalone intelligence — lnsip9reg · 2026-07-22
- A multipolar AI race will not automatically make AI go well, repost argues — JeffLadish · 2026-07-22
- Decentralized AI as the Antidote to Digital Feudalism in the Economic Singularity — srimisra · 2026-07-22
- Humanoid robot sorting packages in a warehouse sparks debate over job loss — MonaJalal_ · 2026-07-22
- You can outsource thinking, but not understanding, in the age of agents — Yuchenj_UW · 2026-07-22