Mollick points to ICLR paper showing small accuracy gains compound exponentially over long-horizon LLM tasks
emollick · x · 2026-10-16
Ethan Mollick highlights an ICLR 2026 paper (Cambridge, Stuttgart, Max Planck et al.) explaining his skepticism that frontier models can easily be replaced by cheaper ones.
- The paper, "The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs," shows marginal single-step accuracy gains compound into exponential improvements in task length a model can complete.
- Most failures come from execution, not reasoning; per-step accuracy degrades with more steps, partly due to a "self-conditioning" effect — models err more when context contains their own earlier mistakes — and scaling alone doesn't fix it. Thinking mitigates it and enables longer single-turn execution.
Mollick's takeaway: cheaper small models save money short-term but produce worse end results; they still do valuable work, but for many tasks you want the best models orchestrating and error-checking them.
Related event: Mollick: Small Accuracy Gains Compound in Long AI Tasks(2 posts)→
More from Research
- SERO augments long-horizon agentic RL with executable oracle continuations — Shanghai-AI-Laboratory · 2026-10-16
- Tsinghua and Amap propose OmniVerifier-C1, a graded verifier beating binary judges — Bowei Liu · 2026-10-16
- Alibaba open-sources MAI-Gym, an all-in-one GUI agent eval and RL training platform — Tongyi-MAI · 2026-10-16
- Peking University's SimForcing distills simulation motion priors into robot world models — PekingUniversity · 2026-10-16
- CoWorld Reasoner uses an evolving video world model as a visual reasoning engine — ModalityDance · 2026-10-16
- Sparse rank-1 maps are often interpretable: a finding said to jump your neural net understanding 5 years ahead — gleech · 2026-10-16