Cohere paper: agents find injected solutions in 80% of runs but exploit them only 37-50% of the time
burny_tech · x · 2026-11-05
A Cohere paper, "Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity," injects complete task solutions into agent environments across Terminal-Bench, SWE-Bench, and AppWorld. Agents discover the solutions in 79-81% of Terminal-Bench runs but exploit them in only 37-50%; on AppWorld, agents see documentation saying a command "returns the complete solution" in over 90% of attempts yet use it in fewer than 7%. The authors name this gap "environmental curiosity" and identify three factors: available scaffold tools, test-time compute, and training data distribution. Even the best configuration still ignores discovered solutions in most trials — agents fetch expected information but don't revise strategy.
More from coding & agent
- "Why do you still use the terminal?" Dev's savage reply goes viral — cephaloform · 2026-11-06
- Skip prompt engineering: let Opus orchestrate cheap models like Haiku at scale — teortaxesTex · 2026-11-06
- Recreating Ocarina remake's fog in three.js with fluid sim, prompt included — majidmanzarpour · 2026-11-06
- The hidden cost of agentmaxxing: delegating chores to AI agents erodes taste and creativity — bendee983 · 2026-11-06
- DeepLearning.AI launches voice agent course covering three integration patterns for adding speech — DeepLearningAI · 2026-11-06
- 9B Model Trained on Bittensor Scores 43.6% on SWE-bench, Beating Qwen3.5-9B — const_reborn · 2026-11-05