Researchers Question the Effectiveness of ReAct Agents
gerardsans · x · 2026-07-17
This post tackles a core argument: what you're facing isn't an agent capable of true "continuous learning," but rather something closer to a frozen state machine or manifold.
It refers to ReAct-style agent frameworks—combining LLMs, tools, environmental observation, and iterative loops—as the infrastructure bridging large models and digital labor. However, two independent 2026 studies challenge this narrative. One of them, CORRAL, involved over 25,000 trials across 8 scientific domains comparing performance with and without harnesses. The conclusions were highly pessimistic: models frequently ignored evidence, barely updated their beliefs, and responded poorly to contradictions.
The overarching thesis is that while many current agent frameworks might appear to "think, act, and observe," they remain unreliable in genuine evidence integration and belief revision.
More from AGI Musings
- Gary Marcus says LLMs still cannot really do math on their own — GaryMarcus · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- AI may make digital work infinitely leveraged while offline life gets more human — illscience · 2026-07-22
- Better AI math could save researchers time by killing false conjectures earlier — prateekj · 2026-07-22
- AI’s economic forecasts are split by nearly a quadrillion dollars by 2035 — bittingthembits · 2026-07-22
- Open source is becoming tech’s soft power, says Kevin Xu — kevinsxu · 2026-07-22