UCLA Introduces LongMemEval-V2: Benchmarking Agent Environmental Memory
dair_ai · x · 2026-08-27
Most agent memory benchmarks test user history recall, but production agents need to internalize environmental characteristics like interface quirks and state dynamics. A new UCLA paper introduces LongMemEval-V2, a benchmark with 451 manually curated questions covering five web agent memory abilities: static state recall, dynamic state tracking, workflow knowledge, environment gotchas, and premise awareness. History trajectories scale up to 500 traces and 115M tokens per question.
The proposed AgentRunbook-C method stores trajectories as files, allowing a coding agent to gather evidence in a sandbox. It achieves an average accuracy of 72.5%, significantly outperforming the strongest RAG baseline (48.5%) and an off-the-shelf coding agent (69.3%).
More from coding & agent
- Blind Agent Struggles with Iframes; Human-in-the-Loop Saves the Day — burkov · 2026-08-27
- GPT-5.6 Builds New Kernel, Achieving 9.7x Speedup on TPU — HuaxiuYaoML · 2026-08-27
- Explaining the Difference Between WebMCP and Regular MCP — VeryWellVersed · 2026-08-27
- Bronx high schoolers host AI dev competition to support local businesses — ziv_ravid · 2026-08-27
- Qwen3.8-Flash-Next generates 112k tokens locally on RTX 3090 — Ok_Ninja7526 · 2026-08-27
- Automating parsing of Indian subcontinental language books with Claude and Codex — aryaman2020 · 2026-08-27