Berkeley paper: LLMs know your preference changed but still use the old one
rohanpaul_ai · x · 2026-10-02
A new Berkeley paper, When Context Changes: Understanding Update Failures in LLMs, studies how LLMs fail at updates within context: after a preference, deadline, or other state changes, the old version stays in the conversation. The model still holds the new value, but its attention keeps drifting back to older mentions, so it acts on stale information.
Key findings: across 5 open models, nudging attention toward the newest value fixed most of these errors without retraining. Even a top-tier model got only 9 of 40 questions right on long agent logs, yet scored 40/40 when the current state was given explicitly.
Practical takeaway: if your agent tracks anything that changes, keep the current state in the prompt instead of making the model dig through history. Paper: arxiv.org/abs/2609.38866
More from coding & agent
- A 5-question interview prompt that turns Claude into a scroll-driven page builder — aziz4ai · 2026-10-02
- Single-prompt workflow: Claude Sonnet + GSAP ScrollTrigger builds scroll-driven glass-shatter page — aziz4ai · 2026-10-02
- After agents write customer-specific code, how do you maintain and deploy it? — Embarrassed-Survey61 · 2026-10-02
- Cloudflare launches Web Search API via AI Gateway with Exa, Linkup and Ceramic — michellechen · 2026-10-02
- 20 tasks × 3 repeats = 120 agent runs: the hidden cost of harness comparisons — RelationshipRound711 · 2026-10-02
- Ant's internal Tiger Agent demos Ling-3.1-flash planning workflows across browser, files and terminal — tinkerbellyie · 2026-10-02