Berkeley paper: LLMs answer with stale values even when the update is in context

ombre-ai · reddit · 2026-11-16

UC Berkeley's Sept 30 paper "When Context Changes: Understanding Update Failures in LLMs" (arXiv 2609.38866) documents stale binding: after a variable is updated in context, models sometimes still answer with the old value (e.g., a meal preference changed from gluten-free to Mediterranean, yet the answer says gluten-free).

Key findings:

The poster asks whether others have seen this in long local runs (agent logs, long roleplay, mid-session config changes), sketches a rough reproduction (several overwritten variables with filler between, then ask for the current value), and asks what mitigations people use: restating state, keeping a state block, or pruning old values.

Original post →

More from coding & agent

coding & agent channel →