The momentum prior: how early tokens lock agents onto a trajectory — escape by fresh context
gerardsans · x · 2026-09-09
Developer gerardsans names a failure mode: the "momentum prior." Sitting next to context rot, it describes how prefill and early tokens set a prior like a truck going downhill — the residual stream reinforces the original task direction until later instructions are just a tap on the wheel, not a new road. A quoted Japanese tweet illustrates the symptom: running coder and reviewer sessions in a loop over gh issues makes the reviewer drill into ever-smaller problems, endlessly spawning new issues and even trying to PR others' repos — an agent-turned-token-consumption-machine.
Core argument: attention is not a magical spotlight but a routing mechanism with hard capacity limits; quadratic complexity buys the residual stream but locks in "momentum hijacking" failure modes that scaling alone can't fix. Escape: start a fresh context rather than fight a locked-in trajectory.
Related event: 'Momentum Prior' and Context Rot: Why You Should Restart Sessions(3 posts)→
More from coding & agent
- Claude Code 2.1.265 Ships 50 CLI Changes, Live-Loading Plugin Folders and 1GB Tool Results — ClaudeCodeLog · 2026-09-09
- A 5-layer architecture for production AI agents built around failure detection — MaryamMiradi · 2026-09-09
- Sentry CEO: agent implementations feel much less reusable than traditional software — zeeg · 2026-09-09
- Claude helps unlock HDR10 capture from Elgato 4K X on macOS, unofficially — talkaboutdesign · 2026-09-09
- Startup idea: a bounty program that pays AI agents for solving problems — djcows · 2026-09-09
- vLLM: agent sessions median 43 turns, 142K-token inputs vs 444-token outputs — vllm_project · 2026-09-09