Bigger Context Windows Solve Fitting, Not Memory: Chroma's Context Rot Shows Accuracy Drops Early

Efficient_Joke3384 · reddit · 2026-09-08

Context windows and memory are two separate investments

Windows now reach 1M tokens (Llama 4 Scout at 10M, Magic at 100M), but the assumption that memory stops mattering doesn't hold.
A big window solves only the fitting problem; three issues remain: sessions still reset to zero, accuracy degrades as context grows — Chroma's context rot study across 18 models found performance dropping well before the limit, with even a single irrelevant distractor measurably hurting accuracy — and you pay to reprocess the same context every time.
The author argues memory must improve independently: recency handling and confidence signals on retrieved memories, rather than waiting for windows to grow.

More from coding & agent