Engram: RL Trains Models to Use Their Notes, Boosting Accuracy at Low Cost

Research company Engram published a new blog post on November 5 examining how reinforcement learning (RL) and text-based memory (notes/skill files) complement each other in continual learning. The core finding: RL training doesn't make models internalize new knowledge—it teaches them how to use their own notes. This training is far cheaper than building the notes themselves, and it stacks with harness memory.

Confirmed

• Core idea: RL complements text-based memory, analogous to a student rereading discrete math notes until they truly understand—training teaches the model to leverage existing notes rather than rewriting knowledge.
• Giving the model fixed high-quality notes, optimizing the prompt, and then applying RL training improved cross-benchmark accuracy by 7-20 percentage points, beating both the untrained version and frontier models given the same notes.
• Training (taking gradients) was surprisingly cheap, far below the cost of building the notes themselves.
• Model performance remained robust even after memory edits spanning millions of tokens.
• The strongest systems combine all forms of memory—training and harness memory stack on top of each other.

Why It Matters

• @realJessyLin pointed out this means memory capabilities can be enhanced on base models through cheap training, offering a practical capability boost for long-horizon Agents.
• @scottlinderman relayed the team's observation: notes taken by models running for long stretches become increasingly inscrutable, and a small amount of training fixes this; the team is optimistic about "models understanding data better than humans" and teased more updates to come.
• This work offers a new paradigm for continual learning: instead of compressing knowledge into parameters, train models to make good use of external memory—balancing cost and robustness.

2026-11-05 ~ 2026-11-05 · 6 related posts

• [source] RL-taught memory: training models to use their notes is cheap and robust — realJessyLin · 2026-11-05
• Engram: brief training fixes long-run models' increasingly inscrutable notes — scott_linderman · 2026-11-05
• Engram: RL Teaches LLMs to Use Notes, Cheap to Train and Robust Across Millions of Token Edits — scott_linderman · 2026-11-05
• Engram blog: RL training complements text memory, stays robust through millions of token edits — canaesseth · 2026-11-05
• Engram shows RL-trained note usage beats text-only agent memory on cost and robustness — scott_linderman · 2026-11-05

1 near-duplicate retellings: sarahcat21