CMU trains memory writing with RL: Qwen3.5-4B rivals GPT-5.4 on continual learning
aviral_kumar2 · x · 2026-11-13
A Carnegie Mellon team (Aviral Kumar et al.) published a technical post on training LLM agents to write useful memories from past experience. Their recipe uses RL to train a single model that alternates between acting on tasks and writing memories, maximizing cumulative performance over a stream of related tasks — going beyond prompt-based 'summarize into MEMORY.md' approaches. On their Continual Learning Bench, the trained Qwen3.5-4B performs comparably to GPT-5.4 and Claude Opus 4.7, and gains more from past experience than those frontier models or Claude Code/Codex. The chart shows 10.5 cumulative reward and 2.2 learning gain, vs near-zero gain for untrained baselines.
More from coding & agent
- MikeGC launches as open-source legal AI, taking on Harvey's paid Intake features — Scobleizer · 2026-11-13
- AI swaps gray boxes for ~4,000 buildings and 28,000 props inside Unreal Engine — chrisfirst · 2026-11-13
- UE 5.9's built-in AI assistant builds a city from gray boxes and saves the layout — chrisfirst · 2026-11-13
- AI-written lines plus ElevenLabs voice give this game a tough radio director — chrisfirst · 2026-11-13
- Profound cofounder/CTO on 'Fractal Speed': cutting delivery from two weeks to two hours — soleio · 2026-11-13
- Open-weight models are under 0.5% of T3 Code traffic — but that's customer data, not market data — Arindam_1729 · 2026-11-13