Remory: Residual Soft Memory Tokens Hit Full-Context Scores With 5.2% of Inputs
pmttyji · reddit · 2026-10-20
- Fudan researchers propose REMORY, a neural memory network that supplements text summaries with a bounded sequence of soft memory tokens for long-horizon agents.
- Acting like a residual connection along the sequence dimension, the tokens help a frozen LLM approximate what it would output with full history.
- On SummHay it improves source attribution with nearly unchanged insight coverage, approaching full-context joint scores using only 5.2% of input positions.
- Qwen3.8-27B and GLM-5.3-Flash show consistent gains on agent benchmarks, with far fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1. Code and weights are open-sourced.
More from coding & agent
- SharePoint MCP Server gives AI assistants search over company docs via Graph API — modelcontextprotocol · 2026-10-20
- Census MCP connector pipes U.S. Census Bureau housing APIs into AI agents — modelcontextprotocol · 2026-10-20
- Open source isn't obsolete in the agent era, says maintainer changing how he contributes — charles_irl · 2026-10-20
- Dev proposes 'background codemode': strongly typed, inspectable agent workflows — lucasmeijer · 2026-10-20
- Builders ditch LangChain: why more devs roll their own agent harness — raftconsensus · 2026-10-20
- Anonymous audits of public MCP servers find read-only tools that can write and spend — Suncly_com · 2026-10-20