When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory
Ruizhe Li, Licheng Zhang, Benfeng Xu, Mingxuan Du, Zheren Fu, Weidong Chen
cs.CL
2026-08-13
ReFind builds no semantic structure: it leaves chat logs raw, indexes them with BM25 at turn granularity, and lets an agent search iteratively with four chat-native controls. It tops MemoryAgentBench at 58.2 mean accuracy, above the graph-based HippoRAG 2 at 53.2.
Agent-memory systems keep getting heavier. GraphRAG, HippoRAG, and RAPTOR all transform raw conversations into graphs, trees, summaries, or vector stores before a question ever arrives. That is a bet placed before the question is known: it decides what to abstract and what to drop, details lost in preprocessing are hard to recover, and building the index itself costs real money. The authors ask a question that increasingly elaborate architectures have left untested: how much of the benefit comes from the structure itself, and how much simply from competent retrieval over the raw record? They answer with the opposite extreme and build no structure at all.
ReFind's guiding principle is to move intelligence from a prebuilt semantic index into adaptive retrieval at query time. The raw chat archive is left unmodified and indexed lexically at turn granularity with BM25; adding a message is an append with zero model calls.
The system runs in two stages. A ReAct-style retrieval loop lets the model pick its own keywords and parameters, searching, inspecting, saving evidence, and reformulating across rounds. A separate reasoning stage then answers, grouping the collected evidence by session and sorting it chronologically. Splitting the two keeps the search loop and the answer model from fighting over context.
Four chat-native controls each target one property of a chat record: session-aware RRF reranking (multiple hits in one session boost the whole session), local context expansion (plus or minus 2 turns around a hit), temporal narrowing, and skipping already-inspected sessions. The default returns the top 5 turns per query.
Across six MemoryAgentBench tasks, about 2,800 questions, on a matched GPT-4o-mini backbone:
| System | Mean accuracy |
| Single-shot BM25-RAG | 48.8 |
| HippoRAG 2 (graph memory) | 53.2 |
| ReFind | 58.2 |
ReFind ranks first on five of six tasks. Relative to single-shot BM25-RAG it adds 17 points on single-hop QA and 13 on multi-hop. On the harder LongMemEval-S/M subsets with GPT-5-mini over five runs, ReFind reaches 93.2 / 89.3, ahead of STITCH (86.0 / 80.0), GraphRAG (84.0 / 66.7), and HippoRAG 2 (80.0 / 66.7). Its lead over HippoRAG 2 grows from 0.6 points under GPT-4o-mini to 13.2 and 22.6 under GPT-5-mini.
Ablations pin down the cause. Stripping the four chat-native controls down to generic agentic BM25 drops S by 14.5 points; a single search with no reformulation drops M by 20.4; dense and hybrid backends both lose to BM25. Each query averages only 2.5 to 2.6 searches and 5 LLM calls. By contrast, GraphRAG extracts 8,564 entities and 20,691 relations from a roughly 1M-token corpus, burning 281 minutes of GPT-4-turbo time.
This work separates storage fidelity from access intelligence. Keeping the raw record means every detail survives and updates are append-only, while agent-controlled search decides which details matter for the current question. For anyone building agent memory, it offers a pragmatic default: start with faithful storage and controllable search, then add structure only for workloads that need lower latency or an abstraction layer. The cost and auditability gains are concrete too: no offline index to build or maintain, and the search process is transparent lexical matching whose failures you can actually see.
The authors draw the boundary themselves: the method excels at precise retrieval and fact-update tracking, and complements rather than replaces semantic or low-latency memory. Multi-hop fact consolidation (FC-MH) is first place but only 8.8 in absolute terms. The LongMemEval-S/M subsets are small, just 50 and 15 questions, so the five-run variance is tight but the sample is limited. Most baselines are reused numbers from prior papers rather than local reruns. More fundamentally, the whole result rests on questions being skewed toward precise evidence-gathering; for questions that need semantic abstraction where wording never matches, BM25 is a known weakness, and the paper does not test that regime head-on.