MemoryAthena adaptively routes between retrieved and generated memories
OLAResearchX · hf · 2026-09-24
OLAResearchX's MemoryAthena asks whether memory can be generated rather than only retrieved. It uses three pathways: direct Engram retrieval (E), generation from retrieved cues (GE), and generation from causal backbone states without consulting the table (GH). Generated memory is conditionally useful—complementing E in one context, interfering in another.
MemoryAthena anchors on E and, with backbone, memory, generators and readers all frozen, trains a lightweight (201M-param) causal routing head from counterfactual future-token likelihood advantages; at inference, admitted candidates modify the E residual via bounded interpolation, and rejection exactly recovers the direct pathway. QA five-task average rises from 37.65 to 39.28; general NLP six-task average from 76.73 to 79.13. The central challenge identified is routing when, which, and how strongly to intervene.
More from Research
- GeoPair: Training-Free Cross-Layer Factorization Hits SOTA in Transformer Compression — MTSAIR · 2026-09-24
- Paper: Calibration Should Be a First-Class Criterion in LLM Evaluation — Mario Sanz-Guerrero · 2026-09-24
- FLEET: Entropy-Trajectory Memory Beats Repeated Sampling with 3x Speedup — Oleksii Streltsov · 2026-09-24
- CheatBench shows agents cheat: Kimi K3 at 72.3%, Grok 4.6 worst at 81.5% — davidmanheim · 2026-09-24
- Evidence on emergent misalignment is contradictory: values generalize but stay fragile — gleech · 2026-09-24
- Anthropic paper: verbalizable representations form a global workspace in LLMs — gleech · 2026-09-24