Metis: A Foundation Model that Internalizes Memory Without External RAG
rohanpaul_ai · x · 2026-08-10
While traditional LLMs rely on external retrieval systems (like RAG) for historical context, Metis introduces a memory foundation model architecture. It compresses past interactions directly into a persistent internal state via a separate memory-attention path, while keeping the model weights frozen.
The model is mid-trained to learn native memory procedures such as remembering, updating, forgetting, and reflecting. In the no-context setting, Metis-27B scores 26.74 on the LoCoMo benchmark, significantly outperforming the vanilla backbone (0.07) and Temp-LoRA (4.24).
However, it still falls short of the base model with full context (65.03). The key failure mode is that performance degrades as more information is compressed into the fixed-size state, and irrelevant stored memories can interfere with normal reasoning tasks.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16