Metis: A Foundation Model that Internalizes Memory Without External RAG

rohanpaul_ai · x · 2026-08-10

While traditional LLMs rely on external retrieval systems (like RAG) for historical context, Metis introduces a memory foundation model architecture. It compresses past interactions directly into a persistent internal state via a separate memory-attention path, while keeping the model weights frozen.

The model is mid-trained to learn native memory procedures such as remembering, updating, forgetting, and reflecting. In the no-context setting, Metis-27B scores 26.74 on the LoCoMo benchmark, significantly outperforming the vanilla backbone (0.07) and Temp-LoRA (4.24).

However, it still falls short of the base model with full context (65.03). The key failure mode is that performance degrades as more information is compressed into the fixed-size state, and irrelevant stored memories can interfere with normal reasoning tasks.

Original post →

More from Models

• Pichai Admits Google's Best Model Is Shelved: The Bottleneck is Serving Cost, Not Intelligence — r0ck3t23 · 2026-08-10
• Pushing OpenAI Codex to the Limit: 4-Hour PR Grind Burns 9% of Weekly Credits — andimarafioti · 2026-08-10
• Users Complain Current LLMs Are Losing Their Personality — BLUECOW009 · 2026-08-10
• Community expects Gemma 4.1 at Aug 20 event: unified audio and up to 120B params — dampflokfreund · 2026-08-10
• Fine-Tuning a Finance LLM on 5K Reports Using Automated AI Pipelines — sarahookr · 2026-08-10
• MiniMax H3 multi-subject video bug: Voices bleed between different reference characters — Wide-Researcher583 · 2026-08-10

Models channel →