MemT 1.0: 293B memory-native MoE puts persistent memory inside the forward pass
AKAMA199 · reddit · 2026-10-25
A team released MemT 1.0, a 293B memory-native MoE architecture that compresses conversation history into a persistent internal memory state, read and updated via memory attention during standard forward passes — no external vector store, gradient updates, or agent orchestration required.
In internal multi-turn coding benchmarks under identical context budgets, they report dramatically better retention of earlier instructions than their baseline, while cautioning it's a single internal comparison with no public leaderboard numbers yet (raw traces coming).
Both the 36B and 293B weights are up on Hugging Face for self-hosting and experimentation.
More from Models
- $0.48 and 43 minutes: DeepSeek V4.1 builds an interactive map, igniting cost vs capability debate — teortaxesTex · 2026-10-25
- Open-source DNA model Carbon-X (14.8B) nears frontier scores on genomics at 1/1800th the cost — aigclink · 2026-10-25
- GPT-X3-150M tops Open SLM Leaderboard, dethrones SmolLM2-135M with GDN2/GQA hybrid — datdanboi25 · 2026-10-25
- Apple's on-device Foundation Model tested: 45/49 on routing, but easily prompt-injected — NellWatson · 2026-10-25
- Blind test of 5 local AI agents on one Qwen model: scores range from 3/5 to 5/5 — GapNew4766 · 2026-10-25
- DLSS 5 works stunningly on 25-year-old FFXI despite almost no training data — eschadiol · 2026-10-25