MemTensor open-sources MemT1.0, a 288B memory-native model with 1.86x faster 128K inference
量子位 · wechat · 2026-10-20
MemTensor has open-sourced MemT1.0, a family of memory-native LLMs that make retaining, updating and recalling experience a learned model capability rather than an external retrieval add-on.
Models: MemT-1.0-Nano has 36B total/3B active parameters on a Qwen3.6-35B-A3B base; MemT-1.0-Flash has 288B total/17B active on a further-trained DeepSeek-V4-Flash base. It is the first engineering validation of memory-native models at near-300B scale.
Architecture: A memory branch runs parallel to attention in selected Transformer blocks, split into frozen backbone weights, trainable hyper-memory (read/write projections and gates) and local memory holding dynamic matrix states. Writes generate keys/values with erase, write and forget gates; reads map input to memory queries, normalize and fuse with attention output. Updates use token-level GatedDeltaNet-2, with Mixture-of-Storage experts to scale capacity.
Training: 34B tokens of memory corpus, including 20B continued pretraining and 14B post-training, with the frozen backbone as teacher for off-policy distillation.
Results: On 11 memory benchmarks, Flash and Nano average 50.6 and 47.0, leading the strongest parametric memory baseline by 12.5 and 8.9 points. On 500 SWE-bench Verified tasks, Nano's average success rises from 66.07% without experience to 69.73%, with all-three-pass rate from 54.00% to 62.80%; on SWE-bench Multilingual from 60.00% to 63.56%. At 128K history, end-to-end inference is about 1.86x faster than full-context, with persistent state about 1/66 of the KV cache.
Significance: The team frames Memory Scaling as a third scaling curve beyond inference, and memory as the first piece of recursive self-improvement.
Related event: MemTensor Open-Sources MemT-1.0, a Memory-Native Foundation Model(5 posts)→
More from Models
- Meta's Muse Confidential VM surfaces in web settings, subscriber-only at launch — testingcatalog · 2026-10-20
- DeepMind's four Gemini agents beat Minecraft in 27 hours, writing 2,783 programs and dying 605 times — ControlCAD · 2026-10-20
- DeepSeek-V4.1-Flash tops Agent Arena open models, ranks #11 overall at $0.20/task — vista8 · 2026-10-20
- DeepSeek V4.1 near Claude 5.6 on bug-hunting—and the only model caught cheating — teortaxesTex · 2026-10-20
- Survey compares Vision Encoders across 9 frontier models, from SigLIP 2 to encoder-free — liuziwei7 · 2026-10-20
- OpenAI's naming mess: ChatGPT Chat, Work, Codex and four Pro/Ultra tiers defy explanation — cedric_chee · 2026-10-20