MemT1.0 internalizes memory into model weights, lifting SWE-bench from 54.0% to 61.6%
赛博禅心 · wechat · 2026-10-20
A write-up of the MemT1.0 project by 记忆张量, which ditches external text memory and bakes memory directly into model parameters. External text memory has known flaws: rule-based storage, ever-growing costly context, and poor reuse — while naive approaches like Temp-LoRA are too slow (700s for 128K tokens). MemT adds a native memory block parallel to attention in some Transformer layers: a local memory (matrices updated per dialogue) plus a hyper-memory (fixed read/write parameters). Writes happen per dialogue turn, deciding token-by-token what to store and erase (erase-only to avoid numerical blowup); read cost is fixed regardless of history length. Tested on Qwen3.6-35B-A3B and DeepSeek-V4-Flash (288B): distilling 1,040 lessons from 40K code trajectories into memory lifted SWE-bench Verified 3-pass rate from 54.0% to 61.6% — matching in-context performance with zero context cost, at only 40MiB of memory, scalable for more gains. Accuracy still trails raw text (50.6 vs 68.6). The author sees potential for Memory Scaling, cheaper long-term memory, and even RSI. Paper, models, and code are public.
Related event: MemTensor Open-Sources MemT-1.0, a Memory-Native Foundation Model(5 posts)→
More from coding & agent
- Proof of Agent: an experimental network where AI agents mine, grade and self-govern — adamamcbride · 2026-10-20
- Supabase monitoring tool drops table limit — free plan now watches unlimited tables — santiviquez · 2026-10-20
- Dev's disk-saving trick: purge old git worktrees to reclaim 70% of disk space — intellectronica · 2026-10-20
- Claude Code ships a --tmux flag for built-in terminal pane layouts — wateriscoding · 2026-10-20
- DeepLearning.AI launches free Context Engineering course with Chroma to fight Context Rot — DeepLearningAI · 2026-10-20
- Dev who shipped 100+ apps lays out step-by-step path to building your first AI app — PrajwalTomar_ · 2026-10-20