ByteDance Seed's Cortex Distills Context into Parametric Memory, Keeping 88.9% Gains
ByteDance-Seed · hf · 2026-10-15
ByteDance Seed, Zhejiang University and Tsinghua formulate Memory Consolidation—internalizing context-induced gains into parameters without hurting other capabilities. Cortex generates prospective queries and distills context-conditioned outputs into mergeable low-rank or structural FFN updates, evaluated on a five-task-family testbed across 4B–120B dense and MoE backbones. After 50 sequential factual edits on a 14B model, Cortex-Structural recovers 88.9% of the context gain while retaining 99.8% of non-target performance, sitting on the consolidation–retention Pareto frontier in most settings.
More from Research
- Gil Kalai presents 'the most outrageous conjecture' on proving primes in sparse sets — _onionesque · 2026-10-15
- 83,017-vacancy RCT: human recruiters add no hires once algorithmic screening exists — danielrock · 2026-10-15
- RAESR Does Super-Resolution in Frozen DINOv3 Latent Space at 37ms per 512px Image — RTP-LLM · 2026-10-15
- BigCell Generates Coherent Gigapixel Whole-Slide Pathology Images, Boosting Few-Shot Classification 18% — StonyBrook-CVLab · 2026-10-15
- NavGPT-3 Reaches Human-Level Navigation: 90.43 SR on RxR-CE vs Human 90.4 — Metacognition-AI · 2026-10-15
- SWEeper-Bench: Best of 15 Frontier Agents Solves Only 59% of Interactive Bug-Finding Tasks — zlab-princeton · 2026-10-15