LeCun amplifies Meta's Memory Mosaics paper, billed as a challenge to Transformers
ylecun · x · 2026-10-09
LeCun retweeted hype around Meta's "Memory Mosaics at scale" paper, which replaces standard attention with networks of associative memories that store and selectively retrieve key-value pairs. The claim: less brute-force data dependence and no exploding long-context compute, with a model trained at 1 trillion tokens showing strong results. The "end of the Transformer era" framing is likely overstated.
More from Research
- Hugging Face launches Robotic Episodes Viewer for 24k+ LeRobot datasets — mishig25 · 2026-10-09
- Blind humanoid walks, plays soccer and lifts suitcases with joint encoders only — accepted at Humanoids 2026 — Jan_R_Peters · 2026-10-09
- Delete object info from observations and PPO learns to search anyway — TU Darmstadt on its Humanoids 2026 paper — Jan_R_Peters · 2026-10-09
- U-Space finds an interpretable subspace for LLM uncertainty, no training needed — Tobias Braun · 2026-10-09
- CARE certifies VLA inference speedups up to 10.8x with statistical guarantees — UMCP · 2026-10-09
- SOL: a sample-based distributional metric proposed for evaluating text diffusion LMs — NandoDF · 2026-10-09