Harvard-led team's MFM memory system lets frozen-weight agents keep learning in Minecraft
du_yilun · x · 2026-11-06
A Harvard/Kempner, NYU and MIT team (Yilun Du's group) introduces Multimodal Filesystem Memory (MFM), storing an embodied agent's past experience as a structured multimodal filesystem spanning episodic, procedural, semantic and working memory—enabling continual learning with fully frozen model weights.
Four key design points:
- Retrieve what matters: resurface old observations (e.g., an iron vein seen thousands of steps earlier);
- Continual learning: distill successful actions into reusable skills;
- Trace failures: attribute mistakes to causes and revise what was learned;
- Reveal knowledge gaps: guide future exploration.
In Minecraft's Ender Dragon task, the agent—playing with raw keyboard/mouse actions—completed 21 of 24 milestones in 58 minutes at real-time speed. In a hill-climbing experiment with larger step budgets and multi-life memory inheritance, MFM kept improving where baselines had plateaued.
More from Embodied
- UC Berkeley's Ctrl-F adds force commands so humanoid robots can move 20 kg loads — ChongZzZhang · 2026-11-06
- Archon Robotics debuts Anchor 1, a humanoid whole-body controller cutting tracking error 30-80% — ChongZzZhang · 2026-11-06
- Lightwheel and Synopsys bring CAE-grade contact physics into robot simulation — lukas_m_ziegler · 2026-11-06
- I watched a humanoid robot clean my apartment: remote-piloted, $30/hour, data-hungry — MIT Tech Review AI · 2026-11-06
- RAX lets coding agents unscrew a real jar with dexterous hands: 6 chained skills, 140° rotation — du_yilun · 2026-11-06
- HeLM: explicit language memory gives VLA robots +12.8 points on non-Markovian manipulation — SeoulNationalUniv · 2026-11-06