Memorizon trains streaming world models beyond context window with only 12% step-time overhead
MBZUAI-IFM · hf · 2026-10-02
MBZUAI-IFM's Memorizon decouples supervision span from attention span for streaming world models: scored chunks retrieve top-K latents via camera co-visibility into a shared bank bounded by kK, so long-span training stays cheap. Extending spans from 100s to 400s adds just 12% step time, while reaching a location's first visit boosts revisit consistency by 24-30% over a sliding-window baseline. Filling the bank from another episode cuts revisit correlation by 83%, confirming the model actually uses retrieved latents.
More from Research
- SECRET: training-free relay steering cuts cross-modal hallucinations in AVLLMs by up to 18 points — Yu Zhang · 2026-10-02
- Meituan LongCat's Neighborhood OPSD boosts math reasoning by up to 2.75 Average@12 points on Qwen3 — meituan-longcat · 2026-10-02
- Decentralized Master-Mind: intent denoising solves 1,598 of 1,600 MAPF tasks and scales to 1M agents — Valeriy Vyaltsev · 2026-10-02
- ProVer paper: LLM judge picks key trajectory steps, beats GRPO by up to 9.9% — omarsar0 · 2026-10-02
- University of Copenhagen PhD opening: tokenization-free LMs, office in a botanical garden observatory — delliott · 2026-10-02
- KaliBench: 8,504 pairs benchmark shows no open-weight LLM exceeds 42% on Kali Linux CLI tasks — RISys-Lab · 2026-10-02