IntBMoE Decouples MoE Participation, Execution and Memory, Deployed in AMap RecSys

Ran Cheng · hf · 2026-09-21

AMap's ML team proposes IntBMoE, a block-conditioned, full-participation Mixture-of-Experts architecture. The paper argues existing MoE designs cannot independently control three per-token quantities: participation (how many experts contribute), execution (how many are computed, i.e., compute cost), and materialization (how many expert-sized parameter sets are stored, i.e., memory cost) — sparse routing shrinks participation, dense mixing inflates execution, and parameter merging balloons storage.

IntBMoE's approach:

• A small learned codebook yields blocks; a lightweight hypernetwork merges all experts in each layer's pool into one composed expert, ensuring full participation;
• A router sends each token to only a few blocks, keeping execution sparse;
• The codebook (not the input) bounds how many blocks exist, capping materialization;
• Dual-Path Residual Gating (DPRG) couples two independently composed paths via multiplicative gating.

Experiments show consistent gains over representative sparse and dense MoE baselines on image classification, with generalization to language modeling and sequential recommendation. IntBMoE is fully deployed in AMap's generative recommendation system, serving hundreds of millions of users within a 60ms latency budget, with a 2.4% relative UVCTR gain in online A/B testing. Code is open-sourced at AMAP-ML/DreamX-Rec on GitHub.

Original post →

More from Research

• Writing a textbook revealed: AI-for-science hinges on "details", not architectures — bravo_abad · 2026-09-21
• The biggest AI-for-science mistakes happen before you pick a model: 11 ideas — bravo_abad · 2026-09-21
• 19 teams, 478 submissions, $50k in prizes: Aletheia's Quest AI lie detection winners announced — gsarti_ · 2026-09-21
• IBM Launches Steerability Challenge: Suppress LLM Dishonesty, Win Cash and NeurIPS Spot — davidbau · 2026-09-21
• Tencent open-sources T-Mem, a memory architecture that anticipates instead of archiving — blaizedsouza · 2026-09-21
• Radial Duality paper turns constrained optimization into unconstrained Lipschitz problems — prof_grimmer · 2026-09-21

Research channel →