Apple study: aligning MoE routing with agent operations boosts success rates by 10+ points

apple · hf · 2026-10-08

An Apple study examines the co-design of agentic post-training and sparse mixture-of-experts structures:

Observation: off-the-shelf MoE models already show expert-selection structure aligned with agentic trajectories — routing overlaps more between turns with semantically similar operations (e.g. READ, UPDATE).
Problem: standard RL ignores this specialization, leaving routing uncontrolled during training, which hurts task performance and inference efficiency.
Method: a hierarchical routing control framework aligns turn-level expert selections with agentic operations, regularizes token-level selections for local consistency, plus an entropy-gated mechanism for post-training stability.
Results: over 10-point success-rate improvements across all evaluated benchmarks, showing agent trajectory structure is an effective signal for optimizing MoE capacity.

More from coding & agent