Ant's PAR trains a 0.5B model to extrapolate state tracking 16x beyond training length
ant-intl · hf · 2026-10-21
Ant International and NUS propose Progress-Aligned Recurrence (PAR), which aligns the prediction at iteration k with the state after the first k operations and rescales recurrent states to first-iteration norms, fixing scale drift under final-answer supervision. On swap-S5, a Qwen2.5-0.5B trained on at most 32 moves hits 95.5% mean full-state accuracy at 512 moves (16x training length) vs 55.7% for RLT and 0.8% for explicit chain-of-thought on the same backbone. Ablations show norm anchoring is essential (extrapolation drops below 2x without it); a learned completion redundancy mechanism handles stopping, and prefix self-distillation works without intermediate labels.
More from Research
- Menlo Research open-sources Cyclotron, an RL training framework for its Asimov humanoid robot — freelerobot · 2026-10-21
- Father-son duo uses Hark to semi-autonomously medal on ARC Prize after days of self-revision — Deep-Firefighter-279 · 2026-10-21
- Berkeley paper: first tractable estimator of empowerment from offline data for exploration — svlevine · 2026-10-21
- Paper: The network around a cortical core is a regulator, not an optimizer — maier_ak · 2026-10-21
- Big day for AI in bio: OpenAI-Orchestra Bio deal, Holon-S1 cancer model, more — iScienceLuvr · 2026-10-21
- GPQA Diamond flaw: question 186 requires an external telescope spec sheet link — charles_irl · 2026-10-21