FTPO fixes small model loops in long reasoning chains
SergioPaniego · x · 2026-08-28
Reading LiquidAI's Antidoom report reveals that small models get stuck in repetitive loops (e.g., repeating "Wait") during long thinking traces on hard problems. Early checkpoints for LFM2.5-2.6B and Qwen3.5-4B showed loop rates of 10.2% and 22.9% respectively.
The fix is FTPO (Final Token Preference Optimization). Key differences from DPO:
- Trains on a single token position mid-generation, not whole sequences;
- Spreads probability across 20 plausible alternatives instead of swapping one overtrained token;
- Precisely targets the token where the loop starts.
More from Research
- Astrophysicists use Keras to model cosmic ray origins from raw waveforms — fchollet · 2026-08-28
- SSMB: Self-Supervised Keypoint Detection Directly on Motion-Blurred Images — zhenjun_zhao · 2026-08-28
- XSquare Robot unveils WALL-SS: long-horizon world model with 0.93 sim-real correlation — chris_j_paxton · 2026-08-28
- Alibaba's Accio launches CommerceAgentBench: 107 real e-commerce tasks testing execution — future_coded · 2026-08-28
- New Paper Critically Evaluates VASCO Studies on Pre-Sputnik Orbiting Objects — DanielWhiteson · 2026-08-28
- Allen Institute video shows neurons don't connect randomly, revealing connectomics — CatAstro_Piyush · 2026-08-28