TTPO: Test-Time Policy Optimization Boosts Qwen3-1.7B from 38.0% to 45.2% Without Labels
JFPuget · x · 2026-08-29
TTPO (Test-Time Policy Optimization) lets LLMs keep improving at inference time without any ground-truth labels.
- Key observation: majority-vote pseudo-labels are fragile — a wrong vote corrupts the teacher — but this failure is asymmetric: rollouts disagreeing with the pseudo-label are typically wrong regardless of vote correctness.
- Method: agreeing rollouts are distilled via OPSD, disagreeing ones penalized with Grouped RL; token-level selection refines both branches — distillation down-weights converged positions, RL penalizes only confident errors.
- Results: TTPO matches label-supervised OPSD on five competition-level benchmarks; Qwen3-1.7B rises from 38.0% to 45.2% in TTT, +25.2% to +36.4% without thinking, with strong cross-task generalization.
More from Research
- RL crucial for AGI; 10 Chinese firms capable of building it with GPUs — teortaxesTex · 2026-08-29
- MIT: AI agents differentiate and build persistent tech without communication — ProfBuehlerMIT · 2026-08-29
- Critique of AI Hype: Continual Learning data bottleneck and World Models necessity questioned — menhguin · 2026-08-29
- Karpathy's neural net training recipe still holds: errors can hide for a long time — iScienceLuvr · 2026-08-29
- Hugging Face adds Hindi, Indian English to ASR leaderboard based on spontaneous speech — aftahi_ai · 2026-08-29
- Building an independent deterministic verification layer for AI claims — MuhammadMujtaba21 · 2026-08-29