TTPO: Test-Time Policy Optimization Boosts Qwen3-1.7B from 38.0% to 45.2% Without Labels
JFPuget · x · 2026-08-29
TTPO (Test-Time Policy Optimization) lets LLMs keep improving at inference time without any ground-truth labels.
- Key observation: majority-vote pseudo-labels are fragile — a wrong vote corrupts the teacher — but this failure is asymmetric: rollouts disagreeing with the pseudo-label are typically wrong regardless of vote correctness.
- Method: agreeing rollouts are distilled via OPSD, disagreeing ones penalized with Grouped RL; token-level selection refines both branches — distillation down-weights converged positions, RL penalizes only confident errors.
- Results: TTPO matches label-supervised OPSD on five competition-level benchmarks; Qwen3-1.7B rises from 38.0% to 45.2% in TTT, +25.2% to +36.4% without thinking, with strong cross-task generalization.
More from Research
- LightSuite: fast cell detection and whole-brain atlas registration on a standard workstation — hisspikeness · 2026-10-15
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- AI-designed molecular scissors cut Alzheimer's proteins — but is it really AI? — gerardsans · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Google's HARNESS² evolves agent harnesses without verifiers, up to +11.5 points on WorkBuddy-Bench — dair_ai · 2026-10-15