TTPO uses disagreement with majority vote as training signal, enabling label-free test-time training
arupbuildsai · reddit · 2026-09-15
Key idea
RL and on-policy self-distillation (OPSD) need ground truth, which rules them out at test time. The usual substitute, majority-vote pseudo-labels, breaks when the vote is wrong. The paper (arXiv 2608.27448, Wang et al.) makes a sharp observation: the failure is asymmetric — rollouts disagreeing with the pseudo-label are usually wrong whether or not the vote was right, so disagreement survives a bad vote while agreement does not.
Method
- Agreeing rollouts distilled with OPSD
- Disagreeing rollouts penalised with Grouped RL
- Token-level selection on both branches: down-weight converged positions for distillation; only penalise confident errors on the RL side
Results
- Qwen3-1.7B label-free TTT: 38.0% → 45.2%
- +25.2% to +36.4% in the no-thinking setting
- Parity with label-supervised OPSD on five competition-level benchmarks; cross-task generalisation claimed
Caveat
The whole approach assumes majority voting converges on a single correct answer — true for competition math and verifiable code, not for summarisation or drafting, so gains likely won't transfer to open-ended workloads. The author also wants an ablation using the disagreement signal alone.
More from Models
- GPT-6 Astra says you can delete scaffolding — just not the control kind — Slight_Republic_4242 · 2026-09-15
- You Don't Need Ultra: Power User's Rule of Thumb Says Low/Medium Tiers Suffice for Daily Tasks — soumitrashukla9 · 2026-09-15
- Reddit user finds Claude one-shots tasks where ChatGPT keeps repeating mistakes — LarryBlink · 2026-09-15
- Cline Desktop hands-on: scheduled background review catches real data-loss risk — HowDevelop · 2026-09-15
- ChatGPT admits project instructions are guidance, not enforced rules — and it's getting worse — dbvirago · 2026-09-15
- Google DeepMind launches SL2T sign-language-to-text model, debuts on Pixel 11 and Gboard — NandoDF · 2026-09-15