Georgia Tech: entropy- and confidence-based auxiliary rewards improve small-model self-improvement
GeorgiaTech · hf · 2026-11-03
A Georgia Tech paper studies whether auxiliary trajectory preferences can improve small-language-model self-improvement and transfer when answer-level rewards cannot distinguish equally correct reasoning traces. The authors compare confidence-based (SC) and entropy-based metrics (UID-Conj., combining global entropy variation and local smoothness) via online reward augmentation and offline training on correct responses (e.g., adapting Owen3-1.7B in 30 steps).
Key finding: entropy- and confidence-based trajectory preferences distinguish learning value among equally correct traces, improving self-improvement and downstream transfer under limited supervision.
More from Research
- Anthropic open-sources FlashProve: provable inference costs ~165x more than raw inference — arthurcolle · 2026-11-03
- AMLab at University of Amsterdam hiring multiple PhDs on physics-inspired deep learning — AnnaCiaunica · 2026-11-03
- CUA-Master: Nested Dual-Loop Framework Scales Computer-Use Agents via Open-World Recursive Improvements — Xinlei Yu · 2026-11-03
- CrossSWE benchmark shows coding agents struggle with cross-repository development — Silin Chen · 2026-11-03
- kalomaze: assuming a step-change that requires dropping attention is even more exotic — kalomaze · 2026-11-03
- LLMs forecast economics RCT effects with 0.054 SD median error, beating naive benchmarks — RobbWiller · 2026-11-03