NUS-Led Paper: Uncertainty-Guided Contraction Cuts Reasoning Post-Training Budget to 0.22% Without Losing Capability
NJU · hf · 2026-10-13
A paper from NUS, Nanjing University, Fudan, Sydney and others proposes an uncertainty-guided progressive contraction framework asking how far reasoning supervision can be compressed without sacrificing capability.
- Instead of treating teacher CoTs as single realized paths, they build Monte Carlo reasoning trees around intermediate reasoning states to expose process-level uncertainty.
- Two-stage contraction: uncertainty first contracts the corpus into a compact teacher-informed pool, then learner-aware online routing concentrates SFT optimization by the student's evolving state.
- Results: as little as 0.22% of the optimization budget of full-data SFT, 450x less training exposure, while maintaining or improving reasoning; generalizes across heterogeneous students, black-box teachers, and weak-to-strong transfer.
More from Research
- Graph theorist spends 20 hours on OpenAI Math Dump, finds solutions to his decade-old conjectures — burny_tech · 2026-10-13
- tszzl predicts mechinterp progress in coming months unlike anything in years — tszzl · 2026-10-13
- Free ML Workshop on the Math Discord Server Covers ML, Deep Learning and Graphs — zeroalephzeta · 2026-10-13
- sansreplace: a novel O(k log k) sampling-without-replacement algorithm with zero extra memory — burny_tech · 2026-10-13
- New Paper Explains Mamba-3's State Rotations as Underdamped Second-Order Dynamics — tri_dao · 2026-10-13
- New Paper Gives Exact Pricing and Analytic Risk Attribution for First-to-Default Basket Swaps — DrCirillo · 2026-10-13