SAGE: Structural Guidance Boosts Long-Horizon Reasoning, 8x Lean Pass Rate on Qwen3
机器之心 · wechat · 2026-10-07
Researchers from Virginia Tech, UW–Madison and Dartmouth propose SAGE (NeurIPS 2026) for long-horizon reasoning where every step looks plausible but the endpoint fails. Symbolic Closure Analysis (SCA) explains two biases: exploration bias — feasible prefix density shrinks multiplicatively with depth, making rare successful trajectories hard to sample — and accumulation bias — with endpoint reward plus KL regularization, early decisions get no direct correction.
SAGE turns the diagnosis into structural guidance during training: algebraic sparsification scores candidate steps by how much of the unresolved residual they explain, raising sampling of relevant branches; hyperbolic structure guidance maps states and targets into hyperbolic space and uses distance as stepwise feedback before the endpoint. No extra inference-time compute is needed.
Across 12 benchmarks and 7 model families versus SFT, GRPO, EMPO and GRPO-PRM: Qwen3.5-35B averages 64.86% on closed-form math (+2.49 over the best baseline); on the Andrews–Curtis symbolic task, Lean verification pass rates rise 13.2–20.8 points, nearly 8x the base model on Qwen3; MMLU-Pro, BBH-H and ARC-C improve on free-text reasoning, with GPQA slightly below EMPO.
More from Research
- Turning mouse sketches into clean drawings via gestalt-like priors and staged interpretation — debreuil · 2026-10-25
- Anthropic x Adaptyv protein design contest challenges participants to design Ozempic-like drugs — AllThingsApx · 2026-10-25
- Vik Paruchuri answers 'self-topped benchmark is suspect' with open audit — VikParuchuri · 2026-10-25
- Datalab releases OmniParseBench and ranks 17 open OCR models — VikParuchuri · 2026-10-25
- Proposal: report AUC only, or add a dev set to tune thresholds per eval — antoine_chaffin · 2026-10-25
- Adaptive Skill Granularity Framework Routes Coding Agent Skills at Runtime, Hits 100% Success — dair_ai · 2026-10-25