SFTProbe Beats OpenThoughts With 5k Examples: 103 vs 4,000+ GPU Hours for Reasoning Data

heghbalz · x · 2026-11-22

A NeurIPS 2026 paper, SFTProbe, proposes directly generating high-quality reasoning data instead of costly generate-then-filter pipelines. Curating OpenThoughts-114k took 4,000+ H100 GPU hours (1.5 weeks on 16 H100s); SFTProbe generates a math-reasoning SFT dataset that beats it in just 103 GPU hours (6 hours on 16 H100s) — a 41x speedup — using only 5k examples vs 89k Openthoughts-Math, plus a new olympiad-winning physics dataset.

Original post →

More from Research

Research channel →