Fin-SD: hindsight self-distillation without external teachers boosts local models' financial reasoning
ctgt-inc · hf · 2026-10-29
A paper from UC Berkeley and CTGT, "How Much Does the Hint Matter?", studies on-policy self-distillation (OPSD) for financial reasoning: under data-residency constraints, financial institutions can only deploy small open-weight models locally, which lag frontier systems on multi-step quantitative reasoning.
The Fin-SD recipe: when the student fails a problem, it identifies its first incorrect reasoning step and writes a short critique, truncates the trajectory before the error to obtain a correct prefix, and conditions a frozen copy of the student on the same prefix—no external teacher or full reference solution required.
The core question: how much of the reference signal (hint/full solution) that traditional OPSD needs is actually necessary? The authors characterize the scope conditions. Accepted at ACM ICAIF '26.
More from Research
- Granary-v2 open speech model expands to 48 languages with open data and pipeline — rdesh26 · 2026-10-29
- Tinker launches Dimuon, a Muon-based optimizer that beats AdamW on LoRA fine-tuning — xennygrimmato_ · 2026-10-29
- Six CUDA kernel optimizations squeeze another 1.78× speedup out of 3DGS training — gkopanas · 2026-10-29
- Cross-lingual LLM effects paper wins Outstanding Paper award at EMNLP 2026 — anas_ant · 2026-10-29
- Lie detector compiled into Qwen3.5 weights flags 91% of lies with zero false positives — teortaxesTex · 2026-10-29
- Practitioner: No serious lab actually trains on the test set today — nrehiew_ · 2026-10-29