DCO: Only Update Direction Matters When Fine-Tuning Instruct Models
Fei Yuan · hf · 2026-09-16
Drift-Constrained Optimization (DCO) reframes instruct-model fine-tuning as a direction-selection problem: within a pre-specified behavioral drift budget, drift fixes distance from the reference model, leaving update direction as the only degree of freedom.
- Stringent test: strong instruct models fine-tuned only on QA final answers must still generate multi-step reasoning at inference — a setting where standard fine-tuning fails.
- Breakthrough: a coarse layer-selective probe reverses this failure, proving effective directions exist.
- Results: substantial gains in scientific reasoning and multilingual translation on Qwen3-8B and Qwen3-14B; across 100+ languages the models match or beat dedicated translation systems and provide stronger RL initialization.
- Takeaway: fine-tuning isn't about how much a model changes, but how that change is spent. Code and models open-sourced.
More from Research
- DeepMind's guide to sharded matrix multiplication: the math behind training LLMs on 10k TPUs — zacharynado · 2026-09-16
- USC lecture notes derive closed-form AUC scaling laws and a 'spectral horizon' for finite-data ML — PTenigma · 2026-09-16
- Just having a tool available drops model answer rate from 98.2% to 63.5%, paper finds — dair_ai · 2026-09-16
- 16-Day Multi-Agent Stress Test: 85万 LLM Calls, No World Fully Resilient — Deepak Akkil · 2026-09-16
- New Paper Maps Roadmap Toward AI Recursive Self-Improvement Across Domains — Yi Duan · 2026-09-16
- ProgramAsWeights: compiling English descriptions into small neural programs, by yuntiandeng — yuntiandeng · 2026-09-16