Divergent Sino-US Training Paradigms and New SFT Alignment Ideas

Recently, observations indicate a clear divergence in training paradigms for reasoning models between US and Chinese AI labs: US labs lean towards Reinforcement Learning (RL), while Chinese labs more frequently employ Supervised Fine-Tuning (SFT) based on successful trajectories. @goyalshaliniuk and @xuanalogue point out that the SFT-preferred path is equally highly competitive, with the recent strong performance of Kimi K3 serving as supporting circumstantial evidence.

Confirmed

The divergence in training paradigms is based on industry observations. @xuanalogue proposes that conducting process-level evaluations of a model's reasoning traces before fine-tuning is feasible. Specifically, before applying SFT to successful reasoning traces, in addition to standard success-based filtering, safety classifiers (such as those used by OpenAI in deployment) could be introduced to evaluate the alignment of these traces, retaining only those that are both effective and safe.

Why it matters

Combined with discussions from relevant RL theory papers, determining whether a model achieves its goals using the correct methods before fine-tuning could help guide models toward safer generalization. This method of performing process-level safety filtering before SFT could potentially train safely aligned reasoning models while avoiding the pathologies associated with RL.

2026-07-22 ~ 2026-07-22 · 5 related posts

Primary sources

U.S. labs lean on RL while Chinese labs favor SFT on successful traces — goyalshaliniuk · 2026-07-22 14:31 (UTC+08:00)
Filtering Reasoning Traces for Alignment Before SFT to Avoid RL Pathologies — xuanalogue · 2026-07-22 14:31 (UTC+08:00)
RL theory paper argues process-level evaluation may steer models toward safer generalization — xuanalogue · 2026-07-22 16:58 (UTC+08:00)

[source] U.S. labs lean on RL while Chinese labs favor SFT on successful traces — goyalshaliniuk · 2026-07-22
US Labs Prefer RL for Reasoning, Chinese Labs Lean Towards SFT on Traces — xuanalogue · 2026-07-22
[source] Filtering Reasoning Traces for Alignment Before SFT to Avoid RL Pathologies — xuanalogue · 2026-07-22
Process-level safety filtering before SFT could train aligned reasoning models — xuanalogue · 2026-07-22
[source] RL theory paper argues process-level evaluation may steer models toward safer generalization — xuanalogue · 2026-07-22