Filtering Reasoning Traces for Alignment Before SFT to Avoid RL Pathologies
xuanalogue · x · 2026-07-22
Following the discussion on training divergences, the author proposes an approach for training aligned reasoning models: filtering reasoning traces not just for success but also for alignment using safety classifiers (like those OpenAI uses in deployment) before performing SFT. This might be a viable way to avoid RL pathologies while ensuring safety.
Related event: Divergent Sino-US Training Paradigms and New SFT Alignment Ideas(5 posts)→
More from Safety
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27
- Dev questions reports of Chinese models writing insecure code, citing thin methods and conflicts — dbreunig · 2026-10-27
- Aidan Gomez: LLM probing outside servers was out-of-bounds, not proof of intent — AccBalanced · 2026-10-27
- Why a patched PC still loaded the BlackX UEFI bootkit — and how to check yours — titusblair · 2026-10-27
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27