Process-level safety filtering before SFT could train aligned reasoning models
xuanalogue · x · 2026-07-22
Before fine-tuning, models could be filtered not only for task success but also for alignment at the process level, using safety classifiers to keep only reasoning traces that are both effective and safe.
The post suggests this may be a promising way to train aligned reasoning models with supervised fine-tuning, while avoiding some of the pathologies associated with RL.
Related event: Divergent Sino-US Training Paradigms and New SFT Alignment Ideas(5 posts)→
More from Safety
- Anthropic's S-1 unusually discusses AI sentience and moral status risks — birchlse · 2026-10-27
- The most dangerous instruction you can give a superintelligent machine: lie to me — r0ck3t23 · 2026-10-27
- Rob Miles: AI employees recording audio logs as automated hacking takes off — Puzzleheaded-King584 · 2026-10-27
- Researcher Asks Whether Suspicious Model Behavior Can Ever Be Explained — or Faked — jd_pressman · 2026-10-27
- UK AISI papers: no good safeguards yet against misuse of biological AI tools — davidmanheim · 2026-10-27
- Washington's message to AI execs: Please shut up — TobyWalsh · 2026-10-27