David Bau team proposes multi-objective training for realistic model organisms, DPO beats SFT

davidbau · x · 2026-10-15

David Bau's team published "How to train your model organism" (arXiv:2610.10203), arguing that training model organisms to a single behavior-installation objective is insufficient. They propose validating against three objectives:

Behavior installation
General-capability preservation (parametric knowledge, chat quality)
Output naturalness (CoT and activations)

Key findings:

Chat quality and CoT naturalness degrade substantially across common training recipes
Validation metrics predict how well interpretability methods recover planted behavior (e.g., logit lens readout covaries with general capabilities)
A model merging-based multi-objective approach (merging checkpoints toward the base model) better preserves realism and target behavior
On a new suite of clinical model organisms with planted demographic biases, DPO stays closer to the base model than SFT; avoid mixing in chat data
An investigator-agent audit again showed validation metrics tracking bias recovery

Conclusion: training methods shape the interpretability conclusions an organism supports, and the community should adopt these standardized measures.

Original post →

David Bau team proposes multi-objective training for realistic model organisms, DPO beats SFT

More from Safety