Dnotitia open-sources DNA Factory, an opinionated LLM fine-tuning framework with 5 trainers

likejazz · reddit · 2026-11-10

Korean company Dnotitia open-sourced DNA Factory, the opinionated LLM fine-tuning framework it has used internally for over a year (Apache-2.0, built on TRL + DeepSpeed).

Philosophy: After consulting for many companies fine-tuning their own LLMs, the author found only a handful trained correctly — modern tools let people launch runs without understanding what happens underneath. DNA Factory ships exactly one recommended way: all defaults live in documented configs/defaults-.yaml files, and you override only what you need.

Key features:

Token-level loss-mask visualization: overrides TRL's loss to color trained tokens blue and excluded tokens gray — catching the most common mistake, "wait, that token was in the loss the whole time?"
Only 5 trainers on purpose: SFT, DPO, GRPO, OPD, SDFT. PPO is dropped, DPO slated for removal; current focus is GRPO and other on-policy methods.

OPD (On-Policy Distillation) reverses classic distillation: the student generates its own answer and a frozen teacher grades it token-by-token via reverse KL, avoiding catastrophic forgetting. SDFT lets the model act as its own teacher given privileged context.

Original post →

Dnotitia open-sources DNA Factory, an opinionated LLM fine-tuning framework with 5 trainers

More from coding & agent