Grafting task vectors enables constitutional model training for under $500
burny_tech · x · 2026-10-30
- RT @voooooogel: character training on an instruct model usually degrades capabilities. Instead, train the desired character on the base model, then sum task vectors to graft it onto the instruct model while preserving its RLVR.
- Quoting @wassname: it works — constitutional training for under $500.
- The attached diagram breaks down the pipeline and costs: pretraining $1.95M, midtraining $74K, full post-training (SFT→DPO→RL) $440K; grafting only requires training an adapter on synthetic documents plus a graft weight update, skipping the expensive post-training stages.
More from Research
- E2E-SWE benchmark: 186 tasks test coding agents building repos from scratch, best pass@1 just 67.7% — jyangballin · 2026-10-30
- LeRoPE makes RoPE frequencies learnable, beating RoPE across 52M-2.5B scales — eliebakouch · 2026-10-30
- ML Commons launches 'benchmarks as a service' to kill benchmaxxing — RexDouglass · 2026-10-30
- Redwood's Screwtape: 35 monitor-evading attacks from 252 runs on real agent transcripts — dl_weekly · 2026-10-30
- DCOCO paper: contextual correction makes parallel speculative decoding up to 6.6x faster — burny_tech · 2026-10-30
- Datology AI builds synthetic pretraining data with controllable knowledge cutoff guarantees — josh_wills · 2026-10-30