Multi-stage LLM training decomposes into additive task vectors

A study suggests that multi-stage LLM training (base → midtrain → RLVR) can be decomposed into additive task vectors, with intermediate stages theoretically optional, since summing the weight deltas still yields expected results.

2026-10-29 ~ 2026-10-29 · 2 related posts

• Training stages as additive task vectors: base + midtrain + RLVR surprisingly just adds up — voooooogel · 2026-10-29
• Training intuition: final = base + (rlvr − base), midtrain collapses into a task vector — voooooogel · 2026-10-29