Sentence Transformers adds training-free model merging via mergekit, with Linear, SLERP and TIES
tomaarsen · x · 2026-10-29
Sentence Transformers maintainer tomaarsen announced built-in model merging: combine compatible checkpoints into one model without further training, powered by Arcee AI's mergekit.
- Usage: pip install "sentence-transformers[merge]", then call SentenceTransformer.merge() with HF model IDs, local paths, or loaded instances; configs, prompts and processors are inherited from the first input.
- Methods: Linear weighted averaging (e.g. a 60/40 merge of two MiniLM models), SLERP, and TIES; SLERP works for dense, sparse, reranker and multi-vector models.
- Use case: merging fine-tunes from the same base model to combine capabilities, or interpolating between a model and a further fine-tuned checkpoint.
- Caveat: merges can improve some tasks while hurting others — evaluate against your own data before choosing.
Related event: Sentence Transformers Adds Training-Free Model Merging(2 posts)→
More from Research
- New paper asks why teams of AI agents fail where a single agent often matches them — JoshPurtell · 2026-10-30
- Prime Intellect makes DeepSeek's MegaMoE trainable, up to 55% faster MoE training — samsja19 · 2026-10-30
- CMU Releases SwarmChat, an Open-Source Slack for AI Agent Swarms — niloofar_mire · 2026-10-30
- Top robotics data companies are applied research labs: ego-data post-training beats Apple EgoDex — ATTlKA · 2026-10-30
- Prime Intellect Residents Release Fully Open Agent Evals: MazeBench, PretrainGym, OpenTTD-Bench — xeophon · 2026-10-30
- DreamFAR world model trained from scratch on robot data only, ~$110k in H200 hours — k7agar · 2026-10-30