Ling-3.0 Open Source: 6 Base Checkpoints Released with WSM Training Method
bclavie · x · 2026-08-19
Ant Ling has released 6 base model checkpoints for Ling-3.0-tiny and Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.
Key Highlights:
- WSM (Weighted State Merging): Replaces LR decay with weighted checkpoint merging. This suits continual pre-training better and allows offline exploration of different LR decay strategies.
- Shared Recipe: Researchers can validate strategies on tiny-base and scale them to flash-base.
The models have not undergone post-training, offering flexible starting points for continued pre-training and fine-tuning.
Related event: AntLing Open-Sources Ling-3.0 Base Model Checkpoints(2 posts)→
More from Models
- Harmony tags confuse ChatGPT, causing incorrect DeepSeek explanation — mitsuhiko · 2026-08-20
- Unsloth Releases Qwen3.8-27B GGUFs with 10% Higher Accuracy — danielhanchen · 2026-08-20
- Hugging Face releases SmolLM3 mid-training checkpoint amid 200x efficiency debate — eliebakouch · 2026-08-20
- Claude 5.6 Sol Ultra with computer use called a qualitative leap like Opus 4.5 with MCP — curious_vii · 2026-08-20
- Gemini 3.7 Flash Test: 350 tok/s Speed, Mixed Coding Results — haider1 · 2026-08-20
- AntLing open-sources Ling-3.0 models using WSM to replace LR decay — AcanthisittaOk1699 · 2026-08-19