AntLing open-sources Ling-3.0 models using WSM to replace LR decay
AcanthisittaOk1699 · reddit · 2026-08-19
AntLing has released 6 base model checkpoints for Ling-3.0-tiny and Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages. These models have not undergone post-training, offering flexible starting points for continued pre-training and fine-tuning.
Key Highlights:
- WSM Technology: Uses Weighted State Merging (WSM) to replace traditional Learning Rate (LR) decay. This makes the training process better suited for continual pre-training and enables offline exploration of different LR decay strategies.
- Unified Recipe: Both tiny-base and flash-base share a single training recipe, allowing the community to validate strategies on the tiny version before scaling to flash.
Model Specs:
- Ling-3.0-tiny-base: 7.9B total | 1.3B active. Despite having half the total parameters of Ling-2.5-mini, it delivers comparable or superior performance on most benchmarks, with particularly strong results in coding.
- Ling-3.0-flash-base: 124B total | 5.1B active. Achieves strong performance in coding, reasoning, and long context tasks, rivaling models 2-3 times larger.
Related event: AntLing Open-Sources Ling-3.0 Base Model Checkpoints(2 posts)→
More from Models
- Harmony tags confuse ChatGPT, causing incorrect DeepSeek explanation — mitsuhiko · 2026-08-20
- Demo: How to setup MINIMAX H3 — Background_Shift_333 · 2026-08-20
- Unsloth Releases Qwen3.8-27B GGUFs with 10% Higher Accuracy — danielhanchen · 2026-08-20
- Hugging Face releases SmolLM3 mid-training checkpoint amid 200x efficiency debate — eliebakouch · 2026-08-20
- Claude 5.6 Sol Ultra with computer use called a qualitative leap like Opus 4.5 with MCP — curious_vii · 2026-08-20
- Gemini 3.7 Flash Test: 350 tok/s Speed, Mixed Coding Results — haider1 · 2026-08-20