AntLing open-sources Ling-3.0 models using WSM to replace LR decay
AcanthisittaOk1699 · reddit · 2026-08-19
AntLing has released 6 base model checkpoints for Ling-3.0-tiny and Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages. These models have not undergone post-training, offering flexible starting points for continued pre-training and fine-tuning.
Key Highlights:
- WSM Technology: Uses Weighted State Merging (WSM) to replace traditional Learning Rate (LR) decay. This makes the training process better suited for continual pre-training and enables offline exploration of different LR decay strategies.
- Unified Recipe: Both tiny-base and flash-base share a single training recipe, allowing the community to validate strategies on the tiny version before scaling to flash.
Model Specs:
- Ling-3.0-tiny-base: 7.9B total | 1.3B active. Despite having half the total parameters of Ling-2.5-mini, it delivers comparable or superior performance on most benchmarks, with particularly strong results in coding.
- Ling-3.0-flash-base: 124B total | 5.1B active. Achieves strong performance in coding, reasoning, and long context tasks, rivaling models 2-3 times larger.
More from Models
- Gemini 4 Argon hits AssemblyAI's LLM Gateway, tops voice bench at 99.5 across 50+ models — AssemblyAI · 2026-10-16
- Meta's open-weight Muse Spark Open hits Ollama with 1M-token context and Claude Code support — ollama · 2026-10-16
- 'Opus 5.5' collage demos spark claim: models have taste, no moat — yuwen_lu_ · 2026-10-16
- PrismML ships Bonsai 2 ternary models: 9B fits in 2GB for local AI — gajesh · 2026-10-16
- Apple MLX Runs 816B Muse Spark Open at 36+ Tokens/s on Four Mac Studios — andrejusb · 2026-10-16
- vLLM Ships Day-0 Support for Meta's 816B Open-Weight Muse Spark Open — woosuk_k · 2026-10-16