Ling-3.0 Open Source: 6 Base Checkpoints Released with WSM Training Method
bclavie · x · 2026-08-19
Ant Ling has released 6 base model checkpoints for Ling-3.0-tiny and Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.
Key Highlights:
- WSM (Weighted State Merging): Replaces LR decay with weighted checkpoint merging. This suits continual pre-training better and allows offline exploration of different LR decay strategies.
- Shared Recipe: Researchers can validate strategies on tiny-base and scale them to flash-base.
The models have not undergone post-training, offering flexible starting points for continued pre-training and fine-tuning.
More from Models
- Grok Bot Ran a 12-Hour Data Analysis Nonstop, Handling API Rate Limits Itself — Daniel_Farinax · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- Users Report Claude Models Suddenly More Compliant After Forced Compaction — RealSharpNinja · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Users Say Krea 2 Output Degraded Sharply Since Krea 3 Beta Launched — szywis · 2026-10-16
- LLM-as-a-Verifier lifts Nemotron 3.5 Super by 9-10 points on agentic benchmarks — simonguozirui · 2026-10-16