Ling-3.0 Architecture Revealed: Native Hybrid Attention with 256K Context
SonglinYang4 · x · 2026-07-24
AntLingAGi has revealed the architectural details of the upcoming Ling-3.0 model, which features a native hybrid-linear attention mechanism.
The core design involves stacking KDA layers (for fine-grained control over long-range memory) with MLA layers at a 5:1 ratio. This hybrid approach not only optimizes long-context handling but also significantly boosts MoE (Mixture of Experts) compute efficiency through a 1/64 expert activation ratio.
According to the post, Ling-3.0 natively supports a 256K context length, with the architectural capability to scale up to 1M.
More from Models
- Same Echo Maze prompt, three frontier models: all passed visually but shipped the same hidden bug — eyishazyer · 2026-09-11
- Benchmark scores drop from 89% to 19% on new evals — how benchmaxxing breaks leaderboard trust — airesearch12 · 2026-09-11
- ChatGPT tells user their question is too hard and to 'accept dumber answers' — phido3000 · 2026-09-11
- Claude is no longer available for minors as Anthropic rolls out age assurance — Muhammad523 · 2026-09-11
- Developer Building a Unified Leaderboard of All Model Benchmark Scores — airesearch12 · 2026-09-11
- Rumor claims Kimi faked performance by serving Claude; DeepSeek new model surprises in evals — realsohamparekh · 2026-09-11