Kimi K3 reportedly reaches 1M tokens without explicit RoPE
chaitjo · x · 2026-07-29
Kimi K3 reportedly reaches 1M-token context length without explicit RoPE.
- The slide says K3 uses no explicit positional embedding, relying instead on KDA's recurrent gating and decay mechanism.
- Because of that design, the model can extrapolate directly to 1M-token contexts without RoPE scaling or interpolation.
More from Models
- From GPT-2 to KimiK3, a thread argues the story is bigger than scale — algo_diver · 2026-07-29
- Perplexity adds Kimi K3 to Search and Computer modes for Pro and Max users — ccerrato147 · 2026-07-29
- Users say Anthropic’s Opus 5 has become nearly unreadable after personalization changes — himanshustwts · 2026-07-29
- Scobleizer says Grok 4.5 is the best coding model right now — Scobleizer · 2026-07-29
- Apple reportedly sues OpenAI over alleged trade-secret misuse tied to future hardware — emmanuelvivier · 2026-07-29
- Kimi K3 and Qwen3.8 show Chinese AI is now a sustained competitive force — emmanuelvivier · 2026-07-29