Kimi K3 Architecture Drops RoPE for NoPE

Recent analysis reveals that the Kimi K3 model has completely removed Rotary Position Embedding (RoPE) in favor of NoPE. Despite this, its recursive KDA mechanism still implicitly encodes sequence order, while researchers also frame RoPE and ALiBi as KDA variants with complex or real logarithmic decay.

2026-07-28 ~ 2026-07-29 · 3 related posts

• Kimi Ditches Positional Encoding Entirely at Scale — tokenbender · 2026-07-28
• RoPE and ALiBi can be seen as KDA variants with imaginary or real log decay — teortaxesTex · 2026-07-28
• Kimi K3 drops RoPE for NoPE, but recurrent KDA still encodes order — cedric_chee · 2026-07-29