RWKV-7 G1k ships: pure-RNN reasoning with no KV cache, 16M-state 13B model
cephaloform · x · 2026-10-04
The RWKV team (BlinkDL) released RWKV-7 G1k, a 100% RNN reasoning model with open weights and demo.
- All reasoning happens within a constant-size state (61×64×4096 ≈ 16M numbers for the 13B model), with no growing KV cache; the author calls this tiny state the model's entire "inner world model".
- RWKV combines RNN and Transformer strengths: linear time, constant space, fast training, unlimited context length, and is fully attention-free as a Linux Foundation AI project.
- The ecosystem is mature: 7B/13B web demos, GGUF and Ollama weights, RWKV-Runner GUI, 7B finetuning on 9GB VRAM, plus Albatross hitting 10,250+ tps for 7B fp16 on a 5090.
More from Infra
- Why AI labs won't push on-device models: cloud inference is their business — AccBalanced · 2026-10-04
- uv fork runs dev sessions ~33% faster with 40-60% less disk, author still unsure it's enough — mitsuhiko · 2026-10-04
- KohakuFA: Blackwell flash attention kernel fixes 10x-1000x gradient bugs in FA4/cuDNN — bdsqlsz · 2026-10-04
- Running a 27B Model on 16GB VRAM: NInfer 4080 Hits 262 tok/s Decode — roofkid · 2026-10-04
- NeoCloud Summit 2026 lands in SF Oct 8, gathering the GPU-native cloud ecosystem — AccBalanced · 2026-10-04
- Modal engineer built Gang Scheduler on K8s Reconciler model — it just worked — emilyzsh · 2026-10-04