vLLM explains looped Inkling internals: shared weights with separate per-pass KV state
vllm_project · x · 2026-11-21
In part 2 of its technical thread on supporting Thinking Machines' looped Inkling model, vLLM explains the architecture handling: "enough weight sharing to make an RNN blush."
- Each pass keeps its own KV cache and convolution state, indexed by pass—shared weights never merge these caches.
- Cross-pass communication flows through a gated message controller: previous-pass AttnRes message reads, gated (destination + message, blended in FP32), fed into the next sublayer input, then RMS-normalized to BF16 output.
- DSpark captures hidden states per layer visit, so repeated uses of one layer remain distinguishable.
More from Infra
- Thinking Machines releases Inkling 2 Mini: 26B looped MoE with 3B active params, 1M context — mervenoyann · 2026-11-21
- Ollama CEO in Fortune: enterprise demand for open models keeps growing — ycombinator · 2026-11-21
- Nvidia plunges 13.17% in a single day to $126.75 — jfischoff · 2026-11-21
- vLLM ships Day-0 support for TML model, cuts GB200 decode latency to 3.84ms — vllm_project · 2026-11-21
- vLLM ships day-0 support for Thinking Machines' new looped Inkling model — vllm_project · 2026-11-21
- Starcloud built a $2.5M space data center with the first H100 in orbit — chetanp · 2026-11-21