SGLang and Miles ship day-0 support for DeepSeek-V4.1 Flash and detail its new architecture
ying11231 · x · 2026-09-10
LMSYS announced day-0 support for DeepSeek-V4.1 Flash in SGLang and its RL framework Miles, with a technical blog dissecting the architecture.
Key points:
- Cross-layer shared compressed KV: fp8 sliding-window cache (128 recent positions) per layer plus fp4 compressed representations for long-range attention; queries attend to the local window and up to 512 indexer-selected compressed positions, shared across layers.
- Two-stage sparse indexer and mHC (manifold hyper-connections): each sublayer reads/writes four parallel residual streams via token-dependent mixing coefficients that overlap with attention/FFN compute.
- Engram lookup memory: layers 1 and 14 retrieve fp8 table rows keyed by hashed token n-grams, gated into the residual stream; blog covers host-memory placement and performance eval.
- Miles RL: a validated reinforcement learning run on day one.
Model specs: 552B backbone (16B active decode / 8B prefill), natively multimodal, up to 1M context. More performance upgrades promised in coming days.
More from Infra
- Matt Barrie burned 4B tokens in a day, cut his bill 500-fold, and now worries about $5T in debt — gaganghotra_ · 2026-09-10
- Analyst: DeepSeek's latest change is a big win for token efficiency, moving toward OpenAI's regime — teortaxesTex · 2026-09-10
- Acellera tests 7 LLM+harness combos on drug discovery: one RTX 5090 holds up — gdefabritiis · 2026-09-10
- Mac mini tested: local 35B runtime hits Haiku-level scores but falls short for agents — PawelHuryn · 2026-09-10
- Miles ships Day-0 RL support for DeepSeek-V4.1-Flash with KL held at 0.0012–0.0017 — ying11231 · 2026-09-10
- The data center is a symbol: why debunked claims about AI infrastructure still spread — ShakeelHashim · 2026-09-10