SGLang and Miles ship day-0 support for DeepSeek-V4.1 Flash and detail its new architecture
ying11231 · x · 2026-09-10
LMSYS announced day-0 support for DeepSeek-V4.1 Flash in SGLang and its RL framework Miles, with a technical blog dissecting the architecture.
Key points:
- Cross-layer shared compressed KV: fp8 sliding-window cache (128 recent positions) per layer plus fp4 compressed representations for long-range attention; queries attend to the local window and up to 512 indexer-selected compressed positions, shared across layers.
- Two-stage sparse indexer and mHC (manifold hyper-connections): each sublayer reads/writes four parallel residual streams via token-dependent mixing coefficients that overlap with attention/FFN compute.
- Engram lookup memory: layers 1 and 14 retrieve fp8 table rows keyed by hashed token n-grams, gated into the residual stream; blog covers host-memory placement and performance eval.
- Miles RL: a validated reinforcement learning run on day one.
Model specs: 552B backbone (16B active decode / 8B prefill), natively multimodal, up to 1M context. More performance upgrades promised in coming days.
More from Infra
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26
- US states halt new data centers as 72% of Americans oppose local builds — YvesMulkers · 2026-10-26
- $5M Seed Round Can't Afford One DeepSeek Training Run: GPU-less AI Startup's Three Paths — FinanceYF5 · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26