vLLM 推送式 KV 迁移：首 token 时间缩短约 3 倍，单 token 输出提速 30%
dl_weekly · x · 2026-10-28
- AWS engineering blog on disaggregated inference in vLLM: moves KV transfer coordination off the critical path with a push-based design.
- Results: 3x faster time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT).
- Covers implementation details of KV transfer coordination, directly relevant for teams running prefill/decode disaggregated serving.
「Infra」频道最新
- SK 海力士 Q3 营收 104.62 万亿韩元超预期,同比暴涨 328% — firstadopter · 2026-10-28
- Google TPU 首次进驻第三方超大规模云,Oracle 2027 年起部署 — rohanpaul_ai · 2026-10-28
- SK 海力士与美光齐声:HBM 存储供不应求将持续到 2030 年 — BenBajarin · 2026-10-28
- Epoch AI 算力版图图:美国份额升至约72%,中国占比持续收缩 — SamuelAlbanie · 2026-10-28
- Liquid Inference 日处理百亿 token,88.7% 输入命中缓存 — AccBalanced · 2026-10-28
- 民调：英国仅东北部支持本地建数据中心，整体净支持 -13% — SamuelAlbanie · 2026-10-28