vLLM makers Inferact partner with AMD to optimize agentic inference on Instinct GPUs
woosuk_k · x · 2026-10-17
Inferact, founded by the creators of vLLM, announced an engineering collaboration with AMD to strengthen production and agentic inference on AMD Instinct GPUs in vLLM.
- Division of work: AMD contributes the Instinct platform, ROCm engineering and the AITER kernel library; Inferact and the vLLM community handle model integration, serving features and day-0 enablement.
- Focus workloads are agentic: long-running sessions, long prefixes and tool calling, with each configuration shipping as a public per-model serving recipe.
- Already live: the work shipped with this summer's major open-model releases, and Inferact runs it on Instinct clusters for its platform customers.
- Everything lands in open source.
More from Infra
- NVIDIA RTX Spark laptops launch with 128GB unified memory to run frontier AI locally — steipete · 2026-10-17
- Retail pours $3.8B into leveraged semiconductor ETF SOXL in record week — toptickcrypto · 2026-10-17
- Strata revives decade-old Maxwell GPUs: local LLM at 150 tk/s prefill on M6000/M40 — jmuff98 · 2026-10-17
- Four GB10s Wired in a Ring, No Switch — 'A Token Ring?' — l4rz · 2026-10-17
- Intel ships LLM-Scaler vLLM PV 1.5 with major inference improvements — Fcking_Chuck · 2026-10-17
- Improved Qwen3.8-Flash chat template cuts answer tokens 40-61% with terse mode — returnity · 2026-10-17