AMD and Inferact bolster vLLM agentic inference on Instinct GPUs, already in production
woosuk_k · x · 2026-10-17
Inferact announced an engineering collaboration with AMD to strengthen production agentic inference on AMD Instinct GPUs in vLLM. AMD brings the Instinct platform, ROCm engineering, and the AITER kernel library, while Inferact and the vLLM community handle model integration, serving features, and day-0 enablement.
The work targets agentic workloads' pain points: long-running sessions, long prefixes, and tool calling, with each configuration shipping as a public per-model serving recipe. It's already in production — shipped with this summer's major open-model releases — and Inferact runs it on Instinct clusters for its platform customers.
zijingliu adds field experience: bootstrapping the cluster is painful, but it's a one-time entry fee — with the job done right, real-world AMD performance on frontier models is "pretty amazing."
Related event: Inferact Partners with AMD to Boost Agentic Inference on Instinct GPUs(2 posts)→
More from Infra
- Litespark serves Nemotron 120B at 962 tokens/sec per user on just two H200 GPUs — NVIDIAAI · 2026-10-17
- SF Signs Data Center Moratorium; Investor Kwok Asks "What Are We Even Doing" — kevinakwok · 2026-10-17
- Inferact, from the creators of vLLM, launches controllable frontier inference platform — Xianbao_QIAN · 2026-10-17
- AMD MI355X beats NVIDIA B300 by 24.4% throughput serving Kimi K3, 87% cheaper per token — vllm_project · 2026-10-17
- Anthropic: Claude Code responds 3x faster, Claude API 1.5x faster via gzip and infra cuts — ClaudeDevs · 2026-10-17
- Reddit user pairs Mac mini M5 Pro with old M4 as sub-agent for local coding agents — Calm-Set6265 · 2026-10-17