vLLM's Transformers backend now serves text, image, audio and video with zero custom code
LysandreJik · x · 2026-10-06
vLLM's Transformers modelling backend now serves every modality — text, images, audio, and video — with zero hand-written vLLM modelling code. Models can be served directly via --model-impl transformers. Video support, contributed by Harshal Janjani, lands in vLLM v0.32.0, meaning any Transformers-supported model can now plug into vLLM's inference stack with no adaptation work.
More from Infra
- Nvidia has a moat but not control: Google's TPUs now hold a quarter of global AI compute — AryHHAry · 2026-10-06
- Local LLM reality check: 4090 caps Qwen context at ~32k before VRAM runs out — BLUECOW009 · 2026-10-06
- DDR5 Overclocking Boosts Out-of-VRAM MoE Inference by Up to 14% in Real Tests — EvolvingDior · 2026-10-06
- DeepSeek raises at least $12B, eyes IPO in early 2027, Bloomberg reports — teortaxesTex · 2026-10-06
- SEC filing: 80% of Anthropic's $518B compute commitments owed even if idle — rohanpaul_ai · 2026-10-06
- llama.cpp Metal PR boosts speculative decoding up to 3.6x on Apple Silicon — pmttyji · 2026-10-06