vLLM PR Adds Native Nemotron-3.5-Transcribe Support: Transcription+Chat, FP8/NVFP4
vllm_project · x · 2026-10-28
A PR under review in vLLM adds native support for NVIDIA Nemotron-3.5-Transcribe, a speech-language model: the new Nemotron35TranscribeForConditionalGeneration architecture uses a native audio tower whose log-mel embeddings replace <|audio|> placeholders in prompts fed to the existing NemotronH decoder.
It loads the released 30B-A4B checkpoint plus FP8 and NVFP4 variants, serves OpenAI-compatible chat completions with audio and /v1/audio/transcriptions, and supports MTP and DFlash2 speculative decoding (including a small DFlash2 fix in shared spec-decode code), without touching the engine, scheduler, or API server. Meanwhile, NeMo Speech's vLLM 0.32.0 plugin already offers similar functionality today.
Related event: vLLM Reviews Native Support for NVIDIA Nemotron-3.5-Transcribe(2 posts)→
More from Infra
- Harmonic's Aristotle Proves 185 Properties of a 12.2M-Gate Chip Block, Cuts Power ~26% — satnam6502 · 2026-10-29
- IT vendor lists $60M AI supercomputer with 9,600 A100 GPUs on eBay — free shipping included — giveen · 2026-10-29
- ENGY claims ~30% cheaper inference via Bittensor with no quality loss on open models — markjeffrey · 2026-10-29
- Pinokio author shares guide to saving tons of disk space with cache cleanup — cocktailpeanut · 2026-10-29
- Hugging Face's Text Embeddings Inference passes 5,000 GitHub stars, multimodal support coming — tomaarsen · 2026-10-29
- Lossless compression of HNSW proximity graph indexes cuts ANN search memory via SIMD codecs — _reachsumit · 2026-10-29