Nemotron-3.5-Transcribe lands native vLLM integration with FP8/NVFP4 and speculative decoding
aakashgupta · x · 2026-10-28
vLLM's native integration of NVIDIA Nemotron-3.5-Transcribe (PR #64282) is in review: a native audio tower replaces <|audio|> placeholders in the NemotronH decoder prompt, with FP8/NVFP4 checkpoint loading, OpenAI-compatible chat completions and /v1/audio/transcriptions endpoints, plus MTP and DFlash2 speculative decoding. Available today via NeMo Speech's vLLM 0.32.0 plugin.
Related event: vLLM Reviews Native Support for NVIDIA Nemotron-3.5-Transcribe(2 posts)→
More from Infra
- Harmonic's Aristotle Proves 185 Properties of a 12.2M-Gate Chip Block, Cuts Power ~26% — satnam6502 · 2026-10-29
- IT vendor lists $60M AI supercomputer with 9,600 A100 GPUs on eBay — free shipping included — giveen · 2026-10-29
- ENGY claims ~30% cheaper inference via Bittensor with no quality loss on open models — markjeffrey · 2026-10-29
- Pinokio author shares guide to saving tons of disk space with cache cleanup — cocktailpeanut · 2026-10-29
- Hugging Face's Text Embeddings Inference passes 5,000 GitHub stars, multimodal support coming — tomaarsen · 2026-10-29
- Lossless compression of HNSW proximity graph indexes cuts ANN search memory via SIMD codecs — _reachsumit · 2026-10-29