NVIDIA Open-Sources Nemotron 3 Diarization, Tops Benchmark

On Sept 23, NVIDIA released and open-sourced Nemotron 3 Diarization, a model that tracks "who spoke when" even in overlapping multi-speaker speech. Despite only 100M parameters, it supports batch and streaming modes, tracks up to 8 speakers across 21 languages, and reaches RTFx 15000. Weights are on Hugging Face (retweets say ModelScope too) under a commercially friendly license, and it can be plugged into existing ASR workflows without replacing the transcription model, with day 0 Transformers support. It ranked first among 12 systems in voicearena.ai's first Diarization-Bench with a 14.72% error rate, about 24% lower than second place, and subsequently climbed Hugging Face's trending charts.

Confirmed

• Officially released and open-sourced by NVIDIA, continuing the Nemotron family's open-weights strategy; deployable in the cloud or locally. It is a streaming-sort-based audio frame classification model with 100M parameters (99.2M per one retweet), up to 8 speaker slots per session, outputting speaker labels and timestamps rather than transcripts.
• Unlike traditional pipelines chaining VAD, speaker embeddings, and other components, the model performs speaker labeling end to end, handles two people speaking simultaneously, and works with any ASR in batch or streaming mode.
• Official benchmark: first place among 12 systems in voicearena.ai's Diarization-Bench with a 14.72% error rate, roughly 24% below the runner-up.
• Developer kwindla shared the model card, weight links, and a full voice demo combining Nemotron 3.5 ASR and Pipecat Smart Turn.

Developer feedback

• @futterneid tested it locally with 1-second streaming chunks, saying it solves the "who is speaking" pain point and can help robots remember different speakers as new faces.
• @andimarafioti argued that distinguishing speakers is the key gap in current voice agents, masked by phone-call demos, and that the commercial-friendly license makes it practical for production.
• @Sam Witteveen published a video calling it a big leap over comparable models on the "who spoke when" task.

Why it matters

• Diarization is a long-standing weakness of real-time voice agents (meeting transcription, customer service, home robots), with prior open-source options struggling on overlapping speech.
• The combination of small size (100M), streaming output, and a commercial-friendly license lowers local deployment barriers — individual developers (per @futterneid's tests and kwindla's demo) can integrate it directly into their voice agent pipelines, plugging into any ASR without swapping the transcription model.

2026-09-23 ~ 2026-09-25 · 16 related posts

Full story(2 episodes)→

• Episode 1: NVIDIA Open-Sources Nemotron 3 Diarization, Tops Benchmark(2026-09-23, 16 posts)
• Episode 2: NVIDIA Nemotron 3 Diarization Launches on Baseten with Day-0 Support(2026-09-24, 2 posts)

• [source] NVIDIA releases Nemotron 3 Diarization model handling up to 8 overlapping speakers with 100M params — NVIDIAAI · 2026-09-23
• [source] Nemotron 3 Diarization tops Diarization-Bench with 14.72% error rate, ~24% lower than runner-up — NVIDIAAI · 2026-09-23
• NVIDIA open-sources Nemotron 3 Diarization to fix voice agents' speaker-blindness — andimarafioti · 2026-09-23
• [source] Nemotron 3 Diarization in action: streaming 8-speaker tracking powers a memory-equipped robot — futterneid · 2026-09-23
• NVIDIA Releases Nemotron 3 Diarization, Massively Outperforming Rivals — Sam Witteveen · 2026-09-23
• NVIDIA releases 99.2M-param streaming speaker diarization model that plugs into any ASR — alexcovo_eth · 2026-09-24
• NVIDIA open-sources Nemotron 3 Diarization model with full voice demo stack — iamrobotbear · 2026-09-24
• NVIDIA's Nemotron-3-Diarization speaker separation model hits HF trending — nvidia · 2026-09-24
• NVIDIA ships Nemotron-3-Diarization: 8 speakers, 21 languages, 15000 RTFx, top-ranked — kuchaev · 2026-09-25
• NVIDIA open-sources 100M-param diarization model that separates 8 speakers locally in real time — Prompt Engineering · 2026-09-25

6 near-duplicate retellings: NVIDIAAI · iamrobotbear · solyarisoftware · andimarafioti · udmrzn · nvidia