Nemotron-3-Diarization: NVIDIA's open model for 'who spoke when'

NVIDIA has published Nemotron-3-Diarization, an open-weight speaker diarization model that answers one question about a recording: who spoke when. A HackerNoon overview dates the release to September 23, 2026, and lists 4,282 downloads. One checkpoint covers offline processing and streaming, with up to eight speakers.

The model follows the Sortformer family and orders its output channels by each speaker's first arrival: the first voice heard takes the first channel, which solves the speaker-permutation problem without fixed identities in advance. In streaming, an Arrival-Order Speaker Cache and a FIFO queue carry speaker information and recent frame context between chunks.

Streaming with configurable latency

Four recommended configurations are documented. The offline-style setup uses a 30.4 s input buffer and favours context, while streaming presets offer 1.04 s, 0.64 s and 0.32 s of input-buffer latency. Those are buffer-latency figures only: they equal (CHUNK_LEN + RIGHT_CONTEXT) x 80 ms and exclude model computation, device transfer and decoding. A single checkpoint reaches 80 ms, but 0.32 s is the lowest recommended operating point.

Output resolution is configurable in multiples of 10 ms, and chunked inference removes any stated maximum duration. The model returns segments with start and end times and a speaker index, plus optional probability tensors; the offline Transformers path outputs eight speaker channels at one frame per 10 ms.

Tooling, licence and open questions

NVIDIA NeMo Speech is the primary Python framework, exposing the model through the SortformerEncLabelModel class; it also runs via the Transformers AutoModelForAudioFrameClassification class. A lightweight C++ runtime, NeMo-Speech.cpp, combines transcription with diarization and emits word-level speaker tags as JSON. The recommended NeMo setup needs Python 3.12 or newer, Cython, PyTorch, libsndfile and ffmpeg.

The model ships under the openmdw-1.1 licence, and the card states it is ready for commercial and non-commercial use. No parameter count, training dataset, VRAM requirement, throughput or DER benchmark is listed, so those values must be measured in-house. Speaker indices reflect arrival order, not names, and the model stops at eight speakers.

Why it matters

Diarization is the missing half of most meeting-transcription workflows: ASR records what was said, diarization records who said it. One open checkpoint covering archived recordings and live meetings alike, with documented latency presets, lets teams prototype speaker-attributed transcripts without stitching two models together. Published accuracy benchmarks are absent, so the decision still rests on in-domain measurements.