Zvukový průvodce AI

Diarizace reproduktoru

Speaker diarization splits a recording into time segments and labels each one by speaker, answering the question of who spoke when without needing to know anyone's name.

Poslechněte si tento návodAI-vyprávěný · ~7 min

Na této stránce8 min čtení
  1. Přehled
  2. Klíčové věci
  3. Hluboký ponor
  4. Diarizing and scoring a five-minute meeting clip
  5. Real-World Implementace
  6. Budoucnost Speaker Diarization
  7. Uvedení do praxe
  8. Zdroje a další čtení
  9. Pokračujte v objevování
  10. Často kladené otázky

Přehled

It turns raw transcripts of meetings, calls and interviews into readable dialogue, and it is measured by diarization error rate, which still ranges from about 8% to 50% across public benchmarks for a leading open-source pipeline.

Klíčové věci

  1. Speaker diarization labels who spoke when in a recording with anonymous speaker tags; naming the speakers is a separate identification task.
  2. Typical pipelines run local segmentation, speaker embeddings and clustering, as described for pyannote.audio in a 2023 Interspeech paper.
  3. Diarization error rate adds missed speech, false alarms and speaker confusion, divided by total reference speech time.
  4. pyannote's speaker-diarization-3.1 reports DER from 7.8% on broadcast news to 50.0% on movie clips under strict scoring with no collar.
  5. A 2025 independent benchmark found missed speech was the largest error source and recordings with many speakers were the hardest.

Hluboký ponor

Speaker diarization is the task of working out who spoke when in an audio recording. The output is a list of time segments, each tagged with a speaker label such as SPEAKER_00 or SPEAKER_01. The labels are anonymous: diarization says that the same voice returns at 3:15 and 7:40, not that the voice belongs to a particular person. Naming speakers is a separate task, speaker identification, which compares voices with enrolled samples. Diarization is usually paired with automatic speech recognition, so a transcript of a meeting or phone call can show turns of dialogue instead of one undivided block of text.

The standard approach is a pipeline. Hervé Bredin's Interspeech 2023 paper on pyannote.audio 2.1 describes three stages that are typical of modern systems. First, a segmentation model slides a short window over the audio and decides, frame by frame, whether anyone is speaking and how many local speakers are active. This folds in voice activity detection, separating speech from silence, music and noise. Second, a speaker embedding model turns each local speaker's speech into a fixed-length vector, a list of numbers designed so that two clips of the same voice land close together and clips of different voices land far apart. Third, a clustering algorithm groups those vectors across the whole recording, so speaker 2 in minute one and speaker 1 in minute ten can be recognized as the same person. Agglomerative clustering, which starts with every segment as its own group and merges the closest pairs, is the method pyannote uses.

Overlapping speech, when two people talk at once, is the hardest part. Older pipelines assigned each moment to one speaker and simply lost the second voice. In 2019 Yusuke Fujita, Shinji Watanabe and colleagues proposed end-to-end neural diarization (EEND), a single network that outputs, for every frame, an independent yes or no for each speaker, which allows overlap. Because speaker labels have no natural order, they trained it with a permutation-free objective that scores the best matching between predicted and true speakers. On simulated mixtures it reached 12.28% diarization error rate against 28.77% for a clustering-based baseline.

pyannote's current segmentation model takes a further step. Alexis Plaquet and Hervé Bredin's 2023 paper replaced the per-speaker yes or no outputs with powerset classification, in which each class is a specific combination of active speakers. The segmentation-3.0 model card lists seven classes for each 10-second chunk: no speech, each of three speakers alone, and each of the three possible pairs. The authors report better results on 9 benchmarks, mostly on overlapping speech, and removal of the detection threshold that the older formulation depended on. The speaker-diarization-3.1 pipeline combines this model with embeddings and clustering, runs in pure PyTorch, and is released under the MIT license.

Performance is measured with diarization error rate (DER), the share of reference speaking time that is wrongly handled. According to the dscore toolkit, which follows NIST's RT-09 evaluation plan, DER is the sum of three errors: missed speech, false alarm speech where silence was labeled as speech, and speaker confusion where speech went to the wrong speaker. Before scoring, system labels are mapped to reference labels so the arbitrary names do not count against the system. Two settings change results a lot: a forgiveness collar, a short window around each boundary that is not scored, and whether overlapping speech is included. pyannote reports its numbers with no collar and with overlap scored, the strictest setup.

Those strict numbers show how much the domain matters. The speaker-diarization-3.1 model card reports DER of 7.8% on the REPERE broadcast set, 11.3% on VoxConverse, 12.2% on the AISHELL-4 meeting corpus, 18.8% on AMI headset mixes, 21.7% on DIHARD 3 and 50.0% on AVA-AVD, a set of movie clips. A September 2025 independent benchmark by Lanzendörfer and colleagues evaluated five systems on 196.6 hours of audio in five languages and found the commercial PyannoteAI model best at 11.2% DER, with the open-source DiariZen at 13.3%. It found most errors came from missed speech, followed by speaker confusion, and that recordings with many speakers were hardest.

Common failure modes follow from the pipeline. Short interjections such as yeah or right are often missed or given to the wrong person, because a fraction of a second carries little voice information. Similar voices, such as relatives, get merged. A single speaker who changes microphone, moves across a room or raises their voice can be split into two. Background music, television and crosstalk create false alarms. Errors also compound: the 2025 benchmark notes that diarization mistakes spread to downstream uses such as transcription and summaries, so a confused speaker turn can put words in the wrong mouth.

In practice, pyannote pipelines run in PyTorch and can be moved to a GPU with one call, which matters for long files. pyannote's pipelines are gated on Hugging Face, so users must accept the conditions and use an access token before downloading. Audio is downmixed to mono and resampled to 16 kHz on loading. Supplying the number of speakers when it is known, or a minimum and maximum, can reduce clustering mistakes. Because accuracy varies so much by domain, the reliable way to choose a system is to hand-label a few minutes of your own audio and measure DER with a fixed collar and overlap setting.

04Zpracovaný příklad

Diarizing and scoring a five-minute meeting clip

  1. Pick a five-minute recording with three or four speakers and some crosstalk, and convert it to 16 kHz mono WAV.

  2. Listen through once and hand-label speaker turns in an RTTM file, the plain-text format diarization scorers read, including overlapping regions.

  3. Accept the model conditions on Hugging Face, then run pyannote's speaker-diarization-3.1 pipeline on the clip and save its output as RTTM.

  4. Score the output against your labels with dscore, once with no collar and once with a 0.25 second collar, and note the missed speech, false alarm and confusion parts.

  5. Re-run the pipeline with the true number of speakers supplied and score it again.

  6. Listen to the segments with the largest errors and classify them as overlap, short interjection or speaker mix-up.

Co ukazuje

Two or three DER figures and a list of error types for your own audio. Expect the collar to lower DER and a known speaker count to reduce confusion; if most error is missed speech in overlaps, that matches the 2025 benchmark's finding.

Real-World Implementace

Producing a speaker-labeled meeting transcript by running pyannote's open-source speaker-diarization-3.1 pipeline, which accepts 16 kHz mono audio and needs no manual speaker count, alongside a speech recognizer.

Adding Speaker 1 and Speaker 2 labels to Whisper output with WhisperX, which calls a pyannote diarization model and attaches its speaker labels to the word-timed transcript.

Scoring a diarization system against hand labels with the dscore toolkit, which computes diarization error rate using NIST's md-eval scoring and the Jaccard error rate introduced for the DIHARD II challenge.

Building speaker-attributed speech-to-text with NVIDIA's Sortformer, a 2024 model released in the NeMo framework that feeds speaker labels into a transcription model's encoder.

Comparing systems across meeting, broadcast and in-the-wild audio using public sets such as AMI, AISHELL-4, DIHARD 3 and VoxConverse, the benchmarks reported on pyannote's model card.

Budoucnost Speaker Diarization

Several research directions are visible in recent primary sources. The September 2025 benchmark by Lanzendörfer and colleagues concluded that diarization remains unsolved, with missed speech the largest error source and many-speaker recordings the hardest, which keeps attention on overlap and detection rather than only clustering. Joint modeling of transcription and speakers is a second direction: NVIDIA's Sortformer, published at ICML 2025, feeds speaker labels into a speech-to-text encoder so that words and speaker turns are predicted together rather than stitched afterward. A third is evaluation itself. The gap between pyannote's 7.8% DER on broadcast news and 50.0% on movie clips shows that single benchmark numbers say little about a new domain, and scoring choices such as collars change results. Multilingual and far-field recordings, many-speaker scenes and long recordings with speakers who change acoustically over time remain difficult. Whether end-to-end models will fully replace modular pipelines is an open question, since current production tools still combine segmentation, embeddings and clustering.

Uvedení do praxe

Proč na tom záleží

  • Přístup a dosah. Zlepšuje dostupnost prostřednictvím přepisu, vyprávění a hlasových rozhraní.
  • Cena a rozpočet. Mediální týmy mohou dodávat vylepšený zvuk rychleji s menšími rozpočty.
  • Rychlost a měřítko. Systémy orientované na zákazníky mohou zpracovávat mluvené interakce ve větším měřítku.

Pozor na

  • Pokud chybí souhlas, zvyšuje se riziko zneužití hlasu a předstírání jiné identity.
  • Přesnost může klesat v přízvuku, dialektech nebo hlučném prostředí.
  • Syntetický zvuk lze bez jasného označení zaměnit za autentickou řeč.

Další kroky

  1. Získejte výslovný souhlas se zachycením hlasu, klonováním a opětovným použitím.
  2. Otestujte kvalitu napříč různými reproduktory a podmínkami pozadí.
  3. Definujte, kdy musí člověk zkontrolovat nebo schválit výstupy.
  4. Označte syntetický zvuk a veďte záznamy o původu pro zajištění odpovědnosti.

Zdroje a další čtení

  1. Hugging Face (pyannote)pyannote/speaker-diarization-3.1
  2. Hugging Face (pyannote)pyannote/segmentation-3.0
  3. ISCA (Interspeech 2023)pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe
  4. arXiv (Interspeech 2023)Powerset multi-class cross entropy loss for neural speaker diarization
  5. arXiv (Interspeech 2019)End-to-End Neural Speaker Diarization with Permutation-Free Objectives
  6. arXiv (ICML 2025)Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems
  7. GitHubnryant/dscore: Diarization scoring tools
  8. arXivBenchmarking Diarization Models

Pokračujte v objevování

Newsletter zdarma

Získejte denní přehled o AI

Tři ověřené příběhy ze světa AI každý všední den ráno, psané srozumitelnou angličtinou. Navždy zdarma, bez reklam.

Jeden e-mail každý všední den. Odhlásit se můžete jedním kliknutím. Vaši adresu nikdy neprodáváme ani nesdílíme.

Otestujte se

Take the Speaker Diarization quiz

Okamžitá zpětná vazba ke každé odpovědi a sdílitelný certifikát s ověřitelným ID po absolvování kurzu.

Spustit kvíz

Podpořte bezplatnou AI vzdělávání. AI Understanding je nezisková organizace 501(c)(3) — žádné reklamy, žádná platba za přístup, nikdy. Darujte

Často kladené otázky

Co je to diarizace mluvčích?

Speaker diarization splits a recording into time segments and labels each one by speaker, answering the question of who spoke when without needing to know anyone's name. It turns raw transcripts of meetings, calls and interviews into readable dialogue, and it is measured by diarization error rate, which still ranges from about 8% to 50% across public benchmarks for a leading open-source pipeline.

Speaker diarization is often summarized by which question?

Diarizace rozděluje zvuk podle reproduktorů v průběhu času a odpovídá „kdo kdy mluvil“ spíše než přepisem samotných slov.

Co je to vložení reproduktoru?

Vložení reproduktoru je vektor s pevnou délkou, kde jsou klipy od stejné osoby blízko u sebe, což umožňuje seskupení podle identity.

Která metrika se běžně používá k hodnocení diarizačních systémů?

DER kombinuje zmeškanou řeč, falešné poplachy a zmatení mluvčího do jediného procenta, což z něj činí standardní metriku diarizování.

Potřebuje standardní diarizace znát skutečná jména mluvčích předem?

Diarizace shlukuje hlasy a přiděluje anonymní štítky; nevyžaduje to znát, kdo ve skutečnosti jsou lidé podle jména.

What problem does permutation-invariant training solve in end-to-end neural diarization?

Speaker-output indices are arbitrary in diarization. Permutation-invariant training finds a matching between predicted and reference speaker streams so the loss does not depend on their index order; this is distinct from representing overlap with multiple active labels.