What is speaker isolation, and how is it different from noise cancellation?
Speaker isolation keeps one speaker's voice and removes everything else, including other people talking. Noise cancellation removes non-speech noise and keeps all speech. Here is the definition, a comparison, and what our own benchmark says about the two-voice case.
Speaker isolation keeps one chosen speaker's voice and removes everything else in the signal, including other people talking. Noise cancellation removes non-speech noise (fans, traffic, keyboards) and keeps all speech, so a second voice passes straight through it.
The difference in one table
| Property | Speaker isolation | Noise cancellation |
|---|---|---|
| Keeps | One speaker | All speech |
| Removes | Other voices, and non-speech noise | Non-speech noise (fans, traffic, keyboards) |
| Typical use | Voice agents, call centres, open offices | Conferencing, headsets, recordings |
The distinction matters for voice agents because a recogniser transcribes every voice it hears. Noise cancellation cannot tell the caller from the television behind them; both are speech. Isolation has to make that choice, which is a harder problem and a riskier one: if the model picks the wrong speaker, or damages the right one, accuracy goes down, not up.
What we measured
We score isolation the same way we score everything else, against a do-nothing control, in Null Test. The two-voice conditions put a second LibriSpeech voice under the target at a stated level. These are our numbers from the run of 2026-08-14 (recogniser faster-whisper:base.en, 10 speakers, 190 clips per engine, 16 kHz), from /benchmark.
| Condition (Null Test, 2026-08-14) | Raw WER | Anecho v8_2 WER |
|---|---|---|
| Two voices, 0 dB | 84.74% | 74.21% |
| Two voices, 5 dB | 26.32% | 31.05% |
| Two voices, 10 dB | 6.32% | 12.11% |
| Babble, 5 dB | 24.74% | 20.53% |
| All 19 conditions pooled | 14.32% | 16.59% |
Isolation helps where the second voice is as loud as the target (0 dB) and in babble, and it hurts at 5 dB and 10 dB, and pooled over all nineteen conditions. We wrote up the pooled result in our row is on the table; the short version is that an isolation model on by default, for all audio, is not what the data supports. The corpus is small and the caveats are in the methodology.
When the difference decides the choice
If your audio is one person in a noisy room, noise cancellation is the matching tool. If your audio is one person plus other people who should not be transcribed, only isolation addresses it, and you should measure it on your own traffic before turning it on everywhere. See how to evaluate speaker isolation.
FAQ
Is speaker isolation the same as speaker diarization?
No. Diarization labels who spoke when and keeps all the audio. Isolation changes the audio itself so that only one speaker remains.
Does speaker isolation improve transcription accuracy?
Not by default. In our 2026-08-14 run it lowered WER in two of the four conditions in the table above and raised pooled WER from 14.32% to 16.59%. Measure it on your own audio.
Can noise cancellation remove a second voice?
Generally no, because a second voice is speech and noise cancellation is designed to preserve speech.