Why turn detection breaks on TV audio
A television is a second speaker, and a voice-activity detector flags speech-like frames whatever voice they belong to. In our matrix Silero VAD marks 58.2% of non-target frames as speech when a competing voice sits at 0 dB, against 7.0% on clean audio.
Turn detection breaks on TV audio because a television is a second speaker, and a voice-activity detector flags speech-like frames whichever voice they belong to. In our matrix Silero VAD marks 58.2% of the frames that contain no target speech as speech when a competing voice sits at 0 dB SNR, against 7.0% on clean audio, and an agent that treats those frames as the caller talking will stop its own sentence.
The numbers below are from benchmark.json (generatedAt 2026-08-14): Silero VAD at threshold 0.5, 512-sample blocks, 19 conditions, ten speakers. We read "false alarm" as the share of reference-silence frames the VAD marked as speech; the field definition is in the harness.
The mechanism
A voice agent has two decisions that depend on the same signal. While the agent talks, the detector decides whether the caller has started to speak (barge-in). While the caller talks, it decides whether the caller has finished. Both are driven by a frame-level speech probability. A television, a radio or a second person in the room produces frames with high speech probability that are not the caller. The detector cannot tell the difference, because it was trained to find speech, not to find your speaker. The result is a false barge-in: the agent cuts itself off, or it waits for an end of turn that never arrives because the TV keeps the line "active".
We did not measure barge-in events directly. Our measurement is the false-alarm rate of the frame-level detector, which is the input to that decision. How a particular agent converts frames into an interruption depends on its own thresholds and hold times (assumption: the conversion is monotonic, so more false-alarm frames means more false interruptions).
What the VAD does on competing voices
| Condition | VAD false alarm, raw | VAD false alarm, focus v8_2 | VAD F1, raw | VAD F1, focus v8_2 |
|---|---|---|---|---|
| clean | 7.0% | 7.3% | 0.976 | 0.966 |
| noise_snr5 | 6.3% | 9.2% | 0.966 | 0.957 |
| babble_snr5 | 97.7% | 92.2% | 0.845 | 0.852 |
| competing_snr0 | 58.2% | 58.2% | 0.898 | 0.897 |
| competing_snr5 | 58.1% | 53.4% | 0.899 | 0.903 |
| competing_snr10 | 57.8% | 44.4% | 0.900 | 0.914 |
Measured 2026-08-14, 190 reference words per condition. F1 barely moves between clean and competing-voice audio, which is why a single F1 figure hides the problem: the detector still finds the target speech, and it also finds everything else.
Does isolation fix it?
Partly, and not everywhere. At 10 dB SNR the focus model cut the false-alarm rate from 57.8% to 44.4%; at 5 dB it went from 58.1% to 53.4%; at 0 dB it did not move. In the same competing_snr10 condition WER went from 6.3% to 12.1% and insertions from 2 to 7, so a lower false-alarm rate did not buy a better transcript there.
| Path | Mean VAD F1 (19 conditions) | Delta vs raw | Insertions |
|---|---|---|---|
| raw / passthrough | 0.9477 | +0.0000 | 103 |
| anecho focus v8_2 | 0.9413 | -0.0065 | 119 |
| ai-coustics Quail L | 0.9427 | -0.0050 | 128 |
| GTCRN | 0.9454 | -0.0023 | 223 |
| DeepFilterNet3 | 0.9011 | -0.0466 | 381 |
Pooled across all 19 conditions, no engine on the table improved mean VAD F1 over the untouched audio, ours included (0.9477 raw against 0.9413). The case for a split pipeline, with the VAD on one branch and the recogniser on another, is in the split pipeline; these numbers are the reason it is a design to be measured and not a switch to flip.
What to do about it
Measure the false-alarm rate on a recording of your own room with the TV on, not on a clean test set. If it is high, raise the VAD threshold and the barge-in hold time before reaching for a model, and treat any isolation model as something to be scored against raw audio, as every row above was.
FAQ
Why does turn detection fail on TV audio? A VAD detects speech, not a speaker, so TV dialogue produces the same speech frames as the caller. In our matrix those frames raise false alarms to 58.2% at 0 dB.
Does speaker isolation stop false barge-in? It lowered the VAD false-alarm rate at 5 and 10 dB in our run and not at 0 dB, and it did not improve pooled VAD F1. We did not measure barge-in events themselves.
Is a better VAD enough? We tested one VAD, Silero at threshold 0.5. We make no claim about other detectors.