Does noise suppression actually help speech-to-text? We measured it.

Updated 14 August 2026. The original version of this post argued that the field was under-measured and described the harness we were building. The harness has since run twice: the illustrative example that used to sit in the middle was replaced by real numbers, and those numbers have now been regenerated across eighteen conditions rather than thirteen. Absolute values moved, the ordering did not, and every figure here is quoted from the published benchmark.json.

Two vendors market speech enhancement as a way to cut speech-to-text error rates roughly in half. A major STT provider tells its customers to switch it off. If you are shipping a voice agent, one of those camps is costing you accuracy or money, and when we first wrote this, nobody had published the evidence that settles it.

So we built the harness and ran it. The short answer, on our data: no enhancement engine we tested beat raw audio on pooled WER, the errors it added were mostly the recogniser inventing words that were never said, and there is a substantial band of conditions where the vendor claim holds up cleanly — 11 of our 18 had an engine that beat raw. All three of those deserve more than a headline, and the third is the one that gets dropped when this post is quoted.

The claims that cannot all be true

Source	Claim as published	What we read it as measuring
Krisp	roughly 46% WER reduction from its noise cancellation	vendor-run evaluation; conditions, corpus and STT engine not independently reproducible
ai-coustics	roughly 43% WER reduction	same shape of claim, different model family and corpus
Chondhekar et al., arXiv:2512.17562	"Original noisy audio achieves lower semWER than enhanced audio in all 40 tested configurations", with degradations of 1.1%–46.6% absolute	independent academic work — but note the scope: MetricGAN+ only, a non-streaming 2021 research model, on medical speech, scored in semantic WER. Not a production real-time enhancer.
AssemblyAI	cited the study above, and separately reported that Krisp's noise cancellation cut false VAD triggers ~3.5x while roughly doubling WER when its output was fed to STT	an STT vendor evaluating enhancement as a preprocessing step in front of its own models
Deepgram	published the position that enhancement hurts STT accuracy	a stated position; we have not found accompanying data

Read those rows carefully and the disagreement is smaller than it looks. Nobody is running the same corpus, the same acoustic conditions, the same enhancement configuration, or the same recognizer. There is no shared control. Every number in that table is defensible inside its own experiment and useless for comparing across them.

That is not a scandal. It is what a field looks like before it has a benchmark.

Five hidden variables that flip the sign

Each of these is enough on its own to turn a 40% improvement into a 40% regression.

1. The enhancement objective is not the STT objective. Suppression models are trained against signal-level or perceptual targets: SI-SDR, PESQ, STOI. Those reward a waveform close to a clean reference and pleasant to a human ear. None of them reward preserving the specific cues an acoustic model uses to discriminate phonemes. A model can improve PESQ by half a point while smearing the exact fricative energy that separates "six" from "fix". The loss function was never asked about that.

2. The recognizer is a hidden variable. Modern STT models are trained on enormous quantities of real-world audio, much of it noisy. Noise is in-distribution for them. The artifacts of a neural suppressor — spectral holes, musical noise, transient smearing, an unnaturally silent noise floor between words — are not. A recognizer trained on messy real audio can be more robust to the original noise than to the cleanup.

3. The condition is a hidden variable. On already-clean audio, enhancement can only remove information; the best possible outcome is a tie. The case for enhancement lives at low SNR, where noise is genuinely destroying phonetic cues. A benchmark weighted toward clean recordings will conclude enhancement is harmful; one weighted toward 0 dB café babble will conclude it is essential. Both are "measuring WER".

4. Double processing. By the time audio reaches your agent it has often been through a handset noise suppressor, a codec, and a platform-level canceller. Adding another suppressor on top is not one enhancement pass, it is a cascade nobody characterised. This is especially true on telephony paths, covered in 8 kHz is where voice AI actually breaks.

5. Aggregation. WER over a corpus is dominated by its hardest files. A single scalar averaged over conditions can be flat while hiding a large win in one condition and an equally large loss in another.

Aggregate WER hides the mechanism

WER is (S + D + I) / N — substitutions plus deletions plus insertions, over reference tokens. It is one number carrying three different failure mechanisms, and for a voice agent those mechanisms are not equally bad.

Over-aggressive suppression has a characteristic signature. It tends to delete rather than substitute, and what it deletes first are short unstressed function words — "a", "the", "of", "to", "not" — because they carry low energy and short duration, and because gating decisions built around a speech-presence probability treat them as noise.

Untreated noise has a different signature: substitutions where a phoneme is masked, and insertions where background speech or transient noise gets decoded as words. Attention-based models are particularly willing to hallucinate fluent text out of babble.

So two systems can post identical WER and be in completely different trouble. A pipeline whose errors are insertions of background chatter is annoying but often survivable, because a downstream LLM discards the incoherent fragment. A pipeline whose errors are deletions of function words is dangerous, because dropping "not" from "I do not want to renew" produces a fluent, confident, inverted sentence the LLM will act on. This is why we record insertions, deletions and substitutions separately in every cell, and why a WER-only claim — from anyone, including us — is not enough information to act on.

What we measured

Provenance first, because these numbers are worth exactly what the method is worth. Single recogniser, faster-whisper base.en (CTranslate2 int8 CPU, greedy), Silero VAD, 10 speakers / 180 clips / roughly 1406 seconds of audio, about 190 reference words per condition and 3420 in total, 18 acoustic conditions, 12 backends, Apple M1 Max, onnxruntime 1.28.0 pinned to one intra-op thread. Pooled across all 18 conditions:

Backend	WER	Insertions	Deletions	RTF	Latency
Raw (no processing)	14.77%	101	60	—	0 ms
ai-coustics Quail L	15.56%	126	57	0.124	30 ms
ai-coustics Quail VF 2.2 L	18.10%	161	75	0.076	30 ms
GTCRN (MIT)	20.26%	210	40	0.050	16 ms
FastEnhancer-L (MIT)	20.44%	159	107	0.376	25.8 ms
FastEnhancer-M (MIT)	22.40%	217	83	0.110	22 ms
FastEnhancer-T (MIT)	22.46%	145	109	0.013	16 ms
DeepFilterNet3	35.18%	380	197	0.133	100 ms

No enhancer beat raw on pooled WER. The best result any engine posted was ai-coustics Quail L at 15.6% against raw's 14.8% — still worse, by eight tenths of a point, which at this sample size is a tie. Everything else was worse by five points or more, and those are not ties.

Insertions went up, not down. Raw produced 101. GTCRN produced 210. DeepFilterNet3 produced 380. This is worth dwelling on, because it is the precise opposite of the marketing claim. An insertion is not a mangled word; it is the recogniser emitting a word that nobody said. The industry story is that enhancement removes the noise that causes hallucinated text. On our data enhancement produced hallucinated text — roughly twice as much of it for GTCRN, close to four times for DFN3, against an unprocessed control. Our reading is that the suppressor's residue, the musical noise and the abrupt gated silences, is more decodable-as-speech than the noise it removed.

Note GTCRN's row in particular: 210 insertions but only 40 deletions, against raw's 101 and 60. It is not deleting your function words. It is adding words. Those are opposite failure modes and a single WER figure conceals which one you bought.

Where the vendor claim does hold

We would be doing exactly what we accuse the vendors of doing if we stopped at the pooled number. In 11 of our 18 conditions, at least one engine beat raw. The margins are real and sizeable, and they land where you would predict: where noise or a competing voice is genuinely destroying phonetic cues, rather than where the recogniser was coping fine on its own.

The clearest is speaker isolation. ai-coustics Quail VF is a Voice Focus model — its job is to isolate the primary speaker and reject other voices. On competing speaker at 5 dB it cut insertions from 19 to 10 and WER from 26.3% to 20.5%. On babble at 5 dB it cut WER from 24.7% to 20.5% with insertions down 10 to 7. Those are the strongest results any enhancer posted in our matrix, and they vindicate a narrow, well-targeted use. If your failure mode is a second human voice in the room, a Voice Focus model is the right tool and our data says so.

Low-SNR broadband noise shows the same shape: at 0 dB, eight of the ten engines beat raw, the best by 4.7 points. The general pattern, and the one sentence to take from this post: enhancement pays where the audio is genuinely bad and costs you where it is not.

The seven conditions with no winner are the mirror image and just as legible: the two reverberant ones, where every engine made things worse, and the five telephony carrier conditions, where in four of five the best an engine achieves is an exact tie with doing nothing.

What none of that vindicates is the general claim. Quail VF pooled across eighteen conditions lands at 18.1%, more than three points worse than doing nothing, because it also runs on clean, reverberant and narrowband audio where it can only remove information. A model that wins its design conditions by five or six points and loses the rest is not a preprocessing default. It is a routing decision, which is the entire argument of The split pipeline.

Caveats, in full

These belong attached to every number above.

• Single recogniser. faster-whisper base.en. A different recogniser gives different absolute numbers. The harness supports Deepgram and AssemblyAI, so the ordering can be re-checked on another engine, and we would like someone to do that.
• About 190 reference words per condition, 3420 pooled. Enough to separate large effects, not enough to resolve one or two points of WER: one word is worth roughly 0.5 points per condition, 0.03 pooled. Treat the raw-versus-Quail-L gap as a tie; the five-to-twenty point gaps below it are not ties.
• DeepFilterNet3 is measured through a block-online adapter, because no public ONNX export carries recurrent state tensors, making per-frame streaming impossible with the published weights. Its 35.2% is a lower bound on quality and an upper bound on cost, not a fair reading of the model. Do not quote that number without this sentence.
• ai-coustics rows are a proprietary baseline run through the licensed SDK. A reference point, never a shipping code path for us.
• Licensing. ESC-50 noise is CC-BY-NC-3.0, so those conditions are evaluation-only. LibriSpeech is CC-BY-4.0, and the babble and competing-speaker conditions are built from LibriSpeech alone, so they carry no NC restriction.
• onnxruntime pinned to one intra-op thread. The RTF column is one stream on one core, not a fan-out across the machine.

What we still do not know

• Whether any STT vendor trains on enhanced audio. If a recognizer has seen suppressor artifacts in training, its sensitivity to them is completely different, and none of us can see inside those training sets.
• How the enhancement models in the published vendor claims were configured. Aggressiveness is usually tunable, and the same model at two settings can land on opposite sides of zero.
• How much of this transfers to streaming STT. Our enhancement is driven block-by-block with state carried across blocks, but the recogniser is scored on complete utterances. Partial stability and latency-to-final are a separate measurement.
• Whether this holds outside English, or on a larger corpus. It will not necessarily. The manifest and the harness are public so that this is a question someone can answer rather than debate.

What we do with the result

Two things follow, and neither of them is "enhancement is useless."

The first is architectural. The same run measured VAD, and there the picture inverts: enhancement barely moves VAD F1 but cuts the false-alarm rate substantially in exactly the conditions that cause spurious barge-ins. That is a real, measured reason to send enhanced audio to your turn detector and raw audio to your recogniser, which is what our SDK does by default — the numbers and the reasoning are in The split pipeline.

The second is where our product went. On the narrowband conditions, no enhancer produced a meaningful improvement and several degraded badly — FastEnhancer-T by 6.8 points and DeepFilterNet3 by 9.5 — and that is where the call volume is. That is Clearline, and it is the thing we sell. Two honesty notes attached: our telephony conditions are a G.711 round trip over clean speech, with no echo, packet loss or handset acoustics, and no natively-8 kHz engine has been benchmarked yet, including ours. So this is a negative result about the existing field, not a positive result about us. The benchmark is not the product — a benchmark has no buyer and no retention. It is how we know what to build and how you check whether we are lying.

Every cell, including the ones we lose, is at /benchmark. The harness, dataset manifest and configs are at github.com/anecho, and the method is described in What we shipped: an open benchmark for speech enhancement. If you want a condition, an engine or a recogniser added, open an issue.