Everything we found out, in the order we found it.
We are asking an industry to accept our measurements, so the reasoning has to be visible. These are the working notes: what the published claims say, where they contradict each other, what our own harness measured, and what it still cannot tell you.
- 16 Aug 202617 min
Amazon Connect, Twilio, NICE CXone and Avaya: four media paths, two places to stand
We read the published specifications for the real-time audio surfaces of four contact-centre platforms and asked one question of each: can a third party receive the audio, process it, and have the processed audio be what the platform's own downstream consumers hear? Twilio and Avaya document yes. Amazon Connect and NICE CXone document no. Along the way: Amazon publishes 8 kHz and 'raw PCM' and nothing else, Avaya is the only one of the five documenting a wideband codec, and Twilio never says the thing we have been saying they say.
Anecho Engineeringtwilioamazon-connectnice-cxoneavaya - 16 Aug 202613 min
We put our own model in our benchmark and it lost
Rule 2 of Null Test is that we score ourselves in the same tables under the same rules. So we did. Clearline posts 39.6% pooled WER against raw's 14.8% — last place in a matrix of twelve, behind every competitor and behind a resample-only control with no model in it. Here is the number, the error decomposition, the control that separates the band limit from the model, and why our own corpus said the opposite.
Anecho Engineeringnulltestbenchmarkclearlinenegative-result - 16 Aug 202611 min
We trained a speaker-isolation model on an assumption that does not hold in real calls
Every clip in our corpus had the target speaker talk alone first. Real calls do not — the television is already on when you dial. With a background running at t=0 the model's speaker embedding locks onto the background in 96% of clips: −14.63 dB SI-SDR, 86% of clips made worse, and 131.9% WER against 121.9% for not running the model at all. Splicing 0.8 s of target-alone speech onto the front recovers it to +4.91 dB. A 19.5 dB swing from timing alone, on the same audio.
Sofia Marchetticlearlinespeaker-isolationtraining-datanegative-result - 16 Aug 20269 min
The resampler that ate 92% of a microphone
soxr.ResampleStream is a burst emitter. Fed the browser's 128-sample AudioWorklet quantum it returns an empty array on 92% of calls and then hands back 95 ms at once. We read that emptiness as 'still priming' and skipped the block — which also skipped the untouched microphone buffer sitting next to it, so 8% of the caller ever reached the model. Our own telemetry reported it as 0.08 for days. This is a control-flow bug, not the resample-tax claim we retracted, and here is the four-line probe that finds it.
Daniel Reissdspresamplingvoice-agentsdebugging - 16 Aug 202612 min
Genesys AudioHook is 8 kHz µ-law, and there is nowhere to stand
The Genesys Cloud AudioHook protocol pins its sample rate in the type system: MediaRate is the literal 8000, and 16000 appears nowhere in the specification. AudioHook Monitor is a one-way tap whose server output is explicitly discarded; Audio Connector can play audio back, but it is a bot fork inside the IVR that pauses the flow and never reaches an agent. A spec-by-spec reading of what a third party can and cannot do in that media path, with the four things we could not confirm named as unconfirmed.
Sofia Marchettigenesystelephonycontact-centrenarrowband - 16 Aug 202610 min
audio_stream_end=True does not end a turn in the Gemini Live API
Measured on Vertex against gemini-live-2.5-flash-native-audio with one 5-second clip: audio alone returns 0 bytes, audio plus audio_stream_end=True returns 0 bytes, and audio plus 1.5 seconds of trailing silence returns 229,994 bytes. The server ends a turn when its own VAD hears silence, so your uplink has to carry a pause as real samples and never as absent chunks. Google's own documentation says two different things about this field, and the Vertex docs do not mention it at all.
Daniel Reissgemini-livevoice-agentsvadturn-taking - 13 Aug 202614 min
Null Test v0.1: ten speech enhancers, eighteen conditions, and nothing beat raw on the pooled average
The harness, the dataset, the configs and now the results. Ten enhancement engines scored against a raw control on WER, error decomposition, VAD and cost. Raw won on pooled WER — but at least one engine beat raw in 11 of the 18 individual conditions, and insertions went up rather than down.
Anecho Engineeringnulltestbenchmarkmethodologyopen-source - 09 Dec 202511 min
The split pipeline: why your VAD and your STT want different audio
Turn detection wants a clean stream. Transcription wants the raw one. We now have our own numbers for both halves of that claim: enhancement cost us five points of pooled WER and cut VAD false alarms by a third — while beating raw outright in 11 of 18 individual conditions.
Daniel Reissarchitecturevadturn-takingsdk - 18 Nov 202511 min
Does noise suppression actually help speech-to-text? We measured it.
Krisp markets a 46% WER reduction. We ran ten enhancement engines across eighteen conditions against a raw control. None beat doing nothing on pooled word error rate, though at least one engine did beat raw in 11 of the 18 individual conditions — and insertions went up rather than down.
Anecho Engineeringspeech-to-textnoise-suppressionbenchmarkswer