Guide · architecture

Your VAD and your transcriber want different audio.

Most voice stacks apply one filter to one track and hand the result to everything downstream. That is a routing decision disguised as a default, and it is usually the wrong one. Here is the alternative, and how to wire it.

One capture, two outputs. Turn-taking gets the clean channel because a suppressor removes exactly the background speech that false-triggers barge-in. Transcription gets the unprocessed channel because the same suppressor removes acoustic detail the recogniser was trained on. You can override either route per session — but this is the default, and defaults are the argument.

This is not a hunch. Every engine below raised word error rate on our test set and several of them cut the voice-activity false-alarm rate by twenty to thirty points in the conditions that break barge-in. Same audio, two consumers, opposite verdicts.

Split pipeline · measuredSame audio, two consumers, opposite verdicts

Effect of enhancement on transcription versus on voice activity detection, per engine.
Engine	Transcription			Turn-taking (Onset)
Engine	WER raw → enh	ΔWER	Ins raw → enh	F1 raw → enh	FA Babble 5 dB	FA Competing 5 dB
ai-coustics Quail L	14.8→15.6	+0.79	101→126	0.950→0.945	98→96	58→58
ai-coustics Quail VF 2.2 L	14.8→18.1	+3.33	101→161	0.950→0.945	98→75	58→37
Resample-only 16→8→16 (control)	14.8→18.6	+3.86	101→173	0.950→0.948	98→97	58→55
GTCRN	14.8→20.3	+5.50	101→210	0.950→0.947	98→92	58→57
FastEnhancer-L	14.8→20.4	+5.67	101→159	0.950→0.942	98→68	58→39
FastEnhancer-S	14.8→20.6	+5.88	101→159	0.950→0.945	98→85	58→56
FastEnhancer-B	14.8→21.1	+6.37	101→159	0.950→0.944	98→87	58→56
FastEnhancer-M	14.8→22.4	+7.63	101→217	0.950→0.941	98→75	58→48
FastEnhancer-T	14.8→22.5	+7.69	101→145	0.950→0.941	98→89	58→57
DeepFilterNet3 (96-frame warm-up)	14.8→31.6	+16.87	101→368	0.950→0.914	98→93	58→57
DeepFilterNet3	14.8→35.2	+20.41	101→380	0.950→0.901	98→92	58→57
Clearline (8 kHz, resampled)	14.8→39.6	+24.88	101→410	0.950→0.835	98→82	58→51

Every engine we tested raised word error rate. Several of them cut the VAD false-alarm rate by twenty to thirty points in the conditions that actually break turn-taking. That is not a contradiction — it is two different consumers with two different requirements, and it is why the SDK emits two channels instead of picking a winner on your behalf.

Raw → your transcriberEnhanced → Onset

Who wants what

Default channel routing per consumer
Consumer	Default channel	Reasoning
Onset · VAD and endpointing	Enhanced	Endpointers trigger on energy and spectral change. Background speech and broadband noise produce false starts, clipped turns and barge-in that fires at the wrong person.
Barge-in / interrupt detection	Enhanced	The failure you are avoiding is a second talker in the room stopping the agent mid-sentence. Suppressing that talker is exactly the job.
Speech-to-text	Raw, by default	Recognisers are trained on noisy audio and have their own robustness. A suppressor removes acoustic detail the acoustic model was relying on, and the errors show up as deletions of short unstressed words.
Recording / compliance	Raw	Processed audio is not evidence of what was said in the room. Archive the capture, not the interpretation.
Human listening / QA review	Enhanced	The one case where sounding better is the requirement.

Wiring it

node · @anecho/sdk

const session = await hear.createSession({
  model: "chamber-s",
  sampleRate: 16000,
  splitPipeline: true,
});

for await (const block of mic) {
  const frame = session.process(block);

  // Clean audio drives the conversation's timing…
  turnDetector.push(frame.enhanced);
  if (frame.speech && agent.isSpeaking) agent.interrupt();

  // …and the original drives what the words were.
  stt.push(frame.raw);
}

When to override the default

The default is an argument, not a law. Send enhanced audio to your transcriber when your own measurements say it helps. On our test set that was almost nowhere, and on band-limited speech nothing produced a meaningful gain in either direction. But that is one recogniser on one corpus. It is a measurement, not a rule, and it is different for every stack.

per-consumer override

const session = await hear.createSession({
  model: "chamber-s",
  splitPipeline: true,
  route: {
    stt: "enhanced",        // you measured it; it helps on your data
    turnTaking: "enhanced",
    recording: "raw",
  },
});

Before you flip it, run the same comparison we run: the comparator shows the shape of the effect, and Null Test shows it across the full matrix.

Cost of the split

One extra buffer per session for the delay line. Memory is a few kilobytes; there is no additional algorithmic latency, because the enhanced path already paid it.
Two consumers to feed instead of one. If your orchestrator assumes a single audio track, the plugins handle the fork for you.
One more thing to measure. That is the point — the routing is now a decision you can put a number on instead of a default you inherited.

The long-form argument Wire it into LiveKit