Skip to content
anecho.ai
Get a key
All writing

How do you add speaker isolation to a LiveKit Agents worker?

Run the Anecho SDK on the inbound audio inside the worker's stt_node, in 240-sample hops at 16 kHz. There is no LiveKit plugin package, so this post shows the code you write yourself, with resampling notes and the numbers.

Anecho Engineering15 Oct 20263 min read
livekitintegrationsdkvoice-agents

You add speaker isolation to a LiveKit Agents worker by passing the inbound audio frames through the Anecho SDK before they reach the speech-to-text stage, in 240-sample hops at 16 kHz. There is no livekit-plugins-anecho package; we have not published one. What follows is the SDK contract applied to LiveKit's frame types, as code you own.

The contract

The SDK has three objects: Model (the .anecho file from your dashboard), Processor (float32 in, the same number of samples out, 15 ms behind) and UsageReporter (the licence heartbeat; it sends durations, never audio). The model works at 16 kHz on 240-sample hops. The first 240 samples of a stream come back as silence, which is the algorithmic delay.

The frame loop

In Agents, stt_node receives an async iterator of rtc.AudioFrame. We wrap it, re-chunk to hops, run the processor and yield frames of the same shape.

import os
import numpy as np
from scipy.signal import resample_poly
from livekit import rtc
from livekit.agents import Agent
from anecho import Model, Processor, UsageReporter

RATE, HOP = 16_000, 240          # model rate; 240 samples = 15 ms

reporter = UsageReporter(os.environ["ANECHO_API_KEY"])
model = Model.from_file("anecho.ai_focus_model_16khz_v8_2.anecho")  # from your dashboard


class IsolatedAgent(Agent):
    async def stt_node(self, audio, model_settings):
        async def isolated():
            proc = Processor(model, reporter.ensure_license())  # one per call
            reporter.attach(proc)
            reporter.start()
            pending = np.zeros(0, dtype=np.float32)
            async for frame in audio:                  # rtc.AudioFrame, int16
                x = np.frombuffer(frame.data, dtype=np.int16).astype(np.float32) / 32768.0
                if frame.num_channels > 1:             # mix down to mono
                    x = x.reshape(-1, frame.num_channels).mean(axis=1)
                if frame.sample_rate != RATE:          # e.g. 48000 -> 16000
                    x = resample_poly(x, RATE, frame.sample_rate).astype(np.float32)
                pending = np.concatenate([pending, x])
                while len(pending) >= HOP:
                    block, pending = pending[:HOP], pending[HOP:]
                    y = proc.process(block)            # same length out, 15 ms behind
                    pcm = (np.clip(y, -1.0, 1.0) * 32767).astype(np.int16)
                    yield rtc.AudioFrame(
                        data=pcm.tobytes(), sample_rate=RATE,
                        num_channels=1, samples_per_channel=HOP,
                    )

        return Agent.default.stt_node(self, isolated(), model_settings)

This is a sketch against the Agents 1.x Agent.stt_node hook. We have not benchmarked a live LiveKit worker, so check the hook signature against your Agents version and run it on your own traffic.

Resampling notes

  • LiveKit frames are int16 and commonly 48 kHz or 24 kHz. The model wants 16 kHz float32, so resample before the processor with a proper polyphase filter (resample_poly above), not sample dropping.
  • Resample once. If your STT also resamples, the isolated 16 kHz audio is already what most recognisers want.
  • A telephony leg arrives at 8 kHz. Upsampling to 16 kHz does not restore bandwidth; see 8 kHz is where voice AI breaks.
  • Keep a raw tap if you also record or run a second consumer; see the split pipeline.

Cost, and the platform alternative

ItemValueSource and date
Algorithmic delay15 msNull Test, 2026-08-14
Real-time factor, one CPU thread0.067Null Test, 2026-08-14 (Apple M1 Max)
Worst block time, p993.96 msNull Test, 2026-08-14
Our entry rate after 10,000 free minutes a month$0.0010 per minuteanecho.ai/pricing
LiveKit Cloud voice isolation, past plan quota$0.0012 per minutelivekit.com/pricing, retrieved 2026-10-15

LiveKit Cloud sells voice isolation as a metered line item, filled by Krisp and ai-coustics models (livekit.com/pricing and docs.livekit.io, retrieved 2026-10-15). If you run your worker on LiveKit Cloud and one of those slots fits, that is a reasonable path. Running the SDK yourself keeps the audio in your process and works wherever the worker runs. The two prices are not like for like: ours is per processed minute on a published ladder, theirs is per plan. Whether isolation helps your traffic at all is a measurement, not a default: how to evaluate speaker isolation.

FAQ

Is there an Anecho plugin for LiveKit Agents?

No. There is no published plugin package. The SDK is a Python and Node library, and this post shows the integration code.

Where in the LiveKit pipeline does the processor go?

On the inbound audio, before the speech-to-text stage, in the stt_node hook.

How much latency does it add?

15 ms of algorithmic delay, measured in Null Test on 2026-08-14 at a real-time factor of 0.067 on one CPU thread.

Next