Amazon Connect, Twilio, NICE CXone and Avaya: four media paths, two places to stand

Every specification claim below is quoted from the vendor's own published documentation and linked. Where we could not confirm something it is marked unconfirmed rather than filled in, and there is a consolidated list near the end. This post also sharpens something we have previously stated too strongly about Twilio; that correction is in its own section. If any of this is wrong or changes, tell us and we will correct it with a date.

The companion post reads Genesys Cloud AudioHook the same way and finds a protocol that pins its sample rate in the type system and a media path with no insertion point in it. This one covers the other four surfaces our buyers actually run on, and the answer is less uniform than we expected.

One question, asked of each platform:

Can a third party receive the call audio, process it, and have the processed audio be what the platform's own bot, recogniser, recording or far end consumes?

Platform	Surface	Audio out to you	Processed audio back in	Documented rate
Amazon Connect	Kinesis Video Streams	Yes	No	8 kHz
Twilio	<Connect><Stream>	Yes	Yes	8000 Hz µ-law
Twilio	<Connect><ConversationRelay>	No — JSON only	No	not published
NICE CXone	Custom Agent Assist	Yes	No	8 kHz µ-law
NICE CXone	Custom Virtual Agent	Yes	Turn-based playback only	G.711, rate not stated
Avaya Infinity	Real-time Contextual Media Streaming	Yes	Yes	8 kHz, or 16 kHz on G.722

Two of these are genuine man-in-the-middle positions. The rest are taps, forks or bot turns wearing the same vocabulary.

Amazon Connect: a one-way fork, and two things everyone gets wrong

It is not started by an API. StartContactStreaming is the obvious candidate and it is the wrong one — its own reference says it "initiates real-time message streaming for a new chat contact", and its only configuration object is a chat streaming config pointing at an SNS endpoint. It is a chat API.

Voice media streaming is enabled at the instance and then turned on inside a flow:

"After you enable live media streaming, add Start media streaming and Stop media streaming blocks to your flow." — enable live media streams

The block offers two options — stream from the customer, or to the customer — and applies to voice only; every other channel takes the error branch. As far as we can find, there is no public API to start voice media streaming, only the flow block.

The format is 8 kHz, and almost nothing else is published. This is the sentence to quote:

"Media streaming uses Kinesis Video Streams multi-track support so that what the customer says is on a separate track from what the customer hears. Audio sent to Kinesis uses a sampling rate of 8 kHz." — plan live media streams

The tracks are AUDIO_FROM_CUSTOMER and AUDIO_TO_CUSTOMER, and the consumer guide says a reader "stores this data as a raw PCM file".

That is the whole documented format. We grepped the four live-media-streaming pages for L16, endian, bit and codec and got nothing. The commonly repeated "audio/L16, 16-bit, little-endian, mono" is not in the Amazon Connect documentation. The only bit-depth signal anywhere in AWS's own material is an Audacity import instruction in a demo repository — set encoding to signed 16-bit PCM and sample rate to 8000 Hz — which tells you what the bytes turned out to be, not what the contract is. Sixteen-bit mono is strongly implied. It is not documented, and we are not going to write it down as though it were.

Note also that Connect is the odd one out on encoding. Twilio and NICE both hand you companded G.711 µ-law; Connect hands you linear PCM at the same 8 kHz. Same band limit, different quantisation noise, and if you are benchmarking a model against one of them you are not benchmarking it against the other.

It is one-way, and AWS says so obliquely but unmistakably. Every documented verb is capture or consume, there is no write path, and the planning page contains this:

"We recommend that you refrain from modifying the streams. Doing so can cause unexpected behavior."

It is a live stream you read fragments from, with a short default retention. "If you select No data retention, data is not retained and is available to be consumed for only 5 minutes." Connect hands your consumer a fragment cursor through contact attributes — StartFragmentNumber, StreamARN, StartTimestamp, StopTimestamp — so your reader seeks into the stream rather than subscribing to a socket. That is an architectural difference worth planning for: it is a pull, not a push, and your latency budget includes whatever your consumer's polling loop costs.

What about the other Connect surfaces? We checked them because the naming invites confusion:

• Contact Lens real-time is transcripts, not audio. It delivers Utterance segments over Kinesis Data Streams — "partial transcripts... to meet ultra-low latency requirements to assist agents on live calls". Text, downstream of a recogniser you do not control.
• External voice systems ingest into Contact Lens over SIPREC — the configuration page names the host that "will receive the SIPREC audio". This is audio flowing into Connect from a third-party telephony platform, not out of it for processing. The codec specification is deferred to Amazon Chime SDK guides that we did not read, so the format there is unconfirmed.
• StartWebRTCContact returns Chime SDK meeting and attendee credentials — AudioHostUrl, SignalingUrl, JoinToken. That is how a client SDK joins the call as a participant. It is not a server-side raw-audio pipe.
• Third-party speech providers is the one place Connect documents somebody else's model in the path: "Connect routes audio to the chosen third-party speech-to-text provider", configured per bot locale with the provider's API key in Secrets Manager. Companion pages exist for third-party TTS. This is vendor selection from a supported list, not an insertion point for your own processing.

One flag for anyone writing about Connect at the moment: the documentation table of contents now carries an end-of-support page for Amazon Connect Voice ID. Check its status before you build on it.

Verdict: no. Kinesis Video Streams is a fork off to the side. You can listen to everything and change nothing.

Twilio: the strongest general-purpose insertion point of the five

Twilio publishes the tightest format contract of anyone here, and we verified these by reading the strings rather than trusting a summary. From the WebSocket messages reference:

"start.mediaFormat.encoding — The encoding of the data in the upcoming payload. Value is always audio/x-mulaw. start.mediaFormat.sampleRate — Value is always 8000. start.mediaFormat.channels — Value is always 1."

Three "always"es. Audio arrives as base64 inside JSON, with a chunk counter from 1 and a timestamp in milliseconds from stream start. The mechanism is a fork: "Twilio forks the raw audio stream of the Call and streams it to your WebSocket server in near real-time", over wss only.

Track selection — inbound_track, outbound_track, both_tracks, defaulting to inbound_track — belongs to the unidirectional form. Inbound is what Twilio receives from the other party; outbound is what Twilio generates toward the call.

The two forms are different products with one noun. This is the distinction that decides your architecture:

	<Start><Stream>	<Connect><Stream>
Direction	Fork out only	Bidirectional
TwiML flow	"immediately continues with the next TwiML instruction"	Blocks: subsequent TwiML runs only "after your server closes the WebSocket connection"
Created via REST	Yes	No
Track selection	Yes	Scoped to unidirectional in the noun reference

For the bidirectional form, audio you send back is played on the call:

"Bidirectional Media Streams are those in which your WebSocket application both receives audio from Twilio and can send audio to Twilio, which is then played on the Call." — Media Streams

With a strict payload rule and a trap in it: "The payload must be encoded audio/x-mulaw with a sample rate of 8000 and must be base64 encoded", and "the media.payload should not contain audio file type header bytes. Providing header bytes causes the media to be streamed incorrectly." If you generate µ-law with a library that writes a WAV header, you will hear it.

You also get mark for playback-completion tracking and clear to flush buffered audio — which is the barge-in primitive, and the reason this surface can support a real interruption model rather than just talking over the caller.

Verdict: yes. <Connect><Stream> is a genuine man-in-the-middle: caller audio in, your audio out, played to the call, with playback and interrupt control. The cost is that <Connect> is terminal — it owns the call for the duration — so you are not decorating a flow, you are becoming it.

One thing Twilio does not publish: the frame size. We looked specifically, because everyone quotes 160 bytes and 20 ms. Twilio documents the chunk counter and the timestamp and gives no bytes-per-message or milliseconds-per-frame figure on either Media Streams page. If you need that number, measure it on your own traffic and label it as observed.

Nor does Twilio publish a sample rate for real-time transcription. The <Transcription> noun documents track selection and labels and names Google or Deepgram as engines; we found no rate on the noun reference or the REST resource.

The Twilio claim we have been making, stated correctly

We have written, on this site, that Twilio's <ConversationRelay> structurally cannot run a <Stream> media fork alongside it. We went looking for the sentence that says so, and there isn't one. The correction matters more to us than the conclusion does, so here is exactly what the documentation does and does not contain.

What ConversationRelay is, confirmed. It "routes a call to the Conversation Relay service, providing advanced AI-powered voice interactions". Twilio performs the speech-to-text and text-to-speech itself, and your WebSocket exchanges JSON only: inbound setup, prompt (carrying voicePrompt, already-transcribed text), dtmf, interrupt, error; outbound text tokens, play-media by URL, send-digits, switch-language, end-session. No raw audio crosses that socket in either direction, and no codec or sample rate is published for it anywhere we could find.

What the documentation does not say, and we checked carefully:

• There is no cardinality rule on <Connect>. The noun reference documents four nouns and two attributes and never states that <Connect> may contain exactly one.
• There is no statement that <Stream> is incompatible with <ConversationRelay>. We grepped the ConversationRelay overview, the noun reference, the onboarding page and the <Connect> reference for "cannot", "not supported", "limitation", "incompatible", "only one", "simultaneously" and "alongside". Every hit was navigation chrome. No body text. There is no limitations section on any ConversationRelay page.
• Whether a <Start><Stream> fork begun before a <Connect><ConversationRelay> survives is neither documented as working nor documented as failing. Twilio simply never addresses the combination.

So the accurate statement is an inference, and we should have labelled it as one. ConversationRelay's protocol carries no audio, and <Connect> is documented as terminal, which together mean Twilio must be terminating and re-originating the media itself — leaving no documented place for a third-party processor between the caller and the recogniser. That reasoning is sound and it is not a citation. We have not run the empirical test of a preceding <Start><Stream>, and until we do, the honest form of the claim is: Twilio does not document the combination; the architecture implies there is nowhere to insert a processor; we have not verified it.

If you have run that experiment either way, we would genuinely like the result, and we will publish it with attribution.

NICE CXone: the documentation is not where you would look for it

The public developer portal lists eighteen API families and none of them is live audio. "Real-Time Data" is metrics. "Media Playback" is recordings. Searching there leads you to conclude that CXone has no real-time audio surface, and that conclusion is wrong — the specification lives in the administrator help centre, under agent-assist integrations.

Before the specifics, one negative finding worth having: we found no product called CXone Real-Time Audio, RTMS, CXone Voice Streaming API or CXone Open Agent Assist. Those names appear in third-party writing. They did not appear in a six-thousand-URL sweep of NICE's own documentation. Do not build a search on them.

Custom Agent Assist Integrations is the real-time audio surface, and its format sentence is unambiguous:

"Audio packets are encoded as G711 μlaw 8-bit 8000 kHz raw audio. This is the same format as all NiCE CXone telephony audio." — agent assist resources

The unit is NICE's own typo; 8000 Hz is plainly meant. The second half of that sentence is the more useful half: this is the same format as all CXone telephony audio. The platform is telling you its internal media representation is narrowband G.711, everywhere.

The channel model is per-connection rather than per-track, which is unusual and easy to get wrong:

"streamPerspective: RX — The audio being transmitted by the agent's phone: the agent talking. TX — The audio that the agent hears: the contact talking... MIX — Contains both agent and contact audio streams. An individual websocket connection only contains audio from one perspective."

So a stereo view of a call is two sockets, correlated by you, with no shared clock in the payload — because the payload has nothing in it but audio:

"Only binary data flows through the webhook. For voice interactions, the only data sent from the call are audio bytes. No call control or other metadata is included."

Operationally: at least 2,000 concurrent requests supported, no connection expiry or maximum duration, and a hold closes the socket while a resume opens a fresh one with an identical handshake. Note also that NICE permits an unsecured endpoint — "The audio relay endpoint must be a websocket. It can be secured (WSS) or unsecured (WS)" — which is a choice you should make deliberately rather than by copying a sample.

Direction: out only. What you return is text and resources rendered to the agent's screen. Audio never re-enters the call. The Agent Assist Hub, which brokers these integrations, therefore streams raw G.711 µ-law audio to the third party and expects the third party to do its own speech-to-text.

Custom Virtual Agent is the closest CXone comes to returning audio, and it is turn-based rather than streaming. Utterances arrive "either as audio in the format of the G-711 codec or as transcribed text", and the response schema carries a field named — with NICE's own misspelling intact — base64EndcodedG711ulawWithWavHeader, described as the encoded WAV "to be played at the next turn". Transport is REST over HTTPS. That is a bot reply, not an inline filter.

Verdict: no for agent assist, and only turn-based playback for virtual agent. You can hear everything, at 8 kHz µ-law, on separate sockets per perspective, and you cannot change what anyone else hears.

Avaya: the surprise, and the one wideband codec in the set

We expected to write that Avaya's real-time audio surface is not publicly documented. It is — in detail, with no login, and it is the most explicitly bidirectional design of the five.

Avaya Infinity publishes Real-time Contextual Media Streaming, last updated 23 July 2026:

"Real-time Contextual Media Streaming is an open WebSocket-based protocol that lets you connect your own AI services directly to Avaya Infinity... No proprietary SDKs. No middleware. Just a WebSocket connection carrying bidirectional audio and structured JSON messages."

The media contract, verbatim:

"Codecs: PCMU (8 kHz), PCMA (8 kHz), G.722 (16 kHz) / Frame size: 20ms default, configurable per session / Channels: Customer audio, agent audio, or both — negotiated at session setup"

That is the only wideband codec in this entire survey. Every other platform here, and Genesys, documents narrowband and only narrowband. Avaya documenting G.722 at 16 kHz does not mean your calls will be wideband — the carrier leg still decides — but it means the platform is not the thing forcing the band limit, which is a different situation from the other four.

The direction model is explicit and per-service:

"Egress (out) (Avaya Infinity → Your Server) — live audio from the endpoint, streamed to your server for processing... Ingress (in) (Your Server → Avaya Infinity) — audio generated by your server, played back to the endpoint." / "A Virtual Agent bot typically uses both flows... A recording service uses egress only. A TTS service uses ingress only. Your server controls which flows are active per endpoint."

Two transport modes are documented — avaya-wss, binary frames over a TLS WebSocket, and avaya-wss-rtp, SRTP over UDP "for ultra-low latency environments" — both using JSON for control and both supporting dynamic codec negotiation. Avaya is the WebSocket client and you are the server, so "no inbound firewall rules required on your side"; authentication is a signed JWT that your server validates; TLS 1.2 or better with public CA certificates. One socket multiplexes multiple endpoints with frames tagged by endpoint ID.

The caveat that keeps this honest: only Virtual Agent is live. The same page carries a "Coming Soon" table listing Agent Assist, Recording, Text-to-Speech, Speech Recognition, Transcription and Translation as future services on the same protocol. So today, the bidirectional path exists and the way to use it is to be the virtual agent. Do not read this section as "Avaya lets you insert third-party ASR or recording today", because the vendor does not say that.

Two further scoping notes. Avaya Experience Platform has no equivalent — its media handling is signed upload and download of files, not live audio; real-time streaming is on Avaya Infinity, the newer platform. And we did not investigate Avaya Aura, AES or Aura Media Server at all. That is an open question in this post, not a negative finding, and we would rather say so than let silence read as absence.

What this adds up to

Two independent conclusions, and they point in different directions.

On sample rate, the wedge holds and is slightly narrower than we have been saying. Four of the five surfaces document 8 kHz and nothing else, and NICE says outright that narrowband G.711 is "the same format as all NiCE CXone telephony audio". That is the channel the money arrives on, and essentially every speech enhancement model on the market is trained wideband — the DNS Challenge, the benchmark series most published denoisers are tuned against, never had a narrowband track. But we should stop writing "it is 8 kHz µ-law everywhere": Amazon Connect is linear PCM, not companded, and Avaya documents G.722 at 16 kHz. Two exceptions out of six surfaces is not a rounding error.

On integration surface, the field splits cleanly, and it is not the split the marketing suggests. Every one of these platforms will let you listen. Two of them — Twilio's <Connect><Stream> and Avaya Infinity's RCMS — document a way to put audio back into the live media path. Genesys, Amazon Connect and NICE CXone do not, and on the surfaces that do return audio, you are being asked to be the bot, not the filter: the audio you send goes to the human, not to somebody else's recogniser.

That distinction is the one to carry into a build decision. "Can I process the caller's audio before the platform's speech recogniser hears it?" is a different question from "can I send audio into this call?", and only one of them has a yes anywhere in this survey — on the platforms where you also own the recogniser.

Which is where our own numbers become the constraint rather than the platform's. On our seven telephony conditions, no enhancer we tested produced a meaningful improvement over the unprocessed audio, several degraded it by five to ten points of word error rate, and our own model did worse than every one of them. The insertion points are the easy part. Having something worth inserting is not solved yet, by us or by anyone whose numbers we can check.

What we could not confirm

• Amazon Connect: bit depth, endianness and channel count. 8 kHz and "raw PCM" is the entire published format. Sixteen-bit mono is implied by AWS's own demo material and is not documented.
• Amazon Connect: any latency figure. None published on any page we read.
• Amazon Connect: the SIPREC ingest codec profile. Deferred by AWS to Amazon Chime SDK configuration guides, which we did not read.
• Twilio: bytes or milliseconds per media message. Not published. The widely quoted 160 bytes / 20 ms is not in Twilio's documentation.
• Twilio: ConversationRelay audio format and sample rate. Not published anywhere we could find, including the voice-configuration page, which covers TTS providers and prosody settings and no codecs.
• Twilio: whether a <Start><Stream> fork survives into a subsequent <Connect><ConversationRelay>. Undocumented in both directions, and untested by us.
• Twilio: real-time transcription sample rate. Not published.
• NICE CXone: the Custom Virtual Agent audio sample rate. The codec is stated as G.711 µ-law; the rate is not stated on the schema page.
• NICE CXone: per-provider formats behind Agent Assist Hub. The integrations page does not restate them; the only format specification is on the custom-integration pages.
• Avaya: the wire-level protocol specification. Avaya publishes a protocol specification PDF, version 1.1, which we did not fetch — so session lifecycle and message definitions are unverified here.
• Avaya Aura, AES and Avaya Aura Media Server. Not investigated. Open question, not a negative finding.

A note on method, because it changes how much weight these findings carry: this survey was done by walking vendor-owned documentation indexes and reading the pages directly, not by searching. That is more reliable for confirming what a document says and less reliable for proving a document does not exist. Every "not documented" above means "we did not find it by crawling the vendor's own index", which is a weaker statement than "the vendor does not document it" — and weaker still than the confirmed quotes, which are verbatim.