audio_stream_end=True does not end a turn in the Gemini Live API

If you are building a voice agent on the Gemini Live API and it connects, streams audio, shows a healthy session and never answers, this post is the two hours you are about to spend.

The measurement first. One 5-second LibriSpeech clip, sent three ways to the same Vertex endpoint, same model, same session configuration. The only thing that varies is how the client signals that the caller has stopped talking:

What the client sent	Bytes returned
Audio only, no end signal	0
Audio, then send_realtime_input(audio_stream_end=True)	0
Audio, then 1.5 s of trailing silence as actual samples	229,994

229,994 bytes of 24 kHz 16-bit mono is about 4.8 seconds of reply. The middle row is the one worth staring at. The field is called audio_stream_end, you set it to True, the socket stays open, no error comes back — and the model does not respond. Ever. We waited.

The server ends a turn when its own voice activity detector hears silence. That is the only mechanism. So the silence has to reach it, as samples, at real-time rate. A client that simply stops sending is not a client that has finished speaking; it is a client that is indistinguishable from a dead one.

What the documentation says, which is two different things

We went looking for this in the docs after we measured it, and the docs are genuinely divided against themselves. All three quotes below are from Google's own published reference and guide.

The normative field reference, in BidiGenerateContentRealtimeInput, describes a stream state, not a turn:

"Indicates that the audio stream has ended, e.g. because the microphone was turned off. This should only be sent when automatic activity detection is enabled (which is the default). The client can reopen the stream by sending an audio message."

The Live API guide, in its automatic-VAD section, says the same thing in operational terms — flush, not finish:

"When the audio stream is paused for more than a second (for example, because the user switched off the microphone), an audioStreamEnd event should be sent to flush any cached audio. The client can resume sending audio data at any time."

And the same guide, in its hybrid-VAD section, says something quite different:

"The server treats the audio_stream_end signal as an immediate finalization prompt, bypassing the default server-side silence detection delay and returning the transcript and model response with minimal latency."

Those cannot both be the general behaviour of one field, and the third is the only sentence anywhere that says it produces a response. Meanwhile the SDK's own type for the containing message states the model's actual contract:

"End of turn is not explicitly specified, but is rather derived from user activity (for example, end of speech)."

There is one more thing we think is the real explanation for why this is so easy to get wrong. The Vertex documentation does not mention audioStreamEnd at all. We could not find the field on a single Vertex Live API page — and Vertex is where gemini-live-2.5-flash-native-audio lives, generally available, as the recommended model for low-latency voice agents. So on the platform we measured, the field is not documented as doing anything, and the paragraph that says it finalises a turn is on the other product's guide.

We are not claiming Google's docs are wrong. We are claiming that on Vertex, with automatic VAD at its defaults, on this model, on this date, the signal produced nothing and 1.5 seconds of silence produced a full response — and that if you read only the sentence about immediate finalization, you will build something that hangs.

We are not the first to hit it. python-genai issue #1328 reports exactly this, step for step: send audioStreamEnd when the microphone goes off, then "observe that the model does not end the user turn or react, even after waiting over a minute." That issue was closed by a stale bot in January with no fix and no confirmation, so treat it as corroboration from another developer rather than as an official statement — which is precisely why we are publishing a measurement instead of a link.

The invariant that fixes it

We wrote this into the service as a one-sentence contract, because everything else in the file is downstream of it:

Once a call is live, a continuous 16 kHz PCM16 stream reaches the API at real-time rate, and a pause is carried as actual silent samples — never as absent chunks.

That is not a style preference. It is the protocol, restated as something you can test. Every place in your code that could decline to forward a chunk is a place that can hang the call forever with the socket still open, and both of the places we had were bugs.

The second consequence is the one that catches browser clients, and it is not obvious: your client will stop sending audio during a pause whether you want it to or not.

• Chrome hands an AudioWorkletProcessor an empty input array whenever the upstream bus is flagged silent. Not a buffer of zeros — an empty array. Naive code forwards nothing.
• A muted track, or a device switch, stops the callbacks outright.

So "the user went quiet" and "the microphone stopped producing callbacks" arrive at your server as the same event, and the one thing the server-side VAD needs in order to answer is exactly the thing the browser has decided not to give you.

Our fix is a keep-alive that synthesises the silence the client should have sent and pushes it through the identical path — the same resamplers, the same model — so the pipeline stays coherent and the stream carries true silent samples rather than a hole. Two details matter more than the idea:

• Key it on client arrival, never on your own uplink. Our uplink is legitimately bursty, because the stream resampler emits in roughly 95 ms bunches. A keep-alive that watched its own output would read every burst gap as a stalled client and inject silence into the middle of live speech. Ours triggers after 150 ms of nothing from the browser, which is about nineteen missed 128-sample quanta: a stall, not jitter.
• Cap the fill per tick. A long stall gets repaired steadily rather than as one enormous late blob that arrives out of proportion to the conversation.

And report it. The amount of silence synthesised per call is a number in our health payload, not a hidden repair, because a session that is 40% synthetic silence is telling you something about the client that you want to know.

The interlock nobody expects

Coalescing came next, for a different reason — two arms of an A/B were being packetised differently, 10.6 against 125.0 frames per second — and both arms now assemble into fixed 100 ms frames before they are sent. Google's own guidance is "send small chunks (between 20 ms and 40 ms) to minimize latency", and 125 frames per second is far outside it in the other direction.

But a part-built frame is audio that has not been sent. So coalescing is only safe because of the keep-alive: without a producer behind the buffer, the end of the caller's last sentence sits in a half-full frame at exactly the moment the server needs it in order to detect that the caller has stopped. The keep-alive guarantees the buffer always has a producer, so a partial frame is always flushed by the silence that follows it.

Two independent-looking changes that are not separable. If you take the frame coalescing without the keep-alive you have built a subtler version of the same hang.

Make the turn-detection settings explicit

The last thing we changed was not code so much as visibility. This service used to leave every VAD parameter implicit, which is how "the agent never answers" was able to look like a mystery instead of a setting.

The documented surface, for reference:

Field	What the docs say
automaticActivityDetection.disabled	Defaults to false — server-side VAD is on
silenceDurationMs	"The server's internal default is approximately 800ms"; the guide recommends 500–800 ms
prefixPaddingMs	No default documented that we could find
startOfSpeechSensitivity / endOfSpeechSensitivity	The SDK states different defaults per platform — LOW on the enterprise branch, HIGH on the Gemini API branch
activityStart / activityEnd	"can only be sent if automatic (i.e. server-side) activity detection is disabled"

Two traps in that table. First, the guide's own configuration example uses prefix_padding_ms: 20 and silence_duration_ms: 100 — illustrative values, and 100 ms sits inside the range the same page calls "Too low", where "the system ends speech turns during natural pauses, splitting a single utterance into multiple small audio fragments." Do not copy the sample into production.

Second, manual activity detection is a different world, not a supplement. With disabled: true you send activityStart and activityEnd yourself, and — per the guide — "an audioStreamEnd isn't sent in this configuration." If you own the VAD, you own the turn boundary and the silence problem goes away; you have simply bought the harder half of the problem, which is detecting turns on a noisy line.

Our own choice is to expose all five as environment variables and per-call query parameters, and to leave them unset by default, so the status endpoint reports "vertex default" rather than a number the service invented. The two configurations we have measured working were measured at Vertex's defaults, and quietly moving them would change a result that is now proven.

The rest of the audio contract, since you will need it anyway

From the documentation, and it is worth getting right in one pass:

• Input: raw, little-endian, 16-bit PCM. "Input audio is natively 16kHz, but the Live API will resample if needed so any sample rate can be sent." Declare the rate in the MIME type of every blob, e.g. audio/pcm;rate=16000. The Vertex troubleshooting page adds the channel count: a single mono channel.
• Output: raw, little-endian, 16-bit PCM, always 24 kHz. Note the asymmetry — you send 16 kHz, you receive 24 kHz. Any downstream mixing or recording has to resample one of them.
• Chunking: 20 to 40 ms; do not buffer around a second before sending.
• Session length: without context-window compression, audio-only sessions are documented at 15 minutes and audio-video at 2. Separately, "the lifetime of a connection is limited to around 10 minutes due to WebSocket connection constraints", with a goAway notice sent 60 seconds before the end. Those are two different limits and the shorter one is the connection, so reconnection is a normal part of a long call, not an error path. We treat it as one: bounded, drop-oldest mic queue across the gap, so a reconnect does not "recover" by replaying a growing backlog and answering questions from ten seconds ago.

One caveat on where these docs live: the Vertex Live API pages have been rebranded and moved, and the old cloud.google.com/vertex-ai/generative-ai/docs/live-api path now redirects. If a link in your notes is dead, that is why.

Caveats

• This is n = 1 on the stimulus. One 5-second clip, three conditions, one model (gemini-live-2.5-flash-native-audio), one region, one date. It is a clean, reproducible demonstration of a behaviour, not a survey. We would not report the byte count as if it were a benchmark.
• The 0 / 0 / 229,994 result is a demonstration that the silence works and the flag did not, on that configuration. It does not establish that audio_stream_end never does anything, on any model, at any setting. If you can show it finalising a turn on Vertex, we want to see the configuration.
• The behaviour is measured on Vertex. The Gemini API branch documents different sensitivity defaults, and the hybrid-VAD paragraph that describes finalization is on that branch's guide.
• The community issue is corroboration, not confirmation. It was closed by automation with no resolution.
• We are not measuring quality here. Nothing in this post says anything about how well the model heard the caller. That is a separate experiment and the audio path in front of it is what we actually work on.

The short version

1. The server's own VAD is the only thing that ends a turn. Design for that and nothing else.
2. Carry pauses as samples. If your client goes quiet, synthesise the silence and send it through the same path as real audio.
3. Never key a keep-alive on your own uplink — a bursty transmit path will look like a stalled client and shred live speech.
4. Coalesce to a fixed frame, but only once something guarantees the buffer has a producer. The two changes are one change.
5. Publish a delivered-audio ratio per session. Uplink seconds over wall-clock seconds has a known correct value near 1.0, which makes a wrong value legible. Ours read 0.08 for days while the UI cheerfully reported a live session, and that is its own post.
6. Do not copy the VAD example values. Leave them unset, report what is in force, and change them deliberately.