How GPT-Live Kills the Turn Detector: A System Design Teardown

Every voice assistant before this one had the same bug: it had to guess when you were done talking. Guess too soon and you cut the user off. Guess too late and the reply feels sluggish. OpenAI’s answer in GPT-Live isn’t a better guesser, it’s removing the guess entirely. The model listens and speaks at the same time, so there’s no turn detector left to get wrong.

That’s the headline, but the more useful part of OpenAI’s writeup is everything underneath it: a real production system with a protocol redesign, a hot-swap mechanism for multi-hour stateful sessions, and a capacity-planning lesson that generalizes way beyond voice. This is written as a study piece: each mechanism gets a plain explanation of the underlying pattern, so you can recognize it the next time you meet it in a different system.

• 6 round trips → 1 to start a session, via a custom protocol called WARP
• p95 latency on the new Go implementation ≈ p50 on the old Python system: same latency floor, far fewer bad outliers
• 75.7% human preference win rate over the previous Advanced Voice Mode
• Third-party testing found GPT-Live is actually ~498ms slower to go silent when a user interrupts it, a real cost of the full-duplex design worth sitting with
• Two independent state tracks per session: a speculative transcript for the UI and an authoritative one for analytics

The core move: delete the turn detector

Classic voice assistants are half-duplex: like a walkie-talkie, only one side transmits at a time. A separate turn-detection model watches the audio stream and decides “the user is done, go.” That decision is a single point of failure sitting on the critical path of every response, and it’s fundamentally a bet against incomplete information, since silence doesn’t reliably mean “finished.”

Instead of segmenting speech into discrete turns up front, GPT-Live’s application server watches partial transcripts and timing signals continuously and infers who has the floor after the fact. Turns are a derived view, not a gate you have to pass through before the model is allowed to respond.

Two paths, one clean boundary

The system splits into a live media path (audio in, audio out, has to be fast, no exceptions) and an asynchronous path (tool calls, web search, delegation to bigger models, can take as long as it needs). The voice model itself stays lightweight enough to keep talking in real time; anything heavy gets handed off.

Delegation isn’t free, so OpenAI pre-pays the cost before it’s needed: when a session starts, the application server opens an inference session with the frontier model and prefills it with the conversation context immediately, then holds that session open with stable affinity for the rest of the call, combined with prompt caching. By the time the user says something that needs GPT-5.5, the expensive part (processing everything said so far) is already done.

Keeping a multi-hour session alive without freezing it

Voice sessions run long, and the model’s context keeps growing the whole time. Eventually you hit two walls: the context window fills up, or you need to move the session to a different model version. A naive fix (pause, reload, resume) would produce an audible gap. GPT-Live avoids it with the same trick both times: run two model instances in parallel and hand off between them.

When a transition is needed, the system warms a replacement model instance alongside the one that’s live, prefills it with the current session’s context, runs inference against both simultaneously, and cuts over to the new one only once it’s fully caught up. Context compaction (summarizing an overlong conversation down to fit the model) is treated as just another instance of this same transition, not a special case.

Six round trips to one

Starting a voice session used to mean: ICE negotiation, DTLS handshake, SCTP handshake, then opening data channels, each its own round trip before a single audio frame could move. OpenAI built a protocol called WARP (WebRTC Abridged Roundtrip Protocol) that piggybacks and pre-negotiates these steps, cutting startup from six round trips to one. Combined with an “Instant Connect” mode that pre-negotiates parameters without reserving server capacity, a client can start a session with a single UDP packet.

Capacity planning isn’t a GPU math problem

The most transferable lesson in the piece has nothing to do with voice specifically. The team’s first instinct was to size capacity by asking “how many requests can a GPU handle?” That’s the wrong question for a system with long-lived, latency-sensitive sessions: the real question became “how many concurrent sessions can we sustain while keeping every audio frame on schedule?” A session that’s live for twenty minutes but idle 80% of the time consumes capacity very differently than a burst of short stateless requests, and production traffic surfaced supporting components that saturated earlier than load tests predicted, causing requests to queue and latency to compound in ways raw GPU throughput numbers never would have shown.

The caveat worth sitting with

OpenAI didn’t publish formal latency or word-error-rate numbers for GPT-Live, only relative comparisons and a 75.7% human-preference win rate against the old system. Independent testing (by Agora, measuring real sessions) found that when a user deliberately interrupts GPT-Live mid-response, it takes about 498ms longer to actually go silent than the old turn-based Advanced Voice Mode did.

That’s the honest tradeoff of full-duplex done this way: a system built to never wrongly cut you off, by continuously weighing whether to yield rather than gating on a hard turn boundary, can end up slower to stop once it’s already decided to keep talking. Removing a turn detector removes its false cutoffs, but the model still has to notice an interruption and decide to yield, and that decision has its own latency. Full-duplex fixes one failure mode by design; it doesn’t automatically fix the other one, and this system apparently hasn’t yet.

Quiz: check your understanding

Five questions, one per pattern. Click each to reveal the answer.

1. A voice model that can listen and generate speech at the same time, with no separate “your turn / my turn” gate, is an example of what communications pattern?

Full-duplex communication, both parties can transmit and receive simultaneously, unlike half-duplex (walkie-talkie style) where only one side transmits at a time.

2. GPT-Live keeps a fast-updating transcript for the UI and a separate, slower, final transcript for analytics. What’s the general name for this pattern?

Speculative (optimistic) state vs. authoritative state: a fast best-effort view users see immediately, decoupled from a slower, durable source of truth.

3. To swap model versions or compact context mid-session without an audible gap, GPT-Live warms a second model instance, prefills it, and cuts over once it’s caught up. What deployment pattern is this analogous to?

Blue-green deployment: keep the old instance serving while the new one comes up to speed in parallel, then flip traffic so the switch is invisible.

4. WARP cuts session startup from six network round trips to one by piggybacking and pre-negotiating handshake steps. What other protocol optimizes startup the same way?

QUIC’s 0-RTT resumption (and TLS 1.3’s shortened handshake) — both collapse sequential negotiation steps so a connection can start useful work sooner.

5. The GPT-Live team found that “how many requests can a GPU handle?” was the wrong capacity question. What was the right one, and why did short-request load tests miss it?

”How many concurrent sessions can we sustain while keeping every frame on schedule?” Long-lived stateful sessions hit a concurrency ceiling (memory, connections, scheduling slots held open) before they hit a raw throughput ceiling, and short synthetic load tests never build up real concurrency, so they don’t surface it.