AI voice agent benchmarks: latency, voicemail, and pick-up rates from production calls

Every voice AI vendor publishes latency numbers. Almost none of them measure the same thing. Some quote time-to-first-token from the language model, which ignores speech-to-text and text-to-speech entirely. Some quote model inference on a warm cache in a datacentre next door. Some quote the number their fastest demo hit once.

The number that matters is round-trip latency on a real phone call: the gap between the caller finishing a sentence and hearing the first syllable back. That figure includes network hops, telephony encoding, speech-to-text, model inference, text-to-speech, and the return trip. It is always worse than the model benchmark, often by a factor of three.

The numbers below come from running that full pipeline on tens of thousands of outbound calls over about 18 months. They are not model benchmarks. They are what the person on the other end of the line actually experiences.

Text-based AI has a generous budget for thinking. A two-second pause in a chat window reads as normal. The same pause on a phone call reads as a dropped connection.

Human conversational turn-taking sits at roughly 200 milliseconds between speakers. That is the gap people expect without noticing it. Push past it and the call degrades in stages. Around 500ms the agent starts feeling slow but stays usable. Around 800ms people say "hello?" into the gap. By 1.2 seconds most assume the line dropped and either hang up or start talking over the agent, which then has to handle an interruption it caused.

The trap is that this cuts your model options down hard. We ran Qwen 3.5, moved to 3.6, tested Gemini Flash, and ran GPT-4o mini. The pattern held across all of them: models fast enough to stay inside the budget are often too limited for the conversational complexity of a real qualification call, and models smart enough to sound natural blow the budget.

The workable answer is not picking one model. It is monitoring latency per call and swapping models when performance drifts, treating the language model as a swappable component rather than a fixed dependency. That is an infrastructure problem, not a prompt engineering problem, and it is where most teams building voice agents are not looking.

Between 60 and 70% of outbound calls reach voicemail. That single number reframes the whole problem. Most teams building voice agents treat voicemail as an exception to handle later. In production it is where most of your call volume lands.

Detecting it is harder than it sounds. Greetings vary enormously. Some are two seconds of silence and a beep. Some are 30-second personal recordings that open exactly like a live person answering. Some carriers play their own pre-roll before the subscriber's greeting starts at all. Get the classification wrong and the agent starts a qualification conversation with an answering machine, which wastes the call and, depending on jurisdiction, creates a compliance problem.

Getting detection to 99% took a dedicated classification layer running parallel to the main conversation engine. It listens to the opening seconds of audio and makes a binary call before the agent speaks a word. Building it as a separate concern rather than a branch inside the conversation logic is what made the accuracy achievable.

The second half is that a voicemail is not a failed call. It is a different call. It needs its own script: short, specific, and giving a concrete reason to call back. A voicemail script that is just the opening of the live script, truncated when the beep arrives, performs about as well as no message at all.

Calls that fire within 30 seconds of a trigger event, a booking, a form submission, a cart abandonment, run around a 20% pick-up rate. The same leads called back by a human hours or days later land between 5 and 8%.

That gap is not about the AI being more persuasive. It is entirely about timing. The lead just interacted with you. Your brand is the thing they were looking at 30 seconds ago. An hour later they have moved on, and the call arrives as an interruption from a company they half remember.

This is the strongest argument for automating the first touch rather than the whole funnel. No human team can call every lead inside 30 seconds at any hour. Software can, and the pick-up rate difference is roughly threefold.

Pick-up rate gets quoted more often, but completion rate is the honest measure. Of the people who answer, what share finish the full qualification flow rather than hanging up midway?

That figure is where everything else compounds. Conversation design, voice quality, personality consistency, and latency all feed into it. Get them all right and completion sits above 70%. Get any single one wrong and it drops below 40%. There is not much middle ground, because the failure modes are individually sufficient: latency alone will kill a call, and so will a voice that sounds synthetic in the first three seconds.

A good production call runs about two minutes. Long enough to confirm the appointment, ask three or four qualification questions, and collect what the sales team needs. Short enough that the person does not feel trapped. Calls that stretch past four minutes show a sharp rise in mid-call drop-off.

Demos run on clean audio, in quiet rooms, with cooperative testers who speak clearly and wait their turn. Production calls happen while someone is driving, cooking, walking through an airport, or managing a toddler.

Background noise alone degrades speech-to-text accuracy by 30 to 40%. The model was benchmarked on podcast-quality audio and is now processing a call from a building site. Interruptions compound it: people talk over the agent, change subject mid-sentence, say "hold on" and go silent for 15 seconds, then return on a different thread. A conversation state manager built for clean turn-taking falls apart on the first real call.

The wins here are unglamorous and specific. Adding a low bed of ambient noise to the agent's own audio so it does not sound sterile. Introducing small imperfections in pacing so the rhythm reads as human. Keeping a consistent personality across calls rather than a script that drifts. None of these show up in a feature comparison. Together they are most of the difference between 40% and 70% completion.

Most teams treat compliance as a checkbox at the end. In practice the regulatory position determines fundamental design decisions, and it is cheaper to accept that early.

The largest one for us: deploying only on inbound leads with clear intent and established consent. Someone books a call, and the agent calls to confirm and qualify. That is a materially different legal position from cold outbound, and it changes the system throughout. Consent already exists. Intent is already expressed. The call is expected rather than intrusive.

Disclosure is the other design decision masquerading as a legal requirement. From August 2026, Article 50 of the EU AI Act requires telling people they are speaking with AI. Handled as a rushed legal preamble, it creates suspicion in the opening seconds. Folded into the opening as a normal thing to say, it builds trust instead. Teams that treat it as part of the pitch rather than a tax on the pitch consistently see better completion rates.

The next meaningful shift is multi-agent orchestration on voice. One agent handles greeting and qualification, then hands off mid-call to a specialist with deeper domain knowledge, without the caller noticing the switch. That needs shared conversation state, unbroken voice continuity, and latency management across two inference pipelines at once.

Nobody has shipped that convincingly yet. But the infrastructure choices teams make now, treating the model as swappable, building real state management, investing in voicemail detection and noise handling, are what will make it feasible or impossible in a year.

The teams building voice agents as chatbots with a microphone attached will hit a wall. A phone call is a different medium. It punishes latency, demands personality, and has no tolerance for the pauses that text interfaces forgive.