Voice AI for inbound calls: how to choose a solution based on latency and quality

Voice AI with “zero latency” does not exist, so choose based on the measured latency of the entire conversation in real calls and on how naturally the system manages turn-taking: when it starts speaking, when it stops, and how it handles interruptions. A figure such as “model response time: N milliseconds” from a demo or vendor website describes only one layer. The customer hears the total: detecting the end of their phrase, speech recognition, the model’s decision, a CRM or calendar request, voice synthesis, and the telephone network.

The main condition for a sound choice is the same test for every candidate: the same scenarios, language, background noise, integrations, and measurement method. After reading this article, you will be able to design that test, choose the metrics to compare, and calculate the full cost of a call rather than the per-minute price of one component.

What makes up the pause in a voice conversation

For a customer, the pause is the time between when they stop speaking and when they hear the bot’s voice. Several steps happen during that interval, sequentially or partly in parallel.

1. End-of-turn detection. The system must decide that the person has finished speaking, rather than paused to think. It may use voice activity detection, text analysis, or a separate turn-detection model. Most platforms wait for a minimum period of silence before closing a turn. For example, one platform’s sample configuration uses a minimum wait of 0.5 seconds and a maximum of 3 seconds. These are configuration values, not a universal standard or a guarantee of response latency.
2. Speech recognition. Audio is converted to text in a cascaded architecture. Some recognition can happen as a stream while the person is still speaking.
3. Model response. A language model forms a reply. The key measure is not total generation time, but the time until the first words are ready to speak.
4. External system requests. Check an open calendar slot, find the customer in the CRM, or retrieve an order status. This step often adds more time than the model itself and depends entirely on the speed of your systems.
5. Speech synthesis. Text is converted into voice; here too, the time to the first sound matters.
6. Telephony and network. Audio travels between the carrier, platform, and servers running the models. The server region and call route matter here.

That is why a browser demo with a headset can seem faster than a real call: it often has no telephone network, background noise, CRM lookup, or customer who speaks in fragments.

Which architectures to compare

Two main approaches are common in the market.

The “recognition → model → synthesis” cascade. Each step uses a separate component, often from a different provider. Benefits: you can select the best recognition for a particular language, the voice you want, and the model you need; it is easier to log the text at each step. Drawbacks: every link adds latency and another point of failure, and orchestration between them is a separate engineering task.

Realtime “speech-to-speech” models. One model takes audio and generates audio, either without explicit intermediate text or with text as an auxiliary output. Potential benefits: fewer transfers between components and more natural intonation. Drawbacks: less control over individual steps, a more limited choice of voices and languages, and a different pricing model.

The table does not pick a winner; it shows which questions to ask each candidate.

Neither approach is always better. A realtime model may win for a short call without integrations and lose when every answer requires a CRM lookup. A cascade may be slower but recognize street names and surnames in your language more accurately. Only a test using your scenarios can answer that.

Also check language limitations for the exact configuration you choose. Different configurations offer several ways to detect the end of a turn, including separate settings for languages other than English. Support for a language in one component does not prove that every pause-detection option will work equally well in your scenario.

Why interruptions matter

In a live conversation, people interrupt each other constantly: “yes, yes,” “no, the other one,” “wait.” A voice agent needs to distinguish three situations:

• a real interruption—the customer wants to change the direction of the conversation, so the bot should stop;
• an acknowledgment or noise—“uh-huh,” a cough, or a television sound; the bot should not stop for every sound;
• a false interruption—the system thinks the customer started speaking, but it was noise; the bot should continue.

Platforms offer separate interruption settings: the minimum speech duration before the bot stops, recovery after a false interruption, a word or voice-duration threshold, and the pause before continuing. Voice activity detection usually reacts faster than analysis of recognized text, but can be more sensitive to background noise. Check the options in the exact platform version you plan to use.

There is another detail that is easy to miss: the bot must remember what the customer actually heard. If the bot begins, “Your appointment is confirmed for Thursday at 3 p.m., and also…,” and the customer interrupts at the word “Thursday,” the conversation history must not imply that the customer heard the time and everything that was meant to follow. Otherwise, the next reply will rely on information the person never received.

Testing a real call

Compare candidates using one protocol. The protocol below is proposed for a pilot; it was not run as part of this article, and no results are reported.

Proposed protocol: 20 identical scenarios

Scenarios. Use the same 20 scenarios for every candidate, for example:

• booking a service with a choice of date and time;
• rescheduling an existing appointment;
• asking a price question answered from a reference source;
• asking to be connected to an employee;
• saying a phone or order number digit by digit;
• a street name, surname, medication, or product name that is difficult to recognize.

Complications. Add background noise (street, office, or car) to some calls, along with a pause inside a sentence when the customer stops to think but has not finished speaking.

Conditions. Use real telephony, not a browser demo; the server region you intend to use; and the language or languages your customers speak.

Integrations. Run each scenario that requires a CRM or calendar request in two versions: without the external system call and with it. This shows how much of the pause comes from the model and how much from your systems.

Repeats. Twenty calls per candidate are too few for a stable 95th percentile. Repeat each scenario several times with different test participants’ voices.

Record this for each call:

Measure latency from the caller-side call recording rather than system logs: logs show processing time, not what the person hears.

Metrics: what to compare

Time to first audible response. From the end of the customer’s speech to the first sound from the bot that the caller hears. This is the main measure of perceived speed.

p50 and p95. p50 is the median: half the responses are faster and half slower. p95 is the value that only 5% of responses exceed. The average is a poor fit here: a few very long pauses can hide behind a good average, while customers remember those long pauses. p95 needs enough measurements; with twenty calls, it is effectively one or two of the slowest values.

False and missed interruptions. How many times the bot stopped because of noise and how many times it kept talking when the customer tried to interrupt.

Recognition of critical data. Names, numbers, dates, and addresses. Mishearing a phone number costs more than mishearing a greeting.

Successful task completion. Was the customer booked? Was an appointment rescheduled? Were they connected to the right employee?

Handover to an agent. How long it takes, whether the call gets through, and whether the agent receives a concise summary.

It is useful to track how the metrics change after tuning end-of-turn detection. A shorter wait may reduce the pause before a reply, but increase cases where the bot starts talking before the customer has finished. Measure this tradeoff on the same calls and under the same conditions.

Choosing and budgeting

The per-minute price of one component is not the cost of a call. A typical cost structure includes several layers:

• telephony: number, inbound minutes, call forwarding;
• a voice-agent orchestration platform, if separate;
• speech recognition;
• a language or realtime model—often priced by audio and text token volume rather than by the minute;
• speech synthesis;
• tool and external service calls;
• storage for recordings and transcripts;
• staff time for calls handed to an agent.

Even one provider may price each layer separately. Some services charge for a voice session by duration and bill model processing and tool use separately. Check each candidate’s current pricing model and calculate a typical call instead of comparing advertised per-minute rates.

A practical way to calculate the cost is to add the layers for the same call:

> call cost = telephony × duration + platform × duration + recognition (per minute or volume) + model (volume of input and output audio or text) + synthesis (volume of the bot’s speech) + tool calls × count + share of calls handed to an agent × agent cost per minute × average duration after handover

No prices are listed here; use the candidates’ current rates on the date of calculation. The more important point is that a cheaper model can produce a more expensive call if poor recognition sends more conversations to an agent or calls last longer because of repeated questions. Calculate cost from test-call recordings, not from assumptions about average duration.

Other selection criteria:

• quality in your language and vocabulary—based on test results, not a list of supported languages;
• p95 behavior, not just the average;
• integrations with your telephony, CRM, and calendar, and their contribution to latency;
• handover to an agent with the conversation context;
• logs and recordings for reviewing problem calls;
• data handling terms: where recordings are processed, whether they are used for training, and how to delete them;
• vendor support and the predictability of API and pricing changes.

What to change in your process

1. Choose 15–20 typical inbound-call scenarios, including connection to an employee and data that is hard to recognize.
2. Set acceptance thresholds before testing: acceptable p50 and p95 time to first audible response, task-completion rate, and correct recognition rate for numbers.
3. Run the same test for two or three candidates over real telephony, with noise and pauses, with and without the CRM.
4. Measure from the caller’s side and save recordings for review.
5. Calculate the cost of a typical call across all layers for each candidate.
6. Assess pause and interruption settings separately: they can affect the experience more than changing the model.
7. Decide based on the full picture: quality, p95, integrations, cost, and support—not just the lowest median.

Frequently asked questions