The small decisions inside a phone call
The pause that ends the call
On the phone, a second of silence feels like a dropped line. A voice agent has less than that to decide what to do.
You finish a sentence on a support call and wait. If the reply takes too long, you say "Hello?" and the call starts to fall apart.
A voice agent is a system that listens, decides and talks back inside a conversational pause. The talking part gets the attention. The deciding part is where the time goes.
This piece is about that deciding: how much of it can be made small, fast and checkable.
Not every decision needs a thinker
Has the caller finished? Is that an interruption or just "mm-hm"? These are quick judgements, not essays.
In every turn of a call a voice agent makes a handful of small decisions. Has the caller finished speaking, or only paused? Is the sound that just came in an interruption, or a murmur of agreement? Does this need a person? Is the action about to be taken one that cannot be undone? Is the sentence about to be spoken safe to say?
These are fast, closed questions with a short list of possible answers. A person on the phone answers them without noticing. A large language model can answer them too, but it answers slowly, in prose, and it cannot say how sure it is.
The idea tested here is to hand those questions to a small decision model that returns a typed answer and a probability, and leave the large model to do only the talking.
Right, in time, at a price
Each kind of decision is scored on labelled cases, against a time allowance, with its cost counted.
The measure is accuracy on labelled cases, one kind of decision at a time. For each kind there is a set of situations from a phone-support setting with the right answer marked. The score is the average accuracy across the kinds, so a kind with few cases counts as much as one with many.
Two guards sit beside it. Each decision has a time allowance for the typical case and a larger one for the slowest cases. And the cost of a thousand decisions is reported with the score.
The labelled cases were written by one model and labelled blind by another; only cases where the two agreed were kept. That makes the labels consistent. It also makes the test easier than real calls, which the limits section returns to.
About 39 in 40, before any tuning
On the first run, with plain one-paragraph instructions and no examples, about 39 decisions in 40 were right.
Out of the box, about 39 decisions in 40 were right. The instructions for each kind of decision were a single plain paragraph, with no worked examples and the simplest possible cut-off.
Two kinds of decision had no errors at all: sending the caller to the right specialist, and spotting an action that needs the caller's confirmation first.
This is a starting point and it is reported as one. Nothing had been tuned.
Fast enough to ask several times a turn
A typical decision took under a quarter of its time allowance, so several fit inside one turn.
A typical decision came back in under a quarter of the time allowed. The slowest ones took about an eighth of their own, larger allowance. No call to the decision model failed.
That changes what is affordable. Several decisions can be asked in every turn, some of them side by side, and together they add only about a tenth to the caller's wait.
They are also cheap enough that the cost of the decisions disappears beside the cost of the talking.
The weak spot: is this sentence safe to say
Checking a drafted sentence against a policy was right about nine times in ten, the lowest of any kind.
One kind of decision stood out: checking a sentence before it is spoken. Judging a drafted reply against a policy with several rules was right about nine times in ten, clearly below the rest.
It makes sense. This is the least reflex-like of the questions: it asks the model to hold several rules against one sentence. In a live test call the same check was also too cautious, stopping harmless sentences.
So the result is uneven, and usefully so. The reflex questions are nearly solved. The one that needs a little thought is where the work is.
The call is still too slow
End to end, the agent answered in nearly three times the target. The small decisions are not why.
Fast decisions do not make a fast call. Measured from the caller's last word to the agent's first, the whole system took nearly three times the target for a natural conversation.
The decisions are about a tenth of that wait. The largest single part is the language model that writes the reply, waiting for its first word. Turning speech into text is the second. Knowing when the caller has really finished is the third: answer too soon and you talk over them.
The measurement was taken on a busy machine, so it is on the pessimistic side. It is still far from the target, and it says where to look.
What cannot be claimed
One run on an easy, machine-made test in one setting. Nothing here is a tuned or a real-call result.
The test is on the easy side. Its cases were written and labelled by models, nearly all of them survived blind re-labelling, and no person has checked them. They are clean text, not the garbled transcripts a real call produces.
It is one run in one setting, a phone-support desk. The accuracy is a starting point, measured before any tuning. The end-to-end timing comes from a scripted call with a synthesised caller, not from people.
No result here has been reproduced by a third party.
What is public, and what is on request
The shape of the result is public. The figures, per kind of decision, are in the full results paper.
Split-second decisions is proprietary. What is public is the shape of the result: how often the small decisions were right, how quickly, and how far the whole call still is from natural. Which models were used, how the questions are worded and how the pieces are wired together are not published.
The accuracy for every kind of decision, the timings by stage and the cost are in the full results paper. If you are building a voice agent and the pause is your problem, ask.
Step 2: the paper