Two seconds to answer

A distributor of motors, drives and gearboxes receives dozens of calls every day with questions that have an answer in its ERP: do you have this motor in stock? When does my order arrive? How much is the 2.2 kW drive? If nobody is free to pick up the phone, the call is lost. With one such distributor, which is rolling out Odoo, we built the proof of concept of a voice agent that answers those calls and queries the ERP in real time while it talks to the customer.

This article explains how it is built: the architecture, the decisions we made from day one and why, the figures we measured and the failures we found when testing it, which taught us the most. Everything comes from the project plan, the code and the logs of the 53 stored test calls. The demo data is fictitious and the client remains anonymous.

01The problem: voice is unforgiving

In a chat, a reply that takes four seconds is acceptable. On the phone, four seconds of silence make the customer ask “hello?”. And here the answer also depends on a query to the ERP. There are five real difficulties:

• Latency and turn-taking. The system has to decide when the customer has finished speaking, understand, query Odoo, compose the answer and start synthesizing speech. Every step adds up, and the customer only perceives the total.
• Dictated references. “E, G, M, dash, I, E, three, slash, one hundred twelve, M, dash, four” has to end up as EGM-IE3/112M-4. And VT-B700-4T-2.2 (2.2 kW) must not be confused with VT-B700-4T-22 (22 kW).
• Identifying the customer without letting the model impersonate them. If the agent answers about orders, it may only do so for the company that is calling, even if someone tries to convince it otherwise.
• Degrading without inventing. If the ERP does not answer, the agent has to say so and offer a way out, not improvise a delivery date.
• Languages. Spanish and Galician, and no others.

02The architecture

The proof of concept is used from the browser: a page with a microphone replaces the phone. Underneath there are four pieces:

A turn goes like this: the customer speaks → the audio reaches the relay → Gemini Live decides whether it needs data → if it does, it asks for a tool (get_stock, get_order_status, get_my_orders, take_message or transfer_to_human) → the relay calls the skill → the skill queries Odoo → the result goes back to the model → the model answers with voice.

In the initial proposal, Apache Airflow appeared as the orchestrator. In the test we left it out of the speaking turn: launching and waiting for a DAG adds hundreds of milliseconds or seconds exactly where the customer notices them. Airflow remains the right piece for everything asynchronous (summaries, dumping into Odoo, messages, WhatsApp), but not for answering.

03Decisions from day one

The model does not touch the ERP; it touches tools

The model never talks to Odoo. It can only ask for five tools with typed arguments, and each skill has a closed list of Odoo models and fields it can read, always with search_read. The Odoo service user has no write permission at all (see section 07: at the beginning it did).

Four states and a ready-to-say sentence

Each skill answers with a state and a sentence designed to be read aloud:

Each call to Odoo has a 2-second limit and each skill a total limit of 2.5 seconds. The skills never return a 500 error: any exception becomes degraded and is logged.

A signed token the model does not see

When the call is picked up, the relay identifies the caller by their number and receives a caller_token: the phone number signed with HMAC. That token travels with every order query, but it is the relay that adds it, not the model, and the model never receives it. Even if someone tells the agent “my phone number is 600 000 002,” it cannot query another company’s orders. If the same phone number appears in two companies, nobody is identified.

Measure from the start

Each call leaves a complete record: what both parties said, each tool with its arguments and the exact response the model received, the latency of each turn, interruptions and errors. A script turns it into a readable report with alerts: repeated queries, non-ok responses, slow turns or silences after a query. Without that record, half the failures in this article would never have been found.

Browser demo, not phone

We decided this so we would not have to wait for a phone number to be verified. It has a cost we always declare: with no telephone leg, the audio is wideband and the latency is more optimistic than on a real call.

From ElevenLabs to Gemini Live, and back

The initial plan used ElevenLabs Agents. With that platform, tools run as webhooks called from its cloud, so the skills would have to be exposed to the internet through a tunnel. We switched to Gemini Live for two reasons:

• with the Live API it is our relay that runs the tools, so the skills never leave the machine;
• we measure latency on the server, on the real audio.

Later we added ElevenLabs as a second engine (section 08), with “client” tools that also end up at the relay.

04Dictated references

Normalization turns what is dictated into something comparable:

• Letters and symbols: “dash,” “slash,” “dot,” “A as in apple.”
• Numbers as words: “one hundred twelve” is 112 and “seventy-five” is 75. “One two three,” dictated digit by digit, is 123. And in Galician, “dous,” “dúas,” “catro.”
• Decimal point: 2.2 and 22 are different products. If the reference only matches once the point is removed, the agent asks and does not give the data.
• Descriptions: “a 4 kW motor” is searched by the terms in the product name, and figures are bound to their unit, so “4 kW” does not match “0.4 kW” or “4 poles.”
• Approximate match with confirmation. Speech recognition once heard “planetary gearbox Brevini 90” instead of “flange 90” (Brevini is a real gearbox brand). Now, if not everything matches, the closest product is offered, but only if all figures match (size, ratio, power) and always asking for confirmation: “The closest is the RD-P90/I9. Is that the one?”.

05Orders: what the agent knows and what it doesn’t

The first version of the orders tool returned status, expected date and tracking. In the first test the customer asked when the order was placed, what it contained and where the shipment was, and the agent had none of that data. It also queried the same order twice hoping for more information. The response is now complete in a single call: creation date, lines with ordered, delivered and pending quantities, availability of each line, amounts with and without VAT, shipping date, carrier and tracking. The prompt tells the model to ask once and answer the rest from that data.

The status of an order is not trivial either. A “ready” delivery note in Odoo may have a line with no stock. The skill looks at the state of each movement and distinguishes between ready, partially available, pending stock and partly shipped.

06Measured latency

We measure at the relay, from the last audio chunk with the customer’s voice to the first audio from the agent. That figure includes the 700 ms the model waits in silence to know the customer has finished, because the customer hears that too. With Gemini Live, in 21 real calls:

Two important caveats:

• Odoo runs on the same machine, so the 70 ms are optimistic. With Odoo in the client’s cloud, the network will have to be added.
• There is no telephone leg.

So the wait is not noticeable, the agent says a short filler (“one moment, let me check”) right before querying, and only then. In the first tests it said it without ever querying anything.

The silence after the query

In the second round of tests the agent said “one moment, let me check,” received the stock from Odoo in 56 ms and did not speak again for 47 seconds. There was no error: the answer had reached the model. Google’s documentation contradicts itself on whether tools are blocking by default in gemini-3.8-live. If they are not, an answer that arrives while the model is still speaking can remain pending and go unsaid. We fixed it twice over:

• by declaring the tools explicitly as BLOCKING;
• with a watchdog in the relay: if the agent does not speak within 6 seconds after a result is delivered, it is reminded that it already has the data. It remains as a warning in the log, to know whether it happens again.

07Failures that made us change things

Before the first call we asked a second model, Claude Opus, to audit the code looking for failures, and then we tested with real voice. These are the ones that weighed most:

Each one has its regression test, except the last, which is checked by the configuration script itself. Today 74 skill tests (against the demo Odoo) and 63 relay tests pass.

08Two engines, same logic

ElevenLabs Agents came in as a second engine so we could compare. It shares with Gemini Live the same prompt and the same tools: the agent is created through the API with a script that checks that the prompt stored in ElevenLabs is identical. The differences:

With ElevenLabs we only have 8 measured turns, with a median of 2.9 s, and the measurement is not done the same way. It is too early to compare them. The cost also differs: with Gemini Live we estimate about $0.004 per test call from the tokens. ElevenLabs charges $0.08 per platform minute, plus the language model and telephony.

09What is missing

It is a proof of concept, and these are the pieces that separate the demo from a production switchboard:

• Telephony. Receiving calls from the client’s virtual switchboard through SIP or call forwarding. First of all we must check that forwarding preserves the caller’s number, because without it there is no identification. With Gemini Live the telephone audio bridge has to be built; ElevenLabs includes it.
• Concurrency. The target is 5 or 6 simultaneous calls. We need to move from a laptop to a machine in the EU and run a load test with seven synthetic simultaneous calls.
• Real Odoo. Adapt the skills to its data and to a catalog of thousands of references, and measure latency with the network in between.
• Compliance. The notice that an AI assistant is answering, paid APIs (free plans use the data or have no commercial license) and a retention policy for transcripts.
• Operations. A console for several simultaneous calls, messages that reach their recipient and a battery of scripted calls that runs after every prompt change.
• Writing to Odoo, with human review, through the asynchronous path.

What the test has shown is that a voice agent can query a real ERP while it speaks and answer in under two seconds, without being able to write or see what it should not, and that when it fails it leaves enough of a trail to fix it.