Memox: AI voice agent development on Telnyx and Gemini Live

This is AI voice agent development inside a product we own: an existing Memox agent given a phone number, call controls and a review trail an operator can use. Memox is an Autonomous-owned product. Businesses use its AI agents to answer questions from company information and hand conversations to staff. Voice extended that product: the objective was to connect an existing agent to a phone number and give operators enough visibility to understand what happened on a call.

A phone channel adds requirements that a chat interface does not. The conversation has to survive real-time audio conditions, recognise interruptions and manage call state. Operators need configuration, test tools and call history. Those responsibilities shaped the build as much as the quality of the model’s spoken answer.

A call has to travel from the carrier to the model and back fast enough to feel like a conversation, so we shortened that path. We built the voice subsystem inside the existing Django backend, using Telnyx Call Control for telephony and Gemini Live for conversational audio. A WebSocket bridge carries audio in both directions, while background tasks handle campaign orchestration and post-call work.

The choice of audio format mattered because conversion happens repeatedly during a live call. Matching the carrier’s supported audio with the model’s expected input reduced avoidable work in the path. Connection setup and greeting preparation were also moved into the ringing period so the system could do some of its work before the person answered.

Workflow: Phone call → Audio bridge → Existing AI agent → Operator review.

The useful part is not one impressive demo call, it is the configuration, call logs and recordings an operator uses every day. The dashboard provides voice-channel configuration, call logs and call-detail views. A browser playground lets an operator try an agent before connecting a real number. Recordings and post-call summaries make a conversation reviewable after it ends.

The campaign workflow handles contact import, number validation and deduplication, with coordination around concurrent workers. Calling-window and consent-related controls sit in the application workflow, making them part of the operator’s configuration and the system’s decisions before a call starts.

Live audio can break in ways an ordinary API test never sees, so we tested the audio path itself rather than only the answers. Audio processing exposed problems that would be easy to miss in a conventional API test. One processor consumed and emitted audio in different-sized blocks from the telephone stream. A direct wrapper could produce silence or uneven output. Buffering was added to preserve a consistent output length while accommodating the processor’s internal timing.

The team also worked on interruptions, speech sensitivity and call recording behaviour. More advanced noise and echo processing defaults to disabled behind a feature flag while its activation checks remained incomplete. That distinction matters: code being present is different from a capability being ready to enable for customers.

Three core contributors developed the voice channel from March through August 2026 alongside the wider Memox platform. It brings telephony, agent configuration and post-call review into one product. Paying-customer voice go-live remains unverified, and the advanced audio-processing feature was disabled by default in the code while activation checks remained open.

For a business considering AI on the phone, the useful starting point is the operational job: what information the agent needs, when a person should intervene and how the call is reviewed. Carrier and model selection follow those requirements. The result needs to function as a manageable business channel, not merely a convincing recording.