Skip to main content

Twilio Voice

Overview​

The Twilio Voice integration enables your AI Agent to handle voice conversations through phone calls and web-based calling. By connecting your Twilio account to Moveo, you can provide automated voice support that combines the power of AI with natural speech interaction.

This integration supports automated phone support through your Twilio phone numbers, web-based calling for browser-to-AI conversations, natural voice interactions with text-to-speech and speech-to-text capabilities, and flexible configuration to optimize voice quality and recognition accuracy.

Prerequisites​

Before setting up the Twilio Voice integration, you'll need to configure both your Twilio account and your Moveo setup.

Twilio account​

You'll need an active Twilio account with voice capabilities enabled and sufficient balance to cover voice call costs. From your Twilio Console, gather the following credentials:

Account SID and Auth Token: Available in the Account Info section of your dashboard. Click Show to reveal the Auth Token if it’s hidden.

Phone number (optional): You can use a Twilio-provided number or a number you’ve ported to Twilio. If you don’t configure a phone number, the integration will be limited to web-based calls.

warning

A phone number is not required if you only plan to handle web calls. It becomes required when supporting incoming phone calls or when using the integration to call customers through campaigns.

Moveo account​

The configuration steps for connecting your AI Agent to Twilio are handled through Moveo's platform. For a complete walkthrough of the Moveo-side setup including AI Agent selection, environment configuration, and rule creation, see the quick deployment guide.

How it works​

The Twilio Voice integration creates a bridge between callers and your Moveo AI Agent. When a user initiates a call through your web application, Twilio establishes the connection and begins streaming audio. The speech-to-text engine converts spoken words into text, which your AI Agent processes to generate an appropriate response. That response is then converted back to natural voice through text-to-speech and streamed to the caller in real-time.

For phone calls, the flow is similar. When someone dials your Twilio phone number, Twilio routes the call to Moveo where the same speech processing and response cycle occurs. The conversation continues until the caller hangs up, or the AI Agent decides that the conversation has been resolved.

When a conversation is resolved, we allow the user to keep talking as long as the keep alive timeout hasn't finished.

Reaching your AI Agent​

Once the integration is configured, users can reach your AI Agent in two ways:

Inbound calls: Customers can call your Twilio phone number directly to start a conversation with your AI Agent. This is particularly useful for customer support hotlines or IVR systems where users initiate contact.

Outbound calls: Your AI Agent can also call customers as part of a campaign. This enables proactive outreach for appointment reminders, notifications, surveys, or follow-ups, with the AI Agent handling the conversation when the customer answers.

Setup guide​

Step 1: Prepare your Twilio credentials​

Log in to your Twilio Console and navigate to the Account Info section on your dashboard. Copy your Account SID and your Auth Token (click "Show" if it's hidden). Keep these credentials handy for the next step.

Step 2: Configure the integration in Moveo​

Navigate to Integrations in your Moveo account, select your environment, find the Twilio Voice integration card, and click Configure.

Step 3: Enter your Twilio account details​

In the configuration panel, you'll see a Twilio Account section where you'll enter your credentials and optionally select a phone number:

Account SID: Your Twilio Account SID from the previous step

Auth Token: Your Twilio Auth Token

Phone Number: Select a phone number from the dropdown to enable phone calls, or leave it set to "None" for web calls only

When no phone number is selected, the integration works exclusively for web calls. When you select a phone number, the integration handles both web calls and incoming phone calls to that number.

Step 4: Optimize voice settings​

The integration includes default settings for text-to-speech (TTS) and speech-to-text (STT) that work well for most use cases. However, you can adjust these settings to better fit your specific needs. The TTS settings let you select the voice model, speaking rate, pitch adjustments, language, and accent. The STT settings allow you to configure the recognition model, language, profanity filtering, and punctuation preferences.

These settings can be found below the Twilio Account section in the configuration panel and adjusted based on your testing results.

Step 5: Activate the integration​

Review your configuration, toggle Active to enable the integration, and click Save to apply your changes.

Configuration options​

Voice quality optimization​

To ensure the best voice experience, structure your AI Agent's responses using clear, concise language without complex formatting. Avoid special characters and symbols that don't translate well to speech, and use natural spoken language patterns. Consider adding pauses with punctuation for better pacing.

AI Agent guidelines for voice​

Voice conversations have unique characteristics compared to text-based chat. To improve your AI Agent's performance in voice interactions, you can add specific guidelines that optimize for the auditory experience. These include using shorter, more conversational responses, avoiding lists or complex structures that are hard to follow audibly, confirming understanding of user input before proceeding, using verbal cues for transitions between topics, and providing clear call-to-action statements.

tip

Check this documentation page to learn more about this: https://elevenlabs.io/docs/best-practices/prompting/normalization#putting-it-all-together

Testing your integration​

Test phone calls​

If you configured a phone number, dial your Twilio phone number and wait for the AI Agent greeting. Test various conversation scenarios, verify proper call handling and termination, and check call quality and response timing.

Key test scenarios​

Make sure to validate call initiation and greeting, clear speech recognition, natural-sounding responses, handling of unclear input or silence, proper call termination, and transcription mismatches.

Troubleshooting​

Connection issues​

Integration fails to connect to Twilio: Verify your Account SID and Auth Token are correct, check that your Twilio account is active and funded, ensure you have voice capabilities enabled, and review Twilio account status for any restrictions.

Phone number not working: Confirm the phone number is active in your Twilio account, check that the number has voice capabilities enabled, verify the number is properly configured in Moveo, and test the number directly in Twilio Console.

Audio quality issues​

Speech recognition is inaccurate: Adjust STT language settings to match the caller's language, enable noise suppression for better accuracy, check microphone quality and audio input levels, consider environment noise factors, and review recognition sensitivity settings.

Voice responses sound unnatural: Review your AI Agent's response formatting, remove special characters and formatting that don't translate to speech, adjust the TTS voice model or speaking rate, simplify complex responses for better verbal delivery, and add punctuation for natural pauses and pacing.

Audio delays or lag: Check your internet connection stability, review Twilio's service status for any issues, verify your server response times are optimal, consider geographic latency factors, and check for webhook processing delays.

Call handling issues​

Calls not being answered: Verify the integration is set to Active, check Twilio webhook configuration, review call logs in Twilio Console, confirm your environment is properly configured, and check for any error messages in Moveo logs.

AI Agent not responding correctly: Test the AI Agent in text mode first to verify logic, review voice-specific guidelines and adjust accordingly, check for speech recognition errors in logs, verify intent recognition is working properly, and consider adding confirmation steps for critical information.

Phone issues​

If the issue seems that the call is not connecting to Moveo at all, ensure that the phone number is set up correctly in Twilio console.

Limitations​

Platform limitations​

Call duration limits are based on your Twilio plan and pricing. Concurrent call limits are determined by your Twilio account tier. Voice availability varies by region due to geographic restrictions. Audio format constraints are defined by the specific codecs supported by Twilio.

Human escalation: Currently the escalation to Human Agents is not available for Twilio integrations. When this action is triggered by the AI Agent, it will ignored and the AI Agent will still handling the conversation.

Integration constraints​

Each integration configuration supports one phone number. Voice quality depends on the caller's connection and device. STT accuracy varies with accents and background noise. Complex multi-turn conversations may require optimization.

Best practices​

Monitor call duration and implement time limits if needed for cost control. Set up call queuing for high-volume scenarios. Provide clear voice prompts for better recognition. Have fallback options for unclear or failed recognition. Regularly review and optimize based on call analytics.

Resources​

Twilio documentation​

Moveo resources​

Support​

Next steps​

Once your Twilio Voice integration is active, optimize your AI Agent with voice-specific guidelines, test thoroughly with various scenarios and voice configuration, monitor call analytics to identify improvement areas, adjust TTS/STT settings based on user feedback, configure fallback options for complex queries, and train your team on monitoring and handling escalations.

Last updated on