AI Agent training and testing
Confidence Level
When the AI Agent is not 100% certain about a user's message, it makes an elaborate prediction based on its smart AI algorithm and responds according to its confidence level.
The only way the AI Agent responds with 100% confidence is if the user's message exactly matches an intent expression you have written.
You can review these predictions and use the Amend functionality inside the test to manually link the intent
with the correct expression.A very effective way to use training and increase the confidence level of your AI Agents is to invest time in feeding them profession-specific language and jargon. This helps the AI Agent learn and adapt to your needs.
With state-of-the-art NLP algorithms, machine learning, and a bit of guidance, your AI Agent will be able to engage in meaningful end-to-end conversations with your users.
Confidence Threshold
You can adjust the confidence threshold of an AI Agent depending on your needs at any time. This threshold can be set to a value between 0 and 1, representing the minimum confidence level your AI Agent should have to answer a question it is not exactly trained on.
For example, for a typo like "Helor there," your AI Agent should have a high level of confidence to respond with the #greetings intent, assuming you have added an expression like "Hello there" to this intent. Conversely, for a phrase like "ffdfdgd," the assistant should have a low level of confidence in replying.
An AI Agent set at a 0.70 confidence threshold would respond to the "Helor there" message and trigger the unknown
intent for the "ffdfdgd" message.You can find the confidence threshold in the Advanced section of your AI Agent.
Testing Your AI Agent
By clicking on Test, you can interact with your AI agent to evaluate its responses. Pretend to be an end user and check if the assistant answers your questions correctly.
You can choose to test your AI agent either within Moveo or with WhatsApp.
- Test with Moveo
- Test with WhatsApp
When testing with WhatsApp, you simulate the conversation using your phone number. This is different from an integration with WhatsApp
, as it tests the integration with our Moveo number. You must have a WhatsApp account with this number to test.Amend
Rephrasing
Before the user's message goes to the AI Agent, it passes through a specialized LLM that understands and routes the message to a specific intent. During this process, the message can be rephrased to improve understanding. However, this rephrasing may not always be accurate, leading to a lower confidence level in matching the intent. In such cases, you can use the Amend feature to correct the rephrased message.
This rephrasing WILL NOT take effect immediately. It will be used to improve the training data for future versions of our LLM.
Add New Training Phrases
In the replayed logs section, if an intent that should have a higher confidence level is not triggered, you can use the Amend feature to add the user’s input as a new training phrase. This helps the AI Agent better understand the user’s intent. If an intent already has a 100% match, the Amend button will not be displayed.
Amend the Response
If the source of the AI Agent's response is synthetic, you can amend the response to improve its accuracy. As with rephrasing, this will not take effect immediately.
Context
You can add your own context variables
, custom user information, or tags to the session to test the behavior of the AI Agent under specific circumstances.