Analytics
Overview
Analytics help you answer questions such as:
- What was the number of conversations/users per week during the last month?
- How often do customers need to speak to a live agent?
- Which days had the largest or smallest numbers of conversations in the last month?
- What time of day do most users contact my business?
- How fast do my live agents reply to the users?
- Are the users satisfied with the AI Agent? What ratings do they leave?
- Are there questions that the assistant is not trained to answer?
The Overview page summarizes interactions between users and your AI Agent. View traffic for a selected time period, the percentage of meaningful conversations, and the most used environments.
Use filters to narrow by date, environment, AI Agent, or channel. Exclude test conversations or filter by minimum message count.
Meaningful conversations
A conversation is labeled as meaningful if either of the following is true:
- LLM meaningful: The conversation has at least one covered LLM response.
- AI Agent meaningful: After the user's first message, an AI Agent has at least one covered and contained response.
A session is meaningful when the AI Agent replies successfully at least once after the user's first message. A successful response does not originate from the Unknown flow.
Below are some examples of meaningful and non-meaningful conversations.
Examples of non-meaningful conversations
- Example #1
- Example #2
- Example #3
--- Conversation Start ---
User: Hi
Assistant: Hi, how can I assist you?
--- Conversation End ---
This is classified as a non-meaningful conversation, since it contains just one user message.
--- Conversation Start ---
User: Hi
Assistant: Hi, how can I help you?
User: How do I change my credit card number?
Assistant: I'm sorry, I'm not sure how to help you change your credit card number.
User: I want to change the four last digits.
Assistant: I apologize, but I'm not sure how to help you change the four last digits of your credit card.
--- Conversation End ---
This is classified as a non-meaningful conversation, since, after the user's first message, there are no covered AI Agent responses.
--- Conversation Start ---
User: Hi
Assistant: Hi, how can I help you?
User: I would like to speak to an agent
Assistant: Ok, let me connect you to an agent
--- Conversation End ---
This is classified as a non-meaningful conversation, since, after the user's first message, there are no contained AI Agent responses.
Examples of meaningful conversations
- Example #1
- Example #2
--- Conversation Start ---
User: Hi
Assistant: Hi, how can I help you?
User: What payment methods do you support?
Assistant: We support Visa, MasterCard and PayPal.
--- Conversation End ---
This is classified as a meaningful conversation, since, after the user's first message, there is a covered and contained response.
--- Conversation Start ---
User: What steps do I need to follow in order to reset my account?
Assistant (LLM): To reset your account, please follow the step-by-step instructions provided in our account recovery guide.
--- Conversation End ---
This is classified as a (LLM) meaningful conversation, since it contains an LLM response.