This section of the user guide is focused on virtual assistant analytics. To help you navigate the article we have broken it down into the following sections:
- What are analytics?
- Who can view Analytics?
- How do I view my Analytics?
- Filtering
- Downloading your Analytics as a PDF
- Exporting Analytics
What are analytics?
The Analytics dashboard allows you to monitor the performance of your virtual assistant(s) in real-time via graphs and other high-level statistics that provide an overview of virtual assistant usage and highlights top topics and trends.
Who can view Analytics?
Analytics is available to individuals with the following permissions:
For more information on user permissions, review the User Roles & Permissions article.How do I view my Analytics?
Virtual Assistant analytics can be accessed in the Ocelot Admin Portal, under the Virtual Assistant section, select Analytics.
Definitions
Conversation: A group of interactions from when the user asked the first question to when they navigated away from the virtual assistant.
Question: One user question (interaction) and one virtual assistant response
You can view additional details about the data on the Analytics dashboard by hovering your cursor over the graph you want to know more about.
Conversations by date range
The graph is dynamic based on the filtered date range.
- A filtered date range of one day will display Conversations Per Hour.
- A filtered date range of more than one day but less than two months will display Conversations Per Day.
- A filtered date range of more than two months but less than two years will display Conversations Per Month.
- A filtered date range of more than two years will display Conversations Per Year.
Questions By Day
The Questions By Day graph aggregates the number of interactions during the filtered timeframe by the day of the week. If you select the bar for a specific day, a new tab will open on the Virtual Assistant>Interactions page with the pre-filtered questions for that day.
Questions By Hour
The Questions By Hour graph aggregates the number of interactions during the filtered timeframe by the hour. if you select the bar for a specific hour, a new tab will open on the Virtual Assistant>Interactions page with the pre-filtered questions for that timeframe.
Response Types
The Response Types graph aggregates the number of interactions based on the type of response the virtual assistant user received. If you select the bar for a specific response type, a new tab will open on the Virtual Assistant>Interactions page with the pre-filtered questions for that timeframe.
- Knowledge base - Your virtual assistant identified a matching response to the user's question within the knowledge base (General Library & Custom Questions).
- Suggestion boxes - Your virtual assistant did not identify a matching response to the user's question within the knowledge base and provided only suggestions of related questions/videos from which the user could choose.
- Search - Your virtual assistant did not identify a matching response to the user's question within the knowledge base and provided relevant links from your institution's website or a combination of suggestions and related links, videos and suggestion boxes, etc.
- Communication Opportunities - Your virtual assistant did not identify a matching response to the user's question from within the knowledge base and was also unable to provide relevant suggestions or links. The user was invited to complete a contact form to be sent to the applicable office for a response from a staff member.
- Generative AI - Your virtual assistant provided a generative AI response.
- Integrated - This category represents responses where the virtual assistant user successfully authenticated and was presented with a personalized response from your Student Information System (SIS).
- Other - This category represents responses that occur when the bot is unable to determine the user's intent. For example, this can happen with random keyboard inputs such as "gjyyjjo"
- Smalltalk - Interactions outside of the virtual assistant's primary content purpose (e.g. "What is your favorite color?")
Knowledge Base Size
- General Library Overrides - Responses to General Library questions that have been customized by your institution.
- Custom Questions - Questions and responses that your institution has created.
Note: The knowledge base size is cumulative and not based on date range.Top Topics
The top 5 topics your users have asked during the filtered timeframe. The topics are hyperlinked to a pre-filtered interactions page displaying the questions under that topic.
Note: The topics listed are not limited to supported AI Department Learning Models.
Top Questions
The top 20 most commonly asked questions from the general library and custom questions include the number of times the question has been asked within the filtered time period. You can scroll within the chart to view the entire list. The questions are hyperlinked to the pre-filtered general library or custom questions page displaying the question for quick access to manage the question.
Top Communication Opportunities
The top 20 topics that received a Communication Opportunities response from your virtual assistant within the filtered timeframe. You can scroll within the chart to view the entire list. The top Communication Opportunities are hyperlinked to a pre-filtered interactions page with those questions.
Top Pages
The top 5 pages on your website from which virtual assistant users opened your virtual assistant within the filtered timeframe.
Top Clicks
The top 5 website links in virtual assistant responses selected by your users within the filtered timeframe.
Filtering
Analytics can also be filtered by date by changing the "Start Date" and "End Date" ranges. While "Conversations (All-Time)" will continue to show the total number of conversations and questions (interactions) over the entire lifespan of your virtual assistant, "Conversations (x/x/xxxx-y/y/yyyy)" will show only the conversations and interactions inside the defined date range.
Page-Based Analytics
Analytics can be filtered by an office. Conversations are classified by the website URLs identified in the URL Matching section of the Look & Feel area in virtual assistant admin unless a user selects a specific office during their conversation with the virtual assistant (e.g., through the Communication Opportunities process or Live Assistant selection).
For more information on URL Matching, please review the following articles:
Choose the applicable office(s) to view statistics related to the virtual assistant embedded on those pages.
Language Analytics
Analytics can be filtered by language using the most recent language used by the virtual assistant user.
AI Departmental Model Analytics
Analytics can be filtered by AI Departmental Model.
Downloading your Analytics as a PDF
If you need to send a PDF report to someone who is not able to view the Analytics dashboard, you can set the filter at the top of the dashboard to the appropriate date range, and then File>Print in the browser to print and/or save it as a PDF.
Exporting Analytics
To export analytics as a .csv file select Export. The bottom left corner of the page will show a green flag that confirms that the report has been sent to your account's email address.
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article