This section of the user guide is focused on FAQs related to post-launch monitoring of the virtual assistant. To help you navigate the article we have broken it down into the following sections:
- General
- Analytics
- Are links in the welcome message counted in analytics?
- Are links from the explore bar incorporated into the analytics page?
- When will page-based analytics by office start once URL Matching is completed?
- If my domain changes, how will this affect page-based analytics?
- Why does the Top Topic statistic on the Dashboard not match the number of Interactions when searching for that topic?
- Why is the school's main URL domain (i.e. school.edu) showing as a Top Page on the Dashboard when the school doesn't have the virtual assistant embedded on that page?
- Why do the statistics on the Dashboard's Response Types graph not match the number of interactions in the Interactions area?
- Why do my analytics not match my interactions?
- Communication Opportunities Behavior
- What email address will the Communication Opportunities Contact Form (Intake) Transcript come from, and what is the subject line of the message?
- How can I see the Communication Opportunities contact form transcripts sent to our office?
- What is the average Communication Opportunities percent across all Ocelot clients?
- Transcripts and Interactions
- Awaiting Review
- User Experience
General
What is the difference between the AI Departmental Model and office filters?
Office filter
When an office is set up, AI Department Learning Models are attached to that office. Depending on the setup of the virtual assistant, multiple offices could have the same AI Department Learning Models, different AI Department Learning Models, or multiple AI Department Learning Models associated with that office. All AI Department Learning Model content associated with that office will display based on that filter.
Example: The financial aid office is always assigned the financial aid and financial wellness AI Department Learning Models. The Office filter for financial aid will include all questions or analytics for both AI Department Learning Models. AI Departmental Model filter
The AI Departmental Model filter displays the knowledge base questions that have the AI Department Learning Models selected associated with them.
Can Ocelot help identify students who use the virtual assistant?
When the inquiry form is enabled and configured, the virtual assistant presents selected questions to users as soon as the virtual assistant is opened in order to collect user information.
If the inquiry form is not configured and enabled, Ocelot is only able to identify the IP address associated with a person using the virtual assistant. Unless the user self-identifies, no identifiable information other than the IP address from which the virtual assistant was accessed can be provided in the event a client needs to determine a user's identity from an interaction.
Analytics
Are links in the welcome message counted in analytics?
If a link is included in the virtual assistant welcome message, the link may appear in the Top Clicks and Top Pages in your virtual assistant analytics:
Are links from the explore bar incorporated into the analytics page?
No. Clicks on links from the explore bar are not currently tracked in our analytics.
When will page-based analytics by office start once URL Matching is completed?
If a client has not yet completed URL matching, their virtual assistant analytics will include an Office filter and will only be associated with an office based on:
- The office a user selects to speak with a live agent.
- The office a user selects to contact because they received a Communication Opportunities response from the virtual assistant.
- If a user does not engage the virtual assistant in any of the above ways, the conversation is not associated with an office for analytics.
If a client has completed URL matching, they have access to by-office (i.e. page-based) virtual assistant analytics. Analytics will be associated with a specific office based on:
- The office the user selects to speak with a live agent.
- The office the user selects to contact because they received a Communication Opportunities response from the virtual assistant.
- The office the user selects to contact when they ask the virtual assistant for office contact information.
For more information on how to set up URL matching, review the Look & Feel: URL Matching article.If my domain changes, how will this affect page-based analytics?
Historical page-based (i.e., by-office) virtual assistant analytics will not be impacted by a domain change. To ensure that all analytics going forward are not impacted, you will need to update your mapped URL for each office with the new URL.
Why does the Top Topic statistic on the Dashboard not match the number of Interactions when searching for that topic?
Searching for a keyword on the Virtual Assistant Interactions page only returns interactions that contain the exact keyword entered in the Search field.
The Top Topics results displayed on the Virtual Assistant Analytics page have been identified using AI and include all virtual assistant interactions related to a keyword, including synonyms, misspellings, etc.
Why is the school's main URL domain (i.e. school.edu) showing as a Top Page on the Dashboard when the school doesn't have the virtual assistant embedded on that page?
This is due to security settings on these webpages. They have elected to include a header or meta tag on some pages restricting the referrer to "origin," meaning that external applications can only see the domain name rather than the full requested URL.
This setup is typically used for sensitive pages where the URL might contain, for example, a user's search keywords. Some schools include this security tag on every page.
Why do the statistics on the Dashboard's Response Types graph not match the number of interactions in the Interactions area?
Some response types are not tracked in the Response Types graph on the Virtual Assistant Analytics page. The Response Types graph includes the following response types:
- Knowledge base
- Smalltalk
- Suggestion boxes
- Search (links)
- Communication Opportunities
- Generative AI
- Integrated
- Other
Other includes responses when the virtual assistant is unable to determine the user's intent and must ask the user to rephrase their request to clarify and be more specific, or to ask one question at a time. These instances primarily include user inputs that are several sentences or paragraphs long, contain a detailed personal narrative, or are a single word.
Why do my analytics not match my interactions?
If you're experiencing a discrepancy between your analytics and interactions, there are a couple of reasons why this might be happening.
- An AI Department Learning Model was removed from your virtual assistant: If you've removed an AI Department Learning Model or office from your virtual assistant, the analytics for that AI Department Learning Model or office will be tracked in your overall analytics. However, you will not be able to filter by that office or AI Departmental Model within the Analytics page or Interactions page.
- A conversation did not have a knowledge base interaction: If a conversation did not have a knowledge base interaction, it will not appear in the analytics filtered by that AI Departmental Model. This can happen if a user asks a question that is not within your knowledge base, such as smalltalk or a Communication Opportunity response. Smalltalk and Communication Opportunity interactions are not assigned an AI Department Learning Model.
Communication Opportunities Behavior
What email address will the Communication Opportunities Contact Form (Intake) Transcript come from, and what is the subject line of the message?
If a client has customized their virtual assistant to provide users with the option to complete a Communication Opportunities contact form when the virtual assistant is unable to provide a response, any completed forms and their accompanying conversation transcripts will be emailed to the Communication Opportunities Form Contact Email set up on the Behavior Settings page. This email will come from webmaster@ocelot.ai and will have the subject "Virtual Assistant Submission from 'Student Name' (name@studentemailaddress.com)." 
Note: Only individuals with Administrator-level access can configure virtual assistant behavior settings globally and on a per-office basis.How can I see the Communication Opportunities contact form transcripts sent to our office?
The Communication Opportunities contact form transcripts can be viewed on the Virtual Assistant Transcripts or Interactions page in the Admin Portal.
View the Communication Opportunities contact forms sent to your office under Virtual Assistant > Transcripts or Interactions.
In the search bar, enter the Form Submission Success Message, found on the Behaviors Settings page, under Communication Opportunities Settings. You can narrow your search by filtering by a date range and/or a specific office.
To find the Form Submission Success Message, go to the Behavior Settings page.
To update the Communication Opportunities behavior, review the Behavior Settings: Communication Opportunities Settings article.What is the average Communication Opportunities percent across all Ocelot clients?
Across all clients, 5-7% of all interactions result in a Communication Opportunities response. This means that 93-95% of virtual assistant responses come from a match in the knowledge base, suggestions to related knowledge base responses, or links from the website.
Although the virtual assistant is always learning and you should see a gradual decrease in the frequency of Communication Opportunities responses over time, you will likely never be able to eliminate all of them; about 5% of questions/comments the virtual assistant receives are undecipherable or about topics that the virtual assistant is not designed to answer. Having less than 10% of your virtual assistant's interactions result in a Communication Opportunities response is considered successful and is often achievable through some moderate response customization (i.e. completing the Question Templates).
Transcripts and Interactions
What is the compass icon next to a question in the virtual assistant interactions?
On the Virtual Assistant > Interactions page of the admin portal, a compass icon next to a question means that the end user selected that question while using the virtual assistant's Explore feature.
What is the average number of interactions per conversation that virtual assistants receive?
The average virtual assistant conversation consists of 2.9 interactions.
Awaiting Review
Who gets the Awaiting Review email notifications?
Users who have been designated to receive the awaiting review notification will be sent an email 30 days prior to the review date so the response(s) can be approved as-is or changed as needed.
If you would like to set up to receive the Awaiting Review email notifications or change your settings by following the steps outlined below.
- To access the Edit Profile page, select your initials in the top right-hand corner, and select Edit Profile.
- On the Edit Profile page, under the Phone Number section, review the Virtual Assistant Question Review Opt Out option.
To receive the email notification, make sure the check box is not selected.
The email notification is not filtered by office permissions. You may receive an email and have no questions that need to be reviewed by your area.
User Experience
What are the differences between the grey suggestions and suggestions in the Explore feature?
Suggestion boxes are the AI model's efforts to find a direct response to a user's input from the questions in a client's knowledge base. Relevant suggestions in response to user input means that the virtual assistant is functioning correctly.
The Explore feature provides added value by supplementing these suggestions with further options for users to delve into.
How does the virtual assistant know what links to present to a user?
Web knowledge source creation is the process of designation URLs that the virtual assistant uses to crawl client webpages and provide search results if a knowledge base answer or video is not available in response to a user’s question.
When there isn't a direct match to a user question in our knowledge base, the AI runs an algorithm against the client's knowledge source content. It then runs each potential match against the AI model and returns the top match, or gives the user a few relevant options.
The crawler can be run at an interval chosen by the client to ensure website content pulled into the virtual assistant is current. Unless otherwise selected by a client, the web knowledge source runs every other week.
For more information on creating and adjusting Knowledge Sources, review the Knowledge Sources article.What is Simplified Chinese? Is this Mandarin?
Mandarin and Cantonese are two dialects of Chinese; "dialect" refers only to spoken language. The virtual assistant communicates in Simplified Chinese, which is written.
Because Simplified Chinese refers only to the written word while Mandarin and Cantonese refer to spoken dialects, document translation--like a virtual assistant interaction--cannot be conducted in Mandarin or Cantonese, and Simplified Chinese is not used in spoken conversation.
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article