Editing the Knowledge Base FAQs

This section of the user guide is focused on FAQs related to editing the knowledge base. To help you navigate the article we have broken it down into the following sections:

• General
  • Does Ocelot review general library content on a regular basis?
  • Is there a character limit for responses?
  • Can I get a spreadsheet of the knowledge base?
  • What is the average number of custom questions per department that clients have?
• Understanding the AI
  • How can I add alternate invocations or different ways to phrase a question?
  • Why can't we change the top-level question wording of general library questions?
  • Why can't I add every program or location to the virtual assistant?
  • What are intents and entities?
  • How does the virtual assistant respond to self-harm questions?
  • How does Ocelot handle gender/ racial biases within the AI?
• Videos
  • What videos will be displayed in the virtual assistant?
  • Can I add videos to the Explore bar?
• Moderation Queue
  • What are the Ocelot moderators looking for in the moderation queue?
  • In the Moderation Queue, why do many of the questions say last edited by user "Ocelot"?
  • How long are questions in the moderation queue?
• Integrated Questions
  • What is the purpose of the Webhook fields in the virtual assistant question editing pages?

General

Does Ocelot review general library content on a regular basis?

Ocelot's Compliance and Content Review Committee oversees the monitoring, updating, and implementation of Ocelot product content related to higher education compliance and best practices. 

The committee creates and revises content which may include creating new virtual assistant templates, virtual assistant general library content, and general library video content.

The following topics are not within the scope of the committee to monitor for compliance within the virtual assistant: 

• Policies related to state higher education 
• Client custom virtual assistant questions, including client edits of general library content
• Human Resources

Is there a character limit for responses?

There is no character limit for virtual assistant responses. However, Ocelot recommends the following best practices to provide the best user experience.

For best practices on writing a response, review the Editing the Knowledge Base Best Practices article. 

Can I get a spreadsheet of the knowledge base?

Due to the proprietary status of Ocelot's general library knowledge base, the entire general library cannot be exported. A limited export of the top 25 General Library questions is accessible.

Custom Questions can be exported in their entirety.

The average department creates about 200 custom questions. Custom questions include any new questions a client has created that are specific to their institution or organization. Custom questions also include completed question templates and client customizations of general library questions.

Understanding the AI

How can I add alternate invocations or different ways to phrase a question?

Alternate invocations are similar questions or statements with the same goal. 

Example:
Question: How do I apply for financial aid? 
Alternate Invocation: What is the application process for financial aid?

Adding alternate invocations to questions helps your virtual assistant understand the many different ways users might ask the same question. The Moderation Queue team adds alternate invocations to the general library and client custom questions on behalf of our clients so they can spend less time thinking about how their users might ask questions and can focus that energy on developing great content. On any given day, the Moderation Queue team creates hundreds of examples in addition to those submitted by our clients.

Every time a client submits a new question, the question comes through the moderation queue, where it is evaluated for functionality and potential AI conflicts. As the Moderation Queue team reviews a newly submitted question, they determine if the question needs to be connected to an existing general library or client custom question. 

In cases where the Moderation Queue team connects a new question to an existing question, they are adding an alternate invocation based on this new phrasing. If the team is not able to connect a new question to an existing question, they add several alternate invocations to this new client custom question on the client's behalf.

If you think you may have a new way to ask a question, be sure to test your phrasing first to see if your virtual assistant understands your question. You can test your phrasing by selecting the Test my Virtual Assistant icon in the upper right-hand corner of the admin portal home screen. If you receive an answer from the knowledge base or relevant suggestion boxes then your virtual assistant understands what you asked and you have had a successful interaction. 

If you receive an response of "Sorry, I don't have the answer to that."  (now referred to as a Communication Opportunity) to a question you would like the virtual assistant to be able to answer, select the Add a Response + button at the bottom of the interaction. You can then create a new question and response. Saving this question will submit it to the moderation queue for the Moderation Queue team to review.

You do not need to create and submit every possible phrasing of a question. Submitting a few examples helps to train our AI model. Over time, your virtual assistant will continue to learn additional phrasings.

Why can't we change the top-level question wording of general library questions?

Top-level questions are the question titles that show in the knowledge base. These are the titles that appear in suggestion boxes and the Explore feature for end-users. The top-level questions represent a singular goal with many similar questions or statements behind the scenes that are not visible in your admin. In your knowledge base, you will see the top-level questions and Ocelot manages the many alternate phrasings that live behind the scenes.

Client-specific responses are tied or linked to similar general library questions so they benefit from our global AI training. When the top-level question wording of a general library question is changed, it breaks the link between the general library and client-specific version of that question.

Why can't I add every program or location to the virtual assistant?

Ocelot teaches our AI model about the numerous program options at a school by creating comprehensive questions such as "What majors or programs are available?" and training it with hundreds of program-specific examples such as “Do you offer nursing?”  In essence, we are combining programs into one broad question as we have found it greatly increases the chances of users getting the information they need and is significantly less upkeep for our clients. If we did not make this decision, there would be thousands of program offerings and questions for clients to review individually, and it would stretch our AI model in a manner that would make it less effective for users. 

Similarly, Ocelot has trained our AI model to recognize numerous location options by creating broad questions such as "Where is the school located?" and training it with specific examples such as "How many campuses does the school have?" 

Additionally, virtual assistant users may not know the exact title of an institution's specific program or precise campus location, and customizing the response to these comprehensive questions provides a much better user experience. For example: an "English" major may be called "English Language and Literature," “English Literature," or "English Language and Writing" etc. A prospective student is the most likely to just ask about an institution's "English major." Likewise, users from outside the Chicagoland area may not make a distinction between the city of Chicago and the city of Naperville and may use "Chicago" to reference both.

With this structure in mind, we cannot add individual questions about programs, classes, or locations since we do not have the framework and training data to effectively parse out each and every program, course option, or location name as individual questions. Our current structure provides the highest chance for a user to get the information they need through a direct knowledge base response, suggestion box, or search link.

What are intents and entities?

Intents

Intents are categories of goals or purposes. For example, if a virtual assistant is asked “Where is the Billing Office?” the intent is "Location Information." We evaluate our intents to ensure we have a comprehensive range and quality to match client submissions and user data. When a client submits a new question, we evaluate if that question should live in an existing intent or if it is more appropriate to create a new one.

Entities

Entities are keywords or terms relevant to the intent. The team manages entity development which includes responsibly expanding the vocabulary of the virtual assistant. We look to connect synonyms such as “Dorms" and “Residence Hall.” We also ensure our AI understands common misspellings or suggestions.

How does the virtual assistant respond to self-harm questions?

Ocelot has trained our AI model to identify language around self-harm. This training is linked to the question "What resources are available to support students experiencing a crisis?" and allows you to provide campus-specific resources to help your students. We encourage you to give a comprehensive answer that covers all types of crises such as mental health, sexual assault, etc. 

You can find the question "What resources are available to support students experiencing a crisis?" in the question templates section of your client admin. 

For information about how to find this question template and how to complete it, review the Complete Question Templates article. 

How does Ocelot handle gender/ racial biases within the AI?

Ocelot implements industry best practices to control bias. Our virtual assistant does not identify users by race, ethnicity, nationality, gender, or religion. We train our AI to identify a user’s stated goal and topic and then provide answers and/or suggestions. We also take additional steps to reduce bias:

Diverse Teams: We ensure that our algorithms, content, and maintenance of our AI model are supervised and written by diverse voices. 

Equitable Data: Our datasets and training are based on more than 11 million user interactions across more than 400 institutions, including Historically Black Colleges and Universities (HCBU) and Hispanic-Serving Institutions (HSI), non-native English speakers, and users outside of the United States.

Supervised Learning: Our AI Conversation Design team monitors our model and user interactions to ensure responsible and equitable growth.

Videos

What videos will be displayed in the virtual assistant?

The virtual assistant will display standard video portal content either in the response directly, or as a suggested video when the virtual assistant determines the video is related to a question the user asked.

If you have created custom videos in your video portal, you can hyperlink to the video in your video portal. 

Custom videos in your video portal will not be displayed as suggested videos in the virtual assistant.

Can I add videos to the Explore bar?

Videos cannot be added to the explore bar. You can direct students to a video with a hyperlink to your video portal.

Moderation Queue

What are the Ocelot moderators looking for in the moderation queue?

When a new question is submitted, the Moderation Queue team reviews the question to ensure your virtual assistant is operating at its best. Our goal is to take our collective 200+ years of higher education knowledge and experience and apply it to our AI model. The Moderation Queue team takes on the management of our AI model so clients can focus on creating the most effective content for their users. 

Here are the steps we take:

1. Testing: Moderators test each submission for relevant suggestion boxes and conflicts with existing general library or client-specific content. If a client submits a question that matches existing content, moderators will need to combine or customize the question in order to maintain Ocelot's existing AI foundation.  
   
   
2. AI Evaluation & Training: Moderators evaluate each submission to determine if we can train our AI model to match it with existing general library or client-specific content. If the submission does not have a high enough degree of relevancy or level of detail that matches existing content, moderators will leave it as a standalone question. 
   
   
3. Similar Phrasings (Alternate Invocations): Moderators add multiple ways of asking the question on behalf of our clients. Alternate invocations teach the AI to understand similar questions or statements with the same goal, as well as additional phrasings and keywords to help your virtual assistant provide a 1:1 response, relevant suggestion boxes, or website links. On any given day, Ocelot creates hundreds of examples in addition to those submitted by our clients.
   
   

We do not review or edit responses, review dates, content locking, content sharing, or expiration dates.

In the Moderation Queue, why do many of the questions say last edited by user "Ocelot"?

When a client submits new content, the Moderation Queue team reviews the question to ensure their virtual assistant is operating at its best. The moderation queue shows the status of a client's custom questions as they're being reviewed by this team.

The User field indicates the last person who edited the question. If "Ocelot" is the User, it means that a member of the Ocelot team is the last person who handled the question; this most often occurs as part of the process of the team reviewing and making the question live in the client's knowledge base.

How long are questions in the moderation queue?

When a client creates a new custom question or changes the wording of an existing custom question, it will move into the moderation queue to be reviewed by Ocelot staff. The Moderation Queue team ensures that the question will work in the client’s virtual assistant and provide a response within the parameters of the AI model. 

Moderators review new questions for AI conflicts, and add multiple alternate invocations, or similar phrasings, in order to train the virtual assistant to understand a variety of ways that users might ask the same question. This process helps ensure users will receive the correct response to the new question regardless of how it is asked. 

Questions in the moderation queue will be reviewed within 3 - 4 business days. Please allow 3 - 4 business days before testing any newly submitted questions in your virtual assistant.

Integrated Questions

What is the purpose of the Webhook fields in the virtual assistant question editing pages?

The webhook fields can be found on the editing page when either creating a new custom question or editing an existing question. These webhooks are used when integrating the virtual assistant with your campus Student Information System (SIS) to provide personalized responses.

Ocelot has set up over 45 web capabilities for common questions about Financial Aid, Admissions, Advising, Student Services, and more.  These web capabilities help schools quickly connect the SIS to the virtual assistant to help answer the most frequently asked questions.

Ocelot can integrate with Peoplesoft Campus Solutions, Ellucian Banner, Ellucian Colleague, and Anthology Student. Other SIS integrations will be available in the future.

For more information on SIS integrations, review the Student Information Systems (SIS) Integration with Ocelot article.