This guide outlines the key steps to successfully launch Automatic Content Generation (Gen AI) in your Ocelot Virtual Assistant. From preparing your Knowledge Sources to testing, activating, and monitoring Gen AI, this article provides best practices and helpful resources to ensure your chatbot delivers accurate, real-time answers using your institution’s web content.

Automatic Content Generation Support Article - full overview of Automatic Content Generation 

1. Knowledge Sources 

   1. Set up Knowledge Sources for all offices that are in Ocelot

   2. Best Practice - each department or concentration area should be its own Knowledge Source

   3. Ensure that all key URLs and documents are setup as Knowledge Sources

   4. Determine if there are any keyword or URL exclusions

   5. Determine how frequently you want AI to re-scrape over your web pages

   6. Knowledge Sources support article

   7. Knowledge Sources FAQs article
      
      

2. Test Your Virtual Assistant - (this step can be skipped if you are comfortable going to the “Live” status)

   1. The testing process is to ensure that Gen AI is answering correctly. 

   2. The first step is to open the Test My Virtual Assistant portal in the top right corner, then click the three action dots at the top right, and toggle on enable in this test conversation. 
      
      This activates Gen AI only in the testing portal for that one user. It will not activate it in live production or for any other users. 
      Once this is activated you can start asking questions. 
      If the response is a Knowledge Base response, you will see a Edit Response icon under the response. If the response is a Gen AI response, you will see an icon labeled “Gen AI” under the response.

   3. You will want to review both the actual response and the sources. Below each response is an icon labeled Sources. When a Gen AI answer is given, the listed Sources are the web pages that we pulled the information from to use in the response. 
      

   4. If you find that answers are incorrect (contextually inaccurate, missing partial information, etc) review the sources to identify if we are pulling in the correct web pages. 
      If it seems that the web page with the best info for the response is missing, be sure to add that as a Knowledge Source. 
      If the correct web page is listed as a source, flag that question and save it for later to be reviewed with your Ocelot CSM.

   5. If Gen AI continues to get a type of question wrong or cannot answer a specific question we can always create a Custom Knowledge Base question to use as the proper response.

   6. IMPORTANT - You will see that a large majority of responses are still going to come from the Ocelot Knowledge Base. As a reminder, Gen AI is only providing a response when a Knowledge Base response is not available. 
      When testing be sure to ask more obscure questions, as traditional chatbot questions will likely yield a Knowledge Base response.

   7. Testing With Gen AI support article
      
      

3. Activate Gen AI for a Single Office

   1. Identify the office where Gen AI will be launched first. Follow the instructions on this article to activate a single office.

      1. If you wish to turn it on for all offices, please review the  "How do I turn on Automatic Content Generation per Office?" section of the "Virtual Assistant: Automatic Content Generation" article. 

   2. Setting Parameters

      1. We provide you the ability to adjust verbosity and creativity of Gen AI answers. The sliders come set to the default setting, feel free to adjust as needed based on your testing
         

   3. Knowledge Source Selection
      

      1. This allows you to limit the scope of what web pages Gen AI uses to answer a question. 
         For example, if you plan to launch Gen AI with the Financial Aid office, you may want it to only respond to questions related to financial aid when users are on financial aid-related web pages. To achieve this, adjust your Knowledge Source selection to include only the Financial Aid Knowledge Source. This ensures that Gen AI will reference only those specific web pages when generating responses on financial aid URLs.

      2. If you want Gen AI to have the ability to answer any question that is asked, no matter what webpage you are on, you will not make any Knowledge Source Selection.

      3. Please note - Knowledge Source Selection does not prioritize or rank certain sources over others. It simply defines which web pages Gen AI is allowed to reference when generating a response. All selected Knowledge Sources are treated equally by the system.
         
         

4. Monitor Gen AI

   1. Now that Gen AI is active, we need to continue to monitor responses for accuracy.

   2. The most effective way to monitor Gen AI responses is by navigating to the Interactions tab and filtering by Response Type: Generative AI. This will limit the view to only show responses generated by Gen AI
      

   3. Be sure to review these responses for accuracy. If you find incorrect answers, review the Sources to ensure the proper web pages are being viewed. You can then adjust the Knowledge Sources to more accurately capture the right pages, or create a custom question to better address the inquiry.

   4. As a best practice, review Gen AI responses weekly for 30 to 60 minutes during the first month. After that initial period, you can scale back to reviewing every other week, and eventually move to a monthly review schedule

   5. Interactions support article

         Side Notes:

• While Gen AI is highly effective, it is not flawless. You can expect an accuracy rate of approximately 95% for its responses.

• It’s essential to regularly review the Sources to ensure your website does not contain conflicting or outdated information. For example, one page might state that a specific form must be hand-signed, while another allows for an electronic signature if a staff member is present. Even small discrepancies like these can cause confusion and impact the accuracy of Gen AI responses

• When major website updates are made, Force Run the Knowledge Sources the ensure the new information is scraped. 

• Gen AI responses cannot be directly edited, and there is no way to control or customize the specific wording used. Additionally, you cannot modify disclaimers or add supplemental information beyond what is automatically generated by the AI

• Each question is treated as a new query for Gen AI. If you ask the exact same question multiple times you will likely see that it is worded differently each time, but the overall information is the same.