- Question. What do I do if I don't get the desired results from my KB?
Answer. Analyze the performance of your KB file using Banter's Workbench tool. See RME Deployment Best Practices Guide, and Banter Workbench User's Guide on Siebel eBusiness Third-Party Bookshelf for more information.
- Question. If I import a new KB that has some of the same categories, what happens to my existing categories?
Answer. After importing a new KB, new categories are appended to the categories list in the Categories Administration screen. Existing categories are not overwritten or deleted and you will not lose the associations of templates and solutions with a category. For new categories, you must associate templates and solutions. See Associating Templates and Solutions with Categories for more information.
- Question. What is Bootstrapping and when should we do this?
Answer. Bootstrapping is a method of training that allows the KB file to learn gradually as Smart Answer processes requests and users select templates and solutions associated with categories. You can use this training method when you start with an untrained KB file containing categories. Use this method when your corpus does not contain emails for all your categories and you need to get up and running quickly.
For example, you want to create a time-sensitive category for information pertaining to a new promotion that you created after training your initial KB. You do not have existing emails to use for learning the new category.
You cannot retrain your entire KB or you will erase your statistics. Therefore, you must add the category in two locations. Add it to your catalog in eMail Response and to your KB file in Banter's Workbench. Then, you inform your agents to choose it for specific emails related to that promotion and your category will gather feedback as it is used by the agents.
- Question. How long does it take a category to learn?
Answer. Learning time depends on the volume of incoming requests. It can take between two hours and two days.
- Question. How do I change Smart Answer to return more or fewer than five categories?
Answer. Smart Answer assigns a confidence score to the five highest-scoring categories and these categories appear at the top of the categories list in the Outgoing Message form in the Communications Detail Response view.
You can override the default value for the number of categories that will appear in the server component parameters for Smart Answer Manager server component. Navigate to Site Map > Server Administration > Server Components, then query for Smart Answer Manager server component. Click the Component Parameters tab and query for the Maximum Number of Categories parameter. You can set the value for the number of returned categories in the Current Value field.
- Question. Which has a higher priority, the keyword or the learning in the KB?
Answer. The keyword has the priority. Keyword matches (or as Banter calls them, Rules) override any statistical modeling or intent analysis that Smart Answer does. That is why they should only be used when you are absolutely sure you want to categorize an email a certain way regardless of the email's intent. Otherwise, you should use the NLP of Smart Answer to determine the intent.
- Question. The KB file hasn't been updated since it was imported to the Siebel Server for production. How will the learning be in this file if it hasn't been updated?
Answer. The learning is stored in memory at first, thus you will not see the file size increase. The file is updated at specified intervals that you can set. There is a component parameter on the Smart Answer Manager server component called KB Write Back interval. This is where you specify the number of feedback messages required before the memory copy is written back to the file. The default state is set at 100, so after 100 emails, or messages where feedback has been sent, the KB will be updated on the file system. Otherwise, you are just accessing the memory copy of the KB. To locate this parameter, navigate to Site Map > Server Components > Smart Answer Manager > Component Parameter KB Write Back Interval.
- Question. The KB has been in production for a while. How can I analyze how my KB is performing with all the learning and feedback that has occurred in production?
Answer. After running Smart Answer for a sufficient amount of time, you should take the .kb file from the cms directory, and put it in the folder where your Workbench Project is for the initial creation of the KB. Then when you open the project using Banter's Workbench Tool, point to this KB and run the various Workbench reports to see the performance of your production KB. You may also create more messages and run Analyze only on those selected messages. The match statistics against the categories should show you the performance of your production KB.
You can also start a new project, point to this new KB, and import your corpus from your existing project folder where you created the KB.
NOTE: Be sure to make a copy of your KB before you load it back into Workbench for analysis.
- Question. After training the KB with additional sample emails, can I load the KB back into production without rebooting the server? Also, can this be done during operations or running, or should this update be done after business hours?
Answer. You can update existing categories with sample emails by applying feedback against a category while the KB is in production within your Siebel system. Send in an email, select the appropriate category, click the Send button and positive feedback is applied to a category. The learning process will occur in real time. If you want to work with the existing KB in production, the learning process will occur in real time.
If you want to add a new category, you will need to add it to your KB using Banter Workbench, apply learning to just that category, copy the new KB file to the cms directory, and add the new category in Catalog Administration screen for the KB. You will not have to reimport the KB, however. The reimporting process loads the categories in the database. As long as the categories in your .kb on the Siebel file system match exactly the categories in your Catalog in the Siebel application, there is no reason to reimport. Similarily, you will not have to stop and start the Siebel Server. Stopping and starting the Siebel Server causes the KB file to be picked up again by the Smart Answer Manager server component, but since the .kb file name has not changed, there is no reason to restart the server.
If you want Smart Answer to point to a completely different KB with totally different categories, then it might be easier to reimport than to create a structure in the Siebel Server and that same structure in KB Editor. And if the name of your .kb file changes, then you should reimport so that Smart Answer Component picks up the correct KB. You would also need to stop and start the Siebel Server, so you would need to do this during nonbusiness hours to avoid disruption of bringing the server component down and up again.
- Question. Does importing the same KB again mean that all the solutions/response templates have to be associated to the categories again?
Answer. No, if you load the KB again, and it has the same categories as the one already in your Siebel file system, the Solutions/Response templates associated with the old KB categories will remain associated with the new KB categories imported. However, if you add a new category to your KB, then you will need to associate new templates and solutions to this new category.
- Question. How does the KB handle a message with multiple intents?
Answer. When replying to an email, the agent is not limited to choosing only one category or one template or solution. Smart Answer suggests categories and the agent can choose one or more categories and one or more templates and solutions that are associated with each category. When the agent clicks Send, feedback is generated and then stored in the KB file for each category associated with the templates and solutions selected by the agent.
- Question. How many emails do you need to train a category?
Answer. At least five per category. However, to fully train a KB file, it is recommended that you use between fifteen and twenty emails per category.
