5Tuning Intelligent Search

Accessing MSQ

To access MSQ in the Oracle Service Cloud Navigation tree, click Service, Knowledge Base, Manage Search Query.

The Manage Search Query screen opens.

    Choosing the Working Locale

    You can set the locale (language and region, for example, English United States) to allow users to see data within the tuning flow from other locales.

    To change the locale, navigate to Manage Search Questions window, Choose a Working Locale pane, and select a language and region from the Locale drop-down menu.

      Working with Tuning Requests

      When you find a user question that is not returning the number or quality of answers you might expect, you can submit it as a tuning request to MSQ. Tuning a question, phrase or even a single word can have significant impact on search results returned for user questions.

      MSQ logs tuning requests that originate from the Tuning Request page within MSQ or from the Knowledge Advanced Analytics Advanced Search Summary component. For more information, see Use the Advanced Search Summary Component to Tune Questions

        Filtering the Tuning Request List

        You can filter the Manage Tuning Request list to display only active requests or only archived requests.

        At the Status drop-down menu on the Manage Search Questions screen, select one of the following options:

        • Archived Requests. This option displays only the requests marked as Reviewed.

        • Active Requests. This option displays only active requests, which are tuning requests that are awaiting some action.

        • All Requests. This option displays all requests.

          Edit a Tuning Note

          Use the following procedure to add or edit notes about tuning a question.

          1. Click the Edit Tuning Notes icon.

          2. Select Next to open the Review Tuning Requests screen.

          3. Enter notes in the text field and click OK.

            Add a Tuning Request

            You can enter a new tuning request or add a tuning request from the Manage Tuning Requests list.

            Use the following procedure to add a new request.

            1. At the Manage Search Query screen, type the question you want to tune in the Enter a Question field, and then click Tune Question. Or, you can select the Add Tuning Request button.

            2. Enter the question that you want to tune.

            3. Select the Priority of the tuning request.

            4. Add optional Notes to help you manage the request.

            5. Select Add.

            To add a tuning request from the Manage Tuning Request list:

            1. Click on the request you want to tune.

            2. Click the Open Tuning Request icon next to the question.

            The Tune Question screen opens with the selected question in the How Search Interprets the Question pane.

              Close a Tuning Request

              When you complete a review of a tuning request as part of your request management process, you change the status to Reviewed. This action closed the request.

              The application moves all reviewed requests from the Active Requests list to the Archived Requests list and changes the request status to Closed.

              The Delete Closed Tasks batch job automatically deletes closed tuning requests. This batch job runs once a week and deletes all requests with the status Closed for at least seven days.

              Use the following procedure to close a tuning request.

              1. Select the request to mark for review and select Mark Reviewed.

                The request you marked is deleted from the Active Requestslist and added to the Archived Requests.

              2. Select Archived Request from the Status drop-down menu to see the reviewed request with a status of Closed.
                Note: Tuning requests remain in the Archived Request list for seven days and then are automatically deleted.

                Deleting a Tuning Request

                You can delete a request from the Manage Tuning Requests list by selecting the request and then selecting Delete.

                The request is deleted.

                  Tuning Questions

                  You tune questions by performing the following tasks.

                    About the Intelligent Search Dictionary

                    When you tune a question you are adding or changing objects in the Intelligent Search Dictionary. The Dictionary is an Intelligent Search component that contains:

                    • Language information, in the form of a hierarchy of semantic relationships that describes words and phrases in multiple locales

                    • A rules engine that determines the best answers to users’ questions

                    Intelligent Search uses the Dictionary to analyze and index your knowledge base content, and to match users’ questions to the best answer.

                      About the Objects and Relationships in the Dictionary

                      Intelligent Search uses concepts to match words in both users' questions and in the knowledge base content based on their meaning. It also recognizes that some concepts are more important to users than others, and uses that information to prioritize answers within search result. See the following example.

                      • For the concept plan, you might add the synonyms agreement or proposal

                      • For the concept picture, you might add the synonyms photo, photograph, or image

                      The Dictionary stores these objects and relationships in base, industry, and customer-specific layers; when you use MSQ to tune user questions, your changes affect only the customer-specific layer.

                      When you use the MSQ, you work with the following types of objects and relationships.

                      • Concepts

                      • Synonyms

                      • Products

                        About Concepts and Synonyms

                        The Dictionary uses concepts and synonyms to associate different words that have the same meaning with one another. A concept is simply a word and its synonyms, which the Dictionary treats as a single object.

                        Intelligent Search uses concepts to match words in both users' questions and in the knowledge base content based on their meaning. It also recognizes that some concepts are more important to users than others, and uses that information to prioritize answers within search results.

                          About Products

                          Knowledge Advanced automatically creates product concepts in the Dictionary for all products in the Service Cloud hierarchy, and automatically updates the Dictionary when you change or delete products. You cannot delete Products from the Dictionary using MSQ; you can delete them only by modifying the Service Cloud product hierarchy.

                            How Intelligent Search Interprets a Question

                            Each word or phrase in your question is represented by a button in the How Search Interprets the Question pane.

                            • Each button with an X represents a word that the tuning process skips.

                            • Each button with a light bulb icon represents a word that exists as a concept.

                            • Each button with two pages represents an unknown word.

                              Actions To Take Menu

                              The Actions To Take menu lists the tuning options for each object. Each object has a set of tuning actions, as displayed in the following table.

                              Table Actions To Take Menu

                              Object Actions To Take

                              Concept

                              Edit Concept. You can modify the:
                              • Enabled selection

                              • Locale

                              • Description of the concept

                              For information on editing concepts, see Edit Concepts.

                              Add to Existing Concept. You can add synonyms to this concept.

                              For information on adding synonyms to existing concepts, see Add to Existing Concepts.

                              Delete Concept. You can delete this concept.

                              For information on deleting concepts, see Delete a Concept.

                              Product

                              Edit Product. You can add a synonym to this product.

                              For information on editing products, see Edit a Product.

                              Unknown word

                              Fix Spelling. You can change or correct the spelling, which may return a concept that already exists.

                              For information on making spelling corrections, see Correct Spelling.

                              Create New Concept. You can complete the form to add the unknown word as a concept

                              For information on adding concepts see Add a Concept.

                              Add to Existing Concept. You can add synonyms to this concept.

                              For information on adding synonyms to existing concepts, see Add to Existing Concepts.

                                Select or Enter a Question to Tune

                                Use the following procedure to add or select a question to tune.

                                1. In the Manage Tuning Requests pane, select a question in the Description column.

                                  • Or you can enter a new question in the Enter a Question field.

                                2. In the Actions column on the far right, click the Open Tuning Request button.

                                  The question appears in the Tune Question window.

                                  Select a Word to Tune

                                  Use the following procedure to select one or more words to tune.

                                  1. Select the word to tune in the How Search Interprets the Question pane.

                                    If you are tuning a word with a down-arrow, you can click to see the synonyms associated with it.

                                  2. In the Actions to Take drop-down list, select the action you want to use to tune this question.

                                    The options are based on the object you select. For example, if you choose to tune a concept, the options in this drop-down menu include Edit Concept or Delete Concept.

                                  3. Click the arrow button.

                                    The Change Summary panel appears, listing the actions taken.

                                    Add a Concept

                                    Adding a new concept enables Knowledge Advanced to return results that match not only on a particular word but also its synonyms and related concepts.

                                    1. Enter the phrase, or word you want to add or edit in the Enter a Question field and then select Tune Question.

                                    2. At the Tune Question screen, select the word or the word in the phrase you want to edit in the How Search Interprets the Question area.

                                    3. Select Create New Concept in the Action to Take drop-down menu, and select the arrow.

                                    4. Complete the following fields on the Create New Concept popup window.

                                    Table Create New Concept Popup Window Fields

                                    Field Action

                                    Concept Name

                                    Enter the name for the new concept.

                                    Enabled?

                                    Select Yes. This is the default.

                                    If you select No, Knowledge Advanced ignores the concept.

                                    Description

                                    Enter the reason adding the concept and a description of the edit, if necessary.

                                    Synonym

                                    Add a synonyms for this concept. A synonym is a word having the same or nearly the same meaning as another word in the language.

                                    For example, for the concept plan, you might enter the synonym contract.

                                    Locale

                                    Select the locale (language and region) for the new concept.

                                    Warning

                                    This column shows warnings associated with the synonyms you add, for example if you add a synonym that is a duplicate within the list of synonyms.

                                    Add a synonym and select Change, and any warning for that synonym displays.

                                      Add to Existing Concepts

                                      You can add synonyms to existing concepts.

                                      Note: It is best to enter all synonyms in lowercase and in their unmarked (stemmed) form. Knowledge Advanced automatically matches the marked forms of that word, such as plural nouns or conjugated verbs. You may enter special characters, such as Chinese characters, into these synonyms field.
                                      1. In the How Search Interprets the Question pane, click the word that represents the concept.

                                      2. In the Action to Take drop-down list, select Add to Existing Concept and click the right arrow.

                                        The Add to Existing Concept popup window appears.

                                      3. Complete fields on the Adding to Existing Concept popup window.

                                        The Adding to Existing Concept Popup Window fields table follows this procedure and describes the fields.

                                      4. Click Add to return to the Tune Question window.

                                        The Change Summary pane shows the edited concept.

                                      5. Click Next to continue tuning.

                                      The following table describes the fields to add to the Existing Concept Popup window.

                                      Table Adding to Existing Concept Popup Window Fields

                                      Field Description/Input

                                      Add This Synonym

                                      Enter the synonym you want to add.

                                      In This Locale

                                      Select the language and region for the synonym.

                                      Add to Which Concept?

                                      Enter the existing concept and select the Search icon. A list of similar concepts displays in the Matches box.

                                      Select the concept and click Add.

                                        Edit Concepts

                                        Use the following procedure to modify the existing concepts.

                                        1. Enter the word you want to add or edit in the Enter a Question field and then select Tune Question.

                                        2. At the Tune Question screen, select the word or the word in the phrase you want to edit in the How Search Interprets the Question area.

                                        3. Select Edit Concept in the Action to Take drop-down menu, and select the arrow.

                                        4. Complete the fields on the Edit Concept popup window.

                                          The Edit Concept Popup Window Fields table follows this procedure and describes the fields.

                                        5. Select Change.

                                          The change action appears in the Change Summary box.

                                        6. Select Next.

                                          The change action appears in the Change Summary box.

                                        The following table lists the fields and actions to take to edit concepts

                                        Table Edit Concept Popup Window Fields

                                        Field Description

                                        Enabled?

                                        Select Yes. This is the default.

                                        If you select No, Knowledge Advanced ignores the concept.

                                        Description

                                        Modify the description of the edit.

                                        Synonym

                                        Add a synonym for this concept.

                                        Locale

                                        Select the language and region for the new concept.

                                        Warning

                                        This column shows warnings associated with the synonyms you add, for example if you add a synonym that is a duplicate within the list of synonyms.

                                        Add a synonym and select Change, and any warning for that synonym displays.

                                          Delete a Concept

                                          You may want to delete concepts that are no longer applicable or are duplicates. Use the following procedure to delete concepts.

                                          1. At the Tune Question screen, select the concept to which you want to delete in the How Search Interprets the Question area.

                                          2. Select Deleting Concept in the Action to Take drop-down menu and select the arrow.

                                          3. At the Deleting Concept popup, view the concept and associated concepts to ensure you really want to delete the concept.

                                          4. Select Delete, and at the confirmation popup, select ok.

                                          The concept is deleted.

                                            Correct Spelling

                                            The tuning process corrects common, misspelled words automatically. However, if you enter a word that the tuning process does not recognize, or it returns a word not synonymous with your meaning, you must correct the spelling manually.

                                            1. In the How Search Interprets the Question pane, click the button of the misspelled word.

                                              The Fix Spelling window appears.

                                            2. In the Should Correct To field, enter the correct word and click the Change button to return to the Tune Question window.

                                              The Change Summary pane shows the corrected word.

                                            3. Select Next to continue tuning.

                                              Edit a Product

                                              Use the following procedure to modify a product concept.

                                              1. Enter the product you want to edit in the Enter a Question field and then select Tune Question.

                                              2. At the Tune Question screen, select the word or the word in the phrase you want to edit in the How Search Interprets the Question area.

                                              3. Select Edit Product in the Action to Take drop-down menu, and select the arrow.

                                              4. Complete the following fields on the Edit Concept popup window.

                                              Table Edit Concept Popup Window Fields

                                              Field Description

                                              Enabled?

                                              Select Yes. This is the default.

                                              If you select No, Knowledge Advanced ignores the concept.

                                              Description

                                              Modify the generated description.

                                              Synonym

                                              Add a synonym for this product.

                                              Locale

                                              Select the language and region for the new synonym.

                                              Warning

                                              This column shows warnings associated with the synonyms you add, for example if you add a synonym that is a duplicate within the list of synonyms.

                                              Add a synonym and select Change, and any warning for that synonym displays.

                                                Review the Tuning Results

                                                You can review the tuning changes you made to determine whether they improved the search results by comparing the number and quality of the results returned before tuning and after tuning.

                                                Use the following procedure to compare the answers returned after you tune a question.

                                                1. At the Tune Question window, click Next or click Review Search Results at the top of the screen.

                                                  The Review Search Results window opens with the tuned question.

                                                2. To understand how results change after tuning, you need to review the search results before tuning. Click the Search Results Before Changes link to view and compare the search results before you tuned the question.

                                                3. Review the search results after tuning the question in the Search Results After Using the Tuned Question pane.

                                                  • The Improved? column shows an up arrow or down arrow next to a search result, which means the answer moved up the list of results (improved) after tuning, or moved down the list (demoted) after tuning. A dash means that the answer stayed in the same position after tuning.

                                                  • The Result column shows the search results for the tuned question.

                                                  • The User Click-Through column shows the number of times users viewed the search result in the Customer Portal. Users viewed answers with a higher click-through more often and viewed answers with a lower click-through less often. You can use this information to help assess how to position answers in the results list.

                                                    Generally, when more answers with higher click-throughs are promoted, or moved up the result list, the tuning had a positive impact. When more answers with lower click-throughs are moved up the result list, this can indicate that the tuning had a negative impact.

                                                4. You can review the tuning changes for this question in the Change Summary panel, and review any notes in the Tuning Notes panel.

                                                  Analyze Tuning Impact on Similar Questions

                                                  You can review how your tuning changes affect the health of your knowledge base by analyzing the impact on similar questions, or questions that share similar words or concepts to the tuned question. For example, for the tuned question Can I add a user to my family plan? you might see the following similar questions.

                                                  • Can I change my family booster?

                                                  • Can I add a line to my family plan?

                                                  • Can I add a family member to my current booster plan?

                                                  MSQ calculates whether the relevancy of the top answers returned to similar questions remains the same, improves, or diminishes after tuning the original question. It uses an algorithm comprised of several metrics to generate a score for each result before and after the tuning changes. Once these scores are calculated, the system compares these scores to each similar question before and after tuning.

                                                  Use the following procedure to analyze the tuning impact on similar questions.

                                                  1. Select Analyze Change (or select Next).

                                                    At the Analyze Change screen you see a list of the similar questions in the Impact of Tuning Changes pane.

                                                  2. At the How Changes Impact drop-down menu, select one of the following options. ( MSQ calculates certain characteristics (such as click-through rates, answer positions, etc.) that provides values for each option.)

                                                    • Clicked Document.

                                                      This is the number of times a user clicked through on an answer after asking a question. This is calculated by using metrics such as index positions and the number of answer click-throughs.

                                                    • Linked To Case. This is the number of incidents (cases) linked to an answer in a time period.

                                                      This is calculated by using metrics such as the answer relevancy score, index positions, and the number of incidents linked to the answer.

                                                    • Popular Document. These are the articles users opened most often in a time period.

                                                      The metric for this option is calculated by information such as the search relevancy score and the number of times the article was opened or clicked on.

                                                  3. View how the results returned by Knowledge Advanced are impacted by your tuning changes in the Impact of Tuning Changes pane. For each similar question in the list, you see:

                                                    • Improved.

                                                      This is a visual indication that represents whether the results set for this question improved. The icons are:

                                                      • The dash indicates the tuning changes added no change to the clicked article

                                                      • The up arrow indicates the tuning changes may have added positive value and made the search better

                                                      • The down arrow indicates the tuning changes may have detracted from the value and made the search worse

                                                    • Estimated Improvement. This is a numeric value that represents the positive or negative change. This is calculated using different algorithms based on the option you selected in the How Changes Impact drop-down menu.

                                                  4. When you have reviewed the tuning impact on all similar questions:

                                                    • If the tuning changes affected your knowledge base in a negative way (detracted from the value), select Cancel to return to the previous screens to modify or cancel your changes.

                                                    • If the tuning changes affected your knowledge base in a positive way (improved the value), select Publish to save all changes.