43 Insights for DA as an Oracle Service Cloud Agent

These insights enable you to monitor how well the intents handle the user requests that are routed through Oracle Service Cloud Chat. These insights apply only to skills that are registered to digital assistants that act as an automated agent, not skills that interact with users and hand off the chat to a live agent.

Report Types

Click Insights (This is an image of the Insights icon.) in the left navbar to access the following reports:.
  • Overview – Lets you monitor the overall trends in intent response and escalation rates as well as the frequency at which individual intents are invoked.
  • Retrainer – Use Insights gathered by this report to improve the training corpus for intents.
  • Export–Lets you download Insights data to a CSV file so that you can use it to create your own report.

The Overview and Retrainer reports are specific to the skills created for integration with Oracle Service Cloud Chat. For other skills, see Insights.

View KPIs for Responses and Escalations

This report tracks the total number of requests handled by the agent skill, which is the sum of utterances that were successfully handled within the skill and the number of utterances that were transferred to an agent. Of this number, the report separates the number of requests that the transactional and answer intents have handled successfully from those that were escalated to a human agent using the Responded and Escalated metrics, respectively.
Description of kpis_agent_insights.png follows
Description of the illustration kpis_agent_insights.png

Some escalations occur because users specifically request the skill connect them to an agent ("I want to speak to a human!") but others happen because the skill could not resolve the request to any of its answer intents within the Confidence Win Margin Threshold.

You can incorporate these unresolved requests into the training corpus using the Retrainer, a tool that enables you to update the training corpus for the draft versions of skills that are registered to the Service Cloud Agent digital assistant. You can access the Retainer from different places within the Overview report:
  • From the Action Items pane when a decrease in the response rate indicates that the intents need updating, or when the number of unresolved requests increases.

  • From the Popular Intents word cloud where color coding indicates the intents in need of retraining.

View Q&A Intent Usage and Performance

The word cloud rendered in the Popular Intents pane weighs the most frequently invoked intents by size and by color. The size of the intent name corresponds to the number of times it was invoked during a given period. Hovering over the intent name reveals the actual invocation number. The color represents the level of success in resolving requests to an intent. The color assignments are as follows:
  • Green represents an answer intent with a high average of resolving requests at, or exceeding, the Confidence Win Margin threshold.
  • Orange represents an answer intent that needs retraining because its Confidence Win Margin threshold is narrow (less than 10%).
  • Red is reserved for unresolvedIntent. This is the collection of user requests that couldn't be matched to any intent. Because these requests ultimately contributed to the work load of the human agents, you need to incorporate them into one of the intents, or create new answer intents for them using the Retrainer.


Retrain from the Popular Intents Word Cloud

Clicking unresolvedIntent, or any of the other intents (regardless of color), opens the Retrainer where the results are filtered to match the selected intent.

View Response Trends

Click the arrow (This is in an image of the chart toggle.) to toggle between the Escalated and Response charts. The Escalated chart plots the rate of automatic escalations that occur because of user requests for an agent against the rate of escalations that occur because of unresolved requests. The Responded chart plots the count of user utterances that were responded to successfully. (That is, the number of utterances that were resolved to intents.)
Description of repsonse_trends_agent.png follows
Description of the illustration repsonse_trends_agent.png

Apply the Retrainer to Intents

The Retrainer report enables you to improve your intents with actual user requests. The insights in this report help you decide which intents to update with these user requests, or if you need to create new intents.

Like the Popular Intents word cloud, the Retrainer report is a guide on how effective the intents have been in deflecting requests from live agents, but this report gives you a more granular view: it assesses the effectiveness of the intent in terms the win margin and confidence level that allowed it to resolve (or potentially resolve) an individual user request. Using this report, you can screen the individual requests as potential additions to the training corpus by reviewing the effectiveness of the intents. A request with a high confidence level and a wide win margin may not be a good candidate for the corpus because it's similar to the current training data, but one that resolved at a lower level because of common misspellings or colloquialisms might be.

You can filter the requests in different ways: by the specific Digital Assistant as Agent channel , by the intent name, by resolution-related properties (Top Confidence, Win Margin), or through combinations of these linked together with less than, equal to, or greater than comparison operators. The Retrainer accompanies each returned request with the answer intent that handled it, its win margin over the next highest ranking intent, and the Intents Score, a 100% stacked bar chart that ranks the skill's answer intents by their respective confidence levels.

Note:

The results get filtered by the name of the intent or by unresolved when you access the Retrainer through the Popular Intents word cloud.


Description of agent_insights_retrainer.png follows
Description of the illustration agent_insights_retrainer.png

Create a Draft Version of the Skill

Before you can update the training corpus with a request, or add a new answer intent, you must first create a draft version of the skill. The Retrainer gathers the requests that have been received from a published version of the skill which can't be updated.

Tip:

Switch off the Show Only Latest toggle to access both the draft and published versions of the skill.

Update Intents

To update an intent:
  1. In the published version of the skill, create a filter. If needed, apply additional filters by clicking + Criteria. Then click Search. If you access this report through the Popular Intents word cloud, then the results are filtered to match the selected intent or unresolved.
  2. Select the draft version of the skill.
  3. Select a request, then choose the appropriate intent from the menu.

    Tip:

    You can add requests to an intent on an individual basis, or you can select multiple intents and then select the target intent from the Add To menu that's located at the upper left of the table. If you want to add all of returned requests to an intent, select Utterances (located at the upper right of the table) and then choose an intent from the Add To menu.
  4. Retrain the skill.
  5. Republish the skill.
  6. Update the digital assistant with the new skill.
  7. Monitor the Overview report for changes in the Responded and Escalated metrics. Repeat the retraining process to achieve continuing increases in responses and downward trends in escalations.

Create an Answer Intent

To add an answer intent:
  1. In the published version of the skill, create a filter. For example, to return requests that can't be resolved to any of the intents, enter Intent match unresolvedIntent. If needed, apply additional filters by clicking + Criteria. Then click Search. If you access this report through the Popular Intents word cloud, then the results are filtered to match the selected answer intent or unresolved.
  2. Select the draft version of the skill.
  3. If you can't assign any of the returned requests to an existing intent:
    1. Select the request (or select a set of requests)
    2. Click Add (This is an image of the Add icon.), located at the upper left of the table.
  4. Complete the dialog by entering an intent name, the user-friendly description for the Conversation Name, and then the answer. Then click Create.

  5. Retrain the skill.
  6. Republish the skill.
  7. Update the digital assistant with the new skill.
  8. Monitor the Overview report for changes in the Responded and Escalated metrics. Repeat the retraining process to achieve continuing increases in responses and downward trends in escalations.