The skill analytics reports enable you to gauge how well your published skills complete conversations both on an individual basis and as a group.
View the Total Conversations Report
- Total Conversations—the overall number of conversations, segmented by completed, incomplete, or in progress.
- Completed Conversations—A ranking of the selected skills by the number of completed conversations. The total represents a maximum of five skills. By default, the total reflects the five skills that have completed the most conversations, which are listed in descending order. Sorting this list in ascending order changes the total, because the five skills that have completed the fewest conversations are listed.
- Incomplete Conversations—A ranking of the selected skills by the
number of incomplete conversations. Customers may not have completed these
conversations because they lost interest, or because they were blocked by errors
handled by the system (not the skill itself), timeouts, or infinite loops caused
by flaws in the skill definition. For each skill, the report provides an overall
count of timeouts, system-handled errors, and infinite loops for the period. To
illustrate how much these factors contributed to a skill’s overall failure, the
report renders a pie chart.
The Incomplete Conversations total reflects a maximum of five skills which can be listed in the pane. By default, the list is sorted in descending order, with the skill having the most incomplete conversations at the top. Sorting this list in ascending order changes the total, because the report ranks the five skills with the fewest incomplete conversations.
Note:Conversations are not the same as metered requests. To find out more about metering, refer to Oracle PaaS and IaaS Universal Credits Service Descriptions.
View Skill Performance
This report tracks the conversation completion rate for the skills for a selected period and on a per-skill basis.
The report includes other factors that influence the skill’s performance during this period: the number of errors (system-handled errors, infinite loops, timeouts) and unresolved intents. For these, the report counts the number of intents that couldn’t resolve to the confidence threshold for all of incomplete conversations during the period. These metrics are aggregated across all versions of the skill. To compare the metrics for a specific version, click Compare Versions.
Compare Metrics Across Different Versions of Skills
To compare how different versions have impacted the success of a skill, click Compare Conversations in the Skills report to open the dialog.
You can then filter the chart and KPIs by the following metrics:
- Total Count – The number of conversations (completed, incomplete, in progress)
- Completion Rate – The ratio of completed conversations to total conversations
- Incomplete Rate – The ratio of incomplete conversation to total conversations
- Unresolved – The number of unresolved conversations
Clicking the summary tile opens the analytics for that version of the skill.
Scenario: Tracking the Unresolved Count Across Versions
- Select versions. By default, the latest two versions of the skill are selected, but you want to see how the refinements made in each version have affected the count, you select all versions.
- Filter by Unresolved Count.
- Select the time frame for the version for the X-Axis time frame.
- Compare versions. The graph plots the decline for each version, but to make a quick comparisons between versions (for example, you want to plot Version 1.2 against Version 1.1), you select and clear the versions in the legend.