Tune Intent Resolution Before Publishing

Before you publish a version of a skill (and thus freeze that version), you should thoroughly test it and, if necessary, adjust its settings to fine tune its intent resolution.

You can use these settings to tune intent resolution:

• Confidence Threshold: Determines the minimum confidence level required for user input to match an intent. It's recommended to set this value to .70 or higher.

• Confidence Win Margin: When a skill has multiple intents that exceed the value of the Confidence Threshold, it displays a list of possible intents and prompts the user to choose one. This property helps the skill determine what intents should be in the list. Set the maximum level to use for the delta between the respective confidence levels for the top intents. The list includes the intents that are greater than or equal to this delta and exceed the value set for the Confidence Threshold.

To access these settings:

• Click to open the side menu, select Development > Skills, and open your bot.

• In the left navigation for the skill, click and select the Configuration tab.

Note:

Once you add a skill to a digital assistant, there is another range of settings that you may need to adjust to better handle intent resolution in the context of the digital assistant. See Tune Routing Behavior.

How Confidence Threshold Works

You use the Confidence Threshold property to adjust the likelihood that given user input will resolve to the skill's intents.

When you increase the confidence threshold, you increase the certainty that any matching intents are accurate (not false positives). However, this also increases the chance that intents that you want to match with certain input will not get high enough confidence scores for the matching to occur, thus resulting in matches to unresolvedIntent.

When you lower the value of the Confidence Threshold property, you reduce the chance that intents that you want to match will fail to match. However, the lower you set this threshold, the greater risk you have of generating false positives in your matches.

As a general rule the underlying language model works better with higher confidence thresholds, so you should set the confidence threshold to 70% (.70) or higher to get the best results.

How Confidence Win Margin Works

With the Confidence Win Margin property (accessed through Settings > Configuration), you can enable your skill to prompt users for an intent when the confidence scores for multiple intents are close. For example, if a user asks the FinancialBot, “I want to check balance or send money,” the skill responds with a select list naming the top intents, Check Balances and Send Money.

Description of the illustration win_margin_list.png

The skill offers these two intents in a select list, because its confidence in them exceeds the value set for the Confidence Threshold property and the difference between their respective confidence levels (that is, the win margin) is within value set for the Win Margin property.