11 Localization

Even though NLP support is in English, you can still add multi-language support for your bot. Using resource bundles and autotranslation services, your bot can automatically translate the users messages that it receives and its own prompts and replies to and from English.

Resource Bundles

Resource bundles allow you to localize your bot based on the language set for messaging channel currently in use. They not only allow your bot to output messages in the user’s language, but in the user’s dialect as well. When you don’t want to rely on the text provided by the translation service, and instead want to control the wording for your bot’s responses in one or several languages, you can opt for resource bundles.

Create a Resource Bundle

You define a single bundle for each bot that’s made up of various keys that identify the output text that needs to be translated.

To create a resource bundle:
  1. Click Resource Bundle in the left navbar (This is an image of the left navbar Resource Bundle icon.).

  2. Click Add Bundle.

  3. Enter the key and the and its text . For example, to localize the user prompt, How old are you?, you’d enter HowOld in the Key field and then How old are you? in the Text field.

  4. Click Create Entry.

  5. By default, the language for your first key is English. To add a foreign language version of the string, click. Add Language.
  6. Complete the Create Entry dialog:
    • Language—Add an IETF BCP 47 language tag like fr for French, de for German, or en-US for U.S. English.

    • Text—The output string. For example, for a French translation (fr) of the HowOld key, you’d add a string like quel âge avez-vous ?

      If the bot can’t match the language set for the browser with a language tag defined in the bundle, it defaults to a less-specific tag (if one is available). For example, it uses fr (a subtag) if the bundle has no entry for fr-CA. If none of the entries match the browser’s language, the bot uses the default entry, English. For more information on this fallback to the most generic entry, see Resource Bundle Entry Resolution

  7. If you want to translate other strings, click Add Key to create another entry in the resource bundle.

  8. Reference the resource bundle in the in the dialog flow. You can define the entity prompts as a resource bundle.

Reference Resource Bundles in the Dialog Flow

To set the output for a built-in component, you need to add a resource bundle context variable and then reference both it and the message key. In the following OBotML snippet for a pizza bot, the resource bundle is declared as the variable, rb, in the context section. Further down, value expressions define the text property for the System.Output components reference the rb variable and the keys, WhatType and OnTheWay. The first outputs a simple string and the other uses dynamic values.
context:
  variables:
    rb: "resourcebundle"
...

pizzaType:
  component: "System.Output"
  properties:
    text: "${rb('WhatType'}" # rb refers to the variable, WhatType is the key to the message in the resource bundle.
  transitions: {}
...

done:
  component: "System.Output"
  properties:
    text: "${rb('OnTheWay',size.value,type.value)}" # size.value and type.value are the arguments for the 'OnTheWay' message code.
  transitions:
    return: "done"
For simple messages, you can also reference resource bundles using dot notation (${rb.WhatType}).

Tip:

To test your resource bundles using the Tester, set your browser to another language.
Complete Translation with Resource Bundles
If you’re returning the user’s language from the browser, then simply setting the resource bundle variable and then referencing both it and the message key in an output component is all you need to do. Keep in mind that using this approach requires users to first enter something in English (like “Hello, Pizzabot!”). To start the session in the user’s language, you need to enable the translation service for the bot and configure the dialog flow accordingly. The following snippet shows this hybrid approach, which enables your bot to detect the user’s language. After the input is translated and the intent is resolved to English, the resource bundles handle the rest. In the following snippet, the rb context variable is set, but in this PizzaBot, it’s accompanied by another variable called translated. Because both the System.DetectLanguage and System.TranslateInput components are positioned before the System.Intent component, they enable the initial user input to be translated into English before it can be used by the System.Intent component and resolved to one of the intents.
metadata:
  platformVersion: "1.0"
main: true
name: "PizzaBot"
parameters:
  age: 18
context:
  variables:
    size: "PizzaSize"
    type: "PizzaType"
    crust: "PizzaCrust"
    iResult: "nlpresult"
    rb: "resourcebundle"
    translated: "string" # holds the user's text that's translated into English.
states:
   # add DetectLanguage and TranslateInput components
  detect:
    component: "System.DetectLanguage"
    properties: {}
    transitions: {}
  
  # translate the user text and store it in the translated variable
  translate:
    component: "System.TranslateInput"
    properties:
      variable: "translated"
    transitions: {}
    
  intent:
    component: "System.Intent"
    properties:
      variable: "iResult"
      sourceVariable: "translated" # this variable now would holds the translated text
    transitions:
      actions:
        OrderPizza: "resolvesize"
        CancelPizza: "cancelorder"
        unresolvedIntent: "unresolved"
  ...

Note:

You can control the output of a response message when you reference a resource bundle from one of the system output components and set the translation override (translate: true). By doing this, you enable the resource bundle to handle the text in the detected language. You can also apply this to content that’s dynamically added, like data returned by a custom component.

Resource Bundle Entry Resolution

The resource bundle that gets applied depends on the value stored for the two location-specific user context variables, profile.locale and profile.languageTag. For example, if ${profile.locale} stores en-AU-sydney then Digital Assistant returns the bundle entry by first searching for an exact match. If it can’t return a match, it broadens its search. In this case, Digital Assistant does the following to localize the output as Australian English:
  1. Searches the bundle using a language-country-variant criteria. In this case, it searches for en-AU-sydney.

  2. If it can’t find that, it searches the bundle by language and country (en-AU).

  3. Failing that, it broadens its search for language (en).

  4. If it can’t locate any entries, then it returns the default language, which is English (en).

Autotranslation

Autotranslation uses services like Microsoft Translator and the Google Translation API to enable the built-in components like System.Text and System.Output to output their prompts in the user’s language.

When a user enters a non-English request or response, the translation service allows the bot to convert this input to English. Once it’s translated, the NLP engine can resolve it to an intent and match the entities. Using both a translation service and an OBotML definition that includes the System.DetectLanguage and System.TranslateInput components, you can enable your bot to automatically detect the user’s language and translate your bot’s messages.

Enable Auto-translation

  1. First, configure a translation service for your Digital Assistant instance. To do this, enter the URL and Authorization token for the Microsoft Translator service or the Google Translation API in the Translation Services dialog.

    Description of translation_services.png follows
    Description of the illustration translation_services.png
    Refer to the documentation for Microsoft Translator and Google Translation API to find out how to get the URL and access token.

    Important:

    To use the Google Translation API, you need to generate the API Key. You create this key from the GCP Console (APIs & services > Credentials). To find out more, see the Google Cloud Platform Documentation.
    You can open this dialog from the menu on the digital assistant landing page.
    This is an image of the menu icon on the landing page.
  2. Next, click Settings in the left navbar and then choose a translation service for your bot.
    This is an image of the Translation Service menu on the Settings page.

  3. Finally, configure the dialog flow:
    1. Add autoTranslate: "boolean" to the context node. This variable is common to all bots (or at least the ones that use translation services). As such, you can’t change its name or define it as string, another type of primitive, or an entity. You can override the autotranslated output generated for the System.Output, System.Text, and System.List components when you set their translate property to false.

      Note:

      Do not set the autoTranslate variable to true if you’re translating text using a resource bundle.
    2. Generally, you’d first add a state with a System.Intent component at the beginning of the states node. But since the NLP engine can only recognize English, you need to begin the states node with a set of language-specific components and variables to translate the user input so that it can be resolved by System.Intent component. The first of these states uses the System.SetVariable component. As shown in the following snippet, its variable property is defined by the autoTranslate context variable. To enable translation, it’s set to true.
        setAutoTranslate:
          component: "System.SetVariable"
          properties:
            variable: "autoTranslate"
            value: true
          transitions: {}
    3. Next, add the System.DetectLanguage component:
        detect:
          component: "System.DetectLanguage"
          properties: {}
          transitions: {}
    4. Add the System.TranslateInput component:
        translate:
          component: "System.TranslateInput"
          properties:
            variable: "translated"
          transitions: {}
    5. Finally, add the System.Intent component. Set sourceVariable to hold the translated input.
        intent:
          component: "System.Intent"
          properties:
            variable: "iResult"
            sourceVariable: "translated" # this variable holds the English translation of the user input.

      Important:

      While the Add Components menu adds template state nodes for these translation components, it doesn’t insert the autoTranslate: “boolean” variable into the context node. You’ll need to add it yourself.
    The following segment shows the PizzaBot equipped for autotranslation. Note that along with the autoTranslate variable, this definition also includes a variable that stores the translated output called translated (translated: “string”) in the context node. The variable property for the System.TranslateInput component names this component, as does the sourceVariable property for the System.Intent component. For example, the System.TranslateInput component would store its English translation (“I want pizza”) in this variable when a user enters “je voudrais commander une pizza.” Because sourceVariable names translated, it holds “I want pizza,” which the System.Intent component can resolve to one of the intents.
    metadata:
      platformVersion: "1.0"
    main: true
    name: "AutoTranslatePizzaBot"
    parameters:
      age: 18
    context:
      variables:
        size: "PizzaSize"
        type: "PizzaType"
        crust: "PizzaCrust"
        iResult: "nlpresult"
        autoTranslate: "boolean" 
        translated: "string"
    states:
       setAutoTranslate:
        component: "System.SetVariable"
        properties:
          variable: "autoTranslate"
          value: true
        transitions: {}
      detect:
        component: "System.DetectLanguage"
        properties: {}
        transitions: {}
      translate:
        component: "System.TranslateInput"
        properties:
          variable: "translated"
        transitions: {}
      intent:
        component: "System.Intent"
        properties:
          variable: "iResult"
          sourceVariable: "translated" 
    

Single-Language Bots with Auto-Translation

You're not limited to defining your corpus or custom entities in English when you use a translation service. Your bot can be trained with utterances written in the language of your choice. Because the language processing framework detects non-English utterances and then translates them into English, your dialog flow is shorter and simpler: you don’t need to configure it for translation using the autoTranslate variable, the System.DetectLanguage, System.TranslateInput components, or the translate property.

Although this approach streamlines your development, it does have a restriction: at runtime, the bot is only fully conversant in a single language, which is the language of the custom entity values. Although auto-translation allows it to reply in the language that corresponds to the user’s locale, it can’t process “foreign” user input that contains any values that need to be matched by a custom entity. These entity values have been defined in a different language, so the framework can’t use them to match the user input. There are also a couple of things to keep in mind:
  • You can’t translate the names of the built-in entities. They’re either in English or Simplified Chinese.

  • Translating the corpus can impact auto-translation costs because it requires more calls to the translation service.

How Do I Translate Custom Component Responses?

There are three methods for returning translated content from your custom components:
  1. You can store data objects in a context variable, which then is referenced by a system output component, a system component label or prompt.

  2. You can save string messages in a context variable for display by system components like System.Output, System.List, System.CommonResponse, or System.TranslateOutput.

  3. You can avoid the system components altogether and send responses directly to the user.

Option 1: Storing Data in a Context Variable

Custom components that query backend services might return data in a simple format like text or number, or in a complex format like an object or an array of objects. If this data requires translation:
  1. Save the data in a context variable in your dialog flow.

    To write data to a context variable, add the following line of code to your custom component:
    conversation.variable('data', dataObject);
    conversation.transition();
    done();
    
    For this particular line of code to work, you need to add a corresponding context variable to your dialog flow definition named data that’s a string type :
    data: "string"

    Tip:

    As a best practice, pass the name of the context variable that the custom component should write to as a input parameter to the custom component. This way, you ensure that the context parameter really exists when accessing it.
  2. Include the translate: true property in an output component definition, or use the System.TranslateOutput component to translate it to the user’s language.

    The system components can’t translate a data object as a whole, so if you need to translate data objects stored in a context variable, then you need to reference the context variable along with the name of the data object attribute that’s displayed to user and gets translated by an output component. In the following example, the context variable is data. The data object that’s passed from the custom component to this context variable is {product: "an apple", type: "fruit", origin: "Spain" }. To display and translate this object, you’d reference it a System.Output component and enable its translation:
    printProduct:
      component: "System.Output"
      Properties:
        text: "The product in your cart is a ${data.value.type}. It is
               ${data.value.product} from ${data.value.origin}"
        translate: true
    
    If a translation service is enabled for the bot, then this message gets translated to the user’s language. The custom component also passes the translated content of the data object.

Option 2: Saving Messages in a Context Variable

In this approach, you can pass the name of the context variable that holds the component response as an input parameter to the custom component. This ensures that custom components are not dependent on a specific OBotML implementation code and remain reusable.

The following OBotML sample references a custom component in the initializeReceipe state and passes the name of the context variable (receipe) that holds the component response and purchaseId as input parameters.

To display messages in the detected user language, you can either use the System.TranslateOutput component or set the translate property on an output component like System.CommonResponse or System.Output, as shown in the printReceipe state:
initializeReceipe:
  component: "sample.receipe.dataresponse"
  properties:
    dataVariable: "receipe"
    purchaseid: "${purchaseId.value}

printReceipe:
  component: "System.Output"
  properties:
    text:"${receipe.value}"
    translate: true
 ...
If a custom component returns a message that’s not in English, but still requires processing by the Natural Language Processing (NLP) engine, then you’d position the System.TranslateInput component before the System.Intentin the dialog flow. This allows the System.Intent component to process the English version of the message.
translateMessage:
  component: "System.TranslateInput"
  properties:
    source: "variable_populated_by_custom_component"
    variable: "translationStringVar"
getIntent:
  component: "System.Intent"
  properties:
    variable: "iResult"
    sourceVariable: "translationStringVar"
...

Option 3: Sending Responses Directly to the User

In some cases, custom components can be self-contained: they don’t send a message directly to a message channel or save data to a context variable. These custom components return responses directly to the user because they use the conversation.reply helper method.

Note:

Responses that are directly sent to a user can’t be translated by an autotranslation service. The component needs to translate the responses before sending them.
There are two ways to translate the bot responses that are sent directly from a custom component:
  1. Use message bundles to provide translated responses for the languages you that want the bot to support. Message bundles can use placeholders in the translated strings in which the custom component inserts the data value at runtime. Message bundles in custom components are a viable option if the custom component queries data from a remote service.

  2. To detect the preferred language in the custom component, you can either pass the locale as an input parameter to the component or access the profile.locale and profile.languageTag variables as shown in the following example:
    //detect user locale. If not set, define a default
    var locale = conversation.variable('profile.locale')?
                 conversation.variable('profile.locale') : 'en';
    //when profile languageTag is set, use it. If not, use profile.locale
    var languageTag = conversation.variable('profile.langageTag')?
                      conversation.variable('profile.langageTag') : locale;