1. Using Oracle Digital Assistant
  2. Skills
  3. Backend Integration

19 Backend Integration

You can integrate skills with backend services, such as Oracle Human Capital Management Cloud and Oracle Financials Cloud, through custom components. For example, you might want to create a skill that lets your employees report their travel expenses to Oracle Financials Cloud.

To integrate a skill with a backend service:

  1. Using JavaScript and the Oracle Digital Assistant Node.js SDK, implement a custom component that transfers data to and from the skill using the SDK's metadata and conversation objects. See Task 1: Implement Custom Components.

  2. Deploy the component package to a Digital Assistant embedded container, an Oracle Mobile Hub backend, or a Node.js server. See Task 2: Deploy the Component Package to a Service.

  3. Configure the component service for the skill. See Task 3: Configure a Component Service.

Task 1: Implement Custom Components

To implement custom components, you use the Oracle Digital Assistant Node.js SDK to interface with Digital Assistant's custom component service.

Here's how to implement custom components that you can deploy to the Digital Assistant embedded container, a Mobile Hub backend, or a Node.js server:

  1. Install the software for building custom components.

  2. Create the custom component package.

  3. Create and build a custom component.

You can find up-to-date examples, as well as more detailed instructions at https://github.com/oracle/bots-node-sdk/tree/master/examples/custom-components/starter.

Step 1: Install the Software for Building Custom Components

To build a custom component package, you need Node.js, Node Package Manager, and the Oracle Digital Assistant Bots Node.js SDK.

Note:

If you plan to deploy to the embedded container, your package should be compatible with the following versions:
  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Infrastructure (sometimes referred to as the Generation 2 cloud infrastructure), your package should be compatible with Node.js 8.11.5 and Bots Node SDK 2.2.2.
  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Platform (as all version 19.4.1 instances are), your package should be compatible with 8.11.4 and Bots Node SDK 2.2.0.
  1. If you haven’t already, download Node.js from https://nodejs.org and install it for global access. Node Package Manager (npm) is distributed with Node.js.

    To test if Node.js and npm are installed, open a terminal window and type these commands: 

    node –v 
npm –v
  2. To install the Oracle Digital Assistant Bots Node.js SDK for global access, enter this command in a terminal window:
    npm install -g @oracle/bots-node-sdk

    On a Mac, you use the sudo command: 

    sudo npm install -g @oracle/bots-node-sdk

    When you use the -g (global) option, you have direct access to the bots-node-sdk command line interface. Otherwise, use npx @oracle/bots-node-sdk.

  3. To verify your Oracle Digital Assistant Bots Node.js SDK installation, type the following command:
    bots-node-sdk -v
    The command should print the Oracle Digital Assistant Bots Node.js SDK version.

Step 2: Create the Custom Component Package

Use the SDK’s command line interface (CLI) to create the necessary files and directory structure.

To create the package folder, and the necessary contents, type the following command in a terminal window:

bots-node-sdk init <top-level folder path>

This command completes the following actions:

  • Creates the top-level folder.

  • Creates a components folder and adds a sample component JavaScript file named hello.world.js. This is where you'll put your component JavaScript files.

  • Adds a package.json file, which specifies main.js as the main entry point and lists @oracle/bots-node-sdk as a devDependency. The package file also points to some bots-node-sdk scripts. 

    {
  "name": "myCustomComponentService",
  "version": "1.0.0",
  "description": "Oracle Bots Custom Component Package",
  "main": "main.js",
  "scripts": {
    "bots-node-sdk": "bots-node-sdk",
    "help": "npm run bots-node-sdk -- --help",
    "prepack": "npm run bots-node-sdk -- pack --dry-run",
    "start": "npm run bots-node-sdk -- service ."
  },
  "repository": {},
  "dependencies": {},
  "devDependencies": {
    "@oracle/bots-node-sdk": "^2.2.2",
    "express": "^4.16.3"
  }
}

  • Adds a main.js file, which exports the package settings and points to the components folder for the location of the components, to the top-level folder.

  • Adds an .npmignore file to the top-level folder. This file is used when you export the component package. It must exclude .tgz files from the package. For example: *.tgz.

  • For some versions of npm, creates a package-lock.json file.

  • Installs all package dependencies into the node_modules subfolder.

If you plan to deploy to the embedded container, your package should be compatible with the following versions:

  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Infrastructure (sometimes referred to as the Generation 2 cloud infrastructure), your package should be compatible with Node.js 8.11.5 and Bots Node SDK 2.2.2.
  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Platform (as all version 19.4.1 instances are), your package should be compatible with 8.11.4 and Bots Node SDK 2.2.0.

This step shows the basic CLI command. For more information, see https://github.com/oracle/bots-node-sdk/blob/master/bin/CLI.md.

Step 3: Create and Build a Custom Component

Here are the steps for building each custom component in your package:

  1. Create the component JavaScript file.

  2. Add code to the metadata and invoke functions.

  3. Access the backend.

  4. Use the SDK to access request and response payloads.

  5. Ensure that the component works in digital assistants.

Create the Component JavaScript File

Use the SDK's CLI to create a JavaScript file with the required functions for the component.

From a terminal window, CD to the package’s top-level folder and type the following command:

bots-node-sdk init component –-name <component name> components

The <component name>.js file, which contains the framework for working with the Oracle Digital Assistant Node.js SDK, is added to the components folder.

Note that the component name can't exceed 100 characters. You can only use alphanumeric characters and underscores in the name. You can't use hyphens. Nor can the name have a System. prefix. Oracle Digital Assistant won't allow you to add a custom component service that has invalid component names.

Tip:

You can put your component and helper files in subfolders to organize your files by component. Use bots-node-sdk init component –-name <component name> components/<subfolder name> to create the component file.

This step shows the basic CLI command. For more information, see https://github.com/oracle/bots-node-sdk/blob/master/bin/CLI.md.

Add Code to the metadata and invoke Functions

The required interface for a custom component implementation is: 

// interface for a custom component implementation
{
  metadata(): {name: string, properties?: {[name:string]: string}, supportedActions?: string[]};
  invoke(conversation: Conversation, done: () => {}): void;
}

  1. Update the metadata function.

    This function provides the component descriptions that are displayed in the Components tab in the UI, and describes how to use the component in the dialog flow. Modify the metadata to provide the component name (which must be unique), the names and types of the input parameters that it expects, and the actions that the component can return, if any.

  2. Add code to the invoke function to make REST calls, access request data, update response data, and return to the skill.

    The invoke function takes two arguments:

    • conversation, which is an instance of the Conversation class in the Digital Assistant Node.js SDK. This class is described in in the SDK documentation at https://oracle.github.io/bots-node-sdk/.
    • done, which is a callback that the component invokes when it has finished processing

Here’s an example:

'use strict';

module.exports = {
    metadata: () => ({
        name: 'hello.world',
        properties: {
            human: {
                required: true,
                type: 'string'
            },
        },
        supportedActions: []
    }),
    invoke: (conversation, done) => {
        // Perform conversation tasks.
        const { human } = conversation.properties();
        conversation
            .reply(`Greetings ${human}.`)
            .reply(`Today is ${new Date().toDateString()}.`)
            .transition();

        done();
    }
};
Access the Backend

There are many JavaScript modules that you can use to access a backend. Here are some examples:

The following steps use the Request module.

  1. If the service requires an Authorization header, add input parameters to the metadata to pass in the necessary information.
    • For Basic Auth, add parameters for user name and password, or for base-64 encoded <username>:<password>.
    • For OAuth2 with Oracle Identity Cloud Service or Oracle Access Management, add a parameter for the access key. The skill can use the System.OAuth2AccountLink component to get this key.
    • For all other authorization types, you have two choices:
      • Add a parameter for the authorization code, and add code to your component to get the access key. The skill can use the System.OAuthAccountLink component to get the authorization code.
      • Obtain the authorization code and access key from your implementation.
  2. Change to the folder that contains the package.json file and enter this command to install the Node.js Request module.
    npm install -save request
  3. Add the Node.js Request module to the code, as shown here:
    var request = require('request');
  4. Inside the invoke function, send a request to the service. For example:
    
    request('<url>', {json: true }, (err, res, body) => {

      var weatherResponse = body;

      if (res.statusCode == 200) {
        ...
        conversation.transition('<action>');
        done();
      } else if (res.statusCode == 401) { 
        ...
      } else if (res.statusCode == 404) {
        ...
        conversation.transition('<action>');
        done();
      } else {
        ...
        conversation.transition('<action>');
        done();
      }
    });
Use the SDK to Access Request and Response Payloads

Use Conversation instance methods to get the context for the invocation, change variables, and send results back to the dialog engine.

Here's how you use these methods for some common tasks:

How Do I? Do this
Get a component property value. Call conversation.properties() to get an object that contains all the component properties.
const MyCustomComponent = {module.exports = {
   metadata: () => ({
     name: 'greeting',
     properties: {
       "name": {        
         "type": "string",
         "required": true
     }    
  }   
}),
invoke: (conversation, done) => {
  // Get property values
  const { name: _name = '' } = conversation.properties();
  conversation.reply('Hello ${_name}');
  conversation.transition();
  done();
  }
}
Get the value of a context variable. Add a metadata property to pass in the name of the variable, and then call conversation.variable(<variable name>)to retrieve the value.
const { latitudeVariable } = conversation.properties();
let _latitude = conversation.variable(latitudeVariable);
Get the value of a profile variable. Call conversation.variable('profile.<name>') to retrieve the value.
Set a context variable's value. Add a metadata property to pass in the name of the variable, and then call conversation.variable(<variable name>, <value>) to set the value.
if (latitudeVariable) {conversation.variable(latitudeVariable, _coordinates.x);}
Use an object to update a variable that's an entity type, such as ADDRESS, and include an entityName property that's set to the entity type.
if (addressVariable){
  let _AddressObject = {};
  _AddressObject['entityName'] = 'ADDRESS';
  _AddressObject['postCode'] = _zip;
  _AddressObject['state'] = _state; 
  conversation.variable(conversation.properties().addressVariable, _addressObject);
}
Send a reply or a message response. Call conversation.reply(<payload>). The payload can be a string, an object, or a MessageModel. You can call this function multiple times. When you call this function, keepTurn is set to false automatically.
Return to the dialog flow. Call transition(<optional state>) followed by done() to return to the dialog flow and transition to another state. Use transition() to transition to the next state in the flow. Otherwise, use transition(<action>), where <action> is one of the supportedActions in the metadata.
module.exports = {
  metadata: () => ({
    name: 'verify',
    properties: {
      "address": {
        "type": "string",
        "required": false
      }
    },
    supportedActions: ['valid', 'invalid']
  }),
  invoke: (conversation, done) => {
    ...
    conversation.transition('valid');
    done();
    ...
}};
Pass a JSON object to the custom component. In the custom component's metadata, specify a property of type string or map (either will work). In the dialog flow, use a FreeMarker expression like this to pass the JSON object:
    properties:
                object: "${myJSONObjectVarName.value}"

The full SDK documentation is at https://github.com/oracle/bots-node-sdk.

Ensure the Component Works in Digital Assistants

In a digital assistant conversation, a user can break a conversation flow by changing the subject. For example, if a user starts a flow to make a purchase, they might interrupt that flow to ask how much credit they have on a gift card. We call this a non sequitur. To enable the digital assistant to identify and handle non sequiturs, call the conversation.invalidInput(payload) method when a user utterance response is not understood in the context of the component.

In a digital conversation, the runtime determines if an invalid input is a non sequitur by searching for response matches in all skills. If it finds matches, it reroutes the flow. If not, it displays the message, if provided, prompts the user for input, and then executes the component again. The new input is passed to the component in the text property.

In a standalone skill conversation, the runtime displays the message, if provided, prompts the user for input, and then executes the component again. The new input is passed to the component in the text property.

This example code calls conversation.invalidInput(payload) whenever the input doesn’t convert to a number. 

"use strict"
 
module.exports = {
 
    metadata: () => ({
        "name": "AgeChecker",
        "properties": {
            "minAge": { "type": "integer", "required": true }
        },
        "supportedActions": [
            "allow",
            "block",
            "unsupportedPayload"
        ]
    }),
 
    invoke: (conversation, done) => {
        // Parse a number out of the incoming message
        const text = conversation.text();
        var age = 0;
        if (text){
          const matches = text.match(/\d+/);
          if (matches) {
              age = matches[0];
          } else {
              conversation.invalidUserInput("Age input not understood. Please try again");
              done();
              return;
          }
        } else {
          conversation.transition('unsupportedPayload");
          done();
          return;
        }
 
        conversation.logger().info('AgeChecker: using age=' + age);
 
        // Set action based on age check
        let minAge = conversation.properties().minAge || 18;
        conversation.transition( age >= minAge ? 'allow' : 'block' );
 
        done();
    }
};

Here’s an example of how a digital assistant handles invalid input at runtime. For the first age response (twentyfive), there are no matches in any skills registered with the digital assistant so the conversation displays the specified conversation.invalidUserInput message. In the second age response (send money), the digital assistant finds a match so it asks if it should reroute to that flow.


Description of components-nonsequitur-conversation.png follows
Description of the illustration components-nonsequitur-conversation.png

You should call either conversation.invalidInput() or conversation.transition(). If you call both operations, ensure that the system.invalidUserInput variable is still set if any additional message is sent. Also note that user input components such as System.CommonResponse, System.Text, System.List, and System.ResolveEntities reset system.invalidUserInput.

Say, for example, that we modify the AgeChecker component as shown below, and call conversation.transition() after conversation.invalidInput(). 

if (matches) {  age = matches[0]; } else { 
      conversation.invalidUserInput("Age input not understood. Please try again"); 
      conversation.transition("invalid"); 
      conversation.keepTurn(true);
      done();
      return;
}

In this case, the data flow needs to transition back to askage so that the user gets two output messages – "Age input not understood. Please try again" followed by "How old are you?". 

  askage:
    component: "System.Output"
    properties:
      text: "How old are you?"
    transitions: {}
  checkage:
    component: "AgeChecker"
    properties:
      minAge: 18
    transitions:
      actions:
        allow: "crust"
        block: "underage"
        invalid: "askage"

Run the Component Service in a Development Environment

During the development phase, you can start a local service to expose the custom component package.

  1. From the top-level folder, open a terminal window and run these commands to start the service:
    npm install
npm start
  2. To verify that the service is running, enter the following URL in a browser:
    localhost:3000/components

    The browser displays the component metadata.

  3. If you have direct Internet access, you can access the development environment from a skill:
    1. Install a tunnel, such as ngrok or Localtunnel.
    2. If you are behind a proxy, go to http://www.whatismyproxy.com/ to get the external IP address of your proxy, and then, in the terminal window that you will use to start the tunnel, enter these commands:
      export https_proxy=http://<external ip>:80
export http_proxy=http://<external ip>:80
    3. Start the tunnel and configure it to expose port 3000.
    4. In Oracle Digital Assistant, go to the skill's Components Components icon tab and add an External component service with the metadata URL set to https://<tunnel-url>/components.
      You can use any value for the user name and password.
You can now add states for the service's components to the dialog flow and test them from the skill tester.

Task 2: Deploy the Component Package to a Service

You can host the custom component package from a Digital Assistant embedded container, Mobile Hub, or a Node.js server.

If you are going to host your component package from an embedded container, then all you need to do is prepare the component package that you'll need to create the component service. When you later create the component service, and choose the Embedded Container option, Digital Assistant hosts the component package automatically. If you don't use an embedded container, then you need to deploy the package to Mobile Hub or a Node.js server.

Prepare the Package for an Embedded Container Service

The easiest way to host your component package is to package it and upload it to the Digital Assistant embedded container service.

To prepare a package for uploading to the embedded container service:

  1. Ensure that you have the latest version of the Bots Node.js command line tools.

    Earlier versions did not bundle the dependencies with the package. The embedded container requires that the package includes all dependencies.

  2. In the directory that contains the main.js file, run the following command for each of the modules that your package depends on. You don't need to do this for devDependencies, such as the Bots Node SDK.

    This command adds the module to the node_modules folder and adds it as a dependency in package.json. 

    npm install <module>

  3. Run this command:

    bots-node-sdk pack

    The command validates the component package and then creates a TGZ file, which you'll upload when you create a service for the embedded container from the skill’s Components tab.

    The command bundles all the libraries that are listed as dependencies in package.json. The embedded container requires that the TGZ file includes all the libraries that the package depends on, with the exception of the Bots Node SDK and Express, which are devDependencies.

Your package should be compatible with the following versions:

  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Infrastructure (sometimes referred to as the Generation 2 cloud infrastructure), your package should be compatible with Node.js 8.11.5 and Bots Node SDK 2.2.2.
  • For Oracle Digital Assistant instances that are provisioned on Oracle Cloud Platform (as all version 19.4.1 instances are), your package should be compatible with 8.11.4 and Bots Node SDK 2.2.0.

For more information about the pack command, see https://github.com/oracle/bots-node-sdk/blob/master/bin/CLI.md.

Deploy to Mobile Hub

To host a custom component package in Mobile Hub, use the bots-node-sdk pack CLI to copy your component package folders and make a few changes that are specific to Mobile Hub, including the RAML file. Then create the custom API from the RAML file, and upload a ZIP of the component package into the custom API.

  1. From the custom component package's top-level folder (the one that contains the main.js file), type this command in a terminal window: 

    bots-node-sdk pack --service mobile-api

    The command does the following:

    • Copies the files and subfolders to service-mobile-api-<package version>.
    • Adds a component.service.raml file, which contains the necessary endpoints and operations.
    • Creates an api.js file, which is a Mobile Hub wrapper for main.js.
    • Modifies the package.json file to set the main file to api.js, set the dependencies, and add the Mobile Hub node configuration.

    This step shows the basic CLI command. For more information, see https://github.com/oracle/bots-node-sdk/blob/master/bin/CLI.md.

  2. Review the package.json file and verify that the package name conforms to the Mobile Hub constraints:
    • The name must consist only of letters (A-Za-z), numbers (0-9), and underscores (_).
    • The name must begin with a letter.
    • The name must be 100 characters or less.

  3. From the Mobile Hub APIs page, click New API > API, and then create the custom API by uploading the component.service.raml file.

  4. From the Security tab, switch off Login Required and then click Save.

  5. Zip up the service-mobile-api-<package version> folder, and then upload the ZIP file from the custom API’s Implementation tab.

  6. From the Test page, invoke the GET request. The response should show the component metadata.

    Tip:

    If you get a status 500, and the error is that it can’t find a matching route definition, check your files for bad JavaScript syntax, as that is the typical cause.

Deploy to a Node.js Server

To host a custom component package on an external Node.js server, use the bots-node-sdk pack CLI to copy your component package folders and make a few changes that are specific to Express, then install the component package and start it on your server.

  1. From the custom component package's top-level folder (the one that contains the main.js file), type this command in a terminal window: 

    bots-node-sdk pack --service express

    The command does the following:

    • Copies the files and subfolders to service-express-<package version>.
    • Adds an index.js service wrapper.
    • Creates an api.js file, which is an Express wrapper for main.js.
    • Modifies the package.json file to set the main file to index.js and add the dependencies.

    This step shows the basic CLI command. For more information, see https://github.com/oracle/bots-node-sdk/blob/master/bin/CLI.md.

  2. Run these commands:

    npm install

npm start

Task 3: Configure a Component Service

You need to create a component service for each custom component package that your skill uses. A service is an active connection from the skill to the components.

A component service has two functions:

  • It queries the component to get the package metadata, including the names of the components, their properties, and allowed actions for each one. After a service is added to the skill, you can see this information in the Components tab, which you access by clicking Components (This is an image of the Components icon.) in the left navbar. You can reference this page to get the component names, properties, and actions, which you will need to use the components in your dialog flow.

  • It allows the skill to invoke the components.

    The JSON payload of the call made by the Dialog Engine to the components includes input parameters, variable values, user-level context, and the user’s message text. The component returns the results by changing the values of existing variables or adding new ones (or both). The Dialog Engine parses the returned payload and proceeds.

To add a custom component package to a skill, go to the skill's Components tab (This is an image of the Components icon.) and click Add Service, which opens a dialog for configuring the service.

How you configure the service depends on where you are hosting the component package – Mobile Hub, your own Node.js server, or the Digital Assistant embedded container, which is the quickest deployment option.

When you upload a package to the embedded container, Digital Assistant verifies that the package is valid, and can reject the package for these reasons:

  • There are JavaScript errors.

  • The package doesn't contain all the node module dependencies.

  • A component name has more than 100 characters, begins with System., or contains other than alphanumeric characters and underscores, then the service creation fails.

  • Your instance already has the maximum number of component services.

See Configure a Service for the Embedded Container for more information about these verification checks.

Configure a Service for the Embedded Container

When you choose the Embedded Container option, which deploys the component package to Oracle Functions Service, you only need to upload a TGZ of your custom component implementation. This TGZ file, which you package using npm pack, must contain the assets and structure described in Task 1: Implement Custom Components.

Note:

If your instance is provisioned on the Oracle Cloud Platform (as all version 19.4.1 instances are), then the service is deployed within the Digital Assistant instance instead of Oracle Functions Service.

During the upload, Digital Assistant verifies that the package is valid, and can reject the package for these reasons:

  • If any component JavaScript contains syntax errors, then that component is not added to the container, which results in this error message:

    Error Message: failed to start service built: Invalid component
      path:

  • If the package doesn't contain all the node module dependencies, then you will get the same error message as above:

    Error Message: failed to start service built: Invalid component
      path:

    To learn how to include node module dependencies, see Prepare the Package for an Embedded Container Service.

  • If a component name has more than 100 characters, begins with System., or contains other than alphanumeric characters and underscores, then the service creation fails.

  • If your instance already has the maximum number of component services, then the service creation fails. To see the component service limit, go to the service limits page in the Infrastructure Console and look for embedded-custom-component-service-count. If you need to raise the limit, you can request an increase. For more information, see Viewing Your Service Limits, Quotas, and Usage and Requesting a Service Limit Increase.

After you upload the TGZ file, the custom component service is built and its components are deployed. If the Components page displays an awaiting deployment message after you upload the TGZ file, it means that the service has been created, but is not yet available. When the service becomes available, the deployment metadata displays in place of the awaiting deployment message.

Tip:

Whenever you upload a new version of a component package, don't forget to click Reload afterwards.

If a custom component incurs a runtime error, then you can click the Diagnostics button for the service to access view logs and crash reports to diagnose the problem.

  • Select View Logs to view messages that the custom component sends to conversation.logger().
  • Select View Crash Report to view details about what may have caused the container to crash.

Configure a Service for Mobile Hub

If your custom component package is hosted on Oracle Mobile Hub, then choose Oracle Mobile Cloud as the container for your custom components. Custom components that are hosted on Mobile Hub can integrate with remote services using various connectors controlled by a Mobile Hub backend and they have access to the Mobile Hub platform APIs.

Because the backend that hosts the custom code handles the authentication for the custom components, you need to refer to the backend’s Settings page to get the information that you need to complete the configuration.

To configure the service for the Mobile Hub backend:

  1. Enter the unique identifier assigned to the Mobile Hub backend in the Backend ID field. This ID is passed in the REST header of every call from the skill.

  2. In the MetadataURL field, enter the /components endpoint from the custom code API. For example, http://<server>:<port>/mobile/custom/ccPackage/components.

  3. Choose Use Anonymous Access if the service allows anonymous login. If you choose this option, enter the anonymous key, which is a unique string that allows your app to access anonymous APIs without sending an encoded username and password. The anonymous key is passed in their place. You can find the anonymous key on the backend's Settings page in Mobile Hub. (You may need to click Show.)

    If the component service requires a login (meaning no anonymous access), then enter the user name and password.

  4. If the service requires specific parameters, click Add HTTP Header and then define the key-value pairs for the headers.

  5. Click Create.

Configure a Service for Your Own Node.js Server

Choose the External option when your custom component package is hosted on your own Node.js server. For this option, enter the The URL that points to the GET endpoint that returns the list of components in the Metadata URL field along with the service's user name and password. Then click Create.

Tip:

You can use the external service option during development, as described in Run the Component Service in a Development Environment.