38 Oracle iOS

Using the Oracle iOS SDK for Oracle Digital Assistant, you can integrate your digital assistant with iOS apps. The SDK connects to the Oracle Chat Server, the intermediary between the Oracle iOS channel configured in Oracle Digital Assistant and the client. The chat server then passes messages to the skill for processing and delivers the skill's response to the client.

This SDK works with instances of Oracle Digital Assistant that were provisioned on Oracle Cloud Infrastructure (sometimes referred to as the Generation 2 cloud infrastructure). If your instance is provisioned on the Oracle Cloud Platform (as all version 19.4.1 instances are), you can use the legacy iOS SDK.

What Do You Need?

  • An Oracle iOS Channel. Creating the channel generates the Channel ID and the Secret Key that you need to initialize the chat app.
  • The URL of the Oracle Chat Server.
  • The Oracle iOS SDK (located under Oracle Native Client SDKs for OCI Native Environments) from Oracle Technology Network’s ODA and OMC download page. Download this ZIP and extract it to your local system. This ZIP includes a user guide that describes the SDK's functions.
  • Swift 4.2 as the Swift Language Version. Xcode 11 or higher. If you want your app to work with earlier versions, keep in mind that we haven't tested these and therefore can't guarantee their compatibility.

Create the Oracle iOS Channel

You can configure the channel to connect to the Oracle Chat Server in two modes: unauthenticated mode and authenticated mode (to protect access to the channel).
  • Unauthenticated mode – Use the unauthenticated mode when the client can't generate signed JWT tokens, when no authentication mechanism is in place, or when the client widget is already secured and visible to authenticated users.
  • Authenticated mode – Authentication is enforced using JSON Web Tokens (JWT). The customer's backend server generates the JWT token, which is then passed to the Oracle iOS SDK. This token is used for each request to an ODA speech, text, or attachment server.

    Note:

    To protect access to the channel, the token must always be generated by a remote server. It must never be generated within by the client app.
    When the app needs to connect to an ODA server, it first requests the token from the backend server and then adds it to the Authorization header. The ODA server validates the token, evaluates the claims, and then either opens the socket or rejects the connection.
    The JWT Token has the following claims: channelId and userId, and the claim names iat (issued at time), and exp (expiration time). iat signifies the time at which the token was issued. This must a number that represents the seconds that have elapsed since the UNIX epoch. exp must be a number representing the seconds that have elapsed since the UNIX epoch. We recommend setting the expiration time to at least 30 minutes after the issued at time (iat). The token header looks something like this:
    {
    
     "alg": "HS256",
    
     "typ": "JWT"
    
    }
    An example token body looks something like this:
    {
    
      "iat": 1569828182,
    
      "exp": 1569831782,
    
      "channelId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    
      "userId": "John"
    
    }

    Note:

    The token illustrated by this example is not signed. The actual tokens are signed by channel's Secret Key.

Configure the Oracle iOS Channel

To configure the Oracle iOS channel:
  1. Choose Development, then Channels from the menu.
  2. Choose Users.
  3. Click Add Channel and then Oracle iOS as the channel type.
  4. Complete the dialog:
    • Enter the channel name.
    • For authenticated connections:
      • Switch on the Client Authentication Enabled toggle to determine whether the SDK is connecting to a client authentication-enabled channel.
      • In the Max. Token Expiration (Minutes) field, set the maximum amount of time for the JWT token.
      • Set the Session expiration time.
    • For unauthenticated connections:
      • Switch off Client Authentication Enable toggle.
      • Set the Session expiration time.
    • Click Create. Oracle Digital Assistant will generate the Channel ID and the Secret Key that you need to initialize the SDK. Keep these close at hand because you'll need them when configuring the HTML page to host the chat widget.
  5. Route the channel to your skill or digital assistant.
  6. Switch Channel Enabled to On.

Add the SDK to the Project

  1. Download the ODA Client SDK for iOS and extract it to your local system.
  2. Add the files to your Xcode project:
    1. Click File > Add Files to "<project name>".
    2. Choose the .framework files that you want to add depending on where you want to run the app (simulator or actual device).
    3. Make sure to that Copy items if needed (located under Destinations) is selected.
    4. Alternatively, you can drag and drop the .framework files into the project file in Xcode.
  3. Embed and sign the frameworks in the Frameworks, Libraries, and Embedded Content category in the General tab. (This may vary according to the version of Xcode that you're using.) Make sure the Targets option is selected.
  4. Add the following keys in the project's .plist file:
    • Privacy - Location Always and When In Use Usage Description or <key>NSLocationAlwaysUsageDescription</key> and the corresponding <string></string> in the source code.
    • Privacy - Location When In Use Usage Description or <key>NSLocationWhenInUseUsageDescription</key> and the corresponding <string></string> in the source code.
    • Privacy - Microphone Usage Description or <key>NSMicrophoneUsageDescription</key> and the corresponding <string></string> in the source code.
    • Privacy - Camera Usage Description or <key>NSCameraUsageDescription</key> and the corresponding <string></string> in the source code.
    • To open a location in Google maps instead of Apple maps when a user taps a map view in a location message, declare the URL schemes used by the Maps SDK for iOS in the app's Info.plist file as described in the Google Maps SDK for iOS documentation.

Initialize the SDK in Your App

You can use the following code to initialize the chat view.
// Import the SDK
import BotClientUISDK
 
// Declare a global BotsViewController variable in your app view controller class
public var chatViewController: BotsViewController?

// Obtain a shared instance of BotsViewController from BotsUIManager
chatViewController = BotsUIManager.shared().viewControllerInstance()

// Specify the color changes, if any, in a particular component. Make sure you set all the required colors in BotsProperties before adding the chat view to the view controller.
  
// Add the chat view to your view controller
self.view.addSubview((chatViewController?.view)!)
  
// Initialize a BotsConfiguration object and set feature flags, if required.
var botsConfiguration = BotsConfiguration(url: baseUrl, userId: botID!, channelId: channelID)
  
// Initialize the configuration in botsViewController. Make sure you set all the feature flag values before passing the botsConfiguration to initConfiguration.
chatViewController?.initConfiguration(botsConfiguration: botsConfiguration)
  
// Get a custom navigation bar view from botsViewController and add it to the navigation bar of your app.
let customView:UIView = (chatViewController?.addCustomViewToNavBar())!
self.navigationController?.navigationBar.addSubview(customView)
 
// Obtain a shared instance of BotsManager
let botsManager = BotsManager.shared()
 
// If you require access to callback methods provided in BotsMessageServiceDelegate. Make sure your class conforms to BotsMessageServiceDelegate
botsManager.delegate = self

// If you require access to callback methods provided in BotsEventListener. Make sure your class conforms to BotsEventListener
botsManager.botsEventListener = self
 
// Initialize and establish connection to the chat server
botsManager.initialize(botsConfiguration: botsConfiguration, completionHandler: { (connectionStatus, error) in
 
})

Note:

The SDK currently provides UI for portrait orientation only.

App Development

Initialize the Feature Flag Settings

Initialize BotsConfiguration object using one of its constructors.
  • clientAuthDisabled
    • BotsConfiguration(url: String, userId: String, channelId: String)
      • Parameters:
        • url - The Oracle Chat Server URL. This cannot be null.
        • userId - The unique identifier for the user. This cannot be null.
        • channelId - The Oracle iOS Channel ID. This cannot be null.
      • When the userId not provided (a randomly generated value is instead)
        • url - The Oracle Chat Server URL. This cannot be null.
        • channelId - The Oracle iOS Channel ID. This cannot be null.
  • clientAuthEnabled
    • BotsConfiguration(url: String, authToken: String)
      • Parameters:
        • url - The Oracle Chat Server URL. This cannot be null.
        • authToken - The authentication token for establishing a connection with an authentication-enabled channel. This cannot be null.
For example:
// Initialize a BotsConfiguration object
var botsConfiguration = BotsConfiguration(url: chatServerUrl, authToken: token)
 
// Set the feature flag values if the desired values are different from the default values
botsConfiguration.showConnectionStatus = true
botsConfiguration.enableBotAudioResponse = true
botsConfiguration.disablePastActions = "none"

Network Configuration

Property Name Description Required? Default Value
url The URL of the Oracle Chat Server Yes N/A
channelId The ID of the Oracle iOS channel. Yes N/A
userId The unique identifier for user. This value gets initialized by the SDK if not provided. No A randomly generated value
authToken The authentication token for establishing a connection with an authentication-enabled channel. Yes N/A

Feature Flags

Property Description Required? Default Value
agentAvatar Passes the asset name of the agent avatar image. No N/A
botAvatar Passes the asset name of the skill avatar image. No N/A
disablePastActions A field for disabling the button clicks on the messages that a user has already interacted with. The allowed values are all, none, and postback. The behavior enabled by this property is independent of the digital assistant-level configuration for disabling the selection of past actions. You need to set the two separately. No all
enableAttachment Configures attachment sharing in the chat app. No true
enableAutoSendSpeechResponse When set to true (the default), the user's speech response is automatically sent to the chat server (and rendered as a sent message in the chat window). When set to false, the user's speech response is rendered in the message text field before it's sent to the chat server, allowing the user to modify it before sending it manually, or delete the message. No true
enableClearMessage Enables the clear message button in the chat widget header. No false
enableSpeechRecognition Enables the microphone button. No false
enableSpeechSynthesis Enables the skill responses to be read aloud. By setting this flag to true, you enable the skill's responses to be read aloud using Swift API. No false
enableTimestamp Enables the timestamp for messages. No true
initSpeechSynthesisMuted Sets the default state of BotAudioResponse as muted. No true
initUserHiddenMessage A user text message that's used to initiate a conversation. This message, which is sent when chat widget is ready, does not actually display in chat. No N/A
initUserProfile Updates the user profile before the start of a conversation. The format of the profile payload must be ["profile": […] ]. For example:
initUserProfile = ["profile": ["givenName": "First", "surname": "Last", "email": "first.last@example.com", "properties": ["lastOrderedItems": "1 medium pepperoni"]]]
This function updates the user context before the initial "hidden" message is sent by initUserHiddenMessage to start the conversation. As a result, the user profile can be reflected in the first response message to the user. For example, the skill can greet the user with a message like "Welcome back, John Smith! Your last order was a medium pepperoni pizza."
No N/A
personAvatar Passes the asset name of the person avatar image. No N/A
sharePopupConfiguration Allows the user to choose the options available as part of the attachment menu. The default value is set to a list of all options. No [.photoAndVideoLibrary, .files, .camera]
showConnectionStatus Enables the connection status to display in the chat widget header. No false
showTypingStatus Enables the display of the Responding… text indicator when waiting for a response from the skill. No true
speechLocale The expected locale of the user's speech that's used for voice recognition. The supported locales are: French (fr-fr), Spanish (es-es), and the default, English (en-US). This string connects to the speech server endpoint for the particular locale when users tap the microphone button. The speech locale can be set dynamically with the setSpeechLocale(string: locale) API. Voice recognition only works when one of the supported locales is passed. No en-us
speechSynthesisVoicePreferences Matches the provided preferences based on both the language-locale and voice name. If no match is found, then the default voice for the given language-locale is used. In the latter case, the Apple API finds best match with the given language-locale. No N/A
timestampFormat Formats the delivery timestamp that accompanies messages. The timestamp format should be supported by the Swift DateFormatter. No MMMM d, yyyy HH:mm
title Sets the title of the app, which is displayed in the app bar. No N/A
typingStatusTimeout Sets the timeout, in seconds, to hide the typing status indicator when no response has been received from the chat server. No 20 seconds

Strings

Configure strings by adding the following key = value pairs in the app's <language-code>.iproj/Localizable.strings file.
Key Description Default Value
odais_chat_title The title of the app that's displayed on the app bar. Ask
odais_connected The status text that displays when the connection between chat widget and the Oracle chat server has been established. Connected
odais_connecting The status text that displays while the chat widget connects to the Oracle chat server. Connecting
odais_disconnected The status text that displays when the connection between the chat view and the Oracle chat server has closed. Disconnected
odais_typing_status The status text that displays while the user waits for the skill's response. Responding...
odais_speech_error The alert message that's displayed when the audio cannot be recorded. Error in voice recognition. Please try again later.
odais_speechrecognition_error The alert message that's displayed when there is a speech recognition error from the speech server. Speech Recognition Error
odais_speech_send The button label text on the speech popup for sending the recorded audio to the speech server. SEND
odais_speech_cancel The button label text on the speech popup for cancelling the sending of recorded audio to the speech server. CANCEL
odais_speech_start The text displayed on the speech popup indicating that the user can now start speaking. Listening...
odais_speech_permission_denied The error message displayed when microphone usage is not allowed. Permission_Denied
odais_file_not_supported The alert message displayed when the user selects file type that's not supported for attachments. File type not supported
odais_file_size_warning The alert message displayed when the file chosen for attachment exceeds 25MB. You can only attach files of size upto 25MB
odais_gallery_permission_denied The alert message displayed when permission to the gallery has not been granted. You don't have permission to access gallery.
odais_photo The action text that appears on the attachment popup for choosing a file from phone's gallery. Photo & Video Library
odais_files The action text that appears on the attachment popup for choosing a file from storage. Files
odais_notification_title The title displayed on the notifications bar. OracleChatBot
odais_warning The title of the alert message that displays when permission to access the gallery has not been granted. Warning
odais_alert The title of the alert message displayed for speech and file-related errors. Alert
odais_check_url The error message displayed for a broken link. Please check the url!
odais_fail_to_load The error message displayed when page is not able to load in the chat view. Failed to load the page
odais_error Title of the error and alert messages in the chat view. Error
odais_ok The label text for the button that closes alert and error messages. Ok
odais_done The label text for the button that closes the chat view. Done
odais_write_a_message The placeholder text in the user message input field. Write a message
odais_camera The action text that appears on the attachment popup for using the device's camera. Camera
odais_speech_unsupported_locale The error message displayed when the set speech locale is not supported by the speech server. The set speech locale is not supported. Cannot start recording.
odais_access_label_skill The accessibility label for a skill message payload read by VoiceOver (the iOS Accessibility feature). This label is then appended with the message specific label. Skill
odais_access_label_user The accessibility label for a user message payload read by VoiceOver (the iOS Accessibility feature). This label is then appended with the message specific label. User
odais_access_label_button_send The accessibility label for the send button Send button
odais_access_label_button_speak The accessibility label for the mic button Speak button
odais_access_label_button_clear The accessibilty label for the clear message button Clear messages
odais_access_label_button_audio_reponse_off The accessibility label for the muted volume button Unmute audio response
odais_access_label_button_audio_reponse_on The accessibility label for the unmuted volume button. Mute audio response
odais_access_label_play The accessibility label for the play button Play
odais_access_label_pause The accessibility label for the pause button. Pause
odais_access_label_button_attach The accessibility label for the upload attachment button. Upload attachment
odais_access_label_chat_title The accessibility label for the chat title which is followed by the chat title string Chat title
odais_access_label_chat_status The accessibility label for the chat status which is followed by the status string Chat status
odais_access_label_file_attachment The accessibility label for a file attachment message file attachment
odais_access_label_image_attachment The accessibility label for an image attachment message image attacment
odais_access_label_audio_attachment The accessibility label for an audio attachment message audio attachment
odais_access_label_video_attachment The accessibility label for a video attachment message video attachment
odais_access_label_location_message The accessibility label for a location message that's followed by the location message title and the latitude and longitude location message
odais_access_label_text_message The accessibility label for a text message that's followed by the text text message
odais_access_label_card_title The accessibility label for a card title that's followed by the text card title
odais_access_label_card_desc The accessibility label for a card description that's followed by the text card description
odais_access_label_header_text The accessibility label for a header text that's followed by the text header text
odais_access_label_footer_text The accessibility label for a footer text that's followed by the text footer text

Colors

You can modify the colors for the following components by using BotsProperties.<component name> = <UIColor type>. As described in Initialize the SDK in Your App, you must set all of the colors in BotsProperties before adding the chat view to the view controller.

Component Description Default Color
ActionButtonColor The color for an action button #0077C2
ActionLabelTextColor The text color for an action label UIColor.white
BotMessageColor The background color for text, attachment, and location messages sent by the skill #BBDEFB
BotTextColor The color for the text in a text message sent by the skill UIColor.black
CardBackgroundColor The background color for a card #BBDEFB
CardDescriptionTextColor The text color for a card description UIColor.black
CardTitleTextColor The text color for a card title UIColor.black
ChatBackgroundColor The color for the chat view background #FAFAFA
FooterColor The background color of the footer. #004C8C
FooterIconsColor The color of the attachment, send, and mic icons located in the footer. #004C8C
HeaderIconsColor The color for the clear message, volume, and mute header icons. UIColor.white
HeaderTextColor The text color for the connection status, typing status, and chat title header items UIColor.white
InputFieldColor The background color of the text input field UIColor.white
InputFieldTextColor The color of the text in the text input field UIColor.black
Theme The UI theme for the application. Valid values are BotUITheme.REDWOOD_DARK and BotUITheme.DEFAULT. For the REDWOOD_DARK theme, we recommend #201E1C. For the DEFAULT theme, it's #004C8C. Set the color of the navigation bar using the sample app. BotUITheme.DEFAULT
TimestampColor The color for the message timestamp UIColor.lightGray
UserMessageColor The background color for a user message #E0E0E0
UserTextColor The color for the text in a text user sent by the user UIColor.black

Features

Agent Avatars

For skills integrated with live agent support, the agentAvatar setting enables the display of an avatar icon for the messages sent by the agents. You configure this with the URL of the icon that displays alongside the agent messages.

Delegation
The delegation feature lets you set a delegate to receive callbacks before certain events in the conversation. To set a delegate, a class must conform to the BotsMessageServiceDelegate protocol and implement the following methods:
public func beforeDisplay(message: [String: Any]?) -> [String: Any]?

This method allows a skill’s message payload to be modified before it is displayed in the conversation. The message payload returned by the method is used to display the message. If it returns nil, then the message is not displayed.

public func beforeSend(message: [String: Any]?) -> [String: Any]?

This method allows a user message payload to be modified before it is sent to the chat server. The message payload returned by the method is sent to the skill. If it returns nil, then the message is not sent.

public func beforeSendPostback(action: [String: Any]?) -> [String: Any]?
The public func beforeSendPostback(action: [String: Any]?) -> [String: Any]? allows a postback action payload to be modified before it is sent to the chat server. The action payload returned by the method is sent to the skill. If it returns nil, then the postback action selected by the user is not sent to the chat server.
public class ViewController: UIViewController, BotsMessageServiceDelegate {
    func beforeSend(message: [String : Any]?) -> [String : Any]? {
        // Handle before send delegate here
    }
    
    func beforeDisplay(message: [String : Any]?) -> [String : Any]? {
        // Handle before display delegate here
    }
    
    func beforeSendPostback(action: [String : Any]?) -> [String : Any]? {
        // Handle before send postback action delegate here
    }
}
The instance, which conforms to the BotsMessageServiceDelegate protocol, should be assigned to the BotsManager.shared().delegate property as shown in the following code snippet for initializing the SDK:
BotsManager.shared().delegate = self
Headless SDK

The SDK can be used without its UI. The SDK maintains the connection to server and provides APIs to send messages, receive messages, and get updates for the network status and for other services. You can use the APIs to interact with the SDK and update the UI.

You can send a message using any of the send() APIs available in BotsManager class. For example, public func send(message: UserMessage) sends text message to skill or digital assistant.

public func send(message: UserMessage)

This function sends a message to the skill. Its message parameter is an instance of a class which conforms to the UserMessage class. In this case, it is UserTextMessage.BotsManager.shared().send(message: UserTextMessage(text: "I want to order a pizza", type: .text))

BotsEventListener
To listen for the connection status change, a message received from skill and attachment upload status events, a class can implement the BotsEventListener protocol which then implements the following methods:
  • onStatusChange(ConnectionStatus connectionStatus) – This method is called when the WebSocket connection status changes. Its connectionStatus parameter is the current status of the connection. Refer to the API docs included in the SDK for more details about the ConnectionStatus enum.
  • onReceiveMessage(message: BotsMessage) – This method is called when a new message is received from the skill. Its message parameter is the message received from the skill. Refer to the API docs included in the SDK for more details about the BotsMessage class.
  • onUploadAttachment(message: BotsAttachmentMessage) – This method is called when an attachment upload has completed. Its message parameter is the BotsAttachmentMessage object for the uploaded attachment.
  • onSpeechResponseReceived(data:String, final:Bool) - This method is called when a response is received from the speech server. It takes the following paramters:
    • data - The text which is returned from the speech server on converting the audio.
    • final - Has a true value if the response is final and false if the response is partial (and the user must wait for the final response)
public class ViewController: UIViewController, BotsEventListener {
    func onReceiveMessage(message: BotsMessage) {
        // Handle the messages received from skill or Digital Assistant
    }
    
    func onUploadAttachment(message: BotsAttachmentMessage) {
        // Handle the post attachment upload actions
    }
    
    func onSpeechResponseReceived(data: String, final: Bool) {
        // Handle the responses received from the speech server
    }
    
    func onStatusChange(connectionStatus: ConnectionStatus) {
        // Handle the connection status change
    }
}
The instance which conforms to the BotsEventListener protocol should be assigned to the BotsManager.shared().botsEventListener property as illustrated in the following code snippet for initializing the SDK:
BotsManager.shared().botsEventListener = self
Message Timestamp Formatting

The timestampFormat flag formats timestamps that display in the messages. It can accept a string consisting of format tokens like "hh:mm:ss" and other formats supported by the Swift DateFormatter.

Share Menu Options
By default, the share menu displays options for the following file types:
  • visual media files (images and videos)
  • audio files
  • general files like documents, PDFs, and spreadsheets
  • location
The sharePopupConfiguration setting allows you to restrict the items that display in the share menu. The setting accepts an array of enum type ShareMenuItem, which has keys that are mapped to the share menu items: ShareMenuItem.photoAndVideoLibrary for the image/ video share item, ShareMenuItem.files for the share file item, and ShareMenuItem.camera for sharing an image directly from the camera. Only the menu items for the values passed in the array display in the share popup. An empty or invalid array results in all of the menu items displaying together in the share popup.
Speech Recognition

Setting the enableSpeechRecognition feature flag to true enables the microphone button to display in place of the send button whenever the user input field is empty. The speech is converted to text and sent to the skill or digital assistant. If the speech is partially recognized, then the partial result is displayed in a popup that's opened by clicking the microphone button.

Setting this property to true also supports the functionality enabled by the enableAutoSendSpeechResponse property, which when also set to true, enables the user's speech response to be sent to the chat server automatically while displaying the response as a sent message in the chat window. You can allow users to first edit (or delete) their dictated messages before they send them manually by setting enableSpeechRecognitionAutoSend to false.

Speech recognition is utilized through the following methods:
public func startRecording()

Starts recording the user's voice message.

public func stopRecording()

Stops recording the user's message.

public func isRecording() -> Bool

Checks whether the voice recording has started or not. Returns true if the recording has started. Otherwise, it returns false.

The onSpeechResponseReceived(data: String, final: Bool) function from the BotsEventListener protocol can be used to handle all the responses from the speech server.
BotsManager.shared().startRecording()
if (BotsManager.shared().isRecording()) {
    BotsManager.shared().stopRecording() // Stop voice recording
}
Speech Synthesis
The SDK has been integrated with speech synthesis to read the skill's message aloud when a new message is received from skill.
  • You enable this feature by setting the enableSpeechSynthesis feature flag to true.
  • You can set the preferred language that read the skill's messages aloud with the speechSynthesisVoicePreferences property. This property enables a fallback when the device doesn't support the preferred language or voice. If the device does not support the preferred voice, then the default voice for the preferred language is used instead. When neither the preferred voice or language are supported, then the default voice and language are used.
public func speak(text: String)
Starts reading the skill's response aloud. Its text parameter is the text for the skill's message that's read aloud.
BotsManager.shared().speak(text: "What kind of crust do you want?")
public func stopSpeech()
Stops reading the skill's response aloud.
BotsManager.shared().stopSpeech()
Voice Visualizer

When voice support is enabled (botsConfiguration.enableSpeechRecognition = true), the footer of the chat widget displays a voice visualizer, a dynamic visualizer graph that indicates the frequency level of the voice input. The visualizer responds to the modulation of the user's voice by indicating whether the user is speaking too softly or too loudly. This visualizer is created using Swift's AVAudioEngine which is exposed in the onAudioReceived method in the SpeechEventListener protocol for use in headless mode.

The chat widget displays a voice visualizer when users click the voice icon. It's an indicator of whether the audio level is sufficiently high enough for the SDK to capture the user’s voice. The user’s message, as it is recognized as text, displays below the visualizer.

Note:

Voice mode is indicated when the keyboard icon appears.

When botsConfiguration.enableSpeechAutoSendSpeechResponse = true, the recognized text is automatically sent to the skill after the user has finished dictating the message. The mode then reverts to text input. When botsConfiguration.enableSpeechAutoSendSpeechResponse = false, the mode also reverts to text input, but in this case, users can modify the recognized text before sending the message to the skill.

Message Model

To use features like headless mode and delegate, you need to understand both user and skill messages. Everything that's received or sent from the Oracle Chat Server is represented as a message, one that's sent from the user to the skill, or from the skill to the user.

Base Types
These are the base types used in all messages sent from the user to the skill and vice versa. They are the building blocks of all messages.
Attachment
Represents an attachment that's sent by the user.
Name Description Type Required?
type The attachment type string (valid values: audio, file, image, video) Yes
url The download URL for the attachment string Yes
For example:
{
    "type": "image",
    "url": "https://www.oracle.com/us/assets/hp07-oow17-promo-02-3737849.jpg"
}
Location
Represents a location object.
Name Description Type Required?
title The location title string No
URL The URL for displaying the location on a map string No
latitude The GPS coordinate's longitude value double Yes
longitude The GPS coordinate's latitude value double Yes
For example:
{
    "title": "Oracle Headquarters",
    "url": "https://www.google.com.au/maps/place/37°31'47.3%22N+122°15'57.6%22W",
    "longitude": -122.265987,
    "latitude": 37.529818
}
Action
An action represents something that the user can select.
Name Description Type Required?
type The action type string Yes
label The descriptive label text for the action. string At least one label or imageUrl must be present.
imageUrl The image for the action string At least one label or imageUrl must be present.
PostbackAction
Sends a predefined postback back to the skill when the user selects an action.
Name Description Type Required?
type The action type "postback" Yes
postback The postback that's returned when the user selects an action. A string or JSON object Yes
For example:
{
    "type": "postback",
    "label": "Large Pizza",
    "imageUrl": "https://example.com/images/gallery/locations/11.jpg",
    "postback": {
        "state": "askSize",
        "action": "getCrust"
    }
}
CallAction
Requests the client to call a specified phone number on behalf of the user.
Name Description Type Required?
type The action type "call" Yes
phoneNumber The phone number to call string Yes
For example:
{
    "type": "call",
    "label": "Call Support",
    "imageUrl": "http://example.com.ar/files/2016/05/cuidado.jpg",
    "phoneNumber": "18005555555"
}
urlAction

Requests the client to open a website in a new tab or in an in-app browser.

Name Description Type Required?
type The action type "call" Yes
URL The URL of the website that's displayed. string Yes
For example:
{
    "type": "url",
    "label": "Open URL",
    "imageUrl": "http://example.com.ar/files/2016/05/cuidado.jpg",
    "url": "https://example.com/images/gallery/locations/11.jpg",
}
LocationAction
Requests the client to ask for the user's location.
Name Description Type Required?
type The action type "location" Yes
For example:
{
    "type": "location",
    "label": "Share location",
    "imageUrl": "http://images.example.com/location-clipart-location-pin-clipart-1.jpg"
}
Card
Represents a single card in the message payload.
Name Description Type Required?
title The title of the card, displayed as the first line on the card. string Yes
description The description of the card string No
imageUrl The URL of the image that is displayed. string No
URL The website URL that's opened by a tap. string No
actions An array of actions related to the text array No
Conversation Message
All of the messages that are part of a conversation have the following structure:
Name Description Type Required?
messagePayload The message payload Message Yes
userId The user ID string Yes
For example:
{
    "messagePayload": {
        "text": "show menu",
        "type": "text"
    },
    "userId": "guest"
}
User Message

Represents a message sent from the user to the skill.

User Text Message
The simple text message that's sent to the server.
Name Description Type Required
type The message type "text" Yes
text The message text string Yes
For example:
{
    "messagePayload": {
        "text": "Order Pizza",
        "type": "text"
    },
    "userId": "guest"
}
User Postback Message
The postback response message that's sent to the server.
Name Description Type Required
type The message type "postback" Yes
text The postback text string No
postback The postback of the selected action A string or JSON object Yes
For example:
{
    "messagePayload": {
        "postback": {
            "variables": {
                "pizza": "Small"
            },
            "system.botId": "69BBBBB-35BB-4BB-82BB-BBBB88B21",
            "system.state": "orderPizza"
        },
        "text": "Small",
        "type": "postback"
    },
    "userId": "guest"
}
User Attachment Message
The attachment response message that's sent to the server.
Name Description Type Required
type The message type "attachment" Yes
attachment The attachment metadata Attachment Yes
For example:
{
    "messagePayload": {
        "attachment": {
            "type": "image",
            "url": "http://oda-instance.com/attachment/v1/attachments/d43fd051-02cf-4c62-a422-313979eb9d55"
        },
        "type": "attachment"
    },
    "userId": "guest"
}
User Location Message
The location response message that's sent to the server.
Name Description Type Required
type The message type "location" Yes
location The user location information Location Yes
For example:
{
    "messagePayload": {
        "location": {
            "latitude": 45.9285271,
            "longitude": 132.6101925
        },
        "type": "location"
    },
    "userId": "guest"
}
Bot Message

Represents the message sent from the skill to the user.

Message
Message is an abstract base type for all other messages. All messages extend it to provide some information.
Name Description Type Required?
type The message type string Yes
Bot Text Message
Represents a text message.
Name Description Type Required
type The message type "text" Yes
text The message text string Yes
headerText The header text for cards string No
footerText The footer text for cards string No
actions An array of actions related to the text. Array No
globalActions An array of global actions related to the text. Array No
For example:
{
    "messagePayload": {
        "type": "text",
        "text": "What do you want to do?",
        "actions": [
            {
                "type": "postback",
                "label": "Order Pizza",
                "postback": {
                    "state": "askAction",
                    "action": "orderPizza"
                }
            },
            {
                "type": "postback",
                "label": "Cancel A Previous Order",
                "postback": {
                    "state": "askAction",
                    "action": "cancelOrder"
                }
            }
        ]
    },
    "userId": "guest"
}
Bot Attachment Message
Represents an attachment message.
Name Description Type Required
type The message type "attachment" Yes
attachment The attachment sent Attachment Yes
headerText The card's header text string No
footerText the card's footer text string No
actions An array of actions related to the text. Array No
globalActions An array of global actions related to the text. Array No
Bot Card Message
Represents a set of choices that are displayed for the user, either horizontally as carousels or vertically as lists.
Name Description Type Required
type The message type "card" Yes
layout Whether to display the messages horizontally or vertically. string (values: horizontal, vertical) Yes
cards An array of cards to be rendered. Array Yes
headerText The cards' header text string No
actions An array of actions related to the text. Array No
globalActions An array of global actions related to the text. Array No
{
    "messagePayload": {
        "type": "card",
        "layout": "horizontal",
        "cards": [
            {
                "title": "Hawaiian Pizza",
                "description": "Ham and pineapple on thin crust",
                "actions": [
                    {
                        "type": "postback",
                        "label": "Order Small",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "hawaiian",
                                "pizzaCrust": "thin",
                                "pizzaSize": "small"
                            }
                        }
                    },
                    {
                        "type": "postback",
                        "label": "Order Large",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "hawaiian",
                                "pizzaCrust": "thin",
                                "pizzaSize": "large"
                            }
                        }
                    }
                ]
            },
            {
                "title": "Cheese Pizza",
                "description": "Cheese pizza (i.e. pizza with NO toppings) on thick crust",
                "actions": [
                    {
                        "type": "postback",
                        "label": "Order Small",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "cheese",
                                "pizzaCrust": "thick",
                                "pizzaSize": "small"
                            }
                        }
                    },
                    {
                        "type": "postback",
                        "label": "Order Large",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "cheese",
                                "pizzaCrust": "thick",
                                "pizzaSize": "large"
                            }
                        }
                    }
                ]
            }
        ],
        "globalActions": [
            {
                "type": "call",
                "label": "Call for Help",
                "phoneNumber": "123456789"
            }
        ]
    },
    "userId": "guest"
}
Bot Raw Message
Used when a component creates the channel-specific payload itself.
Name Description Type Required?
type The message type "raw" Yes
payload The channel-specific payload A JSON object Yes

Oracle iOS Channel Extensions

For Oracle iOS channels, you can extend the functionality of System.CommonResponse components with capabilities that are specific to the Oracle iOS SDK.

You access the extensions by using the channelCustomProperties element in the System.CommonResponse component and setting the appropriate properties. The code has the following format:

...
            channelCustomProperties:
            - channel: "iossdk"
              properties:
                PROPERTY_NAME: "PROPERTY_VALUE"
...

Here are the available custom properties for Oracle iOS channels:

Name Allowed Values Applies To... Description
mediaType
  • A valid media type
  • Response items with the following attributes:
    • type: "attachment"
    • attachmentType: "file"or attachmentType: "image"
  • Cards with imageUrl specified
The media type of the attachment. For example, image/jpeg. If not specified, the media type will be resolved from the attachment URL.

For more general information on channelCustomProperties, see Channel-Specific Extensions.