38 Oracle iOS

Using the Oracle iOS SDK for Oracle Digital Assistant, you can integrate your digital assistant with iOS apps. The SDK connects to the Oracle Chat Server, the intermediary between the Oracle iOS channel configured in Oracle Digital Assistant and the client. The chat server then passes messages to the skill for processing and delivers the skill's response to the client.

This SDK works with instances of Oracle Digital Assistant that were provisioned on Oracle Cloud Infrastructure (sometimes referred to as the Generation 2 cloud infrastructure). If your instance is provisioned on the Oracle Cloud Platform (as all version 19.4.1 instances are), you can use the legacy iOS SDK.

What Do You Need?

• An Oracle iOS Channel. Creating the channel generates the Channel ID and the Secret Key that you need to initialize the chat app.
• The URL of the Oracle Chat Server.
• The Oracle iOS SDK (located under Oracle Native Client SDKs for OCI Native Environments) from Oracle Technology Network’s ODA and OMC download page. Download this ZIP and extract it to your local system. This ZIP includes a user guide that describes the SDK's functions.
• Swift 4.2 as the Swift Language Version. Xcode 11 or higher. If you want your app to work with earlier versions, keep in mind that we haven't tested these and therefore can't guarantee their compatibility.

Create the Oracle iOS Channel

You can configure the channel to connect to the Oracle Chat Server in two modes: unauthenticated mode and authenticated mode (to protect access to the channel).

• Unauthenticated mode – Use the unauthenticated mode when the client can't generate signed JWT tokens, when no authentication mechanism is in place, or when the client widget is already secured and visible to authenticated users.
• Authenticated mode – Authentication is enforced using JSON Web Tokens (JWT). The customer's backend server generates the JWT token, which is then passed to the Oracle iOS SDK. This token is used for each request to an ODA speech, text, or attachment server.

  Note:

  To protect access to the channel, the token must always be generated by a remote server. It must never be generated within by the client app.
  When the app needs to connect to an ODA server, it first requests the token from the backend server and then adds it to the Authorization header. The ODA server validates the token, evaluates the claims, and then either opens the socket or rejects the connection.
  The JWT Token has the following claims: channelId and userId, and the claim names iat (issued at time), and exp (expiration time). iat signifies the time at which the token was issued. This must a number that represents the seconds that have elapsed since the UNIX epoch. exp must be a number representing the seconds that have elapsed since the UNIX epoch. We recommend setting the expiration time to at least 30 minutes after the issued at time (iat). The token header looks something like this:

  {
  
   "alg": "HS256",
  
   "typ": "JWT"
  
  }

  An example token body looks something like this:

  {
  
    "iat": 1569828182,
  
    "exp": 1569831782,
  
    "channelId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  
    "userId": "John"
  
  }

  Note:

  The token illustrated by this example is not signed. The actual tokens are signed by channel's Secret Key.

Configure the Oracle iOS Channel

To configure the Oracle iOS channel:

1. Choose Development, then Channels from the menu.
2. Choose Users.
3. Click Add Channel and then Oracle iOS as the channel type.
4. Complete the dialog:
   • Enter the channel name.
   • For authenticated connections:
     • Switch on the Client Authentication Enabled toggle to determine whether the SDK is connecting to a client authentication-enabled channel.
     • In the Max. Token Expiration (Minutes) field, set the maximum amount of time for the JWT token.
     • Set the Session expiration time.
   • For unauthenticated connections:
     • Switch off Client Authentication Enable toggle.
     • Set the Session expiration time.
   • Click Create. Oracle Digital Assistant will generate the Channel ID and the Secret Key that you need to initialize the SDK. Keep these close at hand because you'll need them when configuring the HTML page to host the chat widget.
5. Route the channel to your skill or digital assistant.
6. Switch Channel Enabled to On.

Add the SDK to the Project

1. Download the ODA Client SDK for iOS and extract it to your local system.
2. Add the files to your Xcode project:
   1. Click File > Add Files to "<project name>".
   2. Choose the .framework files that you want to add depending on where you want to run the app (simulator or actual device).
   3. Make sure to that Copy items if needed (located under Destinations) is selected.
   4. Alternatively, you can drag and drop the .framework files into the project file in Xcode.
3. Embed and sign the frameworks in the Frameworks, Libraries, and Embedded Content category in the General tab. (This may vary according to the version of Xcode that you're using.) Make sure the Targets option is selected.
4. Add the following keys in the project's .plist file:
   • Privacy - Location Always and When In Use Usage Description or <key>NSLocationAlwaysUsageDescription</key> and the corresponding <string></string> in the source code.
   • Privacy - Location When In Use Usage Description or <key>NSLocationWhenInUseUsageDescription</key> and the corresponding <string></string> in the source code.
   • Privacy - Microphone Usage Description or <key>NSMicrophoneUsageDescription</key> and the corresponding <string></string> in the source code.
   • Privacy - Camera Usage Description or <key>NSCameraUsageDescription</key> and the corresponding <string></string> in the source code.
   • To open a location in Google maps instead of Apple maps when a user taps a map view in a location message, declare the URL schemes used by the Maps SDK for iOS in the app's Info.plist file as described in the Google Maps SDK for iOS documentation.

Initialize the SDK in Your App

You can use the following code to initialize the chat view.

// Import the SDK
import BotClientUISDK
 
// Declare a global BotsViewController variable in your app view controller class
public var chatViewController: BotsViewController?

// Obtain a shared instance of BotsViewController from BotsUIManager
chatViewController = BotsUIManager.shared().viewControllerInstance()

// Specify the color changes, if any, in a particular component. Make sure you set all the required colors in BotsProperties before adding the chat view to the view controller.
  
// Add the chat view to your view controller
self.view.addSubview((chatViewController?.view)!)
  
// Initialize a BotsConfiguration object and set feature flags, if required.
var botsConfiguration = BotsConfiguration(url: baseUrl, userId: botID!, channelId: channelID)
  
// Initialize the configuration in botsViewController. Make sure you set all the feature flag values before passing the botsConfiguration to initConfiguration.
chatViewController?.initConfiguration(botsConfiguration: botsConfiguration)
  
// Get a custom navigation bar view from botsViewController and add it to the navigation bar of your app.
let customView:UIView = (chatViewController?.addCustomViewToNavBar())!
self.navigationController?.navigationBar.addSubview(customView)
 
// Obtain a shared instance of BotsManager
let botsManager = BotsManager.shared()
 
// If you require access to callback methods provided in BotsMessageServiceDelegate. Make sure your class conforms to BotsMessageServiceDelegate
botsManager.delegate = self

// If you require access to callback methods provided in BotsEventListener. Make sure your class conforms to BotsEventListener
botsManager.botsEventListener = self
 
// Initialize and establish connection to the chat server
botsManager.initialize(botsConfiguration: botsConfiguration, completionHandler: { (connectionStatus, error) in
 
})

Note:

The SDK currently provides UI for portrait orientation only.

App Development

Initialize the Feature Flag Settings

Initialize BotsConfiguration object using one of its constructors.

• clientAuthDisabled
  • BotsConfiguration(url: String, userId: String, channelId: String)
    • Parameters:
      • url - The Oracle Chat Server URL. This cannot be null.
      • userId - The unique identifier for the user. This cannot be null.
      • channelId - The Oracle iOS Channel ID. This cannot be null.
    • When the userId not provided (a randomly generated value is instead)
      • url - The Oracle Chat Server URL. This cannot be null.
      • channelId - The Oracle iOS Channel ID. This cannot be null.
• clientAuthEnabled
  • BotsConfiguration(url: String, authToken: String)
    • Parameters:
      • url - The Oracle Chat Server URL. This cannot be null.
      • authToken - The authentication token for establishing a connection with an authentication-enabled channel. This cannot be null.

For example:

// Initialize a BotsConfiguration object
var botsConfiguration = BotsConfiguration(url: chatServerUrl, authToken: token)
 
// Set the feature flag values if the desired values are different from the default values
botsConfiguration.showConnectionStatus = true
botsConfiguration.enableBotAudioResponse = true
botsConfiguration.disablePastActions = "none"

Network Configuration

Property Name	Description	Required?	Default Value
url	The URL of the Oracle Chat Server	Yes	N/A
channelId	The ID of the Oracle iOS channel.	Yes	N/A
userId	The unique identifier for user. This value gets initialized by the SDK if not provided.	No	A randomly generated value
authToken	The authentication token for establishing a connection with an authentication-enabled channel.	Yes	N/A

Feature Flags

Property	Description	Required?	Default Value
agentAvatar	Passes the asset name of the agent avatar image.	No	N/A
botAvatar	Passes the asset name of the skill avatar image.	No	N/A
disablePastActions	A field for disabling the button clicks on the messages that a user has already interacted with. The allowed values are all, none, and postback. The behavior enabled by this property is independent of the digital assistant-level configuration for disabling the selection of past actions. You need to set the two separately.	No	all
enableAttachment	Configures attachment sharing in the chat app.	No	true
enableAutoSendSpeechResponse	When set to true (the default), the user's speech response is automatically sent to the chat server (and rendered as a sent message in the chat window). When set to false, the user's speech response is rendered in the message text field before it's sent to the chat server, allowing the user to modify it before sending it manually, or delete the message.	No	true
enableClearMessage	Enables the clear message button in the chat widget header.	No	false
enableSpeechRecognition	Enables the microphone button.	No	false
enableSpeechSynthesis	Enables the skill responses to be read aloud. By setting this flag to true, you enable the skill's responses to be read aloud using Swift API.	No	false
enableTimestamp	Enables the timestamp for messages.	No	true
initSpeechSynthesisMuted	Sets the default state of BotAudioResponse as muted.	No	true
initUserHiddenMessage	A user text message that's used to initiate a conversation. This message, which is sent when chat widget is ready, does not actually display in chat.	No	N/A
initUserProfile	Updates the user profile before the start of a conversation. The format of the profile payload must be ["profile": […] ]. For example: initUserProfile = ["profile": ["givenName": "First", "surname": "Last", "email": "[e-mail]", "properties": ["lastOrderedItems": "1 medium pepperoni"]]] This function updates the user context before the initial "hidden" message is sent by initUserHiddenMessage to start the conversation. As a result, the user profile can be reflected in the first response message to the user. For example, the skill can greet the user with a message like "Welcome back, John Smith! Your last order was a medium pepperoni pizza."	No	N/A
personAvatar	Passes the asset name of the person avatar image.	No	N/A
sharePopupConfiguration	Allows the user to choose the options available as part of the attachment menu. The default value is set to a list of all options.	No	[.photoAndVideoLibrary, .files, .camera]
showConnectionStatus	Enables the connection status to display in the chat widget header.	No	false
showTypingStatus	Enables the display of the Responding… text indicator when waiting for a response from the skill.	No	true
speechLocale	The expected locale of the user's speech that's used for voice recognition. The supported locales are: French (fr-fr), Spanish (es-es), and the default, English (en-US). This string connects to the speech server endpoint for the particular locale when users tap the microphone button. The speech locale can be set dynamically with the setSpeechLocale(string: locale) API. Voice recognition only works when one of the supported locales is passed.	No	en-us
speechSynthesisVoicePreferences	Matches the provided preferences based on both the language-locale and voice name. If no match is found, then the default voice for the given language-locale is used. In the latter case, the Apple API finds best match with the given language-locale.	No	N/A
timestampFormat	Formats the delivery timestamp that accompanies messages. The timestamp format should be supported by the Swift DateFormatter.	No	MMMM d, yyyy HH:mm
title	Sets the title of the app, which is displayed in the app bar.	No	N/A
typingStatusTimeout	Sets the timeout, in seconds, to hide the typing status indicator when no response has been received from the chat server.	No	20 seconds

Strings

Configure strings by adding the following key = value pairs in the app's <language-code>.iproj/Localizable.strings file.

Key	Description	Default Value
odais_chat_title	The title of the app that's displayed on the app bar.	Ask
odais_connected	The status text that displays when the connection between chat widget and the Oracle chat server has been established.	Connected
odais_connecting	The status text that displays while the chat widget connects to the Oracle chat server.	Connecting
odais_disconnected	The status text that displays when the connection between the chat view and the Oracle chat server has closed.	Disconnected
odais_typing_status	The status text that displays while the user waits for the skill's response.	Responding...
odais_speech_error	The alert message that's displayed when the audio cannot be recorded.	Error in voice recognition. Please try again later.
odais_speechrecognition_error	The alert message that's displayed when there is a speech recognition error from the speech server.	Speech Recognition Error
odais_speech_send	The button label text on the speech popup for sending the recorded audio to the speech server.	SEND
odais_speech_cancel	The button label text on the speech popup for cancelling the sending of recorded audio to the speech server.	CANCEL
odais_speech_start	The text displayed on the speech popup indicating that the user can now start speaking.	Listening...
odais_speech_permission_denied	The error message displayed when microphone usage is not allowed.	Permission_Denied
odais_file_not_supported	The alert message displayed when the user selects file type that's not supported for attachments.	File type not supported
odais_file_size_warning	The alert message displayed when the file chosen for attachment exceeds 25MB.	You can only attach files of size upto 25MB
odais_gallery_permission_denied	The alert message displayed when permission to the gallery has not been granted.	You don't have permission to access gallery.
odais_photo	The action text that appears on the attachment popup for choosing a file from phone's gallery.	Photo & Video Library
odais_files	The action text that appears on the attachment popup for choosing a file from storage.	Files
odais_notification_title	The title displayed on the notifications bar.	OracleChatBot
odais_warning	The title of the alert message that displays when permission to access the gallery has not been granted.	Warning
odais_alert	The title of the alert message displayed for speech and file-related errors.	Alert
odais_check_url	The error message displayed for a broken link.	Please check the url!
odais_fail_to_load	The error message displayed when page is not able to load in the chat view.	Failed to load the page
odais_error	Title of the error and alert messages in the chat view.	Error
odais_ok	The label text for the button that closes alert and error messages.	Ok
odais_done	The label text for the button that closes the chat view.	Done
odais_write_a_message	The placeholder text in the user message input field.	Write a message
odais_camera	The action text that appears on the attachment popup for using the device's camera.	Camera
odais_speech_unsupported_locale	The error message displayed when the set speech locale is not supported by the speech server.	The set speech locale is not supported. Cannot start recording.
odais_access_label_skill	The accessibility label for a skill message payload read by VoiceOver (the iOS Accessibility feature). This label is then appended with the message specific label.	Skill
odais_access_label_user	The accessibility label for a user message payload read by VoiceOver (the iOS Accessibility feature). This label is then appended with the message specific label.	User
odais_access_label_button_send	The accessibility label for the send button	Send button
odais_access_label_button_speak	The accessibility label for the mic button	Speak button
odais_access_label_button_clear	The accessibilty label for the clear message button	Clear messages
odais_access_label_button_audio_reponse_off	The accessibility label for the muted volume button	Unmute audio response
odais_access_label_button_audio_reponse_on	The accessibility label for the unmuted volume button.	Mute audio response
odais_access_label_play	The accessibility label for the play button	Play
odais_access_label_pause	The accessibility label for the pause button.	Pause
odais_access_label_button_attach	The accessibility label for the upload attachment button.	Upload attachment
odais_access_label_chat_title	The accessibility label for the chat title which is followed by the chat title string	Chat title
odais_access_label_chat_status	The accessibility label for the chat status which is followed by the status string	Chat status
odais_access_label_file_attachment	The accessibility label for a file attachment message	file attachment
odais_access_label_image_attachment	The accessibility label for an image attachment message	image attacment
odais_access_label_audio_attachment	The accessibility label for an audio attachment message	audio attachment
odais_access_label_video_attachment	The accessibility label for a video attachment message	video attachment
odais_access_label_location_message	The accessibility label for a location message that's followed by the location message title and the latitude and longitude	location message
odais_access_label_text_message	The accessibility label for a text message that's followed by the text	text message
odais_access_label_card_title	The accessibility label for a card title that's followed by the text	card title
odais_access_label_card_desc	The accessibility label for a card description that's followed by the text	card description
odais_access_label_header_text	The accessibility label for a header text that's followed by the text	header text
odais_access_label_footer_text	The accessibility label for a footer text that's followed by the text	footer text

Colors

You can modify the colors for the following components by using BotsProperties.<component name> = <UIColor type>. As described in Initialize the SDK in Your App, you must set all of the colors in BotsProperties before adding the chat view to the view controller.

Component	Description	Default Color
ActionButtonColor	The color for an action button	#0077C2
ActionLabelTextColor	The text color for an action label	UIColor.white
BotMessageColor	The background color for text, attachment, and location messages sent by the skill	#BBDEFB
BotTextColor	The color for the text in a text message sent by the skill	UIColor.black
CardBackgroundColor	The background color for a card	#BBDEFB
CardDescriptionTextColor	The text color for a card description	UIColor.black
CardTitleTextColor	The text color for a card title	UIColor.black
ChatBackgroundColor	The color for the chat view background	#FAFAFA
FooterColor	The background color of the footer.	#004C8C
FooterIconsColor	The color of the attachment, send, and mic icons located in the footer.	#004C8C
HeaderIconsColor	The color for the clear message, volume, and mute header icons.	UIColor.white
HeaderTextColor	The text color for the connection status, typing status, and chat title header items	UIColor.white
InputFieldColor	The background color of the text input field	UIColor.white
InputFieldTextColor	The color of the text in the text input field	UIColor.black
Theme	The UI theme for the application. Valid values are BotUITheme.REDWOOD_DARK and BotUITheme.DEFAULT. For the REDWOOD_DARK theme, we recommend #201E1C. For the DEFAULT theme, it's #004C8C. Set the color of the navigation bar using the sample app.	BotUITheme.DEFAULT
TimestampColor	The color for the message timestamp	UIColor.lightGray
UserMessageColor	The background color for a user message	#E0E0E0
UserTextColor	The color for the text in a text user sent by the user	UIColor.black

Features

Agent Avatars

For skills integrated with live agent support, the agentAvatar setting enables the display of an avatar icon for the messages sent by the agents. You configure this with the URL of the icon that displays alongside the agent messages.

Delegation

The delegation feature lets you set a delegate to receive callbacks before certain events in the conversation. To set a delegate, a class must conform to the BotsMessageServiceDelegate protocol and implement the following methods:

• public func beforeDisplay(message: [String: Any]?) -> [String: Any]?
• public func beforeSend(message: [String: Any]?) -> [String: Any]?
• public func beforeSendPostback(action: [String: Any]?) -> [String: Any]?

public func beforeDisplay(message: [String: Any]?) -> [String: Any]?

This method allows a skill’s message payload to be modified before it is displayed in the conversation. The message payload returned by the method is used to display the message. If it returns nil, then the message is not displayed.

public func beforeSend(message: [String: Any]?) -> [String: Any]?

This method allows a user message payload to be modified before it is sent to the chat server. The message payload returned by the method is sent to the skill. If it returns nil, then the message is not sent.

public func beforeSendPostback(action: [String: Any]?) -> [String: Any]?

The public func beforeSendPostback(action: [String: Any]?) -> [String: Any]? allows a postback action payload to be modified before it is sent to the chat server. The action payload returned by the method is sent to the skill. If it returns nil, then the postback action selected by the user is not sent to the chat server.

public class ViewController: UIViewController, BotsMessageServiceDelegate {
    func beforeSend(message: [String : Any]?) -> [String : Any]? {
        // Handle before send delegate here
    }
    
    func beforeDisplay(message: [String : Any]?) -> [String : Any]? {
        // Handle before display delegate here
    }
    
    func beforeSendPostback(action: [String : Any]?) -> [String : Any]? {
        // Handle before send postback action delegate here
    }
}

The instance, which conforms to the BotsMessageServiceDelegate protocol, should be assigned to the BotsManager.shared().delegate property as shown in the following code snippet for initializing the SDK:

BotsManager.shared().delegate = self

Headless SDK

The SDK can be used without its UI. The SDK maintains the connection to server and provides APIs to send messages, receive messages, and get updates for the network status and for other services. You can use the APIs to interact with the SDK and update the UI.

You can send a message using any of the send() APIs available in BotsManager class. For example, public func send(message: UserMessage) sends text message to skill or digital assistant.

public func send(message: UserMessage)

This function sends a message to the skill. Its message parameter is an instance of a class which conforms to the UserMessage class. In this case, it is UserTextMessage.BotsManager.shared().send(message: UserTextMessage(text: "I want to order a pizza", type: .text))

BotsEventListener

To listen for the connection status change, a message received from skill and attachment upload status events, a class can implement the BotsEventListener protocol which then implements the following methods:

• onStatusChange(ConnectionStatus connectionStatus) – This method is called when the WebSocket connection status changes. Its connectionStatus parameter is the current status of the connection. Refer to the API docs included in the SDK for more details about the ConnectionStatus enum.
• onReceiveMessage(message: BotsMessage) – This method is called when a new message is received from the skill. Its message parameter is the message received from the skill. Refer to the API docs included in the SDK for more details about the BotsMessage class.
• onUploadAttachment(message: BotsAttachmentMessage) – This method is called when an attachment upload has completed. Its message parameter is the BotsAttachmentMessage object for the uploaded attachment.
• onSpeechResponseReceived(data:String, final:Bool) - This method is called when a response is received from the speech server. It takes the following paramters:
  • data - The text which is returned from the speech server on converting the audio.
  • final - Has a true value if the response is final and false if the response is partial (and the user must wait for the final response)

public class ViewController: UIViewController, BotsEventListener {
    func onReceiveMessage(message: BotsMessage) {
        // Handle the messages received from skill or Digital Assistant
    }
    
    func onUploadAttachment(message: BotsAttachmentMessage) {
        // Handle the post attachment upload actions
    }
    
    func onSpeechResponseReceived(data: String, final: Bool) {
        // Handle the responses received from the speech server
    }
    
    func onStatusChange(connectionStatus: ConnectionStatus) {
        // Handle the connection status change
    }
}

The instance which conforms to the BotsEventListener protocol should be assigned to the BotsManager.shared().botsEventListener property as illustrated in the following code snippet for initializing the SDK:

BotsManager.shared().botsEventListener = self

Message Timestamp Formatting

The timestampFormat flag formats timestamps that display in the messages. It can accept a string consisting of format tokens like "hh:mm:ss" and other formats supported by the Swift DateFormatter.

Share Menu Options

By default, the share menu displays options for the following file types:

• visual media files (images and videos)
• audio files
• general files like documents, PDFs, and spreadsheets
• location

The sharePopupConfiguration setting allows you to restrict the items that display in the share menu. The setting accepts an array of enum type ShareMenuItem, which has keys that are mapped to the share menu items: ShareMenuItem.photoAndVideoLibrary for the image/ video share item, ShareMenuItem.files for the share file item, and ShareMenuItem.camera for sharing an image directly from the camera. Only the menu items for the values passed in the array display in the share popup. An empty or invalid array results in all of the menu items displaying together in the share popup.

Speech Recognition

Setting the enableSpeechRecognition feature flag to true enables the microphone button to display in place of the send button whenever the user input field is empty. The speech is converted to text and sent to the skill or digital assistant. If the speech is partially recognized, then the partial result is displayed in a popup that's opened by clicking the microphone button.

Setting this property to true also supports the functionality enabled by the enableAutoSendSpeechResponse property, which when also set to true, enables the user's speech response to be sent to the chat server automatically while displaying the response as a sent message in the chat window. You can allow users to first edit (or delete) their dictated messages before they send them manually by setting enableSpeechRecognitionAutoSend to false.

Speech recognition is utilized through the following methods:

• public func startRecording()
• public func stopRecording()
• public func isRecording() -> Bool

public func startRecording()

Starts recording the user's voice message.

public func stopRecording()

Stops recording the user's message.

public func isRecording() -> Bool

Checks whether the voice recording has started or not. Returns true if the recording has started. Otherwise, it returns false.

The onSpeechResponseReceived(data: String, final: Bool) function from the BotsEventListener protocol can be used to handle all the responses from the speech server.

BotsManager.shared().startRecording()
if (BotsManager.shared().isRecording()) {
    BotsManager.shared().stopRecording() // Stop voice recording
}

Speech Synthesis

The SDK has been integrated with speech synthesis to read the skill's message aloud when a new message is received from skill.

• You enable this feature by setting the enableSpeechSynthesis feature flag to true.
• You can set the preferred language that read the skill's messages aloud with the speechSynthesisVoicePreferences property. This property enables a fallback when the device doesn't support the preferred language or voice. If the device does not support the preferred voice, then the default voice for the preferred language is used instead. When neither the preferred voice or language are supported, then the default voice and language are used.

public func speak(text: String)

Starts reading the skill's response aloud. Its text parameter is the text for the skill's message that's read aloud.

BotsManager.shared().speak(text: "What kind of crust do you want?")

public func stopSpeech()

Stops reading the skill's response aloud.

BotsManager.shared().stopSpeech()

Voice Visualizer

When voice support is enabled (botsConfiguration.enableSpeechRecognition = true), the footer of the chat widget displays a voice visualizer, a dynamic visualizer graph that indicates the frequency level of the voice input. The visualizer responds to the modulation of the user's voice by indicating whether the user is speaking too softly or too loudly. This visualizer is created using Swift's AVAudioEngine which is exposed in the onAudioReceived method in the SpeechEventListener protocol for use in headless mode.

The chat widget displays a voice visualizer when users click the voice icon. It's an indicator of whether the audio level is sufficiently high enough for the SDK to capture the user’s voice. The user’s message, as it is recognized as text, displays below the visualizer.

Note:

Voice mode is indicated when the keyboard icon appears.

When botsConfiguration.enableSpeechAutoSendSpeechResponse = true, the recognized text is automatically sent to the skill after the user has finished dictating the message. The mode then reverts to text input. When botsConfiguration.enableSpeechAutoSendSpeechResponse = false, the mode also reverts to text input, but in this case, users can modify the recognized text before sending the message to the skill.

Message Model

To use features like headless mode and delegate, you need to understand both user and skill messages. Everything that's received or sent from the Oracle Chat Server is represented as a message, one that's sent from the user to the skill, or from the skill to the user.

Base Types

These are the base types used in all messages sent from the user to the skill and vice versa. They are the building blocks of all messages.

• Attachment
• Location
• Action
• Card
• Conversation Message

Attachment

Represents an attachment that's sent by the user.

Name	Description	Type	Required?
type	The attachment type	string (valid values: audio, file, image, video)	Yes
url	The download URL for the attachment	string	Yes

For example:

{
    "type": "image",
    "url": "https://www.oracle.com/us/assets/hp07-oow17-promo-02-3737849.jpg"
}

Location

Represents a location object.

Name	Description	Type	Required?
title	The location title	string	No
URL	The URL for displaying the location on a map	string	No
latitude	The GPS coordinate's longitude value	double	Yes
longitude	The GPS coordinate's latitude value	double	Yes

For example:

{
    "title": "Oracle Headquarters",
    "url": "https://www.google.com.au/maps/place/37°31'47.3%22N+122°15'57.6%22W",
    "longitude": -122.265987,
    "latitude": 37.529818
}

Action

An action represents something that the user can select.

Name	Description	Type	Required?
type	The action type	string	Yes
label	The descriptive label text for the action.	string	At least one label or imageUrl must be present.
imageUrl	The image for the action	string	At least one label or imageUrl must be present.

PostbackAction

Sends a predefined postback back to the skill when the user selects an action.

Name	Description	Type	Required?
type	The action type	"postback"	Yes
postback	The postback that's returned when the user selects an action.	A string or JSON object	Yes

For example:

{
    "type": "postback",
    "label": "Large Pizza",
    "imageUrl": "https://example.com/images/gallery/locations/11.jpg",
    "postback": {
        "state": "askSize",
        "action": "getCrust"
    }
}

CallAction

Requests the client to call a specified phone number on behalf of the user.

Name	Description	Type	Required?
type	The action type	"call"	Yes
phoneNumber	The phone number to call	string	Yes

For example:

{
    "type": "call",
    "label": "Call Support",
    "imageUrl": "http://example.com.ar/files/2016/05/cuidado.jpg",
    "phoneNumber": "18005555555"
}

urlAction

Requests the client to open a website in a new tab or in an in-app browser.

Name	Description	Type	Required?
type	The action type	"call"	Yes
URL	The URL of the website that's displayed.	string	Yes

For example:

{
    "type": "url",
    "label": "Open URL",
    "imageUrl": "http://example.com.ar/files/2016/05/cuidado.jpg",
    "url": "https://example.com/images/gallery/locations/11.jpg",
}

LocationAction

Requests the client to ask for the user's location.

Name	Description	Type	Required?
type	The action type	"location"	Yes

For example:

{
    "type": "location",
    "label": "Share location",
    "imageUrl": "http://images.example.com/location-clipart-location-pin-clipart-1.jpg"
}

Card

Represents a single card in the message payload.

Name	Description	Type	Required?
title	The title of the card, displayed as the first line on the card.	string	Yes
description	The description of the card	string	No
imageUrl	The URL of the image that is displayed.	string	No
URL	The website URL that's opened by a tap.	string	No
actions	An array of actions related to the text	array	No

Conversation Message

All of the messages that are part of a conversation have the following structure:

Name	Description	Type	Required?
messagePayload	The message payload	Message	Yes
userId	The user ID	string	Yes

For example:

{
    "messagePayload": {
        "text": "show menu",
        "type": "text"
    },
    "userId": "guest"
}

User Message

Represents a message sent from the user to the skill.

User Text Message

The simple text message that's sent to the server.

Name	Description	Type	Required
type	The message type	"text"	Yes
text	The message text	string	Yes

For example:

{
    "messagePayload": {
        "text": "Order Pizza",
        "type": "text"
    },
    "userId": "guest"
}

User Postback Message

The postback response message that's sent to the server.

Name	Description	Type	Required
type	The message type	"postback"	Yes
text	The postback text	string	No
postback	The postback of the selected action	A string or JSON object	Yes

For example:

{
    "messagePayload": {
        "postback": {
            "variables": {
                "pizza": "Small"
            },
            "system.botId": "69BBBBB-35BB-4BB-82BB-BBBB88B21",
            "system.state": "orderPizza"
        },
        "text": "Small",
        "type": "postback"
    },
    "userId": "guest"
}

User Attachment Message

The attachment response message that's sent to the server.

Name	Description	Type	Required
type	The message type	"attachment"	Yes
attachment	The attachment metadata	Attachment	Yes

For example:

{
    "messagePayload": {
        "attachment": {
            "type": "image",
            "url": "http://oda-instance.com/attachment/v1/attachments/d43fd051-02cf-4c62-a422-313979eb9d55"
        },
        "type": "attachment"
    },
    "userId": "guest"
}

User Location Message

The location response message that's sent to the server.

Name	Description	Type	Required
type	The message type	"location"	Yes
location	The user location information	Location	Yes

For example:

{
    "messagePayload": {
        "location": {
            "latitude": 45.9285271,
            "longitude": 132.6101925
        },
        "type": "location"
    },
    "userId": "guest"
}

Bot Message

Represents the message sent from the skill to the user.

Message

Message is an abstract base type for all other messages. All messages extend it to provide some information.

Name	Description	Type	Required?
type	The message type	string	Yes

Bot Text Message

Represents a text message.

Name	Description	Type	Required
type	The message type	"text"	Yes
text	The message text	string	Yes
headerText	The header text for cards	string	No
footerText	The footer text for cards	string	No
actions	An array of actions related to the text.	Array	No
globalActions	An array of global actions related to the text.	Array	No

For example:

{
    "messagePayload": {
        "type": "text",
        "text": "What do you want to do?",
        "actions": [
            {
                "type": "postback",
                "label": "Order Pizza",
                "postback": {
                    "state": "askAction",
                    "action": "orderPizza"
                }
            },
            {
                "type": "postback",
                "label": "Cancel A Previous Order",
                "postback": {
                    "state": "askAction",
                    "action": "cancelOrder"
                }
            }
        ]
    },
    "userId": "guest"
}

Bot Attachment Message

Represents an attachment message.

Name	Description	Type	Required
type	The message type	"attachment"	Yes
attachment	The attachment sent	Attachment	Yes
headerText	The card's header text	string	No
footerText	the card's footer text	string	No
actions	An array of actions related to the text.	Array	No
globalActions	An array of global actions related to the text.	Array	No

Bot Card Message

Represents a set of choices that are displayed for the user, either horizontally as carousels or vertically as lists.

Name	Description	Type	Required
type	The message type	"card"	Yes
layout	Whether to display the messages horizontally or vertically.	string (values: horizontal, vertical)	Yes
cards	An array of cards to be rendered.	Array	Yes
headerText	The cards' header text	string	No
actions	An array of actions related to the text.	Array	No
globalActions	An array of global actions related to the text.	Array	No

{
    "messagePayload": {
        "type": "card",
        "layout": "horizontal",
        "cards": [
            {
                "title": "Hawaiian Pizza",
                "description": "Ham and pineapple on thin crust",
                "actions": [
                    {
                        "type": "postback",
                        "label": "Order Small",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "hawaiian",
                                "pizzaCrust": "thin",
                                "pizzaSize": "small"
                            }
                        }
                    },
                    {
                        "type": "postback",
                        "label": "Order Large",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "hawaiian",
                                "pizzaCrust": "thin",
                                "pizzaSize": "large"
                            }
                        }
                    }
                ]
            },
            {
                "title": "Cheese Pizza",
                "description": "Cheese pizza (i.e. pizza with NO toppings) on thick crust",
                "actions": [
                    {
                        "type": "postback",
                        "label": "Order Small",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "cheese",
                                "pizzaCrust": "thick",
                                "pizzaSize": "small"
                            }
                        }
                    },
                    {
                        "type": "postback",
                        "label": "Order Large",
                        "postback": {
                            "state": "GetOrder",
                            "variables": {
                                "pizzaType": "cheese",
                                "pizzaCrust": "thick",
                                "pizzaSize": "large"
                            }
                        }
                    }
                ]
            }
        ],
        "globalActions": [
            {
                "type": "call",
                "label": "Call for Help",
                "phoneNumber": "123456789"
            }
        ]
    },
    "userId": "guest"
}

Bot Raw Message

Used when a component creates the channel-specific payload itself.

Name	Description	Type	Required?
type	The message type	"raw"	Yes
payload	The channel-specific payload	A JSON object	Yes

Oracle iOS Channel Extensions

For Oracle iOS channels, you can extend the functionality of System.CommonResponse components with capabilities that are specific to the Oracle iOS SDK.

You access the extensions by using the channelCustomProperties element in the System.CommonResponse component and setting the appropriate properties. The code has the following format:

...
            channelCustomProperties:
            - channel: "iossdk"
              properties:
                PROPERTY_NAME: "PROPERTY_VALUE"
...

Here are the available custom properties for Oracle iOS channels:

Name	Allowed Values	Applies To...	Description
mediaType	A valid media type	Response items with the following attributes: type: "attachment" attachmentType: "file"or attachmentType: "image" Cards with imageUrl specified	The media type of the attachment. For example, image/jpeg. If not specified, the media type will be resolved from the attachment URL.

For more general information on channelCustomProperties, see Channel-Specific Extensions.