Managing Service Connectors

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.

2. Choose a compartment you have permission to work in (on the left side of the page). The page updates to display only the resources in that compartment. If you're not sure which compartment to use, contact an administrator.

3. Click Create Service Connector.

4. On the Create Service Connector page, fill in the settings:

   • Connector Name: User-friendly name for the new service connector. Avoid entering confidential information.

   • Description: Optional identifier.
   • Resource Compartment: The compartment where you want to store the new service connector.
   • Configure Service Connector:
     • Source: Select the service containing the data you want to transfer from the following options.
       • Logging: Transfer log data from the Logging service. See Logging Overview.
       • Streaming: Transfer stream data from the Streaming service. See Streaming Service Overview.
     • Target: Select the service that you want to transfer the data to.
       • Functions: Send data to a function.
       • Logging Analytics: Send data to a log group.
       • Monitoring: Send metric data points to the Monitoring service.
       • Notifications: Send data to a topic.
       • Object Storage: Send data to a bucket.
       • Streaming: Send data to a stream.

     • Configure your source and task:

       Note
       
       

       By default, this page uses Basic mode. To toggle between Basic Mode and Advanced Mode, click Switch to Advanced mode (to the right of Configure source connection) or Switch to Basic mode(to the right of Configure source and task). Complex queries cannot be displayed in Basic mode. See I can't view my query in Basic mode and Logging Query Language Specification.

       • Configure source connection:

         Source service	Fields
         Logging See Logging Overview.	Compartment Log Group Logs
         Streaming See Streaming Service Overview. Note This source supports the following targets: Functions, Notifications, Object Storage, and Streaming.	Compartment Stream Pool Stream Show Advanced Options: Read Position: Specify the cursor position from which to start reading the stream. For more information, see Using Cursors. Latest: Starts reading at messages published after saving the service connector. Trim_Horizon: Starts reading at the oldest available message in the stream. Note For stream input schema, see Message Reference. Notifications target with Streaming source: All messages are sent as raw JSON blobs.

       • Configure task:

         Task	Fields
         Log Filter Task (filters source logs using the Logging service)	Audit Logs: When _Audit is selected for Log Group: When Attribute is selected for Filter Type: Filter Type: Attribute Attribute Name Attribute Values When Event type is selected for Filter Type: Filter Type: Event type Service Name Event Type Service Logs, Custom Logs: When another log group (not _Audit) is selected for Log Group: Property Operator Value
         Configure Function Task (processes data from the source using the Functions service) Note This task supports the following targets: Functions, Notifications, Object Storage, and Streaming.	Function Application: Select the name of the function application that includes the function you want. Function: Select the name of the function you want to use to process the data received from the source. Show Advanced Options: Optimal batch size: Specify limits for each batch of data sent to the function. Use Automatic Settings Use Manual Settings Batch size limit (KBs) Batch time limit (seconds) Note Service Connector Hub does not parse the output of the function task. The output of the function task is written as-is to the target. For example, when using a Notifications target with a function task, all messages are sent as raw JSON blobs. Functions are invoked synchronously with 6MB of data per invocation. If data exceeds 6MB, then the service connector invokes the function again to move the over-limit data. Such over-limit invocations are handled sequentially. Functions can execute for up to five minutes.

       • Click Advanced Mode or Switch to Advanced Mode to view and edit the source connector and task using the Query Code Editor.

   • Configure target connection: Select the Service Compartment (where the target service resides) and fill in additional fields as needed:

     Target service	Additional fields
     Functions	Function Application: Select the name of the function application that includes the function you want. Function: Select the name of the function you want to send the data to. Note Functions are invoked synchronously with 6MB of data per invocation. If data exceeds 6MB, then the service connector invokes the function again to move the over-limit data. Such over-limit invocations are handled sequentially. Functions can execute for up to five minutes. Do not return data from Functions targets to service connectors. Service Connector Hub does not read data returned from Functions targets.
     Logging Analytics	Log Group: Select the log group you want.
     Monitoring	Note Do not use the reserved oci_ prefix for new metric namespaces and names. Metrics are not ingested when reserved prefixes are used. See Publishing Custom Metrics and PostMetricData Reference (API). When typing a new metric namespace or name, press ENTER to submit it. Namespace: Select the metric namespace that includes the metric you want. It can be an existing or new namespace. Metric Name: Select the name of the metric that you want to send the data to. It can be an existing or new metric. What's included with your metric The following dimensions are included with your metric: connectorId The OCID of the service connector that the metrics apply to. connectorName The name of the service connector that the metrics apply to. connectorSourceType The source service that the metrics apply to. The timestamp of each metric data point is the timestamp of the corresponding log message.
     Notifications	Note Log Group for Notifications is limited to _Audit. Topic: Select the name of the topic that you want to send the data to. Note SMS messages exhibit unexpected results for certain service connector configurations. This issue is limited to topics that contain SMS subscriptions for the indicated service connector configurations. For more information, see Multiple SMS Messages for a Single Notification. Message Format: Select the option you want: Note Message Format is available for service connectors with Logging source only. Not available for service connectors with function tasks. When this option is not available, messages are sent as raw JSON blobs. Send formatted messages: Simplified, user-friendly layout. Note To view supported subscription protocols and message types for formatted messages, see Friendly Formatting. Send raw messages: Raw JSON blob.
     Object Storage	Bucket: Select the name of the bucket that you want to send the data to. Note Batch rollover details: Batch rollover size: 100 MB Batch rollover time: 7 minutes Files saved to Object Storage are compressed using gzip.
     Streaming	Stream: Select the name of the stream that you want to send the data to.

   • Show Advanced Options: If you have permissions to create a resource, then you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure whether to apply tags, skip this option (you can apply tags later) or ask your administrator.

   Default policies are offered for any authorization required for this service connector to access source, task, and target services.

   You can get this authorization through these default policies or through group-based policies. The default policies are offered whenever you use the Console to create or edit a service connector. The only exception is when the exact policy already exists in IAM, in which case the default policy is not offered. For more information about this authorization requirement, see Authentication and Authorization.

5. To accept default policies, click the Create link provided for each default policy.

   Note
   
   If you don't have permissions to accept default policies, contact your administrator.

   View links are provided for you to optionally review the newly created policies.

6. Click Create to create the service connector.

   Within a few minutes, the service connector begins moving data according to its configuration. The service connector applies tasks to data from the source service and then moves the data to the target service.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to edit.
4. Click Edit.

5. Make your changes.

   Note
   
   If you did not previously create the default access policy to allow this service connector to write to the target service, you can do so now. You can get this authorization through these default policies or through group-based policies. The default policies are offered whenever you use the Console to create or edit a service connector. The only exception is when the exact policy already exists in IAM, in which case the default policy is not offered. For more information about this authorization requirement, see Authentication and Authorization.

6. Click Save Changes.

   If you updated the source service or tasks, then data movement may pause for a few minutes, as indicated by Data Freshness metrics. Within a few minutes, the service connector begins moving data according to its configuration. The service connector applies tasks to data from the source service and then moves the data to the target service.

Friendly message formats are available with service connectors that use Notifications as target.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to edit.
4. Click Edit.

5. Under Configure target connection, select the Message Format you want:

   • Send formatted messages: Simplified, user-friendly layout.
     Note
     
     To view supported subscription protocols and message types for formatted messages, see Friendly Formatting.
   • Send raw messages: Raw JSON blob.
   Note
   
   If you did not previously create the default access policy to allow this service connector to write to the target service, you can do so now. You can get this authorization through these default policies or through group-based policies. The default policies are offered whenever you use the Console to create or edit a service connector. The only exception is when the exact policy already exists in IAM, in which case the default policy is not offered. For more information about this authorization requirement, see Authentication and Authorization.

6. Click Save Changes.

   If you updated the source service or tasks, then data movement may pause for a few minutes, as indicated by Data Freshness metrics. Within a few minutes, the service connector begins moving data according to its configuration. The service connector applies tasks to data from the source service and then moves the data to the target service.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to activate.

4. Click Activate and then confirm.

   The service connector immediately begins moving data according to its configuration, applying tasks to data in the source service and then moving the data to the target service.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to deactivate.

4. Click Deactivate and then confirm.

   The service connector stops moving data.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to edit.
4. Click Move Resource.
5. Choose the destination compartment from the list.
6. Click Move Resource.

1. Open the navigation menu and click Analytics & AI. Under Messaging, click Service Connector Hub.
2. Choose the Compartment containing the service connector.
3. Click the name of the service connector you want to deactivate.

4. Click Delete and then confirm.

   The service connector stops moving data.

Open a command prompt and run oci sch service-connector list to list service connectors in the specified compartment:

oci sch service-connector list --compartment-id <compartment_OCID>

Open a command prompt and run oci sch service-connector get to get the specified service connector:

oci sch service-connector get --service-connector-id <service_connector_OCID>

Open a command prompt and run oci sch service-connector create to create a service connector:

oci sch service-connector create --display-name
"<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]

Open a command prompt and run oci sch service-connector update to edit a service connector:

oci sch service-connector update --service-connector-id <service_connector_OCID> --display-name
"<display_name>" --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]

Open a command prompt and run oci sch service-connector activate to activate the specified service connector:

oci sch service-connector activate --service-connector-id <service_connector_OCID>

Open a command prompt and run oci sch service-connector deactivate to deactivate the specified service connector:

oci sch service-connector deactivate --service-connector-id <service_connector_OCID>

Open a command prompt and run oci sch service-connector change-compartment to move the service connector to the specified compartment:

oci sch service-connector change-compartment --service-connector-id <service_connector_OCID> --compartment-id <destination_compartment_OCID>

Open a command prompt and run oci sch service-connector delete to delete the specified service connector:

oci sch service-connector delete --service-connector-id <service_connector_OCID>