Overview of Siebel CRM Event Publication and Subscription

An event is a change of state. For example, ‘Account Status’ changing from ‘Unverified’ to ‘Verified’, new customer creation; existing customer updates; new items in cart or, for that matter, any other change that is important for business.

The Siebel CRM Event Publication and Subscription feature enables you to send event information from Siebel to Kafka and receive event information into Siebel CRM from Kafka. It supports high throughput message exchange between Siebel and Kafka in both synchronous and asynchronous manners.

As a direct benefit, it allows for the decoupling of direct interaction between Siebel and integrated applications, which means, event producers applications (for example Siebel CRM) and consumer applications (for example, payment processing system) can work independently of each other and focus on their domains of operation.

Dedicated Application Interfaces (AIs) can be used to communicate between Siebel Object Managers and Kafka server using Consumer and Producer Java APIs of Kafka Client. These AIs are referred as sidecar AIs throughout this document.

These Sidecar AIs are standard Siebel CRM Application Interfaces that have additional configuration for Siebel CRM Kafka integration for the Siebel CRM Event Publication and Subscription feature. Siebel CRM Event Publication and Subscription functionality therefore works only for those components that are accessible from Application Interface. For example: session-based or interactive components and not of type batch.

The word Sidecar in Sidecar AI refers to the usage of sidecar design pattern, in that the functionalities included in those AIs that are used as sidecars enhance the Siebel core platform functionalities and aid in high throughout event exchange between Siebel CRM and Kafka with easy upscaling as may be required. Therefore, the reference of sidecar pods in a Kubernetes deployment (for example obtained by using Siebel Cloud Manager on OCI OKE) do not apply here and these sidecar AIs are to be deployed as separate containers or pods in the cloud, like traditional AIs in a Kubernetes deployment of Siebel CRM. The installation and configuration process for a sidecar AI therefore is same as that of traditional Siebel AIs, along with additional configurations relevant for Siebel Kafka integration.

Oracle provides Kafka Client as part of Siebel installation. Kafka server (also referred to as broker in this document) needs to be installed, configured and managed by the user. Kafka Client and Server(broker) versions can be independently upgraded (Refer to KIP-97). It is recommended to independently upgrade Kafka servers immediately upon discovery of any critical vulnerability.

The Siebel CRM Kafka integration is implemented over the Java Client APIs of the Kafka distribution from Apache foundation. The producer and consumer Java APIs are implemented within Siebel CRM AIs. If any other Kafka distribution has support for the Java producer and consumer APIs that are from the main Kafka project, this integration may work with Kafka brokers/servers of such distributions. However, since Oracle will not know the implementation details, support from Oracle will be limited in such cases.

Business Service and Class

An important component of Kafka integration in Siebel CRM Event Publication and Subscription is a business service Event Handler. It is based on C++ Class CSSEAIEventHandler. One of its methods, SendEvent, does the job of posting Siebel events to Kafka. For SendEvent method, input is Siebel Property Set data with target Kafka Topic name at the beginning. It posts JSON to Kafka topic.

SendEvent Method’s Input Sample (a simplified Siebel Property Set equivalent representation)

{
    p["topic"] = "CreateContactEvent";
    p["partitions"] = "0,1,3";
    p["BusinessObjName"] = "Contact";
    {
        p["BusinessCompName"] = "Contact";
        {
            p["First Name"] = "John";
            p["Last Name"] = "Doe";
            p["Contact Id"] = "88-37WGM9";
        }
    }
    {
        p["BusinessCompName"] = "Account";
        {
            {
                p["City"] = "Menlo Park";
                p["CSN"] = "10-103";
                p["Location"] = "HQ-Corporate";
                p["Name"] = "A. K. Parker Inc.";
            }
        }
    }
}

SendEvent Method’s Output Sample JSON

{
  "Contact": {
    "Fields": {
      "Contact Id": "88-37WGM9",
      "First Name": "John",
      "Last Name": "Doe"
    },
    "BusinessCompName": "Contact"
  },
  "Account": {
    "Field List": {
      "Fields": {
        "Name": "A. K. Parker Inc.",
        "Location": "HQ-Corporate",
        "CSN": "10-103",
        "City": "Menlo Park"
      }
    },
    "BusinessCompName": "Account"
  },
  "BusinessObjName": "Contact"
}

The Event Publication and Subscription feature requires the use of business services based on classes derived from CSSEAIEventHandler.

From the Siebel 24.12 monthly update release, Siebel CRM Kafka integration through Siebel CRM Event Publication and Subscription supports an optional new payload structure to post messages to Kafka using the Run-Time Events (RTE) mechanism.

You can control the payload structure (old/new) to post messages to Apache Kafka through RTE mechanism using an optional environment variable SIEBEL_EVENT_PUBSUB_PAYLOAD_STRUCT. This variable can take one of the following values:

This is the default value and specifies that the old payload structure will be used to post messages to Kafka using the RTE mechanism. If the value of SIEBEL_EVENT_PUBSUB_PAYLOAD_STRUCT is set to an invalid value or left empty, it will default to 1.

Sample use case and default (old) payload:

Use case
- Primary Business Component: Contact
- Child Business Component Data: One Account

Payload:

{
  "Contact": {
    "Fields": {
        "Contact Id": "101",
        "First Name": "PayLoad",
        "Last Name": "EPSSiebel",
    },
    "BusinessCompName": "Contact"
  },
  "Account": {
    "Field List": {
      "Fields": {
            "Name": "A. K. Parker Inc.",
            "Location": "HQ-Corporate",
            "CSN": "10-103",
            "City": "Menlo Park"
           }
    },
    "BusinessCompName": "Account"
  },
  "BusinessObjName": "Contact"
}

Specifies that the new payload structure will be used to post messages to Kafka using RTE. This new message structure is optional and in this format Siebel CRM will:
- Not send the business component name (BusinessCompName) and business object name (BusinessObjName) in the payload structure.
- Post primary business component as JSON object.
- Determine how a single data record of a child business component is posted using the value of the SingleRecordAsJSONObject property. For more information, see Configuration Files forPayloadGeneration.
Sample use case and (new optional ) payload:
- Use case
  - Primary Business Component: Contact
  - Child Business Component Data: One Account
- Payload
```
{
  "Contact": {
    "Contact Id": "101",
    "First Name": "PayLoad",
    "Last Name": "EPSSiebel",
    "Account": [
      {
        "Name": "A. K. Parker Inc.",
        "Location": "HQ-Corporate",
        "CSN": "10-103",
        "City": "Menlo Park"      
      }
    ]
  }
}
```

Note: The old JSON payload structure, is the default and will continue to work but support for the same will be dropped in the future releases. Hence, it is recommended that, you transition to the new payload structure at the earliest.