Avro Serialization in Siebel CRM Kafka Integration

This topic explains how to configure Avro serialization for Siebel CRM Kafka integration to optimize message exchange between the Kafka broker and client. It includes the following sections:

• Creating Avro Schema Files

• Configuring Schema Mapping in aieventconfig.txt for Avro serialization

• Avro Schema Mapping Rules in Siebel CRM Event Publication and Subscription

You can optimize message exchanges in Siebel CRM Kafka integration by using Avro serialization.

Avro is an open-source data serialization framework developed as part of the Apache Hadoop project and is widely used. It is the most common serialization framework used with Apache Kafka.

Avro relies on schemas. Avro schemas used during producing message and reading message from Kafka must be compatible. Avro also supports schema evolution, making it flexible as data requirements change. At present, Siebel CRM Event Pub/Sub does not support schema registry from Siebel CRM.

Using Avro serialization with Siebel CRM Kafka integration offers the following benefits:

• Compact size: Avro serialization produces smaller message payloads, which helps reduce storage and data transfer costs.
• Fast processing: Minimal overhead enables efficient, high-speed data processing.
• Schema consistency: Avro enforces schema consistency, which is ideal for systems that require data format standardization.
• Interoperability: Avro makes it easy for different producers and consumers to exchange data seamlessly.

Siebel CRM Kafka integration uses Avro version 1.9.2.

To implement Avro serialization in Siebel CRM Kafka integration, you must:

1. Create the Avro schema files. For more information, see Creating Avro Schema Files.
2. Configure the schemamapping section in the aieventconfig.txt file. For more information, see Configuring Schema Mapping in aieventconfig.txt for Avro serialization.