Consume Messages from an Apache Kafka Topic

You can configure a scheduled orchestrated integration to use the Apache Kafka Adapter to consume messages from an Apache Kafka topic at specific intervals and invoke a child integration (for example, an Oracle Database-based integration for insertion of records into a database table) to process the consumed messages.

The following integration provides one example of how to implement this pattern:

• A scheduled orchestrated integration that runs every ten minutes.
• A mapper to perform appropriate source-to-target mappings between the schedule and an Apache Kafka Adapter.
• An Apache Kafka Adapter configured to:
  • Consume records from a Kafka topic.
  • Specify the consumer group to attach. Kafka assigns the partitions of a topic to the consumers in a group.
  • Specify Read latest as the option for consuming messages. The latest messages are read starting at the time at which the integration is activated.
  • Specify the message structure to use (for this example, an XML schema (XSD) document) and the headers to use for the message.
• A mapper for performing appropriate source-to-target mappings between the Apache Kafka Adapter and a SOAP Adapter.
• A SOAP Adapter invoked to insert the Kafka topic record into an Oracle Database table.