Topic & Partition Page
Select the operation and topic on which to perform the operation and optionally specify the message structure.
Configure the Apache Kafka Adapter as an Invoke Connection
Element | Description |
---|---|
Select a Topic |
Select the topic on which to perform the operation. You can also enter the beginning letters of the topic to filter the display of topics. A topic is a category in which applications can add, process, and reprocess messages. You subscribe to messages in topics. |
Specify the Partition (This field is only displayed if you select Publish records to a Kafka topic or Consume records from a Kafka topic.) |
Specify the partition to which to push the selected topic. Kafka topics are divided into partitions that enable you to split data across multiple brokers. If you do not select a specific partition and use the Default selection, Kafka considers all available partitions and decides which one to use. |
Consumer Group (This field is only displayed if you select Consume records from a Kafka topic.) |
Specify the consumer group to attach.
Consumers join a group by using the same group ID. Kafka assigns the partitions of a topic to the consumers in a group. |
Specify the option for consuming
messages (This field is only displayed if you select Consume records from a Kafka topic.) |
Once
configured, if you edit it again, the following options are
displayed:
Note: When you select the Read from beginning or Read latest option, not all messages are guaranteed to be consumed in a single run. However, any remaining messages are consumed in subsequent runs. |
Maximum Number of Records to be
fetched (This field is only displayed if you select Consume records from a Kafka topic or Consume records from a Kafka topic by specifying offset.) |
Specify the number of messages to read. The threshold
for the complete message payload is 10 MB.
Note: This field specifies the upper boundary of records to fetch. It does not guarantee the specified amount of records to retrieve from the stream in a single run. Remaining messages are fetched in subsequent runs. |
Do you want to specify the message structure? |
Select Yes if you want to define the message structure to use on the Message Structure page of the wizard. Otherwise, select No. |
Do you want to specify the headers for the message? |
Select Yes if you want to define the message headers to use on the Headers page of the wizard. Otherwise, select No. |
Review and update advanced configurations | Click Edit to open the
Advanced Options section to enable or disable the transactional
producer.
|
Configure the Apache Kafka Adapter as a Trigger Connection
Element | Description |
---|---|
Select a Topic |
Select the topic on which to perform the operation. You can also enter the beginning letters of the topic to filter the display of topics. A topic is a category in which applications can add, process, and reprocess messages. You subscribe to messages in topics. |
Specify the Partition |
Specify the partition to which to push the selected topic. Kafka topics are divided into partitions that enable you to split data across multiple brokers. If you do not select a specific partition and use the Default selection, Kafka considers all available partitions and decides which one to use. |
Consumer Group | Specify the consumer group to attach.
Consumers join a group by using the same group ID. Kafka assigns the partitions of a topic to the consumers in a group. |
Polling Frequency (Sec) | Specify the frequency at which to fetch records. |
Maximum Number of Records to be fetched | Specify the number of messages to read. The threshold
for the complete message payload is 10 MB.
Note: This field specifies the upper boundary of records to fetch. It does not guarantee the specified amount of records to retrieve from the stream in a single run. Remaining messages are fetched in subsequent runs. |
Do you want to specify the message structure? |
Select Yes if you want to define the message structure to use on the Message Structure page of the wizard. Otherwise, select No. |
Do you want to specify the headers for the message? |
Select Yes if you want to define the message headers to use on the Headers page of the wizard. Otherwise, select No. |
Review and update advanced configurations | Click Edit to open the
Advanced Options section to enable or disable the transactional
producer.
|