5.8.1 Kafka Configuration Fields

Table 5-7 Kafka fields

Field	Description	Data Input Notes
Remote Server Name*	Unique identifier used to label a Remote Server.	Format: Text box Default: NA Range: A 32 character string. Valid characters are alphanumeric, minus sign, period, and underscore. Must start with an alphanumeric or an underscore and end with an alphanumeric. A value is required.
Broker IPs*	IP and port of Kafka cluster.	Format: Text box Default: NA Range: Any valid IPv4 address with port. A value is required.
Topic Name *	Defines Kafka's topic name which is a fundamental unit for event or message organization. Messages with the same topic name will be appended one after another creating a Log file. Producers can push messages into the tail of these logs while consumers pull messages off from the head. It is possible to get logical segregation between messages and events, similar to how various tables in a database can contain different sorts of data. Note: Topic name is default and cannot be modified.	Format: Text box Default: com.oracle.dsr.idih.ttr Range: Topic name can include the following characters: a-z, A-Z, 0-9, . (dot), _ (underscore), and - (dash)] [A value is required.]
Compression Codec	To be used for compressing message sets.	Format: Drop-down Default: Gzip Range: None, Gzip, Snappy, Lz4
Compression Level	Compression level parameter for algorithm selected by configuration property compression.codec. Higher values will result in better compression at the cost of more CPU usage.	Format: Text box Default: 0 Range: [0-9] for gzip, [0-12] for lz4, only 0 for snappy.]
Batch Size	The producer will attempt to batch records together into fewer requests whenever multiple records are being sent to the same partition.	Format: Text box Default: 16384 Range: 0-16384
Max Request Size	The maximum size of a request in bytes. This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.	Format: Text box Default: 1048576 Range: 0-1048576
Linger Time	Delay in milliseconds to wait for messages in the producer queue to accumulate before constructing message batches (MessageSets) to transmit to brokers.	Format: Text box Default: 0 Range: 0
Acknowledgements	Number of replicas to be written successfully before acknowledging the publish request.	Format: Text box Default: 1 Range: -1 or all, 0, 1
Max Idle Time	Closes idle connections after the number of milliseconds specified by this configuration.	Format: Text box Default: 540000 Range: 0-3600000
Socket Connection Timeout	The amount of time the client will wait for the socket connection to be established. If the connection is not built before the timeout elapses, clients will close the socket channel.	Format: Text box Default: 10000 Range: 0-10000
Send Buffer Size	The size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.	Format: Text box Default: 131072 Range: 0-131072
Max In Flight Request	The maximum number of unacknowledged requests the client will send on a single connection before blocking.	Format: Text box Default: 5 Range: 0 - 5
Max Request Timeout	This controls the maximum amount of time (in milliseconds) the client will wait for the response of a request. If the response is not received before the timeout elapses, the client will resend the request if necessary or fail the request if retries are exhausted.	Format: Text box Default: 30000 Range: 0-30000
Max Retries	Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error.	Format: Text box Default: 2147483647 Range: 0-2147483647
Max Delivery Timeout	An upper bound on how long it takes to communicate success or failure after calling send() returns. This limits the total time that a record will be delayed prior to sending the report, the time to await acknowledgement from the broker (if expected), and the time allowed for retriable send failure.	Format: Text box Default: 120000 Range: 0-120000
Metadata Max Age	The period of time in milliseconds after which we force a refresh of metadata even if we haven’t seen any partition leadership changes to proactively discover any new brokers or partitions.	Format: Text box Default: 300000 Range: 0-300000
Reconnect Backoff Max Time	The maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. If provided, the backoff per host will increase exponentially for each consecutive connection failure, up to this maximum time. After calculating the backoff increase, 20% random jitter is added to avoid connection storms.	Format: Text box Default: 1000 Range: 0-1000
Reconnect Backoff Time	The base amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop. This backoff applies to all connection attempts by the client to a broker.	Format: Text box Default: 50 Range: 0-50
Retry Backoff Time	The amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.	Format: Text box Default: 100 Range: 0-100
Max Connection Retry	Time in minutes within which the producer instance makes maximum number of retries to fetch the metadata from the kafka server.	Format: Text box Default: 10 Range: 2-10 minutes
Security Protocol	Protocol used to communicate with brokers.	Format: Text box Default: SSL Range: PLAINTEXT, SSL

Table 5-8 SSL Properties

Field	Description	Data Input Notes
SSL Certificate	File name of file in PEM format containing the client certificate.	Format: Browse to select the file.
SSL Key	File name of file in PEM format containing the client private key in encrypted format.	Format: Browse to select the file.
SSL CA	File name of CA in PEM format to be used for certificate verification	Format: Browse to select the file.
SASL Username	Enter the SASL (Simple Authentication and Security Layer) user name that will be used for authenticating your Kafka client.	Format: Text box Default: NA Range: The length of the user name should be between 8 and 64 characters. The user name accepts only alphanumeric characters.
SASL Password	Enter the SASL (Simple Authentication and Security Layer) password that will be used for authenticating your Kafka client.	Format: Text box Default: NA Range: The length of the user name should be between 8 and 64 characters. The user name accepts only alphanumeric characters