6 Kafka & Communication Management

This chapter outlines the administrative, security, and operational procedures required to manage Kafka infrastructure and external communication within OCNADD, ensuring reliable data flow, controlled access, and secure service interactions.

6.1 Kafka Cluster Management Procedures

This section outlines the operational steps required to maintain Kafka clusters used by OCNADD.

To perform the following operations, see "Kafka Cluster Management Procedures" section in the Oracle Communications Network Analytics Data Director User Guide.

  • Kafka topic creation
  • Kafka cluster capacity expansion
    • Adding a broker to an existing Kafka cluster
    • Adding a partition to an existing topic
    • Partition reassignment in Kafka cluster
  • Kafka cluster external access
    • External access with OCCNE LBVM
    • External access with OCCNE CNLB
  • Enabling Kafka log retention policy
  • Expanding Kafka storage
  • Enabling RAM storage in Kafka cluster
  • Disabling RAM storage in Kafka cluster

Note:

For each worker group, source topics (inbound Diameter data from Diameter applications to the Data Director), such as vcollector, dsr, and pcf, are created and managed in the Relay Agent’s Kafka cluster. In contrast, destination topics (outbound Diameter data from the Data Director to third-party applications), such as diameter and <xdr>-correlated, are created and managed in the Mediation Group’s Kafka cluster.

6.2 Enable External Communication Between OCNADD Gateways

Prerequisites

  • mTLS should be enabled
  • External IPs must be used to create the certificates. There will not be any dynamic IP addresses for gateway external communication; users need to provide static IPs and configure the certificates with these IPs.

To perform the following operations, see "Enable External Communication Between OCNADD Gateways" section in the Oracle Communications Network Analytics Data Director User Guide.

  • OCNADD Gateway External Access in OCCNE LBVM
  • OCNADD Management Gateway External Access
  • OCNADD Mediation Gateway External Access
  • OCNADD Relay Agent Gateway External Access
  • OCNADD Gateway External Access in OCCNE CNLB

6.3 Update Certificate of The Existing Services

To update OCNADD service certificates, see "Update Certificate of the Existing Services" section in the Oracle Communications Network Analytics Data Director User Guide.

6.4 Enable Kafka Feed Configuration Support

This section lists the prerequisites for the Diameter Node or vCollector to communicate with the Data Director Relay Agent Kafka cluster, and for third-party consumer applications to communicate with the Data Director Mediation Kafka cluster securely. The section also lists the configuration settings that need to be done on the Kafka broker.

There are certain preconditions that must be met before the Kafka feed for external consumer applications can work correctly. Some of these settings may disrupt communication with producer clients, especially if any client ACL rule is configured in Kafka. In that case, Kafka will authenticate and authorize each and every client, and existing clients will be disrupted if they are not already using SASL_SSL or SSL (mTLS) connections and recommendations from the Oracle Communications Network Analytics Data Director Security Guide.

Note:

The procedure mentioned below should be executed on the corresponding Relay Agent and Mediation Group on which the Kafka feed configuration support is being enabled.

To perform the following operations, see "Enable Kafka Feed Configuration Support" section in the Oracle Communications Network Analytics Data Director User Guide.

  • Prerequisites for Diameter producer (The steps for NF Producer also apply to Diameter producers like vCollector and DSR, etc.)
  • Prerequisites for External Consumers
  • Update OCNADD Configuration
  • Update JAAS Configuration with Users
  • Update SCRAM Configuration with Users
  • Create Client ACLs
  • Delete Generic Producer Client ACLs

6.5 Disable Kafka Feed Configuration Support

The section defines the procedure that should be executed when external Kafka feeds are no longer used in the Data Director deployment.

External Kafka feeds require TLS and access control in the Kafka server; if external Kafka feed support is not required, then access control in Kafka can be disabled.

The steps in this procedure should only be executed on the Mediation Group in which Kafka feed support is required to be disabled.

Note:

  • In the case of a rollback to a release where Kafka feed support was not present, it is mandatory to delete the producer client ACLs and Kafka feeds before the rollback is initiated. Follow steps 1 and 3 for deleting the feeds and ACLs.
  • In the case of a rollback to a revision where Kafka feeds were supported and configured, there is no need to delete Kafka feeds and producer client ACLs.
  • If it is not possible to delete the ACLs and feeds before the rollback, contact Oracle Support using MOS.

To disable Kafka Feed, see the steps mentioned in the section "Disable Kafka Feed Configuration Support" section in the Oracle Communications Network Analytics Data Director User Guide.

6.6 Configuring "Host" based ACLs for Kafka Feed

The Kafka Feed supports optional "host"-based ACLs for the external consumer application. This allows an external application to connect from a specific client machine with a specific IP address. The client application can be running inside a pod in a Kubernetes cluster where OCNADD is deployed, or in a different cluster. Since pods do not have static IP addresses, "host"-based ACLs are optional for Kafka feeds. The client machine hosting the external Kafka application can also be a separate virtual machine in the customer cloud environment; in this case, a static IP address can be given to the client VM running the external Kafka consumer application.

The Kafka Feed configuration has a "hostname" field, which is optional and currently supports only a single IP address. The default behavior of the Kafka feed is to allow all hosts. This default behavior applies when the user leaves the Host Name field blank or provides the wildcard character *.

The Host Name field can be either of the following:

  • IPv4 address of the host where the consumer application is running
  • Blank or wildcard character * (this allows all host IPs)

Note:

  • Pod/VM hostname-based ACLs are not yet supported in Kafka
  • IPv6 is not supported
  • A specific host IP ACL is recommended when a static IP is used for the client machine
  • The host IP should not be configured for cloud-native client applications running in a K8s cluster, since pods have dynamic IP assignment

To perform the following operations, see "Configuring 'Host' based ACLs for Kafka Feed" section in the Oracle Communications Network Analytics Data Director User Guide.

  • Adding network IP "Host" ACLs in Kafka Feed
  • Deleting network IP "Host" ACLs in Kafka Feed

6.7 Enable/Disable Traffic Segregation Using CNLB in the Data Director

This section defines the procedure to enable or disable traffic segregation in the Data Director. The procedures are applicable only when CNLB is supported in OCCNE. The Data Director currently supports traffic segregation and external access using CNLB for the following:

  • Kafka cluster external access using CNLB ingress NADs and external IPs

To perform the following operations, see "Enable/Disable Traffic Segregation Using CNLB in the Data Director" in the Oracle Communications Network Analytics Data Director User Guide.

  • Enable traffic segregation in the Data Director
  • Disable traffic segregation in the Data Director