Changing Kafka Retention Policies
Oracle Communications Unified Assurance uses Apache Kafka message buses linked to the ingestion pipeline to provide redundancy for the Metric database. A Telegraf instance on each server sends metrics to the Kafka message bus instead of directly to the Metric database, and Telegraf fails over to the redundant Kafka server as needed.
By default, Kafka retains data for three days. If you need a longer retention period, for example, if you are performing extended maintenance on your Metric database, you can temporarily extend this retention period for running Kafka servers. The change does not persist after rebooting the Kafka server or updating Unified Assurance.
To permanently change the retention period, you can override the default configuration. See the details for Kafka in Overriding Default Vendor Configurations for more information.
Verifying the Kafka Retention Policy Details
To check the existing retention period setting, run the following commands, replacing <hostFQDN> with the host server where Kafka is running:
cd $A1BASEDIR/vendor/kafka
bin/kafka-configs.sh --bootstrap-server <hostFQDN>:9093 --describe --command-config config/consumer.properties --topic influxdb
If you have not changed the retention period (or other configurations), the response is the following single line, with no values:
Dynamic configs for topic influxdb are:
If you have changed any Kafka configurations for the influxdb topic, they are listed.
Temporarily Changing the Kafka Retention Policy Details
To temporarily change the Kafka retention period, run the following command, replacing <hostFQDN> with the host server where Kafka is running:
cd $A1BASEDIR/vendor/kafka
bin/kafka-configs.sh --bootstrap-server <hostFQDN>:9093 --add-config retention.ms=<time_in_milliseconds> --command-config config/consumer.properties --topic influxdb --alter
You can also change other retention policy configurations by providing them in a comma-separated list along with retention.ms.
For example, run the following command to:
-
Set the retention period to 1728000000 milliseconds (20 days)
-
Set the maximum partition size to 1 byte
-
Set the cleanup policy to delete old segments and compact retained segments
bin/kafka-configs.sh --bootstrap-server <hostFQDN>:9093 --add-config retention.ms=1728000000,retention.bytes=-1,cleanup.policy=[compact,delete] --command-config config/consumer.properties --topic influxdb --alter
You can confirm your changes by running the --describe option again, as described in Verifying the Kafka Retention Policy Details. The changes are only retained until the Kafka server is rebooted.
For more information about the configuration options, see Topic-Level Configs in the Kafka documentation.