Kafka PVC Storage Requirements

The following table describes the retention period per topic for different NFs:

Table 3-8 Retention Period Per Topic

Topic Name	Retention Period
SCP	5 Minutes
NRF	5 Minutes
MAIN	6 Hours

The following calculation is for storage requirements for a topic:

Important:

For the 6 hrs storage in the MAIN topic, the storage requirement must be calculated using the following explanation.

Storage Requirement for a topic = MPS * Retention Period * RF * Average Message Size

Where,

MPS: Message Per Second

RF: Replication Factor

Examples:

Average Message Size = 1941 Bytes

The following example uses the values from the first row of the Table 3-7 table. For more information about the table, see Disk Throughput Requirements:

Storage Requirement for SCP and NRF Topics = MPS * Retention Period * RF * Message Size = 39000 * 5 Minutes * 3 * 1941
                                                                               = 39000 * 5 * 60 * 3 * 1941
                                                                               = ~ 63.45 GB
Storage Requirement for MAIN = MPS * Retention Period * RF * Message Size  =  39000 * 6 Hours * 3 * 1941
                                                                               =  39000 * 6 * 60 * 60 * 3 * 1941
                                                                               =  ~ 4.46 TB
Total Storage Requirement for the Broker Cluster = Storage for SCP + Storage for NRF + Storage for MAIN
                                                                               = 63.45 GB + 4.46 TB
                                                                               = ~ 4.53 TB
Total Storage for each broker = (4.53/Number of Brokers) TB = (4.53/3) TB = ~ 1.51 TB [Assuming 3 Broker cluster]

Average Message Size = 3769 Bytes

The following example uses the values from the fourth row of the Table 3-7 table. For more information about the table, see Disk Throughput Requirements:

Storage Requirement for SCP and NRF Topics = MPS * Retention Period * RF * Message Size = 39000 * 5 Minutes * 3 * 3769
                                                                                        = 39000 * 5 * 60 * 3 * 3769
                                                                                        = ~ 123.20 GB

Storage Requirement for MAIN = MPS * Retention Period * RF * Message Size  =  39000 * 6 Hours * 3 * 3769
                                                                                        =  39000 * 6 * 60 * 60 * 3 * 3769
                                                                                        = ~ 8.66 TB
Total Storage Requirement for the Broker Cluster = Storage for SCP + Storage for NRF + Storage for MAIN
                                                                                        = 123.20 GB + 8.66 TB
                                                                                        = ~ 8.79 TB
Total Storage for each broker = (8.79/Number of Brokers) TB = (8.79/3) TB = ~ 2.93 TB [Assuming 3 Broker cluster]