3 BSF Benchmark Testing

This chapters describes BSF Application related test scenarios that are validated for benchmarking BSF.

This section describes the set-up and configurations used to benchmark BSF, and then the specific results for the benchmarks are provided for each benchmark infrastructure.

3.1 Test Scenario1: BSF Call Flow Model on Two-Site GeoRedundant (GR) setup, with 15K TPS on each site

BSF Call Flow Model: BSF Performance Benchmark for 15kTPS on a ASM enabled two-site georedundant setup. The following image represents the BSF Deployed Environment for two Sites.

Figure 3-1 Two-Site BSF Deployment

3.1.1 Test Case and Setup Details

Table 3-1 Test Case Parameters

Parameters	Values
Call Rate	30K TPS (15k TPS each BSF Instance)
Call Mix	2.1k nbsf Traffic (1.05k each site) and 27.9k Rx traffic (13.95k each site)
Traffic Ratio	Nbsf TPS - 1.05k Each Site Binding Create ratio -1 Binding Delete ratio - 1 Rx TPS - 13.95k Each Site Rx AAR-I ratio - 1 Other Rx messages ratio (UPDATE and STR) - 2
Active Binding Sessions	~15000000
Execution Time	11 Hours
Site Redundancy	Two-Site deployment with replication enabled

Table 3-2 Resource Footprint for BSF Microservices

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replicas
appinfo	1	1	0.5	1	2
audit-service	1	2	1	1	2
Bsf Management Service	3	4	1	4	10
cm-service	2	4	0.5	2	2
config server	2	4	0.5	2	2
Diameter Gateway	3	4	1	2	10
Egress Gateway	3	4	4	4	2
Ingress Gateway	3	4	2	4	3
nrf-client-nfmanagement	1	1	1	1	2
queryservice	1	2	1	1	2
performance	1	1	0.5	1	2

Note:

Min Replica = Max Replica

Table 3-3 Resource Footprint for cnDBTier Microservices

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replica Count	PVC
Management node	4	4	8	10	3	ndbdisksize: 15Gi
Data node	10	10	96	112	10	ndbdisksize: 125Gi ndbbackupdisksize: 288Gi
mysqld	4	4	8	8	2	ndbdisksize: 50Gi
appmysqld	8	8	2	2	7	ndbdisksize: 16Gi
db-replication-service	0.2	0.2	0.5	0.5	1	ndbdisksize: 20Gi

Note:

Min Replica = Max Replica

Table 3-4 BSF Feautures

Features Name	Feature Status(Enabled/Disabled)
Nbsf	Enabled
Rx	Enabled

3.1.2 CPU and Memory Utilization

This section lists the CPU and memory utilization for BSF and cnDBTier.

Table 3-5 BSF Microservices and their Resource Utilization - Site1 and Site2

Service	CPU-Site1	CPU - Site2	Memory - Site1	Memory - Site2
db-backup-manager-svc	0 (0%)	0 (0%)	0.03 (23.44%)	0.03 (23.44%)
db-monitor-svc	0.01 (5%)	0.01 (5%)	0.25 (50%)	0.23 (46%)
delhi-mumbai-replication-svc	0 (0%)	0 (0%)	0.42 (84%)	0.0 (0%)
init-sidecar	0 (0%)	0 (0%)	0 (0%)	0 (0%)
mysqlndbcluster	7.9 (14.11%)	7.4 (13.21%)	5.26 (37.57%)	5.07 (36.21%)
mysqlndbcluster	0.03 (0.25%)	0.03 (0.25%)	0.07 (0.23%)	0.07 (0.23%)
db-backup-executor-svc	0 (0%)	0 (0%)	0 (0%)	0 (0%)
mysqlndbcluster	8.96 (8.96%)	8.72 (8.72%)	471.79 (42.12%)	471.78 (42.12%)
init-sidecar	0 (0%)	0 (0%)	0 (0%)	0 (0%)
mysqlndbcluster	0.34 (4.25%)	0.36 (4.5%)	1.74 (10.88%)	1.76 (11%)
appinfo	0.03 (1.5%)	0.03 (1.5%)	0.44 (22%)	0.44 (22%)
bsf-management-service	7.58 (18.95%)	7.24 (18.1%)	12.92 (32.3%)	14.13 (35.33%)
diam-gateway	7.45 (18.63%)	7.12 (17.8%)	8.96 (44.8%)	8.95 (44.75%)
config-server	0.15 (1.88%)	0.14 (1.75%)	1.55 (38.75%)	1.64 (41%)
ocbsf-egress-gateway	0.01 (0.13%)	0.01 (0.13%)	1.32 (16.5%)	1.33 (16.63%)
ocbsf-ingress-gateway	1.49 (12.42%)	1.38 (11.5%)	6.77 (56.42%)	5.13 (42.75%)
nrf-client-nfmanagement	0.01 (0.5%)	0.01 (0.5%)	0.92 (46%)	0.79 (39.5%)
audit-service	0.01 (0.25%)	0.01 (0.25%)	0.7 (35%)	0.72 (36%)
cm-service	0.02 (0.25%)	0.01 (0.13%)	1.72 (43%)	1.66 (41.5%)
queryservice	0 (0%)	0 (0%)	0.5 (25%)	0.54 (27%)
ocbsf-performance/perf-info	0 (0%)	0.01 (0.5%)	0.14 (7%)	0.1 (5%)

Table 3-6 Observed Values of cnDBTier Services

Name	Targets
ndbappmysqld	36%/80%
ndbmgmd	0%/80%
ndbmtd	15%/80%
ndbmysqld	6%/80%
ocbsf-appinfo	1%/80%
ocbsf-bsf-management	39%/80%
ocbsf-config-server	7%/80%
ocbsf-egress-gateway	0%/80%
ocbsf-ingress-gateway	22%/80%
ocbsf-nrf-client-nfmanagement	0%/80%
ocbsf-query-service	0%/80%

3.1.3 TPS and DB Statistics

This section lists the TPS (Transactions Per Second) and DB (Database) Statistics for BSF and cnDBTier.

Table 3-7 Overall TPS Ingress/Egress TPS to/from NF

Site	Rx	nbsf	Total TPS
Site1	13.95K	1.05k	15K
Site2	13.95K	1.05k	15K

Table 3-8 DB (Database) Statisitics

DB Stats	Site1	Site2
Read Rate	5.94k	5.60k
Write Rate	11.6k	11.7k
Commit Rate	6.18k	5.85k

3.1.4 Results

Table 3-9 Latency Observations

Attributes	Average Latency (ms)
Binding Create	7.6 ms
AAR-RX-INIT	11.1 ms
AAR-RX-UPDATE	1.5 ms
RX-STR	0.7 ms

3.2 Test Scenario 2: BSF Call Flow Model Performance deployed on Single Site, with 30K TPS and ASM enabled

This test run benchmarks the performance and capacity of BSF data call model. The BSF application handles an incoming traffic of 30K TPS with replication enabled. For this setup Aspen Service Mesh (ASM) was enabled.

3.2.1 Test Case and Setup Details

Table 3-10 Test Case Parameters

Parameters	Values
Call Rate	30K TPS
Call Mix	2.1k nbsf Traffic and 27.9k Rx traffic
Traffic Ratio:	Nbsf TPS - 2100 Binding Create ratio -1 Binding Delete ratio - 1 Rx TPS - 27900 Rx AAR-I ratio - 1 Other Rx messages ratio (UPDATE and STR) - 2
Active User/Sessions	~15000000
Execution Time	14 Hours

Table 3-11 BSF Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replicas
appinfo	1	1	0.5	1	2
audit-service	1	2	1	1	2
Bsf Management Service	3	4	1	4	10
cm-service	2	4	0.5	2	2
config server	2	4	0.5	2	2
Diameter Gateway	3	4	1	2	10
Egress Gateway	3	4	4	4	2
Ingress Gateway	3	4	2	4	3
nrf-client-nfmanagement	1	1	1	1	2
queryservice	1	2	1	1	2
performance	1	1	0.5	1	2

Table 3-12 cnDBTier Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replica Count
Management node	4	4	8	10	3
Data node	10	10	96	112	10
mysqld	4	4	8	8	2
appmysqld	8	8	2	2	7
db-monitor-service	0.2	0.2	0.5	0.5	1
db-replication-service	0.2	0.2	0.5	0.5	1
db-backup-service	0.2	0.2	0.128	0.128	1

Table 3-13 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

3.2.2 CPU and Memory Utilization

This section lists the CPU and memory utilization for BSF and cnDBTier.

Table 3-14 BSF Services and their Resource Utilization

Microservices	Container Count	CPU	Memory
db-backup-manager-svc	1	0 (0%)	0.03 (23.44%)
db-monitor-svc	1	0.01 (5%)	0.25 (50%)
init-sidecar	7	0 (0%)	0 (0%)
mysqlndbcluster	7	7.9 (14.11%)	5.26 (37.57%)
mysqlndbcluster	3	0.03 (0.25%)	0.07 (0.23%)
db-backup-executor-svc	10	0 (0%)	0 (0%)
mysqlndbcluster	10	8.96 (8.96%)	471.79 (42.12%)
init-sidecar	2	0 (0%)	0 (0%)
mysqlndbcluster	2	0.34 (4.25%)	1.74 (10.88%)
appinfo	2	0.03 (1.5%)	0.44 (22%)
bsf-management-service	10	7.58 (18.95%)	12.92 (32.3%)
diam-gateway	10	7.45 (18.63%)	8.96 (44.8%)
config-server	2	0.15 (1.88%)	1.55 (38.75%)
ocbsf-egress-gateway	2	0.01 (0.13%)	1.32 (16.5%)
ocbsf-ingress-gateway	3	1.49 (12.42%)	6.77 (56.42%)
nrf-client-nfmanagement	2	0.01 (0.5%)	0.92 (46%)
audit-service	2	0.01 (0.25%)	0.7 (35%)
cm-service	2	0.02 (0.25%)	1.72 (43%)
queryservice	2	0 (0%)	0.5 (25%)
perf-info	2	0 (0%)	0.14 (7%)
delhi-mumbai-replication-svc	1	0 (0%)	0.42 (84%)

Table 3-15 cnDBTier Services and their Resource Utilization

Name	Targets	MinPods	MaxPods
ndbappmysqld	35%/80%	7	7
ndbmgmd	0%/80%	3	3
ndbmtd	21%/80%	10	10
ndbmysqld	5%/80%	2	2
ocbsf-appinfo	1%/80%	2	2
ocbsf-bsf-management	67%/80%	10	10
ocbsf-config-server	6%/80%	2	2
ocbsf-egress-gateway	0%/80%	2	2
ocbsf-ingress-gateway	41%/80%	3	3
ocbsf-nrf-client-nfmanagement	0%/80%	2	2
ocbsf-query-service	0%/80%	2	2

3.2.3 TPS and DB Statistics

This section lists the TPS (Transactions Per Second) and DB (Database) Statistics for BSF.

Table 3-16 TPS Statistics

Site	Rx	nbsf	Total TPS
Site1	27.9k	2.1k	30K

Table 3-17 DB(Database) Records

DB Stats	Site
Read Rate	11.1k
Write Rate	11.7k
Commit Rate	11.9k

3.2.4 Results

Table 3-18 Latency Observations

Attributes	Average Latency (ms)
Binding Create	8.1 ms
Binding Delete	8.5 ms
AAR-RX-INIT	12.2 ms
AAR-RX-UPDATE	0.8 ms
RX-STR	0.7 ms

3.3 Test Scenario 3: BSF Call Model deployed on Three-Site GeoRedundant (GR) setup, with 35K TPS and ASM enabled

This test run benchmarks the performance and capacity of BSF data call model that is deployed on a three-site georedundant setup. At each site, nbsf interface handles 1.2k TPS traffic and Rx interface handles 10.4k TPS traffic. For this setup Aspen Service Mesh (ASM) was enabled.

3.3.1 Test Case and Setup Details

Table 3-19 Test Case Parameters

Parameters	Values
Call Rate	35K TPS
Call Mix	3.6k nbsf traffic (1.2k each site) and 31.2k Rx traffic (10.4k each site)
Traffic Ratio:	Nbsf TPS - 1.2k each Site Binding Create ratio -1 Binding Delete ratio - 1 Rx TPS - 10.4k each Site Rx AAR-I ratio - 1 Other Rx messages ratio (UPDATE and STR) - 2
Active User/Sessions	~18.5M
Execution Time	14 Hours

Table 3-20 BSF Microservices Resource

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replicas
Appinfo	1	1	0.5	1	3
Audit Service	1	2	1	1	3
Bsf Management Service	3	4	1	4	8
CM Service	2	4	0.5	2	3
Config Service	2	4	0.5	2	3
Diameter Gateway	3	4	0.5	2	9
Egress Gateway	3	4	4	6	3
Ingress Gateway	3	4	4	6	3
Nrf Client Management	1	1	1	1	3
Query Service	1	2	1	1	3
Performance	3	4	0.5	1	3

Note:

Min Replica = Max Replica

Table 3-21 cnDBTier Microservices Resource

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Replicas
Management node	3	3	8	10	2
Data node	12	12	64	73	14
mysqld	4	4	16	16	4
appmysqld	12	12	20	20	2
db-monitor-service	0.2	0.2	0.5	0.5	1
db-replication-service	0.2	0.2	12	12	2
db-backup-service	0.1	0.128	0.128	0.128	1

Note:

Min Replica = Max Replica

Table 3-22 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

3.3.2 CPU and Memory Utilization

This section lists the CPU and memory utilization for BSF and cnDBTier microservices. The CPU utilization is the ratio between the (total CPU utilization against total CPU request (X)) versus (target CPU Utilization (Y) set configured for the pod).

Table 3-23 BSF Microservices CPU and Memory Utilization

Service	CPU Site1	CPU Site2	CPU Site3	Memory Site1	Memory Site2	Memory Site3
Appinfo	0.04 (1.33%)	0.04 (1.33%)	0.05 (1.67%)	0.67 (22.33%)	0.67 (22.33%)	0.67 (22.33%)
Audit Service	0.01 (0.17%)	0.01 (0.17%)	0.01 (0.17%)	0.92 (30.67%)	0.9 (30%)	0.9 (30%)
CM Service	0.02 (0.17%)	0.02 (0.17%)	0.02 (0.17%)	1.42 (23.67%)	1.29 (21.5%)	1.39 (23.17%)
Bsf Management Service	5.28 (16.5%)	5.3 (16.56%)	4.92 (15.38%)	8.34 (26.06%)	8 (25%)	7.53 (23.53%)
Diameter Gateway	5.62 (15.61%)	5.37 (14.92%)	5.39 (14.97%)	6.63 (36.83%)	6.5 (36.11%)	6.35 (35.28%)
Egress Gateway	0.01 (0.08%)	0.01 (0.08%)	0.01 (0.08%)	1.97 (16.42%)	2.04 (17%)	1.97 (16.42%)
Ingress Gateway	1.36 (11.33%)	1.49 (12.42%)	1.46 (12.17%)	4.45 (37.08%)	4.47 (37.25%)	4.66 (38.83%)
Nrf Client Management	0.01 (0.33%)	0.01 (0.33%)	0.01 (0.33%)	0.95 (31.67%)	0.95 (31.67%)	0.96 (32%)
Query Service	0 (0%)	0 (0%)	0 (0%)	0.66 (22%)	0.66 (22%)	0.66 (22%)
Perf Info	0.02 (0.67%)	0.08 (2.67%)	0.09 (3%)	0.3 (10%)	0.3 (10%)	0.3 (10%)

Table 3-24 cnDBTier Microservices CPU and Memory Utilization

Service	CPU Site1	CPU Site2	CPU Site3	Memory Site1	Memory Site2	Memory Site3
ndbappmysqld	5.46 (34.13%)	5.69 (35.56%)	5.15 (32.19%)	3.18 (79.5%)	3.12 (78%)	2.96 (74%)
ndbmtd	10.29 (7.35%)	9.34 (6.67%)	8.56 (6.11%)	662.41 (42.25%)	662.4 (42.24%)	662.38 (42.24%)
ndbmysqld	0.54 (3.38%)	0.47 (2.94%)	0.46 (2.88%)	4.35 (13.59%)	4.3 (13.44%)	4.24 (13.25%)
db-backup-manager-svc	0 (0%)	0 (0%)	0 (0%)	0.04 (31.25%)	0.04 (31.25%)	0.04 (31.25%)
db-monitor-svc	0.01 (5%)	0.01 (5%)	0.01 (5%)	0.24 (48%)	0.27 (54%)	0.24 (48%)

3.3.3 TPS and DB Statistics

This section lists the TPS (Transactions Per Second) and DB (Database) statistics for BSF.

Table 3-25 TPS Statistics

Site	Rx	nbsf	Total TPS
Site1	10.04k	1.2k	11.6k
Site2	10.04k	1.2k	11.6k
Site3	10.04k	1.2k	11.6k

Table 3-26 Database Records

DB Statistics	Site1	Site2	Site3
Read Rate	8.77k	8.77k	8.08k
Write Rate	13.4k	14.1k	13.5k
Commit Rate	9.67k	9.51k	8.79k

3.3.4 Results

Table 3-27 Latency Observations

Attributes	Average Latency (ms)
Binding Create	7.4 ms
AAR-RX-INIT	13.7 ms
AAR-RX-UPDATE	0.9 ms
RX-STR	0.9 ms

3.4 Test Scenario 4: BSF Call Model deployed on Single Site setup, with 35K TPS traffic and ASM enabled

This test run benchmarks the performance and capacity of BSF call model that is deployed on a single site setup with Aspen Service Mesh (ASM) enabled. In the setup, nbsf interface handles 3.6k TPS and Rx interface hanldes 31.2K TPS traffic respectively.

3.4.1 Test Case and Setup Details

Table 3-28 Test Case Parameters

Parameters	Values
Call Rate	35K TPS
Call Mix	3.6k nbsf and 31.2k Rx traffic
Traffic Ratio:	Nbsf TPS - 3.6k Binding Create ratio -1 Binding Delete ratio - 1 Rx TPS - 31.2k Rx AAR-I ratio - 1 Other Rx messages ratio (UPDATE and STR) - 2
Active User/Sessions	~18.5M
Execution Time	45 Hours

Table 3-29 BSF Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isitio CPU Limit	Isitio Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	1	1	0.5	1	2	2	2	2	3
Audit Service	1	2	1	1	2	2	2	2	3
BSF Management Service	3	4	1	4	2	2	2	2	10
CM Service	2	4	0.5	2	2	2	2	2	3
Config Service	2	4	0.5	2	2	2	2	2	3
Diameter Gateway	3	4	0.5	2	2	2	2	2	9
Egress Gateway	3	4	4	6	2	2	2	2	3
Ingress Gateway	3	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	3
Query Service	1	2	1	1	2	2	2	2	3
Performance	3	4	0.5	1	2	2	2	2	3

Note:

Min Replica = Max Replica

Table 3-30 cnDBTier Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Management node	3	3	8	10	2	2	2	2	2
Data node	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3
db-monitor-service	0.2	0.2	0.5	0.5	2	2	2	2	1
db-replication-service	0.2	0.2	12	12	2	2	2	2	2
db-backup-service	0.1	0.1	0.128	0.128	0	0	0	0	1

Note:

Min Replica = Max Replica

Table 3-31 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

Table 3-32 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	5	Specifies the number of execution threads used by ndbmtd.
datamemory	78 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.
SERVER_XNIO_TASK_THREAD_SIZE	180	Specifies the BSF management service server thread size count.

3.4.2 CPU and Memory Utilization

This section lists the CPU and memory utilization for BSF and cnDBTier microservices. The CPU utilization is the ratio between the (total CPU utilization against total CPU request (X)) versus (target CPU Utilization (Y) configured for the pod).

Table 3-33 BSF Microservices CPU and Memory Utilization

Service	CPU	Memory
Appinfo	0.0405(1.35%)	0.736(22.9%)
Audit Service	0.0147(0.245%)	1.18(36.6%)
Bsf Management Service	16.6(41.5%)	19.3(44.9%)
CM Service	0.0364(0.303%)	2.19(34.0%)
Config Server	0.134(1.19%)	2.08(32.3%)
Diameter Gateway	14.0(38.8%)	9.01(46.6%)
Egress Gateway	0.0189(0.158%)	1.94(10.0%)
Ingress Gateway	3.61(0.158%)	7.99(41.3%)
Nrf Client Management	0.0180(0.600%)	1.26(39.2%)
Query Service	0.00911(0.152%)	0.854(26.5%)
Perf Info	0.0251(0.209%)	0.354(11.0%)

Table 3-34 cnDBTier Microservices CPU and Memory Utilization.

Service	CPU	Memory
ndbappmysqld	16.8(46.6%)	2.94(11.6%)
ndbmtd	7.66(21.3%)	5.31(90.5%)
ndbmysqld	0.818(0.835%)	0.831(7.39%)
db-backup-manager-svc	0.00353(3.53%)	3.42(54.6%)
db-monitor-svc	0.0122(6.10%)	5.48(50.6%)
replication-svc	0.0811(0.405%)	5.32(3.30%)

3.4.3 TPS and DB Statistics

This section lists the TPS (Transactions Per Second) and DB (Database) statistics for BSF.

Table 3-35 TPS Statistics

Site	Rx	nbsf	Total TPS
Site1	31.2k	3.6k	34.8k

Table 3-36 Database Records

DB Statistics	Site1
Read Rate	11.8k
Write Rate	12.9k
Commit Rate	13.0k

3.4.4 Results

Table 3-37 Latency Observations

Attributes	Average Latency (ms)
Binding Create	7.6 ms
Binding Delete	8.9 ms
AAR-RX-INIT	13.9 ms
AAR-RX-UPDATE	0.8 ms
RX-STR	0.7 ms

3.5 Test Scenario 5: BSF Performance Benchmarking 47.7K TPS traffic on three-site Setup

BSF Call Flow Model: BSF Performance Benchmark for 47.7k TPS on a ASM enabled three-site georedundant setup. The following image represents the BSF deployment on a ASM enabled three-site georedundant setup.

Figure 3-2 Three-Site BSF Deployment

3.5.1 Test Case and Setup Details

Table 3-38 Test Case Parameters

Parameters	Values
Call Rate	47.7K TPS (Approximately, 16k TPS each site)
Call Mix	7.2k nbsf traffic (2.4K each site) and 40.5K Rx traffic (13.5K each site)
Traffic Ratio	Nbsf TPS - 2.4K Each Site Binding Create ratio -1 Binding Delete ratio - 1 Rx TPS - 13.5K Each Site Rx AAR-I ratio - 1 Other Rx messages ratio (UPDATE and STR) - 2
Active Binding Sessions	~18500000
Execution Time	58 Hours

Table 3-39 BSF Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isitio CPU Limit	Isitio Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	1	1	0.5	1	2	2	2	2	3
Audit Service	1	2	1	1	2	2	2	2	3
BSF Management Service	3	4	1	4	2	2	2	2	10
CM Service	2	4	0.5	2	2	2	2	2	3
Config Service	2	4	0.5	2	2	2	2	2	3
Diameter Gateway	3	4	0.5	2	2	2	2	2	9
Egress Gateway	3	4	4	6	2	2	2	2	3
Ingress Gateway	3	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	3
Query Service	1	2	1	1	2	2	2	2	3
Performance	3	4	0.5	1	2	2	2	2	3

Note:

Min Replica = Max Replica

Table 3-40 cnDBTier Microservices Resource Allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Management node	3	3	8	10	2	2	2	2	2
Data node	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

Note:

Min Replica = Max Replica

Table 3-41 BSF Feautures

Features Name	Feature Status (Enabled/Disabled)
Nbsf	Enabled
Rx	Enabled

Table 3-42 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	5	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	78 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.
SERVER_XNIO_TASK_THREAD_SIZE	180	Specifies the BSF management service server thread size count.

3.5.2 CPU and Memory Utilization

This section lists the CPU and memory utilization for BSF and cnDBTier microservices. The CPU utilization is the ratio between the (total CPU utilization against total CPU request (X)) versus (target CPU Utilization (Y) configured for the pod).

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-43 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site2 CPU	Site2 Memory	Site3 CPU	Site3 Memory
queryservice	0.00145(0.0241%)	788(24.5%)	0.00130(0.0217%)	790(24.5%)	0.00130(0.0217%)	791(24.6%)
perf-info	0.117(0.977%)	434(13.5%)	0.117(0.973%)	4.34(13.5%)	0.116(0.967%)	447(13.9%)
nrf-client-nfmanagement	0.00457(0.152%)	1.52(47.3%)	0.00519(0.173%)	1.50(46.6%)	0.00471(0.157%)	1.50(46.7%)
ingress-gateway	1.42(11.8%)	7.06(36.5%)	1.49(12.4%)	6.95(35.9%)	1.47(12.2%)	6.40(33.1%)
egress-gateway	0.00499(0.0416%)	2.49(12.9%)	0.00446(0.0371%)	2.82(14.6%)	0.00444(0.0370%)	2.69(13.9%)
diam-gateway	5.63(15.6%)	9.66(50.0%)	5.53(15.3%)	9.57(49.5%)	5.63(15.6%)	9.41(48.7%)
config-server	0.0935(0.779%)	2.50(38.9%)	0.0910(0.759%)	2.52(39.2%)	0.0890(0.742%)	2.70(42.0%)
cm-service	0.0131(0.109%)	2.08(32.2%)	0.0133(0.111%)	2.01(31.2%)	0.0128(0.107%)	2.12(32.8%)
bsf-management-service	4.24(10.6%)	15.8(36.7%)	4.37(10.9%)	15.1(35.3%)	4.30(10.7%)	16.8(39.1%)
audit-service	0.00524(0.0874%)	1.16(36.1%)	0.00497((0.0828%)	1.17(36.4%)	0.00530(0.0883%)	1.18(36.5%)
appinfo	0.0463(1.54%)	786(24.4%)	0.0466(1.55%)	797(24.7%)	0.0473(1.58%)	791(24.6%)

The following table provides information about observed values of cnDBTier services.

Table 3-44 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site2 CPU	Site2 Memory	Site3 CPU	Site3 Memory
ndbappmysqld	4.86(13.5%)	17.3(26.8%)	5.23(14.5%)	17.3(26.9%)	5.20(14.4%)	17.2(26.7%)
ndbmtd	4.16(11.6%)	692(80.6%)	4.48(12.4%)	691(80.5%)	4.26(11.8%)	691(80.5%)
ndbmysqld	0.329(0.214%)	67.9(6.70%)	0.337(0.219%)	67.7(6.68%)	0.327(0.212%)	67.5(6.66%)

3.5.3 TPS and DB Statistics

This section lists the TPS (Transactions Per Second) and DB (Database) Statistics for BSF and cnDBTier.

Table 3-45 Overall TPS Ingress/Egress TPS to/from NF

Site	Rx	nbsf	Total TPS
Site1	13.5K	2.4K	15.9K
Site2	13.5K	2.4K	15.9K
Site3	13.5K	2.4K	15.9K

Table 3-46 DB (Database) Statisitics

DB Stats	Read Rate	Write Rate	Commit Rate
Site1	4.97K	20.3K	6.92K
Site2	5.02K	20.3K	6.92K
Site3	4.97K	20.3K	7.01K

3.5.4 Results

Table 3-47 Latency Observations

Attributes	Average Latency (ms)
Binding Create	5.2ms
Binding Delete	4.9ms
AAR-RX-INIT	8.48ms
AAR-RX-UPDATE	0.7ms
RX-STR	0.7ms

3.6 Test Scenario 6: BSF Performance Benchmarking 45K TPS traffic on a three-site Setup with traffic on one Site

BSF Call Flow Model

BSF Performance Benchmark for 45kTPS on an ASM enabled three-site georedundant setup with traffic on one site.

The following image represents the BSF deployment on an ASM enabled three-site georedundant setup:

Figure 3-3 ASM enabled three-site georedundant setup

3.6.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-48 Testcase Parameters

Parameters	Values
Call Rate	45K TPS
Call Mix	Approximately, 3.6k nbsf traffic and 41.4k Rx traffic on one site
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~25000000
Execution Time	~7 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-49 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	1	1	1	1	2	2	2	2	4
Audit Service	2	2	1	1	2	2	2	2	4
BSF Management Service	4	4	4	4	2	2	2	2	9
CM Service	4	4	2	2	2	2	2	2	4
Config Service	4	4	2	2	2	2	2	2	4
Diameter Gateway	4	4	2	2	2	2	2	2	10
Egress Gateway	4	4	6	6	2	2	2	2	4
Ingress Gateway	4	4	6	6	2	2	2	2	4
Nrf Client Management	1	1	1	1	2	2	2	2	4
Query Service	2	2	1	1	2	2	2	2	4
Performance	4	4	1	1	2	2	2	2	4

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-50 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
Management node	3	3	10	10	2	2	2	2	2
Data node	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-51 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-52 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	5	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	78 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.6.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-53 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	14.2(35.5%)	9.38 Bil(43.7%)	3.11(15.5%)	1.96 Bil(9.13%)	-	-	-	-	-	-	-	-
Perfinfo	0.148(0.924%)	591 Mil(13.7%)	0.317(3.96%)	756 Mil(8.81%)	-	-	-	-	-	-	-	-
Appinfo	0.105(2.63%)	2.12 Bil(49.4%)	0.293(3.66%)	1.35 Bil(15.7%)	-	-	-	-	-	-	-	-
cm-service	0.0201(0.126%)	2.80 Bil(32.6%)	0.327((4.08%)	779 Nil(9.06%)	-	-	-	-	-	-	-	-
Config-server	0.0940(0.588%)	3.22 Bil((37.5%)	0.368((4.60%)	758 Mil(8.82%)	-	-	-	-	-	-	-	-
bsf-management-service	12.0(33.3%)	25.4 Bil(65.8%)	9.00(50.0%)	1.87 Bil(9.67%)	-	-	-	-	-	-	-	-
egress-gateway	0.00893(0.0558%)	3.22 Bil(12.5%)	0.280(3.50%)	762 Mil(8.87%)	-	-	-	-	-	-	-	-
ingress-gateway	2.81(17.6%)	7.83 Bil(30.4%)	2.08(26.0%)	832 Mil(9.68%)	-	-	-	-	-	-	-	-
nrf-client-nfmanagement	0.0106(0.265%)	1.95 Bil(45.5%)	0.299(3.74%)	769 Mil(8.96%)	-	-	-	-	-	-	-	-
audit-service	0.0539(0.673%)	1.70 Bil((39.6%)	0.294(3.68%)	779 Mil(9.07%)	-	-	-	-	-	-	-	-
queryservice	0.00163(0.0204%)	1.06 Bil(24.8%)	0.277(3.47%)	748 Mil(8.71%)	-	-	-	-	-	-	-	-

The following table provides information about observed values of cnDBTier services.

Table 3-54 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
ndbappmysqld	13.8 (38.3%)	41.1 Bil (63.8%)	6.47 (40.5%)	786 Mil (4.08%)	-	-	-	-	-	-	-	-
ndbmtd	7.85 (21.8%)	73.3 Bil (85.3%)	8.94 (54.9%)	1.78 Bil (9.30%)	-	-	-	-	-	-	-	-
ndbmysqld	0.366 (0.238%)	68.7 Bil (6.78%)	0.503 (0.286%)	769 Mil (1.33%)	-	-	-	-	-	-	-	-

3.6.3 TPS and DB Statistics

Table 3-55 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	45K	41.4K	3.6K
Site 2	-	-	-
Site 3	-	-	-

Observed values of cnDBTier Record counts:

Table 3-56 Database Records

DB Statistics	Read	Write	Commit
Site1	12.9K	14K	14.2K
Site2	-	-	-
Site3	-	-	-

3.6.4 Results

Table 3-57 Results

Attributes	Average Latency (ms)
Binding Create	7.3
Binding Delete	6.55
AAR-RX-INIT	8.4
RX-RAR	0.81
AAR-RX-UPDATE	1.07
RX-STR	0.77

Table 3-58 Average NF Latency

NF	50th Percentile (Site 1)	99th Percentile (Site 1)
Ingress Gateway	6.05 ms	16.2 ms
Egress Gateway	1.03 ms	1.15 ms
Diameter Gateway	-	0.368 ms

3.7 Test Scenario 7: BSF Performance Benchmarking 45K TPS traffic on a three-site Setup with traffic on Two Site

BSF Call Flow Model: BSF Performance Benchmark for 45kTPS on an ASM enabled three-site georedundant setup with traffic on two sites

3.7.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-59 Testcase Parameters

Parameters	Values
Call Rate	45K TPS
Call Mix	3.6k nbsf(1.8K on each each site) traffic and 41.4k (20.7K each site) Rx traffic on a ASM enabled Setup.
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~25000000
Execution Time	~17 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-60 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	1	1	1	1	2	2	2	2	4
Audit Service	2	2	1	1	2	2	2	2	4
BSF Management Service	4	4	4	4	2	2	2	2	9
CM Service	4	4	2	2	2	2	2	2	4
Config Service	4	4	2	2	2	2	2	2	4
Diameter Gateway	4	4	2	2	2	2	2	2	10
Egress Gateway	4	4	6	6	2	2	2	2	4
Ingress Gateway	4	4	6	6	2	2	2	2	4
Nrf Client Management	1	1	1	1	2	2	2	2	4
Query Service	2	2	1	1	2	2	2	2	4
Performance	4	4	1	1	2	2	2	2	4

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-61 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
Management node	3	3	10	10	2	2	2	2	2
Data node	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-62 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-63 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	5	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	78 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.7.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-64 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	7.74 (19.3%)	7.85 Bil (36.5%)	1.11 (5.54%)	1.86 Bil (8.66%)	8.42 (21.0%)	7.94 Bil (37.0%)	1.35 (6.44%)	1.84 Bil (8.55%)	-	-	-	-
Perfinfo	0.166 (1.04%)	5.79 Mil (13.5%)	0.0246 (0.307%)	710 Mil (8.27%)	0.167 (1.04%)	579 Mil (13.5%)	0.0264 (0.265%)	713 Mil (8.30 %)	-	-	-	-
Appinfo	0.0951 (2.38%)	1.07 Bil (24.9%)	0.0186 (0.232%)	728 Mil (8.48%)	0.107 (2.66%)	1.06 Bil (24.7%)	0.0191 (0.143%)	726 Mil (8.45%)	-	-	-	-
cm-service	0.0222 (0.139%)	2.82 Bil (32.8%)	0.0233 (0.291%)	742 Mil (8.64%)	0.0161 (0.100%)	2.77 Bil (32.3%)	0.0223 (0.211%)	722 Mil (8.40 %)	-	-	-	-
Config-server	0.0940 (0.588%)	3.61 Bil (42.0%)	0.0953 (1.19%)	737 Mil (8.58%)	0.0891 (0.557%)	3.44 Bil (40.0%)	0.0885 (1.00%)	730 Mil (8.49%)	-	-	-	-
bsf-management-service	5.54 (15.4%)	25.4 Bil (65.7%)	3.74 (20.8%)	1.73 Bil (8.97%)	6.43 (17.9%)	25.8 Bil (66.7%)	4.38 (23.6%)	1.71 Bil (8.85%)	-	-	-	-
egress-gateway	0.0163 (0.102%)	3.23 Bil (12.5%)	0.00980 (0.122%)	719 Mil (8.37%)	0.0134 (0.0839%)	3.07 Bil (11.9%)	0.0110 (0.0644%)	712 Mil (8.39%)	-	-	-	-
ingress-gateway	1.37 (8.55%)	9.65 Bil (37.4%)	0.859 (10.7%)	964 Mil (11.2%)	1.45 (9.05%)	9.40 Bil (36.5%)	0.890 (9.87%)	850 Mil (9.90%)	-	-	-	-
nrf-client-nfmanagement	0.00892 (0.223%)	1.96 Bil (45.6%)	0.0151 (0.188%)	748 Mil (8.71%)	0.00871 (0.218%)	1.96 Bil (45.6%)	0.0157 (0.119%)	727 Mil (8.46%)	-	-	-	-
audit-service	0.0105 (0.131%)	1.80 Bil (42.0%)	0.0121 (0.151%)	754 Mil (8.77%)	0.0103 (0.129%)	1.73 Bil (40.2%)	0.0139 (0.949%)	715 Mil (8.33%)	-	-	-	-
queryservice	0.00303 (0.0379%)	1.06 Bil (24.8%)	0.0105 (0.132)	725 Mil (8.44%)	0.00240 (0.0300%)	1.04 Bil (24.1%)	0.0114 (0.788%)	715 Mil (8.32%)	-	-	-	-

The following table provides information about observed values of cnDBTier services.

Table 3-65 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
ndbappmysqld	6.27 (16.4%)	38.8 Bil	3.03 (20.2%)	770 Mil (3.94%)	7.58 (21.1%)	38.8 Bil (60.3%)	3.56 (23.7%)	774 Mil (4.01%)	-	-	-	-
ndbmtd	5.00 (12.9%)	750 Bil	4.62 (30.8%)	1.75 Bil (8.93%)	5.44 (15.1%)	748 Bil (87.1%)	5.07 (33.8%)	1.75 Bil (9.05%)	-	-	-	-
ndbmysqld	0.628 (0.190%)	68.7 Bil	0.202 (0.270%)	747 Mil (1.23%)	0.305 (0.198%)	68.7 Bil (6.78%)	0.192 (0.256%)	755 Mil (1.30%)	-	-	-	-

3.7.3 TPS and DB Statistics

Table 3-66 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	22.5k	20.7k	1.8k
Site2	22.5k	20.7k	1.8k
Site 3	-	-	-

Observed values of cnDBTier Record counts:

Table 3-67 Database Records

DB Statistics	Read	Write	Commit
Site1	6.25k	14k	6.59k
Site2	7.38k	14.1k	7.84k
Site3	-	-	-

3.7.4 Results

Table 3-68 Results

Attributes	Average Latency (ms)
Binding Create	5.85
Binding Delete	5.17
AAR-RX-INIT	6.73
RX-RAR	0.51
AAR-RX-UPDATE	0.57
RX-STR	0.49

Table 3-69 Average NF Latency

NF	50th Percentile (Site 1)	99th Percentile (Site 1)	50th Percentile (Site 2)	99th Percentile (Site 2)
Ingress Gateway	4.97 ms	7.93 ms	5.21 ms	9.20 ms
Egress Gateway	1.39 ms	1.54 ms	1.95 ms	2.09 ms
Diameter Gateway	-	0.0639 ms	-	0.0866 ms

3.8 Test Scenario 8: BSF Performance Benchmarking 45K TPS traffic on a three-site Setup with traffic on three Sites

BSF Call Flow Model: BSF Performance Benchmark for 45kTPS on an ASM enabled three-site georedundant setup with traffic on three sites.

3.8.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-70 Testcase Parameters

Parameters	Values
Call Rate	45K TPS
Call Mix	Approximately, 3.6k nbsf traffic and 41.4k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~25000000
Execution Time	~40 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-71 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	1	1	1	1	2	2	2	2	4
Audit Service	2	2	1	1	2	2	2	2	4
BSF Management Service	4	4	4	4	2	2	2	2	9
CM Service	4	4	2	2	2	2	2	2	4
Config Service	4	4	2	2	2	2	2	2	4
Diameter Gateway	4	4	2	2	2	2	2	2	10
Egress Gateway	4	4	6	6	2	2	2	2	4
Ingress Gateway	4	4	6	6	2	2	2	2	4
Nrf Client Management	1	1	1	1	2	2	2	2	4
Query Service	2	2	1	1	2	2	2	2	4
Performance	4	4	1	1	2	2	2	2	4

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-72 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
Management node	3	3	10	10	2	2	2	2	2
Data node	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-73 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-74 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	5	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	78 GB	Specifes the cnDBTier ndb data memory capacity configuration.
cnDBTier App node (ndbappmysqld) istio side car parameter (proxy.istio.io/config)	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
cnDBTier Data node (ndbmtd) istio side car parameter (proxy.istio.io/config)	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.8.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-75 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	5.88 (14.7%)	8.73 Bil (40.6%)	0.973 (4.87%)	2.14 Bil (9.97%)	5.58 (14.0%)	8.38 Bil (37.5%)	0.883 (4.42%)	2.01 Bil (9.36%)	5.73 (14.3%)	8.44 Bil (39.3%)	0.952 (4.76%)	1.95 Bil (9.09 %)
Perfinfo	0.174 (1.09%)	608 Mil (14.2%)	0.0302 (0.377%)	745 Mil (8.67%)	0.172 (1.07%)	579 Mil (13.5%)	0.0353 (0.441%)	757 Mil (8.81%)	0.168 (1.05%)	567 Mil (13.2%)	0.0330 (0.413%)	749 Mil (8.72%)
Appinfo	0.107 (2.67%)	1.07 Bil (24.9%)	0.0257 (0.322%)	745 Mil (8.67%)	0.119 (2.98%)	1.06 Bil (24.7%)	0.0238 (0.298%)	754 Mil (8.78%)	0.104 (2.60%)	1.07 Bil (24.8%)	0.0251 (0.314%)	756 Mil (8.80%)
cm-service	0.0204 (0.128%)	2.85 Bil (33.2%)	0.0325 (0.406%)	776 Mil (9.03%)	0.0183 (0.114%)	2.78 Bil (32.3%)	0.0313 (0.392%)	748 Mil (8.70%)	0.0199 (0.125%)	2.92 Bil (34.0%)	0.0313 ((0.391%)	755 Mil (8.79%)
config-server	0.107 (0.670%)	3.70 Bil (43.1%)	0.105 (1.31%)	760 Mil (8.85%)	0.102 (0.635%)	3.54 Bil (41.3%)	0.101 (1.26%)	751 Mil (8.75%)	0.0988 (0.617%)	3.46 Bil (40.3%)	0.103 (1.29%)	777 Mil (9.04%)
bsf-management-service	4.37 (12.1%)	28.5 Bil (73.8%)	2.95 (16.4%)	1.96 Bil (10.1%)	4.20 (11.7%)	26.7 Bil (69.0%)	2.77 (15.4%)	1.85 Bil (9.57%)	4.28 (11.9%)	25.3 Bil (65.5%)	2.81 (15.6%)	1.85 Bil (9.57%)
egress-gateway	0.0118 (0.0737%)	3.30 Bil (12.8%)	0.0204 (0.255%)	743 Mil (8.65%)	0.00856 (0.0535%)	3.14 Bil (12.2%)	0.0170 (0.213%)	746 Mil (8.69%)	0.0107 (0.0668%)	3.38 Bil (13.1%)	0.0183 (0.229%)	741 Mil (8.63%)
ingress-gateway	1.03 (6.46%)	10.3 Bil (40.1%)	0.626 (7.82%)	1.23 Bil (14.3%)	1.11 (6.92%)	9.71 Bil (37.7%)	0.662 (8.27%)	973 Mil (11.3%)	0.994 (6.21%)	6.98 Bil (27.1%)	0.636 (7.95%)	865 Mil (10.1%)
nrf-client-nfmanagement	0.0161 (0.403%)	1.97 Bil (45.9%)	0.0212 (0.265%)	764 Mil (8.90%)	0.0157 (0.391%)	1.97 Bil (45.8%)	0.0242 (0.302%)	754 Mil (8.78%)	0.0139 ((0.348%)	1.97 Bil (46.0%)	0.0215 (0.269%)	766 Mil (8.92%)
audit-service	0.0499 (0.624%)	1.88 Bil (43.7%)	0.0358 (0.448%)	774 Mil (9.01%)	0.0541 (0.676%)	1.89 Bil (43.9%)	0.0446 (0.558%)	751 Mil (8.75%)	0.0180 (0.225%)	1.85 Bil (43.1%)	0.0194 (0.243%)	741 Mil (8.62%)
queryservice	0.00297 (0.0371%)	1.06 Bil (24.8%)	0.0183 (0.228%)	745 Mil (8.67%)	0.00260 (0.0325%)	1.04 Bil (24.1%)	0.0196 (0.245%)	740 Mil (8.61%)	0.00262 (0.0327%)	1.05 Bil (24.5%)	0.0179 (0.223%)	738 Mil (8.59%)

The following table provides information about observed values of cnDBTier services.

Table 3-76 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
ndbappmysqld	4.93 (13.7%)	59.9 Bil (93.0%)	2.45 (16.3%)	791 Mil (4.09%)	14.8 (41.2%)	60.2 Bil (85.6%)	2.34 (15.6%)	789 Mil (4.08%)	4.89 (13.6%)	47.5 Bil (75.3%)	2.45 (16.4%)	786 Mil (4.06%)
ndbmtd	4.88 (13.6%)	770 Bil (89.7%)	4.10 (27.3%)	1.81 Bil (9.38%)	4.98 (13.7%)	770 Bil (89.7%)	3.94 (26.3%)	1.80 Bil (9.31%)	4.98 (13.8%)	742 Bil (86.8%)	4.09 (27.3%)	1.85 Bil (9.59%)
ndbmysqld	0.338 (0.220%)	68.7 (6.78%)	0.206 (0.274%)	776 Mil (1.34%)	0.320 (0.208%)	68.7 Bil (6.61%)	0.209 (0.278%)	786 Mil (1.36%)	0.369 (0.240%)	39.8 Bil (5.14%)	0.209 (0.279%)	770 Mil (1.33%)

3.8.3 TPS and DB Statistics

Table 3-77 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	15k	13.8k	1.2k
Site2	15k	13.8k	1.2k
Site3	15k	13.8k	1.2k

Observed values of cnDBTier Record counts:

Table 3-78 Database Records

DB Statistics	Read	Write	Commit
Site1	4.86k	14k	4.96k
Site2	4.66k	14.1k	4.76k
Site3	4.74k	14k	4.92k

3.8.4 Results

Table 3-79 Results

Attributes	Average Latency (ms)
Binding Create	6.6
Binding Delete	5.8
AAR-RX-INIT	7.57
RX-RAR	0.57
AAR-RX-UPDATE	0.63
RX-STR	0.58

Table 3-80 Average NF Latency

NF	50th Percentile (Site 1)	99th Percentile (Site 1)	50th Percentile (Site 2)	99th Percentile (Site 2)	50th Percentile (Site 3)	99th Percentile (Site 3)
Ingress Gateway	5.35 ms	11.7 ms	5.34 ms	18.5 ms	5.50 ms	10.2 ms
Egress Gateway	1.40 ms	1.53 ms	1.85 ms	1.99 ms	0.855 ms	1.15 ms
Diameter Gateway	-	0.126 ms	-	0.116 ms	-	0.346 ms

3.9 Test Scenario 9: BSF Performance Benchmarking 54K TPS traffic on a three-site Setup with traffic on one Site (25 Million Binding Sessions)

BSF Call Flow Model

BSF Performance Benchmark for 54K TPS on an ASM enabled three-site georedundant setup with traffic on one site.

The following image represents the BSF deployment on an ASM enabled three-site georedundant setup:

Figure 3-4 ASM enabled three-site georedundant setup

3.9.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-81 Testcase Parameters

Parameters	Values
Call Rate	54K TPS
Call Mix	4.5k nbsf Traffic and 50k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~25000000
Execution Time	~84 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-82 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	3
BSF Management Service	4	4	1	4	2	2	2	2	9
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	3
Diameter Gateway	4	4	2	2	2	2	2	2	8
Egress Gateway	4	4	4	6	2	2	2	2	3
Ingress Gateway	4	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	2
Query Service	2	2	1	1	2	2	2	2	3
Performance	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-83 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
ndbmgmd	3	3	10	10	2	2	2	2	2
ndbmtd	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-84 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-85 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	11	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	37 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.9.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-86 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	15.666 (48.96%)	8.421 (52.63%)	3.058 (19.11%)	2.431 (15.19%)	0.124 (0.39%)	5.824 (36.40%)	0.033 (0.21%)	2.325 (14.53%)	0.119 (0.37%)	5.782 (36.14%)	0.034 (0.21%)	2.298 (14.36%)
Perfinfo	0.130 (1.08%)	0.400 (13.35%)	0.012 (0.20%)	0.749 (12.48%)	0.126 (1.05%)	0.399 (13.31%)	0.012 (0.20%)	0.737 (12.29%)	0.111 (0.92%)	0.404 (13.48%)	0.011 (0.18%)	0.746 (12.43%)
Appinfo	0.055 (0.92%)	0.758 (25.26%)	0.010 (0.17%)	0.741 (12.35%)	0.054 (0.90%)	0.749 (24.97%)	0.012 (0.20%)	0.732 (12.21%)	0.054 (0.90%)	0.749 (24.97%)	0.010 (0.17%)	0.741 (12.35%)
cm-service	0.018 (0.15%)	2.175 (36.25%)	0.017 (0.28%)	0.830 (13.83%)	0.016 (0.13%)	2.037 (33.95%)	0.017 (0.28%)	0.793 (13.22%)	0.015 (0.12%)	2.148 (35.81%)	0.017 (0.28%)	0.796 (13.26%)
Config-server	0.110 (0.92%)	2.527 (42.12%)	0.093 (1.55%)	0.831 (13.85%)	0.100 (0.83%)	2.410 (40.17%)	0.082 (1.37%)-	0.805 (13.41%)	0.089 (0.74%)	2.488 (41.47%)	0.077 (1.28%)	0.794 (13.23%)
bsf-management-service	13.006 (36.13%)	22.446 (62.35%)	7.737 (42.98%)	4.297 (23.87%)	0.019 (0.05%)	14.439 (40.11%)	0.052 (0.29%)	2.616 (14.53%)	0.018 (0.05%)	15.061 (41.83%)	0.040 (0.22%)	2.666 (14.81%)
egress-gateway	0.008 (0.07%)	2.375 (13.19%)	0.010 (0.17%)	0.894 (14.89%)	0.006 (0.05%)	2.271 (12.61%)	0.013 (0.22%)	0.846 (14.10%)	0.007 (0.06%)	2.413 (13.41%)	0.011 (0.18%)	0.856 (14.27%)
ingress-gateway	3.271 (27.26%)	9.154 (50.86%)	2.084 (34.73%)	3.658 (60.97%)	0.034 (0.28%)	2.626 (14.59%)	0.012 (0.20%)	0.846 (14.10%)	0.027 (0.23%)	2.633 (14.63%)	0.012 (0.20%)	0.850 (14.16%)
nrf-client-nfmanagement	0.006 (0.30%)	0.982 (49.12%)	0.007 (0.18%)	0.532 (13.31%)	0.007 (0.35%)	0.959 (47.95%)	0.006 (0.15%)	0.534 (13.35%)	0.007 (0.35%)	0.964 (48.19%)	0.007 (0.18%)	0.531 (13.28%)
audit-service	0.004 (0.07%)	1.069 (35.64%)	0.025 (0.42%)	0.741 (12.35%)	0.005 (0.08%)	1.081 (36.04%)	0.008 (0.13%)	0.733 (12.22%)	0.003 (0.05%)	1.073 (35.77%)	0.008 (0.13%)	0.735 (12.26%)
queryservice	0.003 (0.05%)	0.703 (23.44%)	0.007 (0.12%)	0.746 (12.43%)	0.003 (0.05%)	0.706 (23.54%)	0.008 (0.13%)	0.743 (12.39%)	0.008 (0.13%)	0.737 (12.29%)	0.015 (0.12%)	2.148 (35.81%)

The following table provides information about observed values of cnDBTier services.

Table 3-87 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
ndbappmysqld	17.699 (73.75%)	14.835 (32.97%)	5.412 (60.13%)	1.071 (17.85%)	0.153 (0.64%)	12.697 (28.22%)	0.137 (1.52%)	1.031 (17.19%)	0.132 (0.55%)	12.756 (28.35%)	0.133 (1.48%)	1.074 (17.90%)
ndbmtd	17.231 (17.95%)	325.102 (63.50%)	14.501 (36.25%)	2.490 (15.56%)	5.671 (5.91%)	323.887 (63.26%)	1.745 (4.36%)	2.434 (15.21%)	6.554 (6.83%)	323.833 (63.25%)	1.640 (4.10%)	2.412 (15.08%)
ndbmysqld	0.526 (3.29%)	16.396 (25.62%)	0.328 (4.10%)	1.074 (13.43%)	0.405 (2.53%)	15.003 (23.44%)	0.280 (3.50%)	1.048 (13.10%)	0.365 (2.28%)	14.976 (23.40%)	0.290 (3.62%)	1.064 (13.31%)

3.9.3 TPS and DB Statistics

Table 3-88 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	54K	50K	4.5K
Site 2	-	-	-
Site 3	-	-	-

Observed values of cnDBTier Record counts:

Table 3-89 Database Records

DB Statistics	Read	Write	Commit
Site1	9.05K	6.10K	2.14K
Site2	-	-	-
Site3	-	-	-

3.9.4 Results

Table 3-90 Results

Attributes	Average Latency (ms)
Binding Create	16.1
Binding Delete	18.2
AAR-RX-INIT	16.9
RX-RAR	0.7
AAR-RX-UPDATE	0.6
RX-STR	0.4

3.10 Test Scenario 10: BSF Performance Benchmarking 54K TPS traffic on a three-site Setup with traffic on one Site (50 Million Binding Sessions)

BSF Call Flow Model

BSF Performance Benchmark for 54K TPS on an ASM enabled three-site georedundant setup with traffic on one site.

The following image represents the BSF deployment on an ASM enabled three-site georedundant setup:

Figure 3-5 ASM enabled three-site georedundant setup

3.10.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-91 Testcase Parameters

Parameters	Values
Call Rate	54K TPS
Call Mix	4.5k nbsf Traffic and 50k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~50000000
Execution Time	~15 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-92 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	3
BSF Management Service	4	4	1	4	2	2	2	2	9
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	3
Diameter Gateway	4	4	2	2	2	2	2	2	8
Egress Gateway	4	4	4	6	2	2	2	2	3
Ingress Gateway	4	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	2
Query Service	2	2	1	1	2	2	2	2	3
Performance	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-93 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
ndbmgmd	3	3	10	10	2	2	2	2	2
ndbmtd	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-94 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-95 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	11	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	37 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.10.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-96 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Istio Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	14.7 (45.9 %)	9.13 (53.2 %)	2.76 (17.2 %)	3.19 (18.6 %)	0.196 (0.615 %)	7.29 (42.4 %)	0.0368 ( 0.230 %)	2.90 (16.9 %)	0.191 (0.597 %)	7.01 (40 %)	0.0358 (0.224 %)	2.85 (16.6 %)
Perfinfo	0.114 (0.952 %)	0.439 (13.6 %)	0.0125 (0.209 %)	0.914 (14.2 %)	0.112 (0.940 %)	0.395 (12.2 %)	0.0119 (0.198 %)	0.932 (14.5 %)	0.112 (0.976 %)	0.395 (12.3 %)	0.0127 (0.211 %)	0.962 (14.9 %)
Appinfo	0.0492 (0.821 %)	0.814 (25.3 %)	0.00868 (0.145 %)	0.925 (14.4 %)	0.055 (0.891 %)	0.803 (24.9 %)	0.00806 (0.134 %)	0.926 (14.4 %)	0.0574 (0.953 %)	0.799 (24.8 %)	0.008 (0.133 %)	0.930 (14.4 %)
cm-service	0.013 (0.113 %)	1.99 (30.9 %)	0.0170 (0.284 %)	1.02 (15.9 %)	0.139 (0.115 %)	2.01 (3.11 %)	0.0153 (0.255 %)	1.04 (16.2 %)	0.0136 (0.114 %)	2.01 (31.3 %)	0.015 (0.25 %)	1.04 (16.1 %)
Config-server	0.102 (0.85 %)	2.47 (38.2 %)	0.0869 (1.45 %)	1.00 (15.6 %)	0.0914 (0.755 %)	2.56 (39.7 %)	0.0842 (1.40 %)	1.04 (16.1 %)	0.0890 (0.723%)	2.61 (40.6 %)	0.0811 (1.35 %)	1.02 (15.9 %)
bsf-management-service	13.1 (36.3 %)	26.9 (69.6 %)	7.83 (43.5 %)	3.84 (19.8 %)	0.0117 (0.0328 %)	15.1 (39.0 %)	0.0505 (0.280 %)	3.32 (17.2 %)	0.0116 (0.0849 %)	15.5 (40.0 %)	0.0455 (0.253 %)	3.35 (17.3 %)
egress-gateway	0.006 (0.05 %)	2.71 (14 %)	0.0113 (0.188 %)	1.07 (16.7 %)	0.00545 (0.0455 %)	2.16 (11.2 %)	0.0121 (0.202 %)	1.08 (16.7 %)	0.00573 (0.0478 %)	2.14 (11.1 %)	0.00936 (0.156 %)	1.07 (16.5 %)
ingress-gateway	3.35 (27.9 %)	8.37 (43.3 %)	2.17 (36.1 %)	1.65 (25.6 %)	0.0343 (0.285 %)	4.19 (21.7 %)	0.0172 (0.286 %)	1.08 (16.8 %)	0.0325 (0.271 %)	4.18 (21.6 %)	0.0150 (0.249 %)	1.09 (17.0 %)
nrf-client-nfmanagement	0.00540 (0.27 %)	1.05 (49 %)	0.00568 (0.142 %)	0.653 (15.2 %)	0.00558 (0.279 %)	1.02 (47.7 %)	0.00691 (0.173 %)	0.656 (15.3 %)	0.00571 (0.286 %)	1.02 (47.7 %)	0.00569 (0.142 %)	0.654 (15.2 %)
audit-service	0.00789 (0.132 %)	1.62 (50.3 %)	0.00661 (0.110 %)	0.933 (14.5 %)	0.00323 (0.0536 %)	1.46 (46.4 %)	0.00614 (0.102 %)	0.942 (14.6 %)	0.00325(0.0541 %)	1.49 (46.4 %)	0.00787 (0.131 %)	0.940 (14.6 %)
queryservice	0.00140 (0.0233 %)	0.745 (23.1 %)	0.00580 (0.0967 %)	0.921 (14.3 %)	0.00204 (0.0335 %)	0.810 (25.2 %)	0.00613 (0.102 %)	0.923 (14.3 %)	0.00163 (0.0273 %)	0.798 (24.8 %)	0.00644 (0.107 %)	0.926 (14.4 %)

The following table provides information about observed values of cnDBTier services.

Table 3-97 Observed Values of cnDBTier Services

Name	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
ndbappmysqld	17.6 (73.2 %)	14.3 (29.6 %)	5.54 (61.6 %)	1.19 (18.4 %)	0.141 (0.391 %)	8.22 (12.8 %)	0.131 (0.872 %)	1.19 (6.14 %)	0.139 (0.387 %)	8.19 (12.7 %)	0.132 (0.881 %)	1.18 (6.09 %)
ndbmtd	17.7 (73.9 %)	439 (79.9 %)	14.0 (156 %)	2.90 (45.1 %)	5.65 (15.7 %)	786 (91.5 %)	1.09 (7.27 %)	2.97 (15.3 %)	5.85 (16.2 %)	786 (91.4 %)	1.32 (8.80 %)	2.94 (15.2 %)
ndbmysqld	0.527 (0.371 %)	68.7 (9.98 %)	0.31 (0.530 %)	1.24 (3.40 %)	0.416 (0.270 %)	13.3 (1.31 %)	0.310 (0.414 %)	1.48 (2.55 %)	0.410 (0.267 %)	7.03 (0.694 %)	0.310 (0.414 %)	1.49 (2.57 %)

3.10.3 TPS and DB Statistics

Table 3-98 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	54K	50K	4.5K
Site 2	-	-	-
Site 3	-	-	-

Observed values of cnDBTier Record counts:

Table 3-99 Database Records

DB Statistics	Read	Write	Commit
Site1	9.05K	6.08K	2.13K
Site2	-	-	-
Site3	-	-	-

3.10.4 Results

Table 3-100 Results

Attributes	Average Latency (ms)
Binding Create	11.2
Binding Delete	12.8
AAR-RX-INIT	18.2
RX-RAR	0.6
AAR-RX-UPDATE	0.5
RX-STR	0.4

3.11 BSF Performance Benchmarking 54K TPS traffic on a Four-site Setup with traffic on one Site (Congestion Disabled)

BSF Call Flow Model

BSF Performance Benchmark for 54K TPS on an ASM enabled four-site georedundant setup with traffic on one site.

The following image represents the BSF deployment on an ASM enabled four-site georedundant setup:

Figure 3-6 ASM Enabled Four-site Georedundant Setup

3.11.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-101 Testcase Parameters

Parameters	Values
Call Rate	54K TPS
Call Mix	4.5k nbsf Traffic and 50k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~50000000
Execution Time	~72 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-102 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	3
BSF Management Service	4	4	1	4	2	2	2	2	9
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	3
Diameter Gateway	4	4	2	2	2	2	2	2	8
Egress Gateway	4	4	4	6	2	2	2	2	3
Ingress Gateway	4	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	2
Query Service	2	2	1	1	2	2	2	2	3
Performance	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-103 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
ndbmgmd	3	3	10	10	2	2	2	2	2
ndbmtd	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-104 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-105 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	11	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	37 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.11.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

BSF Microservices and their Resource Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory	Site4 CPU	Site4 Memory	Site4 Istio CPU	Site4 Istio Memory
diam-gateway	16.036 (50.11%)	10.645 (66.53%)	6.384 (39.90%)	2.750 (17.19%)	0.075 (0.47%)	3.136 (39.20%)	0.024 (0.30%)	1.155 (14.44%)	0.061 (0.38%)	2.508 (31.35%)	0.022 (0.27%)	1.136 (14.20%)	0.059 (0.37%)	2.328 (29.10%)	0.042 (0.53%)	1.191 (14.89%)
Perfinfo	0.115 (0.96%)	0.412 (13.74%)	0.012 (0.20%)	0.846 (14.10%)	0.083 (1.04%)	0.272 (13.62%)	0.009 (0.22%)	0.492 (12.30%)	0.013 (0.22%)	0.732 (12.21%)	0.002 (0.05%)	0.621 (31.05%)	0.002 (0.05%)	0.620 (31.01%)	0.005 (0.12%)	0.490 (12.26%)
Appinfo	0.056 (0.93%)	0.763 (25.42%)	0.011 (0.18%)	0.820 (13.67%)	0.038 (0.95%)	0.497 (24.85%)	0.007 (0.18%)	0.512 (12.79%)	0.034 (0.85%)	0.507 (25.34%)	0.007 (0.18%)	0.487 (12.18%)	0.028 (0.70%)	0.508 (25.39%)	0.007 (0.18%)	0.490 (12.26%)
cm-service	0.020 (0.17%)	2.216 (36.93%)	0.018 (0.30%)	0.888 (14.79%)	0.011 (0.14%)	1.257 (31.42%)	0.012 (0.30%)	0.547 (13.67%)	0.012 (0.10%)	1.860 (31.01%)	0.014 (0.23%)	0.814 (13.57%)	0.010 (0.12%)	1.399 (34.99%)	0.015 (0.38%)	0.564 (14.11%)
Config-server	0.115 (0.96%)	2.527 (42.12%)	0.085 (1.42%)	0.899 (14.99%)	0.056 (0.70%)	1.508 (37.70%)	0.045 (1.12%)	0.544 (13.60%)	0.047 (0.59%)	1.493 (37.33%)	0.037 (0.92%)	0.536 (13.40%)	0.053 (0.66%)	1.583 (39.58%)	0.047 (1.18%)	0.553 (13.82%)
bsf-management-service	13.145 (36.51%)	23.284 (64.68%)	7.855 (43.64%)	3.112 (17.29%)	0.093 (0.58%)	4.022 (25.14%)	0.056 (0.70%)	1.182 (14.77%)	0.007 (0.04%)	4.583 (28.64%)	0.016 (0.20%)	1.174 (14.67%)	0.008 (0.05%)	3.684 (23.02%)	0.022 (0.27%)	1.189 (14.87%)
egress-gateway	0.095 (0.79%)	2.789 (15.49%)	0.039 (0.65%)	0.990 (16.50%)	0.085 (1.06%)	1.696 (14.14%)	0.031 (0.78%)	0.581 (14.53%)	0.006 (0.07%)	1.762 (14.68%)	0.008 (0.20%)	0.573 (14.33%)	['0.006 (0.07%)	1.861 (15.51%)	0.009 (0.22%)	0.589 (14.72%)
ingress-gateway	3.301 (27.51%)	6.412 (35.62%)	2.006 (33.43%)	1.103 (18.38%)	0.026 (0.33%)	1.909 (15.91%)	0.010 (0.25%)	0.571 (14.28%)	0.023 (0.29%)	1.875 (15.62%)	0.012 (0.30%)	0.562 (14.04%)	['0.028 (0.35%)	1.904 (15.87%)	0.016 (0.40%)	0.571 (14.28%)
nrf-client-nfmanagement	0.006 (0.30%)	1.006 (50.29%)	0.007 (0.18%)	0.684 (17.09%)	0.007 (0.35%)	0.985 (49.27%)	0.007 (0.18%)	0.526 (13.16%)	0.007 (0.35%)	0.994 (49.71%)	0.006 (0.15%)	0.532 (13.31%)	['0.006 (0.30%)	['0.973 (48.63%)	0.006 (0.15%)	['0.533 (13.33%)
audit-service	0.024 (0.40%)	1.235 (41.18%)	0.025 (0.42%)	0.854 (14.23%)	0.020 (0.50%)	0.862 (43.12%)	0.025 (0.62%)	0.508 (12.70%)	0.004 (0.10%)	0.729 (36.43%)	0.007 (0.18%)	0.496 (12.40%)	['0.004 (0.10%)	['0.711 (35.55%)	['0.008 (0.20%)	['0.490 (12.26%)
queryservice	0.003 (0.05%)	0.923 (30.76%)	0.007 (0.12%)	0.819 (13.66%)	0.002 (0.05%)	0.615 (30.76%)	0.005 (0.12%)	0.498 (12.45%)	0.002 (0.05%)	0.621 (31.05%)	0.006 (0.15%)	0.493 (12.33%)	['0.002 (0.05%)	0.620 (31.01%)	0.005 (0.12%)	0.490 (12.26%)

Observed Values of cnDBTier Services

The following table provides information about observed values of cnDBTier services.

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory	Site4 CPU	Site4 Memory	Site4 Istio CPU	Site4 Istio Memory
ndbappmysqld	17.826 (74.28%)	14.117 (31.37%)	6.909 (76.77%)']	1.322 (22.04%)	0.152 (0.63%)	12.361 (27.47%)	0.110 (1.22%)	1.293 (21.55%)	0.110 (1.22%)	1.293 (21.55%)	0.083 (0.92%)	1.249 (20.82%)	0.093 (0.39%)	11.559 (25.69%)	0.081 (0.90%)	1.237 (20.62%)
ndbmtd	17.960 (18.71%)	324.243 (63.33%)	16.106 (40.27%)	3.091 (19.32%)	7.353 (7.66%)	324.522 (63.38%)	1.617 (4.04%)	3.116 (19.48%)	7.636 (7.95%)']	324.854 (63.45%)	1.695 (4.24%)	3.123 (19.52%)	7.305 (7.61%)	324.876 (63.45%)	1.599 (4.00%)	3.191 (19.95%)
ndbmysqld	0.818 (3.41%)	24.810 (25.84%)	0.513 (4.28%)	1.938 (16.15%)	0.487 (2.03%)	22.604 (23.55%)	['0.383 (3.19%)	1.972 (16.43%)	0.466 (1.94%)	22.377 (23.31%)	0.407 (3.39%)	1.916 (15.97%)	0.467 (1.95%)	22.298 (23.23%)	0.400 (3.33%)	1.938 (16.15%)

3.11.3 TPS and DB Statistics

Table 3-106 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	54K	50K	4.5K
Site 2	-	-	-
Site 3	-	-	-
Site 4	-	-	-

Observed values of cnDBTier Record counts:

Table 3-107 Database Records

DB Statistics	Read	Write	Commit
Site1	9.05K	6.25K	2.25K
Site2	-	-	-
Site3	-	-	-
Site4	-	-	-

Replication Delay

Observed values of latency on replication channels:

DB Statistics	Latency Min (Seconds)	Latency Max (Seconds)
Site1	0	0
Site2	0	1
Site3	0	1
Site4	0	1

3.11.4 Results

Table 3-108 Results

Attributes	Average Latency (ms)
Binding Create	8.3
Binding Delete	8.1
AAR-RX-INIT	10.4
RX-RAR	0.9
AAR-RX-UPDATE	0.5
RX-STR	0.5

3.12 BSF Performance Benchmarking 54K TPS traffic on a Four-site Setup with traffic on one Site (Congestion Enabled)

BSF Call Flow Model

BSF Performance Benchmark for 54K TPS on an ASM enabled four-site georedundant setup with traffic on one site.

The following image represents the BSF deployment on an ASM enabled four-site georedundant setup:

Figure 3-7 ASM Enabled Four-site Georedundant Setup

3.12.1 Test Case and Setup Details

The following table describes the testcase parameters and their values:

Table 3-109 Testcase Parameters

Parameters	Values
Call Rate	54K TPS
Call Mix	4.5k nbsf Traffic and 50k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Binding Sessions	~50000000
Execution Time	~12 Hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-110 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod	Memory Limit Per Pod	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	3
BSF Management Service	4	4	1	4	2	2	2	2	9
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	3
Diameter Gateway	4	4	2	2	2	2	2	2	8
Egress Gateway	4	4	4	6	2	2	2	2	3
Ingress Gateway	4	4	4	6	2	2	2	2	3
Nrf Client Management	1	1	1	1	2	2	2	2	2
Query Service	2	2	1	1	2	2	2	2	3
Performance	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-111 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replica Count
ndbmgmd	3	3	10	10	2	2	2	2	2
ndbmtd	12	12	100	100	5	5	2	2	8
mysqld	4	4	16	16	4	4	4	4	4
appmysqld	12	12	20	20	5	5	6	6	3

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-112 BSF Features

Features Name	Feature Status
Nbsf	Enabled
Rx	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-113 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	11	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	37 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.12.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

BSF Microservices and their Resource Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF Microservices.

Table 3-114 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory	Site4 CPU	Site4 Memory	Site4 Istio CPU	Site4 Istio Memory
diam-gateway	16.405 (51.27%)	9.913 (61.96%)	6.686 (41.79%)	2.468 (15.42%)	0.074 (0.46%)	3.385 (42.31%)	0.056 (0.70%)	1.168 (14.60%)	0.061 (0.38%)	2.771 (34.64%)	0.024 (0.30%)	1.157 (14.47%)	0.058 (0.36%)	2.665 (33.31%)	0.028 (0.35%)	1.227 (15.33%)
Perfinfo	0.113 (0.94%)	0.416 (13.87%)	0.013 (0.22%)	0.851 (14.18%)	0.086 (1.07%)	0.273 (13.67%)	0.009 (0.22%)	0.500 (12.50%)	0.106 (0.88%)	0.400 (13.35%)	0.012 (0.20%)	0.737 (12.29%)	0.071 (0.89%)	0.271 (13.57%)	0.010 (0.25%)	0.532 (13.31%)
Appinfo	0.052 (0.87%)	0.764 (25.46%)	0.012 (0.20%)	0.818 (13.64%)	0.030 (0.75%)	0.497 (24.85%)	0.007 (0.18%)	0.503 (12.57%)	0.032 (0.80%)	0.507 (25.34%)	0.006 (0.15%)	0.499 (12.48%)	0.031 (0.78%)	0.510 (25.49%)	0.007 (0.18%)	0.505 (12.62%)
cm-service	0.018 (0.15%)	2.307 (38.44%)	0.019 (0.32%)	0.893 (14.88%)	0.011 (0.14%)	1.282 (32.06%)	0.011 (0.27%)	0.551 (13.77%)	0.013 (0.11%)	1.897 (31.62%)	0.015 (0.25%)	0.809 (13.48%)	0.017 (0.43%)	0.564 (14.11%)	0.004 (0.10%)	0.721 (36.04%)
Config-server	0.119 (0.99%)	2.448 (40.80%)	0.087 (1.45%)	0.892 (14.86%)	0.064 (0.80%)	1.514 (37.84%)	0.050 (1.25%)	0.552 (13.79%)	0.047 (0.59%)	1.801 (45.02%)	0.038 (0.95%)	0.549 (13.72%)	0.049 (0.61%)	1.827 (45.68%)	0.045 (1.12%)	0.570 (14.26%)
bsf-management-service	13.320 (37.00%)	22.150 (61.53%)	7.895 (43.86%)	3.896 (21.64%)	0.094 (0.59%)	4.045 (25.28%)	0.063 (0.79%)	1.185 (14.81%)	0.007 (0.04%)	7.717 (48.23%)	0.018 (0.22%)	1.194 (14.93%)	0.008 (0.05%)	7.717 (48.23%)	0.022 (0.27%)	1.206 (15.08%)
egress-gateway	0.092 (0.77%)	2.934 (16.30%)	0.039 (0.65%)	0.963 (16.05%)	0.084 (1.05%)	1.923 (16.02%)	0.033 (0.83%)	0.590 (14.75%)	0.006 (0.15%)	0.583 (14.58%)	0.047 (0.59%)	1.801 (45.02%)	0.012 (0.30%)	0.583 (14.58%)	0.049 (0.61%)	1.827 (45.68%)
ingress-gateway	3.281 (27.34%)	9.870 (54.83%)	2.022 (33.70%)	1.447 (24.12%)	0.027 (0.34%)	1.932 (16.10%)	0.015 (0.38%)	0.577 (14.43%)	0.022 (0.27%)	1.888 (15.73%)	0.010 (0.25%)	0.573 (14.33%)	0.012 (0.30%)	0.584 (14.60%)	0.006 (0.07%)	1.872 (15.60%)
nrf-client-nfmanagement	0.007 (0.35%)	1.013 (50.63%)	0.006 (0.15%)	0.686 (17.14%)	0.006 (0.30%)	0.998 (49.90%)	0.007 (0.18%)	0.530 (13.26%)	0.006 (0.15%)	0.535 (13.38%)	0.022 (0.27%)	1.888 (15.73%)	0.022 (0.27%)	1.923 (16.02%)	0.012 (0.30%)	0.584 (14.60%)
audit-service	0.023 (0.38%)	1.254 (41.80%)	0.023 (0.38%)	0.851 (14.18%)	0.019 (0.47%)	0.872 (43.60%)	0.022 (0.55%)	0.513 (12.82%)	0.004 (0.10%)	0.735 (36.77%)	0.007 (0.18%)	0.506 (12.65%)	0.004 (0.10%)	0.721 (36.04%)	0.006 (0.15%)	0.505 (12.62%)
queryservice	0.003 (0.05%)	0.931 (31.02%)	0.008 (0.13%)	0.809 (13.48%)	0.002 (0.05%)	0.623 (31.15%)	0.004 (0.10%)	0.502 (12.55%)	0.002 (0.05%)	0.615 (30.76%)	0.005 (0.12%)	0.490 (12.26%)	0.005 (0.12%)	0.499 (12.48%)	0.011 (0.14%)	1.462 (36.55%)

Observed Values of cnDBTier Services

The following table provides information about observed values of cnDBTier services.

Table 3-115 Observed Values of cnDBTier Services

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory	Site4 CPU	Site4 Memory	Site4 Istio CPU	Site4 Istio Memory
ndbappmysqld	18.292 (76.22%)	'14.170 (31.49%)	6.909 (76.77%)	1.316 (21.94%)	0.162 (0.67%)	12.379 (27.51%)	0.117 (1.30%)	1.255 (20.91%)	0.098 (0.41%)	12.281 (27.29%)	0.090 (1.00%)	1.246 (20.77%)	0.093 (0.39%)	11.750 (26.11%)	0.102 (1.13%)	1.252 (20.87%)
ndbmtd	19.268 (20.07%)	324.049 (63.29%)	16.136 (40.34%)	3.081 (19.26%)	6.304 (6.57%)	324.378 (63.36%)	1.622 (4.06%)	3.139 (19.62%)	7.216 (7.52%)	324.667 (63.41%)	1.684 (4.21%)	3.124 (19.53%)	6.931 (7.22%)	321.891 (62.87%)	1.627 (4.07%)	3.175 (19.84%)
ndbmysqld	0.815 (3.40%)	24.730 (25.76%)	0.512 (4.27%)	1.944 (16.20%)	0.486 (2.02%)	22.572 (23.51%)	0.381 (3.18%)	1.978 (16.48%)	0.465 (1.94%)	22.442 (23.38%)	0.408 (3.40%)	1.912 (15.93%)	0.456 (1.90%)	22.481 (23.42%)	0.403 (3.36%)	1.928 (16.06%)

3.12.3 TPS and DB Statistics

Table 3-116 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	54K	50K	4.5K
Site 2	-	-	-
Site 3	-	-	-
Site 4	-	-	-

Observed values of cnDBTier Record counts:

Table 3-117 Database Records

DB Statistics	Read	Write	Commit
Site1	8.63K	6.212K	2.3K
Site2	-	-	-
Site3	-	-	-
Site4	-	-	-

Replication Delay

Observed values of latency on replication channels:

DB Statistics	Latency Min (Seconds)	Latency Max (Seconds)
Site1	0	0
Site2	0	1
Site3	0	0
Site4	0	1

3.12.4 Results

Table 3-118 Results

Attributes	Average Latency (ms)
Binding Create	8.2
Binding Delete	8.0
AAR-RX-INIT	10.8
RX-RAR	0.9
AAR-RX-UPDATE	0.5
RX-STR	0.5

3.13 BSF Call Model Deployed on a Three-site Setup with 63K TPS Traffic on One Site and ASM Enabled

This test run benchmarks the performance and capacity of BSF call model that is deployed on a three-site setup with 63K TPS traffic on one site and Aspen Service Mesh (ASM) enabled.

3.13.1 Test Case and Setup Details

The following table describes the test case parameters and their values:

Table 3-119 Test Case Parameters

Parameters	Values
Call Rate	63K TPS
Call Mix	8.43K nbsf traffic and 54.7K Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Binding Get ratio (Discovery) - 0.15 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Subscribers	32M subscribers and 64M sessions
Execution Time	~60 hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-120 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	3
BSF Management Service	4	4	1	4	2	2	2	2	14
Alternate Route Service	2	2	2	4	2	2	2	2	3
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	3
Diameter Gateway	4	4	2	2	2	2	2	2	15
Egress Gateway	4	4	4	6	2	2	2	2	3
Ingress Gateway	4	4	4	6	2	2	2	2	7
Nrf Client Management	1	1	1	1	2	2	2	2	3
Query Service	2	2	1	1	2	2	2	2	3
Perf-info	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-121 cnDBTier services resource allocation

Microservice Name	CPU Request per Pod	CPU Limit Per Pod	Memory Request per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
ndbmgmd	3	3	10	10	2	2	2	2	2
ndbmtd	12	12	100	100	5	5	2	2	12
mysqld	4	4	16	16	4	4	4	4	6
appmysqld	12	12	20	20	5	5	6	6	9
db-monitor-service	0.2	0.2	0.5	0.5	2	2	2	2	1
db-replication-service	0.2	0.2	12	12	2	2	2	2	3
db-backup-service	0.1	0.1	0.128	0.128	0	0	0	0	1

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-122 BSF Features

Features Name	Feature Status
Audit	Enabled
Overload	Enabled
dnssrv	Enabled
Contolled Shutdown	Enabled
NF Scoring	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-123 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	13	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	51 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 4}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.13.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF microservices.

BSF Microservices and their Resource Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF microservices.

Table 3-124 BSF Microservices and their Resource Utilization

Service	Site1 CPU	Site1 Memory	Site1 Istio CPU	Site1 Istio Memory	Site2 CPU	Site2 Memory	Site2 Istio CPU	Site2 Istio Memory	Site3 CPU	Site3 Memory	Site3 Istio CPU	Site3 Istio Memory
diam-gateway	21.271 (35.45%)	15.829 (52.76%)	3.875 (12.92%)	3.907 (13.02%)	-	-	-	-	-	-	-	-
Perfinfo	0.122 (1.02%)	0.431 (14.36%	0.016 (0.27%)	0.755 (12.58%)	-	-	-	-	-	-	-	-
Appinfo	0.070 (1.17%)	0.814 (27.15%)	0.012 (0.20%)	0.753 (12.55%)	-	-	-	-	-	-	-	-
cm-service	0.027 (0.23%)	2.254 (37.57%)	0.027 (0.45%)	0.780 (13.00%)	-	-	-	-	-	-	-	-
Config-server	0.198 (1.65%)	1.298 (21.63%)	0.189 (3.15%)	0.691 (11.52%)	-	-	-	-	-	-	-	-
bsf-management-service	19.866 (35.48%)	22.906 (40.90%)	12.000 (42.86%	4.305 (15.37%)	-	-	-	-	-	-	-	-
egress-gateway	0.013 (0.11%)	2.316 (12.87%)	0.012 (0.20%)	0.813 (13.56%)	-	-	-	-	-	-	-	-
ingress-gateway	7.079 (25.28%)	11.938 (28.42%)	4.674 (33.39%)	2.415 (17.25%)	-	-	-	-	-	-	-	-
nrf-client-nfmanagement	0.014 (0.47%)	1.431 (47.69%)	0.010 (0.17%)	0.763 (12.71%)	-	-	-	-	-	-	-	-
audit-service	0.016 (0.27%)	1.121 (37.37%)	0.038 (0.63%)	0.729 (12.16%)	-	-	-	-	-	-	-	-
queryservice	0.003 (0.05%)	0.933 (31.09%)	0.006 (0.10%)	0.736 (12.27%)	-	-	-	-	-	-	-	-

Observed Values of cnDBTier Services

The following table provides information about observed values of cnDBTier services.

Table 3-125 cnDBTier Services and their Resource Utilization

Services	Site1 CPU	Site1 Memory	Site2 CPU	Site2 Memory	Site3 CPU	Site3 Memory
mysql-cluster-db-db-backup-manager-svc/db-backup-manager-svc	['0.002 (2.00%)']	['0.086 (68.75%)']	['0.007 (7.00%)']	['0.087 (69.53%)']	['0.001 (1.00%)']	['0.091 (72.66%)']
mysql-cluster-db-db-monitor-svc/istio-proxy	['0.005 (0.10%)']	['0.259 (5.18%)']	['0.004 (0.08%)']	['0.254 (5.08%)']	['0.004 (0.08%)']	['0.259 (5.18%)']
mysql-cluster-db-db-monitor-svc/db-monitor-svc	['0.004 (2.00%)']	['0.346 (70.80%)']	['0.015 (7.50%)']	['0.334 (68.40%)']	['0.028 (14.00%)']	['0.329 (67.40%)']
mysql-cluster-db-two-four-replication-svc/istio-proxy	None	None	['0.003 (0.06%)']	['0.250 (5.00%)']	None	None
mysql-cluster-db-two-four-replication-svc/two-four-replication-svc	None	None	['0.006 (0.30%)']	['0.278 (2.32%)']	None	None
mysql-cluster-db-two-one-replication-svc/istio-proxy	None	None	['0.003 (0.15%)']	['0.248 (12.40%)']	None	None
mysql-cluster-db-two-one-replication-svc/two-one-replication-svc	None	None	['0.005 (0.25%)']	['0.295 (2.46%)']	None	None
mysql-cluster-db-two-one-replication-svc/db-infra-monitor-svc	None	None	['0.001 (1.00%)']	['0.052 (20.70%)']	None	None
mysql-cluster-db-two-three-replication-svc/istio-proxy	None	None	['0.004 (0.08%)']	['0.249 (4.98%)']	None	None
mysql-cluster-db-two-three-replication-svc/two-three-replication-svc	None	None	['0.006 (0.30%)']	['0.292 (2.43%)']	None	None
ndbappmysqld/istio-proxy	['11.494 (25.54%)']	['2.851 (6.33%)']	['0.119 (0.26%)']	['2.694 (5.99%)']	['0.112 (0.25%)']	['2.681 (5.96%)']
ndbappmysqld/mysqlndbcluster	['22.608 (31.40%)']	['36.985 (27.40%)']	['0.110 (0.15%)']	['32.576 (24.13%)']	['0.110 (0.15%)']	['32.562 (24.12%)']
ndbappmysqld/db-infra-monitor-svc	['0.011 (1.22%)']	['0.487 (21.66%)']	['0.011 (1.22%)']	['0.488 (21.70%)']	['0.014 (1.56%)']	['0.493 (21.92%)']
ndbappmysqld/init-sidecar	['0.018 (2.00%)']	['0.009 (0.39%)']	['0.018 (2.00%)']	['0.009 (0.39%)']	['0.018 (2.00%)']	['0.009 (0.39%)']
ndbmgmd/istio-proxy	['0.014 (0.70%)']	['0.503 (12.57%)']	['0.014 (0.70%)']	['0.504 (12.60%)']	['0.014 (0.70%)']	['0.507 (12.67%)']
ndbmgmd/mysqlndbcluster	['0.011 (0.18%)']	['0.095 (0.47%)']	['0.010 (0.17%)']	['0.093 (0.46%)']	['0.011 (0.18%)']	['0.095 (0.47%)']
ndbmgmd/db-infra-monitor-svc	['0.002 (1.00%)']	['0.099 (19.73%)']	['0.002 (1.00%)']	['0.100 (19.92%)']	['0.002 (1.00%)']	['0.101 (20.12%)']
ndbmtd/istio-proxy	['25.237 (42.06%)']	['3.642 (6.07%)']	['1.611 (2.69%)']	['3.644 (6.07%)']	['4.060 (6.77%)']	['3.672 (6.12%)']
ndbmtd/mysqlndbcluster	['42.015 (29.18%)']	['677.226 (88.18%)']	['9.079 (6.30%)']	['677.260 (88.18%)']	['11.299 (7.85%)']	['677.258 (88.18%)']
ndbmtd/db-backup-executor-svc	['0.012 (1.00%)']	['0.656 (43.75%)']	['0.012 (1.00%)']	['0.658 (43.88%)']	['0.012 (1.00%)']	['0.659 (43.95%)']
ndbmtd/db-infra-monitor-svc	['0.087 (7.25%)']	['0.628 (20.93%)']	['0.066 (5.50%)']	['0.627 (20.90%)']	['0.045 (3.75%)']	['0.632 (21.06%)']
ndbmysqld/istio-proxy	['0.676 (2.25%)']	['1.791 (5.97%)']	['0.473 (1.58%)']	['1.795 (5.98%)']	['0.478 (1.59%)']	['1.820 (6.07%)']
ndbmysqld/mysqlndbcluster	['0.977 (4.07%)']	['25.205 (26.26%)']	['0.582 (2.42%)']	['22.083 (23.00%)']	['0.599 (2.50%)']	['22.086 (23.01%)']
ndbmysqld/init-sidecar	['0.012 (2.00%)']	['0.009 (0.59%)']	['0.012 (2.00%)']	['0.007 (0.46%)']	['0.013 (2.17%)']	['0.007 (0.46%)']
ndbmysqld/db-infra-monitor-svc	['0.010 (1.67%)']	['0.371 (24.74%)']	['0.013 (2.17%)']	['0.385 (25.65%)']	['0.013 (2.17%)']	['0.365 (24.35%)']
mysql-cluster-db-three-four-replication-svc/istio-proxy	None	None	None	None	['0.003 (0.06%)']	['0.256 (5.12%)']
mysql-cluster-db-three-four-replication-svc/three-four-replication-svc	None	None	None	None	['0.007 (0.35%)']	['0.284 (2.37%)']
mysql-cluster-db-three-one-replication-svc/istio-proxy	None	None	None	None	['0.003 (0.15%)']	['0.250 (12.50%)']
mysql-cluster-db-three-one-replication-svc/three-one-replication-svc	None	None	None	None	['0.007 (0.35%)']	['0.274 (2.29%)']
mysql-cluster-db-three-one-replication-svc/db-infra-monitor-svc	None	None	None	None	['0.001 (1.00%)']	['0.051 (20.31%)']
mysql-cluster-db-three-two-replication-svc/istio-proxy	None	None	None	None	['0.003 (0.06%)']	['0.248 (4.96%)']
mysql-cluster-db-three-two-replication-svc/three-two-replication-svc	None	None	None	None	['0.005 (0.25%)']	['0.284 (2.37%)']
mysql-cluster-db-one-four-replication-svc/istio-proxy	['0.004 (0.08%)']	['0.252 (5.04%)']	None	None	None	None
mysql-cluster-db-one-four-replication-svc/one-four-replication-svc	['0.005 (0.25%)']	['0.271 (2.26%)']	None	None	None	None
mysql-cluster-db-one-three-replication-svc/istio-proxy	['0.003 (0.06%)']	['0.264 (5.27%)']	None	None	None	None
mysql-cluster-db-one-three-replication-svc/one-three-replication-svc	['0.006 (0.30%)']	['0.293 (2.44%)']	None	None	None	None
mysql-cluster-db-one-two-replication-svc/istio-proxy	['0.003 (0.15%)']	['0.262 (13.09%)']	None	None	None	None
mysql-cluster-db-one-two-replication-svc/one-two-replication-svc	['0.012 (0.60%)']	['0.280 (2.34%)']	None	None	None	None
mysql-cluster-db-one-two-replication-svc/db-infra-monitor-svc	['0.001 (1.00%)']	['0.052 (20.70%)']	None	None	None	None

3.13.3 TPS and DB Statistics

Table 3-126 Overall TPS Ingress & Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	63.1K	54.7K	8.43K
Site 2	-	-	-
Site 3	-	-	-

Observed values of cnDBTier Record counts:

Table 3-127 Database Records

DB Statistics	Read	Write	Commit
Site1	7.298K	5.254K	1.934K
Site2	2.387K	7.244K	0.037K
Site3	1.834K	5.569K	0.059K

3.13.4 Results

Table 3-128 Average Latency Observations (in milliseconds)

NF service	Average Latency (ms) Site1
Ingress	7.420
Egress	0.881
Diameter Gateway	0.355
BSF Management	6.070

Table 3-129 Average Current Percentile Latency Observations (in milliseconds)

NF	50th Percentile (Site 1)	99th Percentile (Site 1)
Ingress Gateway	7.15	29.2
Diameter Gateway	-	5.650

3.14 BSF Call Model Deployed on a Four-site Setup with 22K TPS Traffic on One Site and ASM Enabled

This test run benchmarks the performance and capacity of BSF call model that is deployed on a four-site setup with 22K TPS traffic on one site and Aspen Service Mesh (ASM) enabled.

The following image represents the BSF deployment on an ASM enabled four-site georedundant setup:

Figure 3-8 ASM Enabled Four-site Georedundant Setup

3.14.1 Test Case and Setup Details

The following table describes the test case parameters and their values:

Table 3-130 Test Case Parameters

Parameters	Values
Call Rate	22K TPS
Call Mix	14k nbsf traffic and 8k Rx traffic
Traffic Ratio	Binding Create ratio -1 Binding Delete ratio - 1 Rx AAR-I ratio - 1 Rx RAR ratio - 1 Rx AAR-U ratio - 1 Rx STR ratio - 1
Active Subscribers	7M subscribers and 14M sessions
Execution Time	~10 hours

The following table describes the resource capacity allocated to the BSF microservices:

Table 3-131 BSF microservices Resource allocation

Service Name	CPU Request Per Pod	CPU Limit Per Pod	Memory Request Per Pod (Gi)	Memory Limit Per Pod (Gi)	Isito CPU Request	Isito CPU Limit	Isito Memory Request (Gi)	Isito Memory Limit (Gi)	Replicas
Alternate Route Service	2	2	2	4	2	2	2	2	2
Appinfo	2	2	1	1	2	2	2	2	3
Audit Service	2	2	1	1	2	2	2	2	2
BSF Management Service	4	4	1	4	2	2	2	2	11
CM Service	4	4	1	2	2	2	2	2	3
Config Service	4	4	1	2	2	2	2	2	2
Diameter Gateway	4	4	1	2	2	2	2	2	2
Egress Gateway	4	4	4	6	2	2	2	2	2
Ingress Gateway	4	4	4	6	2	2	2	2	9
Nrf Client Management	1	1	1	1	2	2	2	2	2
Query Service	2	2	1	1	2	2	2	2	3
Perf-info	4	4	1	1	2	2	2	2	3

The following table describes the resource capacity allocated to the cnDBTier microservices:

Table 3-132 cnDBTier microservices Resource Allocation - Site 1

Microservices	Replicas	CPU Limit Per Container	CPU Request Per Container	Memory Limit Per Container	Memory Request Per Container
mysql-cluster-db-db-backup-manager-svc/db-backup-manager-svc	1	100m	100m	128Mi	128Mi
mysql-cluster-db-db-monitor-svc/db-monitor-svc	1	200m	200m	500Mi	500Mi
mysql-cluster-db-db-monitor-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-one-four-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-one-four-replication-svc/one-four-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-one-three-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-one-three-replication-svc/one-three-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-one-two-replication-svc/db-infra-monitor-svc	1	100m	100m	256Mi	256Mi
mysql-cluster-db-one-two-replication-svc/istio-proxy	1	2	2	2Gi	2Gi
mysql-cluster-db-one-two-replication-svc/one-two-replication-svc	1	2	2	12Gi	12Gi
ndbappmysqld/db-infra-monitor-svc	9	100m	100m	256Mi	256Mi
ndbappmysqld/init-sidecar	9	100m	100m	256Mi	256Mi
ndbappmysqld/istio-proxy	9	5	5	5Gi	5Gi
ndbappmysqld/mysqlndbcluster	9	8	8	15Gi	15Gi
ndbmgmd/db-infra-monitor-svc	2	100m	100m	256Mi	256Mi
ndbmgmd/istio-proxy	2	1	1	2Gi	2Gi
ndbmgmd/mysqlndbcluster	2	3	3	10Gi	10Gi
ndbmtd/db-backup-executor-svc	12	100m	100m	128Mi	128Mi
ndbmtd/db-infra-monitor-svc	12	100m	100m	256Mi	256Mi
ndbmtd/istio-proxy	12	5	5	5Gi	5Gi
ndbmtd/mysqlndbcluster	12	12	12	64Gi	64Gi
ndbmysqld/db-infra-monitor-svc	6	100m	100m	256Mi	256Mi
ndbmysqld/init-sidecar	6	100m	100m	256Mi	256Mi
ndbmysqld/istio-proxy	6	5	5	5Gi	5Gi
ndbmysqld/mysqlndbcluster	6	4	4	16Gi	16Gi

Table 3-133 cnDBTier microservices Resource Allocation - Site 2

Microservices	Replicas	CPU Limit Per Container	CPU Request Per Container	Memory Limit Per Container	Memory Request Per Container
ndbmtd/istio-proxy	12	5	5	5Gi	5Gi
ndbmtd/mysqlndbcluster	12	12	12	64Gi	64Gi
ndbmtd/db-backup-executor-svc	12	100m	100m	128Mi	128Mi
ndbmtd/db-infra-monitor-svc	12	100m	100m	256Mi	256Mi
ndbmysqld/istio-proxy	6	5	5	5Gi	5Gi
ndbmysqld/mysqlndbcluster	6	4	4	16Gi	16Gi
ndbmysqld/init-sidecar	6	100m	100m	256Mi	256Mi
ndbmysqld/db-infra-monitor-svc	6	100m	100m	256Mi	256Mi
mysql-cluster-db-db-backup-manager-svc/db-backup-manager-svc	1	100m	100m	128Mi	128Mi
mysql-cluster-db-db-monitor-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-db-monitor-svc/db-monitor-svc	1	200m	200m	500Mi	500Mi
mysql-cluster-db-two-four-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-two-four-replication-svc/two-four-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-two-one-replication-svc/istio-proxy	1	2	2	2Gi	2Gi
mysql-cluster-db-two-one-replication-svc/two-one-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-two-one-replication-svc/db-infra-monitor-svc	1	100m	100m	256Mi	256Mi
mysql-cluster-db-two-three-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-two-three-replication-svc/two-three-replication-svc	1	2	2	12Gi	12Gi
ndbappmysqld/istio-proxy	9	5	5	5Gi	5Gi
ndbappmysqld/mysqlndbcluster	9	8	8	15Gi	15Gi
ndbappmysqld/db-infra-monitor-svc	9	100m	100m	256Mi	256Mi
ndbappmysqld/init-sidecar	9	100m	100m	256Mi	256Mi
ndbmgmd/istio-proxy	2	1	1	2Gi	2Gi
ndbmgmd/mysqlndbcluster	2	3	3	10Gi	10Gi
ndbmgmd/db-infra-monitor-svc	2	100m	100m	256Mi	256Mi

Table 3-134 cnDBTier microservices Resource Allocation - Site 3

Microservices	Replicas	CPU Limit Per Container	CPU Request Per Container	Memory Limit Per Container	Memory Request Per Container
ndbappmysqld/istio-proxy	9	5	5	5Gi	5Gi
ndbappmysqld/mysqlndbcluster	9	8	8	15Gi	15Gi
ndbappmysqld/db-infra-monitor-svc	9	100m	100m	256Mi	256Mi
ndbappmysqld/init-sidecar	9	100m	100m	256Mi	256Mi
ndbmgmd/istio-proxy	2	1	1	2Gi	2Gi
ndbmgmd/mysqlndbcluster	2	3	3	10Gi	10Gi
ndbmgmd/db-infra-monitor-svc	2	100m	100m	256Mi	256Mi
ndbmtd/istio-proxy	12	5	5	5Gi	5Gi
ndbmtd/mysqlndbcluster	12	12	12	64Gi	64Gi
ndbmtd/db-backup-executor-svc	12	100m	100m	128Mi	128Mi
ndbmtd/db-infra-monitor-svc	12	100m	100m	256Mi	256Mi
ndbmysqld/istio-proxy	6	5	5	5Gi	5Gi
ndbmysqld/mysqlndbcluster	6	4	4	16Gi	16Gi
ndbmysqld/init-sidecar	6	100m	100m	256Mi	256Mi
ndbmysqld/db-infra-monitor-svc	6	100m	100m	256Mi	256Mi
mysql-cluster-db-db-backup-manager-svc/db-backup-manager-svc	1	100m	100m	128Mi	128Mi
mysql-cluster-db-db-monitor-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-db-monitor-svc/db-monitor-svc	1	200m	200m	500Mi	500Mi
mysql-cluster-db-three-four-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-three-four-replication-svc/three-four-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-three-one-replication-svc/istio-proxy	1	2	2	2Gi	2Gi
mysql-cluster-db-three-one-replication-svc/three-one-replication-svc	1	2	2	12Gi	12Gi
mysql-cluster-db-three-one-replication-svc/db-infra-monitor-svc	1	100m	100m	256Mi	256Mi
mysql-cluster-db-three-two-replication-svc/istio-proxy	1	5	5	5Gi	5Gi
mysql-cluster-db-three-two-replication-svc/three-two-replication-svc	1	2	2	12Gi	12Gi

The following table describes the interfaces that were either enabled or disabled for this run:

Table 3-135 BSF Features

Features Name	Feature Status
Audit	Enabled
Overload	Enabled
dnssrv	Enabled
Controlled Shutdown	Enabled
NF Scoring	Enabled

The following cnDBTier and Policy helm parameters configured for this run:

Table 3-136 Configuring Helm Parameters

cnDBTier Parameter	Value	Description
MaxNoOfExecutionThreads	11	Specifies the number of execution threads used by ndbmtd.
MaxNoOfConcurrentScans	495	Specifies the maximum number of scans executing concurrently on ndbmtd.
datamemory	51 GB	Specifes the cnDBTier ndb data memory capacity configuration.
proxy.istio.io/config	'{concurrency: 2}'	Specifies the cnDBTier App node (ndbappmysqld) istio side car configuration.
proxy.istio.io/config	'{concurrency: 8}'	Specifies the cnDBTier Data node (ndbmtd) istio side car configuration.
BSF Parameter
proxy.istio.io/config	'{concurrency: 4}'	Specifies the BSF management service istio side car configuration.

3.14.2 CPU and Memory Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF microservices.

BSF Microservices and their Resource Utilization

The following table describes the benchmarking number as per the system maximum capacity utilization for BSF microservices.

Table 3-137 BSF Microservices and their Resource Utilization

Microservices	Site1 CPU	Site1 Memory	Site2 CPU	Site2 Memory	Site3 CPU	Site3 Memory	Site4 CPU	Site4 Memory
Appinfo Istio	0.20%	14.31%	-	-	-	-	-	-
Appinfo	0.95%	27.93%	-	-	-	-	-	-
BSF Management Service Istio	37.30%	18.54%	-	-	-	-	-	-
BSF Management Service	29.81%	58.44%	-	-	-	-	-	-
Alternate Route Service Istio	0.18%	14.87%	-	-	-	-	-	-
Alternate Route Service	0.12%	13.39%	-	-	-	-	-	-
Configuration Service Istio	1.20%	15.92%	-	-	-	-	-	-
Configuration Service	0.55%	44.78%	-	-	-	-	-	-
Egress Service Istio	0.75%	16.75%	-	-	-	-	-	-
Egress Gateway	0.09%	22.62%	-	-	-	-	-	-
Ingress Gateway Istio	41.70%	21.38%	-	-	-	-	-	-
Ingress Gateway	31.32%	44.15%	-	-	-	-	-	-
NRF Client NF Management Istio	0.22%	15.23%	-	-	-	-	-	-
NRF Client NF Management	0.40%	50.54%	-	-	-	-	-	-
Audit Service Istio	0.20%	14.33%	-	-	-	-	-	-
Audit Service	0.10%	47.36%	-	-	-	-	-	-
CM Service Istio	0.43%	15.80%	-	-	-	-	-	-
CM Service	0.15%	36.65%	-	-	-	-	-	-
Query Service Istio	0.12%	14.01%	-	-	-	-	-	-
Query Service	0.05%	30.91%	-	-	-	-	-	-
Perfinfo Istio	0.25%	14.39%	-	-	-	-	-	-
Perfinfo	0.93%	14.16%	-	-	-	-	-	-
Diameter Gateway Istio	14.57%	16.75%	-	-	-	-	-	-
Diameter Gateway	31.72%	50.90%	-	-	-	-	-	-

Observed Values of cnDBTier Services

The following table provides information about observed values of cnDBTier services.

Table 3-138 Observed Values of cnDBTier Services

Microservices	Site1 CPU	Site1 Memory	Site2 CPU	Site2 Memory	Site3 CPU	Site3 Memory	Site4 CPU	Site4 Memory
mysql-cluster-db-db-backup-manager-svc/db-backup-manager-svc	1.00%	78.12%	1.00%	69.53%	3.00%	66.41%	-	-
mysql-cluster-db-db-monitor-svc/istio-proxy	0.06%	5.74%	0.08%	5.78%	0.06%	5.68%	-	-
mysql-cluster-db-db-monitor-svc/db-monitor-svc	2.00%	68.40%	6.00%	66.40%	33.00%	69.20%	-	-
mysql-cluster-db-one-four-replication-svc/istio-proxy	0.06%	5.62%	-	-	-	-	-	-
mysql-cluster-db-one-four-replication-svc/one-four-replication-svc	0.25%	2.42%	-	-	-	-	-	-
mysql-cluster-db-one-three-replication-svc/istio-proxy	0.06%	5.74%	-	-	-	-	-	-
mysql-cluster-db-one-three-replication-svc/one-three-replication-svc	0.30%	2.39%	-	-	-	-	-	-
mysql-cluster-db-one-two-replication-svc/istio-proxy	0.15%	14.31%	-	-	-	-	-	-
mysql-cluster-db-one-two-replication-svc/one-two-replication-svc	0.30%	2.40%	-	-	-	-	-	-
mysql-cluster-db-one-two-replication-svc/db-infra-monitor-svc	1.00%	19.53%	-	-	-	-	-	-
ndbappmysqld/istio-proxy	10.54%	5.92%	0.23%	5.42%	0.26%	5.54%	-	-
ndbappmysqld/mysqlndbcluster	14.12%	26.73%	0.13%	23.79%	0.14%	23.69%	-	-
ndbappmysqld/db-infra-monitor-svc	1.11%	21.57%	1.44%	21.57%	1.44%	21.70%	-	-
ndbappmysqld/init-sidecar	2.00%	0.43%	2.00%	0.39%	2.00%	0.43%	-	-
ndbmgmd/istio-proxy	0.70%	14.31%	0.70%	14.16%	0.60%	13.84%	-	-
ndbmgmd/mysqlndbcluster	0.15%	0.48%	0.17%	0.47%	0.17%	0.45%	-	-
ndbmgmd/db-infra-monitor-svc	1.00%	20.70%	1.00%	20.70%	1.00%	21.09%	-	-
ndbmtd/istio-proxy	18.59%	6.78%	0.83%	6.77%	0.83%	6.80%	-	-
ndbmtd/mysqlndbcluster	11.59%	88.13%	1.29%	88.13%	1.31%	88.14%	-	-
ndbmtd/db-backup-executor-svc	1.00%	43.55%	1.00%	43.68%	1.00%	44.01%	-	-
ndbmtd/db-infra-monitor-svc	4.42%	21.45%	3.00%	22.14%	1.33%	22.10%	-	-
ndbmysqld/istio-proxy	1.18%	5.47%	0.29%	5.49%	0.26%	5.51%	-	-
ndbmysqld/mysqlndbcluster	2.01%	25.00%	0.65%	22.51%	0.69%	22.55%	-	-
ndbmysqld/init-sidecar	2.00%	0.39%	2.00%	0.59%	2.00%	0.52%	-	-
ndbmysqld/db-infra-monitor-svc	1.83%	23.11%	1.83%	23.96%	1.17%	23.63%	-	-
mysql-cluster-db-two-four-replication-svc/istio-proxy	-	-	0.08%	5.62%	-	-	-	-
mysql-cluster-db-two-four-replication-svc/two-four-replication-svc	-	-	0.25%	2.45%	-	-	-	-
mysql-cluster-db-two-one-replication-svc/istio-proxy	-	-	0.15%	14.21%	-	-	-	-
mysql-cluster-db-two-one-replication-svc/two-one-replication-svc	-	-	0.30%	2.39%	-	-	-	-
mysql-cluster-db-two-one-replication-svc/db-infra-monitor-svc	-	-	1.00%	19.92%	-	-	-	-
mysql-cluster-db-two-three-replication-svc/istio-proxy	-	-	0.06%	5.68%	-	-	-	-
mysql-cluster-db-two-three-replication-svc/two-three-replication-svc	-	-	0.30%	2.43%	-	-	-	-
mysql-cluster-db-three-four-replication-svc/istio-proxy	-	-	-	-	0.56%	5.55%	-	-
mysql-cluster-db-three-four-replication-svc/three-four-replication-svc	-	-	-	-	0.30%	2.35%	-	-
mysql-cluster-db-three-one-replication-svc/istio-proxy	-	-	-	-	1.45%	15.04%	-	-
mysql-cluster-db-three-one-replication-svc/three-one-replication-svc	-	-	-	-	0.30%	2.37%	-	-
mysql-cluster-db-three-one-replication-svc/db-infra-monitor-svc	-	-	-	-	1.00%	21.09%	-	-
mysql-cluster-db-three-two-replication-svc/istio-proxy	-	-	-	-	0.78%	5.61%	-	-
mysql-cluster-db-three-two-replication-svc/three-two-replication-svc	-	-	-	-	0.35%	2.40%	-	-

3.14.3 TPS and DB Statistics

Table 3-139 Overall TPS Ingress and Egress TPS to/from NF

Site	Total TPS	Rx	Nbsf
Site1	22.59K	8.2K	14.366K
Site 2	-	-	-
Site 3	-	-	-
Site 4	-	-	-

Observed values of cnDBTier Record counts:

Table 3-140 Database Records

DB Statistics	Read	Write	Commit
Site1	0.524K	0.901K	0.906K
Site2	0.036K	1.263K	0.046K
Site3	0.046K	1.433K	0.052K
Site4	-	-	-

3.14.4 Results

Table 3-141 Average Latency Observations (in milliseconds)

Attributes	Average Latency (ms)
Ingress	4.76
Egress	0.759
Diameter Gateway	0.175
BSF Management	3.09

Table 3-142 Average Current Percentile Latency Observations (in milliseconds)

NF	50th Percentile (ms)	90th Percentile (ms)	95th Percentile (ms)	99th Percentile (ms)
Ingress Gateway	3.42	4.34	4.83	12.9
Egress Gateway	0.874	1.07	1.07	1.07
Diameter Gateway	-	0.306	0.289	0.311