4 Benchmark Testing
This section describes the environment used for benchmarking Oracle Communications Cloud Native Core, Network Repository Function (NRF). Benchmarking is performed with the settings described in this section.
The default values or recommendations for any required software or resource are available from third-party vendors. The operators may choose different values.
4.1 Test Scenario-1: NRF Performance with 24K TPS
Overview
To qualify the test run, the following elements are considered:
- Pod restart
- CPU and Memory utilization
- Error rate
- Ingress and egress traffic rate
- Success rate
- Message request and response processing time
- Infrastructure resource requirements and utilization
4.2 NRF Features
Table 4-1 NRF Features
| NRF Features | Feature Status | Notes |
|---|---|---|
| Subscriber Location Function | ENABLED | SLF is enabled for 3 NfTypes (AUSF, UDR, and UDM) in slfLookupConfig. |
| SLF Selection from Registered NF Profiles | ENABLED | Dynamic SLF feature is enabled. |
| Access Token Caching during SLF Query | ENABLED | Access Token Caching during SLF Query is enabled with 24 hour validity period based caching. For more information about the accessTokenCacheEnabled under slfOptions, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Overload Control Based on Percentage Discards | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Preferred Locality Feature Set | ENABLED | Extended Preferred locality is configured with 2 locationTypes,8 locationMappings, 56 preferredLocationDetails. |
| NRF Forwarding per NF Type/Service Type | ENABLED | Enabled for UDM. Forwarding is enabled for discovery services. 1 host is present in nrfHostConfig and 1 rule is present in forwardingRulesConfig. |
| EmptyList in Discovery Response | ENABLED | EmptyList feature is enabled and discoveryValidityPeriodCfg under nfDiscoveryOptions is configured with 5 rules. |
| Access Token Request Authorization | ENABLED | AuthFeature is enabled with 8 rules in authRulesConfig under nfAccessTokenOptions. |
| Pod Protection Support for NRF Subscription Microservice | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Subscription Limit | ENABLED | Set the subscription limit to 1000. For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| User-Agent Header for Outgoing Requests | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| NRF to Pick Port from ipEndpoints and FQDN from the NF Service or NF level when Selecting SLF | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Ingress Gateway Pod Protection | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| ASM Sidecar | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| TLS SNI Header Validation | DISABLED | TLS is enabled by sidecar. For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Maximum Number of Profiles in Discovery Response | ENABLED | Set the value as 12 for this parameter. For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide.Note: If the value is > 12, there will be a performance impact. |
| NRF Georedundancy | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Discovery Validity | ENABLED | Discovery validity feature is enabled with a
value as 1 minute (for all NF Types).
For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| Access Token Expire Time | NA | The expiry duration for a NRF signed
certificate. The limit is 24 hours.
For more information about NRF Access Token Service Usage Details, see Oracle Communications Cloud Native Core, Network Repository Function User Guide. |
| Load Change Threshold for Notifications from NRF | NA | This is nfNotifyLoadThreshold. This is set as
10%.
For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| NF Screening Feature (Rules) | DISABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
| NRF Roaming support (hNRF + vNRF) | DISABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
Note:
Apart from these features being enabled, rest of the feature configurations have been set to their default values. For more information about the feature configurations, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. For more information about the features, see Oracle Communications Cloud Native Core, Network Repository Function User Guide.4.3 Software Test Constraints
Table 4-2 Software Test Constraints
| Test Constraint | Details |
|---|---|
| NRF Version | 23.4.6 |
| % of traffic forwarded to another NRF | 10% |
| % of traffic forwarded to remote PLMN (case Visited NRF) | Not included in this test |
| % of traffic received from remote PLMN (case Home NRF) | Not included in this test |
| Ingress incoming connections | 221 |
| Ingress outgoing connections | 1.728K |
| Egress incoming connections | 152 |
| Egress outgoing connections | 520 |
| Average HTTP Request Packet Size | ~630 to 11300 bytes (Based on NfRegister Request) |
| Average HTTP Response Packet Size | ~2520-44000 bytes (Discovery Response with nfProfiles) |
| Average NF Heartbeats Timer per NF | 30s |
| Average size of NF Profiles | ~630 to 11300 bytes |
4.4 STATE Data Quantum at NRF before Performance Run Begins
Table 4-3 STATE Data Quantum at NRF before Performance Run Begins
| Parameters | Values | Details |
|---|---|---|
| Number of Subscriptions in DB | 1000 |
Subscriptions are created with the below conditions:
|
4.5 Call-Mix
Table 4-4 Call-Mix
| Type of Request | Traffic Distribution (in %) | Targeted TPS |
|---|---|---|
| NF Register | 0.00002 | 1 |
| NF Update | 0.00282 | 1 |
| NF Heartbeat | Not considered | 52 |
| NF Subscribe + Update Subscription | 0.01416 | 7 |
| NF Discover | 99% | 21600 + 10% forwarded traffic = 24000 |
| SLF Query | Extra 95% of Discovery | 22875* |
| NF Access Token for SLF | Extra | 0.1 |
| NF Profile Retrieval | 0.00141 | 1 |
| NF List Retrieval | 0.00141 | 1 |
| NF Status Notify | 1.52328 | 750 |
| NF Status UnSubscribe | 0.00006 | 1 |
| NF DeRegister | 0.00002 | 1 |
| NF Access Token for Consumers | 0.00141 | 1 |
| NRF Forwarding (Outgoing) | Extra 10% of 24K | 2400* |
Note:
* This is not included in the calculation of the overall traffic of 24K TPS.4.5.1 NF Count Per NF Types
Table 4-5 NF Count Per NF Types
| Target-NF Types | NF Count | Profile Size (with pretty printing) (in bytes) |
|---|---|---|
| AMF | 26 | 8700-11300 |
| AUSF | 6 | 4640 |
| BSF | 3 | 1700 |
| CHF | 3 | 1800 |
| GMLC | 4 | 0630 |
| LMF | 4 | 2400 |
| PCF | 10 | 1500-2500 |
| SMF | 80 | 5700-7400 |
| UDM | 6 | 3200 |
| UDR | 30 | 1700-2000 |
| UPF | 78 | 1500 |
| SLF | 3 | 2100 |
Total number of NF Count is 250.
4.5.2 Number of Profiles in Response
Table 4-6 Number of Profiles in Response for NF Types
| Target-NF Types | TPS (in %) | Absolute TPS | SLF Queries Posted | Number of Profiles in Response |
|---|---|---|---|---|
| AMF | 1.34 | 320 | 0 | 4 |
| AUSF | 9.65 | 2352 | 2352 | 6 |
| BSF | 0.00 | 0 | 0 | 3 |
| CHF | 0.10 | 24 | 0 | 3 |
| GMLC | 0.00 | 0 | 0 | 4 |
| LMF | 0.00 | 0 | 0 | 4 |
| PCF | 0.27 | 64 | 0 | 10 |
| SMF | 2.96 | 699 | 0 | 4 |
| UDM | 66.49 | 16050 | 16050 | 6 |
| UDR | 18.60 | 4473 | 4473 | 12 |
| UPF | 0 | 0 | 0 | 0 |
| SLF | 0 | 0 | 0 | 3 |
| Total | 99.41 | 23982 | 22875 | - |
4.6 Service Operations
This section list the various service operations with the required query parameters.
NfDiscover Service Operation
The NfDiscover service operations are sent with the following query parameters:
- Target-nf-type, guami, and requester-nssais
- Target-nf-type, requester-nf-type, routing-indicator, and group-id-list
- Target-nf-type, requester-nf-type, supi, and service-names
- Target-nf-type, requester-nf-type, and service-names
- Target-nf-type, requester-nf-type, and group-id-list
- Target-nf-type and pgw
- Target-nf-type, requester-nf-type, supi, and service-names
- Target-nf-type, requester-nf-type, and service-names
- Target-nf-type, dnn, snssai, TAI, and pgw-ind
- Target-nf-type, SUPI, group-id-list, and service-names
- Target-nf-type, dnn, snssai, preferred-locality, and service-names
- Target-nf-type and guami
- Target-nf-type, requester-nf-type, and preferred-locality
- Target-nf-type, requester-nf-type, preferred-locality, and SUPI
- Target-nf-type, requester-nf-type, SUPI, group-id-list, requester-nf-type, and data-set
- Target-nf-type, requester-nf-type, routing-indicator, group-id-list, and service-names
4.7 Test Observations
The following table provides the details of test duration and TPS used for the benchmarking test.
Table 4-7 Test Observations
| Parameter | Values |
|---|---|
| Test Duration | 60 hours |
| TPS Achieved | 24K |
4.8 Resource Utilization
The following table describes NRF microservices and their utilization.
Table 4-8 Resource Utilization
| NRF Microservices | Number of Pods | Average CPU | Average Memory (in Gi) |
|---|---|---|---|
| nfregistration | 2 | 0.27 (13.5%) | 1.21 (40.33%) |
| nfdiscovery | 60 | 135.08 (56.28%) | 74.71 (41.51%) |
| nfsubscription | 2 | 0.05 (2.5%) | 1.18 (39.33%) |
| nrfAuditor | 2 | 0.02 (1%) | 0.89 (29.67%) |
| nrfconfiguration | 1 | 0.03 (1.5%) | 0.84 (42%) |
| nfaccesstoken | 2 | 0.02 (1%) | 1.69 (34.5%) |
| nrfartisan | 1 | 0 (0%) | 0.47 (47%) |
| nrfcachedata | 2 | 0.58 (7.25%) | 2.27 (28.38%) |
| alternate-route | 2 | 0 (0%) | 0 (0%) |
| appinfo | 2 | 0.06 (6%) | 0.25 (25%) |
| perfinfo | 2 | 0.12 (12%) | 0.12 (12%) |
| ingressgateway | 27 | 53.25 (49.31%) | 56.81 (52.6%) |
| egressgateway | 19 | 33.95 (44.67%) | 22.84 (30.05%) |
Note:
The features enabled for this benchmarking testing do not utilize alternate-route service.4.8.1 CNE Common Services Resource Utilization
The following table describes CNE microservices and their utilization.
Table 4-9 CNE Common Services Resource Utilization
| CNE Observability Services | Pod Count | Average CPU | Average Memory |
|---|---|---|---|
| occne-prometheus-server-1 | 2 | 8.07% | 64% |
| occne-elastic-elasticsearch-master-0 | 1 | 0.53% | 65% |
| occne-elastic-elasticsearch-master-1 | 1 | 0.56% | 68% |
| occne-elastic-elasticsearch-master-2 | 1 | 0.53% | 65% |
| occne-elastic-elasticsearch-data-1 | 1 | 0.31% | 53% |
| occne-elastic-elasticsearch-data-2 | 1 | 0.72% | 93% |
| occne-prometheus-node-exporter | 16 | 0.80% | 8% |
4.8.2 cnDBTier Services Resource Utilization
The following table provides observed values of cnDBTier services.
Table 4-10 cnDBTier Services Resource Requirements
| Service Name | Pod Count | Average CPU | Average Memory |
|---|---|---|---|
| SQL (ndbappmysqld) | 2 | 0.4 (10%) | 3.47 (57.83%) |
| DB (ndbmtd) | 4 | 0.56 (2.8%) | 15.87 (79.35%) |
| SQL (ndbmysqld) | 2 | 0.04 (0.5%) | 1.92 (19.2%) |
| MGMT (ndbmgmd) | 2 | 0.01 (0.13%) | 0.06 (0.5%) |
| Monitor Service (db-monitor-svc) | 1 | 0.01 (2.5%) | 0.35 (70%) |
| Backup Manager Service (db-backup-manager-svc) | 1 | 0 (0%) | 0.06 (46.88%) |
| Replication Service | 1 | 0.0 (0.0%) | 0.41 (20.5%) |
Table 4-11 cnDBTier Statistics
| DBTier Statistics | Value |
|---|---|
| Memory usage of data nodes | 1.98% |
| CPU usage of data nodes | 1.88% |
| Write operations per second | 45.6 |
| Read operations per second | 2.5k |
| Transaction rates on data nodes | 2.4k |
4.8.3 Latency Observations
Table 4-12 Latency Observations
| Latency Parameter | Details | ||
|---|---|---|---|
| Min (s) | Max (s) | Avg (s) | |
| Average Turnaround time at Ingress Gateway, Discovery Processing, and SLF Egress Transaction | 0.0781 | 0.199 | 0.0881 |
| Average Turnaround time for Discovery Processing and SLF Egress Transaction | 0.0953 | 0.116 | 0.104 |
| Average Turnaround time for cnDBTier access for Discovery Processing | 0.0 | 0.000551 | 0.0000695 |
| Turnaround time for SLF Egress Transaction (see Note) | 0.109 | 0.150 | 0.126 |
Note:
Following simulators are used for performance measurement and their induced latency:- NF simulator to NRF: 50ms
- NRF to forwarding simulator and forwarding simulator to NRF: 100ms
- NRF to SLF: 150 ms
- Packet Error Ratio Induced: 0.01
The values of Bucketed View, Max, and Average for discovery service operation is maximum among all the other service operations.