4 Benchmark Testing
This section describes the environment used for benchmarking Oracle Communications Cloud Native Core, Network Repository Function (NRF). Benchmarking is performed with the settings described in this section.
The default values or recommendations for any required software or resource are available from third-party vendors. The operators may choose different values.
4.1 Overview
To qualify the test run, the following elements are considered:
- Pod restart
- CPU and Memory utilization
- Error rate
- Ingress and egress traffic rate
- Success rate
- Message request and response processing time
- Infrastructure resource requirements and utilization
4.2 NRF Features
The following table lists the features that are enabled during benchmarking.
Table 4-1 NRF Features
NRF Features | Feature Status | Notes |
---|---|---|
Subscriber Location Function | ENABLED | SLF is enabled for 3 NfTypes (AUSF, UDR, and UDM) in slfLookupConfig. |
SLF Selection from Registered NF Profiles | ENABLED | Dynamic SLF feature is enabled. |
Access Token Caching during SLF Query | ENABLED | Access Token Caching during SLF Query is enabled with 24
hour validity period based caching. For more information about the
accessTokenCacheEnabled under
slfOptions , see Oracle
Communications Cloud Native Core, Network Repository Function REST
Specification Guide.
|
Overload Control | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Preferred Locality Feature Set | ENABLED | Extended Preferred locality is configured with 2 locationTypes, 8 locationMappings, 56 preferredLocationDetails. |
NRF Forwarding per NF Type/Service Type | ENABLED for UDM | Forwarding is enabled for discovery services. 1 host is present in nrfHostConfig and 1 rule is present in forwardingRulesConfig. |
EmptyList in Discovery Response | ENABLED | EmptyList feature is enabled and discoveryValidityPeriodCfg under nfDiscoveryOptions is configured with 5 rules. |
Access Token Request Authorization | ENABLED | AuthFeature is enabled with 8 rules in authRulesConfig under nfAccessTokenOptions. |
Pod Protection Support for NRF Subscription Microservice | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Subscription Limit | ENABLED | Set the subscription limit to 1000. For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
User-Agent Header for Outgoing Requests | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
NRF to Pick Port from ipEndpoints and FQDN from the NF Service or NF level when Selecting SLF | ENABLED | For more information about the configuration, seeOracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
ASM Sidecar | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Maximum Number of Profiles in Discovery Response | ENABLED | Set the value as 12 for the
profilesCountInDiscoveryResponse parameter. For
more information about the configuration, see Oracle
Communications Cloud Native Core, Network Repository Function REST
Specification Guide.
Note: If the value is greater than 12, there will be a performance impact. |
NRF Georedundancy | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
NRF Growth | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
SLF Routing using SCP | ENABLED | For more information about the configuration, seeOracle Communications Cloud Native Core, Network Repository Function Rest Specification Guide. |
Discovery Validity | ENABLED | Discovery validity feature is enabled with a value as 1
minute (for all NF Types).
For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Access Token Expiry Time | NA | The expiry duration for a NRF signed certificate. The
limit is 24 hours.
For more information about NRF Access Token Service Usage Details, see Oracle Communications Cloud Native Core, Network Repository Function User Guide. |
Load Change Threshold for Notifications from NRF | NA | This feature is configured using
nfNotifyLoadThreshold parameter. Set the value to
10%.
For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Monitoring the Availability of SCP Using SCP Health APIs | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
NRF to Retry Alternate Producer to Support Maximum Diversity | NA |
The value of maxSlfAttempts parameter is set to 1 and the value of useAlternateScp parameter is set to true. For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Error Response Enhancement | NA | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Error Log Enhancements | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
NRF support for A_PFD DataSetId in registration and discovery (DataSetId Enhancements) | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
NFService Priority Update | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Subscriber Identity in messages (3GPP-SBI-Correlation-Info) | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Ingress Gateway Pod Protection | DISABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network
Repository Function REST Specification Guide. Note: The Ingress Gateway Pod Protection using Rate Limiting feature is recommended over the Ingress Gateway Pod Protection feature. |
Egress Gateway Pod Throttling | DISABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network
Repository Function Installation, Upgrade and Fault Recovery
Guide.
Note: The Egress Gateway Pod Protection using Rate Limiting feature is recommended over the Egress Gateway Pod Throttling feature. |
NF Profile Size Limit | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Ingress Gateway Pod Protection Using Rate Limiting | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Egress Gateway Pod Protection Using Rate Limiting | ENABLED | For more information about the configuration, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. |
Note:
Apart from these features being enabled, rest of the feature configurations have been set to their default values. For more information about the feature configurations, see Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide. For more information about the features, see Oracle Communications Cloud Native Core, Network Repository Function User Guide.4.3 Software Test Constraints
Table 4-2 Software Test Constraints
Test Constraint | Details |
---|---|
NRF Version | 25.1.202 |
Average HTTP Request Packet Size | ~630 to 15000 bytes (Based on NfRegister Request) |
Average HTTP Response Packet Size | ~5500-131000 bytes (Discovery Response with nfProfiles) |
Average NF Heartbeats Timer per NF | 30s |
Average size of NF Profiles | ~630 to 15000 bytes |
4.4 Service Operations
This section list the various service operations with the required query parameters.
NfDiscover Service Operation
The NfDiscover service operations are sent with the following query parameters:
- Target-nf-type, guami, and requester-nssais
- Target-nf-type, requester-nf-type, routing-indicator, and group-id-list
- Target-nf-type, requester-nf-type, supi, and service-names
- Target-nf-type, requester-nf-type, and service-names
- Target-nf-type, requester-nf-type, and group-id-list
- Target-nf-type and pgw
- Target-nf-type, requester-nf-type, supi, and service-names
- Target-nf-type, requester-nf-type, and service-names
- Target-nf-type, dnn, snssai, TAI, and pgw-ind
- Target-nf-type, SUPI, group-id-list, and service-names
- Target-nf-type, dnn, snssai, preferred-locality, and service-names
- Target-nf-type and guami
- Target-nf-type, requester-nf-type, and preferred-locality
- Target-nf-type, requester-nf-type, preferred-locality, and SUPI
- Target-nf-type, requester-nf-type, SUPI, group-id-list, requester-nf-type, and data-set
- Target-nf-type, requester-nf-type, routing-indicator, group-id-list, and service-names
4.5 CNE Common Services Resource Utilization
The following table describes CNE microservices and their utilization.
Table 4-3 CNE Common Services Resource Utilization
CNE Observability Services | Pod Count |
---|---|
occne-prometheus-server-1 | 2 |
occne-opensearch-cluster | 10 |
occne-prometheus-node-exporter | 16 |
4.6 53K TPS per NRF Set/106K TPS for Two NRF Sets
The per set capacity of NRF is 53K TPS. Higher TPS rate can be achieved by deploying additional NRF sets. This section provides the details for 106K TPS.
4.6.1 NRF Services Resource Requirements
This section provides the NRF services resource requirement per NRF deployment.
Note:
Changes in the resource requirements are highlighted in bold.Table 4-4 NRF Services Resource Requirements
Service Name | Min Pod Replica | Max Pod Replica | Min CPU/Pod | Max CPU/Pod | Min Memory/Pod (in Gi) | Max Memory/Pod (in Gi) | Min (Mi) Ephemeral Storage | Max (Gi) Ephemeral Storage |
---|---|---|---|---|---|---|---|---|
Helm test | 1 | 1 | 1 | 2 | 1 | 2 | 78.1 | 1 |
<helm-release-name>-nfregistration | 2 | 2 | 4 | 4 | 3 | 3 | 78.1 | 1 |
<helm-release-name>-nfdiscovery | 2 | 90 | 8 | 8 | 5 | 5 | 78.1 | 2 |
<helm-release-name>-nfsubscription | 2 | 2 | 2 | 2 | 3 | 3 | 78.1 | 1 |
<helm-release-name>-nrfauditor | 2 | 2 | 2 | 2 | 3 | 3 | 78.1 | 1 |
<helm-release-name>-nrfconfiguration | 1 | 1 | 2 | 2 | 2 | 2 | 78.1 | 1 |
<helm-release-name>-nfaccesstoken | 2 | 2 | 2 | 2 | 2 | 2 | 78.1 | 1 |
<helm-release-name>-nrfartisan | 1 | 1 | 2 | 2 | 2 | 2 | 78.1 | 1 |
<helm-release-name>-nrfcachedata | 2 | 2 | 4 | 4 | 4 | 4 | 78.1 | 1 |
<helm-release-name>-ingressgateway | 2 | 34 | 4 | 4 | 5 | 5 | 78.1 | 1 |
<helm-release-name>-egressgateway | 2 | 30 | 4 | 4 | 4 | 4 | 78.1 | 1 |
<helm-release-name>-alternate-route | 2 | 2 | 2 | 2 | 4 | 4 | 78.1 | 1 |
<helm-release-name>-appinfo | 2 | 2 | 1 | 1 | 1 | 1 | 78.1 | 1 |
<helm-release-name>-perfinfo | 2 | 2 | 1 | 1 | 1 | 1 | 78.1 | 1 |
Table 4-5 Service Mesh Sidecar Resource Requirements
Service Name | CPU/Pod | Memory/Pod (in G) | Concurrency | ||
---|---|---|---|---|---|
Min | Max | Min | Max | ||
Helm test | 0 | 0 | 0 | 0 | NA |
<helm-release-name>-nfregistration | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nfdiscovery | 2 | 2 | 3 | 3 | 4 |
<helm-release-name>-nfsubscription | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nrfauditor | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nrfconfiguration | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nfaccesstoken | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nrfartisan | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-nrfcachedata | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-ingressgateway | 4 | 4 | 3 | 3 | 8 |
<helm-release-name>-egressgateway | 4 | 4 | 3 | 3 | 8 |
<helm-release-name>-alternate-route | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-appinfo | 2 | 2 | 3 | 3 | 2 |
<helm-release-name>-perfinfo | 2 | 2 | 3 | 3 | 2 |
4.6.2 cnDBTier Resource Requirements per NRF Deployment
This section provides the details of cnDBTier services resource requirements.
Note:
Changes in the resource requirements are highlighted in bold.Table 4-6 cnDBTier Services Resource Requirements
Service Name | Pod Replica # | CPU/Pod | Memory/Pod (in Gi) | PVC Size (in Gi) | Ephemeral Storage | ||||
---|---|---|---|---|---|---|---|---|---|
Min | Min | Max | Min | Max | PVC1 | PVC2 | Min (Mi) | Max (Gi) | |
MGMT (ndbmgmd) | 2 | 4 | 4 | 8 | 10 | 15 | NA | 90 | 1 |
DB (ndbmtd) | 4 | 4 | 4 | 5 | 5 | 4 | 5 | 90 | 1 |
SQL (ndbmysqld) | 2 | 4 | 4 | 20 | 20 | 13 | NA | 90 | 1 |
SQL (ndbappmysqld) | 2 | 8 | 8 | 3 | 3 | 1 | NA | 90 | 1 |
Monitor Service (db-monitor-svc) | 1 | 4 | 4 | 4 | 4 | NA | NA | 90 | 1 |
Backup Manager Service (db-backup-manager-svc) | 1 | 1 | 1 | 1 | 1 | NA | NA | 90 | 1 |
Replication Service - Leader | 1 | 2 | 2 | 12 | 12 | 4 | NA | 90 | 1 |
Replication Service - Other | 0 | 0.6 | 1 | 1 | 2 | NA | NA | 90 | 1 |
The following table describes the resources required by cnDBTier service mesh sidecar to perform NRF benchmark tests.
Table 4-7 Service Mesh Sidecar
Service Name | CPU | Memory (in Gi) | Concurrency | ||
---|---|---|---|---|---|
Min | Max | Min | Max | ||
MGMT (ndbmgmd) | 2 | 2 | 1 | 1 | 8 |
DB (ndbmtd) | 2 | 2 | 1 | 1 | 8 |
SQL (ndbmysqld) | 2 | 2 | 1 | 1 | 8 |
SQL (ndbappmysqld) | 4 | 4 | 1 | 1 | 8 |
Monitor Service (db-monitor-svc) | 2 | 2 | 1 | 1 | 2 |
Backup Manager Service (db-backup-manager-svc) | 2 | 2 | 1 | 1 | 2 |
Replication Service-Leader | 2 | 2 | 1 | 1 | 2 |
Replication Service-Other | 2 | 2 | 1 | 1 | 2 |
The following default values are added for the service mesh specific
annotations in the
ocnrf_dbtier_CNDBTIER_VERSION_custom_values_NRF_VERSION.yaml
file.
The following table describes the default values for resources required by cnDBTier service mesh sidecar to perform NRF benchmark tests.
Table 4-8 Default Values for Service Mesh Specific Annotations
Parameter Name | Annotations |
---|---|
mgm.annotations |
|
ndb.annotations |
|
api.annotations |
|
ndbapp.annotations |
|
db-replication-svc.podAnnotations |
Note: The annotations for
|
db-replication-svc.podAnnotations |
traffic.sidecar.istio.io/excludeInboundPorts: "8081" The inbound ports are added only to the db-replication-svc (leader pod). |
db-monitor-svc.podAnnotations |
|
db-backup-manager-svc.pod.annotations |
|
4.6.3 State Data Quantum at NRF
Table 4-9 STATE Data Quantum at NRF
Parameters | Values | Details |
---|---|---|
Number of Subscriptions | 1000 (500 for NRF set 1 + 500 for NRF set 2) |
Subscriptions are created with the below conditions:
|
4.6.3.1 NF Count Per NF Types
The total number of NFs supported by per NRF set is 300. For two sets a total of 600 NFs are supported.
The following table lists the NF count per NF type along with the NF profile size.
Table 4-10 NF Count Per NF Types
Target-NF Types | NF Count | Profile Size (in bytes) |
---|---|---|
AMF | 46 | 15000 |
AUSF | 48 | 4640 |
BSF | 3 | 1700 |
CBCF | 3 | 11300 |
CHF | 3 | 1800 |
GMLC | 3 | 0630 |
LMF | 3 | 2400 |
NEF | 3 | 11300 |
PCF | 79 | 1500-2500 |
SCP | 27 | 1200 |
SLF | 3 | 2100 |
SMF | 148 | 15000 |
SMSF | 3 | 11300 |
UDM | 48 | 3200 |
UDR | 180 | 1700-2000 |
4.6.4 Call-Mix
Table 4-11 Call-Mix
S.No | Type of Request | Traffic Distribution (in %) | Targeted TPS |
---|---|---|---|
A | NF Register | 0.0094 | 5 |
B | NF Update | 0.0810 | 43 |
C | NF Heartbeat | Not considered | 155 |
D | NF Subscribe + Update Subscription | 0.0377 | 20 |
E | NF Discover | 97.24 | 51600 |
F | NRF Forwarded Discovery (Incoming) | 10% of Incoming discovery traffic | 0 |
G | % of traffic forwarded to remote PLMN (case Visited NRF) | 0 | Not included in this test. |
H | % of traffic received from remote PLMN (case Home NRF) | 0 | Not included in this test. |
I | SLF Query | 88.4% of Discovery | 45600 |
J | NF Access Token for SLF | Extra | 1 |
K | NF Profile Retrieval | 2.280 | 1210 |
L | NF List Retrieval | 0.0358 | 19 |
M | NF Status Notify | 4.0612 | 2155 |
N | NF Status UnSubscribe | 0.0056 | 3 |
O | NF DeRegister | 0.0056 | 3 |
P | NF Access Token for Consumers | 0.0188 | 10 |
Q | NRF Forwarding (Outgoing) | 10% of discovery traffic | 5160 |
Table 4-12 Total Traffic
Traffic Type | Message Considered | Targeted TPS |
---|---|---|
Total Incoming Traffic | A + B + C + D + E + F + G + H + J + K + L + N + O + P | 53069 |
Total Outgoing Traffic | I +J+ M + Q | 52916 |
4.6.4.1 Discovery TPS per NfType and Number of Profiles in Response
The following table indicates the discovery TPS per NfType per NRF set and the number of profiles sent in the response.
Table 4-13 Discovery TPS per NfType and Number of Profiles in Response
Target-Nf Types | TPS (in %) | Absolute Discovery TPS | SLF Queries Posted | Number of Profiles in Response |
---|---|---|---|---|
AMF | 4.19 | 2162.4 | 0 | 4 |
AUSF | 0.40 | 204 | 0 | 6 |
BSF | 0.00075 | 0.4 | 0 | 3 |
CBCF | 0.0000 | 0 | 0 | 3 |
CHF | 0.02 | 9.6 | 0 | 3 |
GMLC | 0.0000 | 0 | 0 | 4 |
LMF | 0.00299 | 1.6 | 0 | 4 |
NEF | 0.0001 | 0.0516 | 0 | 3 |
PCF | 0.37 | 191.4 | 0 | 10 |
SLF | 0.00075 | 0.4 | 0 | 3 |
SMF | 8.03 | 4143.4 | 0 | 4 |
SMSF | 0.0001 | 0.0516 | 0 | 3 |
UDM | 85.85 | 44297 | 44297 | 6 |
UDR | 1.14 | 590.4 | 590.4 | 12 |
4.6.5 Test Observations
This section following table provides the details of test duration used for the benchmarking test.
Table 4-14 Test Observations
Parameter | Values |
---|---|
Test Duration | 72 hours |
Ingress incoming connections | 337 |
Ingress outgoing connections | 2.719K |
Egress incoming connections | 240 |
Egress outgoing connections | 1.200K |
4.6.6 NRF Resource Utilization
Table 4-15 NRF Resource Utilization
NRF Microservices | Number of Pods | Average CPU | Average Memory |
---|---|---|---|
nfregistration | 2 | 41% | 43.9% |
nfdiscovery | 90 | 59% | 47.5% |
nfsubscription | 2 | 41.8% | 32% |
nrfauditor | 2 | 1.34% | 23.2% |
nrfconfiguration | 1 | 2.09% | 33.9% |
nfaccesstoken | 2 | 5.76% | 39.4% |
nrfartisan | 1 | 0.10% | 24.6% |
nrfcachedata | 2 | 48.0% | 39.7% |
alternate-route | 2 | 0.10% | 17% |
perf-info | 2 | 8.65% | 13.5% |
appinfo | 2 | 5.41% | 26.3% |
ingressgateway | 34 | 63.0% | 56.5% |
egressgateway | 30 | 62.0% | 40.4% |
Note:
If you enable Message Feed feature at Ingress Gateway and Egress Gateway, approximately 33% pod capacity is impacted. The features enabled for this benchmark testing does not utilize alternate-route service.4.6.7 cnDBTier Services Resource Utilization
The following table provides observed values of cnDBTier services.
Table 4-16 cnDBTier Services Resource Utilization
Service Name | Pod Count | Average CPU | Memory |
---|---|---|---|
SQL (ndbappmysqld) | 2 | 17.8% | 36% |
DB (ndbmtd) | 4 | 14.1% | 87% |
SQL (ndbmysqld) | 2 | 1.7% | 5% |
MGMT (ndbmgmd) | 2 | 0.3% | 0.38% |
Monitor Service (db-monitor-svc) | 1 | 0.5% | 11.3 % |
Backup Manager Service (db-backup-manager-svc) | 1 | 0.6% | 9.5% |
Table 4-17 cnDBTier Statistics
DBTier Statistics | Value |
---|---|
Disk Write operations per second | 2.3K |
Disk Read operations per second | 10.2K |
Transaction rates on data nodes | 3.7K |
4.6.8 Latency Observations
This section provides information about the latency observations.
Table 4-18 Latency Observations
Latency Parameters | 99th Percentile (ms) | 95th Percentile (ms) | 90th Percentile (ms) | 50th Percentile (ms) |
---|---|---|---|---|
Average Turnaround time at Ingress Gateway, Discovery Processing, and SLF Egress Transaction | 200 | 196 | 191 | 151 |
Average Turnaround time for Discovery Processing and SLF Egress Transaction | 200 | 195 | 190 | 150 |
Average Turnaround time for SLF Egress Transaction (see Note) | 200 | 195 | 190 | 150 |
Note:
Following simulators are used for performance measurement and their induced latency:- NF simulator to NRF: 50ms
- NRF to forwarding simulator and forwarding simulator to NRF: 150ms
- NRF to another remote set NRF: 150 ms
- NRF to SLF: 150 ms
4.6.9 NRF Configuration
The configurations for the following features are specific for 53K TPS per NRF set. These features must be configured with the recommended values as mentioned in the Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide.
- Egress Gateway Pod Protection Using Rate Limiting- For more information
about the configurations, see the following sections in Oracle
Communications Cloud Native Core, Network Repository Function REST Specification
Guide.
- "Pod Protection By Rate Limiting" section in Egress Gateway Configuration
- "Congestion Level Configuration" section in Ingress Gateway Configuration
- Ingress Gateway Pod Protection Using Rate Limiting - For more
information about the configurations, see the following sections in Oracle Communications Cloud Native Core, Network Repository
Function REST Specification Guide.
- Pod Protection By Rate Limiting section in Ingress Gateway Configuration
- Congestion Level Configuration section in Ingress Gateway Configuration
- Subscription Pod Protection- For more information about the configuration, see the "Pod Protection Options" section in Oracle Communications Cloud Native Core, Network Repository Function REST Specification Guide.
- Overload Control- For more information about the configurations, see the
following sections in Oracle Communications Cloud Native Core,
Network Repository Function REST Specification Guide.
- Overload Level Threshold Configuration in Perf-Info
- Discard Policy Configuration in Ingress Gateway Configuration
- Policy Mapping Configuration in Ingress Gateway Configuration