3 SEPP Benchmark Testing
This section provides information about the SEPP testcases run in different scenarios.
3.1 Test Scenario 1: SEPP 24K MPS, 24 Hrs Run with ASM Enabled and without any Feature Enabled with 50ms Delay at Server End
This test scenario describes performance and capacity of SEPP and provides the benchmarking results for 24K MPS,24Hrs with ASM enabled and without any feature enabled with 50ms delay at server end.
3.1.1 Testcase and Setup Details
Following are the testcase and setup details:
Traffic Model Details:
Table 3-1 TPS (Transactions Per Second)
| Over all TPS | SITE-1 | SITE-2 |
|---|---|---|
| 24K MPS | 12K MPS | 12K MPS |
Call Mix
Table 3-2 Call Mix
| NF-Consumer | NF-Producer | Reference Point ID | Traffic % |
|---|---|---|---|
| vAMF | hUDM | N8 | 32.35 |
| vAMF | hAUSF | N12 | 8.82 |
| vNRF | hNRF | N27 | 11.76 |
| vNSSF | hNSSF | N31 | 2.94 |
| vPCF | hPCF | N24 | 11.76 |
| vSMF | hUDM | N10 | 8.82 |
| vSMF | hSMF | N16 | 17.65 |
| vSMSF | hUDM | N21 | 2.94 |
| vPCF | hAMF | N15 | 2.94 |
Setup Details
Table 3-3 Setup Details
| Setup Details | Values |
|---|---|
| Active User | NA for SEPP |
| Execution Timeline | 24Hrs |
| Environment | VCNE |
| Cluster | Hardhead1 |
| Bastion-1 IP | 10.148.209.173 |
| OCCNE Version | 23.3.3 |
| cnDBTier | 25.1.100 |
| cSEPP | 25.1.100 |
| pSEPP | 25.1.100 |
| CNC Console | 25.1.100 |
| Set up Configuration |
|
| List of SEPP Features enabled | None. This execution is done on vanilla SEPP deployment. |
Resource Footprint
Table 3-4 Resource Footprint
| Microservices / container | Container Count | CPU Resource per container (Limit) | CPU Resource per container (Request) | Memory Resource per container (Limit) | Memory Resource per container (Request) |
|---|---|---|---|---|---|
| Site1-ocsepp-alternate-route/alternate-route | 2 | 2 | 2 | 4Gi | 4Gi |
| Site1-ocsepp-appinfo/appinfo | 2 | 1 | 1 | 2Gi | 1Gi |
| Site1-ocsepp-cn32c-svc/cn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-cn32f-svc/cn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site1-ocsepp-coherence-svc/coherence-svc | 1 | 1 | 1 | 2Gi | 2Gi |
| Site1-ocsepp-config-mgr-svc/config-mgr-svc | 1 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-n32-egress-gateway/n32-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-n32-ingress-gateway/n32-ingress-gateway | 7 | 6 | 6 | 5Gi | 5Gi |
| ocsepp-nf-mediation/nf-mediation | 2 | 8 | 8 | 8Gi | 8Gi |
| Site1-ocsepp-ocpm-config/config-server | 2 | 1 | 1 | 1Gi | 1Gi |
| Site1-ocsepp-performance/perf-info | 2 | 2 | 2 | 4Gi | 200Mi |
| Site1-ocsepp-plmn-egress-gateway/plmn-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-plmn-ingress-gateway/plmn-ingress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-pn32c-svc/pn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-pn32f-svc/pn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site1-ocsepp-sepp-nrf-client-nfdiscovery/nrf-client-nfdiscovery | 2 | 1 | 1 | 2Gi | 2Gi |
| Site1-ocsepp-sepp-nrf-client-nfmanagement/nrf-client-nfmanagement | 1 | 1 | 1 | 1Gi | 1Gi |
| Site2-ocsepp-alternate-route/alternate-route | 1 | 2 | 2 | 4Gi | 4Gi |
| Site2-ocsepp-appinfo/appinfo | 2 | 1 | 1 | 2Gi | 1Gi |
| Site2-ocsepp-cn32c-svc/cn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-cn32f-svc/cn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site2-ocsepp-coherence-svc/coherence-svc | 1 | 1 | 1 | 2Gi | 2Gi |
| Site2-ocsepp-config-mgr-svc/config-mgr-svc | 1 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-n32-egress-gateway/n32-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-n32-ingress-gateway/n32-ingress-gateway | 7 | 6 | 6 | 5Gi | 5Gi |
| Site2-ocsepp-ocpm-config/config-server | 2 | 1 | 1 | 1Gi | 1Gi |
| Site2-ocsepp-performance/perf-info | 2 | 2 | 2 | 4Gi | 200Mi |
| Site2-ocsepp-plmn-egress-gateway/plmn-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-plmn-ingress-gateway/plmn-ingress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-pn32c-svc/pn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-pn32f-svc/pn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site2-ocsepp-sepp-nrf-client-nfdiscovery/nrf-client-nfdiscovery | 2 | 1 | 1 | 2Gi | 2Gi |
| Site2-ocsepp-sepp-nrf-client-nfmanagement/nrf-client-nfmanagement | 1 | 1 | 1 | 1Gi | 1Gi |
| Site2-mysql-cluster-db-backup-manager-svc/db-backup-manager-svc | 1 | 100m | 100m | 128Mi | 128Mi |
| Site2-mysql-cluster-db-monitor-svc/db-monitor-svc | 1 | 4 | 4 | 4Gi | 4Gi |
| Site2-ndbappmysqld/mysqlndbcluster | 2 | 8 | 8 | 10Gi | 10Gi |
| Site2-ndbappmysqld/init-sidecar | 2 | 100m | 100m | 256Mi | 256Mi |
| Site2-ndbmgmd/mysqlndbcluster | 2 | 4 | 4 | 10Gi | 8Gi |
| Site2-ndbmgmd/db-infra-monitor-svc | 2 | 100m | 100m | 256Mi | 256Mi |
| Site2-ndbmtd/mysqlndbcluster | 4 | 10 | 10 | 18Gi | 16Gi |
| Site1-mysql-cluster-db-backup-manager-svc/db-backup-manager-svc | 1 | 100m | 100m | 128Mi | 128Mi |
| Site2-ndbmtd/db-backup-executor-svc | 4 | 100m | 100m | 256Mi | 256Mi |
| Site1-mysql-cluster-db-monitor-svc/db-monitor-svc | 1 | 4 | 4 | 4Gi | 4Gi |
| Site2-ndbmtd/db-infra-monitor-svc | 4 | 100m | 100m | 256Mi | 256Mi |
| Site1-ndbappmysqld/mysqlndbcluster | 2 | 8 | 8 | 10Gi | 10Gi |
| Site1-ndbappmysqld/init-sidecar | 2 | 100m | 100m | 256Mi | 256Mi |
| Site1-ndbmgmd/mysqlndbcluster | 2 | 4 | 4 | 10Gi | 8Gi |
| Site1-ndbmgmd/db-infra-monitor-svc | 2 | 100m | 100m | 256Mi | 256Mi |
| Site1-ndbmtd/mysqlndbcluster | 4 | 10 | 10 | 18Gi | 16Gi |
| Site1-ndbmtd/db-backup-executor-svc | 4 | 100m | 100m | 256Mi | 256Mi |
| Site1-ndbmtd/db-infra-monitor-svc | 4 | 100m | 100m | 256Mi | 256Mi |
| Site1-hello-world2/hello-world2 | 2 | NA | NA | NA | NA |
| Site1-occne-alertmanager-snmp-notifier/alertmanager-snmp-notifier | 1 | NA | NA | NA | NA |
| Site1-occne-bastion-controller/bastion-controller | 1 | 200m | 10m | 256Mi | 128Mi |
| Site1-occne-kube-prom-stack-grafana/grafana-sc-dashboard | 1 | NA | NA | NA | NA |
| Site1-occne-kube-prom-stack-grafana/grafana-sc-datasources | 1 | NA | NA | NA | NA |
| Site1-occne-kube-prom-stack-grafana/grafana | 1 | 500m | 500m | 500Mi | 500Mi |
| Site1-occne-kube-prom-stack-kube-operator/kube-prometheus-stack | 1 | 200m | 100m | 200Mi | 100Mi |
| Site1-occne-kube-prom-stack-kube-state-metrics/kube-state-metrics | 1 | 20m | 20m | 500Mi | 32Mi |
| Site1-occne-metrics-server/metrics-server | 1 | 100m | 100m | 200Mi | 200Mi |
| Site1-occne-promxy/promxy | 1 | 100m | 100m | 512Mi | 512Mi |
| Site1-occne-promxy-apigw-nginx/nginx | 2 | 2 | 1 | 1536Mi | 1Gi |
| Site1-occne-tracer-jaeger-collector/occne-tracer-jaeger-collector | 1 | 1250m | 500m | 1Gi | 512Mi |
| Site1-occne-tracer-jaeger-query/occne-tracer-jaeger-query | 1 | 500m | 256m | 512Mi | 128Mi |
| Site1-occne-tracer-jaeger-query/occne-tracer-jaeger-agent-sidecar | 1 | NA | NA | NA | NA |
| Site1-alertmanager-occne-kube-prom-stack-kube-alertmanager/alertmanager | 2 | 20m | 20m | 64Mi | 64Mi |
| Site1-alertmanager-occne-kube-prom-stack-kube-alertmanager/config-reloader | 2 | 200m | 200m | 50Mi | 50Mi |
| Site1-occne-opensearch-cluster-client/opensearch | 3 | 1 | 1 | 2Gi | 2Gi |
| Site1-occne-opensearch-cluster-data/opensearch | 5 | 2 | 1 | 32Gi | 16Gi |
| Site1-occne-opensearch-cluster-master/opensearch | 3 | 2 | 1 | 32Gi | 16Gi |
| Site1-prometheus-occne-kube-prom-stack-kube-prometheus/prometheus | 2 | 12 | 12 | 55Gi | 55Gi |
| Site1-prometheus-occne-kube-prom-stack-kube-prometheus/config-reloader | 2 | 200m | 200m | 50Mi | 50Mi |
3.1.2 Traffic and Latency
The following tables describe the traffic and latency details:
Traffic Details
Table 3-5 Traffic Details
| TPS | Site-1 | Site-2 |
|---|---|---|
| PLMN-IGW-requests-rate | 10290.83 | 10305.44 |
| CN32F-requests-rate | 10148.51 | 10237.15 |
| N32-IGW-requests-rate | 10237.13 | 10148.64 |
| N32-EGW-requests-rate | 10148.65 | 10237.16 |
| PN32F-requests-rate | 10236.35 | 10147.94 |
| PLMN-EGW-requests-rate | 10537.70 | 10445.02 |
| Total TPS | 10266.5 | 10253.5 |
Latency Details
Table 3-6 Latency Details
| NF Service Latency( In MilliSecond) | Site-1 | Site-2 |
|---|---|---|
| IGW(s) | 0.17 | 0.24 |
| EGW(s) | 0.25 | 0.18 |
| cn32f(s) | 0.13 | 0.10 |
| pn32f(s) | 0.03 | 0.03 |
3.1.3 Results
- csepp_call success rate: 99.999%
- psepp_call success rate: 99.999%
- csepp_Avg_Latency_rate: 73.67 ms
- psepp_Avg_Latency_rate: 72.71 ms
- No pod restarts are observed.
- Perfgo deployed on hardhead 1 cluster with 15 server and 4 client each side.
- Run with 50ms server delay.
- Feature enabled: NA (Vanilla run)
3.2 Test Scenario 2: SEPP 24K MPS, 102 Hrs Run with ASM Enabled and without any Feature Enabled with 50ms Delay at Server End
This test scenario describes performance and capacity of SEPP and provides the benchmarking results for 102 Hrs with ASM enabled and without any feature enabled with 50ms delay at server end.
3.2.1 Test Case and Setup Details
Following are the testcase and setup details:
Traffic Model Details
Table 3-7 TPS (Transactions Per Second)
| Over all TPS | Site 1 | Site 2 | DB size | CPU Threshold | Memory Threshold |
|---|---|---|---|---|---|
| 40K MPS | 20K MPS | 20K MPS | 21MB at each SEPP | 75 | 75 |
Call Mix
Table 3-8 Call Mix
| NF-Consumer | NF-Producer | Reference Point ID | Traffic % |
|---|---|---|---|
| vAMF | hUDM | N8 | 32.35 |
| vAMF | hAUSF | N12 | 8.82 |
| vNRF | hNRF | N27 | 11.76 |
| vNSSF | hNSSF | N31 | 2.94 |
| vPCF | hPCF | N24 | 11.76 |
| vSMF | hUDM | N10 | 8.82 |
| vSMF | hSMF | N16 | 17.65 |
| vSMSF | hUDM | N21 | 2.94 |
| vPCF | hAMF | N15 | 2.94 |
Setup Details
Table 3-9 Setup Details
| Setup Details | Values |
|---|---|
| Active User | NA for SEPP |
| Execution Timeline | 75Hrs |
| Environment | VCNE |
| Cluster | Hardhead1 |
| OCCNE Version | 23.3.3 |
| cnDBTier | 25.1.200 |
| cSEPP | 25.1.200 |
| pSEPP | 25.1.200 |
| CNC Console | 25.1.200 |
| Set up Configuration |
|
| List of SEPP Features enabled | None. This execution is done on vanilla SEPP deployment. |
Resource Footprint
Table 3-10 Resource Footprint
| Micro services/container | Replicas | CPU (Limit) | CPU (Request) | Memory (Limit) | Memory (Request) |
|---|---|---|---|---|---|
| Site2-ocsepp-alternate-route/alternate-route | 2 | 2 | 2 | 4Gi | 4Gi |
| Site2-ocsepp-appinfo/appinfo | 2 | 1 | 1 | 2Gi | 1Gi |
| Site2-ocsepp-cn32c-svc/cn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-cn32f-svc/cn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site2-ocsepp-coherence-svc/coherence-svc | 1 | 1 | 1 | 2Gi | 2Gi |
| Site2-ocsepp-config-mgr-svc/config-mgr-svc | 1 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-n32-egress-gateway/n32-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-n32-ingress-gateway/n32-ingress-gateway | 7 | 6 | 6 | 5Gi | 5Gi |
| Site2-ocsepp-nf-mediation/nf-mediation | 2 | 8 | 8 | 8Gi | 8Gi |
| Site2-ocsepp-ocpm-config/config-server | 2 | 1 | 1 | 1Gi | 1Gi |
| Site2-ocsepp-performance/perf-info | 2 | 2 | 2 | 4Gi | 200Mi |
| Site2-ocsepp-plmn-egress-gateway/plmn-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-plmn-ingress-gateway/plmn-ingress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site2-ocsepp-pn32c-svc/pn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site2-ocsepp-pn32f-svc/pn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site2-ocsepp-sepp-nrf-client-nfdiscovery/nrf-client-nfdiscovery | 2 | 1 | 1 | 2Gi | 2Gi |
| Site2-ocsepp-sepp-nrf-client-nfmanagement/nrf-client-nfmanagement | 1 | 1 | 1 | 1Gi | 1Gi |
| Site1-ocsepp-alternate-route/alternate-route | 2 | 2 | 2 | 4Gi | 4Gi |
| Site1-ocsepp-appinfo/appinfo | 2 | 1 | 1 | 2Gi | 1Gi |
| Site1-ocsepp-cn32c-svc/cn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-cn32f-svc/cn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site1-ocsepp-coherence-svc/coherence-svc | 1 | 1 | 1 | 2Gi | 2Gi |
| Site1-ocsepp-config-mgr-svc/config-mgr-svc | 1 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-n32-egress-gateway/n32-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-n32-ingress-gateway/n32-ingress-gateway | 7 | 6 | 6 | 5Gi | 5Gi |
| Site1-ocsepp-nf-mediation/nf-mediation | 2 | 8 | 8 | 8Gi | 8Gi |
| Site1-ocsepp-ocpm-config/config-server | 2 | 1 | 1 | 1Gi | 1Gi |
| Site1-ocsepp-performance/perf-info | 2 | 2 | 2 | 4Gi | 200Mi |
| Site1-ocsepp-plmn-egress-gateway/plmn-egress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-plmn-ingress-gateway/plmn-ingress-gateway | 7 | 5 | 5 | 5Gi | 5Gi |
| Site1-ocsepp-pn32c-svc/pn32c-svc | 2 | 2 | 2 | 2Gi | 2Gi |
| Site1-ocsepp-pn32f-svc/pn32f-svc | 7 | 5 | 5 | 8Gi | 8Gi |
| Site1-ocsepp-sepp-nrf-client-nfdiscovery/nrf-client-nfdiscovery | 2 | 1 | 1 | 2Gi | 2Gi |
| Site1-ocsepp-sepp-nrf-client-nfmanagement/nrf-client-nfmanagement | 1 | 1 | 1 | 1Gi | 1Gi |
3.2.2 Traffic and Latency
The following tables describe the traffic and latency details:
Traffic Details
Table 3-11 Traffic Details
| TPS | Site-1 | Site-2 |
|---|---|---|
| PLMN-IGW-requests-rate | 10289.46 | 10289.58 |
| CN32F-requests-rate | 10289.45 | 10289.59 |
| N32-IGW-requests-rate | 10289.61 | 10289.47 |
| N32-EGW-requests-rate | 10289.45 | 10289.57 |
| PN32F-requests-rate | 10289.61 | 10289.42 |
| PLMN-EGW-requests-rate | 10289.60 | 10289.46 |
| Total TPS | 10289.5 | 10289.5 |
Latency Details
Table 3-12 Latency Details
| NF Service Latency( In MilliSecond) | Site-1 | Site-2 |
|---|---|---|
| IGW(s) | 0.06 | 0.06 |
| EGW(s) | 0.07 | 0.07 |
| cn32f(s) | 0.04 | 0.04 |
| pn32f(s) | 0.03 | 0.03 |
3.2.3 Results
- csepp_call success rate: 99.999%
- psepp_call success rate: 99.999%
- csepp_Avg_Latency_rate: 72.04 ms
- psepp_Avg_Latency_rate: 71.78 ms
- No pod restarts are observed.
- Perfgo deployed on hardhead 1 cluster with 5 server and 4 client each side.
- Run with 50ms server delay.
- Feature enabled: NA (Vanilla run)