Implementing Security Recommendations and Guidelines for OCNWDAF

4.1 Implementing Security Recommendations and Guidelines for OCNWDAF

This section provides specific recommendations and guidelines for Oracle Communications Network Data Analytics Function (OCNWDAF) security.

4.1.1 Common Security Recommendations and Procedures

This section provides details of the common security recommendations and guidelines.

4.1.1.1 4G and 5G Application Authentication and Authorization

Cloud Native Core NFs support integration with platform service meshes and mTLS may be provided by the platform service mesh, thereby securing communication flows between all applications that participate in the platform service mesh. mTLS also encrypts the data flows so that only the endpoints of the mTLS session can access the contents of the communication data.

4G and 5G NFs use Mutual Transport Layer Security (mTLS) authentication to secure communication. All NFs require establishing a trust relationship with all peers by exchanging and trusting peer root or intermediate certificates. The peer certificates must be available in the truststore (K8s Secrets) to establish secure communication.

4G and 5G NFs also support manual importation and a semiautomatic import using the cert-manager external provider.

4.1.1.2 cnDBTier Authentication and Authorization

The cnDBTier provides a highly available multisite database used to store NF state and configuration. When installed, the MySQL DB is configured with a root account whose password is randomly generated. Each NF must have additional accounts for that particular NF. The procedures in this section explains how to change these account passwords. Additionally, communication between the NFs and the MySQL query nodes are protected using TLS.

Procedure: Modify MySQL NDB Root Password

This procedure is executed by the DB Administrator

For each of the MySQL Query nodes, perform the following steps :

Table 4-1 Modify MySQL NDB Root Password

Step	Description	Est Time
1.	Log into the next query node using ssh: `$ ssh admusr@<mysql query node>`	1m
2.	Execute the following command to make the node as root:`$ sudo su`	1m
3.	Invoke mysql using existing DB Root credentials: `# mysql -h 127.0.0.1 -uroot -p` Enter password: `<enter existing root password>`	1m
4.	Change the DB Root credentials: `mysql> ALTER USER'root'@'localhost'IDENTIFIED BY'<NEW_PASSWORD>'; mysql> FLUSH PRIVILEGES;`	1m
5	Repeat steps 1 through 4 for each MySQL Query node.

Note:

If you are accessing a DB instance for the first time, the DB Root password is stored in the


						/var/occnedb/mysqld_expired.log

file. (The system generates a random password at installation time).

Note:

Recommendation 1: Separation of Roles

The roles of DB Administrator and Cluster Administration must be kept separate. The DB Administrator must be responsible for securing and maintaining the cnDBTier MySQL NDM cluster. The Cluster Administrator must be responsible for securing and operating the Bastion Host and K8s Cluster. When 5G NFs are installed, the DB Administrator is required to create new NF database and NF DB accounts (using the DB Root credentials). Once this is completed, the Cluster Administrator installs the NF (using helm).

Recommendation 2: Use Strong Passwords

The DB Administrator must choose a complex DB Root password as per their organization's security. guidelines.

Procedure: Configure TLS for MySQL NDB Query Nodes

The MySQL NDB comes preconfigured to use a self-signed certificate that expires after 365 days. User can replace this certificate using the following procedure:

Table 4-2 Configure TLS for MySQL NDB Query Nodes

Step	Description	Est Time
1.	Create private CA and a set of Keys/Certificate pairs for use in securing MySQL : `$ my_ssl_rsa_setup`	1m
2.	The available set of PEM files containing CA, server, and client certificates and keys must be installed on all the MySQL Query Nodes.
3.	Using SCP, copy the PEM files to the MySQL Query Node: `$ scp *.pem admusr@<mysql query node>`	1m
4.	Login to the MySQL Query Node using ssh: `$ ssh admusr@<mysql query node>`	1m
5	Create a directory to hold the TLS keys and certs, and move them into root: `$ sudo mkdir /var/occnedb/opensslcerts $ sudo chmod 700 /var/occnedb/opensslcerts $ sudo mv ~admusr/*pem /var/occnedb/opensslcerts`	1m
6	Configure MySQL to use the new TLS certs and keys by executing the following command: `$ sudo vi /etc/my.cnf` Update the [mysqld] section as follows: `[mysqld] ssl-ca=/var/occnedb/opensslcerts/ca.pem ssl-cert=/var/occnedb/opensslcerts/server-cert.pem ssl-key=/var/occnedb/opensslcerts/server-key.pem tls_version=TLSv1.2 ssl-cipher=DHE-RSA-AES128-GCM-SHA256`	5m
7	Restart the mysql daemon: `$ sudo /etc/init.d/mysql.server restart`	1m
	Repeat steps 3 through 7 for each MySQL Query node.

Note:

It is possible to integrate into an existing Public Key Infrastructure (PKI) by creating signing requests and having the PKI to generate the needed key/certificate pairs.

4.1.1.3 Cloud Native Core Ingress and Egress Gateways Specific Security Recommendations and Guidelines

This section provides Ingress and Egress Gateways specific security recommendations and guidelines.

Note:

The following procedures can be performed by any authenticated user who has privileged access to the system. This user can create different roles for specific operations. For creation of role and role binding, see the NF or component-specific Installation and Upgrade Guide.

The procedures are:

Enabling TLS and Ciphers in Ingress and Egress Gateway

Use the following procedure to enable TLS and Ciphers:

Helm Configuration to enable TLS:
To open HTTPS port in Ingress Gateway, set the enableIncomingHttps parameter to true.

To configure the HTTPS client in Ingress Gateway, set the enableOutgoingHttps parameter to true.
Create the following files:
1. RSA or ECDSA Private key (Example: rsa_private_key_pkcs1.pem)
2. Truststore password (Example:trust.txt)
3. Key store password (Example: key.txt)
4. Certificate chain for truststore (Example: caroot.cer)
5. Signed server certificate (Example: ocingress.cer) or Signed client certificate (Example: ocegress.cer)
Note: Creation of private keys, certificates, and passwords is at the discretion of user.

Run the following command to create secret:

$ kubectl create secret generic ocingress-secret --from-file=rsa_private_key_pkcs1.pem 
--from-file=trust.txt --from-file=key.txt --from-file=ocingress.cer --from-file=caroot.cer -n ocingress

Enable the cipher suites:

Cipher Suites to be enabled on Server side (Ingress Gateway).
Cipher Suites to be enabled on Client side (Egress Gateway).

cipher Suites:


-TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Certificate Management and Dynamic Reload of Certificates in Gateways

Whenever certificates get compromised or a new certificate chain is required to be added to the truststore, you can update the key and truststore used by the application.

To update the key and the truststore, update or replace the secret:

$ kubectl create secret generic ocingress-secret --from-file=rsa_private_key_pkcs1.pem
 --from-file=trust.txt --from-file=key.txt --from-file=tmp.cer --from-file=caroot.cer --dry-run -o yaml
 -n ocingress| kubectl replace -f - -n ocingress

Whenever there is an update in the certificate chain or signed certificate placed in secret, Kubernetes watcher implemented in update container checks for a change in file state and replaces the key and truststore accordingly in the mounted shared volume.

Dynamic reload of certificates is not supported in Ingress Gateway as of now, so a manual restart of pod is required when any update in the configuration is made with respect to https.

In case of Egress Gateway, update container will trigger the rest endpoint to reload key and truststore dynamically. Then Egress Gateway will pickup new store files from shared volume and reload trust and key managers. Egress Gateway will use the replaced store to establish new connections and gracefully terminate existing connections by sending a GOAWAY frame.

4.1.2 Oracle Communications Networks Data Analytics Function (OCNWDAF) Specific Security Recommendations and Guidelines

This section provides specific recommendations and guidelines for Oracle Communications Networks Data Analytics Function (OCNWDAF) security.

Note:

The following procedures can be performed by any authenticated user who has privileged access to the system. This user can create different roles for specific operations. For creation of role and role binding, see the Oracle Communications Networks Data Analytics Function Installation and Fault Recovery Guide.

Note:

kubectl commands might vary based on the platform deployment. Replace kubectl with Kubernetes environment-specific command line tool to configure Kubernetes resources through kube-api server. The instructions provided in this document are as per the Oracle Communications Cloud Native Environment (CNE) version of kube-api server.

Caution:

User, computer and applications, and character encoding settings may cause an issue when copy-pasting commands or any content from PDF. PDF reader version also affects the copy-pasting functionality. It is recommended to verify the pasted content especially when the hyphens or any special characters are part of the copied content.

The procedures are:

Oauth Token Validation Configuration

Use the following procedure for Oauth Token validation configuration:

NRF creates access tokens using following private keys:
- ECDSA private key
  Example:
```
ecdsa_private_key_pkcs8.pem
```
- RSA private key
  Example:
```
rsa_private_key_pkcs1.pem
```
  In order to validate access token secret needs to be created and configured in ocnwdaf ingress gateway with certificates fetched from nrf.
  Example:
```
 6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c_ES256.crt
```
Log in to Bastion Host or server from where kubectl can be executed.
Create namespace for the secret.
```
$ kubectl create namespace ocnwdaf
```
Create Kubernetes secret for NF Access token validation

Note:
The file names in below command are same as in Step 1.
```
$ kubectl create secret generic oauthsecret --from-file=6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c_ES256.crt-n ocnwdaf
```
Run the following command to verify if the secret is created successfully:
```
$ kubectl describe secret oauthsecret -n ocnwdaf
```

Rest Configuration

We need REST based configurations to distinguish certificates configured from different NRF and use them properly to validate token received from specific NRF. These configurations can be added from CNCC GUI which internally uses config API and payload as below:

“/nwdaf/nf-common-component/v1/igw/oauthvalidatorconfiguration”

Payload:

{
              "keyIdList": [{

                             "keyId": "664b344e74294c8fa5d2e7dfaaaba407",
                             "kSecretName": "samplesecret1",
                             "certName": "samplecert1.crt",
                             "certAlgorithm": "ES256"
                           }],
              "instanceIdList": [{

                             "instanceId": "664b344e74294c8fa5d2e7dfaaaba407",
                             "kSecretName": "samplesecret2",
                             "certName": "samplecert2.crt",
                             "certAlgorithm": "ES256"

                              }],
              "oAuthValidationMode": "INSTANCEID_ONLY"

}

The multiple keyId and instanceId object of different NRFs can be configured.

Using oAuthValidationMode mode of validation can be selected.

Example: INSTANCEID_ONLY, KID_ONLY or KID_PREFERRED

KID_PREFERRED is a fall back mode where it checks for keyId in token, if token contains keyId then validation mode is KID_ONLY or else it falls back to INSTANCEID_ONLY.

Public key Update for Changed Access Token

Use the following procedure for public key update for changed access token:

Update the secret with new or updated details:

# Delete the secret and recreate it
$ kubectl delete secret oauthsecret -n ocnwdaf

# Fetch updated certificates from nrf 

# Recreate the secret with updated details
$ kubectl create secret generic oauthsecret --from-file=0263663c-f5c2-4d1b-9170-f7b1a9116337_ES256.crt

-n ocudr

Certificate configuration update request needs to be sent using CNCC GUI with the updated keyIdList and instanceIdList with new certificates.

Disabling the Signature Validation for Oauth

If serviceMeshCheck flag is enabled under ingress gateway in custom-values file, signature validation is disabled by default.

In this case, only header and payload are validated, and request is successful even if token has wrong signature.

Create HTTPS Certificates

Access to the OCNWDAF subscription endpoint is enabled through the gateway service. To access the endpoint, use a HTTPS version 2 POST request.

Endpoint address: <path>nnwdaf-eventssubscription/v1/subscriptions

To configure HTTPS version 2, enable the property in the configuration file and generate a SSL certificate. Configure the property in the project using the .properties file. See below example:
Example:
```
server.http2.enabled=true
server.ssl.enabled=true
server.ssl.key-store-type=PKCS12
server.ssl.key-store=classpath:localhost.p12 
server.ssl.key-store-password=<password>
```

To create the certificate, run the following commands:

openssl req -x509 -out localhost.crt -keyout localhost.key \
-newkey rsa:2048 -nodes -sha256 \
-subj '/CN=localhost' -extensions EXT -config <( \
printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

Two files are generated - a .crt file and a .key file.

Use openssl to create a PKCS truststore file that contains both the certificate and key, see below example:
Example:
```
openssl pkcs12 -export -in localhost.crt -inkey localhost.key -name localhost -out localhost.p12
```

Configure the properties, see below example:

Example:

server.ssl.enabled=true
server.ssl.key-store-type=PKCS12 
server.ssl.key-store=classpath:localhost.p12 
server.ssl.key-store-password=<password>

Obtain List of Certificate Providers Allowed

The following command lists the SSL certificate providers allowed:

cd JAVA_HOME/jre/lib/security 
keytool -list -keystore cacerts

SSL Providers allow jdk version 17.0.2.

actalisauthenticationrootca
affirmtrustcommercialca
affirmtrustnetworkingca
affirmtrustpremiumca
affirmtrustpremiumeccca
amazonrootca1
amazonrootca2
amazonrootca3
amazonrootca4
baltimorecybertrustca
buypassclass2ca
buypassclass3ca
camerfirmachambersca
camerfirmachamberscommerceca
camerfirmachambersignca
cert_100_emsign_root_ca___g1100
cert_101_emsign_ecc_root_ca___g3101
cert_102_emsign_root_ca___c1102
cert_103_emsign_ecc_root_ca___c3103
cert_104_hongkong_post_root_ca_3104
cert_106_microsoft_ecc_root_certificate_authority_2017106
cert_107_microsoft_rsa_root_certificate_authority_2017107
cert_108_e_szigno_root_ca_2017108
cert_109_certsign_root_ca_g2109
cert_110_trustwave_global_certification_authority110
cert_111_trustwave_global_ecc_p256_certification_authority111
cert_112_trustwave_global_ecc_p384_certification_authority112
cert_113_naver_global_root_certification_authority113
cert_114_ac_raiz_fnmt_rcm_servidores_seguros114
cert_115_globalsign_root_r46115
cert_116_globalsign_root_e46116
cert_117_globaltrust_2020117
cert_118_anf_secure_server_root_ca118
cert_119_certum_ec_384_ca119
cert_120_certum_trusted_root_ca120
cert_121_tuntrust_root_ca121
cert_122_harica_tls_rsa_root_ca_2021122
cert_123_harica_tls_ecc_root_ca_2021123
cert_124_autoridad_de_certificacion_firmaprofesional_cif_a62634068124
cert_125_vtrus_ecc_root_ca125
cert_126_vtrus_root_ca126
cert_127_isrg_root_x2127
cert_128_hipki_root_ca___g1128
cert_129_globalsign_ecc_root_ca___r4129
cert_130_gts_root_r1130
cert_131_gts_root_r2131
cert_132_gts_root_r3132
cert_18_secure_global_ca18
cert_19_comodo_certification_authority19
cert_20_network_solutions_certificate_authority20
cert_22_certigna22
cert_24_certsign_root_ca24
cert_25_netlock_arany__class_gold__f__tan__s__tv__ny25
cert_26_hongkong_post_root_ca_126
cert_27_securesign_rootca1127
cert_28_microsec_e_szigno_root_ca_200928
cert_30_autoridad_de_certificacion_firmaprofesional_cif_a6263406830
cert_31_izenpe_com31
cert_40_twca_root_certification_authority40
cert_42_ec_acc42
cert_43_hellenic_academic_and_research_institutions_rootca_201143
cert_50_ca_disig_root_r250
cert_51_accvraiz151
cert_52_twca_global_root_ca52
cert_54_e_tugra_certification_authority54
cert_56_atos_trustedroot_201156
cert_69_staat_der_nederlanden_ev_root_ca69
cert_74_cfca_ev_root74, Feb 22, 2022, trustedCertEntry
cert_75_oiste_wisekey_global_root_gb_ca75
cert_76_szafir_root_ca276
cert_77_certum_trusted_network_ca_277
cert_81_ac_raiz_fnmt_rcm81
cert_86_tubitak_kamu_sm_ssl_kok_sertifikasi___surum_186
cert_87_gdca_trustauth_r5_root87
cert_88_trustcor_rootcert_ca_188
cert_89_trustcor_rootcert_ca_289
cert_90_trustcor_eca_190
cert_94_ssl_com_ev_root_certification_authority_ecc94
cert_96_oiste_wisekey_global_root_gc_ca96
cert_97_uca_global_g2_root97
cert_98_uca_extended_validation_root98
cert_99_certigna_root_ca99
certumca
certumtrustednetworkca
chunghwaepkirootca
comodoaaaca
comodoeccca
comodorsaca
digicertassuredidg2
digicertassuredidg3
digicertassuredidrootca
digicertglobalrootca
digicertglobalrootg2
digicertglobalrootg3
digicerthighassuranceevrootca
digicerttrustedrootg4
dtrustclass3ca2
dtrustclass3ca2ev
entrust2048ca
entrustevca
entrustrootcaec1
entrustrootcag2
entrustrootcag4
geotrustglobalca
geotrustprimaryca
geotrustprimarycag2
geotrustprimarycag3
geotrustuniversalca
globalsignca
globalsigneccrootcar4
globalsigneccrootcar5
globalsignr3ca
globalsignrootcar6
godaddyclass2ca
godaddyrootg2ca
haricaeccrootca2015
haricarootca2015
identrustcommercial
identrustpublicca
letsencryptisrgx1
luxtrustglobalroot2ca
quovadisrootca1g3
quovadisrootca2
quovadisrootca2g3
quovadisrootca3
quovadisrootca3g3
secomscrootca1
secomscrootca2
securetrustca
sslrooteccca
sslrootevrsaca
sslrootrsaca
starfieldclass2ca
starfieldrootg2ca
starfieldservicesrootg2ca
swisssigngoldg2ca
swisssignplatinumg2ca
swisssignsilverg2ca
teliasonerarootcav1
thawteprimaryrootca
thawteprimaryrootcag2
thawteprimaryrootcag3
ttelesecglobalrootclass2ca
ttelesecglobalrootclass3ca
usertrusteccca
usertrustrsaca
verisignclass3g3ca
verisignclass3g4ca
verisignclass3g5ca
verisignuniversalrootca
xrampglobalca Kafka security configuration

Configuring Kafka Security

Use the following procedure to configure basic Kafka security:

Prerequisite: SSL certificates and keys.

Configuring Kafka Brokers

Configure Kafka to use TLS or SSL encryption, edit the configuration file server.properties. This file is usually stored in the Kafka config directory.

Note:
Login as su to ensure the ownership of the file is not changed.
```
kafka config cmd

/opt/kafka/config# su -s /bin/bash kafka
```

Add the following properties for each broker configuration, see below example:

Example:

listeners=PLAINTEXT://kafka-broker-host-name:9092,SSL://kafka-broker-host-name:9093
advertised.listeners=PLAINTEXT://kafka-broker-host-name:9092,SSL://kafka-broker-host-name:9093
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks

ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks

ssl.truststore.password=<secret_value>

ssl.keystore.password=<secret_value>

ssl.key.password=<secret_value>

security.inter.broker.protocol=SSL

In the above sample configurations, PLAINTEXT and SSL protocols are used for the SSL enabled brokers.

Complete the above process for each broker, then restart the Kafka Cluster. Run the following scripts to reset the Kafka Cluster:
```
kafka-server-stop.sh
kafka-server-start.sh 
```
Enable Client Authentication (two way authentication), run the following command:
```
ssl.client.auth=required
```

Configuring Kafka Client

On the client side both certificates and keys are required. Configure all the clients communicating with SSL enabled Kafka Cluster. See below example:

Example:


bootstrap.servers=kafka-broker-host-name:9093

security.protocol=SSL

ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks

ssl.truststore.password=<secret_value>

ssl.keystore.location=/var/private/ssl/kafka.client.keystore.jks

ssl.keystore.password=<secret_value>

ssl.key.password=<secret_value>

Where kafka-broker-host-name is the FQDN of the broker.

ACL authorization

To enable and use the AclAuthorizer, set its full class name for your broker configuration in file server.properties.

Run the following command:

authorizer.class.name=kafka.security.authorizer.AclAuthorizer

Kafka ACLs control the principals that perform operations on Kafka resources. Kafka brokers can use ZooKeeper ACLs by enabling broker configuration in the properties file.
```
zookeeper.set.acl=true
```

Create ACLs for every topic, see below example:

Example:

kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \
--allow-principal User:* --operation All --topic mesa.report.location

MySQL Configuration

cnDBTier is used for DB connections. This ensures security issues are avoided and the data is encrypted.

OCNWDAF database passwords are encrypted using cipher.

Figure 4-1 OCNWDAF database password

OCNWDAF Secret Configuration to Enable HTTPS

Use the following procedure to create Kubernetes secret for HTTPS:

Prerequisites:

ECDSA private key and CA signed certificate of CNCC (if initial algorithm is ES256)
RSA private key and CA signed certificate of CNCC (if initial algorithm is RSA256)
TrustStore password file
KeyStore password file
CA certificate

Note:

Private Keys, certificates, passwords are created at the discretion of users.

Log in to Bastion Host or server from where you can run kubectl commands.
Create a namespace for the secret by following:
1. Verify if the required namespace already exists in the system:
```
$ kubectl get namespaces
```
2. In the output of the above command, check if required namespace is available. If unavailable, create the namespace using following command:
  Note:
  This is an optional step. In case required namespace already exists, proceed with next procedures.
```
$ kubectl create namespace <required namespace>
```
  Example:
```
 $ kubectl create namespace ocnwdaf
```

Create nwdaf-tls secret based on TLS certificates:

K8_NAMESPACE=<insert-your-namespace>
CERT_PATH=<certificate-path>
kubectl create secret generic nwdaf-tls \
--save-config \
--dry-run=client \
--from-file=${CERT_PATH}/nwdaf.crt --from-file=${CERT_PATH}/nwdaf.key -o yaml | \
kubectl -n ${K8_NAMESPACE} apply -f -

Note:

.crt and .key files should be named nwdaf.crt and nwdaf.key

After successful secret creation, the following message is displayed:
```
secret/nwdaf-tls created
```