Data Director Configuration

3 Data Director Configuration

This chapter lists the Data Director Configuration requirements on Oracle Communications Cloud Native Environment.

3.1 Requirements

Before you begin with the procedure for setting up Data Director in Cloud Native Core, ensure that the following requirements are met:

The metadata from NFs
The third-party target that receives the packets
Optional TLS config for For HTTP2 and Synthetic TCP Feeds
The HTTP standards require a response for every message sent. Oracle will provide a configuration option to ignore the response for HTTP2 Feed.
The message acquisition point is configurable (ingress or egreess, or both) at the NF level.
Kafka consumer Feed is enabled only when OCNADD services are on TLS.
Create ACL user prior to creating Kafka feeds.

Note:

With the HTTP2 ignore the response option enabled the OCNADD considers message transfer as successful as soon as data is sent to 3rd party monitoring consumers. OCNADD does not wait for 200 OK response to consider the message transfer as successful. Message retransmission is not attempted. However, for maintaining the connection status to 3rd party monitoring App endpoints, OCNADD still expects response for each post request sent to 3rd party monitoring App.

3.2 Outbound Protocols

Data Director currently supports three Egress Feed types, over which the 5G SBI messages and metadata added by the NFs are forwarded to third-party consumers:

HTTP2 Feed: The HTTP2 Feed is used for monitoring purposes. It employs the HTTP/2 protocol, utilizing JSON as the application layer serialization protocol. Additionally, there is an option to implement TLS for security protection at the transport layer.
Synthetic Feed: The Synthetic Feed operates through a TCP connection, enabling the transmission of synthetic packets. These packets contain comprehensive L2 to L7 information, complete with synthesized layers and necessary information. For added security at the transport layer, optional TLS is available.
Kafka Consumer Feed: The Kafka Consumer Feed allows 3rd-party consumers to retrieve the 5G SBI messages and metadata introduced by the NFs in the form of JSON documents, using the Kafka consumer API. To enhance security during transmission, TLS is employed at the transport layer.

3.2.1 Metadata

The following table lists the metadata that are part of the available metadata from SCP:

Note:

For more information on each metadata component, see Data Stream Contents.

Table 3-1 Format based on 3GPP

Metadata	Information
correlation-id	This is a unique identifier in the message for correlation within a single transaction. If an intermediate Oracle NF like SCP or SEPP sees a correlation-id custom header in the message, then it forwards the header without any modification. Oracle NFs add the correlation-id custom header in the responses.
consumer-id	The 5G NF Instance ID of the NF that originated the received message. Conditions: For SCP-initiated requests and response messages (e.g., delegated Discovery and OAuth access token requests): The `consumer-id` metadata is always present and matches the value of `feed-source->nf-instance-id` metadata. Note: Since this corresponds to SCP's own instance ID, there is no dependency on the user-header to retrieve the `nf-instance-id` information. For Consumer NF-initiated messages: The `consumer-id` metadata depends on the presence of the `User-Agent` header in the received service request. Recommended User-Agent header format:`<NF Type>-<NF Instance ID> <NF FQDN>`
producer-id	This is a 5G NF Instance ID of the destination NF. Oracle SCP can find the destination NF instance Id using the authority in the service request and learning from the NRF. Other Oracle NFs may not be able to find NF instance id of destination in be able to put destination FQDN
consumer-fqdn	The FQDN of the NF that originated the received message. Conditions: For SCP-initiated requests and response messages (e.g., delegated Discovery and OAuth access token requests): The `consumer-fqdn` metadata is always present and matches the value of the `feed-source->nf-fqdn` metadata. Note: Since this corresponds to SCP's own FQDN, there is no dependency on the user-header to retrieve the FQDN information. For Consumer NF-initiated messages: The `consumer-fqdn` metadata depends on the presence of the `User-Agent` header in the received service request. Recommended User-Agent header format:`<NF Type>-<NF Instance ID> <NF FQDN>`
producer-fqdn	This is an FQDN of the destination NF. It depends on the presence of FQDN in the authority of service request.
hop-by-hop-id	Oracle NFs can add Hop-by-Hop id to identify a request and response pair to the next node. This is required in addition to correlation-id for uniquely identifying the request-response pair in case of re-routing. hop-by-hop-id format is as shown below RxRequest/TxResponse: If the consumer is provided in the request: Format: `<First 30 characters of consumerFqdn>_<Last 30 characters of worker-pod-instance-Id>` If the consumer is not provided: Format: `NA_< Last 30 characters of worker-pod- instanceId>` TxRequest/RxResponse: Format: `< Last 30 characters of worker-pod-instance-Id>_< First 30 characters of producerFqdn>_Suffix` Suffix: An incrementing integer that increases with each routing hop.
reroute-cause	Indicate the re-route cause. Contains one of the following: Circuit breaking: Flag to indicate that a message is an alternate attempt due to circuit breaking functionality at the SCP Outlier detection: Flag to indicate that a message is an alternate attempt due to outlier detection functionality at the SCP Egress-rate-limit: Flag to indicate that a message is an alternate attempt due to egress rate limiting functionality at the SCP producer-nf-congestion: Flag to indicate that a message is an alternate attempt due to producer NF congestion Error received Timeout Not Available
timestamp	This is a timestamp (in nanoseconds) at the traffic feed trigger point when the message is received or forwarded by the SCP. It is an epoch time.
message-direction	This is a parameter to indicate whether a message is ingress to or egress from NF. It can be indicated by putting the traffic feed trigger point name. RxRequest TxRequest RxResponse TxResponse
feed-source	Source of this traffic feed. This contains the key-value of different identity of the node sending the traffic feed. Feed-source : nf-type = SCP nf-fqdn = SCP's FQDN nf-instance-id = SCP's NF instance id pod-instance-id = SCP-worker's pod instance id
dd-ingress-timestamp	This is a timestamp (in nanoseconds) at Data Director when the message is received from NF’s traffic feed and written to Data Director. It is an epoch time. Data Director uses this timestamp for calculating the end-to-end Data Director latency for the feed. Note: For HTTP2 feeds, end-to-end Data Director latency includes the time taken by the HTTP application on the feed consumer side to acknowledge the HTTP2 messages. For TCP/Synthetic feeds, end-to-end Data Director latency includes the time taken by the feed consumer node to acknowledge the TCP packets.
source-ip	The origination IP address of the Message./responses. It's applicable to SEPP.
destination-ip	The destination IP address of the message/response. It's applicable to SEPP.
source-port	The port on which the message/response was received. It's applicable to SEPP.
destination-port	The port on which the message/response is delivered. It's applicable to SEPP.

Following metadata can be optionally generated on DD using DD metadata framework:

Table 3-2 Additional Metadata with DD Metadata Framework

Metadata	Description
path	The path and query parts of the target URI are present in the HEADERS frame. They are taken from the header-list and populated from the first occurrence of the request message. For SCP and SEPP, it is RxRequest. For NRF/PCF/BSF, it can be either RxRequest or TxRequest, as an NRF transaction could be RxRequest→TxResponse or TxRequest→RxResponse. Example: /nausf-auth/v1/ue-authentications/reg-helm-charts-ausfauth-6bf59-kx.34/5g-aka-confirmation
user-agent	The User Agent identifies which equipment made the request. It is present in the HEADERS frame. It is taken from the header-list and populated from the first occurrence of the request message. For SCP and SEPP, it is RxRequest. For NRF/PCF/BSF, it can be either RxRequest or TxRequest as an NRF transaction could be RxRequest→TxResponse or TxRequest→RxResponse. Example: UDM-26740918-e9cd-0205-aada-71a76214d33c udm12.oracle.com
method	It represents the type of request for a transaction. It is present in the HEADERS frame. It is taken from the header-list and populated from the first occurrence of the request message. For SCP and SEPP, it is RxRequest. For NRF/PCF/BSF, it can be either RxRequest or TxRequest as an NRF transaction could be RxRequest→TxResponse or TxRequest→RxResponse. Value: POST, PUT, DELETE, PATCH
consumer-via	It contains a branch unique in space and time identifying the transaction with the next hop. It is taken from the header-list and populated from the first occurrence of RxRequest (In case of an array of "via" in a message, the last occurrence from the list). For SCP and SEPP, it will be RxRequest. For NRF/PCF/BSF, it will be RxRequest or TxRequest as an NRF transaction could be RxRequest→TxResponse or TxRequest→RxResponse. Example: SCP-scp1.5gc.mnc001.mcc208.3gppnetwork.org
ingress-authority	The Node's local IP/FQDN on the ingress side. It is taken from the header-list and populated from the last occurrence of RxRequest. For SCP and SEPP, it will be RxRequest. For NRF/PCF/BSF, it will be either from RxRequest or TxRequest, as NRF transactions could be RxRequest→TxResponse or TxRequest→RxResponse. Example: 172.19.100.5:9443
supi	It represents the subscription identifier with the pattern: '^(imsi-[0-9]{5,15}\|nai-.+\|gci-.+\|gli-.+\|.+)$'. It is populated from the first occurrence in the RxRequest path header or the 3GPP_SBI_DISCOVERY_SUPI header. For SCP and SEPP, it will be RxRequest. For NRF/PCF/BSF, it can be either RxRequest or TxRequest, as an NRF transaction could be RxRequest→TxResponse or TxRequest→RxResponse. Example: imsi-208014489186000

3.2.2 Data Director Message

The 5G SBI message that is received or forwarded contains the following components:

HTTP2: HTTP2 Headers - All received HTTP2 standard and 3gpp defined headers.
Received Data Director Message Payload

3.2.2.1 Data Director Message Format

The Data Director supports the following message formats:

HTTP2 Message Format
Synthetic Packet Message Format
Kafka Consumer Egress Feed Message Format

HTTP2 Message Format

Data Director supports HTTP2 feed for forwarding from Data Director to third-party monitoring consumer applications. 5G monitoring data is forwarded to third-party monitoring consumer using HTTP2 POST requests. The following components are delivered as JSON payload in the HTTP2 data frames:

Original received 5G SBI message headers as a header-list.
Original received 5g sbi data as 5g-sbi-message
Metadata-list added by NF

The 5g-sbi-message forwarded to third-party consumer application is Base64 encoded if:

The content-type header in the received 5G SBI message header list is multipart

The content-encoding in the received 5G SBI message header list indicates compression

If the "content-type" header in the received 5G SBI message header list is labeled as "multipart," the third-party consumer application performs an initial base64 decode. Subsequently, the application proceeds to process the message as multipart content.

When the "content-encoding" header in the received 5G SBI message header list shows compression, the third-party consumer application first applies base64 decode. Then, it processes the message further based on the content-encoding and content-type in the header list.

Synthetic Packet Message Format

OCNADD converts incoming JSON data into network transfer wire format and sends the converted packets to the third-party monitoring applications in a secure manner. The third-party probe feeds the synthetic packets to the internal monitoring applications. The feature helps third-party vendors to eliminate the need of creating additional applications to receive JSON data and converting the data into probe compatible format, thereby saving critical compute resources and associated costs.

Kafka Consumer Egress Feed Message Format

OCNADD supports the external Kafka consumer applications using the external Kafka Feeds. This enables third-party consumer applications to directly consume data from the Data Director Kafka topic, eliminating the need for any egress adapter.

Clients need to be authenticated through either SASL or SSL (mTLS) for authorization by Kafka. As a result, enabling external Kafka feed support requires specific settings to be activated within the Kafka broker. This ensures mandatory authentication of Kafka clients by the Kafka service.

OCNADD only allows authorized and authenticated third-party applications to use the Data Director Kafka service. Application authorization is handled using the KAFKA ACL (Access Control List) functionality. Access control for the external feed is established during Kafka feed creation. Presently, third-party applications are exclusively allowed to READ from a specific topic using a designated consumer group.

3.2.2.2 Third-Party Feed Format

Third-Party HTTP2 Feed Format

A third-party HTTP2 feed contains the following components:

Figure 3-1 Third-Party HTTP2 Feed Format

Following TLS options are supported:

TLSv1.2 (minimum) with oracle approved TLS Ciphers
TLSv1.2 with Static Key Cipher support (TLS_RSA_WITH_AES_128_GCM_SHA256)
No TLS (H2C)

Third-Party Synthetic Feed Format

A third-party synthetic feed contains the following components:

3.2.2.3 Example for the JSON Data

Note:

For more information on the metadata list, see Metadata.

Following is an example for the JSON data:

{
    "version":"Major.Minor.Patch",
    "metadata-list":{},
    "header-list":{
        ":authority":"10.75.224.64:30065",
        ":method":"PUT",
        ":path":"/USEast/nudm-uecm/v1/imsi-556670000000000/registrations/amf-3gpp-access",
        ":scheme":"http",
        "content-type":"application/json",
        "3gpp-sbi-target-apiroot":"http://udm1svc.default.svc.cluster.local:8080/USEast",
        "3gpp-sbi-message-priority":"5",
        "content-length":"501",
        "accept-encoding":"gzip",
        "user-agent":"Go-http-client/2.0"
    },
    "5g-sbi-message":{
        "guami":{
            "plmnId":{
                "mcc":"233",
                "mnc":"23"
            },
            "amfId":"100000"
        },
        "pei":"imei-456565651000000",
        "attrib1":"abcdefghijklmnopqurestuvwxqweoeqwowertyo123445678",
        "attrib2":"abcdefghijklmnopqurestuvwxqweoeqwowertyo123445678",
        "attrib3":"abcdefghijklmnopqurestuvwxqweoeqwowertyo123445678",
        "attrib4":"abcdefghijklmnopqurestuvwxqweoeqwowertyo432123445678",
        "attrib5":"abcdefghijklmnopqurestuvwxqweoeqwowert234yo123445678",
        "pcscfRestorationCallbackUri":"http://pcf1.pcf1svc.svc.cluster.local/notification/udmtest"
    }
}

3.3 Inbound Protocols for Non-Oracle NFs

Data Director currently supports HTTP2 Ingress Feed type, over which it received the 5GSBI messages and metadata added by the Non-Oracle NFs.

Non-Oracle NFs should be sending mirrored copy of actual HTTP2 request or response message (HTTP Header + Body) in the body of HTTP2 messages using POST method. The metadata fields from Non-Oracle NFs can be present either in "MESSAGE_HEADER" or "MESSAGE_BODY".

3.3.1 Data Transformation & Metadata Mapping

The message transformation functionality will allow data conversion and mapping from Non-Oracle NF to Oracle NF data which will be consumed by DD internal services for data processing. The conversion framework will provide capabilities to map the following metadata fields with OCNADD for processing.

The metadata fields from Non-Oracle NFs can be present either in "MESSAGE_HEADER" (as custom headers) or "MESSAGE_BODY". Based on the value of the parameter "metadataLocation" while creating configuration, the ingress adapter will take the attributes and perform the transformation of these fields to the Oracle Data Director format. If metadata is present in message body, then additional fields are required to be configured.

Metadata Mapping

Table 3-3 Metadata Mapping

Oracle Attribute Name	Ingress Attribute Name	Presence	Static Value(Default)	Description
correlation-id	<Ingress-attribute-name>	M	NA	Correlation id is mandatory to correlate all mirrored request and response messages of a transaction. If custom correlation id is not provided DD will attempt to retrieve this from 3gpp-Sbi-Correlation-Info header if available. It must be present in either of the two attributes.
timestamp	<Ingress-attribute-name>	M	NA	This property defines the timestamp of the request when it is initiated.
message-direction	<Ingress Attribute name(list)>	M	NA	It consists of both the messages direction (ingress or egress) and the message type (Request or Response). The non-Oracle feeds may send messages direction and message type in different custom headers. Oracle ingress adapter will combine both and map it to the supported OracleNfFeedDto.
consumer-fqdn	<Ingress Attribute name>	O	NA	The consumer fqdn will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
consumer-id	<Ingress Attribute name>	O	NA	The consumer id will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
hop-by-hop-id	<Ingress Attribute name>	O	NA	The hop by hop id will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
producer-fqdn	<Ingress Attribute name>	O	NA	The producer fqdn will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
producer-id	<Ingress Attribute name>	O	NA	The producer id will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
reroute-cause	<Ingress Attribute name>	O	NA	The reroute cause will be mapped with the received value of configured ingress attribute name in custom headers. If the value is not present, then it will be skipped.
feed-source-nf-type	<Ingress Attribute name>, Use feed-source Host Address mapping	M	<default-nf-type>	The "nf type" for OracleNfFeedDto will be mapped from the ingress attribute name which is provided during feed creation. However, if attribute name is not present in the custom headers, then feed-source host IP address will be taken from "custom-forward-for" or "x-forwarded-for" header if present and a look up will be performed from feed source host address mapping to get the nf-type. If the x-forwarded-for header is not present then Source IP of the request will be used. If the source IP is not present in the feed source host address map, then default value will be used for mapping. However, it is recommended to use default value when only one NF is producing data.
feed-source-nf-instance-id	<Ingress Attribute name>, Use feed-source Host Address mapping	C	<default-nf-instance-id>	The "nf instance id" for OracleNfFeedDto will be mapped from the ingress attribute name which is provided during feed creation. However, if attribute name is not present in the custom headers, then feed-source host IP address will be taken from "custom-forward-for" or "x-forwarded-for" header if present and a look up will be performed from feed source host address mapping to get the nf-instance-id. If the x-forwarded-for header is not present then Source IP of the request will be used. If the source IP is not present in the feed source host address map, then default value will be used for mapping. However, it is recommended to use default value when only one NF is producing data.
feed-source-nf-fqdn	<Ingress Attribute name>, Use feed-source Host Address mapping	C	<default-nf-fqdn>	The "nf instance fqdn" for OracleNfFeedDto will be mapped from the ingress attribute name which is provided during feed creation. However, if attribute name is not present in the custom headers, then feed-source host IP address will be taken from "custom-forward-for" or "x-forwarded-for" header if present and a look up will be performed from feed source host address mapping to get the nf-fqdn. If the x-forwarded-for header is not present then Source IP of the request will be used. If the source IP is not present in the feed source host address map, then default value will be used for mapping. However, it is recommended to use default value when only one NF is producing data.
feed-source-nf-pod-instance-id	<Ingress Attribute name>	O	<default-nf-pod-instance-id>	The "nf pod instance id" for OracleNfFeedDto will be mapped from the ingress attribute name which is provided during feed creation. However, if attribute name is not present in the custom headers, then default value will be used for mapping. It is recommended to use default value when only one NF is producing data.

3.3.2 Ingress Message Format for Non-Oracle NFs

The following diagram explains the format of the Ingress messages for non-Oracle network functions (NFs).

HTTP2 Ingress feed for non-Oracle NFs – Ingress Message Format

3.4 xDR

xDRs generated by correlation services are stored in corresponding xDR topic as JSON data. External application acting as Kafka consumer can subscribe to the xDR Kafka topic to read the xDR data. For more information on mandatory and optional xRD contents, see Oracle Communications Network Analytics Data Director User Guide.

3.4.1 xDR Format

Following is an example for xDR when includeMessageWithxDR option is set to none.

[
  {
    "version": "1.0.0",
    "beginTime": "2023-01-23T07:03:36.311Z",
    "endTime": "2023-01-23T07:03:36.311Z",
    "configurationName": "corr-test-2",
    "xdrStatus": "SUDR",
    "path": "/nudm-uecm/v1/imsi-208014489186000/registrations/smf-registrations/1",
    "supi": "imsi-208014489186000",
    "methodType": "PUT",
    "producerNfType": "SCP",
    "consumerFqdn": "SMF.5g.oracle.com",
    "producerFqdn": "UDM.5g.oracle.com",
    "contentType": "application/json",
    "ueId": "imsi-208014489186000",
    "pduSessionId": 1,
    "smfInstanceId": "8e81-4010-a4a0-30324ce870b2",
    "snssai": "{\u0022sst\u0022:1,\u0022sd\u0022:\u0022000001\u0022}",
    "pcfInstanceId": "8e81-4010-a4a0-30324823334"
  }
]

Note:

includeMessageWithxDR option allows user to select whether original feed message will be included with xDR or not and If included, which part of message to be included.

Below examples capture xDRs with includeMessageWithxDR

includeMessageWithxDR is set to DATA

[
    // XDR
    {
        "version": "1.0.0",
        "configurationName": "cap4c",
        "beginTime": "2023-06-26T14:40:29.950313200",
        "endTime": "2023-06-26T14:40:29.950313200",
        "xdrStatus": "SUDR",
        ….
    },
    // MESSAGE 1
    {
        "5g-sbi-message":
        {
            No change in data(
                if incoming data to DD is nested, same will be transferred)
        }
    },
    // END OF MESSAGE 1
    // MESSAGE 2
    {
        "5g-sbi-message":
        {
            No change in data(
                if incoming data to DD is nested, same will be transferred)
        }
    }
    // END OF MESSAGE 2
]

includeMessageWithxDR is set to HEADERS_DATA

[
  // XDR
  {
   "version": "1.0.0",
    "configurationName" : "cap4c",
    "beginTime" : "2023-06-26T14:40:29.950313200",
    "endTime" : "2023-06-26T14:40:29.950313200",
    "xdrStatus" : "SUDR",
          ….
  },
  // MESSAGE 1
 {
  "header-list":
     {
      No change in data
     },
   "5g-sbi-message":
     {                
      No change in data
     }
  }
  // END OF MESSAGE 1
]

includeMessageWithxDR is set to METADATA_HEADERS_DATA

[
 // XDR
  {
    "version": "1.0.0",
    "configurationName" : "cap4c",
    "beginTime" : "2023-06-26T14:40:29.950313200",
    "endTime" : "2023-06-26T14:40:29.950313200",
    "xdrStatus" : "SUDR",
     ….
  },
  // MESSAGE1
  {
    "metadata-list":{No change in format},
    "header-list":{No change in format},
    "5g-sbi-message":{No change in format}
  },
  // END OF MESSAGE 1
  // MESSAGE2
  {
    "metadata-list":{No change in format},
    "header-list":{No change in format},
    "5g-sbi-message":{No change in format}
  }
  // END OF MESSAGE 2 
]

Note:

The format of message would be same that is received in OCNADD from Oracle NFs.