Logs

2 Logs

This chapter explains the process to retrieve the logs and status that can be used for effective troubleshooting. The Oracle Communications Cloud Native Core, Network Repository Function (NRF) provides various sources of information that may be helpful in the troubleshooting process.

2.1 Log Levels

Logs register system events along with their date and time of occurrence. They also provide important details about a chain of events that could have led to an error or problem.

Supported Log Levels

For NRF, the log level for a microservice can be set to any of the following valid values:

TRACE: A log level describing events showing step by step execution of your code that can be ignored during the standard operation, but may be useful during extended debugging sessions.
DEBUG: A log level used for events considered to be useful during software debugging when more granular information is needed.
INFO: The standard log level indicating that something happened, the application entered a certain state, etc.
WARN: Indicates that something unexpected happened in the application, a problem, or a situation that might disturb one of the processes. But that doesn’t mean that the application failed. The WARN level should be used in situations that are unexpected, but the code can continue the work.
ERROR: The log level that should be used when the application hits an issue preventing one or more functionalities from properly functioning.

Configuring Log Levels

To view logging configurations and update logging levels, use the Logging Level page under Logging Level Options on the Cloud Native Core (CNC) Console. For more information, see the section "Logging Level Options" in Oracle Communications Cloud Native Core, Network Repository Function User Guide.

Configuring the Stack Trace Depth

NRF collects the stack trace when an exception occurs. These stack trace runs through multiple classes. NRF includes all the error stack traces in the log messages resulting in occupying the log storage space. Currently, NRF does not allow the users to configure the stack trace depth when a Java exception occurs.

From 25.1.100, NRF allows the users to configure the depth of the stack trace to be included when a Java exception occurs. The user can decide the depth of stack traces that should be included in the error log messages. Configure the global.maxJavaStackTraceDepth parameter in the global parameters section of ocnrf_custom_values_25.1.202.yaml to define the depth of the stack trace depth that should be included in the log messages. This is a reference variable. This value can also be configured at each NRF microservice. For more information about the above-mentioned parameter, see the "Global Parameters" section in Oracle Communications Cloud Native Core, Network Repository Function Installation, Upgrade, and Fault Recovery Guide.

Sample Stack Trace

When the value of is set as 0, all the stack traces are included:

{"instant":{"epochSecond":1733138161,"nanoOfSecond":529210823},"thread":"NfRegistration-DbSchemaVersionInfo-TablePolling2-thread-1","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.nfregistration.service.helper.DbSchemaVersionInfo","message":"{logMsg=An exception thrown by Table Polling Thread, Exception message=null, stackTrace=
[com.oracle.cgbu.cne.nrf.nfregistration.service.helper.DbSchemaVersionInfo.reloadSiteAndNetworkVersionInfo(DbSchemaVersionInfo.java:148),
 com.oracle.cgbu.cne.nrf.nfregistration.service.helper.DbSchemaVersionInfo.lambda$startTablePollingThread$0(DbSchemaVersionInfo.java:120),
 org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54),
 java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539),
 java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305),
 java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305),
 java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136),
 java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635), 
 java.base/java.lang.Thread.run(Thread.java:833)]}",
"endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":62,"threadPriority":5,"messageTimestamp":"2024-12-02T11:16:01.529+0000","configuredLevel":"WARN","processId":"1","nrfTxId":"","ocLogId":"","xRequestId":"","nfInstanceID":"","requesterNfType":"","nfFqdn":"","httpMethod":"","serviceOperation":"","featureStatus":"","requestUrl":"","hostname":"","subsystem":""}

When the value of maxJavaStackTraceDepth is set to 10, stack trace is included in the log messages as per the configuration defined for maxJavaStackTraceDepth.

{"instant":{"epochSecond":1730956275,"nanoOfSecond":188744679},"thread":"XNIO-2 task-3","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.actuator.CommonUtils","message":"DB connection health check failed with problemDetails {\      "title\":\"Internal Server Error\",\"status\":500,\"detail\":\"Failed to obtain JDBC Connection\",\"cause\":\"TIMED_OUT_REQUEST\"}. Exception occurred is Failed to obtain JDBC Connection and stackTrace is 
[
 org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:84),
 org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:378),
 org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:466),
 org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:476),
 com.oracle.cgbu.cne.nrf.actuator.CommonUtils.dbConnectionCheck(CommonUtils.java:33),
 com.oracle.cgbu.cne.nrf.actuator.CustomReadinessActuator.health(CustomReadinessActuator.java:25),
 org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37),
 org.springframework.boot.actuate.health.HealthEndpointWebExtension.getHealth(HealthEndpointWebExtension.java:94),
 org.springframew ork.boot.actuate.health.HealthEndpointWebExtension.getHealth(HealthEndpointWebExtension.java:47),
 org.springframework.boot.actuate.health.HealthEndpointSupport.getLoggedHealth(HealthEndpointSupport.java:172)
]",
 "endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":70,"threadPriority":5,"messageTimestamp":"2024-11-07T05:11:15.188+0000","configuredLevel":"WARN","processId":"1","nrfTxId":"nrf-tx-155440036","ocLogId":"","xReques      tId":"","nfInstanceID":"","requesterNfType":"","nfFqdn":"","httpMethod":"","serviceOperation":"","featureStatus":"","requestUrl":"","hostname":"ocnrf-nfregistration-69995d6c68-cml74","subsystem":"reloadNrfSystemOptions"}

2.2 Collecting Logs

This section describes steps to collect the logs from PODs or containers.

Run the following command to get the POD details:
```
$ kubectl -n <namespace_name> get pods
```
Collect the logs from the specific pods or containers:
```
$ kubectl logs <podname> -n <namespace>
```
Example:
```
$ kubectl logs ocnrf-nfaccesstoken-xxxxxxxxxx-xxxxx -n ocnrf
```

Store the log in a file using the following command:

$ kubectl logs <podname> -n <namespace>  > <filename>

Example:

$ kubectl logs ocnrf-nfaccesstoken-xxxxxxxxxx-xxxxx -n ocnrf > logs.txt

(Optional) You can also use the following commands for the log stream with file redirection starting with last 100 lines of log:
```
$ kubectl logs <podname> -n <namespace> -f --tail <number of lines> > <filename>
```
Example:
```
$ kubectl logs ocnrf-nfaccesstoken-xxxxxxxxxx-xxxxx -n ocnrf -f --tail 100 > logs.txt
```

For more information on how to collect the logs, see Oracle Communications Cloud Native Core Data Collector Guide.

2.3 Understanding Logs

This chapter explains the logs you need to look into, to handle different NRF debugging issues.

This section provides log level attribute details for following services:

nfregistration
nfdiscovery

Sample Logs

Sample log statement nfregistration:

{"instant":{"epochSecond":1604654859,"nanoOfSecond":278610000},"thread":"XNIO-1 task-1","level":"WARN","loggerName":"com.oracle.cgbu.cne.nrf.service.NfMgmtServiceImpl","message":"NF status changed for nfInstanceId 8faf1bbc-5324-4454-a507-a14ef8e1bc2c, whose previous status was SUSPENDED and current status is REGISTERED.","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","nfInstanceID": "8faf1bbc-5324-4454-a507-a14ef8e1bc2c","threadId":49,"threadPriority":5,"messageTimestamp":"2020-11-06T09:27:39.278+0000","configuredLevel":"INFO","serviceOperation":"nfRegister","requesterNfType": "UDM","nfFqdn": "UNKNOWN","processId":"1","ocLogId":"1604654852868_56_ocnrf-ingressgateway-59548646fc-g7qtk:1604654859244_49_ocnrf-nfregistration-65b999468b-gchg8"}

Sample ERROR log statement for nfregistration when Error Log Messages Enhancement feature is enabled:

{"instant":{"epochSecond":1717082050,"nanoOfSecond":263756204},"thread":"XNIO-1
                task-2","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.rest.NFManagementController","message":"Response
                sent: 400 Bad Request for uri :
                http://ocnrf-ingressgateway.nrf1-ns/nnrf-nfm/v1/nf-instances/fcff26ea-6c8d-4502-968b-982532f48dcf
                with the problem cause : MANDATORY_IE_INCORRECT and problem details :
                NRF-d5g.oracle.com: Nnrf_NFManagement: Bad Request:
                ONRF-REG-REGN-E0007","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2024-05-30T15:14:10.263+0000","configuredLevel":"WARN","processId":"1","ocLogId":"1717082050214_77_ocnrf-ingressgateway-854464d548-426bz:1717082050236_76_ocnrf-nfregistration-d5d7bb9ff-gdlct","xRequestId":"","nfInstanceID":"fcff26ea-6c8d-4502-968b-982532f48dcf","requesterNfType":"AMF","nfFqdn":"UNKNOWN","httpMethod":"PUT","serviceOperation":"nfRegister","featureStatus":"","requestUrl":"","hostname":"ocnrf-nfregistration-d5d7bb9ff-gdlct","subsystem":"","errorStatus":"400","errorTitle":"Bad
                Request","errorDetails":"NRF-d5g.oracle.com:   Nnrf_NFManagement: Bad Request:
                ONRF-REG-REGN-E0007","errorCause":"MANDATORY_IE_INCORRECT","sender":"NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c","subscriberId":"imsi-345012123123125"}

Sample log statement nfdiscovery:

{"instant":{"epochSecond":1604655402,"nanoOfSecond":946649000},"thread":"XNIO-1 task-1","level":"INFO","loggerName":"com.oracle.cgbu.cne.nrf.rest.NFDiscoveryController","message":"Request received with uri http://10.75.226.148:30547/nnrf-disc/v1/nf-instances and http-method GET"}","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":47,"threadPriority":5,"messageTimestamp":"2020-11-06T09:36:42.946+0000","configuredLevel":"INFO","serviceOperation":"NFDiscover","processId":"1","ocLogId":"1604655396595_56_ocnrf-ingressgateway-59548646fc-g7qtk:1604655402932_47_ocnrf-nfdiscovery-6df9dbb6bb-kzbgl"}

Sample ERROR log statement for nfdiscovery when Error Log Messages Enhancement feature is enabled:

{"instant":{"epochSecond":1717082579,"nanoOfSecond":276265496},"thread":"discovery-h2c-epoll-2","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.rest.NFDiscoveryController","message":"Response
      sent: 400 Bad Request for uri : /nnrf-disc/v1/nf-instances?{target-nf-type=[AMF],
      requester-nf-type=[ABC]} with the problem cause : MANDATORY_QUERY_PARAM_INCORRECT and problem
      details : NRF-d5g.oracle.com: Nnrf_NFDiscovery: requesterNfType should be a valid value:
      ONRF-DIS-DISC-E0004","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":5286,"threadPriority":5,"messageTimestamp":"2024-05-30T15:22:59.276+0000","configuredLevel":"WARN","processId":"1","ocLogId":"1717082579244_250_ocnrf-ingressgateway-854464d548-426bz:1717082579263_5286_ocnrf-nfdiscovery-76f946759c-gg7lg","serviceOperation":"NFDiscover","xRequestId":"","requesterNfType":"ABC","targetNfType":"AMF","discoveryQuery":"/nnrf-disc/v1/nf-instances?{target-nf-type=[AMF],
      requester-nf-type=[ABC]}","hostname":"ocnrf-nfdiscovery-76f946759c-gg7lg","subsystem":"","errorStatus":"400","errorTitle":"Bad
      Request","errorDetails":"NRF-d5g.oracle.com: Nnrf_NFDiscovery: requesterNfType should be a
      valid value:
      ONRF-DIS-DISC-E0004","errorCause":"MANDATORY_QUERY_PARAM_INCORRECT","sender":"NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c","subscriberId":"imsi-345012123123126"}

Table 2-1 Log Attribute Details

Log Attribute	Details	Sample Value	Data Type
instant	Epoch time Note: It is group of two values epochSecond and nanoOfSecond	{"epochSecond":1604655402,"nanoOfSecond":946649000}	Object
thread	Logging Thread Name	"XNIO-1 task-1"	String
level	Log Level of the log printed	"WARN"	String
loggerName	Class or Module which printed the log	"com.oracle.cgbu.cne.nrf.rest.NFDiscoveryController"	String
message	Message related to the log providing brief details Note: Indicates that no NFProfiles found for mentioned search query	"{Discovery Query=target-nf-type=AMF&requester-nf-type=UDM, logMsg=No NFProfiles found for query}"	String
endOfBatch	Log4j2 Internal Default from log4j2: false	false	boolean
loggerFqcn	Log4j2 Internal Fully Qualified class name of logger module	org.apache.logging.log4j.spi.AbstractLogger	String
threadId	Thread Id generated internally by Log4j2	47	Integer
threadPriority	Thread Priority set internally by Log4j2	5	Integer
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-11-06T09:36:42.946+0000"	String
configuredLevel	Log Level configured	"INFO"	String
serviceOperation	Indicates the service operation	"NFDiscover"	String
processId	Process ID internally assigned	"1"	String
ocLogId	End to End Log Identifier across the NRF microservices Note: Helps to correlate the logs across the microservices in NRF application	"ocLogId":"1604655396595_56_ocnrf-ingressgateway-59548646fc-g7qtk :1604655402932_47_ocnrf-nfdiscovery-6df9dbb6bb-kzbgl"	String
nfInstanceID	Provide the NF Instance Id details		String
requesterNfType	For Registration request, this indicates the message originator fetched from NF Type in the User Agent HeaderFor Discovery query, this indicates the Requester NF Type		String
targetNfType	Indicates the Discovery Query Target NF Type		String
nfFqdn	Value of NF FQDN. This is extracted from XFCC header.		String
httpMethod	Indicates the httpMethod. For example: PUT, GET, PATCH		String
requestUrl	Indicates the URL of the message		String
discoveryQuery	This is populated in the log for response message in case of response for NFDiscover service operation.		String
errorStatus	Indicates the status sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	400	String
errorTitle	Indicates the title sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	Bad Request	String
errorDetails	Indicates the detail sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-d5g.oracle.com: Nnrf_NFDiscovery: requesterNfType should be a valid value: ONRF-DIS-DISC-E0004	String
errorCause	Indicates the cause sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	MANDATORY_QUERY_PARAM_INCORRECT	String
sender	Indicates the NRF-NRF Instance Id or Server header value as received in the error response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c	String
receiver	Indicates the NRF-NRF Instance Id Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.		String
subscriberId	Indicates the 3GPP-Sbi-Correlation-Info header present in the request header received at NRF 3GPP-Sbi-Correlation-Info header present in the response header received from the peer in the case of NRF acting as a client. Either SUPI or GPSI values or both SUPI and GPSI values. Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	imsi-345012123123126	String

This section provides log level attribute details for following services:

nfsubscription
nfaccesstoken

Sample Logs

Sample log statement nfsubscription:

{"instant":{"epochSecond":1604654864,"nanoOfSecond":237018000},"thread":"XNIO-1 task-1","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.service.NfSubsServiceImpl","message":"{logMsg=No subscriptions found matching the notification for Profile, nfProfile={\"nfInstanceId\":\"dd94d4b5-0ce6-4571-a661-3ff7c21a79a7\",\"nfType\":\"CUSTOM_SPF\",\"nfStatus\":\"REGISTERED\",\"heartBeatTimer\":30,\"fqdn\":\"UPF.d5g.oracle.com\",\"interPlmnFqdn\":\"UPF-d5g.oracle.com\",\"ipv4Addresses\":[\"192.168.2.100\",\"192.168.3.100\",\"192.168.2.110\",\"192.168.3.110\"],\"ipv6Addresses\":[\"2001:0db8:85a3:0000:0000:8a2e:0370:7334\"],\"additionalAttributes\":{\"recordCreator\":\"6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c\",\"creationTimestamp\":1604654861984000,\"requesterNfFqdn\":null,\"lastUpdateTimestamp\":1604654861984000},\"customInfo\":{\"key1\":\"value1\",\"key2\":\"value2\"}}, eventType=NF_DEREGISTERED}","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":49,"threadPriority":5,"messageTimestamp":"2020-11-06T09:27:44.237+0000","configuredLevel":"INFO","subsystem":"notifyProfileDeregistration","processId":"1","ocLogId":"1604654857899_56_ocnrf-ingressgateway-59548646fc-g7qtk:1604654864210_49_ocnrf-nfregistration-65b999468b-gchg8"}

Sample ERROR log statement for nfsubscription when Error Log Messages Enhancement feature is enabled:

{"instant":{"epochSecond":1717077558,"nanoOfSecond":256932930},"thread":"boundedElastic-8","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.routes.SubscriptionHandler","message":"Response
                sent: 500 Subscription global limit breached for uri :
                http://ocnrf-ingressgateway.madhu-ns/nnrf-nfm/v1/subscriptions with the   problem
                cause : INSUFFICIENT_RESOURCES and problem details : NRF-d5g.oracle.com:
                Nnrf_NFManagement: Subscription global limit breached:
                ONRF-SUB-SUBSCR-E2003","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":39547,"threadPriority":5,"messageTimestamp":"2024-05-30T13:59:18.256+0000","configuredLevel":"WARN","subsystem":"NfSubscribe","processId":"1","ocLogId":"1717077558225_77_ocnrf-ingressgateway-854464d548-426bz:1717077558237_110_ocnrf-nfsubscription-766d45f5c7-t4xp8","xRequestId":"","numberOfRetriesAttempted":"","hostname":"ocnrf-nfsubscription-766d45f5c7-t4xp8","errorStatus":"500","errorTitle":"Subscription
                global limit breached","errorDetails":"NRF-d5g.oracle.com:   Nnrf_NFManagement:
                Subscription global limit breached:
                ONRF-SUB-SUBSCR-E2003","errorCause":"INSUFFICIENT_RESOURCES","sender":"NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c","subscriberId":"imsi-345012123123126"}

Sample log statement nfaccesstoken:

{"instant":{"epochSecond":1604512030,"nanoOfSecond":714594000},"thread":"XNIO-1 task-1","level":"INFO","loggerName":"com.oracle.cgbu.cne.nrf.service.AccessTokenServiceImpl","message":"{logMsg=Successfully generated token}","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":53,"threadPriority":5,"messageTimeStamp":"20-11-0417:47:10.714+0000","configuredlevel":"INFO","subsystem":"accessToken","processId":"1","ocLogId":"1604512030507_73_ocnrf-ingressgateway-7cbcff6b47-9qw9b:1604512030525_53_ocnrf-nfaccesstoken-6bddf44c66-86lmf"}

Sample ERROR log statement for nfaccesstoken when Error Log Messages Enhancement feature is enabled:

{"instant":{"epochSecond":1717078943,"nanoOfSecond":335260003},"thread":"XNIO-1
                task-2","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.rest.NFAccessTokenController","message":"Response
                sent: 400 Bad Request for uri :   http://ocnrf-ingressgateway.madhu-ns/oauth2/token
                with the problem cause :   Bad Request and problem details : Bad
                Request","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":71,"threadPriority":5,"messageTimestamp":"2024-05-30T14:22:23.335+0000","configuredLevel":"WARN","subsystem":"accessToken","processId":"1","ocLogId":"1717078942906_77_ocnrf-ingressgateway-854464d548-426bz:1717078942947_71_ocnrf-nfaccesstoken-79b55d9b45-qm4gn","xRequestId":"","hostname":"ocnrf-nfaccesstoken-79b55d9b45-qm4gn","errorStatus":"400","errorTitle":"Bad
                Request","errorDetails":"NRF-d5g.oracle.com:   Nnrf_AccessToken: NfType in oAuth request is different from the registered   profile:
                ONRF-ACC-ACTOK-E4004","errorCause":"unauthorized_client","sender":"NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c","subscriberId":"imsi-345012123123129"}

Table 2-2 Log Attribute Details

Log Attribute	Details	Sample Value	Data Type
instant	Epoch time Note: It is group of two values epochSecond and nanoOfSecond	{"epochSecond":1604655402,"nanoOfSecond":946649000}	Object
thread	Logging Thread Name	"XNIO-1 task-1"	String
level	Log Level of the log printed	"WARN"	String
loggerName	Class or Module which printed the log	"com.oracle.cgbu.cne.nrf.rest.NFDiscoveryController"	String
message	Message related to the log providing brief details Note: Indicates that no NFProfiles found for mentioned search query	"{Discovery Query=target-nf-type=AMF&requester-nf-type=UDM, logMsg=No NFProfiles found for query}"	String
endOfBatch	Log4j2 Internal Default from log4j2: false	false	boolean
loggerFqcn	Log4j2 Internal Fully Qualified class name of logger module	org.apache.logging.log4j.spi.AbstractLogger	String
threadId	Thread Id generated internally by Log4j2	47	Integer
threadPriority	Thread Priority set internally by Log4j2	5	Integer
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-11-06T09:36:42.946+0000"	String
configuredLevel	Log Level configured	"INFO"	String
subsystem	Subsystem inside microservice internal to microservice	discoveryNfInstances	String
processId	Process ID internally assigned	"1"	String
ocLogId	End to End Log Identifier across the NRF microservices Note: Helps to correlate the logs across the microservices in NRF application	"ocLogId":"1604655396595_56_ocnrf-ingressgateway-59548646fc-g7qtk :1604655402932_47_ocnrf-nfdiscovery-6df9dbb6bb-kzbgl"	String
errorStatus	Indicates the status sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	400	String
errorTitle	Indicates the title sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	Bad Request	String
errorDetails	Indicates the detail sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-d5g.oracle.com: Nnrf_AccessToken: NfType in oAuth request is different from the registered profile: ONRF-ACC-ACTOK-E4004	String
errorCause	Indicates the cause sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	unauthorized_client	String
sender	Indicates the NRF-NRF Instance Id or Server header value as received in the error response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c	String
receiver	Indicates the NRF-NRF Instance Id Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.		String
subscriberId	Indicates the 3GPP-Sbi-Correlation-Info header present in the request header received at NRF 3GPP-Sbi-Correlation-Info header present in the response header received from the peer in the case of NRF acting as a client. Either SUPI or GPSI values or both SUPI and GPSI values. Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	imsi-345012123123129	String

This section provides log level attribute details for following services:

nrfauditor
nrfconfiguration

Sample log statement nrfauditor:

{"instant":{"epochSecond":1604654345,"nanoOfSecond":125124000},"thread":"main","level":"INFO","loggerName":"com.oracle.cgbu.cne.nrf.audit.AuditManager","message":"{logMsg=Started NF Profile Auditor Thread, Id=25, Name=NF Profile Auditor}","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5,"messageTimestamp":"2020-11-06T09:19:05.125+0000","configuredLevel":"INFO","subsystem":"AuditManager","processId":"1"}

Sample log statement nrfconfiguration:

{"instant":{"epochSecond":1604654450,"nanoOfSecond":117406000},"thread":"XNIO-1 task-1","level":"INFO","loggerName":"com.oracle.cgbu.cne.nrf.rest.NrfConfigurationController","message":"{logMsg=Update GeneralOptions request received with generalOptions, generalOptions={\"additionalAttributes\":null,\"nrfPlmnList\":[{\"mcc\":\"310\",\"mnc\":\"14\"}],\"enableF3\":null,\"enableF5\":null,\"maximumHopCount\":null,\"defaultLoad\":null,\"defaultPriority\":null,\"defaultPriorityAssignment\":null,\"defaultLoadAssignment\":null,\"ocnrfHost\":null,\"ocnrfPort\":null}}","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":34,"threadPriority":5,"messageTimestamp":"2020-11-06T09:20:50.117+0000","configuredLevel":"INFO","subsystem":"updateNrfGeneralOptions","processId":"1"

Sample ERROR log statement for nrfconfiguration when Error Log Messages Enhancement feature is enabled:

{"instant":{"epochSecond":1717082305,"nanoOfSecond":790424465},"thread":"XNIO-1
      task-3","level":"ERROR","loggerName":"com.oracle.cgbu.cne.nrf.rest.NrfConfigurationController","message":"Response
      sent: 400 Bad Request for uri :
      http://ocnrf-nrfconfiguration:8080/nrf-configuration/v1/generalOptions with the problem cause
      : MANDATORY_IE_INCORRECT and problem details : NRF-d5g.oracle.com: Nnrf_Internal_Config:
      Multiple attributes are missing or incorrect:
      ONRF-CFG-GENOPT-E0021","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":68,"threadPriority":5,"messageTimestamp":"2024-05-30T15:18:25.790+0000","configuredLevel":"WARN","subsystem":"updateNrfGeneralOptions","processId":"1","xRequestId":"","hostname":"ocnrf-nrfconfiguration-6956dd5454-27vh2","errorStatus":"400","errorTitle":"Bad
      Request","errorDetails":"NRF-d5g.oracle.com: Nnrf_Internal_Config: Multiple attributes are
      missing or incorrect:
      ONRF-CFG-GENOPT-E0021","errorCause":"MANDATORY_IE_INCORRECT","sender":"NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c"}

Table 2-3 Log Attribute Details for nrfauditor and nrfconfiguration

Log Attribute	Details	Sample Value	Data Type	Notes
instant	Epoch time	{"epochSecond":1604654450,"nanoOfSecond":117406000}	Object	It is group of two values epochSecond and nanoOfSecond.
thread	Logging Thread Name	"XNIO-1 task-1"	String
level	Log Level of the log printed	"INFO"	String
loggerName	Class or Module which printed the log	"com.oracle.cgbu.cne.nrf.rest.NrfConfigurationController"	String
message	Message related to the log providing brief details	"{logMsg=Update GeneralOptions request received with generalOptions, generalOptions={\"additionalAttributes\":null, \"nrfPlmnList\":[{\"mcc\":\"310\",\"mnc\":\"14\"}], \"enableF3\":null,\"enableF5\":null,\"maximumHopCount\":null,\"defaultLoad\":null, \"defaultPriority\":null,\"defaultPriorityAssignment\":null,\"defaultLoadAssignment\":null, \"ocnrfHost\":null,\"ocnrfPort\":null}}"	String	It states that a request is received to update the generaloptions. generaloptions received is also printed in the message.
endOfBatch	Log4j2 Internal Default from log4j2: false	false	Boolean
loggerFqcn	Log4j2 Internal Fully Qualified class name of logger module	org.apache.logging.log4j.spi.AbstractLogger	String
threadId	Thread Id generated internally by Log4j2	34	Integer
threadPriority	Thread Priority set internally by Log4j2	5	Integer
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-11-06T09:20:50.117+0000"	String
configuredLevel	Log Level configured	"INFO"	String
subsystem	Subsystem inside microservice internal to microservice	"updateNrfGeneralOptions"	String
processId	Process ID internally assigned	"1"	String
errorStatus	Indicates the status sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	400	String
errorTitle	Indicates the title sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	Bad Request	String
errorDetails	Indicates the detail sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-d5g.oracle.com: Nnrf_Internal_Config: Multiple attributes are missing or incorrect: ONRF-CFG-GENOPT-E0021	String
errorCause	Indicates the cause sent or received by NRF in ProblemDetails of HTTP response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	MANDATORY_IE_INCORRECT	String
sender	Indicates the NRF-NRF Instance Id or Server header value as received in the error response Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.	NRF-6faf1bbc-6e4a-4454-a507-a14ef8e1bc5c	String
receiver	Indicates the NRF-NRF Instance Id Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.		String
subscriberId	Indicates the 3GPP-Sbi-Correlation-Info header present in the request header received at NRF 3GPP-Sbi-Correlation-Info header present in the response header received from the peer in the case of NRF acting as a client. Either SUPI or GPSI values or both SUPI and GPSI values. Note:This attribute will be added only for the ERROR logs when the Error Log Enhancement feature is enabled.		String

This section provides log level attribute details for the ingressgateway service:

Sample log statement ingressgateway:

{"thread":"ingress-h2c-epoll-3","level":"DEBUG","loggerName":"ocpm.cne.gateway.filters.PreGatewayFilter","message":"Exiting PreGatewayFilter","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","instant":{"epochSecond":1604650229,"nanoOfSecond":4993000},"contextMap":{"hostname":"ocnrf-ingressgateway-69f6544b8d-cdbgx","ingressTxId":"ingress-tx-1087436877","ocLogId":"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx"},"threadId":72,"threadPriority":5,"messageTimestamp":"2020-11-06 08:10:29.004","ocLogId":"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx","pod":"ocnrf-ingressgateway-69f6544b8d-cdbgx","processId":"1","instanceType":"prod","ingressTxId":"ingress-tx-1087436877"}

Table 2-4 Log Attribute Details for ingressgateway

Log Attribute	Details	Sample Value	Data Type	Notes
thread	Logging Thread Name	"ingress-h2c-epoll-3"	String
level	Log Level of the log printed	"DEBUG"	String
loggerName	Class or Module which printed the log	"ocpm.cne.gateway.filters.PreGatewayFilter"	String
message	Message related to the log providing brief details	"Exiting PreGatewayFilter"	String	Indicates that the method PreGatewayFilter is being exited.
endOfBatch	Log4j2 Internal Default from log4j2: false	false	boolean
loggerFqcn	Log4j2 Internal Fully Qualified class name of logger module	org.apache.logging.log4j.spi.AbstractLogger	String
instant	Epoch timestamp	{"epochSecond":1604650229,"nanoOfSecond":4993000}	Object	It is group of two values epochSecond and nanoOfSecond
contextMap	contents of log4j ThreadContext map	{"hostname":"ocnrf-ingressgateway-69f6544b8d-cdbgx", "ingressTxId":"ingress-tx-1087436877", "ocLogId":"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx"}	Object
threadId	Thread Id generated internally by Log4j2	72	Integer
threadPriority	Thread Priority set internally by Log4j2	5	Integer
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-11-06 08:10:29.004"	String
ocLogId	End to End Log Identifier across the NRF microservices.	"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx"	String	Helps to correlate the logs across the microservices in OCNRF application
pod	Pod Name	"ocnrf-ingressgateway-69f6544b8d-cdbgx"	String
processId	Process ID internally assigned	"1"	String
instanceType	Instance type	"prod"	String
ingressTxId	Transaction id that is added to log4j ThreadContext map and is unique to every transaction	"ingress-tx-1087436877"	String

This section provides log level attribute details for the egressgateway service:

Sample log statement egressgateway:

{"thread":"main","level":"DEBUG","loggerName":"ocpm.cne.gateway.config.ScpDynamicBeanConfiguration","message":"Property name: server.port and value: 8080","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","instant":{"epochSecond":1604564777,"nanoOfSecond":135977000},"contextMap":{},"threadId":1,"threadPriority":5,"messageTimestamp":"2020-11-05 08:26:17.135","ocLogId":"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx","pod":"ocnrf-egressgateway-69f6544b8d-cdbgx","processId":"1","instanceType":"prod","egressTxId":"egress-tx-1087436877"}

Table 2-5 Log Attribute Details for egressgateway

Log Attribute	Details	Sample Value	Data Type	Notes
thread	Logging Thread Name	"main"	String
level	Log Level of the log printed	"DEBUG"	String
loggerName	Class or Module which printed the log	"ocpm.cne.gateway.config.ScpDynamicBeanConfiguration"	String
message	Message related to the log providing brief details	"Property name: server.port and value: 8080"	String
endOfBatch	Log4j2 Internal Default from log4j2: false	false	boolean
loggerFqcn	Log4j2 Internal Fully Qualified class name of logger module	org.apache.logging.log4j.spi.AbstractLogger	String
instant	Epoch timestamp	{"epochSecond":1604564777,"nanoOfSecond":135977000	Object	It is group of two values epochSecond and nanoOfSecond
contextMap	Elements in log4j ThreadContext map	{}	Object
threadId	Thread Id generated internally by Log4j2	1	Integer
threadPriority	Thread Priority set internally by Log4j2	5	Integer
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-11-05 08:26:17.135"	String
ocLogId	End to End Log Identifier across the NRF microservices	"1604650229002_72_ocnrf-ingressgateway-69f6544b8d-cdbgx"	String	Helps to correlate the logs across the microservices in OCNRF application
pod	Name of the egress pod	"ocnrf-egressgateway-69f6544b8d-cdbgx"	String
processId	Process ID internally assigned	"1"	String
instanceType	Instance type	"prod"	String
egressTxId	Transaction id that is added to log4j ThreadContext map and is unique to every transaction	"egress-tx-1087436877"	String

This section provides log level attribute details for the appinfo service:

Sample log statement appinfo:

{"name": "gunicorn.access","message": "127.0.0.1 - - [29/Oct/2020:18:39:35 +0000]\"GET /v1/liveness HTTP/1.1\" 200 16 \"-\"\"kube-probe/1.17+\"", "level": "INFO","filename":"glogging.py","lineno": 344,"module": "glogging","func":"access","thread":"MainThread","created": "2020-10-29T18:39:35.799448"}

Table 2-6 Log Attribute Details for appinfo

Log Attribute	Details	Sample Value	Data Type
name	Logger name	"gunicorn.access"	String
message	Message related to the log providing brief details	"message": "127.0.0.1 - - [29/Oct/2020:18:39:35 +0000] \"GET /v1/liveness HTTP/1.1\" 200 16 \"-\" \"kube-probe/1.17+\""	String
level	Log Level of the log printed	"INFO"	String
filename	File name which generates the log	"glogging.py"	String
lineno	Line number	344	Integer
module	Python module name	"glogging"	String
func	Function Name	"func"	String
thread	Thread Name	"MainThread"	String
messageTimestamp	Timestamp of log from application container. Format: yyyy-MM-dd'T'HH:mm:ss.SSSZ	"2020-10-29T18:39:35.799448"	String

Common Useful log attributes

The below log attributes will be available only through Kibana. These attribute names are part of Kubernetes labels which are added in each NRF POD.

Table 2-7 Common useful log attributes

Log Attribute	Details	Sample Value	Data Type	Notes
application	NF Type	"ocnrf"	String	complete attribute name in Kibana: kubernetes.labels.application
microservice	Kubernetes service name Format: <Helm-Release-Name>-<Microservice-Name>	(assuming the <Helm-Release-Name> is "nrfnorth" "nrfnorth-nfregistration" "nrfnorth-nfdiscovery" "nrfnorth-nfsubscription" "nrfnorth-nfaccesstoken" "nrfnorth-appinfo" "nrfnorth-nrfauditor" "nrfnorth-nrfconfiguration" "nrfnorth-ingressgateway" "nrfnorth-egressgateway"	String
engVersion	Engineering version	"1.8.0"	String
mktgVersion	Marketing version	"1.8.0.0.0"	String
vendor	Vendor Name	"Oracle"	Integer