2 Logs

This chapter explains the process to retrieve the logs and status that can be used for effective troubleshooting.

2.1 Log Levels

Logs register system events along with their date and time of occurrence. They also provide important details about a chain of events that could have led to an error or problem.

A log level helps in defining the severity level of a log message. For OCCM, the log level of a microservice can be set to any one of the following valid values:

  • DEBUG: A log level used for events during software debugging when more granular information is needed.
  • INFO: A standard log level indicating that something has happened, an application has entered a certain state, etc.
  • WARN: A log level indicates that something unexpected has happened in the application, a problem, or a situation that might disturb one of the processes. But this does not mean that the application has failed. The WARN level should be used in situations that are unexpected, but the code can continue to work.
  • ERROR: A log level that should be used when an application hits an issue preventing one or more functionalities from functioning.

Using this information, the logs can be filtered based on the system requirements. For instance, if you want to filter the critical information about your system from the informational log messages, set a filter to view messages with only WARN log level in Kibana.

2.2 Collecting Logs

This section describes the steps to collect logs from PODs and containers. Perform the following steps:

  • Run the following command to get the PODs details:
    $ kubectl -n <namespace_name> get pods
  • Run the following command to collect the logs from the specific pods or containers:
    $ kubectl logs <podname> -n <namespace> -c <containername>
  • Store the log in a file using the following command:
    $ kubectl logs <podname> -n <namespace>  > <filename>
  • (Optional) You can also run the following command for the log stream with file redirection starting with last 100 lines of log:
    $ kubectl logs <podname> -n <namespace> -f --tail <number of lines> > <filename>

2.3 Understanding Logs

This section provides information on how to read logs

Log JSON Format

The following is a sample log for OCCM services:

{
     "instant": <epoch time in nanoseconds>,
     "thread": <threadName>,
     "level": <log_level>,
     "loggerName": <name_of_the_logging_class>,
     "message": <message>,
     "contextMap": <context_map>,
     "threadId": <threadId>,
     "threadPriority": <threadPriority>,
     "messageTimestamp": <timestamp_in_readable_format>,
     "application": occm,
     "microservice": occm,
     "cluster": <Deployment cluster name>,
     "namespace": <release namespace>,
     "node": <K8s node on which pod is running>, 
     "pod": <name_of_the_pod>
}

The log message format is same for all the OCCM services. All logs are represented in JSON format.

The following table describes key attributes of a log message:

Table 2-1 Log Attributes

Attribute Description Example
instant The Date and Time the event occurred in epoch second and nano seconds "instant": { "epochSecond": 1590045388, "nanoOfSecond": 339789000}
thread Name of the thread "thread": "pool-7-thread-2"
level Level of the log. It can be: Log level (INFO, WARN, DEBUG, ERROR) "level": "INFO"
loggerName Name of the class that generated the log "loggerName": "com.oracle.cgbu.cne.occm.util.CmpUtils"
message Information about the event "message": "Started Application....."
contextMap It holds information added to threadContext, if any. "contextMap": { }
threadId Id of the thread "threadId":"34"
threadPriority Priority assigned to the thread "threadPriority": 5
messageTimestamp Time represented in human readable format and in UTC. Format is date:yyyy-MM-dd'T'HH:mm:ss.SSSZ EFK friendly and also follows Oracle Standards. "messageTimestamp": 2023-12-04T02:51:46.458+0000
application Application name "application":"occm"
microservice Micro service name "microservice":"occm"
cluster Deployemnt cluster name "cluster":"occm"
namespace Release namespace "namespace":"occncc-thrust5-01"
node K8s node where pod is running "node": "thrust5-k8s-node-31"
pod Name of the pods where the log is generated "pod" : "occm-occm-949b9c7d4-qh5k7"

OCCM Message Format

Name Description Example Possible Values
UUID UUID of the resource. Can be CertId or IssuerId CertId= 7e2aa1e2-17ee-4601-b334-3bfd0d677d55 Any randomly generated UUID.
Name Name of the resource.CertName or IssuerName CertName= SCPTLS Any string
NF Name of the NF NF= SCP Any string
ErrorCode In case of any failure, OCCM error code is logged. ErrorCode= ERR_PROCESS_START_FAILURE Error code from a list of OCCM error codes.
ErrorMsg In case of any failure, error message is logged. ErrorMsg=An error occurred during cert creation. Error running CMP command. NA
ErrorCause In case of any failure, error cause is logged. ErrorCause=java.io.IOException: error=2, No such file or directory NA

2.4 Accessing Logs

This section provides information about how to access the logs.

The OCCM application logs can be accessed in following ways:

  1. Run the following command to view logs of an OCCM application pod:
    $ kubectl logs -f -n <occm_namespace> <pod_name>

    For example:

    $ kubectl logs -f -n occm occm-occm-77df795fb5-wv2sb
  2. OCCM uses cloud native supported logging framework to view the logs.

    For example: Elasticsearch, Fluentd, and Kibana (EFK) can be used with CNC Console to view the OCCM logs.