3 Logs

This chapter explains the process to retrieve the logs and status that can be used for effective troubleshooting.

3.1 Log Levels

Logs register system events along with their date and time of occurrence. They also provide important details about a chain of events that could have led to an error or problem.

A log level helps in defining the severity level of a log message. For OCNADD, the log level of a microservice can be set to any one of the following valid values:

  • TRACE: A log level that describes events, as a step by step execution of code. This can be ignored during the standard operation, but may be useful during extended debugging sessions.
  • DEBUG: A log level used for events during software debugging when more granular information is needed.
  • INFO: A standard log level indicating that something has happened, an application has entered a certain state, etc.
  • WARN: A log level indicates that something unexpected has happened in the application, a problem, or a situation that might disturb one of the processes. But this does not mean that the application has failed. The WARN level should be used in situations that are unexpected, but the code can continue to work.
  • ERROR: A log level that should be used when an application hits an issue preventing one or more functionalities from functioning.

Using this information, the logs can be filtered based on the system requirements. For instance, if you want to filter the critical information about your system from the informational log messages, set a filter to view messages with only WARN log level in Kibana.

3.2 Configuring Log Levels

To view logging configurations and update logging levels, check the respective service child values.yaml.

Following is an example from the Configurations service:
env:
     CONFIGURATION_ROOT_LOG_LEVEL: INFO
     CONFIGURATION_WEB_LOG_LEVEL: INFO

Once the service child values.yaml is modified, perform helm upgrade for the OCNADD charts.

3.3 Collecting Logs

This section describes the steps to collect logs from PODs or containers. Perform the following steps:

  1. Run the following command to get the POD details:
    $ kubectl -n <namespace_name> get pods
  2. Collect the logs from the specific pods or containers:
    $ kubectl logs <podname> -n <namespace>
    Example:
    $ kubectl logs ocnaddconfiguration-xxxxxxxxxx-xxxxx -n ocnadd
  3. Store the log in a file using the following command:
    $ kubectl logs <podname> -n <namespace> > <filename>
    Example:
    $ kubectl logs ocnaddconfiguration-xxxxxxxxxx-xxxxx -n ocnadd > logs.txt
  4. (Optional) You can also use the following commands for the log stream with file redirection starting with the last 100 lines of the log:
    $ kubectl logs <podname> -n <namespace> -f --tail <number of lines> > <filename>
    Example:
    $ kubectl logs ocnaddconfiguration-xxxxxxxxxx-xxxxx -n ocnadd -f --tail 100 > logs.txt

For information on the OCNADD GUI user logs, see Oracle Communications Cloud Native Configuration Console Troubleshooting Guide.

3.4 Collect logs using Deployment Data Collector Tool

Perform this procedure to start the NF Deployment Data Collector module and generate the tarballs. If the user does not specify the output storage path, then this module generates the output in the same directory where the module is executed.

nfDataCapture.sh is a script which can be used for collecting all required logs from NF deployment for debugging issues. The script will collect logs from all Micro-Service PODs of specified helm input, helm deployment details,the status, description of all the kafka topics, offset details server properties and description of all the pods, services and events.

  • Ensure that you have appropriate privileges to access the system and execute kubectl and helm commands.
  • Perform this procedure on the same machine where the OCNADD is deployed using helm or kubectl.
  • Execute the chmod +x nfDataCapture.sh command on the tool to provide the executable permission.
  • Execute the following command to start the module:
      ./nfDataCapture.sh -n|--k8Namespace=[K8 Namespace] -u|--username=[User
        Name] -p|--password=[Password] -k|--kubectl=[KUBE_SCRIPT_NAME]  -h|--helm=[HELM_SCRIPT_NAME]
         -s|--size=[SIZE_OF_EACH_TARBALL]  -d|--cnDBTierStatus=[CN DB TIER STATUS]
        -x|--kafkaDetails=[KAFKA  DETAILS] -b|--binlogCollectionStatus=[BIN LOG COLLECTION STATUS]
        -o|--toolOutputPath -helm3=false

Examples:

./nfDataCapture.sh -k="kubectl --kubeconfig=admin.conf" -h="helm --kubeconfig admin.conf" -n=ocnrf -s=5M -o=/tmp/

./nfDataCapture.sh -n=ocnadd -s=5M -o=/tmp/

./nfDataCapture.sh -n=ocnadd

./nfDataCapture.sh -n=ocnadd -x=false

./nfDataCapture.sh -n=ocnadd -helm3=true

./nfDataCapture.sh -n=ocnadd -u=username -p=password -s=5M -o=/home/root/datacollector/data -helm3=true

./nfDataCapture.sh -n=ocnadd -u=username -p=password -s=5M -o=/home/root/datacollector/data -helm3=true -b=false

./nfDataCapture.sh -n=ocnadd -u=username -p=password -s=5M -o=/home/root/datacollector/data -helm3=true -d=true -b=false

Note:

Default size of tarball generated will be 10M, if not provided, and default location of output will be tool working directory.

Kafka Detailed Staus will be by default true and if we do not want to collect the details we have to pass it as false.

By default, helm2 is used. Use proper argument in command to use helm3.

Note:

If the database is not in same namespace , then the script should be run again for the namespace in which database is deployed to capture the database related logs.
  • Only if the size of the tar [example: ocnadd.debugData.2023.01.16_09.15.01.tar.gz] generated is greater than "SIZE_OF_EACH_TARBALL" specified in the command ,tar is split into several tarball based on the size specified.
  • After execution of command, tar-balls will be created based on size specified in the following format:

    <namespace>.debugData.<timestamp>

Example:

ocnadd.debugData.2023.01.16_09.15.01-part01

Each tarball can then be combined into one tarball with the following command:

cat ocnadd.debugData.2023.01.16_09.15.01-part* > onadd.debugData.2023.01.16_09.15.01-combined.tar.gz