2 Logs

This chapter explains the process to retrieve the logs and status that can be used for effective troubleshooting.

2.1 Collecting Logs

This section describes the steps to collect logs from PODs and containers. Perform the following steps:

  1. Run the following command to get the PODs details:
    $ kubectl -n <namespace_name> get pods
  2. Collect the logs from the specific pods or containers:
    $ kubectl logs <podname> -n <namespace> -c <containername>
  3. Store the log in a file using the following command:
    $ kubectl logs <podname> -n <namespace>  > <filename>
  4. (Optional) You can also use the following commands for the log stream with file redirection starting with last 100 lines of log:
    $ kubectl logs <podname> -n <namespace> -f --tail <number of lines> > <filename>

For more information on kubectl commands, see Kubernetes website.

2.2 Log Formats

This section provides information about the log formats.

Log4j JSON Format

CNCC Message Format

Log4j JSON Format

Following is the log format in JSON:

{
     "thread": <threadName>,
     "level": <log_level>,
     "loggerName": <name_of_the_logging_class>,
     "message": <message>,
     "instant": <timestamp_in_miliseconds>,
     "messageTimestamp": <timestamp_in_readable_format>,
     "threadId": <threadId>,
     "threadPriority": <threadPriority>,
     "pod": <name_of_the_pod>,
     "processId": <processId>,
     "contextMap": <context_map>,
     "ocLogId": <unique_trace_id_for_every_request>,
     "instanceType": <instanceType>,
     "ingressTxId": <IngressTransactionId>
 }

Table 2-1 Log Details

Name Description Example
thread Name of the thread. "thread": "reactor-http-epoll-1"
level Level of the log. It can be: Log level (INFO, WARN, DEBUG, TRACE) "level": "INFO"
loggerName Name of the class that generated the log. "loggerName": "ocpm.cne.gateway.cncc.GatewayApplication"
messageTimestamp Time represented in human readable format and in UTC. Format is date:yyyy-MM-dd'T'HH:mm:ss.SSSZ EFK friendly and also follows Oracle Standards. "messageTimestamp": 2020-07-04'T'12:00:40.702Z
message Information about the event. "message": "Started Application....."

By default, all messages are in simple string except Audit Log, Security Log which are represented in CNC Console Message Format..

instant The Date and Time the event occurred in epoch second and nano seconds. "instant": { "epochSecond": 1590045388, "nanoOfSecond": 339789000}
processId Linux process Identifier (for a multi-process host. Linux process Identifier (for a multi-process host).
threadId Id of the thread. "threadId":"43"
threadPriority Priority assigned to the thread "threadPriority": 5
pod Name of the pods where the log is generated "cncc-mcore-ingress-gateway-77df795fb5-wv2sb"
contextMap It holds information added to threadContext. "contextMap": { "hostname": "cncc-mcore-ingress-gateway-77df795fb5-wv2sb", "ingressTxId": "ingress-tx-1460885598"}
ocLogId It contains the trace id that is uniquely generated for every request of the format "<timestamp(in milliseconds)>_<thread Id>_<POD name>" It contains the trace id that is uniquely generated for every request of the format "<timestamp (in milliseconds)>_<thread Id>_<POD name>"
instanceType Static tag which implies that instance type is production "instanceType": "prod"
ingressTxId It contains id of the format "ingress-tx-<Random no>" to track every transaction "ingressTxId": ingress-tx-1904660570

CNC Console Message Format

Table 2-2 CNC Console Message Format

Name Description Example Possible Values
logType Indicates whether it is Security Log or Audit Log. logType=AUDIT AUDIT

SECURITY
type Indicates nature or action of the log. type=REQUEST For Security Log: REQUEST, RESPONSE

For Audit Log: LOGIN, ACCESS_RESOURCE, ACCESS_RESOURCE_ERROR, LOGOUT
resourceType Indicates what is the resource being requested for. resourceType=SCP CM_SERVICE (For default route)

CNC Console (For User Login Activity)

SCP

UDR

NRF

PCF

(all CNC Console supported NFs)
userId Id of the user who triggered request or action. userId=3314f54f-08bf-489d-b395-27bf56da1262 NA
username Name of the user username= "user1" NA
status HTTP status of the response. status=200 OK NA
operationType HTTP method of the request. operationType=GET NA
scheme Indicates the scheme of the request. scheme=http NA
remoteAddress The remote address that is associated with the request. It also means the remote address to where this request is connected when available. remoteAddress=/192.168.219.64:53587 NA
localAddress The local address that is associated with the request. It also means the local address to where this request is connected when available. localAddress=cncc-mcore-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075 NA
resourcePath Request URI resourcePath=/soothsayer/v1/canaryrelease/ NA
queryParams Query parameters associated with request. queryParams={form_id=9, page=1, view_id=78} NA
headers Headers associated with request or response. headers={Accept=*/*, X-Requested-With=XMLHttpRequest, User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0, Connection=keep-alive, Host=cncc-core-ingress-gateway.cncc.svc.cluster.local:30075, Accept-Language=en-US,en;q=0.5, Accept-Encoding=gzip, deflate, DNT=1, Content-Type=application/json; charset=utf-8} NA
payload Payload or Data associated with request or response. payload=[{"serviceName":"n5g-eir-eic","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5} NA
authenticationType This indicates whether user is requesting resource logged in using CNC Console or directly accessing through postman or curl. authenticationType=OAUTH OAUTH -> User is logged in through CNC Console application and accessing resource.

JWT -> User is accessing resource directly through postman or curl.

2.3 Log Levels

Logs register system events along with their date and time of occurrence. They also provide important details about a chain of events that could have led to an error or problem.

A log level helps in defining the severity level of a log message. For CNC Console, the log level of a microservice can be set to any one of the following valid values:

  • TRACE: A log level that describes events, as a step by step execution of code. This can be ignored during the standard operation, but may be useful during extended debugging sessions.
  • DEBUG: A log level used for events during software debugging when more granular information is needed.
  • INFO: A standard log level indicating that something has happened, an application has entered a certain state, etc.
  • WARN: A log level indicates that something unexpected has happened in the application, a problem, or a situation that might disturb one of the processes. But this does not mean that the application has failed. The WARN level should be used in situations that are unexpected, but the code can continue to work.
  • ERROR: A log level that should be used when an application hits an issue preventing one or more functionalities from functioning.

Using this information, the logs can be filtered based on the system requirements. For instance, if you want to filter the critical information about your system from the informational log messages, set a filter to view messages with only WARN log level in Kibana.

2.4 Types of Logs

The CNC Console logs can be categorized into following types:

Regular logs

These logs contain error messages, warnings, or other events written within the application that provide logical, high level information about the application and ongoing events.

Example:

{"level": "INFO","message": "Started GatewayApplication in 10.748 seconds (JVM running for 12.825)"}
{"level": "INFO","message": "Creating plain httpClient"}
{"level": "INFO","message": "Creating plain restTemplate"}
{"level": "ERROR","message": "Can't get cfgs of topic public.dynamic.datamodel,  exception is:\n
javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused (Connection
        refused)"}

Audit Logs

Note:

CNC Console IAM Audit logs are not applicable for OCI Deployment.

These logs contain user related information and the activity within the system.

The following events are logged in CNC Console Core:

  • Log in: A user has logged in.
  • Access Resource: A user is accessing a particular NF resource.
  • Access Resource Error: A user is denied from accessing a particular NF resource.
  • Logout: A user has logged out.

    Note:

    The user can find the CNC Console Core User Activity logs as part of cncc-core-ingress-gateway and are represented in CNC Console message format.

The following events are logged in CNC Console IAM:
  • Login events
  • Log in: An admin user has logged in.
  • Register: An admin user has registered.
  • Logout: An admin user has logged out.
  • Code to Token: An application or a client has exchanged a code for a token.
  • Refresh Token: An application or a client has refreshed a token.

    Account events

  • Update Email: The email address for an account has changed.
  • Update Profile: The profile for an account has changed.
  • Send Password Reset: A password reset email has been sent.
  • Update Password: The password for an account has changed.

    Note:

    The user can find the CNC Console IAM User Activity logs as part of cncc-iam-0, represented in Keycloak format. These events are provided by keycloak and documented under Keycloak Auditing End Events.
    Logging Error Logs are recorded by keycloak container as :
    ^[[0m^[[33m10:12:57,388 WARN  [org.keycloak.events] (default task-3) type=LOGIN_ERROR, realmId=master, clientId=security-admin-console,
      userId=ef58d62e-a0a8-4f4e-bcc6-abccf917641c, ipAddress=192.168.203.108, error=invalid_user_credentials, auth_method=openid-connect, auth_type=code,
      redirect_uri=http://10.75.240.33:30085/cncc/auth/admin/master/console/,
      code_id=3e6d822a-9e82-4660-bb01-a814f7ae8f97, username=admin,
      authSessionParentId=3e6d822a-9e82-4660-bb01-a814f7ae8f97,
    authSessionTabId=2ak6Xwal-28

Security Logs

Note:

M-CNCC IAM Security logs are not applicable for OCI Deployment.

The security logs contain the header, payload, method, scheme, URI information for all the requests and corresponding responses.

Disabling Security Logs

By default Security Log will be enabled for M-CCNC IAM, M-CNCC Core, and A-CNCC Core. You can disable this by setting securityLogEnabled flag to false in custom values.yaml file. 

# CNC Console configuration
cncc:
  # Enable security logs for CNC Console
  securityLogEnabled: false

Header Information

At all the log levels, sensitive information like Cookies are masked.

Note:

The user can find the Security logs:
  • For M-CNCC Core and A-CNCC Core, these are logged as part of cncc-mcore-ingress-gateway or cncc-acore-ingress-gateway and are represented in CNC Console message format.
  • For M-CNCC IAM, these are logged as part of cncc-iam-ingress-gateway and are represented in CNCC message format.

Log Levels

Note:

M-CNCC IAM ingress-gateway default log levels are not applicable for OCI deployment.

Default log levels set for M-CNCC Core and A-CNCC Core:


ingress-gateway:
  log:
    level:
      cncc:
        root: WARN
        audit: INFO
        security: INFO

Default log levels set for M-CNCC IAM:


ingress-gateway:
  log:
    level:
      cncc:
        root: WARN
        security: INFO
Updating M-CNCC IAM Kubernetes Cluster (KC) log level

Note:

M-CNCC IAM KC logs levels are not applicable for OCI Deployment.
  • By default the log level of M-CNCC IAM KC is set to WARN, org.keycloak.events:DEBUG
    • This means the root log-level is set to WARN and the org.kecyalok.events package is set to DEBUG
  • In the level label, set the log level. Following are the different options available to set the log level:
    • Log level
      • TRACE
      • DEBUG
      • INFO
      • WARN
      • ERROR
      • FATAL
    • level: OFF

      No logs will appear

  • Sample M-CNCC IAM KC log configuration: 
    kc:    
  preferIpv6Stack:      
    enabled: false
  log:       
    level: WARN,org.keycloak.events:DEBUG

Supported Headers for Logging

Header Header values (regex)
Content-Type ^application/x-www-form-urlencoded.*
^application/json.*
^application/problem+json.*
Accept ^application/json.*
^application/ld+json.*
^application/xml.*
^multipart/form-data.*
Role of supporting headers in CNC Console Audit and Security logs
  • At INFO level, only those request and response that match the supporting headers and values are logged.
  • At DEBUG level, no supporting headers used and all request and response are logged.
  • At ERROR / WARN, no supporting headers used and only error or warnings are logged.

    Note:

    Any failure in authorizing a request will always be logged irrespective of the supported header configuration.

2.5 Examples of Logs

This section lists the examples of audit and security logs.

Examples of Audit Logs

Examples of Security Logs

Examples of Audit Logs

CNC Console Core

Only message part of the JSON log is shown in the example.

  • User successfully logging into CNC Console Core 
    
logType=AUDIT, type=LOGIN, resourceType=CNCC,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=user,
operationType=GET, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/login/oauth2/code/cncc-iam,
authenticationType=OAUTH
  • User accessing SCP resource having SCP_READ role 
    
logType=AUDIT, type=ACCESS_RESOURCE, resourceType=SCP,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=user,
operationType=GET, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/soothsayer/v1/canaryrelease/,
 authenticationType=OAUTH
  • User updating(PATCH) SCP resource having SCP_WRITE role 
    
logType=AUDIT, type=ACCESS_RESOURCE, resourceType=SCP,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=user,
operationType=PATCH, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/soothsayer/v1/canaryrelease/n5g-eir-eic,
authenticationType=OAUTH
  • User accessing NRF resource without having NRF_READ role 
    
logType=AUDIT, type=ACCESS_RESOURCE_ERROR, resourceType=NRF,
 userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=user,
status=403 FORBIDDEN, operationType=GET,
remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/nrf-configuration/v1/system-options,
authenticationType=OAUTH
  • User successful logout 
    
logType=AUDIT, type=LOGOUT, resourceType=CNCC,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=user,
operationType=POST, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/logout, authenticationType=OAUTH

CNC Console IAM:

Note:

  • The defined CNC Console IAM Audit log examples are not applicable for OCI Deployment.
  • The CNC Console IAM Security log examples are not applicable for OCI Deployment.
  • To see M CNCC-IAM KC Audit event logs, set the org.keycloak.events package log to DEBUG in the custom_values.yaml. Refer Updating M-CNCC IAM KC loglevel for log level configuration.
  • Login Error when password entered was wrong 
    04:56:35,890 WARN  [org.keycloak.events] (default task-22)
                              type=LOGIN_ERROR, realmId=master, clientId=security-admin-console,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              error=invalid_user_credentials, auth_method=openid-connect,
                              auth_type=code, redirect_uri=http://10.75.225.28:31373/cncc/auth/admin/master/console/, code_id=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              username=admin,
                              authSessionParentId=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              authSessionTabId=8sruELA1WWs
  • Login with correct credential 
    04:57:24,581 DEBUG  [org.keycloak.events] (default task-22)
                              type=LOGIN, realmId=master, clientId=security-admin-console,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              auth_method=openid-connect, auth_type=code, redirect_uri=http://10.75.225.28:31373/cncc/auth/admin/master/console/, consent=no_consent_required,
                              code_id=5aca4960-eecf-406b-a7eb-92e249c2beeb, username=admin,
                              authSessionParentId=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              authSessionTabId=8sruELA1WWs
  • User created 
    04:58:41,804 DEBUG  [org.keycloak.events] (default task-22)
                              operationType=CREATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=USER,
                              resourcePath=users/070911f5-c397-42c1-b5a4-cd92fa435a33
  • Deleted user 
    05:00:08,226 DEBUG  [org.keycloak.events] (default task-22)
                              operationType=DELETE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=USER,
                              resourcePath=users/2b931bbb-7f97-4f04-9f75-e0d0974ab73d
  • Admin Role removed for a user 
    05:01:07,781 DEBUG  [org.keycloak.events] (default task-22)
                              operationType=DELETE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM_ROLE_MAPPING,
                              resourcePath=users/08fc0058-133b-4288-9165-14c96c5dcd7a/role-mappings/realm
  • Admin Role added for a user 
    05:01:33,664 DEBUG  [org.keycloak.events] (default task-27)
                              operationType=CREATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM_ROLE_MAPPING,
                              resourcePath=users/08fc0058-133b-4288-9165-14c96c5dcd7a/role-mappings/realm
  • Realm setting update 
    05:02:29,222 DEBUG  [org.keycloak.events] (default task-26)
                              operationType=UPDATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM, resourcePath=null

  • Logout all session on keycloak

    05:05:02,383 DEBUG  [org.keycloak.events] (default task-29)
                              operationType=ACTION, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM, resourcePath=logout-all

Examples of Security Logs

Representation for IAM and Core are same as these logs are part of ingress-gateway. Only message part of the JSON log is shown in the example.

CNC Console Core

  • SCP request 
    logType=SECURITY, type=REQUEST, resourceType=SCP,
                                    userId=3314f54f-08bf-489d-b395-27bf56da1262, username=user,
                                    operationType=GET, scheme=http,
                                    remoteAddress=/192.168.219.64:53587,
                                    localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
                                    resourcePath=/soothsayer/v1/canaryrelease/, queryParams={},
                                    headers={Accept=*/*, X-Requested-With=XMLHttpRequest,
                                    User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0)
                                    Gecko/20100101 Firefox/68.0, Connection=keep-alive,
                                    Host=cncc-core-ingress-gateway.cncc.svc.cluster.local:30075,
                                    Accept-Language=en-US,en;q=0.5, Accept-Encoding=gzip, deflate,
                                    DNT=1, Content-Type=application/json; charset=utf-8},
                                    payload={}, authenticationType=OAUTH
  • SCP response 
    logType=SECURITY, type=RESPONSE, resourceType=SCP,
                                    userId=3314f54f-08bf-489d-b395-27bf56da1262, username=user,
                                    status=200 OK, operationType=GET, scheme=http,
                                    resourcePath=/soothsayer/v1/canaryrelease/,
                                    headers={transfer-encoding=chunked, Connection=keep-alive,
                                    Transfer-Encoding=chunked, Content-Type=application/json,
                                    Date=Sat, 04 Jul 2020 11:58:20 GMT},
                                    payload=[{"serviceName":"n5g-eir-eic","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-comm","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-evts","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-loc","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-mt","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"nausf-auth","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"nausf-sorprotection","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5}}],
                                    authenticationType=OAUTH

CNC Console IAM

The logs for IAM Ingress Gateway contain the username and user ID for all actions performed by CNCC-IAM admin users post-login, alongside other relevant information.

  • Request
    logType=SECURITY, type=REQUEST, userId=2811c05e-bc84-4597-8014-3a88ef7535c2, username=admin, operationType=GET, scheme=http, remoteAddress=/10.233.118.0:52597, localAddress=/10.233.80.179:8081, resourcePath=/cncc/auth/admin/realms/master/users/0931c7a1-34a6-45f1-9db5-5cd8a043fefb, queryParams={userProfileMetadata=true}, headers={Connection=keep-alive, User-Agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36, uidToken=07338817-dOac-4b18-948d-e3d5c0c91879, Host=10.121.42.53, Accept-Encoding=gzip, deflate, svcName=cncc-iam-kc-http.amanp-ns.svc.thrust2a, ocLogId=1733063953903_110_cncc-iam-ingress-gateway-5f7d9b9d87-dt6vj,
accept=application/json, text/plain, */*, sbi-timer-publish-headers=false, content-type-application/json, Accept-Language=en-GB,
en; q=0.9,
sbi-timer-feature=false}, payload={}, authenticationType=UNKNOWN", "endOfBatch": false, "loggerFqcn": "org-apache. logging.
10g4j.internal.DefaultLogBuilder", "threadId":112, "threadPriority":5, "messageTimestamp":"2024-12-01T14:39:13.910+0000"' ,
"processId": "1", "ingressTxId":"ingress-tx-1093411995", "pod": "cncc-iam-ingress-gateway-5f7d9b9d87-dt6vj" ,
"ocLogId":"1733063953903 110 cncc-iam-ingress-qateway-5f7d9b9d87-dt6vi" "instanceType":"prod" "RequestId": ''}
  • Response 
    "message":" logType=SECURITY, type=RESPONSE, status=200 0K, userId=2811c05e-bc84-4597-8014-3a88ef7535c2, username=admin, operationType=GET, schemeshttp, resourcePath=/cncc/auth/admin/ realms/ master/clients/a6f486d4-0d0f-468d-a609-f655e9f1c492, headers={transfer-encoding=chunked, content-length=772, Cache-Control=no-cache, Content-Type=application/json;charset=UTF-8, Referrer-Policy=no-referrer, Strict-Transport-Security=max-age=31536000; includeSubDomains, X-Content-Type-Options=nosniff, X-Frame-Options=SAMEORIGIN, X-XSS-Protection=1; mode=block, NettyLatency=1733064487125, RequestMethod=GET}, payload= {\"id\":\"a6f486d4-0d0f-468d-a609-f655e9f1c492\", \"clientId\" :\"cncc-realm\", \"name\": \"cncc Realm\" ,
\"surrogateAuthRequired)": false, \"enabled)": true, \"alwaysDisplayInConsole\": false, \"clientAuthenticatorType\" :\"client-secret\", \"redirectUris\": [1, \"webOrigins\": [1, \"notBefore\":0, \"beareronly)": true, \"consentRequired\": false, \"standardF lowEnabled\": true, \"implicitFlowEnabled\": false, \"directAccessGrantsEnabled\": false, \"serviceAccountsEnabled\": false, \"publicClient\": false, \"frontchannelLogout)": false, \"attributes)": {}, \"authenticationFlowBindingOverrides\": {},\"fullScopeAllowed\": true, \"nodeReRegistrationTimeout\":0, \"defaultClientScopes\": [\"web-origins\", \"roles\", \"profile\", \"email\"], \"optionalClientScopes\": [\"address\", \"phone\", \"offline_access\", \"microprofile-jwt\"], \"access\":{\"view|": true, \"configure)": true, \"manage)": true}}, authenticationType=UNKNOWN", "endofBatch": false, "loggerFqcn": "org-apache. logging. log4j. internal.DefaultLogBuilder", "threadId":122, "threadPriority":5, "messageTimestamp": "2024-12-01T14:48:07.152+0000", "processId": "1" ,
"ingressTxId" :"ingress-tx-577080018", "pod":"cncc-iam-ingress-gateway-5f7d9b9d87-dt6vj", "ocLogId" :"'","instanceType": "prod" ,
"RequestId" : ""}

2.6 Configuring Security Logs

This section provides the details about configuring security logs.

Setting at Log Level

By default, Security Log is set to the "INFO" level for both CNC Console Core and CNC Console IAM. You can change the log level by setting log.level.cncc.security to the required level in core and iam values.yaml file.

values.yaml
#Set the root log level
log:
  level:
    root: WARN
    ingress: INFO
    oauth: INFO
    cncc:
      security: INFO
Disabling Security Log

By default, the Security Log is enabled for both CNCC Core and CNCC IAM. You can disable this by setting securityLogEnabled flag to false in core and iam values.yaml file.

values.yaml
# CNCC configuration
cncc:
  enabled: false
  enablehttp1: false
  securityLogEnabled: false

2.6.1 Accessing logs

This section gives information about how to access the logs.

2.6.1.1 Accessing Logs in Non OCI Deployment

The CNC Console application logs can be accessed in following ways:

  1. Run the following command to view logs of a CNC Console application pod: 
    kubectl logs -f -n <cncc_namespace> <pod_name> -c <container_name>

    Example:

    CNC Console Core:
    $ kubectl logs -f -n cncc cncc-mcore-ingress-gateway-77df795fb5-wv2sb -c ingress-gateway (Security & Audit Log)
    CNC Console IAM: 
    $ kubectl logs -f -n cncc cncc-iam-ingress-gateway-77df795fb5-wv2sb -c ingress-gateway (Security Log)

    $ kubectl logs -f -n cncc cncc-iam-0 (Audit Log)
  2. CNC Console uses cloud native supported logging framework to view the logs.

    Example : Elasticsearch, Fluentd, and Kibana (EFK) can be used here with CNC Console to view the logs as follows:

    Figure 2-1 Log View

    img/kibana.png
2.6.1.2 Accessing logs in OCI Deployment
  1. Viewing logs for OCI IAM in OCI Console GUI:

    See the Generate Identity and Access Management Reports from Oracle Cloud Infrastructure Audit document to view OCI IAM logs.

  2. Viewing logs for M-CNCC Core and A-CNCC Core in OCI Console GUI:
    1. Log in to OCI Console GUI and search for Log Explorer or Click the Hamburger menu, click Observability and Management, then click Log Explorer in Logging Analytics section. This will open the Log Explorer page.

      Figure 2-2 Log Explorer


      Log Explorer

    2. On the Log Explorer page, click Kubernetes Container Generic logs.

      Figure 2-3 Kubernetes Container Generic logs


      Kubernetes Container Generic logs

    3. After clicking the Kubernetes Container Generic Logs link, logs will be visible.

      Figure 2-4 View Logs


      View Logs

    4. You can filter the logs by the container by clicking Add to search. Similarly other log source fields can also be added to filter logs.

      Figure 2-5 Filter the logs by the container


      Filter the logs by the container

    5. Filter logs based on the logType regex pattern for security and audit logs. Similarly required regex patterns can be used to filter the logs.

      Figure 2-6 Regex Patterns for Filtering Logs


      Regex Patterns for Filtering Logs

Note:

For more details, see the Explore Logs section in the Oracle Cloud Infrastructure Logging Analytics Quick Start Guide.

2.6.2 Debugging using Logs

This section provides information to debug CNC Console using Logs.

Table 2-3 CNCC Core Debugging through Logs

Scenario Level Logs to be searched
CNC Core Login INFO Login successful
Session Timeout Value INFO Session timeout
Validating user authorization INFO User Authorization Details
Accessing a resource DEBUG Mapping [Exchange: GET
Updating a resource DEBUG Mapping [Exchange: PATCH 'or' Mapping [Exchange: PUT
Creating a new resource DEBUG Mapping [Exchange: POST 'or' Mapping [Exchange: PUT
CNCC Core Logout INFO Logout successful