1. Cloud Native Core Console User's Guide
  2. CNCC Logs

6 CNCC Logs

This section describes about the cncc logs. It contains the following topics:

Log Formats and Details

Types of Logs

Configuring the Logs

Examples of Logs

Accessing the Logs

Log Formats

This section describes about the log formats.

Log4j JSON Format

CNCC Message Format

Log4j JSON Format

Following are the log format and filelds. All logs are represented in JSON format.

{
     "thread": <threadId>,
     "level": <log_level>,
     "loggerName": <name_of_the_logging_class>,
     "message": <message>,
     "instant": <timestamp_in_miliseconds>,
     "messageTimestamp": <timestamp_in_readable_format>
     "threadId": <threadId>,
     "threadPriority": <threadPriority>,
     "pod": <name_of_the_pod>,
     "processId": <processId>,
     "contextMap": <context_map>
}

Table 6-1 Log Details

Name Description Example
thread Name of the thread "thread": "reactor-http-epoll-1"
level Level of the log. It can be: Log level (INFO, WARN, DEBUG, TRACE) "level": "INFO"
loggerName Name of the class that generated the log "loggerName": "ocpm.cne.gateway.cncc.GatewayApplication"
messageTimestamp Time represented in human readable format and in UTC. "messageTimestamp": 2020-07-04 12:00:40.702
message Information about the event "message": "Started Application....."

By default, all messages are in simple string except Audit Log, Security Log which are represented in CNCC Message Format.

instant The Date and Time the event occurred in epoch second and nano seconds "instant": { "epochSecond": 1590045388, "nanoOfSecond": 339789000}
processId Linux process Identifier (for a multi-process host) Linux process Identifier (for a multi-process host).
threadId Id of the thread "threadId":"43"
threadPriority Priority assigned to the thread "threadPriority": 5
pod Name of the pods where the log is generated "cncc-core-ingress-gateway-77df795fb5-wv2sb"
contextMap It hold information that are added to threadContext. "contextMap": { "hostname": "cncc-core-ingress-gateway-77df795fb5-wv2sb", "ingressTxId": "ingress-tx-1460885598"}

CNCC Message Format

Table 6-2 CNCC Message Format

Name Description Example Possible Values
logType Indicates whether it is Security Log or Audit Log logType=AUDIT AUDIT

SECURITY
type Indicates nature/action of the log type=REQUEST For Security Log,REQUEST

RESPONSE

For Audit Log,LOGIN

ACCESS_RESOURCE

ACCESS_RESOURCE_ERROR

LOGOUT
resourceType Indicates what is the resource being requested for resourceType=SCP CM_SERVICE (For default route)

CNCC (For User Login Activity)

SCP

UDR

NRF

PCF

... all CNCC supported NF's
userId Id of the user. Basically to know who triggered request/action userId=3314f54f-08bf-489d-b395-27bf56da1262  
username Name of the user username=shreb  
status Http status of the response. status=200 OK  
operationType HTTP method of the request operationType=GET  
scheme Indicates the scheme of the request scheme=http  
remoteAddress Remote Address associated with request. i.e remote address where this request is connected to, if available. remoteAddress=/192.168.219.64:53587  
localAddress Local Address associated with request. i.e local address the request was accepted on, if available. localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075  
resourcePath Request uri resourcePath=/soothsayer/v1/canaryrelease/  
queryParams Query parameters associated with request queryParams={form_id=9, page=1, view_id=78}  
headers Headers associated with request or response headers={Accept=*/*, X-Requested-With=XMLHttpRequest, User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0, Connection=keep-alive, Host=cncc-core-ingress-gateway.cncc.svc.cluster.local:30075, Accept-Language=en-US,en;q=0.5, Accept-Encoding=gzip, deflate, DNT=1, Content-Type=application/json; charset=utf-8}  
payload Payload/Data associated with request or response payload=[{"serviceName":"n5g-eir-eic","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5}  
authenticationType This indicates whether user is requesting resource logged in using CNCC or directly accessing through postman/curl. authenticationType=OAUTH OAUTH -> User is logged in through CNCC application and accessing resource

JWT -> User is accessing resource directly through postman/curl

Types of Logs

The CNCC logs can be categorized into following types:

Regular logs

These logs contains all kinds of error messages, warnings or other events written within the application which provide logical, highlevel information about the application and ongoing events.

Example:

{"level": "INFO","message": "Started GatewayApplication in 10.748 seconds (JVM running for 12.825)"}
{"level": "INFO","message": "Creating plain httpClient"}
{"level": "INFO","message": "Creating plain restTemplate"}
{"level": "ERROR","message": "Can't get cfgs of topic public.dynamic.datamodel,  exception is:\n
javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused (Connection
        refused)"}

Audit Logs

These logs contains user related information and his activity within the system.

Following events are logged in CNCC Core:

  • Login - A user has logged in.
  • Access Resource- A user is accessing particular NF resource.
  • Access Resource Error - A user is denied from accessing particular NF resource.
  • Logout - A user has logged out.

    Note:

    The user can find the CNCC Core User Activity logs as part of cncc-core-ingress-gateway and are represented in CNCC message Format

    Following events are logged in CNCC IAM:

    Login events

  • Login - A admin user has logged in.
  • Register - A admin user has registered.
  • Logout - A admin user has logged out.
  • Code to Token - An application/client has exchanged a code for a token.
  • Refresh Token - An application/client has refreshed a token.

    Account events

  • Update Email - The email address for an account has changed.
  • Update Profile - The profile for an account has changed.
  • Send Password Reset - A password reset email has been sent.
  • Update Password - The password for an account has changed.

    Note:

    The user can find the CNCC IAM User Activity logs as part of cncc-iam-0 and are represented in Keycloak format. These events are provided by keycloak and are documented under Keycloak Auditing End Events.

Security Logs

The security logs holds information of all the request and its corresponding response. Information such as header, payload, method, scheme, uri etc.

At INFO level it logs,

  • only those request, with header Content-Type/Accept is set to application-json/www-form-urlencoded
  • corresponding response, with header Content-Type is set to application-json/prolem+json/www-form-urlencoded

At DEBUG level it logs,

  • all request.
  • all response.

Request/Response Payload

At all the log levels:

  • payload is logged only for request/response with header Content-Type/Accept is set to application-json/prolem+json/www-form-urlencoded
  • all html, css, javascript, icon, woff payload are masked.

Header Information

AT all the log levels, sensitive information like Cookies are masked.

Note:

The user can find the Security logs :
  • For CNCC Core, these logs are logged as part of cncc-core-ingress-gateway and are represented in CNCC message Format.
  • For CNCC IAM, these logs are logged as part of cncc-iam-ingress-gateway and are represented in CNCC message Format.

Configuring Security Logs

This section details about configuring security logs.

Setting at Log Level

By default Security Log will be set to "INFO" level for both CCNC Core and CNCC IAM. But user can change it log level by setting log.level.cncc.security to required level in core and iam values.yaml

values.yaml
#Set the root log level
log:
  level:
    root: WARN
    ingress: INFO
    oauth: INFO
    cncc:
      security: INFO
Disabling Security Log

By default Security Log will be enabled for both CCNC Core and CNCC IAM. But user can disable this by setting securityLogEnabled flag to false in core/iam values.yaml

values.yaml
# CNCC configuration
cncc:
  enabled: false
  enablehttp1: false
  securityLogEnabled: false

Examples of Logs

This section lists the examples of audit and security logs.

Examples of Audit Logs

Examples of Security Logs

Examples of Audit Logs

CNCC Core

Only message part of the JSON log is shown in the example.

  • User successfully logging into CNCC Core 
    
logType=AUDIT, type=LOGIN, resourceType=CNCC,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=shreb,
operationType=GET, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/login/oauth2/code/cncc-iam,
authenticationType=OAUTH
  • User accessing SCP resource having SCP_READ role 
    
logType=AUDIT, type=ACCESS_RESOURCE, resourceType=SCP,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=shreb,
operationType=GET, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/soothsayer/v1/canaryrelease/,
 authenticationType=OAUTH
  • User updating(PATCH) SCP resource having SCP_WRITE role 
    
logType=AUDIT, type=ACCESS_RESOURCE, resourceType=SCP,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=shreb,
operationType=PATCH, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/soothsayer/v1/canaryrelease/n5g-eir-eic,
authenticationType=OAUTH
  • User accessing NRF resource without having NRF_READ role 
    
logType=AUDIT, type=ACCESS_RESOURCE_ERROR, resourceType=NRF,
 userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=shreb,
status=403 FORBIDDEN, operationType=GET,
remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/nrf-configuration/v1/system-options,
authenticationType=OAUTH
  • User successful logout 
    
logType=AUDIT, type=LOGOUT, resourceType=CNCC,
userId=186f6f2a-ba6a-4812-8a18-b906a5f9e3f6, username=shreb,
operationType=POST, remoteAddress=/192.168.219.64:53587,
localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
resourcePath=/logout, authenticationType=OAUTH

CNCC IAM:

  • Login Error when password entered was wrong 
    04:56:35,890 WARN  [org.keycloak.events] (default task-22)
                              type=LOGIN_ERROR, realmId=master, clientId=security-admin-console,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              error=invalid_user_credentials, auth_method=openid-connect,
                              auth_type=code, redirect_uri=http://10.75.225.28:31373/cncc/auth/admin/master/console/, code_id=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              username=admin,
                              authSessionParentId=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              authSessionTabId=8sruELA1WWs
  • Login with correct credential 
    04:57:24,581 INFO  [org.keycloak.events] (default task-22)
                              type=LOGIN, realmId=master, clientId=security-admin-console,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              auth_method=openid-connect, auth_type=code, redirect_uri=http://10.75.225.28:31373/cncc/auth/admin/master/console/, consent=no_consent_required,
                              code_id=5aca4960-eecf-406b-a7eb-92e249c2beeb, username=admin,
                              authSessionParentId=5aca4960-eecf-406b-a7eb-92e249c2beeb,
                              authSessionTabId=8sruELA1WWs
  • User created 
    04:58:41,804 INFO  [org.keycloak.events] (default task-22)
                              operationType=CREATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=USER,
                              resourcePath=users/070911f5-c397-42c1-b5a4-cd92fa435a33
  • Deleted user 
    05:00:08,226 INFO  [org.keycloak.events] (default task-22)
                              operationType=DELETE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=USER,
                              resourcePath=users/2b931bbb-7f97-4f04-9f75-e0d0974ab73d
  • Admin Role removed for a user 
    05:01:07,781 INFO  [org.keycloak.events] (default task-22)
                              operationType=DELETE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM_ROLE_MAPPING,
                              resourcePath=users/08fc0058-133b-4288-9165-14c96c5dcd7a/role-mappings/realm
  • Admin Role added for a user 
    05:01:33,664 INFO  [org.keycloak.events] (default task-27)
                              operationType=CREATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM_ROLE_MAPPING,
                              resourcePath=users/08fc0058-133b-4288-9165-14c96c5dcd7a/role-mappings/realm
  • Realm setting update 
    05:02:29,222 INFO  [org.keycloak.events] (default task-26)
                              operationType=UPDATE, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM, resourcePath=null

  • Logout all session on keycloak

    05:05:02,383 INFO  [org.keycloak.events] (default task-29)
                              operationType=ACTION, realmId=master,
                              clientId=819ce4a5-ddbd-4717-908f-a204bdabc808,
                              userId=d7cde46f-15e1-4ff8-a2cb-c5825e481438, ipAddress=192.168.219.64,
                              resourceType=REALM, resourcePath=logout-all

Examples of Security Logs

Representation for IAM and Core are same as these logs are part of ingress-gateway. Only message part of the JSON log is shown in the example.

CNCC Core

  • SCP request 
    logType=SECURITY, type=REQUEST, resourceType=SCP,
                                    userId=3314f54f-08bf-489d-b395-27bf56da1262, username=shreb,
                                    operationType=GET, scheme=http,
                                    remoteAddress=/192.168.219.64:53587,
                                    localAddress=cncc-core-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30075,
                                    resourcePath=/soothsayer/v1/canaryrelease/, queryParams={},
                                    headers={Accept=*/*, X-Requested-With=XMLHttpRequest,
                                    User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0)
                                    Gecko/20100101 Firefox/68.0, Connection=keep-alive,
                                    Host=cncc-core-ingress-gateway.cncc.svc.cluster.local:30075,
                                    Accept-Language=en-US,en;q=0.5, Accept-Encoding=gzip, deflate,
                                    DNT=1, Content-Type=application/json; charset=utf-8},
                                    payload={}, authenticationType=OAUTH
  • SCP response 
    logType=SECURITY, type=RESPONSE, resourceType=SCP,
                                    userId=3314f54f-08bf-489d-b395-27bf56da1262, username=shreb,
                                    status=200 OK, operationType=GET, scheme=http,
                                    resourcePath=/soothsayer/v1/canaryrelease/,
                                    headers={transfer-encoding=chunked, Connection=keep-alive,
                                    Transfer-Encoding=chunked, Content-Type=application/json,
                                    Date=Sat, 04 Jul 2020 11:58:20 GMT},
                                    payload=[{"serviceName":"n5g-eir-eic","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-comm","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-evts","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-loc","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"namf-mt","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"nausf-auth","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5},{"serviceName":"nausf-sorprotection","canaryReleaseFlag":true,"apiFullVersion":"2.0.0","canaryTraffic":5}}],
                                    authenticationType=OAUTH

CNCC IAM

  • Request 
    logType=SECURITY, type=REQUEST, operationType=GET, scheme=http,
                                    remoteAddress=/192.168.219.64:53587,
                                    localAddress=cncc-iam-ingress-gateway.cncc.svc.cluster.local/<unresolved>:30085,
                                    resourcePath=/cncc/auth/admin/master/console/config,
                                    queryParams={}, headers={Accept=application/json,
                                    User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0)
                                    Gecko/20100101 Firefox/68.0, Referer=http://cncc-iam-ingress-gateway.cncc.svc.cluster.local:30085/cncc/auth/admin/master/console/, Connection=keep-alive,
                                    Host=cncc-iam-ingress-gateway.cncc.svc.cluster.local:30085,
                                    Accept-Language=en-US,en;q=0.5, Accept-Encoding=gzip, deflate,
                                    DNT=1}, payload={},
                              authenticationType=NONE
  • Response 
    logType=SECURITY, type=RESPONSE, status=200 OK,
                                    operationType=GET, scheme=http,
                                    resourcePath=/cncc/auth/admin/master/console/config,
                                    headers={transfer-encoding=chunked, Cache-Control=no-cache,
                                    X-XSS-Protection=1; mode=block, X-Frame-Options=SAMEORIGIN,
                                    Date=Mon, 06 Jul 2020 10:54:16 GMT, Connection=keep-alive,
                                    Strict-Transport-Security=max-age=31536000; includeSubDomains,
                                    X-Content-Type-Options=nosniff, Content-Type=application/json,
                                    Content-Length=211},
                                      payload={\"realm\":\"master\",\"auth-server-url\":\"http://cncc-iam-ingress-gateway.cncc.svc.cluster.local:30085/cncc/auth/\",\"ssl-required\":\"none\",\"resource\":\"security-admin-console\",\"public-client\":true,\"confidential-port\":0},
                                    authenticationType=NONE

Accessing logs

This section gives information about how to access the logs.

The CNCC application logs can be accessed in following ways:

  1. Viewing logs of a cncc application pod running. This can be achieved by executing the command: 
    kubectl logs -f -n <cncc_namespace> <pod_name> -c <container_name>

    Example:

    CNCC Core:
    $ kubectl logs -f -n cncc cncc-core-ingress-gateway-77df795fb5-wv2sb
                              -c ingress-gateway (Security & Audit Log)
    CNCC IAM: 
    $ kubectl logs -f -n cncc cncc-iam-ingress-gateway-77df795fb5-wv2sb
                              -c ingress-gateway (Security Log)
  2. CNCC uses cloud native supported logging framework to view the logs.

    Example : EFK can be used here with CNCC to view the logs as given below.

    img/kibana.png