4.3 Service Related Issues

This section describes the most common service related issues and their resolution steps. It is recommended to perform the resolution steps provided in this guide. If the issue still persists, then contact Oracle Support.

4.3.1 SM Service Issues

This section describes the most common SM service issues and their resolution steps. It is recommended to for users to attempt the resolution steps provided in this guide before contacting Oracle Support.

Failed BSF register or deregister binding

Symptom

On sending BSF register or deregister binding request, the SM service receives 406 NOT_ACCEPTABLE binding reply message from BSF.

Problem

When the SM service initiates a request to register with or deregister from BSF a session, BSF sends 4xx in the response code. It is assumed that bindingSvcenabled parameter is set to true while deploying the Policy instance.

Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1621844941,"nanoOfSecond":854958774},"thread":"boundedElastic-1","level":"INFO","loggerName":"ocpm.pcf.service.sm.serviceconnector.BsfConnector","message":"Sent Binding Request to BSF Service: https://bsf.apigateway:8001/nbsf-management/v2/pcfBindings, {\"supi\":\"imsi-10000000002\",\"contextId\":\"afa7e0cb-87f3-4e6c-a867-166705acfcfe\",\"gpsi\":\"msisdn-10000000001\",\"ipv4Addr\":\"192.168.10.10\",\"ipv6Prefix\":\"2800:a00:cc01::/64\",\"ipDomain\":\"ora.com\",\"dnn\":\"dnn1\",\"pcfFqdn\":\"pcf-smservice.pcf\",\"pcfDiamHost\":\"pcf-smservice\",\"pcfDiamRealm\":\"pcf-smservice.svc\",\"snssai\":{\"sst\":11,\"sd\":\"abc123\"}}","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":37,"threadPriority":5,"messageTimestamp":"2021-05-24T08:29:01.854+0000"}
{"instant":{"epochSecond":1621844941,"nanoOfSecond":945868600},"thread":"boundedElastic-1","level":"INFO","loggerName":"ocpm.pcf.service.sm.serviceconnector.BsfConnector","message":"Receive Binding Reply from BSF: 406 NOT_ACCEPTABLE","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":37,"threadPriority":5,"messageTimestamp":"2021-05-24T08:29:01.945+0000"}
{"instant":{"epochSecond":1621844941,"nanoOfSecond":946569461},"thread":"boundedElastic-1","level":"DEBUG","loggerName":"ocpm.pcf.service.sm.domain.component.metrics.SmMetrics","message":"Pegging binding response metric. Dnn :dnn1, snssai : 11-abc123, operationType : create ,mode : synchronous ,responseCode : 4xx","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":37,"threadPriority":5,"messageTimestamp":"2021-05-24T08:29:01.946+0000"}
Resolution Steps

Policy not evaluated, and instead default policy got applied

Symptom

On sending POST request to binding service, SM service receives failed to call Binding service error.

Problem
When the SM service initiates a POST request towards BSF such as http://my-cnpolicy-occnp-binding:8000/binding/v1/contextBinding/context-owner/PCF-SM, an error occurs and a message is received at SM service stating that the system failed to call Binding service. User may search for a response similar to the following:
logMsg=Failed to call policy service: {}
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1623052588,
    "nanoOfSecond": 652897378
  },
  "thread": "boundedElastic-7",
  "level": "ERROR",
  "loggerName": "ocpm.pcf.service.common.domain.serviceconnector.PolicyServiceException",
  "message": "Max Attempts Reached for PRE connections",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 125,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T07:56:28.652+0000"
}
{
  "instant": {
    "epochSecond": 1623052588,
    "nanoOfSecond": 653125047
  },
  "thread": "boundedElastic-7",
  "level": "ERROR",
  "loggerName": "ocpm.pcf.service.common.domain.serviceconnector.PolicyServiceConnector",
  "message": "Failed to call policy service: {} ",
  "thrown": {
    "commonElementCount": 0,
    "name": "ocpm.pcf.service.common.domain.serviceconnector.PolicyServiceException",
    "extendedStackTrace": "ocpm.pcf.service.common.domain.serviceconnector.PolicyServiceException: null\n\tat ocpm.pcf.service.common.domain.serviceconnector.PolicyServiceConnector.lambda$processObject$4(PolicyServiceConnector.java:106) ~[classes!/:?]\n\tat reactor.util.retry.RetryBackoffSpec.lambda$generateCompanion$4(RetryBackoffSpec.java:557) ~[reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:374) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:250) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:97) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:189) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) [reactor-core-3.4.3.jar!/:3.4.3]\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]\n\tat java.lang.Thread.run(Thread.java:832) [?:?]\n"
  },
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 125,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T07:56:28.653+0000"
}
{
  "instant": {
    "epochSecond": 1623052588,
    "nanoOfSecond": 653405431
  },
  "thread": "boundedElastic-7",
  "level": "DEBUG",
  "loggerName": "ocpm.pcf.service.common.domain.component.policy.PolicyManager",
  "message": "process PolicyReply",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 125,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T07:56:28.653+0000"
}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check whether all the pods are running or not.
  2. Verify that correct URLs have been mentioned in the deployment file.
  3. Update any incorrect or missing information in the deployment file.
  4. Run the policy again.

Inter-microservice communication failures

Symptom

SM service receives failed to call Policy service error.

Problem
When the microservices are unable to establish communication with the each other, an error occurs and a message is received at SM service stating that the system failed to call Policy service. Search for an error message similar to the following:
logMsg=Failed to call Binding service for
    a8f8cf48-b889-44ee-95e6-a9b82fdeeef3


Error has been observed at the following site(s):
|_ checkpoint ? Request to POST http://my-cnpolicy-occnp-binding:8000/binding/v1/contextBinding/context-owner/PCF-SM [DefaultWebClient]
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1622547969,"nanoOfSecond":166956262},"thread":"boundedElastic-14","level":"ERROR","loggerName":"ocpm.pcf.service.sm.serviceconnector.BindingServiceConnector","message":"Failed to call Binding service for
      a8f8cf48-b889-44ee-95e6-a9b82fdeeef3 :
      org.springframework.web.reactive.function.client.WebClientRequestException: Connection
      refused; nested exception is java.net.ConnectException: Connection refused
      ","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":23640,"threadPriority":5,"messageTimestamp":"2021-06-01T11:46:09.166+0000"}org.springframework.web.reactive.function.client.WebClientRequestException: Connection
      refused; nested exception is java.net.ConnectException: Connection refusedat
      org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:Error has been observed at the following site(s):|_ checkpoint ? Request to POST http://my-cnpolicy-occnp-binding:8000/binding/v1/contextBinding/context-owner/PCF-SM [DefaultWebClient]Stack trace:at
      org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:137)at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:70)at reactor.core.publisher.Mono.subscribe(Mono.java:4046)at
      reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)at
      reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234)at
      reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onError(FluxPeekFuseable.java:234)at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1862)at
      reactor.core.publisher.MonoIgnoreThen$ThenAcceptInner.onError(MonoIgnoreThen.java:315)at
      org.eclipse.jetty.reactive.client.internal.AbstractSingleProcessor.onError(AbstractSingleProcessor.java:119)at
      org.eclipse.jetty.reactive.client.internal.ResponseListenerProcessor.onComplete(ResponseListenerProcessor.java:140)at
      org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:218)at
      org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:210)at
      org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:269)at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:240)at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:149)at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:818)at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:506)at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:253)at
      org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:551)at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:136)at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:633)at
      org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2$SessionListenerPromise.failConnectionPromise(HttpClientTransportOverHTTP2.java:261)at
      org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2$SessionListenerPromise.failed(HttpClientTransportOverHTTP2.java:194)at
      org.eclipse.jetty.http2.client.HTTP2Client$ClientSelectorManager.connectionFailed(HTTP2Client.java:516)at org.eclipse.jetty.io.ManagedSelector$Connect.failed(ManagedSelector.java:929)at org.eclipse.jetty.io.ManagedSelector.processConnect(ManagedSelector.java:335)at org.eclipse.jetty.io.ManagedSelector.access$1600(ManagedSelector.java:62)at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:639)at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:501)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)at
      org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)at
      org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)at java.base/java.lang.Thread.run(Thread.java:832)Caused by: java.net.ConnectException: Connection refusedat java.base/sun.nio.ch.Net.pollConnect(Native Method)at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:660)at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:875)at org.eclipse.jetty.io.SelectorManager.doFinishConnect(SelectorManager.java:355)at org.eclipse.jetty.io.ManagedSelector.processConnect(ManagedSelector.java:313)at org.eclipse.jetty.io.ManagedSelector.access$1600(ManagedSelector.java:62)at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:639)at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:501)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)at
      org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)at
      org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)at
      org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)at java.base/java.lang.Thread.run(Thread.java:832)
Resolution Steps

To resolve this issue, perform the following steps:

PCF is suspended with both primary and secondary NRF

Symptom

SM service receives status of services associated with NfType: PCF is Deregistration service warning.

Problem
When the SM service tries to establish communication with the NRF client, but PCF is suspended with both primary and secondary NRF, a warning message is received. User may search for the following log message:

logMsg=Status of services associated with NfType :PCF is Deregistration, nrfInstanceId=<>,
      response=<>

Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1622098819,
    "nanoOfSecond": 615494329
  },
  "thread": "main",
  "level": "WARN",
  "loggerName": "com.oracle.cgbu.cnc.nrf.NRFManagement",
  "message": "Status of services associated with NfType :PCF is Deregistration",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 1,
  "threadPriority": 5,
  "source": {
    "method": "registerNfInstance",
    "file": "NRFManagement.java",
    "line": 843,
    "class": "com.oracle.cgbu.cnc.nrf.NRFManagement"
  },
  "messageTimestamp": "2021-05-27T07:00:19.615+0000"
}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check whether the primaryNrfApiRoot and secondaryNrfApiRoot point to the correct endpoints.
  2. Check the status of the pcf-sm-servicepod.
  3. Check the logs of nrf-management pod.
  4. If the pcf-sm-service is down and NRF discovery is not able to register then restart the pcf-sm-service pod.
  5. Check the logs of nrf-management pod again to verify if the registration has happened successfully.

Failed to write to database

Symptom

SM service receives error on trying to save data in database.

Problem
When the SM service tries to write to database, but the request is not processed and the following error message is generated:
logMsg="Could not create connection to database server"
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1622819336,
    "nanoOfSecond": 250368963
  },
  "thread": "main",
  "level": "INFO",
  "loggerName": "ocpm.cne.common.db.JdbcDbClient",
  "message": "Maximum Pool Size is: 32 ",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 1,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-04T15:08:56.250+0000"
}
{
  "instant": {
    "epochSecond": 1622819336,
    "nanoOfSecond": 265776725
  },
  "thread": "main",
  "level": "WARN",
  "loggerName": "com.zaxxer.hikari.HikariConfig",
  "message": "HikariPool-1 - idleTimeout has been set but has no effect because the pool is operating as a fixed size pool.",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 1,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-04T15:08:56.265+0000"
}
{"instant":{"epochSecond":1622819342,"nanoOfSecond":432547862},"thread":"main","level":"ERROR","loggerName":"com.zaxxer.hikari.pool.HikariPool","message":"HikariPool-1
      - Exception during pool
      initialization.","thrown":{"commonElementCount":0,"localizedMessage":"Could not create
      connection to database server. Attempted reconnect 3 times. Giving up.","message":"Could
      not create connection to database server. Attempted reconnect 3 times. Giving
      up.","name":"java.sql.SQLNonTransientConnectionException","cause":{"commonElementCount":67,"localizedMessage":"Communications
      link failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The
      driver has not received any packets from the server.","message":"Communications link
      failure\n\nThe last packet sent successfully to the server was 0 milliseconds ago. The driver
      has not received any packets from the
      server.","name":"com.mysql.cj.exceptions.CJCommunicationsException","cause":{"commonElementCount":67,"localizedMessage":"Connection
      refused","message":"Connection refused","name":"java.net.ConnectException","extendedStackTrace":"java.net.ConnectException: Connection refused\n\tat sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]\n\tat sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
      ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333) ~[?:?]\n\tat java.net.Socket.connect(Socket.java:648) ~[?:?]\n\tat
      com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:63)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.NativeSession.connect(NativeSession.java:144)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:847)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n"},"extendedStackTrace":"com.mysql.cj.exceptions.CJCommunicationsException:
      Communications link failure\n\nThe last packet sent successfully to the server was 0
      milliseconds ago. The driver has not received any packets from the server.\n\tat
      jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]\n\tat
      jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64)
      ~[?:?]\n\tat
      jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      ~[?:?]\n\tat java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
      ~[?:?]\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]\n\tat
      com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:89)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.NativeSession.connect(NativeSession.java:144)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]\n\tat
      com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:847)
      ~[mysql-connector-java-8.0.23.jar!/:8.0.23]
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check the database connection details of the specific deployment file.
  2. Update any missing or incorrect details.
  3. Check whether the database tables are created properly and include all the required columns.

4.3.2 CM Service Issues

This section describes the most common Configuration Management (CM) service issues and their resolution steps. It is recommended for users to attempt the resolution steps provided in this guide before contacting Oracle Support.

Configuration Management GUI not loading configuration data

Symptom

Configuration data is not updated in the GUI, that is, Cloud Native Configuration Console (CNC Console).

Problem
When the configuration data is not loaded in the configuration management GUI, the following error message is generated:
logMsg=Error fetching config-items for topic: common.logging.config-mgmt, retry after 1000
      milliseconds and retry count = 1
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1622099002,"nanoOfSecond":761814372},"thread":"pool-6-thread-1","level":"ERROR","loggerName":"ocpm.cne.common.configclient.ConfigServerConnectionWithRetry","message":"Could
      not fetch config-items for topic: common.logging.config-mgmt, maxRetry is
      exhausted.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":28,"threadPriority":5,"messageTimestamp":"2021-05-27T07:03:22.761+0000"}{"instant":{"epochSecond":1622099005,"nanoOfSecond":792401204},"thread":"pool-6-thread-1","level":"WARN","loggerName":"ocpm.cne.common.configclient.ConfigServerConnectionWithRetry","message":"Error
      fetching config-items for topic: common.logging.config-mgmt, retry after 1000 milliseconds and
      retry count = 1. Exception:
      ","thrown":{"commonElementCount":0,"localizedMessage":"java.net.ConnectException: Connection refused","message":"java.net.ConnectException: Connection refused","name":"javax.ws.rs.ProcessingException","cause":{"commonElementCount":16,"localizedMessage":"Connection
      refused","message":"Connection refused","name":"java.net.ConnectException","extendedStackTrace":"java.net.ConnectException: Connection refused\n\tat sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]\n\tat sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:549)
      ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]\n\tat java.net.Socket.connect(Socket.java:648) ~[?:?]\n\tat sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[?:?]\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]\n\tat sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:?]\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[?:?]\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1261)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1194)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1082)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1016)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1600)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1528)
        ~[?:?]\n\tat java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[?:?]\n\tat
      org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:367)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:259)
      ~[jersey-client-2.30.1.jar!/:?]\n"},"extendedStackTrace":"javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused\n\tat
      org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:261)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:296)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$2(JerseyInvocation.java:643)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.internal.Errors.process(Errors.java:292)
      ~[jersey-common-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.internal.Errors.process(Errors.java:274)
      ~[jersey-common-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.internal.Errors.process(Errors.java:205)
      ~[jersey-common-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
      ~[jersey-common-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:641)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:414)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:305)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      ocpm.cne.common.configclient.ConfigServerConnectionWithRetry.getConfigurationItemByTopicWithRetry(ConfigServerConnectionWithRetry.java:313)
      [cne-common-1.11.0.jar!/:?]\n\tat
      ocpm.cne.common.configclient.ConfigClient.getConfigurationItemByTopic(ConfigClient.java:178)
      [cne-common-1.11.0.jar!/:?]\n\tat
      ocpm.cne.common.configclient.ConfigClient.getConfigurationItemByTopic(ConfigClient.java:149)
      [cne-common-1.11.0.jar!/:?]\n\tat
      ocpm.cne.common.logging.level.PullLogLevelConfigTask.run(PullLogLevelConfigTask.java:68)
      [cne-common-1.11.0.jar!/:?]\n\tat
      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]\n\tat
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]\n\tat
      java.lang.Thread.run(Thread.java:832) [?:?]\nCaused by: java.net.ConnectException: Connection refused\n\tat sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]\n\tat sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:549)
      ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]\n\tat java.net.Socket.connect(Socket.java:648) ~[?:?]\n\tat sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[?:?]\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]\n\tat sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:?]\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[?:?]\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1261)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1194)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1082)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1016)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1600)
        ~[?:?]\n\tat sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1528)
        ~[?:?]\n\tat java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[?:?]\n\tat
      org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:367)
      ~[jersey-client-2.30.1.jar!/:?]\n\tat
      org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:259)
      ~[jersey-client-2.30.1.jar!/:?]\n\t... 16
      more\n"},"endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":28,"threadPriority":5,"messageTimestamp":"2021-05-27T07:03:25.792+0000"}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check the status of the config-management and config-server pods.
  2. Check the logs of config-server pod and rule out errors related to database or connection.
  3. If the config-server is itself down, restart the config-server pod. Then, check the logs of config-management.

    After performing these steps, the data should be available on CNC Console.

4.3.3 Audit Service Issues

This section describes the most common Audit service issues and their resolution steps. It is recommended for users to attempt the resolution steps provided in this guide before contacting Oracle Support.

Audit service unable to notify services about stale session

Symptom

Audit service receives error message on sending a notification request.

Problem
When the audit service detects a stale session, it sends a notification to the owner service about the stale records. When the notification request sent by Audit service is not successful, it receives a response similar to the following error: 
logMsg=Error sending notification request to http://my-cnpolicy-occnp-pcf-sm:8005/audit/notify
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1623075339,
    "nanoOfSecond": 71945796
  },
  "thread": "main",
  "level": "WARN",
  "loggerName": "org.hibernate.orm.connections.pooling",
  "message": "HHH10001002: Using Hibernate built-in connection pool (not for production use!)",
  "endOfBatch": false,
  "loggerFqcn": "org.hibernate.internal.log.ConnectionPoolingLogger_$logger",
  "threadId": 1,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T14:15:39.071+0000"
}
{
  "instant": {
    "epochSecond": 1623075340,
    "nanoOfSecond": 282816509
  },
  "thread": "Thread-6",
  "level": "ERROR",
  "loggerName": "ocpm.common.service.audit.services.NotifyTask",
  "message": "Error sending notification request to http://my-cnpolicy-occnp-pcf-sm:8005/audit/notify",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 69,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T14:15:40.282+0000"
}
{
  "instant": {
    "epochSecond": 1623075340,
    "nanoOfSecond": 285152533
  },
  "thread": "pool-4-thread-1",
  "level": "ERROR",
  "loggerName": "ocpm.common.service.audit.services.NotifyTask",
  "message": "Notification was not sent to http://my-cnpolicy-occnp-pcf-sm:8005/audit/notify due to an error",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 62,
  "threadPriority": 5,
  "messageTimestamp": "2021-06-07T14:15:40.285+0000"
}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Verify if the correct value has been entered for AUDIT_NOTIFICATION_URL in the deployment file of SM service.
  2. Ensure that the value of notificationUri in the AuditRegistrations database table has the same value as mentioned in the SM service deployment file.

    After performing these steps, check the logs again.

4.3.4 UDR Connector Issues

This section describes the most common UDR Connector issues and their resolution steps. Users are recommended to attempt the resolution steps provided in this guide before contacting Oracle Support.

Failed or no UDR on-demand discovery to NRF on Egress Gateway

Symptom

UDR returns status code: 424 FAILED_DEPENDENCY on receiving a request from UDR connector.

Problem
When the UDR connector sends a request to UDR to fetch for example SmPolicyData, UDR tries to establish connection with NRF on Egress Gateway to process the on-demand discovery request. However, when it is unable to establish the connection, it returns the following status code:
ClientResponse has erroneous status code: 424 FAILED_DEPENDENCY
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1622112585,
    "nanoOfSecond": 184133637
  },
  "thread": "XNIO-1 task-5",
  "level": "INFO",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.UserDataUniformApiController",
  "message": "Received GET request, ueIdList: [imsi-650081000000606], reqParam: {\"smPolicyDataReq\":{\"subscription\":false,\"params\":null,\"snssai\":{\"sst\":11,\"sd\":\"abc123\"},\"dnn\":\"dnn1\",\"fields\":null},\"ldapDataReq\":{\"subscription\":false,\"params\":null},\"ssvEnabled\":false}",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 86,
  "threadPriority": 5,
  "messageTimestamp": "2021-05-27T10:49:45.184+0000"
}
{
  "instant": {
    "epochSecond": 1622112585,
    "nanoOfSecond": 186728033
  },
  "thread": "UserService_ThreadPool_6",
  "level": "DEBUG",
  "loggerName": "ocpm.pcf.service.ud.core.AbstractCommonService",
  "message": "Initialize user for [imsi-650081000000606], result: {\"pk\":\"6966884214826339058\",\"ueIdList\":[\"imsi-650081000000606\"],\"policyDataProfile\":{\"subscriptionMap\":{}}}",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 115,
  "threadPriority": 5,
  "messageTimestamp": "2021-05-27T10:49:45.186+0000"
}
{
  "instant": {
    "epochSecond": 1622112585,
    "nanoOfSecond": 189665164
  },
  "thread": "UserService_ThreadPool_6",
  "level": "INFO",
  "loggerName": "ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService",
  "message": "discover UDR instance on demand: http://pcf1111-occnp-nrf-client-nfdiscovery:8000/nnrf-disc/v1/nf-instances?target-nf-type=UDR&requester-nf-type=PCF&service-names=nudr-dr&supi=imsi-650081000000606",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 115,
  "threadPriority": 5,
  "messageTimestamp": "2021-05-27T10:49:45.189+0000"
}

{"instant":{"epochSecond":1622112585,"nanoOfSecond":184133637},"thread":"XNIO-1 task-5","level":"INFO","loggerName":"ocpm.pcf.service.ud.intf.restful.api.UserDataUniformApiController","message":"Received GET request, ueIdList: [imsi-650081000000606], reqParam: {\"smPolicyDataReq\":{\"subscription\":false,\"params\":null,\"snssai\":{\"sst\":11,\"sd\":\"abc123\"},\"dnn\":\"dnn1\",\"fields\":null},\"ldapDataReq\":{\"subscription\":false,\"params\":null},\"ssvEnabled\":false}","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":86,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:45.184+0000"}
{"instant":{"epochSecond":1622112585,"nanoOfSecond":186728033},"thread":"UserService_ThreadPool_6","level":"DEBUG","loggerName":"ocpm.pcf.service.ud.core.AbstractCommonService","message":"Initialize user for [imsi-650081000000606], result: {\"pk\":\"6966884214826339058\",\"ueIdList\":[\"imsi-650081000000606\"],\"policyDataProfile\":{\"subscriptionMap\":{}}}","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:45.186+0000"}
{"instant":{"epochSecond":1622112585,"nanoOfSecond":189665164},"thread":"UserService_ThreadPool_6","level":"INFO","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"discover UDR instance on demand: http://pcf1111-occnp-nrf-client-nfdiscovery:8000/nnrf-disc/v1/nf-instances?target-nf-type=UDR&requester-nf-type=PCF&service-names=nudr-dr&supi=imsi-650081000000606","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:45.189+0000"}

{"instant":{"epochSecond":1622112586,"nanoOfSecond":782028662},"thread":"UserService_ThreadPool_6","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"ClientResponse has erroneous status code: 424 FAILED_DEPENDENCY, body: ","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.782+0000"}
{"instant":{"epochSecond":1622112586,"nanoOfSecond":782274717},"thread":"UserService_ThreadPool_6","level":"DEBUG","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"Check for response:null","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.782+0000"}
{"instant":{"epochSecond":1622112586,"nanoOfSecond":782622756},"thread":"UserService_ThreadPool_6","level":"DEBUG","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"Retry Exception result: true","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.782+0000"}
{"instant":{"epochSecond":1622112586,"nanoOfSecond":782858203},"thread":"UserService_ThreadPool_6","level":"DEBUG","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.AlternateRouteServiceHelper","message":"Check Retry: Profile: RetryProfilebject [name = udr-retry, enableRetrySettings = true, enableAlternateRouting = true] SubQuery_Count: 0 retryCount: 2","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.782+0000"}
{"instant":{"epochSecond":1622112586,"nanoOfSecond":783021549},"thread":"UserService_ThreadPool_6","level":"DEBUG","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.AlternateRouteServiceHelper","message":"RETRY status: true, subQueryRetry: 1, ConfiguredRetryCount: 2","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.783+0000"}
{"instant":{"epochSecond":1622112586,"nanoOfSecond":783292038},"thread":"UserService_ThreadPool_6","level":"INFO","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"discover UDR instance on demand: http://pcf1111-occnp-nrf-client-nfdiscovery:8000/nnrf-disc/v1/nf-instances?target-nf-type=UDR&requester-nf-type=PCF&service-names=nudr-dr&supi=imsi-650081000000606","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":115,"threadPriority":5,"messageTimestamp":"2021-05-27T10:49:46.783+0000"}
Resolution Steps
To resolve this issue, perform the following steps:
  1. In the application-config yaml file, check whether UDR is present in nrfClientSubscribeTypes values or not. If it is not present, then add UDR.
  2. Restart nrf-management pod.
  3. Verify on-demand flag in PCF user connector GUI.
  4. After resending the request, check the UDR and NRF-management logs to verify if the request for on-demand discovery has been processed successfully.

Failed or no Policy data request to UDR on Egress Gateway

Symptom

UDR returns Could NOT find any NFProfile, set NullDataSource for UDR on receiving a policy data request from UDR connector on Egress Gateway.

Problem
To fetch policy data, for example SmPolicyData, UDR connector sends a request to UDR. However, when the UDR is unable to process the policy data request successfully, the following response is received in the log message:
Could NOT find any NFProfile, set NullDataSource for UDR
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1622025233,"nanoOfSecond":476028210},"thread":"main","level":"WARN","loggerName":"io.undertow.websockets.jsr","message":"UT026010:
      Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be
      used","endOfBatch":false,"loggerFqcn":"io.undertow.websockets.jsr.JsrWebSocketLogger_$logger","threadId":1,"threadPriority":5,"messageTimestamp":"2021-05-26T10:33:53.476+0000"}{"instant":{"epochSecond":1622027281,"nanoOfSecond":565754129},"thread":"UserService_ThreadPool_1","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"Could
      NOT find any NFProfile, set NullDataSource for
      UDR","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":50,"threadPriority":5,"messageTimestamp":"2021-05-26T11:08:01.565+0000"}{"instant":{"epochSecond":1622027281,"nanoOfSecond":576403066},"thread":"UserService_ThreadPool_1","level":"WARN","loggerName":"ocpm.pcf.service.ud.core.UdrService","message":"Failed
      GET class ocpm.pcf.service.ud.domain.SmPolicyData, ueId: imsi-650081000000606, result:
      FAILURE_DATASOURCENOTFOUND","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":50,"threadPriority":5,"messageTimestamp":"2021-05-26T11:08:01.576+0000"}{"instant":{"epochSecond":1622027281,"nanoOfSecond":580718514},"thread":"UserService_ThreadPool_1","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"A
      child [GET] request failed, <500 INTERNAL_SERVER_ERROR Internal Server
      Error,[]>","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":50,"threadPriority":5,"messageTimestamp":"2021-05-26T11:08:01.580+0000"}{"instant":{"epochSecond":1622027281,"nanoOfSecond":582259973},"thread":"UserService_ThreadPool_1","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"Request
      failed 500 INTERNAL_SERVER_ERROR,
      requestContext=RequestContext{userIds=[imsi-650081000000606],
      requestParams='{\"smPolicyDataReq\":{\"subscription\":false,\"params\":null,\"snssai\":{\"sst\":11,\"sd\":\"abc123\"},\"dnn\":\"dnn1\",\"fields\":null},\"ldapDataReq\":{\"subscription\":false,\"params\":null}}',
      requestType='GET'}.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":50,"threadPriority":5,"messageTimestamp":"2021-05-26T11:08:01.582+0000"}{"instant":{"epochSecond":1622027658,"nanoOfSecond":279448141},"thread":"UserService_ThreadPool_2","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.udr.UdrDataSourceService","message":"Could
      NOT find any NFProfile, set NullDataSource for
      UDR","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":55,"threadPriority":5,"messageTimestamp":"2021-05-26T11:14:18.279+0000"}{"instant":{"epochSecond":1622027658,"nanoOfSecond":280438989},"thread":"UserService_ThreadPool_2","level":"WARN","loggerName":"ocpm.pcf.service.ud.core.UdrService","message":"Failed
      GET class ocpm.pcf.service.ud.domain.SmPolicyData, ueId: imsi-650081000000606, result:
      FAILURE_DATASOURCENOTFOUND","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":55,"threadPriority":5,"messageTimestamp":"2021-05-26T11:14:18.280+0000"}{"instant":{"epochSecond":1622027658,"nanoOfSecond":280904616},"thread":"UserService_ThreadPool_2","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"A
      child [GET] request failed, <500 INTERNAL_SERVER_ERROR Internal Server
      Error,[]>","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":55,"threadPriority":5,"messageTimestamp":"2021-05-26T11:14:18.280+0000"}{"instant":{"epochSecond":1622027658,"nanoOfSecond":282162222},"thread":"UserService_ThreadPool_2","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"Request
      failed 500 INTERNAL_SERVER_ERROR,
      requestContext=RequestContext{userIds=[imsi-650081000000606],
      requestParams='{\"smPolicyDataReq\":{\"subscription\":false,\"params\":null,\"snssai\":{\"sst\":11,\"sd\":\"abc123\"},\"dnn\":\"dnn1\",\"fields\":null},\"ldapDataReq\":{\"subscription\":false,\"params\":null}}',
      requestType='GET'}.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":55,"threadPriority":5,"messageTimestamp":"2021-05-26T11:14:18.282+0000"}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check in the application-config yaml file whether UDR is present in nrfClientSubscribeTypes values or not. If it is not present, then add UDR.
  2. Restart nrf-management pod.
  3. Check the logs of udr-connector to verify if the policy data request registration has been sent successfully.

UDR profile is found, but UDR request fails

Symptom

UDR returns error response with code 503 on receiving a request from UDR connector.

Problem
To fetch policy data, for example SmPolicyData, UDR connector sends a request to UDR. However, when the UDR is unable to process the policy data request despite finding the UDR profile, the following response is received in the log message:
logMsg=<500 INTERNAL_SERVER_ERROR Internal Server
      Error,{\"type\":null,\"title\":\"Service
    Unavailable\",\"status\":503,\
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1627388645,
    "nanoOfSecond": 749163068
  },
  "thread": "UserService_ThreadPool_16",
  "level": "WARN",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper",
  "message": "A child [GET] request failed, <500 INTERNAL_SERVER_ERROR Internal Server Error,{\"type\":null,\"title\":\"Service Unavailable\",\"status\":503,\"detail\":\"Service Unavailable\",\"instance\":null,\"cause\":null,\"invalidParams\":null},[]>",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 3092,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T12:24:05.749+0000"
}
{
  "instant": {
    "epochSecond": 1627388645,
    "nanoOfSecond": 749294213
  },
  "thread": "UserService_ThreadPool_16",
  "level": "WARN",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper",
  "message": "Request failed 500 INTERNAL_SERVER_ERROR, requestContext=RequestContext{userIds=[imsi-450081000000001], requestParams='{\"smPolicyDataReq\":{\"subscription\":false,\"params\":null,\"snssai\":{\"sst\":11,\"sd\":\"abc123\"},\"dnn\":\"dnn1\",\"fields\":null},\"ldapDataReq\":{\"subscription\":false,\"params\":null},\"ssvEnabled\":true}', requestType='GET'}.",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 3092,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T12:24:05.749+0000"
}
Resolution Steps
To resolve this issue, perform the following steps:
  • As UDR discovery is on-demand, when a request is received, check whether NRF-Management returns correct FQDN for UDR. If the FQDN is incorrect, update with the current value and initiate the request again.

Retry to CHF or UDR alternate route on timeout or error

Symptom

UDR returns error response for retrying to CHF or UDR alternate route on timeout or error in previous attempt.

Problem
To fetch policy data, for example SmPolicyData, UDR connector sends a request to UDR. However, when the UDR is unable to process the policy data request despite finding the UDR profile, the following response is received in the log message:
logMsg=Error performing GET operation for URI
      /nf-common-component/v1/nrf-client-nfmanagement/nfProfileList"
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1627368379,"nanoOfSecond":172423065},"thread":"CmAgentTask1","level":"ERROR","loggerName":"ocpm.cne.common.cmclient.CmRestClient","message":"Error
      performing GET operation for URI
      /nf-common-component/v1/nrf-client-nfmanagement/nfProfileList","thrown":{"commonElementCount":0,"localizedMessage":"I/O
      error on GET request for \"http://localhost:5000/nf-common-component/v1/nrf-client-nfmanagement/nfProfileList\": Connect to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
      Connection refused; nested exception is org.apache.http.conn.HttpHostConnectException: Connect
      to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection
      refused","message":"I/O error on GET request for \"http://localhost:5000/nf-common-component/v1/nrf-client-nfmanagement/nfProfileList\": Connect to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
      Connection refused; nested exception is org.apache.http.conn.HttpHostConnectException: Connect
      to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection
      refused","name":"org.springframework.web.client.ResourceAccessException","cause":{"commonElementCount":14,"localizedMessage":"Connect
      to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection
      refused","message":"Connect to localhost:5000 [localhost/127.0.0.1,
      localhost/0:0:0:0:0:0:0:1] failed: Connection
      refused","name":"org.apache.http.conn.HttpHostConnectException","cause":{"commonElementCount":14,"localizedMessage":"Connection
      refused","message":"Connection refused","name":"java.net.ConnectException","extendedStackTrace":"java.net.ConnectException: Connection refused\n\tat sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]\n\tat sun.nio.ch.Net.pollConnectNow(Net.java:669) ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
      ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333) ~[?:?]\n\tat java.net.Socket.connect(Socket.java:645) ~[?:?]\n\tat
      org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:87)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:776)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n"},"extendedStackTrace":"org.apache.http.conn.HttpHostConnectException:
      Connect to localhost:5000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection
      refused\n\tat
      org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
      ~[httpclient-4.5.13.jar!/:4.5.13]\n\tat
      org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:87)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
      ~[spring-web-5.3.4.jar!/:5.3.4]\n\tat
      org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:776)
      ~[spring-web-5.3.4.jar!/:5.3.4]\nCaused by: java.net.ConnectException: Connection refused\n\tat sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]\n\tat sun.nio.ch.Net.pollConnectNow(Net.java:669) ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
      ~[?:?]\n\tat sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333) ~[?:?]\n\tat java.net.Socket.connect(Socket.java:645) ~[?:?]\n\tat
      org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75)
Resolution Steps

4.3.5 CHF Connector Issues

This section describes the most common CHF connector issues and their resolution steps. It is recommended for users to attempt the resolution steps provided in this guide before contacting Oracle Support.

No CHF profile found

Symptom

CHF Connector receives an error response message saying no CHF Profile found.

Problem
When the CHF connector tries to establish communication with the CHF to process a request, but the request is rejected by CHF because the end user specified in the request cannot be served by the CHF. In the response message, it sends the following response:
message":"Not found matching CHF, refuse this request
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1621844768,"nanoOfSecond":666347628},"thread":"UserService_ThreadPool_7","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.chf.ChfDataSourceService","message":"Not
      found matching CHF, refuse this
      request","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":103,"threadPriority":5,"messageTimestamp":"2021-05-24T08:26:08.666+0000"}{"instant":{"epochSecond":1621844768,"nanoOfSecond":666674276},"thread":"UserService_ThreadPool_7","level":"ERROR","loggerName":"ocpm.pcf.service.ud.core.SpendingLimitService","message":"No
      Data Source found for
      op:SUBSCRIBE","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":103,"threadPriority":5,"messageTimestamp":"2021-05-24T08:26:08.666+0000"}{"instant":{"epochSecond":1621844768,"nanoOfSecond":667139735},"thread":"UserService_ThreadPool_7","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"A
      child [GET] request failed, <500 INTERNAL_SERVER_ERROR Internal Server
      Error,[]>","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":103,"threadPriority":5,"messageTimestamp":"2021-05-24T08:26:08.667+0000"}{"instant":{"epochSecond":1621844768,"nanoOfSecond":667602486},"thread":"UserService_ThreadPool_7","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"Request
      failed 500 INTERNAL_SERVER_ERROR, requestContext=RequestContext{userIds=[imsi-10000000002],
      requestParams='{\"spendingLimitReq\":{\"gpsi\":\"msisdn-10000000001\",\"plmn\":{\"mcc\":\"450\",\"mnc\":\"08\"},\"policyCounterIds\":null,\"supportedFeatures\":null,\"asyncQuery\":false},\"ldapDataReq\":{\"subscription\":false,\"params\":null}}',
      requestType='GET'}.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":103,"threadPriority":5,"messageTimestamp":"2021-05-24T08:26:08.667+0000"}{"instant":{"epochSecond":1621844768,"nanoOfSecond":672460909},"thread":"XNIO-1
      task-1","level":"INFO","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"Send
      reply: \n<500 INTERNAL_SERVER_ERROR Internal Server Error,All sub request
      failed.,[]>","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":65,"threadPriority":5,"messageTimestamp":"2021-05-24T08:26:08.672+0000"}
Resolution Steps
To resolve this issue, perform the following steps:
  1. In the application-config yaml file, check whether CHF is added as a value to the nrfClientSubscriberTypes parameter. If it is not added, add CHF.
  2. Restart nrf-management pod.
  3. Initiate a request again.
  4. Check the logs of chf-connector again to verify if the request has been sent successfully.

CHF Profile found, but CHF request fails

Symptom

CHF Connector receives an error response with code 503.

Problem
When the CHF connector tries to establish communication with the CHF to process a request, but the request cannot be served by the CHF despite finding the CHF profile. In the response message, it sends the following response:
logMsg=:"WARN","loggerName":"com.oracle.cgbu.cnc.nrf.api.NRFClientApi","message":"Error
      Response received with code 503
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{
  "instant": {
    "epochSecond": 1627391543,
    "nanoOfSecond": 982425153
  },
  "thread": "HttpLoggingJettyHttpClient@720ce1b8-7778",
  "level": "WARN",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper",
  "message": "A child [GET] request failed, <503 SERVICE_UNAVAILABLE Service Unavailable,{\"type\":null,\"title\":\"Service Unavailable\",\"status\":503,\"detail\":\"Service Unavailable\",\"instance\":null,\"cause\":null,\"invalidParams\":null},[]>",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 7778,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T13:12:23.982+0000"
}
{
  "instant": {
    "epochSecond": 1627391543,
    "nanoOfSecond": 982547688
  },
  "thread": "HttpLoggingJettyHttpClient@720ce1b8-7778",
  "level": "WARN",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper",
  "message": "Request failed 500 INTERNAL_SERVER_ERROR, requestContext=RequestContext{userIds=[imsi-450081000011001], requestParams='{\"spendingLimitReq\":{\"gpsi\":\"msisdn-81000000002\",\"plmn\":{\"mcc\":\"450\",\"mnc\":\"08\"},\"policyCounterIds\":null,\"supportedFeatures\":null,\"asyncQuery\":false},\"ldapDataReq\":{\"subscription\":false,\"params\":null},\"ssvEnabled\":true}', requestType='GET'}.",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 7778,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T13:12:23.982+0000"
}
{
  "instant": {
    "epochSecond": 1627391543,
    "nanoOfSecond": 982811514
  },
  "thread": "HttpLoggingJettyHttpClient@720ce1b8-7778",
  "level": "DEBUG",
  "loggerName": "ocpm.pcf.service.ud.common.metrics.ChfDataSourceMetricsHelper",
  "message": "Pegging CHF response metric. OperationType : SUBSCRIBE, nfInstanceId : fe7d992b-0541-4c7d-ab84-666666666666, ServiceName : nchf-spendinglimitcontrol, ServiceVersion : v1, ServiceResource : subscriptions, ResponseCode : 5xx",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 7778,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T13:12:23.982+0000"
}
{
  "instant": {
    "epochSecond": 1627391543,
    "nanoOfSecond": 984365884
  },
  "thread": "XNIO-1 task-2",
  "level": "INFO",
  "loggerName": "ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper",
  "message": "Send reply: \n<500 INTERNAL_SERVER_ERROR Internal Server Error,All sub request failed.,[]>",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 7692,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T13:12:23.984+0000"
}
{
  "instant": {
    "epochSecond": 1627391586,
    "nanoOfSecond": 37087769
  },
  "thread": "Thread-2",
  "level": "INFO",
  "loggerName": "ocpm.cne.common.configclient.ConfigurationAgent",
  "message": "Configuration removed from topic=NRF.UDR, key=fe7d992b-0541-4c7d-ab84-555552222222",
  "endOfBatch": false,
  "loggerFqcn": "org.apache.logging.slf4j.Log4jLogger",
  "threadId": 27,
  "threadPriority": 5,
  "messageTimestamp": "2021-07-27T13:13:06.037+0000"
}
Resolution Steps
To resolve this issue, perform the following steps:
  1. Check whether CHF simulator is registered with NRF management. If it is not registered, register it.
  2. If it is registered, verify that correct FQDN is added.
  3. Initiate a request again.
  4. Check the logs of chf-connector again to verify if the request has been processed successfully.

Failed or No Spending Limit data request to CHF on egress

Symptom

CHF Connector receives an error response message saying no CHF Profile found.

Problem
When the CHF connector tries to establish communication with the CHF to process a request from PCF to retrieve policy counter status information for a specific UE, but the request is rejected by CHF because the end user specified in the request cannot be served by the CHF. In the response message, it sends the following response:
message":"No CHF NFProfile found
Sample Error Logs
The following is a sample for error logs that the user may see for this issue:
{"instant":{"epochSecond":1622028135,"nanoOfSecond":231441903},"thread":"UserService_ThreadPool_3","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.chf.ChfDataSourceService","message":"No CHF NFProfile found.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.231+0000"}
{"instant":{"epochSecond":1622028135,"nanoOfSecond":232856398},"thread":"UserService_ThreadPool_3","level":"ERROR","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.chf.ChfDataSourceService","message":"No CHF DataSource found","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.232+0000"}
{"instant":{"epochSecond":1622028135,"nanoOfSecond":233241908},"thread":"UserService_ThreadPool_3","level":"WARN","loggerName":"ocpm.pcf.service.ud.dbplugin.ds.chf.ChfDataSourceService","message":"Not found matching CHF, refuse this request","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.233+0000"}
{"instant":{"epochSecond":1622028135,"nanoOfSecond":238565694},"thread":"UserService_ThreadPool_3","level":"ERROR","loggerName":"ocpm.pcf.service.ud.core.SpendingLimitService","message":"No Data Source found for op:SUBSCRIBE","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.238+0000"}
{"instant":{"epochSecond":1622028135,"nanoOfSecond":239436215},"thread":"UserService_ThreadPool_3","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"A child [GET] request failed, <500 INTERNAL_SERVER_ERROR Internal Server Error,[]>","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.239+0000"}
{"instant":{"epochSecond":1622028135,"nanoOfSecond":240114557},"thread":"UserService_ThreadPool_3","level":"WARN","loggerName":"ocpm.pcf.service.ud.intf.restful.api.ApiControllerHelper","message":"Request failed 500 INTERNAL_SERVER_ERROR, requestContext=RequestContext{userIds=[imsi-650081000000606], requestParams='{\"spendingLimitReq\":{\"gpsi\":\"msisdn-20000000606\",\"plmn\":{\"mcc\":\"313\",\"mnc\":\"350\"},\"policyCounterIds\":null,\"supportedFeatures\":null,\"asyncQuery\":false},\"ldapDataReq\":{\"subscription\":false,\"params\":null}}', requestType='GET'}.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","threadId":76,"threadPriority":5,"messageTimestamp":"2021-05-26T11:22:15.240+0000"}
Resolution Steps
To resolve this issue, perform the following steps:
  1. In the application-config yaml file, check whether CHF is added as a value to the nrfClientSubscriberTypes parameter. If it is not added, add CHF.
  2. Restart nrf-management pod.
  3. Initiate a request again.
  4. Check the logs of chf-connector again to verify if the request has been sent successfully.