5 OCSS Maintenance
The Oracle® Communications Security Shield (OCSS) provides you with tools to monitor and manage your deployment. You can view information about Cloud Communication Service activities in your deployment, change or update certificates, run scripts to change, deactivate, reinstall, upgrade, and downgrade the components, and run show commands.
Cloud Communication Service Metrics, Events, Alarms, and Logs
The Cloud Communication Service (CCS) can help you monitor its operations and your applications traffic by providing metrics, events, alarms, and logs.
Cloud Communication Service Logs
The Cloud Communication Service (CCS) provides logs to help you monitor the health of the service.
- NET—network
- DBG—debug
- INF—info
- ERR—error
- EVT—event
- <date> <timestamp> <thread> <type> <details>
- <date> <timestamp> <thread> <type> <function> <details>
- 0001 1100b (EVT+ ERR+ INF+ DBG- NET-).
The CCS writes logs to the console (std::clog).
- 10 files of ~1Mb (required file system space: ~10Mb)
- Filenames are <path>/ccs<0-9>.log where <path> is configured (0 is latest log, 9 is oldest log)
Log Examples
The following is an example of a log file:
2020-04-08 10:58:51.652 (0x7fada7b0d240) EVT: CCS
v1.0.0 (build 0)2020-04-08 10:58:51.652 (0x7fada7b0d240) INF: main() limits...
2020-04-08 10:58:51.652 (0x7fada7b0d240) INF: main() parsing...2020-04-08 10:58:51.668 (0x7fada7b0d240) INF: main() spawning...
2020-04-08 10:58:51.668 (0x7fada7b0d240) INF: Base::Shard::Shard() shard=0
2020-04-08 10:58:51.669 (0x7fada2dff700) INF: Http11::Client::Client() client=WAN/idcs.oraclecloud.com:443/oauth2 on shard=0
2020-04-08 10:58:51.669 (0x7fada2dff700) INF: Http20::Client::Client() client=WAN/ocss.oraclecloud.com:443/ocss on shard=0
2020-04-08 10:58:51.669 (0x7fada2dff700) INF: Http20::Client::Client() client=WAN/osdmc.oraclecloud.com:443/osdmc on shard=0
2020-04-08 10:58:51.675 (0x7fada2dff700) INF: Http11::Server::Server()server=WAN/0.0.0.0:9000 on shard=0
2020-04-08 10:58:51.679 (0x7fada2dff700) INF: Http20::Server::Server()server=LAN/0.0.0.0:8000 on shard=0
2020-04-08 10:58:51.683 (0x7fada2dff700) INF: Http20::Server::Server()server=OAM/0.0.0.0:2000 on shard=0
2020-04-08 10:58:51.683 (0x7fada2dff700) EVT: event=WanAuthClientToken/Fsm state=Idle->NoToken service=ocss
2020-04-08 10:58:51.683 (0x7fada2dff700) EVT: alarm=WanAuthClientToken/Impaired state=CL->CR cause=no auth token service=ocss
2020-04-08 10:58:51.684 (0x7fada2dff700) EVT: event=WanAuthClientToken/Fsm state=Idle->NoToken service=osdmc
2020-04-08 10:58:51.684(0x7fada2dff700) EVT: alarm=WanAuthClientToken/Impaired state=CL->CR cause=no auth token service=osdmc
2020-04-08 10:58:51.684(0x7fada2dff700) EVT: event=WanAuthClientKey/Fsm state=Idle->NoKey
2020-04-08 10:58:51.684 (0x7fada2dff700) EVT: alarm=WanAuthClientKey/Impaired state=CL->CR cause=no auth key
2020-04-08 10:58:51.688 (0x7fada2dff700) EVT: event=HttpClient/SessionError shard=0 client=WAN/idcs.oraclecloud.com:443/oauth2 host=unkownn op=net::ip::tcp::resolver::async_resolve() err=Host not found (authoritative)
2020-04-08 10:58:51.690 0x7fada2dff700) EVT: event=RegServer/Fsm state=Idle->Active service=ocss
2020-04-08 10:58:51.690 (0x7fada2dff700) EVT: event=RegServer/Fsm state=Idle->Active service=osdmc
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: event=RegClient/Fsm state=Idle->Post service=ocss
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: alarm=RegClient/Isolated state=CL->CR service=ocss
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: event=HttpClient/SessionError shard=0 client=WAN/ocss.oraclecloud.com:443/ocss host=unkownn op=net::ip::tcp::resolver::async_resolve() err=Host not found(authoritative)
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: event=RegClient/Fsm state=Idle->Post service=osdmc
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: alarm=RegClient/Isolated state=CL->CR service=osdmc
2020-04-08 10:58:51.691 (0x7fada2dff700) EVT: event=HttpClient/SessionError shard=0 client=WAN/osdmc.oraclecloud.com:443/osdmc host=unkownn op=net::ip::tcp::resolver::async_resolve() err=Host not found(authoritative)
2020-04-08 10:58:51.692 (0x7fada2dff700) EVT: event=OamServer/Fsm state=Idle->Active
The following example shows a stat log.
System/CpuUsage 0 0 0 0
System/MemUsage 0 0 0 0
OamServer/RxGet 0
OamServer/TxError 0
> LAN service "ocss"
RegServer/RxGet 0
RegServer/RxPost 0
RegServer/RxPut 0
RegServer/RxDelete 0
RegServer/TxError 0
> LAN service "osdmc"
RegServer/RxGet 0
RegServer/RxPost 0
RegServer/RxPut 0
RegServer/RxDelete 0
RegServer/TxError 0
> WAN service "ocss"
RegClient/TxPost 3
RegClient/TxPut 0
RegClient/TxDelete 0
RegClient/RxError 2
> WAN service "osdmc"
RegClient/TxPost 3
RegClient/TxPut 0
RegClient/TxDelete 0
RegClient/RxError 2
LanAuth/Failed 0
WanAuth/Failed 0
WanAuthClientKey/TxPost 0
WanAuthClientKey/RxError 0
> WAN service "ocss"
WanAuthClientToken/TxPost 2
WanAuthClientToken/RxError 1
> WAN service "osdmc"
WanAuthClientToken/TxPost 2
WanAuthClientToken/RxError 1
> Shard 0 Host LAN/0.0.0.0:8000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 0 0 0
HttpServer/TxRsp 0
HttpServer/TxRspSize 0 0 0
> Shard 0 Host OAM/0.0.0.0:2000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 0 0 0
HttpServer/TxRsp 0
HttpServer/TxRspSize 0 0 0
> Shard 0 Host WAN/0.0.0.0:9000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 0 0 0
HttpServer/TxRsp 0
HttpServer/TxRspSize 0 0 0
> Shard 0 Peer WAN/idcs.oraclecloud.com:443/oauth2
HttpClient/Sessions 2 1 0 1HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
> Shard 0 Peer WAN/ocss.oraclecloud.com:443/ocss
HttpClient/Sessions 1 0 0 1
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
> Shard 0 Peer WAN/osdmc.oraclecloud.com:443/osdmc
HttpClient/Sessions 1 0 0 1
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
Cloud Communication Service Metrics
The Cloud Communication Service (CCS) collects and reports metrics to keep you informed about traffic, system, and authentication activities. The CCS collects metrics every 15 seconds and reports them every 15 minutes.
- Count—A cumulative number that can only increase or reset to zero upon a restart. Count provides a value.
- Gauge—A single value that can go up or down. Gauge provides a value, a minimum, maximum, and average.
- Meter—A specialized gauge that represents a per second rate that can arbitrarily go up or down. Meter provides a value, a minimum, maximum, and average.
- Histogram—A summary of
observations marked at 50th, 90th, and 99th percentiles. Historic intervals
persist as text in rotating log files with the following defaults:
- 24 hours of 15 minute intervals (96 files with required system space less than 1 MB).
- The path to both Regular Logs and Stat Logs is /opt/oracle/ccs/log.
- Filenames are <path>/stat<0-95>.log, where <path> is configured (0 is latest log, 95 is oldest log).
The following table summarizes CCS metrics.
Note:
In the CCS context, "registration" refers to enabling the ground-to-cloud communication path.Source | ID | Type | Description | Details | Instancing |
---|---|---|---|---|---|
System | CpuUsage | Gauge | Gauge of CCS process CPU utilization | CCS process CPU utilization (no per thread stats) | Global |
System | MemUsage | Gauge | Gauge of CCS process memory utilization | CCS process memory utilization (no per thread stats) | |
HttpServer | Sessions | Gauge | Gauge of HTTP server sessions | HTTP server sessions established |
Instanced by:
|
HttpServer | RxReq | Count | Count of HTTP requests received | HTTP server requests received | |
HttpServer | RxReqRate | Meter | Gauge of HTTP requests received | HTTP server requests received (requests per second) | |
HttpServer | RxReqSize | Histogram | Histogram of HTTP request sizes received | HTTP server request sizes received (bytes) | |
HttpServer | TxRsp | Count | Count of HTTP responses transmitted | HTTP server responses transmitted | |
HttpServer | TxRspSize | Histogram | Histogram of HTTP response sizes transmitted | HTTP server response sizes transmitted (bytes) | |
HttpClient | Sessions | Gauge | Gauge of HTTP client sessions | HTTP client sessions established |
Instanced by:
|
HttpClient | TxReq | Count | Count of HTTP requests transmitted | HTTP client requests transmitted | |
HttpClient | TxReqRate | Meter | Gauge of HTTP requests transmitted | HTTP client requests transmitted (requests per second) | |
HttpClient | TxReqSize | Histogram | Histogram of HTTP request sizes transmitted | HTTP client request sizes transmitted (bytes) | |
HttpClient | RxRsp | Count | Count of HTTP responses received | HTTP client responses received | |
HttpClient | RxRspSize | Histogram | Histogram of HTTP response sizes received | HTTP client response sizes received (bytes) | |
HttpClient | RxRspLatency | Histogram | Histogram of HTTP response latency | HTTP client response latency (msec) | |
OamServer | RxGet | Count | Count of server GET requests | OAM server GET requests processed | Global |
OamServer | TxError | Count | Count of server requests that failed | OAM server requests received that failed (error response) | |
RegServer | RxGet | Count | Count of server GET requests | Registration server GET requests processed |
Instanced by:
|
RegServer | RxPost | Count | Count of server POST requests | Registration server POST requests processed | |
RegServer | RxPut | Count | Count of server PUT requests | Registration server PUT requests processed | |
RegServer | RxDelete | Count | Count of server DELETE requests | Registration server DELETE requests processed | |
RegServer | TxError | Count | Count of server requests that failed | Registration server requests received that failed (error response) | |
RegClient | TxPost | Count | Count of client POST requests | Registration client POST requests generated |
Instanced by:
|
RegClient | TxPut | Count | Count of client PUT requests | Registration client PUT requests generated | |
RegClient | TxDelete | Count | Count of client DELETE requests | Registration client DELETE requests generated | |
RegClient | RxError | Count | Count of client requests that failed | Registration client requests transmitted that failed (error response) | |
LanAuth | Failed | Count | Count of LAN and OAM authentication failures | LAN and OAM authentication failures (bad API key) | Global |
WanAuth | Failed | Count | Count of WAN authentication failures | WAN authentication failures (bad Identity Domain token) | Global |
WanAuthClientKey | TxPost | Count | Count of Identity Domain key client POST requests | Identity Domain key client POST requests generated | Global |
WanAuthClientKey | RxError | Count | Count of Identity Domain key client requests that failed | Identity Domain key client requests transmitted that failed (error response) | |
WanAuthClientToken | TxPost | Count | Count of Identity Domain token client POST requests | Identity Domain token client POST requests generated |
Instanced by:
|
WanAuthClientToken | RxError | Count | Count of Identity Domain token client requests that failed | Identity Domain token client requests transmitted that failed (error response) |
Example - Statistics Log File
The following example shows a sample log file with statistics for the server (line 14) and the client (line 29).
System/CpuUsage 0 0 0 0
System/MemUsage 0 0 0 0
Auth/LanAuthFailed 0
Auth/WanAuthFailed 0
RegClient/Post 0
RegClient/Put 1
RegClient/Del 0
RegClient/Error 0
RegServer/Post 0
RegServer/Put 30
RegServer/Del 0
RegServer/Get 0
RegServer/Error 0
0.0.0.0:2000 (this is a server header for its associated stats below, and there will be a set per server instance)
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/TxRsp 0
0.0.0.0:443
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/TxRsp 0
0.0.0.0:8080
HttpServer/Sessions 1 0 0 3
HttpServer/RxReq 32
HttpServer/RxReqRate 0 0 0 0
HttpServer/TxRsp 32
144.25.17.233:443 (this is a client header for its associated stats below, and there will be a set per client instance)
HttpClient/Sessions 0 2 0 5
HttpClient/TxReq 1
HttpClient/TxReqRate 0 0 0 0
HttpClient/RxRsp 1
HttpClient/RxRspLatency 0 0 0
2.0.0.2:5808
HttpClient/Sessions 0 0 0 0
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspLatency 0 0 0
Cloud Communication Service Events
The Cloud Communication Service (CCS) records the following stateless events for your information, which typically do not require corrective action. The following table summarizes the supported CCS events.
Source | ID | Fields | Description | Details | Instancing |
---|---|---|---|---|---|
HttpServer | Exhausted |
Shard Interface HostAddr Host Port |
HTTP server exhausted | HTTP server session pool exhausted |
Instanced by:
|
HttpServer | SessionError |
Shard Interface HostAddr HostPort PeerAddr PeerPort Operation Error |
HTTP server session failed |
HTTP server session error Cause is error as returned by networking stack (includes TLS) |
|
HttpClient | SessionError |
Shard Interface PeerAddr PeerPort HostAddr HostPort Operation Error |
HTTP client session failed |
HTTP client session error Cause is error as returned by networking stack (includes TLS) |
Instanced by:
|
OamServer | Fsm |
OldState NewState |
OAM server FSM state change | OAM server FSM state change | Global |
RegServer | Fsm |
OldState NewState Service |
Registration server FSM state change | Registration server FSM state change |
Instanced by:
|
RegServer | DeviceCreated |
DeviceId Service |
Creation of a peer device registration | Registration server created a device registration record | |
RegServer | DeviceDeleted |
DeviceId Cause Service |
Deletion of a peer device registration |
Registration server deleted a device registration record Cause is one of requested or expired |
|
RegClient | Fsm |
OldState NewState Service |
Registration client FSM state change | Registration client FSM state change |
Instanced by:
|
LanAuth | Failed |
PeerAddr PeerPort |
LAN or OAM API authentication failed | LAN or OAM API authentication failed | Global |
WanAuth | Failed |
PeerAddr PeerPort |
WAN API authentication failed | WAN API authentication failed | Global |
WanAuthClientKey | Fsm |
OldState NewState |
Identity Domain key client FSM state change | Identity Domain client key FSM state change | Global |
WanAuthClientToken | Fsm |
OldState NewState Service |
Identity Domain token client FSM state change | Identity Domain client token FSM state change |
Instanced by:
|
Cloud Communication Service Alarms
The Cloud Communication Service (CCS) provides the following alarms or your information. Unlike events, alarms are stateful, ranked by severity, and typically require corrective action. See "CCS Operations, Administration, and Maintenance Interface" for information about how to view the data.
If the resolution to an alarm is to check the configuration, you can verify CCS attributes by examining them in the /opt/oracle/ccs/cfg/cfg.json file, or by using the Operations, Administration, and Maintenance (OAM) interface to dump the configuration. Configuration issues with Oracle Cloud Infrastructure (OCI), Identity Domains, and Oracle Communications Security Shield (OCSSC) require assistance from Oracle.
The following table summaizes the supported CCS alarms.
Note:
Network issues are out of scope for this guide.Source | ID | Severity | Fields | Description | Details | Resolution | Instancing |
---|---|---|---|---|---|---|---|
HttpServer | Down | Critical |
Shard Interface HostAddr HostPort Error |
HTTP server is unavailable |
Raised while HTTP server is not listening Cleared when HTTP server is listening |
Investigate reported error and correct. Root causes may include the following: CCS misconfig of host, IP, HTTP server port Network outage |
Instanced by:
|
Reg Client | Isolated | Critical | Service | Registration of CCS pending |
Raised while CCS is not registered with the cloud service Cleared when CCS is registered with the cloud service |
Determine why CCS is isolated from the cloud service and correct. Root cause may include... CCS misconfig of WAN IP, HTTP server port, TLS CCS misconfig of Identity Domain FQDN, credentials CCS misconfig of OCSS FQDN Identity Domain misconfig Cloud service misconfig Network outage |
Instanced by:
|
LanAuth | Impaired | Minor | Cause | LAN API and OAM authentication is impaired |
Raised while CCS is configured with an invalid API key Cleared when CCS is configured with a valid API key |
Determine why CCS is configured with an invalid API key and correct. Root causes may include the following: CCS misconfig of API key |
Global |
WanAuthClientKey | Impaired | Critical | Cause | WAN API authentication is impaired |
Raised while CCS has not acquired an Identity Domain key Cleared when CCS has acquired an Identity Domain key |
Determine why CCS is isolated from Identity Domain and correct. Root causes may include the following: CCS misconfig of WAN IP, HTTP server port, TLS CCS misconfig of Identity Domain FQDN, credentials Identity Domain misconfig Network outage |
Global |
WanAuthClientToken | Impaired | Critical |
Cause Service |
WAN API authentication is impaired |
Raised while CCS has not acquired an Identity Domain token Cleared when CCS has acquired an Identity Domain token |
Determine why CCS is isolated from Identity Domain and correct. Root causes may include the following: CCS misconfig of WAN IP, HTTP server port, TLS CCS misconfig of Identity Domain FQDN, credentials Identity Domain misconfig Network outage |
Instanced by:
|
Example - Regular Log File
The following example shows a sample log file with an alarm entry in line 9.
2019-07-16 07:44:58.275 (0x7f2f10be3d80) DBG: parsing..
2019-07-16 07:44:58.306 (0x7f2f10be3d80) DBG: configuring...
2019-07-16 07:44:58.309 (0x7f2f10be3d80) DBG: spawning...
2019-07-16 07:44:58.309 (0x7f2f10be3d80) DBG: Appl::Shards::enable()shards=1
2019-07-16 07:44:58.342 (0x7f2f0bbff700) INF: Http::HttpServer::HttpServer() HTTP/1.1 server on LAN interface (ipAdress:port)
2019-07-16 07:44:58.346 (0x7f2f0bbff700) INF: Http::HttpServer::HttpServer() HTTP/1.1 server on WAN interface (ipAdress:port)
2019-07-16 07:44:58.346 (0x7f2f0bbff700) INF: Http::HttpClient::HttpClient() HTTP/1.1 client on WAN interface (icds.<company>.com:port)
2019-07-16 07:44:58.346 (0x7f2f0bbff700) INF: Http::HttpClient::HttpClient() HTTP/1.1 client on WAN interface (icds.<company>.com:port)
2019-07-16 07:44:58.356 (0x7f2f0bbff700) EVT: alarm=RegClient/Isolated state=CR
2019-07-16 07:44:58.367 (0x7f2f0bbff700) ERR: Base::Client::connect() async_connect()failed for fqdn=ocss.<company>.com:port port=<port number> with ec=Connection refused
2019-07-16 07:51:19.450 (0x7f2f10be3d80) INF: Util::Signal::block() caught signal=2
2019-07-16 07:51:19.451 (0x7f2f10be3d80) DBG: shutdown...
Cloud Communication Service Management
Oracle provides a set of scripts that you run on the host to install and manage the Cloud Communication Service (CCS). After the initial installation you can run or re-run any of the scripts to further manage your deployment, including the installation scripts if you need to reinstall the service. The following topics describe the operations you can perform after installation.
Topics:
- The Cloud Communication Service Operations, Administration, and Maintenance Interface
- Cloud Communication Service Certificate Management
- Change the Cloud Communication Service Configuration
- Deactivate the Cloud Communication Service
- Downgrade the Cloud Communication Service
- Uninstall the Cloud Communication Service
- Upgrade the Cloud Communication Service
The Cloud Communication Service Operations, Administration, and Maintenance Interface
When you want to view the Cloud Communication Service (CCS) configuration, metrics, and alarms status, the CCS provides a REST API to enable you to get the information. Through the Operations, Administration, and Maintenance (OAM) interface, you can use any programming language capable of sending and receiving HTTP requests to get the information, for example, client URL Request Library (cURL) and Postman. Note that the information is read-only.
When you want to change the configuration, you must log on to the host, deactivate, make the changes, and reactivate because the CCS does not support dynamic configuration. Specify the host address as 0.0.0.0 and the port as 2000. You must provide a configured API key for authentication.
Note:
If your deployment requires a different host address and port number, your Oracle support representative can help you edit them in the configuration file.The CCS supports GET operations on the following URI paths.
Path | Description |
---|---|
/ | Help summary |
/help | Help summary |
/host | Host status |
/sys | CCS system status |
/cfg | CCS configuration dump |
/reg | CCS registration dump |
/alarms | CCS alarms |
/metrics | CCS metrics |
/alarms/server | HTTP server alarms |
/alarms/client | HTTP client alarms |
/metrics/server | HTTP server metrics |
/metrics/client | HTTP client metrics |
/raw | All metrics in raw format |
Note:
All metrics refresh every 15 seconds and the display refreshes every 15 minutes.Example for / GET
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
Path Description
----------------------------------------------------------------------------
/—This help summary
/help—This help summary
/host—Host status
/sys—CCS system status
/cfg—CCS configuration dump
/reg—CCS registration dump
/alarms—CCS alarms
/metrics—CCS metrics
/alarms/server—HTTP server alarms
/alarms/client—HTTP client alarms
/metrics/server—HTTP server metrics
/metrics/client—HTTP client metrics
/raw—All metrics in raw format
Example for /help GET
The following example shows the results of the GET operation for /help.
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
Path Description
----------------------------------------------------------------------------
/—This help summary
/help—This help summary
/host—Host status
/sys—CCS system status
/cfg—CCS configuration dump
/reg—CCS registration dump
/alarms—CCS alarms
/metrics—CCS metrics
/alarms/server—HTTP server alarms
/alarms/client—HTTP client alarms
/metrics/server—HTTP server metrics
/metrics/client—HTTP client metrics
/raw—All metrics in raw format
---------------------------------------------------------------------------
Example for /host GET
The following example shows the results of the GET operation for /host.
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
Hostname : <hostname>
Uptime : 3d 11:04:57
OS Variant : Linux
OS Release : 4.1.12-124.27.1.e17uek.x86_64
OS Version : #2 SMP Mon May 13 08:56:12 PDT 2019
Host Arch : x86_64
Num CPUs : 4
Max CPUs : 4
Load 1m: 0.00 CPU User : 0.14% Mem Total: 14400M
Load 5m: 0.08 CPU System: 0.06% Mem Used : 1196M
Load 15m: 0.36 CPU Idle : 99.79% Mem Free : 13203M
----------------------------------------------------------------------------
Example for /sys GET
The following example shows the results of the GET operation for /sys.
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
Version: CCS v1.0.0 (build 0)
Uptime : 0d 00:17:16
Alarms : (MN 0) (MJ 0) (CR 5)
Process: ccs.exe (PID 104550) (CPU 0.00%) (Mem 0.00%)
----------------------------------------------------------------------------
Example for /cfg GET
The following example shows the results of the GET operation for /cfg.
The following example shows a log that the system can generate upon request to help Oracle Customer Support personnel see the active configuration, for example, if adjustments are needed.
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
{
"Version": 1,
"System": {
"Shards": 1,
"Log-Path": "./log"
},
"HTTP": {
"Trans-Limit": 1000,
"Req-Size-Limit": 1,
"Rsp-Size-Limit": 200,
"Req-Rx-Timeout": 180,
"Rsp-Rx-Timeout": 30,
"Server-Session-Pool": 100,
"Server-Retry-Timeout": 60,
"Client-Session-Pool": 4,
"Client-Retry-Timeout": 10,
"ALPN-Negotiation": true,
"Stream-Limit": 128,
"Stream-Init-Window": 1,
"Session-Init-Window": 10
},
"IDCS": {
"Refresh-Percentage": 90,
"Retry-Timeout": 30
},
"Registration": {
"Server-Expiration-Timeout": 60,
"Server-Expiration-Padding": 10,
"Client-Device-Name": "ccs.tesla.com",
"Client-Retry-Timeout": 30,
"Client-Throttle-Timeout": 10
},
"OAM": {
"Server-Addr": "0.0.0.0",
"Server-Port": 2000
},
"LAN": {
"Server-Addr": "0.0.0.0",
"Server-Port": 8000,
"TLS-Cipher-Suite": "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384",
"TLS-Server-Cert": "./ssl/lan-cert.pem",
"TLS-Server-Key": "./ssl/lan-key.pem",
"TLS-Server-DH": "./ssl/dh2048.pem",
"TLS-Client-CA-Path": "./ssl/ca",
"TLS-Client-Verify": true,
"API-Key-Verify": true
},
"WAN": {
"Server-FQDN": "ccs.tesla.com",
"Server-Addr": "0.0.0.0",
"Server-Port": 9000,
"TLS-Cipher-Suite": "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384",
"TLS-Server-Cert": "./ssl/wan-cert.pem",
"TLS-Server-Key": "./ssl/wan-key.pem",
"TLS-Server-DH": "./ssl/dh2048.pem",
"TLS-Client-CA-Path": "./ssl/ca",
"TLS-Client-Verify": true,
"Identity Domain-FQDN": "idcs.oraclecloud.com",
"Identity Domain-Port": 443,
"Identity Domain-Tenant-ID": "idcs-tenant-id",
"Identity Domain-Verify": true
},
"Services": [
{
"Prefix": "ocss",
"FQDN": "ocss.oraclecloud.com",
"Port": 443,
"Tenant-ID": "ocss-tenant-id",
"API-Key": "ocss-lan-api-key",
"API-Key-Alt": "ocss-lan-api-key-alt",
"Identity Domain-Client-ID": "ocss-idcs-client-id",
"Identity Domain-Client-Secret": "ocss-idcs-client-secret"
},
{
"Prefix": "osdmc",
"FQDN": "osdmc.oraclecloud.com",
"Port": 443,
"Tenant-ID": "osdmc-tenant-id",
"API-Key": "osdmc-lan-api-key",
"API-Key-Alt": "osdmc-lan-api-key-alt",
"Identity Domain-Client-ID": "osdmc-idcs-client-id",
"Identity Domain-Client-Secret": "osdmc-idcs-client-secret"
}
]
}
--------------------------------------------------------------------------
Example for /reg GET
The following example shows the results of the GET operation for /reg.
The following example reflects the current registration status of on-premises devices using the Cloud Communication Service (CCS). The example shows only the CCS because the Policy Decision Engine has not yet registered with CCS.
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
> LAN service "ocss"
{
"name": "ccs.tesla.com",
"type": "CCS",
"version": "CCS v1.0.0 (build 0)",
"httpAddress": "ccs.tesla.com",
"httpPort": 9000,
"devices": []
}
> LAN service "osdmc"
{
"name": "ccs.tesla.com",
"type": "CCS",
"version": "CCS v1.0.0 (build 0)",
"httpAddress": "ccs.tesla.com",
"httpPort": 9000,
"devices": []
}
---------------------------------------------------------------------------
Example for /alarms GET
The following example shows the results of the GET operation for /alarms.
- CL—Clear
- MN—Minor
- MJ—Major
- CR—Critical
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
> WAN service "ocss"
RegClient Lvl Timestamp
------------------------------ -------- -----------------------
Isolated CR 2020-04-11 17:42:54.156
> WAN service "osdmc"
RegClient Lvl Timestamp
------------------------------ -------- -----------------------
Isolated CR 2020-04-11 17:42:54.157
LanAuth Lvl Timestamp
------------------------------ -------- -----------------------
Impaired CL 2020-04-11 18:02:01.890
WanAuthClientKey Lvl Timestamp
------------------------------ -------- -----------------------
Impaired CR 2020-04-11 17:42:54.151
> WAN service "ocss"
WanAuthClientToken Lvl Timestamp
------------------------------ -------- -----------------------
Impaired CR 2020-04-11 17:42:54.151
> WAN service "osdmc"
WanAuthClientToken Lvl Timestamp
------------------------------ -------- -----------------------
Impaired CR 2020-04-11 17:42:54.151 Cl 2020-01-23 13:11:05.620
Example for /metrics GET
The following example shows the results of the GET operation for /metrics.
In the following example, the Avg, Min, and Max column headings correlate to the percentile values that you can see on the Histogram on the Dashboard. (50th, 90th, and 99th percentiles)
$ curl -k -H "Authorization: Bearer <api-key>" https://123.0.0.1:2020/
----------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS <version> / <timestamp>
----------------------------------------------------------------------------
System Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
CpuUsage 0 0 0 0
MemUsage 0 0 0 0
OamServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
RxGet 4 - - -
TxError 0 - - -
> LAN service "ocss"
RegServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
RxGet 0 - - -
RxPost 0 - - -
RxPut 0 - - -
RxDelete 0 - - -
TxError 0 - - -
> LAN service "osdmc"
RegServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
RxGet 0 - - -
RxPost 0 - - -
RxPut 0 - - -
RxDelete 0 - - -
TxError 0 - - -
> WAN service "ocss"
RegClient Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
TxPost 4 - - -
TxPut 0 - - -
TxDelete 0 - - -
RxError 4 - - -
> WAN service "osdmc"
RegClient Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
TxPost 4 - - -
TxPut 0 - - -
TxDelete 0 - - -
RxError 4 - - -
LanAuth Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Failed 0 - - -
WanAuth Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Failed 0 - - -
WanAuthClientKey Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
TxPost 0 - - -
RxError 0 - - -
> WAN service "ocss"
WanAuthClientToken Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
TxPost 2 - - -
RxError 2 - - -
> WAN service "osdmc"
WanAuthClientToken Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
TxPost 2 - - -
RxError 2 - - -
Example for /alarms/server GET
The following example shows the results of the GET operation for /alarms/server.
$ curl -k -H "Authorization: Bearer <api-key>" https://127.0.0.1:2000/alarms/server
-------------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS v1.0.0 (build 0) /alarms/server @ 2020-04-11 18:04:53.848
-------------------------------------------------------------------------------
Shard 0 Host LAN/0.0.0.0:8000
HttpServer Lvl Timestamp
------------------------------ -------- -----------------------
Down CL 2020-04-11 17:42:54.146
Shard 0 Host OAM/0.0.0.0:2000
HttpServer Lvl Timestamp
------------------------------ -------- -----------------------
Down CL 2020-04-11 17:42:54.150
Shard 0 Host WAN/0.0.0.0:9000
HttpServer Lvl Timestamp
------------------------------ -------- -----------------------
Down CL 2020-04-11 17:42:54.142
Example for /alarms/client GET
The following example shows the results of the GET operation for /alarms/client.
$ curl -k -H "Authorization: Bearer <api-key>" https://127.0.0.1:2000/alarms/client
-------------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS v1.0.0 (build 0) /alarms/client @ 2020-04-11 18:05:20.337
-------------------------------------------------------------------------------
None yet defined.
Example for /metrics/server GET
The following example shows the results of the GET operation for /metrics/server.
$ curl -k -H "Authorization: Bearer <api-key>" https://127.0.0.1:2000/metrics/server
-------------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS v1.0.0 (build 0) /metrics/server @ 2020-04-11 18:05:56.481
-------------------------------------------------------------------------------
> Shard 0 Host LAN/0.0.0.0:8000
HttpServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 0 0 0 0
RxReq 0 - - -
RxReqRate 0 0 0 0
RxReqSize - 0 0 0
TxRsp 0 - - -
TxRspSize - 0 0 0
> Shard 0 Host OAM/0.0.0.0:2000
HttpServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 0 0 0 0
RxReq 9 - - -
RxReqRate 0 0 0 0
RxReqSize - 1 1 1
TxRsp 9 - - -
TxRspSize - 1024 3072 3072
> Shard 0 Host WAN/0.0.0.0:9000
HttpServer Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 0 0 0 0
RxReq 0 - - -
RxReqRate 0 0 0 0
RxReqSize - 0 0 0
TxRsp 0 - - -
TxRspSize - 0 0 0
Example for /metrics/client GET
The following example shows the results of the GET operation for /metrics/client.
$ curl -k -H "Authorization: Bearer <api-key>" https://127.0.0.1:2000/metrics/client
-------------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS v1.0.0 (build 0) /metrics/client @ 2020-04-11 18:06:10.228
-------------------------------------------------------------------------------
> Shard 0 Peer WAN/idcs.oraclecloud.com:443/oauth2
HttpClient Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 2 1 0 1
TxReq 0 - - -
TxReqRate 0 0 0 0
TxReqSize - 0 0 0
RxRsp 0 - - -
RxRspSize - 0 0 0
RxRspLatency - 0 0 0
> Shard 0 Peer WAN/ocss.oraclecloud.com:443/ocss
HttpClient Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 1 0 0 0
TxReq 0 - - -
TxReqRate 0 0 0 0
TxReqSize - 0 0 0
RxRsp 0 - - -
RxRspSize - 0 0 0
RxRspLatency - 0 0 0
> Shard 0 Peer WAN/osdmc.oraclecloud.com:443/osdmc
HttpClient Val Avg|P50 Min|P90 Max|P99
------------------------------ -------- -------- -------- --------
Sessions 1 0 0 0
TxReq 0 - - -
TxReqRate 0 0 0 0
TxReqSize - 0 0 0
RxRsp 0 - - -
RxRspSize - 0 0 0
RxRspLatency - 0 0 0
Example for /raw GET
The following example shows the results of the GET operation for /raw.
$ curl -k -H "Authorization: Bearer <api-key>" https://127.0.0.1:2000/raw
-------------------------------------------------------------------------------
Oracle Cloud Communications Service, (c) 2020 Oracle
CCS v1.0.0 (build 0) /raw @ 2020-04-11 18:07:03.175
-------------------------------------------------------------------------------
System/CpuUsage 0 0 0 0
System/MemUsage 0 0 0 0
OamServer/RxGet 11
OamServer/TxError 0
> LAN service "ocss"
RegServer/RxGet 0
RegServer/RxPost 0
RegServer/RxPut 0
RegServer/RxDelete 0
RegServer/TxError 0
> LAN service "osdmc"
RegServer/RxGet 0
RegServer/RxPost 0
RegServer/RxPut 0
RegServer/RxDelete 0
RegServer/TxError 0
> WAN service "ocss"
RegClient/TxPost 14
RegClient/TxPut 0
RegClient/TxDelete 0
RegClient/RxError 14
> WAN service "osdmc"
RegClient/TxPost 14
RegClient/TxPut 0
RegClient/TxDelete 0
RegClient/RxError 14
LanAuth/Failed 0
WanAuth/Failed 0
WanAuthClientKey/TxPost 0
WanAuthClientKey/RxError 0
> WAN service "ocss"
WanAuthClientToken/TxPost 7
WanAuthClientToken/RxError 7
> WAN service "osdmc"
WanAuthClientToken/TxPost 7
WanAuthClientToken/RxError 7
> Shard 0 Host LAN/0.0.0.0:8000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 0 0 0
HttpServer/TxRsp 0
HttpServer/TxRspSize 0 0 0
> Shard 0 Host OAM/0.0.0.0:2000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 11
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 1 1 1
HttpServer/TxRsp 11
HttpServer/TxRspSize 1024 2432 3072
> Shard 0 Host WAN/0.0.0.0:9000
HttpServer/Sessions 0 0 0 0
HttpServer/RxReq 0
HttpServer/RxReqRate 0 0 0 0
HttpServer/RxReqSize 0 0 0
HttpServer/TxRsp 0
HttpServer/TxRspSize 0 0 0
> Shard 0 Peer WAN/idcs.oraclecloud.com:443/oauth2
HttpClient/Sessions 2 1 0 1
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
> Shard 0 Peer WAN/ocss.oraclecloud.com:443/ocss
HttpClient/Sessions 1 0 0 0
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
> Shard 0 Peer WAN/osdmc.oraclecloud.com:443/osdmc
HttpClient/Sessions 1 0 0 0
HttpClient/TxReq 0
HttpClient/TxReqRate 0 0 0 0
HttpClient/TxReqSize 0 0 0
HttpClient/RxRsp 0
HttpClient/RxRspSize 0 0 0
HttpClient/RxRspLatency 0 0 0
Cloud Communication Service Certificate Management
The Cloud Communication Service (CCS) activation script requires you to enter certain information about the authentication credentials that you want CCS to use when communicating to the Session Border Controller (SBC). The CCS uses certificates and keys to authenticate the SBC.
LAN
For the Local Area Network (LAN) interface you must supply a server certificate, a public key, and a signing certificate.
WAN
For the Wide Area Network (WAN) interface you must supply a server certificate and a public key. You do not need to provide the signing certificate for the WAN because the WAN requires a commercial Certificate Authority. Oracle ships CCS with the root signing keys already provided by the commercial Certificate Authorities.
Use the CCS installation procedure to set the certificates that you want to use for the Oracle® Communications Security Shield service. See Install, Configure, and Activate the Cloud Communication Service.
Change the Cloud Communication Service Configuration
When you want to change the IP address, FQDN, or port for the Cloud Communication Service (CCS), use the CCS installation procedure. Then go to sip-configuration > spl-options on the Session Border Controller and change the ocss-service-address to point to the CCS.
Deactivate the Cloud Communication Service
If you want to deactivate the Cloud Communication Service (CCS) installation, for example to add new certificates or migrate to another host, you can do so without uninstalling the CCS. Use the deactivate script to stop the CCS service from running, while leaving the CCS installed on the system for future re-activation.
- Confirm that CSS is installed and activated. See Install, Configure, and Activate the Cloud Communication Service for confirmation instructions.
- Be aware of consequences that can affect service.
- Ensure that Perl5 is installed on the host.
- Ensure that you have root privileges.
Uninstall the Cloud Communication Service
Use the following procedure when you want to remove the Cloud Communication Service (CCS) from the system, for example, when migrating to another host. If you want to re-install CCS after using this procedure, see "Install, Configure, and Activate CCS."
- Confirm that CCS is installed and deactivated. See "Deactivate the CSS Installation."
- Be aware of consequences that can affect service.
- Ensure that Perl5 is installed on the host.
- Ensure that you have root privileges.
Procedure
Upgrade the Cloud Communication Service
Use the following procedure to upgrade the Cloud Communication Service (CCS). The upgrade preserves a snapshot of the most recent installation in the /opt/ocss/ccs directory for a future downgrade, if needed. After you upgrade, ensure that CCS works as expected. If not, downgrade immediately. Note that any configuration changes you made between the last upgrade and the downgrade do not persist.
- Confirm that CCS is installed and activated.
- Know the consequences of an upgrade, such as behavior changes. See the Release Documentation.
- Download the CCS archive file (ccs-<version>.<build>.tgz) that you want to upgrade to from My Oracle Support (MOS) or Oracle SaaSOps
Downgrade the Cloud Communication Service
Use the following procedure to downgrade the Cloud Communication Service (CCS).
- Confirm that OCSS is installed and activated.
- Know the consequences of a downgrade, such as behavior changes and the loss of configuration changes since the last upgrade.
Activate Debug in the CCS
When you want to activate Debug for the Cloud Communication Service (CCS), you must deactivate CCS, edit the ccs/perl/activate.pl script, and reactivate CCS.
system("docker run --detach --network=host --restart=unless-stopped --volume=$ccs_dir:/mnt
--name=$img $img_id --cfg=/mnt/cfg/cfg.json --mask=31 > /dev/null 2>&1") ==
0 or die "error: $!";
- NET (1)
- DBG (2)
- INF (4)
- ERR (8)
- EVT (16)
Note:
Remove “--mask=<value>” when you are done debugging.OCSS Show Commands
The following information describes the show commands available through the Acme Command Line Interface (ACLI) on your Session Border Controller for viewing Session Plug-in Language (SPL) Application statistics about the Oracle® Communications Security Shield (OCSS).
The statistics reports are divided into groups. You can view all groups at once or you can specify a single group to view.
Note:
You must load an OCSS.pkg created on or after August 1, 2019 to see the following commands.Show all SPL Application Stats
Syntax: show spl
appstats
- ocss—Displays the SPL build informatiuon.
- ocss-http-stats—Displays Client Requests Sent and Client Requests Received
- ocss-http-stats-detailed—Displays Policy Evaluations Requests (POST), Policy Results, Call Updates (PUT), Call Termination Updates (PUT), Registration Requests (POST), Registration Refresh (PUT), Reregistration (DELETE), Mid-Call Updates, and Mid-Call Actions for up to three Session Border Controllers (SBC) configured for OCSS.
- ocss-connection-check-stats—Displays statistics for Requests Sent, Response 2xx, Response 400, Response 403, Response 404, Response 4xx, Response 5xx, Response Other, Response Timeout, and Response Invalid.
- ocss-policy-response-time—Displays statistics for 0ms-200ms, 201ms to 500ms, 501ms to 1000ms, 1001ms-1500ms, 1501ms-2000ms, 2001ms-2500ms, 2501ms-3000ms, 3001-3500ms, 3501ms-4000ms, and Above 10000ms, Policy requests, and Average Response Time.
- ocss-policy-rtt (round trip time)—Displays statistics for 0ms-200ms, 201ms to 500ms, 501ms to 1000ms, 1001ms-1500ms, 1501ms-2000ms, 2001ms-2500ms, 2501-3000ms, 3001ms-3500ms, 3501ms-4000ms, 4001ms-10000ms, Above 10000ms, Policy requests, and Average Round TripTime.
- ocss-registration-status—Displays Device Name, Device Type, Resource ID, Registration Interval, OCSS Service Address, Registration State, Registered At, and Local Expire.
- spl show sip circuit-breaker ocss-policy—Displays failureThreshold, retryTimePeriod, checked, checkedHalfOpen, errorCount, notSendOpen, errors, state OPEN, nthSendHalfOpen, windowDuration, notSendHalfOpen, okCount, sendClosed, sendHalfOpen, and ratePrevious. Also, displays the State (Active | Available | Unavailable) and Circuit Breaker State (Closed | Open) of up to three SBCs connected to OCSS through the Cloud Communication Service (CCS). Also displays the IP address of the active server.
- reset spl-stats application—
Show a Specific SPL Stats Group
show spl appstats
command with the
group name. For example, to view only the ocss-policy-rtt
report:show spl appstats ocss-policy-rtt
Reset the SPL Stats by Group
reset spl-stats
application
command with the group name. For
example, to reset the ocss-policy-rtt
report:reset spl-stats application ocss-policy-rtt