3 New Features in ECE

The Oracle Communications Elastic Charging Engine (ECE) 12.0 Patch Set releases include several new features.

Caution:

Deploying charging for 5G with HTTP Gateway (5G CHF) requires a cloud native deployment of ECE and BRM components. The HTTP Gateway can be used only on an ECE cloud native system.

Topics in this document:

• New Features in ECE 12.0 Patch Set 8
• New Features in ECE 12.0 Patch Set 7
• New Features in ECE 12.0 Patch Set 6
• New Features in ECE 12.0 Patch Set 5
• New Features in ECE 12.0 Patch Set 4
• New Features in ECE 12.0 Patch Set 3
• New Features in ECE 12.0 Patch Set 2
• New Features in ECE 12.0/12.0 Patch Set 1

New Features in ECE 12.0 Patch Set 8

ECE 12.0 Patch Set 8 includes the following enhancements:

• Extensions for Midsession-Rated Events
• Reasons Now Included for Midsession-Rated Events
• Midsession-Rated Events Created When USU Block Missing
• Extensions for Sy Request-Response Flow
• Active-Active Mode Can Now Process Requests in Local Sites
• ECE Enhancements for Processing Free Allowances
• Support for Tax Selectors and Tax Exemption Selectors
• ECE Supports Flexible Proration Settings
• ECE Cloud Native Contains New ConfigLoader Pod
• Support for Prepaid Usage Overage
• Rated Event Publisher Support for CDR Streaming

Extensions for Midsession-Rated Events

By default, ECE generates a rated event for a network session only when a Terminate operation ends the session. You can configure ECE to generate a rated event in the middle of an ongoing active session based on trigger criteria. In previous releases, the available trigger criteria for creating a midsession-rated event was limited to the following:

• Duration (for example, create a rated event every 4 hours that a session is active)

• Quantity (for example, create a rated event whenever downloaded data totals 70 MB or more)

• Time of day (for example, create a rated event daily at 23:00:00 during the life of the session)

You can also trigger a midsession-rated event based on custom criteria, such as tariff changes or specific network conditions. To do so, you extend ECE at the pre-rating or post-rating extension points.

For more information, see the following:

• These ECE SDK sample programs: SamplePreRatingMidSessionExtension and SamplePostRatingMidSessionExtension

• "Configuring ECE to Generate Midsession Rated Events" in ECE Implementing Charging

Reasons Now Included for Midsession-Rated Events

When creating midsession-rated events, ECE now adds why a rated event was triggered during an active session. For example, the reason could be that:

• The configured volume was reached

• The configured duration was reached

• The configured time of the day was crossed

In previous releases, ECE did not include the reason for creating the midsession-rated event.

For more information, see "Viewing Reasons for Midsession-Rated Events" in ECE Implementing Charging.

Midsession-Rated Events Created When USU Block Missing

When the network sends a Final Unit Indicator (FUI) followed by a top up, ECE generates a reauthorization request (RAR). The network then sends an update request, sometimes not including the Used Service Units (USU) block for the ongoing session. When this occurs, ECE can now generate a midsession-rated event. This ensures that any part of the reservation consumed by the account is reported, preventing revenue loss.

In previous releases, if the USU block was missing from the update request, ECE cleared the reservation and information about the amount of the reservation consumed, causing revenue exposure.

For information, see "Generating Midsession-Rated Events When USU Block Missing" in ECE Implementing Charging.

Extensions for Sy Request-Response Flow

The Diameter Sy request-response flow now includes extension hooks for customizing Sy processes. For example, you could customize ECE to suspend Sy signaling when a subscriber's account is suspended.

For information, see "Diameter-Request Processing Extension Points" in ECE Implementing Charging.

Active-Active Mode Can Now Process Requests in Local Sites

In previous releases, ECE active-active disaster recovery configurations always processed usage requests according to the preferred site assignments in the customerGroup list. For example, if subscriber A's preferred primary site was site 1, ECE would always process subscriber A's usage requests in site 1. If subscriber A's usage request was received by production site 2, it was always sent to production site 1 for processing.

Now, you can configure the ECE active-active mode to process usage requests in the site that received the request, regardless of the subscriber's preferred site. For example, if a subscriber's usage request is received by production site 1, it is processed in production site 1. Likewise, if the usage request is received by production site 2, it is processed in production site 2.

Note:

This configuration does not apply to usage charging requests for members of a sharing group. Usage requests for sharing group members continue to be processed in the same site as the sharing group parent.

For more information, see "Processing Usage Requests in the Site Received" in BRM System Administrator's Guide.

ECE Enhancements for Processing Free Allowances

ECE includes enhancements for processing free allowances, such as 100 free minutes or 5 free movie rentals.

In previous releases, ECE first applied charges for all usage and then applied a discount for the granted allowance. This ensured that the customer was not charged until the allowance was exhausted. For example, assume a customer owned a plan that granted 500 GB of free data per month and charged $10 per 100 GB of data. If a customer used 600 GB of data in a month, ECE would first apply a $60 charge for the 600 GB of data and then apply a $50 discount for the 500 GB of free data. Applying charges first and then applying discounts was based on how PDC was being configured to address allowances.

PDC now allows you to create product offerings that consume granted allowances before applying any charges. To improve processing performance, ECE also supports this rating flow. ECE can now consume free allowances first and then charges for usage based on pricing configurations in PDC. Using the previous example, if the customer uses 600 GB of data in a month, ECE now consumes the 500 GB of free data and then applies a $10 charge for the remaining 100 GB of data usage.

For information, see "Processing Granted Allowances Before Applying Usage Charges" in ECE Implementing Charging.

Support for Tax Selectors and Tax Exemption Selectors

ECE supports the tax selectors and tax exemption selectors configured in PDC for taxation during charging.

Tax selectors and tax exemption selectors are used to apply taxes based on account, service, event, and profile attributes. This allows you to choose whether to use the direct tax code or to choose the tax code using the selectors in the rate plan while creating the product.

For more information, see "Configuring Taxation in ECE" in ECE Implementing Charging.

ECE Supports Flexible Proration Settings

ECE now supports flexible proration settings. When charging for usage, ECE now uses the charge-offer-level settings for proration that are configured in PDC. For more information, see "PDC Supports Flexible Proration Options".

ECE Cloud Native Contains New ConfigLoader Pod

In previous ECE cloud native releases, ConfigLoader was a Kubernetes job. This meant that if an ECE cluster went down, you had to manually restart the cluster.

In Patch Set 8, ConfigLoader has been changed to a Pod. This enables data to be automatically loaded into the ECS pods from the persistence database whenever the ECE cluster restarts.

For more information, see "Enabling Persistence in ECE" in BRM Cloud Native System Administrator's Guide.

Support for Prepaid Usage Overage

You can now configure ECE to capture any overage amounts by prepaid customers during an active session, which can help you prevent revenue leakage. If the network reports that the number of used units during a session is greater than a customer's available allowance, ECE charges the customer up to the available allowance. It then creates a CDR with information about the overage amount and sends it to a new ECE Overage Kafka topic.

To enable ECE to check for and capture any overage:

1. Access the ECE configuration MBeans in a JMX editor, such as JConsole. See "Accessing ECE Configuration MBeans" in ECE Implementing Charging.

2. Expand the ECE Configuration node.

3. Expand charging.server.

4. Expand Attributes.

5. Set the checkReservationOverImpact attribute to true.

For more information, see "Configuring ECE to Support Prepaid Usage Overage" in ECE Implementing Charging.

Rated Event Publisher Support for CDR Streaming

ECE's Rated Event Publisher now supports streaming events to and from Kafka. This enables near-real-time reporting of rated event data and makes rated event data available for consumption from Apache Kafka by third party systems as well as by BRM. See "Event Streaming Mode (Patch Set 8 or later)" in Loading Rated Events for more information.

New Features in ECE 12.0 Patch Set 7

ECE 12.0 Patch Set 7 includes the following enhancements:

• Time-Based Price Overrides and Dynamic Usage Rating
• Bundle Renewal Triggered from ECE
• Non-Linear Rating Configurable by Event Type
• Balance Query Returns Lifecycle Information
• Current Balance Now Returned with Balance Impact
• First Usage Activation Enhancements
• Network Signaling Optimization
• Balance Monitoring and Group Credit Limits
• Product Sharing
• Logging of Usage Rating Failures
• Robust Handling of Configuration Data
• Quorum Policy Handling in ECE
• Rolling Upgrade Enhancements
• Access to Bundle Information in ECE
• Ability to Call a BRM Opcode from ECE
• CDR Generator Enhancements
• Policy-Based Charging Notification Enhancements
• Usage Consumption Management Report Certification

Time-Based Price Overrides and Dynamic Usage Rating

Data related to time-based price overrides is now synchronized with ECE. Multiple overrides are available for different date ranges for both charge offers and discount offers.

ECE also now allows you to override currency or non-currency resources for a limited or unlimited period of time without customization. These overrides can be defined at the balance impact level for charge offers and discount offers.

For more information, see "Managing Dynamic Charging Overrides for Online Sessions" in ECE Implementing Charging.

Bundle Renewal Triggered from ECE

If an active session crosses a bundle validity boundary, ECE now requests a bundle renewal from BRM. BRM then renews the bundle and the updated grant is synchronized with ECE, to ensure that rating continues correctly.

Non-Linear Rating Configurable by Event Type

Previously, you could only configure non-linear rating at the product level. Now, you can configure non-linear rating for specific events configured for a product. This provides more flexibility when configuring products.

For more information, see "Configuring Incremental Rating for Midsession Rating Condition Changes" in ECE Implementing Charging.

Balance Query Returns Lifecycle Information

The balance query operation now returns the lifecycle state and lifecycle expiry information. The calling application can use this information to take appropriate action, such as disconnecting a voice call if the subscriber is in a suspended state.

Current Balance Now Returned with Balance Impact

You can now configure ECE to include the current balance and current loan balance as part of the balance impact in rated events. CSRs will then have this information available if a customer calls to inquire about a particular transaction.

First Usage Activation Enhancements

Data for configuring bundles or packages to start on first usage is now synchronized with ECE. When you use this option, the first usage of any charge offer or discount offer activates all of the offers in the bundle or package.

Network Signaling Optimization

Many times, bundles expire at midnight of a particular day. If your customers are using the bundles at that time, the renewal and request messages that are transmitted at midnight can cause an undue network load. To prevent this, ECE can now randomize the validity times for a service so that renewal requests do not occur simultaneously.

For more information, see "Optimizing Network Signaling" in ECE Implementing Charging.

Balance Monitoring and Group Credit Limits

ECE now joins BRM in supporting balance monitoring and monitor group shared balances. You can set up shared balances for specific services and ECE will update BRM with the balance impacts after rating.

Product Sharing

ECE, like BRM, has added support for product sharing. This allows you to change rates once at the group owner level, rather than having to change each of the subscriber accounts.

In support of this, ECE has added product sharing agreements. During usage rating, ECE will consider priority to determine whether to use the shared package.

Logging of Usage Rating Failures

A new logger has been created for failed usage rating requests. Failed usage requests now include the reason for the failure, along with the customer and session identifiers. For example, in the case of a Diameter Gateway usage rating failure with NO_QUALIFIED_CHARGE_OFFERS, you might see:

ERROR -  -  -  - Failing Usage Request for subscriber ID : <<PUID>>, session ID :<<Session ID>, 
reasons : [NO_QUALIFIED_CHARGE_OFFERS, ZERO_RUM_QUANTITY]

This feature is configuration-driven and works even if debug mode is not enabled.

For more information, see "Troubleshooting Failed Usage Requests" in BRM System Administrator's Guide.

Robust Handling of Configuration Data

ECE uses the charging-settings.xml file to store configuration information. This file is loaded into the cache when ECE starts, and run-time changes can be made using JConsole. Starting with this release, if this configuration file is accidentally deleted or corrupted while ECE is running, you can rebuild the file from the cache. The rebuilt file will also contain any run-time configuration changes that were made using JConsole.

For more information, see "Troubleshooting a Corrupted ECE Configuration File" in BRM System Administrator's Guide.

Quorum Policy Handling in ECE

You can now set a timeout survivor quorum, which is the minimum number of cluster members that must remain running to avoid data loss when the cluster service terminates suspect members. For more information:

• For ECE cloud native environments, see "Adding Elastic Charging Engine Keys" in BRM Cloud Native Deployment Guide.

• For ECE on-premise systems, see "Configuring Cluster Quorum Policy" in BRM System Administrator’s Guide.

Rolling Upgrade Enhancements

ECE previously had a single wait parameter (rollingUpgradeGatewayReadinessWait in the ece.properties file) to ensure that all of the gateways bootstrapped completely during a rolling upgrade. However, different gateways can take different amounts of time to bootstrap.

ECE now has the following separate ece.properties parameters for each gateway so that each gateway does not have to wait for the time required by the slowest gateway:

• rollingUpgradeEmGatewayReadinessWait

• rollingUpgradeCdrGatewayReadinessWait

• rollingUpgradeDiameterGatewayReadinessWait

• rollingUpgradeRadiusGatewayReadinessWait

• rollingUpgradeHttpGatewayReadinessWait

For more information, see "Starting the Rolling Upgrade Process" in ECE Installation Guide.

Access to Bundle Information in ECE

Bundle (deal) information is now cached in ECE, so that it can be sent with rated events. The deal POID, deal name, and purchased bundle name are available through APIs in extension points, so that you can use this information when you are extending the functionality.

Ability to Call a BRM Opcode from ECE

It is now possible to call a BRM opcode from ECE as a response to an event by extending BRM Gateway functionality. Any updates to data due to this opcode call in BRM are synchronized back into ECE by the update flow through External Manager (EM) Gateway.

For more information, see "BRM Gateway Request Processing Extension Points" in ECE Implementing Charging.

CDR Generator Enhancements

Several enhancements to the 5G Charging Function CDR Generator have been included in this release:

• Scalability: It is now possible to have more than one CDR Gateway and CDR Formatter per schema.

• Disaster Recovery: The system will now process any orphan records on a failed site as soon as the site becomes active, in parallel with live records. In addition, a configuration parameter has been added to set the duration after which, if no updates to the record occur, the record will be considered an orphan.

• Performance: Many performance improvements are provided in the release, including Oracle Database and NoSQL CDR data store improvements, optimized CPU utilization for object mappers, and improved metrics for CDR generation and the HTTP gateway.

• CDR Files: A new parameter has been introduced to limit the number of unrated CDRs included in a single file when CDR Formatter is writing out the CDRs.

  Previously, only the timestamps were included in the file name (in the format StartTimeStamp_EndTimeStamp.Extension). The file name format has been updated to ClusterName_StartTimeStamp_EndTimeStamp_SequenceNumber.Extension in order to uniquely distinguish CDR files generated by different clusters, and also to allow for proper sequencing of files so that the downstream systems can process them in the correct order.

For more information, see "Generating CDRs for External Systems" in ECE Implementing Charging.

Policy-Based Charging Notification Enhancements

Previously, all notifications for policy label breaches were sent in parallel. Now, policy label breaches for the same session are sent sequentially, with the later message being sent after an acknowledgment is received for the earlier notification. Notifications for breaches in different sessions are still sent in parallel.

For more information, see "How ECE Processes Policy Requests for Online Network Mediation System" in ECE Implementing Charging.

Usage Consumption Management Report Certification

The "TMF677 - Usage Consumption Management" report, added in Patch Set 3, has now received Open API certification from TM Forum.

New Features in ECE 12.0 Patch Set 6

ECE 12.0 Patch Set 6 includes the following enhancements:

• ECE SDK Includes New Rated Events Custom Plugin

• ECE Now Supports Temporary Credit Limits

• ECE Now Supports Loan Management

• ECE Can Now Record Failed Usage Requests

• HTTP Gateway Supports Charging Type-Based Configuration for CDR Generation

• ECE Supports 5G SA Nchf_OfflineOnlyCharging Service

• Enhancements for HTTP Gateway Registration with an NRF

• ECE REST API Supports New CHF Operation Types

• HTTP Gateway Now Determines Charging Type

ECE SDK Includes New Rated Events Custom Plugin

The ECE SDK package includes the new SampleRatedEventFormatterKafkaCustomPlugin.java sample plug-in, which writes fully rated events into a JSON file that is published to Apache Kafka topics.

For more information, see "ECE Sample Programs" in ECE Implementing Charging.

ECE Now Supports Temporary Credit Limits

When creating or modifying an account in BRM, you can add temporary credit limits to a customer's balance of minutes, US dollars (USD), or so on. For example, you could add a temporary credit limit of 400 USD that is valid from 1 November 2022 through 1 January 2023. Account Synchronization Manager now includes this information about temporary credit limits when synchronizing account updates between BRM and ECE.

The temporary credit limit is used by:

• The ECE Balance Query Java API when querying subscriber balances.

• ECE when performing credit breach checks.

To synchronize temporary credit limits between BRM and ECE, ensure that:

• Your event notification file (pin_notify_ifw_sync or pin_notify_kafka_sync) includes the /event/notification/billing/temp_limit and /event/billing/limit events.

  For more information, see the ECE_home/oceceserver/brm_config/pin_notify file.

• Your payload configuration file (payloadconfig_ifw_sync.xml or payloadconfig_kafka_sync.xml) includes the TemporaryLimit and TemporaryLimitExpiry business events and their event definitions.

  For more information, see the ECE_home/oceceserver/brm_config/payloadconfig_ece_sync.xml file.

ECE Now Supports Loan Management

You can grant loans in BRM to prepaid customers when their credit is insufficient for rating (dynamic loans), or when their balance falls below a configured threshold (automatic loans). Account Synchronization Manager now includes information about any loan balance items when synchronizing customer data between BRM and ECE.

Loan balance items are used by:

• The ECE Balance Query Java API when returning user balances.

• ECE when creating loan threshold notifications during the rating process.

To synchronize loan balance items between BRM and ECE, ensure that:

• Your event notification file (pin_notify_ifw_sync or pin_notify_kafka_sync) includes these events:

  • /event/billing/loan_credit

  • /event/billing/loan_debit

  • /event/billing/loan_fee

  • /event/billing/loan_grant

  • /event/billing/loan_late_fee

  • /event/billing/loan_pullback

  • /event/billing/loan_recovery

  • /event/billing/loan_recovery_reversal

  For more information, see the ECE_home/oceceserver/brm_config/pin_notify file.

• Your payload configuration file (payloadconfig_ifw_sync.xml or payloadconfig_kafka_sync.xml) includes these business events and their event definitions:

  • LoanCredit

  • LoanDebit

  • LoanFee

  • LoanGrant

  • LoanLateFee

  • LoanPullback

  • LoanRecovery

  • LoanRecoveryReversal

  For more information, see the ECE_home/oceceserver/brm_config/payloadconfig_ece_sync.xml file.

ECE Can Now Record Failed Usage Requests

ECE may occasionally fail to process usage requests. For example, a data usage request could fail because a customer has insufficient funds. You can now configure ECE to publish details about failed usage requests, such as the user ID and request payload, to the ECE failure topic in your Kafka server. Later on, you can reprocess the usage requests or view the failure details for analysis and reporting. For more information, see "Recording Failed ECE Usage Requests" in ECE Implementing Charging.

HTTP Gateway Supports Charging Type-Based Configuration for CDR Generation

By default, HTTP Gateway sends all 5G usage requests to ECE Server for online and offline charging. You can now configure HTTP Gateway to convert some of the usage requests into Call Detail Records (CDRs) based on the charging type. You can specify whether to route online charging requests, offline charging requests, or both to ECE Server for charging or to the CDR Gateway for generating CDRs.

For more information, see "About Using the HTTP Gateway" in ECE Implementing Charging.

ECE Supports 5G SA Nchf_OfflineOnlyCharging Service

ECE HTTP Gateway supports Nchf_OfflineOnlyCharging service operations for offline-only charging requests in 5G standalone (SA) deployments.

For more information, see ECE 5G CHF Protocol Implementation Conformance Statement.

Enhancements for HTTP Gateway Registration with an NRF

HTTP Gateway includes the following enhancements for NRF registration:

• You can now register one HTTP Gateway server with multiple NRF endpoints.

• You can now register multiple HTTP Gateways with one NRF endpoint.

For more information, see "Configuring Registration Details for the HTTP Gateway Server" in ECE Implementing Charging.

ECE REST API Supports New CHF Operation Types

The ECE REST API now supports the following CHF operation types:

• Creating an initial request for an offline-only charging session

• Updating a request for an offline-only charging session

• Ending an offline-only charging session

In addition, all ECE REST API endpoints are now documented in this new book: REST API for Elastic Charging Engine.

HTTP Gateway Now Determines Charging Type

HTTP Gateway now determines whether a usage request requires online or offline charging. It does so as follows:

• For INITIATE requests, based on the multipleUnitUsage block. If the block is present, the request needs online charging. If the block is missing, the request needs offline charging.

• For UPDATE requests, based on the value of the quotaManagementIndicator field in the request. If the value is set to ONLINE_CHARGING, the request needs online charging. If the field is missing or the value is set to OFFLINE_CHARGING, the request needs offline charging.

• For TERMINATE requests, based on the value of the quotaManagementIndicator field in the request. If the value is set to ONLINE_CHARGING, the request needs online charging. If the field is missing or the value is set to OFFLINE_CHARGING, the request needs offline charging.

For more information, see "Connecting ECE to a 5G Client" in ECE Implementing Charging.

New Features in ECE 12.0 Patch Set 5

ECE 12.0 Patch Set 5 includes the following enhancements:

• ECE Now Generates CDRs for Unrated 5G Usage Events

• SSL Now Between ECE and Persistence Database

• ECE Cloud Native Supports Subscriber Tracing

ECE Now Generates CDRs for Unrated 5G Usage Events

ECE can now generate call detail records (CDRs) for unrated 5G usage events. You need to enable generating CDRs in the HTTP Gateway. You can configure ECE to publish CDRs as files or send them to a Kafka messaging service.

For more information, see "Generating CDRs" in ECE Implementing Charging and "CHF-CDR Format" in ECE 5G CHF Protocol Implementation Conformance Statement.

SSL Now Between ECE and Persistence Database

The ECE installer now enables SSL communication between ECE and the Oracle persistence database by default. Also, the installer's ECE Persistence Database Details screen includes new fields.

For more information, see "Installing ECE by Using the GUI Installation" in ECE Installation Guide.

ECE Cloud Native Supports Subscriber Tracing

In ECE cloud native, you can now selectively trace your subscribers' sessions based on one or more subscriber IDs. You can also specify to trace and log selective functions, such as alterations (discounts), charges, and distributions (charge sharing), for each subscriber.

This feature introduces the following keys to the oc-cn-ece-helm-chart/values.yaml file:

subscriberTrace:
   filePath: "/home/charging/config/subscriber-trace.xml"
   logMaxSubscribers: "100"
   logMaxSubscriberSessions: "24"
   logExpiryWaitTime: "1"
   logCleanupInterval: "2"
   logLevel: "DEBUG"
   subscriberList: ""

For more information, see "Configuring Subscriber-Based Tracing for ECE Services" in BRM Cloud Native System Administrator's Guide.

Rated Event Formatter Changes for Rated Event Manager

The Rated Event Formatter (RE Formatter) was changed to support the Rated Event Manager feature. A new RE Formatter plug-in, RatedEventManagerCdrPlugin, is now available for sending rated events to BRM. For more information about configuring RE Formatter for the new plug-in, see "Configuring RE Formatter" in BRM Loading Rated Events.

For more information about the Rated Event Manager feature, see "New Method for Loading Rated Events from ECE into BRM".

New Features in ECE 12.0 Patch Set 4

ECE 12.0 Patch Set 4 includes the following enhancements:

• It now supports the active-active disaster recovery configuration. In this configuration, ECE can be deployed with multiple clusters across multiple geographic sites and each site performs active rating simultaneously. All sites utilize resources and continue rating operations after a site failure.

  For more information, see "Configuring ECE for Disaster Recovery" in BRM System Administrator's Guide.

• It can now spawn multiple BRM gateway processes for Kafka-enabled ECE notification queues, with each process catering to a fixed set of customer notification handling.

  For more information, see "Configuring Multiple BRM Gateway Instances" in BRM System Administrator's Guide.

• ECE persistence databases can now support multischema systems.

  For more information, see "Enabling Multischema Support in Persistence Databases" in BRM System Administrator's Guide.

• ECE now supports 5G Charging Function (CHF) communication through a Services Communication Proxy (SCP).

  For more information, see "Configuring Communication through SCP" in ECE Implementing Charging.

• The HTTP Gateway now supports secure communication through SSL. You configure the HTTP Gateway to use SSL when you install ECE.

  See "Installing ECE" and "Post-Installation Tasks for an ECE Integrated Installation" in ECE Installation Guide.

• ECE now supports the latest 3GPP standards.

  For more information, see ECE 5G CHF Protocol Implementation Conformance Statement and ECE Diameter Gateway Protocol Implementation Conformance Statement.

• HTTP Gateway now supports four request processing extension points:

  • RequestReceived extension. The role of the RequestReceived extension is to manipulate the charging data before the usage request is processed by HTTP Gateway and to provide an immediate response that bypasses the online charging system (OCS) completely.

  • PreOCS extension. The role of the PreOCS extension is to manipulate the mapped ECE usage request payload to perform enrichments that are not possible in the RequestReceived extension.

  • PostOCS extension. The role of the PostOCS extension is to manipulate the ECE usage request before the HTTP Gateway response is returned to the 5G client.

  • PostOCSBalanceQuery extension. The role of the PostOCSBalanceQuery extension is to manipulate the ECE usage response before the HTTP Gateway response is returned to the 5G client.

  For more information, see "Customizing Rating" in ECE Implementing Charging.

New Features in ECE 12.0 Patch Set 3

ECE 12.0 Patch Set 3 includes the following enhancements:

• Apache Kafka Now Supported for Notification Handling

• Support for Configuring Diameter Gateway to Bypass Rating During ECE Downtime

• Diameter Gateway Now Supports the 5G Non-Standalone Architecture

• ECE Now Supports 5G CHF Protocol

• New HTTP Gateway for 5G Networks

• Active-Hot Standby and Segmented Active-Active DR Support Persistence-Enabled Deployments

Apache Kafka Now Supported for Notification Handling

In addition to using Oracle WebLogic queues, you can now use Apache Kafka topics for notification handling. The following gateways support Kafka topics:

• HTTP Gateway supports only Apache Kafka topics. See "Connecting ECE to a 5G Client" in ECE Implementing Charging.

• Diameter Gateway supports both WebLogic queues and Kafka topics. See "Connecting ECE to a Diameter Client" in ECE Implementing Charging.

• BRM Gateway supports both WebLogic queues and Kafka topics. See "Configuring BRM Gateway" in BRM System Administrator's Guide.

Support for Configuring Diameter Gateway to Bypass Rating During ECE Downtime

During a planned maintenance or an unplanned downtime of ECE nodes, you can configure Diameter Gateway to continue receiving CCRs and responding to the service network to ensure continued service delivery. When Diameter Gateway is configured to bypass rating, it persists the CCRs to the Oracle NoSQL database. Later, when ECE is restored, you can replay the persisted CCRs to the ECE charging servers for rating and updating balance impacts. With this functionality, services can be rendered to the subscribers without any interruption. For more details, see ECE Implementing Charging.

Diameter Gateway Now Supports the 5G Non-Standalone Architecture

You can now use Diameter Gateway to integrate ECE with 5G networks. For more information, see "Connecting ECE to a Diameter Client" in ECE Implementing Charging.

It also has been enhanced to support Extended Bandwidth AVPs and the new NR Secondary RAT AVPs. For more information, see "Diameter Gateway Support for 5G Non-Standalone Architecture" in ECE Diameter Gateway Protocol Implementation Conformance Statement.

ECE Now Supports 5G CHF Protocol

ECE now supports the 5G Charging Function (CHF) protocol. It supports Nchf interfaces for converged charging and spending limit control for policy. For more information, see ECE 5G CHF Protocol Implementation Conformance Statement.

New HTTP Gateway for 5G Networks

ECE now supports online charging for 5G networks through the new HTTP Gateway. HTTP Gateway integrates ECE with your 5G clients and performs the following functions:

• Receives ECE REST API requests from 5G clients and then translates them into batch request server (BRS) requests.

• Submits BRS requests to ECE servers.

• Receives responses from ECE servers and then translates them into REST API responses.

• Responds to the 5G clients.

• Consumes notifications from the ECE Notification topic and then notifies the 5G clients by making a REST call to the URL stored in the system.

After the HTTP Gateway is set up, your 5G clients submit requests to ECE by using these new ECE REST API endpoints:

• Create a Spending Limit Control Subscription Endpoint

• Update a Spending Limit Control Subscription Endpoint

• End a Spending Limit Control Subscription Endpoint

• Create an Nchf Converged Charging Service Endpoint

• Update an Nchf Converged Charging Service Endpoint

• End an Nchf Converged Charging Service Endpoint

• Get Balances Endpoint

• Get a Usage Consumption Report Endpoint

For more information, see "Connecting ECE to a 5G Client" in ECE Implementing Charging.

Active-Hot Standby and Segmented Active-Active DR Support Persistence-Enabled Deployments

In previous releases, active-hot standby and segmented active-active configurations were not supported when ECE persistence was enabled.

With this enhancement, you can now configure active-hot standby and segmented active-active disaster recovery (DR) configurations when ECE persistence is enabled. Earlier, the RatedEventFormatter instance was running in both primary and secondary sites, but now there is only one RatedEventFormatter instance running among the sites.

For more information, see "Configuring ECE for Disaster Recovery" in BRM System Administrator’s Guide.

New Features in ECE 12.0 Patch Set 2

ECE 12.0 Patch Set 2 includes the following enhancements:

• New Tools to Monitor ECE

• Duplicate Check Enhancement in ECE

• Charging Operation Type Can be Configured for Expired Active Session Cleanup

• ECE Now Generates POID for Events

• New Tool for Querying ECE Cache Data

• ECE Now Supports Wildcard in Item Type Selectors

• Support for Persisting BRS Configuration Data

• Rated Event Partition in the ECE Persistent Database

New Tools to Monitor ECE

You can now use the following new monitoring tools to collect the ECE data for monitoring the ECE cache size, rating performance, latency, and so on:

• ece_cluster_overview. Collects the ECE cluster summary. Use this tool to display the summary in a text or comma-separated value (CSV) format in Linux.
• ece_full_gc. Collects the complete Garbage Collection (GC) debug logs for any ECE grid member or a specific java application by using the process ID.
• ece_log. Collects the specified log-level data. You can also use this script to set log levels. See "Modifying Log Levels by Using Scripts" for more information.
• ece_metrics_collector. Runs the ECE Metrics Collector tool. This tool starts a light-weight HTTP Server to collect the ECE metrics data and present the data to third-party monitoring tools. Use this tool to view the ECE metrics using open-source tools, such as Grafana and Prometheus. See "Monitoring ECE Using Grafana and Prometheus" for more information.
• jvm_info. Collects the JVM data on a periodic basis. The JVM data includes memory usage, central processing unit (CPU) usage, open files, and so on.
• ece_brs_info. Connects to a specific Java process in which the batch request service (BRS) is running and collects the data such as the number of requests processed and latency over time.
• ece_queues.sh. Collects Oracle Communications Pricing Design Center (PDC) metrics from WebLogic Server, such as the size of the pricing data, the number of messages in the queue, and so on.
• cohql_file_processor. Starts the query tool (query.sh) and allows it to remain connected to the ECE cluster. Handles the queries on ECE Coherence caches in the non-interactive or interactive mode. Use this script to query ECE data instead of running the query.sh script frequently.
• count_ref_files. Counts the number of Rated Event Formatter output files and their volume. You can use this script to estimate the volume of data that can be processed by Rated Event Formatter and the storage space required to store the rated event data in the event of failure.
• get_log_slices. Collects data from the specified set of ECE log files for the given time period to perform error analysis. You can use this script to collect the detailed information regarding a specific problem.
• parse_ece_chronicler. Parses the ECE batch request service (BRS) chronicler metrics to aggregate the data into configurable time periods. This script provides a general view of the ECE rating performance.
• parse_ecedc. Parses the Oracle Communications Offline Mediation Controller Elastic Charging Engine (ECE) Distribution Cartridge (DC) log files to collect the data for monitoring the rating performance and throughput of ECE DC deployed in Offline Mediation Controller; for example, batch processing time, data read time, batch submission time, timeouts, and so on.
• parse_ece_pricing_load. Parses the INFO-level log data in the Pricing Updater log files. This script provides the information required to monitor the Pricing Updater performance, such as the volume of data loaded, number of items processed, and the time taken to process the pricing data.
• parse_ece_start. Parses the INFO-level data in the ECE server log files generated during initial data loading. This script enables you to identify the performance problems in the initial data loading and startup process.

You can use these tools along with Coherence reports and ECE Monitoring Agent to monitor your ECE system. By default, the ECE monitoring tools are installed in the ECE_home/tools directory. If you change the location of these tools, run the following command in Bash shell, which sets the ECE_TOOLS_DIR and PATH environment variables to point to the new location:

export ECE_TOOLS_DIR=path_to_tools
export PATH=path_to_tools

For more information on these tools, see the following:

• Running ECE Monitoring Tools
• Customizing JMX-Based Tool Reports
• Monitoring ECE Cluster
• Monitoring ECE Using Grafana and Prometheus
• Setting Log Levels by Using Scripts
• Using cohql_file_processor for Query

Running ECE Monitoring Tools

You can run all these ECE monitoring tools with the default settings by using the following wrapper utilities:

Note:

You do not have to run each monitoring tool individually.

• ece_util_launcher.sh. Runs the following Groovy/JMX-based tools by using the default JMX credentials: ece_cluster_overview, ece_metrics_collector, ece_brs_info, jvm_info, ece_log, and ece_full_gc.
• perl_util_launcher.sh. Runs the following Perl-based tools: cohql_file_processor, count_ref_files, get_log_slices, parse_ece_chronicler, parse_ecedc, parse_ece_pricing_load, and parse_ece_start.

You can also create symbolic link between tools to simplify the start up process. You can create symbolic links by running the following command in the ECE_home/tools directory or the directory in which the tools are available:

./mklinks all 

For example, this creates symbolic links from ece_cluster_overview and start_ece_metrics_collector to the ece_util_launcher.sh and perl_util_launcher.sh utilities. This enables ece_cluster_overview and start_ece_metrics_collector to run the ece_util_launcher.sh and perl_util_launcher.sh utilities internally without any intervention.

Later, if you want to remove the symbolic link, run the following command:

./mklinks -r all 

For the syntax and parameters used with the ECE monitoring tools, run the help (-h) command. For example:

ece_cluster_overview.groovy -h

Following is an example for running ece_cluster_overview to collect the cluster summary:

./ece_cluster_overview -e /scratch/ri-user-1/opt/OracleCommunications/ECE/ECE/oceceserver/config/eceTopology.conf -U controlRole -P 'R&D' -c

Customizing JMX-Based Tool Reports

The JMX-based tools generate reports in the tabular format. The output for these tools are defined in the ECE_home/tools/TableDefinition.properties file. You can customize the reports generated by these tools by updating this file or by using the filters when you run the tool command. For example, you can define the ECE attributes to be included in the reports and also change the width and format of the table based on your requirements.

For more information, see the ECE_home/tools/TableDefinition.properties file.

Monitoring ECE Cluster

The ece_cluster_overview tool connects to an ECE cluster through a JMX-enabled node and queries data about the cluster members and the state of ECE caches. This tool can be used to retrieve data on periodic basis in the text or comma-separated value (CSV) format.

Note:

When using this tool to query data from Coherence clusters, which have a large number of members, retrieving data from remote members frequently might affect overall cluster performance.

Oracle recommends to review the management refresh expiry (tangosol.coherence.management.refresh.expiry value in the ECE_home/oceceserver/config/ece.properties file) for the cluster you are monitoring and adjust the interval to reduce the frequency of data retrieval from remote members. However, the data collected may not be completely accurate.

You can also use this tool to query batch request service (BRS) data for charging clients, such as Diameter Gateway and Oracle Communications Offline Mediation Controller, and retrieve general JVM information, such as heap usage and CPU usage.

You can set the parameters used for collecting ECE cluster details by running the ece_cluster_overview.groovy script. For information on the parameters, default values, and examples, run the help (-h) command.

Monitoring ECE Using Grafana and Prometheus

You use the ECE Metrics Collector (ece_metrics_collector) tool that runs a light-weight HTTP server to collect system metrics from ECE on demand and convert the data into a format that can be processed by third-party monitoring tools. This tool provides different metric groups to collect different ECE metrics and present them to the third-party monitoring tools at different intervals.

You can use ECE Metrics Collector with the following third-party monitoring tools for monitoring ECE:

• Prometheus. An open-source monitoring tool which:

  • Stores system metrics in a time-series database.

  • Provides flexible queries to retrieve data for analysis.

  • Supports real-time alerting.

• Grafana. An open-source visualization tool that presents graphical dashboards by retrieving data from Prometheus. This tool contains multiple presentation formats and also provides an SDK for customizing dashboards.

The ECE monitoring process that uses Grafana and Prometheus is as follows:

1. ECE Metrics Collector collects the ECE metrics data in its different groups and presents them in the text-based exposition format to Prometheus by using HTTP endpoints.

   To configure, start, or stop ECE Metrics Collector, see "Starting and Stopping ECE Metrics Collector".

2. The Prometheus monitoring server connects to each HTTP endpoint in ECE Metrics Collector at different intervals periodically and queries the metrics data. For example, Prometheus collects the ECE cache metrics every 30 seconds and the usage rating metrics every 10 seconds.

   To configure Prometheus for querying ECE metrics data, see the Prometheus documentation.

3. The Prometheus monitoring server stores the metrics data in a time-series database and runs rules on the metrics data to aggregate the data and generate alerts or notifications based on the configuration.

4. Grafana retrieves the ECE metrics data from the Prometheus monitoring server and presents different graphical dashboards for different users. The dashboard contains one or more panels which displays the metrics for ECE.

   To configure Grafana for displaying the ECE metrics data, see the Grafana documentation.

For information on the metrics collected by ECE Metrics Collector, see "Collecting ECE Metrics Data".

Starting and Stopping ECE Metrics Collector

Before starting ECE Metrics Collector, you can set the ECE Metrics Collector parameters used for collecting ECE metrics by running the ece_metrics_collector.groovy script. For information on the parameters, default values, and examples, run the help (-h) command.

To start or stop ECE Metrics Collector:

1. On the driver machine, change to the ECE_home/oceceserver/bin directory.

2. Start ECC:

   ./ecc
   

3. Do one of the following:

   • To start ECE Metrics Collector, run the following command:

     start_ece_metrics_collector
     

     The HTTP server is started and it listens on the specified port. The default port is 5050.

   • To stop ECE Metrics Collector, run the following command:

     stop_ece_metrics_collector

     The HTTP server is stopped.

You can verify if ECE Metrics Collector is running by connecting to the HTTP server using the following URL:

http://your_server:5050/metrics

Collecting ECE Metrics Data

ECE Metrics Collector collects metrics in different groups to produce data for monitoring ECE. See the following for more information:

• ECE_GRID_MEMBER_METRICS
• BRS_CLIENT_METRICS
• CACHE_METRICS
• JVM_METRICS
• SESSION_AND_EVENT_METRICS
• SERVICE_METRICS
• FEDERATION_METRICS

ECE_GRID_MEMBER_METRICS

The ECE_GRID_MEMBER_METRICS group contains the ece_grid_members metric which provides information about the running status of nodes in the topology. ECE Metrics Collector compares the information retrieved from the Coherence grid with the information in the ECE topology file to retrieve the list of all nodes in the grid with their status.

Table 3-1 describes the metrics that ECE Metrics Collector retrieves for grid members.

Table 3-1 Grid Member Metrics

Metric Name	Type	Description
ece_grid_members	Gauge	Contains the state of all the members in the ECE grid. The metric shows 1 for nodes that are running and 0 for nodes that are down.

For example, if the diameterGateway1 and ratedEventFormatter1 nodes are running and the query and Customer Loader utilities are down, the ece_grid_members metric that is retrieved is as follows:

ece_grid_members{id="21", pid="2313", machine="abc.us.example.com", nodeName="diameterGateway1", role="diameterGateway"} 1
ece_grid_members{id="22", pid="3239", machine="abc.us.example.com", nodeName="ratedEventFormatter1", role="ratedEventFormatter"} 1
ece_grid_members{id="0", pid="0", machine="abc.us.example.com", nodeName="CohqlShell", role="query"} 0
ece_grid_members{id="0", pid="0", machine="abc.us.example.com", nodeName="customerLoader", role="customerLoader"} 0

BRS_CLIENT_METRICS

The BRS_CLIENT_METRICS group contains the metrics for tracking throughput and latency of the charging clients, which use batch request service (BRS), such as Diameter Gateway, Offline Mediation Controller, or any custom charging clients.

Table 3-2 describes the metrics that ECE Metrics Collector retrieves for ECE charging clients.

Table 3-2 ECE Charging Client Metrics

Metric Name	Type	Description
ece_brs_total_task_count	counter	Tracks the total number of requests processed by a charging client. This metric shows the number of requests accepted, processed, timed-out, or rejected by the ECE component. You can use this in Prometheus to track the approximate processing rate over time, aggregate over all client applications, and so on.
ece_brs_current_task_count	gauge	Tracks the number of requests processed by a charging client in the current scrape interval. This metric shows the number of requests accepted, processed, timed-out, or rejected by the ECE component since the last query. You can use this metric to determine the delta.
ece_brs_current_throughput_per_second	gauge	Tracks the throughput of a charging client for each charging operation type in the current query interval. For example, if the data is collected every 10 seconds, the current value is considered as the throughput for that specific operation type since the last query. This metric provides the throughput information for the following operation types: Initiate, Update, Terminate, Cancel, Price_Enquiry, Balance_Query, Debit_Amount, Debit_Unit, Refund_Amount, and Refund_Unit. You can use this metric to gauge the overall system throughput for each type of usage requests.
ece_brs_current_latency_milliseconds	gauge	Tracks the current operation latency(in milliseconds) for a charging client in the current scrape interval. This metric contains the BRS statistics tracked using the charging.brsConfigurations MBean attributes. This configuration tracks minimum, maximum, and average latency for an operation type since the last query. The maximum window size for the collecting this data is 30 seconds, so the query has to be run within every 30 seconds. This metric provides the latency information for the following operation types: Initiate, Update, Terminate, Cancel, Price_Enquiry, Balance_Query, Debit_Amount, Debit_Unit, Refund_Amount, Refund_Unit, and Spending_Limit_Report.
ece_total_requests_by_result_code	counter	Tracks the total requests processed by Diameter Gateway by using the result code. This metric shows the total number of Gy and Sy requests processed by a Diameter Gateway instance for each result code.
ece_current_request_rate_milliseconds_by_result_code	gauge	Tracks the current processing rate of Diameter Gateway (in milliseconds) by using the Diameter protocols (Sy/Gy) and the result code. This metric provides the current processing rate for Gy and Sy operations handled by a Diameter Gateway instance for each result code.
ece_session_metrics	gauge	Tracks the overall number of Gy sessions that are open or close for an ECE component, such as Diameter Gateway and ECE server nodes. This metric shows the total number of open and closed sessions for each Diameter Gateway instance and rating group. Note: By default, ECE Metrics Collector aggregates ECE server metrics by rating group at the system-level. You can refine this metric to include the information per ECS node by running ECE Metrics Collector with the -mp parameter.

Example

The following are the examples for the metrics in the BRS_CLIENT_METRICS group:

• ece_brs_total_task_count. This example shows that diameterGateway1 has processed 340845 requests without error and rejected or timed-out 0 requests:

  ece_brs_total_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", type="accepted"} 340845
  ece_brs_total_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", type="processed"} 340845
  ece_brs_total_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", type="rejected"} 0
  ece_brs_total_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", type="timeout"} 0

• ece_brs_current_task_count. This example shows that diameterGateway1 has accepted and processed 54 requests in the current scrape interval:

  ece_brs_current_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="pending"} 0
  ece_brs_current_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="accepted"} 54
  ece_brs_current_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="processed"} 54
  ece_brs_current_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="timeout"} 0
  ece_brs_current_task_count{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="rejected"} 0

• ece_brs_current_throughput_per_second. This example shows that diameterGateway1 has processed Initiate, Update, and Terminate (IUT) for voice calls at a rate 0f 30 calls per second:

  ece_brs_current_throughput_per_second{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" opType="initiate"} 26
  ece_brs_current_throughput_per_second{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" opType="update"} 27
  ece_brs_current_throughput_per_second{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" opType="terminate"} 29

• ece_brs_current_latency_milliseconds. This example shows the minimum, maximum, and average latency in milliseconds for Initiate and Update requests:

  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="average", opType="initiate"} 5.9
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="min", opType="initiate"} 4.6
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="max", opType="initiate"} 7.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="average", opType="update"} 0.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="min", opType="update"} 0.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="max", opType="update"} 0.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="average", opType="terminate"} 0.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="min", opType="terminate"} 0.0
  ece_brs_current_latency_milliseconds{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com" type="max", opType="terminate"} 0.0

• ece_total_requests_by_result_code. This example shows the total number of Gy and Sy requests processed by diameterGateway1 for the result code 2001:

  ece_total_requests_by_result_code{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", protocol="Gy", resultCode="2001"} 237114
  ece_total_requests_by_result_code{nodeName="diameterGateway1", role="diameterGateway", machine="abc.us.example.com", protocol="Sy", resultCode="2001"} 45883

• ece_current_request_rate_milliseconds_by_result_code. This example shows the current processing rate in milliseconds for Gy and Sy requests processed by diameterGateway1 for the result code 2001:

  ece_current_request_rate_milliseconds_by_result_code{instance="abc.us.example.com:5050",job="ece_brs",machine="abc.us.example.com",nodeName="diameterGateway1",protocol="Gy",resultCode="2001",role="diameterGateway"} 140.6
  ece_current_request_rate_milliseconds_by_result_code{instance="abc.us.example.com:5050",job="ece_brs",machine="abc.us.example.com",nodeName="diameterGateway1",protocol="Sy",resultCode="2001",role="diameterGateway"} 10.4

• ece_session_metrics. This example shows the total number of open and closed sessions for diameterGateway1 and the rating group 10:

  ece_session_metrics{nodeName="diameterGateway1",rating_group="*",role="diameterGateway",type="Close"} 8140.0
  ece_session_metrics{nodeName="diameterGateway1",rating_group="*",role="diameterGateway",type="Open"} 11750.0
  ece_session_metrics{rating_group="10",type="Close"} 7639.0
  ece_session_metrics{rating_group="10",type="Open"} 10645.0

CACHE_METRICS

The CACHE_METRICS group contains metrics for ECE caches. You can use this metric to track the overall growth rate of certain caches along with other metrics.

Table 3-3 describes the metrics that ECE Metrics Collector retrieves for ECE caches.

Table 3-3 ECE Cache Metrics

Metric Name	Type	Description
ece_cache_avg_size	Gauge	Contains the average size of objects (in bytes) in the ECE caches.
ece_cache_entries	Gauge	Contains the total number of entries present in an ECE cache on an ECE node at the time the query is run.
ece_cache_gets	Counter	Contains the total number of gets performed on an ECE cache.
ece_cache_puts	Counter	Contains the total number of updates made to an ECE cache. It is the mirror of the ece_cache_gets metric.
ece_cache_store_millis	Counter	Contains the cumulative time spent (in milliseconds) on the cache-store operations for an ECE cache. This metric is applicable only for the following caches: RatedEvent and ServiceContext.
ece_cache_store_writes	Counter	Contains the number of updates attempted on a cache store, such as RatedEvent and ServiceContext.
ece_cache_total_size	Gauge	Contains the total size of an ECE cache (by default in megabytes). You can also define the unit of the metric by using the -u parameter.

Examples

The following are the examples for the metrics in the CACHE_METRICS group:

• ece_cache_avg_size. This example shows the average size in bytes for the ActiveSession, Customer, and RatedEvent caches:

  ece_cache_avg_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ActiveSession"} 1968
  ece_cache_avg_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Customer"} 3344
  ece_cache_avg_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="RatedEvent"} 2557

• ece_cache_entries. This example shows the total number of entries in the ActiveSession, Customer, and RatedEvent caches:

  ece_cache_entries{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ActiveSession"} 3068
  ece_cache_entries{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Customer"} 1117
  ece_cache_entries{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="RatedEvent"} 836

• ece_cache_gets. This example shows the total number of gets for the ActiveSession and Customer caches:

  ece_cache_gets{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ActiveSession", tier="back"} 13906794
  ece_cache_gets{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Customer", tier="back"} 8720803

• ece_cache_store_millis. This example shows that 244.931 milliseconds and 5.019 milliseconds are spent on the RatedEvent and ServiceContext cache updates respectively:

  ece_cache_store_millis{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="RatedEvent", tier="back"} 244.931
  ece_cache_store_millis{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ServiceContext", tier="back"} 5.019

• ece_cache_store_writes. This example shows the number of updates attempted for the RatedEvent and ServiceContext caches on 2 ECE charging server nodes:

  ece_cache_store_writes{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="RatedEvent", tier="back"} 3182
  ece_cache_store_writes{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ServiceContext", tier="back"} 1
  ece_cache_store_writes{nodeName="ecs2", service="BRMFederatedCache", machine="abc.us.example.com", cache="RatedEvent", tier="back"} 2281
  ece_cache_store_writes{nodeName="ecs2", service="BRMFederatedCache", machine="abc.us.example.com", cache="ServiceContext", tier="back"} 0

• ece_cache_total_size. This example shows the size of the ActiveSession, Balance, and Customer cache in megabytes:

  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ActiveSession", unit="mb"} 1.73
  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Balance", unit="mb"} 0.52
  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Customer", unit="mb"} 1.17

JVM_METRICS

The JVM_METRICS group contains standard metrics about the central processing unit (CPU) and memory utilization of JVMs, which are members of the ECE grid.

Table 3-4 describes the metrics that ECE Metrics Collector retrieves for JVMs.

Table 3-4 JVM Metrics

Metric Name	Type	Description
ece_jvm_memory	Gauge	Contains the Java heap usage information (by default in megabytes) for each ECE charging server (ECS) node. This metric contains a lot of memory-related attributes of JVMs, which you can use to get a detailed view of the memory usage for different ECE components. Note: The off-heap memory data is tracked only if the -Mparameter is not used while running the ECE Metrics Collector.
ece_jvm_process_cpu_percent	Gauge	Contains the CPU usage information (in percentage) for each ECE component on the server. This data is collected from the corresponding MBean attributes by JVMs.
ece_jvm_process_file_descriptors	Gauge	Contains the total number of file-descriptors currently available for an ECE component and the descriptors that are in use for that ECE component.
ece_jvm_server_cpu_percent	Gauge	Contains the CPU load information (in percentage) for each system in the cluster. These statistics are based on the average data collected from all the ECE grid members running on a server.
ece_jvm_server_load_average	Gauge	Contains the system load average (the number of items waiting in the CPU run-queue) information for each machine in the cluster. These statistics are based on the average data collected from all the ECE grid members running on a server.
ece_jvm_server_swap	Gauge	Contains system swap usage information (by default in megabytes) for each system in the cluster. These statistics are based on the average data collected from all the ECE grid members running on a server.

Example

The following are the examples for the metrics in the JVM_METRICS group:

• ece_jvm_memory. This example shows the values in megabytes for a single ECS node:

  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="heap", class="committed"} 1536.00
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="heap", class="init"} 1536.00
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="heap", class="percent"} 42.56
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="heap", class="used"} 653.79
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="mapped", class="count"} 0.0
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="mapped", class="memory_used"} 0.00
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="mapped", class="total_capacity"} 0.00
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="off_heap", class="committed"} 173.06
  ece_jvm_memory{nodeName="ecs1", role="server", machine="abc.us.example.com", type="off_heap", class="init"} 2.44
  

• ece_jvm_process_cpu. This example shows that the 3 ECS nodes are using 6.41%, 6.31%, and 6.04% of CPU respectively:

  ece_jvm_process_cpu_percent{nodeName="ecs1", role="server", machine="abc.us.example.com"} 6.409681757059616
  ece_jvm_process_cpu_percent{nodeName="ecs2", role="server", machine="abc.us.example.com"} 6.308724832214765
  ece_jvm_process_cpu_percent{nodeName="ecs3", role="server", machine="abc.us.example.com"} 6.037567084078712
  

• ece_jvm_process_file_descriptors. This example shows that BRM Gateway is using 144 descriptors and there are maximum of 65536 descriptors available for BRM Gateway:

  ece_jvm_process_file_descriptors{nodeName="brmGateway", role="brmGateway", machine="abc.us.example.com", type="max"} 65536
  ece_jvm_process_file_descriptors{nodeName="brmGateway", role="brmGateway", machine="abc.us.example.com", type="open"} 144

• ece_jvm_server_cpu_percent. The following example shows that the abc server is using approximately 31% of CPU:

  ece_jvm_server_cpu_percent{machine="abc.us.example.com"} 30.95

• ece_jvm_server_load_average. This example shows the abc server's load average is 3.08 at the time of query (if it is an 8 CPU sever, this indicates that the server is not heavily loaded):

  ece_jvm_server_load_average{machine="abc.us.example.com"} 3.08

• ece_jvm_server_swap. This example shows that abc server has 4096 megabytes of swap in a total of 4 gigabytes and has about 2.8 gigabytes free space:

  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="ActiveSession", unit="mb"} 1.73
  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Balance", unit="mb"} 0.52
  ece_cache_total_size{nodeName="ecs1", service="BRMFederatedCache", machine="abc.us.example.com", cache="Customer", unit="mb"} 1.17

SESSION_AND_EVENT_METRICS

The SESSION_AND_EVENT_METRICS group contains metrics on ECE server sessions and rated events processed by these sessions. See "BRS_CLIENT_METRICS" for the Diameter session metrics.

Table 3-5 describes the metrics that ECE Metrics Collector retrieves for ECE server sessions and rated events.

Table 3-5 ECE Server Sessions and Rated Events Metrics

Metric Name	Type	Description
ece_rated_events_current_by_node	Gauge	Contains the total number of RatedEvent objects extracted from the Oracle NoSQL database since the last query. Note: In single-schema systems, only one Rated Event Formatter instance is tracked for collecting metrics.
ece_rated_events_total_by_node	Counter	Contains the total number of rated events processed by each ECS node. This includes both the rated events that are stored in the RatedEvent cache and the Oracle NoSQL database. This data is tracked only when you run ECE Metrics Collector with the -mp parameter.
ece_rated_events_total	Counter	Tracks the processing of rated events by ECE server nodes.
ece_session_metrics	Counter	Contains the total number of sessions opened or closed on each ECE server node by rating group. This metric can be refined when you run ECE Metrics Collector with the -mp parameter.

Example

The following are the examples for the metrics in the SESSION_AND_EVENT_METRICS group:

• ece_rated_events_current_by_node. This example shows that 4477 events were extracted in the last scrape interval:

  ece_rated_events_current_by_node{nodeName="ratedEventFormatter1", machine="abc.us.example.com", role="ratedEventFormatter", type="extracted"} 4477.0

• ece_rated_events_total_by_node. This example shows the rated events extracted and processed by ECS node ecs1. There are 158421 events stored in the RatedEvent cache, of which 158290 have been stored in the Oracle NoSQL database:

  ece_rated_events_total_by_node{nodeName="ecs3", machine="abc.us.example.com", role="server", type="inserted"} 177644.0
  ece_rated_events_total_by_node{nodeName="ecs3", machine="abc.us.example.com", role="server", type="pushed"} 177502.0
  ece_rated_events_total_by_node{nodeName="ratedEventFormatter1", machine="abc.us.example.com", role="ratedEventFormatter", type="extracted"} 503713.0

• ece_rated_events_total. This example shows the number of rated events extracted from Rated Event Formatter and the number of rated events stored in the RatedEvent cache and Oracle NoSQL database respectively:

  ece_rated_events_total{type="extracted"} 503713.0
  ece_rated_events_total{type="inserted"} 511258.0
  ece_rated_events_total{type="pushed"} 510842.0

• ece_session_metrics. This example shows the number of sessions opened and closed for rating group 10 for the ECE server node ecs3 (if -mp parameter is used):

  ece_session_metrics{nodeName="ecs3", machine="abc.us.example.com", role="server", type="Close", rating_group="10"} 174647.0
  ece_session_metrics{nodeName="ecs3", machine="abc.us.example.com", role="server", type="Open", rating_group="10"} 184504.0
  ece_session_metrics{type="Close", rating_group="10"} 506267.0
  ece_session_metrics{type="Open", rating_group="10"} 534832.0

SERVICE_METRICS

The SERVICE_METRICS group contains metrics for the Oracle Coherence cache services. You can use this data to monitor the server latency and load (per node) and the backlogs which may accumulate on the ECE nodes. You can configure the frequency to reset the Coherence service statistics by running the ECE Metrics Collector with the -r <duration> parameter.

Table 3-6 describes the metrics that ECE Metrics Collector retrieves for Oracle Coherence cache services.

Table 3-6 Oracle Coherence Cache Services Metrics

Metric Name	Type	Description
ece_service_avg_thread_count	Gauge	Contains the average active thread count as determined by Coherence.
ece_service_endangered_partitions	Gauge	Contains the total number of endangered partitions on an ECE service. The metric value is 0 if the partition is not endangered and a number greater than zero if any node has failed.
ece_service_ha_status	Gauge	Contains a numeric representation of the Coherence high-availability status. The metric value is 0 for ENDANGERED node, 1 for NODE-SAFE, 2 for RACK-SAFE, 3 for MACHINE-SAFE, and 4 for SITE-SAFE. Typically, nodes can be endangered temporarily during rebalancing operations when the nodes are added or removed.
ece_service_request_avg_duration	Gauge	Contains the average server-side request latency in milliseconds.
ece_service_request_count	Counter	Contains the total number of requests run by the service since the last reset.
ece_service_request_max_duration	Gauge	Contains the maximum server-side request latency (in milliseconds) since the last reset.
ece_service_request_pending_count	Gauge	Contains the total number of requests currently pending for a service. Large number of pending tasks may indicate a performance or capacity problem.
ece_service_request_pending_duration	Gauge	Contains the duration of the request (in milliseconds) pending in a service.
ece_service_task_avg_duration	Gauge	Contains the average server-side task latency in milliseconds since the last reset.
ece_service_task_backlog	Gauge	Contains the current server-side task backlog for each service. A large backlog is indicative of a performance or capacity problem.
ece_service_task_count	Counter	Contains the total number of tasks processed by a service since the last reset.
ece_service_task_max_backlog	Gauge	Contains the maximum task backlog for each service since the last reset.
ece_service_unbalanced_partitions	Gauge	Contains the total number of unbalanced partitions for a particular service. Typically, unbalanced partitions can occur temporarily during rebalancing operations when the nodes are added or removed.

Example

The following are the examples for the metrics in the SERVICE_METRICS group:

• ece_service_avg_thread_count. This example shows the number of BRMFederatedCache and InvocationService service threads in use. By default, ECE provides 4 threads for the BRMFederatedCache service and 33 threads for the InvocationService:

  ece_service_avg_thread_count{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 0.5684553
  ece_service_avg_thread_count{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 1.3715283

• ece_service_endangered_partitions. This example shows that there are no endangered partitions for the ECS server node ecs1:

  ece_service_endangered_partitions{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 0
  ece_service_endangered_partitions{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} -1

• ece_service_ha_status. This example shows that the BRMFederatedCache services on the ECS server nodes ecs1 and ecs2 are 1 (NODE-SAFE):

  ece_service_ha_status{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 1
  ece_service_ha_status{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} -1
  ece_service_ha_status{nodeName="ecs2", machine="abc.us.example.com",  service="BRMFederatedCache"} 1
  ece_service_ha_status{nodeName="ecs2", machine="abc.us.example.com",  service="InvocationService"} -1

• ece_service_request_avg_duration. This example shows the ECE server node ecs1 with average request for the BRMFederatedCache service as 5 milliseconds and 0 for the InvocationService service (this service processes only tasks):

  ece_service_request_avg_duration{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 5.173
  ece_service_request_avg_duration{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0.000

• ece_service_request_count. This example shows the ECS server node ecs1 has processed approximately 30000 requests since the last statistics reset:

  ece_service_request_count{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 30044
  ece_service_request_count{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0

• ece_service_request_max_duration. This example shows the maximum request duration on the ECE server node ecs1 is approximately 66 milliseconds:

  ece_service_request_max_duration{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 67
  ece_service_request_max_duration{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0

• ece_service_request_pending_count. This example shows the ECE server node ecs1 with 12 pending requests:

  ece_service_request_pending_count{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 12
  ece_service_request_pending_count{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0

• ece_service_request_pending_duration. This example shows the ECE server node ecs1 with requests pending for 13 milliseconds:

  ece_service_request_pending_duration{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 13
  ece_service_request_pending_duration{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0

• ece_service_task_avg_duration. This example shows the average task durations on the ECE server node ecs1 for the InvocationService and BRMFederatedCache services in milliseconds:

  ece_service_task_avg_duration{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 3.276
  ece_service_task_avg_duration{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 9.142

• ece_service_task_backlog. This example shows the ECE server node ecs1 with a backlog of 4 requests on the BRMFederatedCache service and 1 task on the InvocationService on ECE server node ecs2:

  ece_service_task_backlog{nodeName="ecs1", machine="slc15ejg.us.example.com",  service="BRMFederatedCache"} 4
  ece_service_task_backlog{nodeName="ecs1", machine="slc15ejg.us.example.com",  service="InvocationService"} 0
  ece_service_task_backlog{nodeName="ecs2", machine="slc15ejg.us.example.com",  service="BRMFederatedCache"} 0
  ece_service_task_backlog{nodeName="ecs2", machine="slc15ejg.us.example.com",  service="InvocationService"} 1

• ece_service_task_count. This example shows that all services have handled approximately 15000 tasks:

  ece_service_task_count{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 14995
  ece_service_task_count{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 15010
  ece_service_task_count{nodeName="ecs2", machine="abc.us.example.com",  service="BRMFederatedCache"} 14272
  ece_service_task_count{nodeName="ecs2", machine="abc.us.example.com",  service="InvocationService"} 14264

• ece_service_task_max_backlog. This example shows the ECE server node ecs1 and ecs2 with a backlog of 26 and 13 tasks for the BRMFederatedCache and 0 and 1 for the InvocationService service:

  ece_service_task_max_backlog{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 26
  ece_service_task_max_backlog{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} 0
  ece_service_task_max_backlog{nodeName="ecs2", machine="abc.us.example.com",  service="BRMFederatedCache"} 13
  ece_service_task_max_backlog{nodeName="ecs2", machine="abc.us.example.com",  service="InvocationService"} 1

• ece_service_unbalanced_partitions. This example shows that there are no unbalanced partitions on ECE server node ecs1:

  ece_service_unbalanced_partitions{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 0
  ece_service_unbalanced_partitions{nodeName="ecs1", machine="abc.us.example.com",  service="InvocationService"} -1

FEDERATION_METRICS

The FEDERATION_METRICS group contains metrics for ECE federated caches when ECE persistence is disabled. The metrics in this group provide information regarding the volume of data transferred, the number of objects transferred, and so on. You can use this metric to monitor the data transferred from the primary production system to the remote or backup systems. This data is typically used for disaster recovery where the Oracle NoSQL database is used for storing rated events.

You can run the ECE Metrics Collector with the -f ClusterName and -F ServiceName parameters to retrieve the federated cache metrics.

Table 3-7 describes the metrics that ECE Metrics Collector retrieves for ECE federated caches when ECE persistence is disabled.

Table 3-7 ECE Federated Caches Metrics

Metric Name	Type	Description
ece_federated_service_bandwidth	Gauge	Tracks the current or maximum bandwidth used to transfer data from ECE charging nodes to a secondary ECE cluster.
ece_federated_service_rate	Gauge	Tracks the approximate rate of data transfer in bytes and number of messages sent. This metric uses the Coherence Mbean attributes for tracking data.
ece_federated_service_replicate_millis	Gauge	Contains the cache replication latency (in milliseconds) for initial data replication. The type can be: total (total time taken for replication), estimate_ttc (estimated time to complete).
ece_federated_service_replicate_percent	Gauge	Contains the percentage of cache replication completed for a service.
ece_federated_service_status	Gauge	Contains the status of the service which is on federation. The state of a service can be: 1 (Initial), 2 (Idle), 3 Ready, 4 (Sending), 5 (Connecting), 6 (Connect_Wait), 7 (Stopped), 8 (Paused), 9 (Error), 10 (Yielding), 11 (Backlog_Excessive), 12 (Backlog_Normal), and 13 (Disconnected). The status of a service can be: 1 (OK), 2 (Warning), and 3 (Error).
ece_federated_service_time_millis	Gauge	Contains the cache replication latency (in milliseconds) for data replicated to the remote cache. The type can be: apply, backlog_delay, and round_trip. These are the 90th percentile latency times.
ece_federated_service_total	Gauge	Contains the total number of bytes, cache entries, and messages that are replicated to the remote cache. The entity type can be: records (total number of journal records), bytes (total number of bytes sent), entries (total number of cache entries sent), message (total number of messages sent), response (total number of message responses received). The status can be: sent (entity shipped to remote cluster), unacked (messages sent without acknowledgment), and error (messages failed).

Example

The following are the examples for the metrics in the FEDERATION_METRICS group:

• ece_federated_service_bandwidth. This example shows the bandwidth at the current moment for the BRMFederatedCache and ReplicatedFedratedCache services on the ECE charging server node ecs1 (in MBps). It also shows that there is no maximum bandwidth configured (as the value is -1.0) for these services:

  ece_federated_service_bandwidth{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="current"} 9.159515380859375
  ece_federated_service_bandwidth{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="max"} -1.0
  ece_federated_service_bandwidth{nodeName="ecs1", machine="abc.us.example.com",  service="ReplicatedFederatedCache", type="current"} 0.0
  ece_federated_service_bandwidth{nodeName="ecs1", machine="abc.us.example.com",  service="ReplicatedFederatedCache", type="max"} -1.0

• ece_federated_service_rate. This example shows the approximate number of bytes sent per second and number of messages sent for the BRMFederatedCache service on the ECE charging server node ecs1:

  ece_federated_service_rate{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="bytes", type="sent"} 1183714
  ece_federated_service_rate{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="message", type="sent"} 182

• ece_federated_service_replicate_millis. This example shows the replication completed time in milliseconds for the BRMFederatedCache service on the ECE charging server node ecs1:

  ece_federated_service_replicate_millis{nodeName="ecs1", machine="abc.us.example.com", service="BRMFederatedCache", type="estimate_ttc"} 0
  ece_federated_service_replicate_millis{nodeName="ecs1", machine="abc.us.example.com", service="BRMFederatedCache", type="total"} 378
  ece_federated_service_replicate_millis{nodeName="ecs1", machine="abc.us.example.com", service="ReplicatedFederatedCache", type="estimate_ttc"} 0
  ece_federated_service_replicate_millis{nodeName="ecs1", machine="abc.us.example.com", service="ReplicatedFederatedCache", type="total"} 938

• ece_federated_service_replicate_percent. This example shows that the replication is 100% complete for BRMFederatedCache and ReplicatedFedratedCache services on the ECE charging server node ecs1:

  ece_federated_service_replicate_percent{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache"} 100
  ece_federated_service_replicate_percent{nodeName="ecs1", machine="abc.us.example.com",  service="ReplicatedFederatedCache"} 100

• ece_federated_service_status. This example shows the state and statuses of the BRMFederatedCache and ReplicatedFedratedCache services on the ECE charging server node ecs1. The service status is directly reflected from Coherence.

  ece_federated_service_status{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="state"} 2
  ece_federated_service_status{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="status"} 0
  ece_federated_service_status{nodeName="ecs1", machine="abc.us.example.com",  service="ReplicatedFederatedCache", type="state"} 2
  ece_federated_service_status{nodeName="ecs1", machine="abc.us.example.com",  service="ReplicatedFederatedCache", type="status"} 0

• ece_federated_service_time_millis. This example shows the time taken to apply the changes to the BRMFederatedCache service on the ECE charging server node ecs1, with a round-trip time of approximately 5 minutes:

  ce_federated_service_time_millis{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="apply"} 20
  ece_federated_service_time_millis{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="backlog_delay"} 10
  ece_federated_service_time_millis{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", type="round_trip"} 5

• ece_federated_service_total. This example shows the total number of BRMFederatedCache service entries, messages, data bytes, records, and responses replicated from the ECE charging server node ecs1. The changes to the cache entries are grouped into journal records and then into messages:

  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="bytes", type="sent"} 28868414
  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="entries", type="sent"} 11292
  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="message", type="sent"} 5590
  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="message", type="unacked"} 0
  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="records", type="sent"} 5646
  ece_federated_service_total{nodeName="ecs1", machine="abc.us.example.com",  service="BRMFederatedCache", entity="response", type="error"} 0

Setting Log Levels by Using Scripts

You can set the log level of the ECE module or modules by ECE functional domain.

To set log levels by using the ece_log script:

1. On the driver machine, change to the ECE_home/tools directory.
2. Start ECC, if not started:

   ./ecc

3. Run the following command:

   ece_log -l Levels -c Classes -S Level

   where:

   • Levels specifies the log-levels to be selected; for example, "debug|info".
   • Classes specifies the log class names.
   • Level specifies the log-level to be set.

   For example:

   ece_log -l debug -c "rating|client" -S error

   This command sets all the DEBUG-level log entries to ERROR for the rating and client classes.

Using cohql_file_processor for Query

The cohql_file_processor tool is a wrapper around the query tool (query.sh), which runs queries on ECE Coherence caches. You can use the following scripts to query ECE data instead of running the query.sh script frequently:

• start_cohql_file_processor. Use this script to run the query in the non-interactive mode using input files.
• interactive_cohql. Use this script to run the query in the interactive mode using command-line.

You can stop the query by using the stop_cohql_file_processor script.

For information on the parameters, default values, and examples for this tool, run the help (-h) command.

Duplicate Check Enhancement in ECE

ECE performs duplicate check on the incoming TERMINATE, DIRECT_DEBIT, and REFUND requests to verify the session ID in the requests. By default, the session ID is stored in the aggregate usage and rated event objects. In this case, sometimes the aggregate usage and rated event objects have to be kept in the ECE cache for a longer duration when duplicate checks are performed on these requests.

With this enhancement, when data persistence is enabled in ECE, the session IDs are stored in the new Session ID cache, TerminatedSessionHistory. This cache stores only the session IDs of the requests that are processed. For performing duplicate checks on TERMINATE, DIRECT_DEBIT, and REFUND requests, ECE checks the session IDs in this new cache. This ensures that the aggregate usage and rated event objects are evicted from the cache immediately after the successful cache update.

Charging Operation Type Can be Configured for Expired Active Session Cleanup

In the previous releases, ECE removed the expired active session and the corresponding expired balance reservations only when TERMINATE or CANCEL operation type in usage requests were processed. It was not possible to configure other charging operation types for cleaning up the expired objects.

With this enhancement, you can configure for which charging operation types the expired active sessions and balance reservations must be cleaned up. For example, you can configure ECE to remove the expired active sessions and balance reservations when the INITIATE or UPDATE operation type in a usage request is processed. You can specify the charging operation types to be used by setting the asoCleanupOperationList Mbean attribute. See "Configuring Expired Objects Cleanup".

Configuring Expired Objects Cleanup

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.server.

3. Expand Attributes.

4. Set the asoCleanupOperationList attribute to the charging operation types for which the expired active session and balance reservation objects must be cleaned up.

   The default values are TERMINATE, CANCEL. If this attribute is set to null, the default values are used for the cleanup.

   For the list of charging operation types supported by ECE, see the discussion about the Charging API in ECE Implementing Charging.

ECE Now Generates POID for Events

In the previous releases, ECE was using the Portal object IDs (POIDs) received from BRM for tracking events rated by ECE.

With this enhancement, POIDs can be generated in ECE for tracking the rated events. ECE uses Rated Event Formatter to generate the required POIDs and persists the last allocated POID ID in the Oracle NoSQL database. This ensures that the POIDs are generated without any duplication even if the ECE system is restarted.

The POID generated in ECE contains the following information:

event_type date cluster_id BRM_schema_id unique_id

See Table 3-8 for the description of each entry in the POID.

Table 3-8 POID Entries in ECE

Entry	Description
event_type	A unique 4-bit number assigned to each event type. For example, 0 is assigned to subscription events, 1 is assigned to postpaid events (USAGE_POSTPAID), and 2 to 7 is assigned to prepaid events (USAGE_PREPAID) depending on the prepaidParttitionSet value specified in BRM. The default value for event_type is 0.
date	The 16-bit date on which the POID is generated. The date is determined based on ECE virtualTime if it is enabled. For more information on virtualTime, see the discussion about changing time and date to test ECE in ECE Implementing Charging.
cluster_id	A unique 4-bit number assigned to the Coherence cluster to identify ECE in the cluster. The cluster_id is limited to 0 to 15 and the maximum number of ECE clusters allowed in a deployment is 16. The default value for cluster_id is 0. If ECE is configured for disaster recovery, you must specify the cluster ID for each cluster used in the Active-hot standby or Active-cold standby systems.
BRM_schema_id	A unique 6-bit number assigned to the BRM schema. The BRM_schema_id is limited to 0 to 31.
unique_id	A unique 34-bit number assigned to each POID.

For tracking the events rated by ECE, Rated Event Formatter uses the POIDs generated in ECE. You can configure multiple instances of Rated Event Formatter to ensure high availability and uninterrupted POID allocation. In case if the primary Rated Event Formatter instance fails, the secondary Rated Event Formatter instance ensures that the POIDs are allocated without any interruption. In a disaster recovery deployment, if the Rated Event Formatter instance in the primary site fails, the Rated Event Formatter instance in the backup site continues the POID allocation for the events. To connect the instances in different sites or systems, you must specify the name of the primary Rated Event Formatter instance in the primary and secondary Rated Event Formatter instances.

For tracking the bill items created in ECE, ECE continues to use the POIDs received from BRM. However, ECE now persists the POID pool received from BRM in the Oracle NoSQL database. This ensures that the reserved POID pool is retained in ECE even after the ECE restart. It allows ECE to continue the POID allocation for the bill items using the existing POID pool, which in turn reduces the dependency on BRM.

To enable POID generation in ECE for events, you must perform the following:

1. Enable prepaid-event partitions in BRM. For instructions, see "Enabling Prepaid-Event Partitions".

2. Ensure that the cluster ID is configured for ECE clusters. The cluster ID must be specified if you have ECE configured for disaster recovery. See "Configuring Cluster ID".

3. Ensure that the name of the primary Rated Event Formatter instance is specified in each Rated Event Formatter instance. The primary Rated Event Formatter instance must be specified if you have ECE configured for disaster recovery. See "Connecting Rated Event Formatter Instances".

4. Enable prepaid-event partitions in ECE. See "Enabling Prepaid-Event Partitions".

Configuring Cluster ID

To configure the cluster ID for ECE clusters:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.clusters.Cluster_Name, where Cluster_Name is the name of the ECE cluster that you are configuring.

3. Expand Attributes.

4. Set the id attribute to a unique number that indicates the ID of the cluster in the POID generated in ECE.

   Rated Event Formatter uses the cluster ID in the POID to identify the ECE clusters. The cluster ID must be unique for each cluster.

Connecting Rated Event Formatter Instances

To connect the Rated Event Formatter instances in different sites or systems, you must perform this for each Rated Event Formatter instance

To connect Rated Event Formatter instances:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.ratedEventFormatters.Instance_Name, where Instance_Name is the name of the instance you want to configure; for example, ratedEventFormatter2.

3. Expand Attributes.

4. Set the primaryInstanceName attribute to the name of the primary Rated Event Formatter instance.

   For example, if the name of the primary Rated Event Formatter instance is ratedEventFormatter1, specify ratedEventFormatter1 as primaryInstanceName in the primary and all secondary instances.

5. Change directory to the ECE_home/oceceserver/bin directory.

6. Start ECC:

   ./ecc
   

7. Stop and restart any Rated Event Formatter instances that you configured.

   Each instance reads its configuration information by name.

   For information about stopping and starting Rated Event Formatter instances, see the discussion about starting and stopping ECE in BRM Elastic Charging Engine System Administrator's Guide.

Enabling Prepaid-Event Partitions

To enable prepaid-event partitions:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.brmCdrPlugins.Instance_Name, where Instance_Name is the name of the BrmCdrPluginDirect Plug-in instance you are configuring.

3. Expand Attributes.

4. Set the prepaidPartitionSet attribute to the value that you specified in the prepaid_partition_set entry in the BRM_Home/sys/dm_oracle/pin.conf file.

   To enable prepaid-event partitions, you need to set this attribute to a number between 2 and 7. If this attribute is set to 0, ECE continues to use the POIDs received from BRM for events instead of generating them.

New Tool for Querying ECE Cache Data

In previous releases, the ECE cache data query took a long time to complete if the data was not properly indexed.

With this enhancement, a new query tool, query_cache.sh, has been introduced to query data in the ECE caches that are associated with customer keys. The new query tool provides access to the ECE cache content by way of CohQL. This tool is included with the ECE Server software in ECE_home/ocecesdk/bin/query, where ECE_home is the directory in which ECE Server software is installed. You can use this tool for debugging, development, generating reports, and other-related queries. This tool supports only the non-interactive mode.

The following is the syntax for the non-interactive use of the new query tool:

sh query_cache.sh run  cache_name Associated_key ID 

where:

• Cache_name specifies the name of the cache you are querying.

• Associated_key specifies the customer ID (composite key) that is associated with the cache.

• ID specifies the unique identifier, such as balance ID, subscriber ID, and so on.

For example, to query a customer's balance, you can run the query on the Balance cache using the associated customer ID (479665838) and balance ID (479664046):

sh query_cache.sh run Balance 479665838 479664046

This query returns the balance information for the specified customer ID.

For information on the query tool options, use the help command:

sh query_cache.sh -h

ECE Now Supports Wildcard in Item Type Selectors

ECE now supports wildcard (*) in item type selectors for services and events. You can use the wildcard to substitute one or more characters in the service or event type to indicate that any value is acceptable; for example, /service/telco/gsm/*.

If wildcard is used in the service or event type, ECE uses the applicableToAllChildServices and applicableToAllChildEvents values to identify if the service or event type and item type selector is applicable for all the child services or events. If the value is true, the item type selector is considered for all the child services or events. If the value is false, the item type selector is not considered for the child services or events.

For more information on using wildcard in item type selectors, see PDC Creating Product Offerings.

Support for Persisting BRS Configuration Data

In the previous releases, batch request service (BRS) parameters were not persisted in the ECE cache and were lost during ECE restart. You had to manually set these parameters again after ECE restart.

With this enhancement, you can persist the BRS parameters in the ECE cache by adding them using the brsConfigurations MBean attribute. You can now use this attribute instead of the BatchRequestService and ChargingClient.BatchRequestService MBean attributes for configuring system overload protection and client-side ECE request queues.

Note:

If you have multiple instances of a component and you want to use the same BRS configuration for all the instances, you can add the BRS configuration for the role instead of each instance. For example, if you have diameterGateway1, diameterGateway2, and diameterGateway3, add the BRSconfiguration instance as diameterGateway to use the same BRS configuration for all the three instances.

If the BRS configuration is not added or removed for any of the instance, the BRS configuration for the role is applied to that instance by default. If the BRS configuration for the role is not defined, the default (system-level) BRS configuration is applied to that instance.

To add the BRS configuration:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.brsConfigurations.

3. Expand Operations.

4. Click addBrsConfiguration.

5. Set the name attribute to the name of the BrsConfiguration instance; for example, diameterGateway1.

6. Click addBrsConfiguration.

   The new BRS configuration is added. For example, charging.brsConfigurations.diameterGateway1.

7. Expand charging.brsConfigurations.instancename, where instancename is the name of the instance for which you are configuring BRS.
8. Expand Attributes.
9. Specify the values for the attributes.

For descriptions of each attribute, see the discussion about configuring system overload protection and configuring client-side ECE request queues in BRM System Administrator’s Guide. For the default values, see the charging.brsConfigurations.default Mbean attribute.

You can also delete the BRS configuration. To delete the BRS configuration:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.brsConfigurations.

3. Expand Operations.

4. Click removeBrsConfiguration.

5. Set the name attribute to the name of the BrsConfiguration instance that you want to remove.
6. Click removeBrsConfiguration.

   The BRS Configuration is deleted.

Rated Event Partition in the ECE Persistent Database

In the previous releases, Rated Event Formatter was using SQL-DELETE query for deleting the rated events stored in the ECE persistence database. When SQL-DELETE query was used for deleting rated events, there was a considerable increase in the table space. This might impact the overall performance.

With this enhancement, Rated Event Formatter uses Oracle Drop Partition instead of SQL-DELETE query for deleting rated events. By default, the partition time for creating rated event partitions is set to 5 minutes. You can change the default partition time depending on your sizing and performance requirements by setting the ratedEventTablePartitionByMinute MBean attribute. During installation you can configure the initial storage and subpartition values for rated event table. For more information, see Elastic Charging Engine Installation Guide.

To change the partition time:

1. Access the ECE MBeans:

   1. Log on to the driver machine.

   2. Start the ECE charging servers (if they are not started).

   3. Start a JMX editor, such as JConsole, that enables you to edit MBean attributes.

   4. Connect to the ECE charging server node set to start CohMgt = true in the ECE_home/oceceserver/config/eceTopology.conf file.

      The eceTopology.conf file also contains the host name and port number for the node.

   5. In the editor's MBean hierarchy, expand the ECE Configuration node.

2. Expand charging.connectionConfigurations.Connection_Name; where Connection_Name is the name of the persistence database connection.

3. Expand Attributes.

4. Click ratedEventTablePartitionByMinute to set the partition time.

5. Save your changes.

New Features in ECE 12.0/12.0 Patch Set 1

For the new features in ECE 12.0/12.0 Patch Set 1 (12.0.0.1.0), see ECE Release Notes.