Configuring Node Chains

Configuring Node Chain Input

All input into Offline Mediation Controller comes through a collection cartridge (CC) node. To receive input, Offline Mediation Controller supports the following methods, depending on the capabilities of the network device providing input:

You can configure the CC node to look in a specified directory. You need to configure the network device to send files to that directory.
You can configure the CC node to use FTP to retrieve files from an external location.
You can configure the CC node to communicate directly with the network element. For example, you can configure a CC node to receive records from a Diameter client.

If the CC node receives input from a network connection, such as from a Diameter client, you can configure options such as the port to connect to and time out parameters.

If the CC node receives input in files, you can configure file management settings, such as:

How to recognize which files the node should process, usually by identifying a file suffix or prefix.
Specify to change the file suffix after the CC node processes it.
Specify the FTP connection settings to retrieve files by using FTP.
Specify whether to delete input files after processing. If you choose not to delete the input files, they are moved from the input directory to the directory specified in the inputBackupDir property in the OMC_home/config/node_id/general.cfg file for the node.
By default, inputBackupDir is set to OMC_home/backup/node_id.

In addition, you might edit a rule file to customize how data is handled, or specify the type of rule file that applies the input format.

The following nodes handle protocol-specific input:

The GTP Data Collection CC node receives GSM data directly from the network.
The GTP Prime Collection CC node receives data from the GPRS Tunneling Protocol (GTP) protocol.

Configuring Input from BRM

The Oracle CDR format is used by Oracle Communications Billing and Revenue Management (BRM) to process CDRs for offline charging. The Oracle CDR Format CC node collects CDR input files conforming to the Oracle CDR format and parses it into a network accounting record (NAR).

The Oracle CDR Format CC node uses the schema information in a schema file to parse the CDR files conforming to the Oracle CDR format into NARs. If the CDR files you process include information that has no corresponding fields in the default schema file, you can configure a custom schema file to accommodate the custom data.

To use a custom schema, copy your new schema file into the OMC_home/config/sol42/schema directory. You then choose your custom schema when configuring the Oracle CDR Format CC node in Administration Client.

Configuring Node Chain Output

Node chain output is created by a distribution cartridge (DC) node. The DC node reads the NAR and converts it to the format required by the destination.

Offline Mediation Controller supports several methods to provide output to external systems. For example:

You can generate files in various formats and send them to an external system by using FTP.
You can use JDBC to send output to a database. You can configure the DC node rule file to enable sending large objects, such as images, audio, and multimedia, to the database.
You can connect directly to an external system and send data directly to their receiving interface. For example, you could use ASN.1.

When you configure a DC node, most of the configuration specifies how to send the output. For example:

If the output is a file sent by FTP, you configure the output directory, file name attributes, FTP connection information, and so on.
If the output is to a database, you configure database connection information, such as the SID, host name, and so on.

You can also configure how to handle files after output. For example, you could specify to archive or delete the files.

Managing Sequencing

Sequencing ensures that records are processed in the correct order. You can configure sequencing at:

The record level. See "About Record-Level Sequencing".
The file level. See "About File-Level Sequencing".

You can also customize the file name syntax for file-level sequencing. See "Customizing the Sequencing File Name Syntax".

About Record-Level Sequencing

Sequencing EP nodes detect records that are not in the correct order and then correct the processing order. If the EP node discovers records that are not in sequence, it holds on to those records until it can sequence them, or until the flush timer expires. If the EP node is able to sequence the records, it flushes them right away. If the EP node is not able to sequence the records, it flushes them when the timer expires. You can set the timer when you configure the node.

The logic for determining the correct sequence is defined in the node rule file. You usually do not need to edit the rule file.

You can use the following EP nodes for sequencing records:

Sequencing EP node
3GPP Session Sequencing EP node

About File-Level Sequencing

CC nodes process input files based on the alphabetical order of the CDR input file name. You can also configure the CC nodes to validate the file names based on a sequence number.

When file-level sequencing is enabled, the CC node performs the following validations on each CDR input file:

Checks whether the file name includes a sequence number.
Checks whether the sequence number is in order.

If the file name passes both validations, the CC node processes the input file and sets seqNum in memory and in the OMC_home/ocomc/scratch/node_id/file_seq_order_info file to the next sequence number. For example, if the CDR input file has a sequence number of 12, the CC node sets seqNum to 13.

If the file name fails one or both validations, the CC node adds a warning message to the node log file and displays an alarm before processing the input file.

Note:

You can use information in the node log file to identify the out-of-sequence file names.
If the CC node stops while processing CDR input files, Offline Mediation Controller uses file_seq_order_info to resume validations when the CC node is restarted.

For example, this shows how the CC node would process CDR input files received in the following order:

ABC_10.txt: The CC node changes seqNum to 11 and processes the input file.
ABC_11.txt: The CC node changes seqNum to 12 and processes the input file.
ABC_15.txt: The CC node logs the out-of-sequence file name in the node log file, displays an alarm, processes the input file, and changes seqNum to 16.
ABC_5.txt: The CC node logs the out-of-sequence file name in the node log file, displays an alarm, processes the input file, but does not change seqNum.

To use file-level sequencing, configure your file-based CC node as follows:

Enable sequence ordering in the CC node.

In the CC node's Node Configuration dialog box, select the Advanced tab and then select the Sequence Ordering by File Name option.
(Optional) Enable multi-threading in the CC node to improve performance. See "Improving CC Node Processing Performance with Multi-Threaded Processing".
Ensure that your CDR input files adhere to the file name syntax required for sequencing. Your files can use one of the following:
- The default file name syntax for sequencing:
```
sourceFilename[_seqNum].fileExtension
```
  where:
  - sourceFilename is the name of the original CDR input file.
  - seqNum is the sequence number of the CDR input file. This should contain a fixed number of digits.
  - fileExtension is the extension of the original CDR input file.
- A custom file name syntax for sequencing. See "Customizing the Sequencing File Name Syntax".

Customizing the Sequencing File Name Syntax

To customize the file name syntax for sequencing:

Stop the CC node. See "Stopping Nodes" in Offline Mediation Controller System Administrator's Guide.
Open the OMC_home/config/nodeID/general.cfg file in a text editor.
Set the fileNamePattern entry to the syntax of your CDR input file names (without the file name extension).
```
fileNamePattern 'X_SEQ.X'
```
where:
- X represents a group of alpha-numeric characters. The group cannot contain symbols or special characters.
- SEQ is the place holder for the sequence number.
- . is a dot that appears between a group of alpha-numeric characters.
- _ is an underscore that appears between a group of alpha-numeric characters.
Save and close the general.cfg file.
Start the CC node. See "Starting Nodes" in Offline Mediation Controller System Administrator's Guide.

Sample File Name Syntax

The following shows a group of sample input file names with the sequence number shown in bold:

APSCDR_M_USERBILLING.2030.02.20.1031.STD.000036.1.rcd
APSCDR_M_USERBILLING.2030.02.20.1032.STD.000037.1.rcd
APSCDR_M_USERBILLING.2030.02.20.1033.STD.000038.1.rcd

In this case, you would configure the fileNamePattern parameter as follows:

fileNamePattern 'X_X_X.X.X.X.X.X.SEQ.X'

Sample File Name Syntax with Sequence Number Last

The following shows a group of sample input file names with the sequence number shown in bold:

BW-CDR-20300604093000-2-00505691D073-000503.csv
BW-CDR-20300604093000-2-00505691D073-000504.csv
BW-CDR-20300604093000-2-00505691D073-000505.csv

In this case, you would configure the fileNamePattern parameter as follows:

fileNamePattern 'X-X-X-X-X-X.SEQ'

Managing Duplicate Records

Offline Mediation Controller includes the following methods of detecting and removing duplicate records:

You can remove duplicate CDR files. See "About Removing Duplicate CDR Files".
You can remove duplicate records. See "About Removing Duplicate Records".

About Removing Duplicate CDR Files

By default, the file-based CC nodes do not validate the input files based on the file names. The CC nodes process the input files based on the alphabetical order of the file name. You can configure a CC node to identify duplicate files based on the file name.

When configured to detect duplicates, the CC node reads the CDR input file names and stores them in memory and in the OMC_home/ocomc/scratch/node_id/file_duplicate_info file for a specified length of time, where node_id is the unique ID assigned to the node when the node configuration is saved. The CC node compares each incoming file's name to those in memory.

If the file name does not exist in memory, the file is processed.
If the file name exists in memory, the CC node renames the file to sourceFilename.fileExtension.duplicate, logs a warning in the node log, and displays an alarm.

If the CC node stops while processing CDR input files, Offline Mediation Controller uses the information in the file_duplicate_info files to resume validations when the CC node is restarted.

About Removing Duplicate Records

Use the duplicate check EP node to find duplicate records while processing CDRs. This prevents you from charging a customer twice for the same usage.

To configure the duplicate check EP node, you define a duplicate check key. A duplicate check key is a set of fields in the CDR that uniquely identify the record and the record timestamp. When you run the EP node, the key is added to a memory partition, and assigned an amount of time for which it is used.

When CDRs are processed, the duplicate check node compares the keys in the records to the keys in the partition. If the key already exists in the partition, the record is rejected and is written to a file in the duplicate record storage directory, and the information related to the duplicate records is written to a log file.

If a duplicate key is not found, the key is added to the list of keys in the partition and the duplicate check file, and the CDR is distributed to the next node in the mediation node chain.

When you configure the duplicate check EP node, the configuration includes:

Editing the EP node rule file to specify the data used for the duplicate check key. For example:
```
DUP_CHECK_KEYS "calling_number session_id seq_no";
DUP_CHECK_EVENT_TIME_FIELD "start_time";
```
Configuring the EP node to specify parameters such as how often to create a partition to store duplicate check keys (daily or monthly), how often to flush records, where to store duplicate records files, and so on.

In addition to the duplicate check EP node, you can also detect duplicate records by configuring some sequencing EP nodes, for example:

Sequencing EP node
3GPP Session Sequencing EP node
Hot Billing Duplicate CDR Removal EP node

Oracle recommends using the duplicate check EP node.

About Aggregation

Aggregation can be used for a variety of purposes. The primary function is to aggregate multiple records for a single session into one record.

When you configure aggregation, you can specify to aggregate data based on the following:

Time. You can specify to aggregate records based on time. There are two options:
- Aggregate records based on the first record received. For example, if the time is set for 10 minutes, records are aggregated for 10 minutes and then aggregated. Following that, records are aggregated for another 10 minutes and so on.
- Aggregate records based on the time since the last record was received. For example, if the time is set for 10 minutes, and ten minutes pass before no record is processed, the records are aggregated.
Volume. For example, you could create a record when the volume reaches 1 MB.

When using volume-based aggregation, the AP node may create NARs for a partial session. That is, if the session contains two CDRs of 6 MB and 2 MB respectively, and the maximum volume is set to 1 MB, Offline Mediation Controller creates two NARs for the session instead of one NAR per session.
Time and volume. If you specify to aggregate based on time and volume, the records are aggregated based on which value is reached first, time or volume.

About Time-Based Aggregation of CDRs

Offline Mediation Controller supports aggregating the CDRs based on the arrival time of the CDRs before the AP node writes the aggregated CDRs as one or more NARs to a NAR archive file.

If you set the option for session segmentation by the arrival time of the first CDR, the CDRs are aggregated in memory from the arrival time of the first CDR until the time configured in the Flush Time field (or until the arrival of a termination CDR). When the flush time is reached, the AP node writes the aggregated CDRs as a NAR to a NAR archive file. With this option, if the flush time occurs before the arrival of the termination CDR, the CDRs are written as multiple NARs to a NAR archive file.

For example, if the segmentation of the aggregated CDRs is time-based set by the arrival of the first CDR, the flush time is set to 600 (10 minutes), and the flow of CDRs is as follows:

The first CDR for session A arrives at 10:10 am
The second CDR for session A arrives at 10:15 am
The third CDR for session A arrives at 10:17 am
The fourth CDR for session A arrives at 10:21 am

The first three CDRs are aggregated and stored in the memory until 10:20 am (time of arrival of the first CDR + flush time). At 10:20 am, the AP node writes the aggregated CDRs as a NAR to a NAR archive file. The fourth CDR is aggregated and stored in the memory in a new aggregation session.

If you set the option for session segmentation by the arrival time of the last CDR, the CDRs are aggregated in memory from the arrival time of the last CDR until the time configured in the Flush Time field (or until the arrival of a termination CDR). When the flush time is reached, the AP node writes the aggregated CDRs as a NAR to a NAR archive file. If a new CDR arrives before the flush time is reached, the flush time is reset to begin from the arrival time of the new CDR. With this option, the CDRs for long-duration sessions are aggregated into a single NAR unless the flush time is reached before the arrival of the termination CDR.

For example, if the segmentation of the aggregated CDRs is time-based set by the arrival of the last CDR, the flush time set to 600 (10 minutes), and the flow of CDRs is as follows:

The first CDR for session A arrives at 10:10 am
The second CDR for session A arrives at 10:15 am
The third CDR for session A arrives at 10:17 am

If no more CDRs arrive, the three CDRs are aggregated and stored in memory until 10:27 am (time of arrival of the last CDR + flush time). At 10:27 am, the AP node writes the aggregated CDRs as a NAR to a NAR archive file.

About Volume-Based Aggregation of CDRs

Offline Mediation Controller supports aggregating the CDRs based on volume. When the volume of the aggregated CDRs reaches the configured maximum volume, the AP node writes the aggregated CDRs as a NAR to a NAR archive file.

For example, if the segmentation of the aggregated CDRs is volume-based, the data volume is a maximum of 1 MB, and the flow of CDRs is as follows:

The first CDR for session A has the value of 300 KB in the CDR's dataVolume field
The second CDR for session A has the value of 700 KB in the CDR's dataVolume field
The third CDR for session A has the value of 100 KB in the CDR's dataVolume field

The maximum volume limit is exceeded when the third CDR arrives and the AP node writes the aggregated CDRs as a NAR to a NAR archive file.

Note:

When using volume-based aggregation, the AP node may create NARs for a partial session. That is, if the session contains two CDRs of 6 MB and 2 MB respectively, and the maximum volume is set to 1 MB, Offline Mediation Controller creates two NARs for the session instead of one NAR per session.

About Mapping and Enrichment

Charging often requires that you change or add data to records before they are rated. To do so, use EP nodes. See the following topics:

Record Enhancement for BRM Charging

Use the record enhancement charging EP node to prepare records for charging by BRM. This node configures the following:

Social number
Prefix description
Service code
Usage class
Access point name (APN)

To configure the record enhancement charging EP node, define the mapping and enrichment values in the rule file.

Before you create the record enhancement charging EP node, configure the database connection information to Oracle Communication Billing and Revenue Management (BRM) database. The record enhancement charging EP node uses the database connection information to retrieve the mapping information while enhancing the CDR data.

To configure the database connection:

Open the OMC_home/config/nodemgr/prc.properties file in a text editor, where OMC_home is the directory in which Offline Mediation Controller is installed.
Add the following text:
```
driver=oracle.jdbc.driver.OracleDriver
url= jdbc:oracle:thin:@server_name:port_number:database_alias
username=login
password=password
```
where:
- server_name is the system on which the BRM database is installed.
- port_number is the port number of the system running the BRM database.
- database_alias is the BRM database alias of the schema you are connecting.
- login is the username for the database schema you are connecting.
- password is the encoded password for login.
Save and close the file. You now need to encode the database connection password.
Go to the OMC_home/bin directory.
Run the following command:
```
./encode password
```
where password is the password for the user name for the database schema you are connecting.

The command window displays password in encoded form. For example:
```
72,-46,62,71,41,-115,14,-68,-34,-4,-105,-113,-125,1,-18,-70
```

Filtering Records Based on Data

The Record Filter EP node deletes records according to multiple selection criteria. Filtering is based on:

CDR Type: G-CDR, S-CDR, M-CDR, S-SMO-CDR, S-SMT-CDR·
List of APNs
List of SMS-Cs

The EP node also suppresses certain optional fields if they are not required.

Restoring Excluded Records

Use the Partial CDR Completion AP node to restore specific fields to records. Even though this is an AP node, it does not aggregate any records.

The rule file specifies to restore eight optional fields to the record, which are normally excluded. The Nortel SGSN generates S-CDRs that exclude eight optional fields in non-initial partial records. These eight fields are excluded because their values do not change throughout the PDP session. If you use this rule file, the node adds the eight fields to the non-initial S-CDRs, using the values from the initial S-CDR. The eight optional fields are:

MS-Network-Capability
Charging-Characteristics
Served-MSISDN
APN-Selection-Mode
Node-ID
Network-Initiation
QosNegotiated
QosRequested

Adding Data to Records

You can add data to records by using the following EP nodes:

Use the Record Processing EP node to enhance records with the user-defined rules set in the rule file.
Use the Record Enhancement (Local File) EP node to enhance records with data from a local file.
Use the Record Enhancement (Remote File) EP node to enhance records with data from a remote file.

File and Record Processing Options

See the following topics:

Processing CDRs at File Level

By default, Offline Mediation Controller processes the call detail record (CDR) input file as individual records. Offline Mediation Controller can also process the CDR input file as a single unit. This enables Offline Mediation Controller to reject the original CDR input file at any stage of processing by the nodes in the node chain.

When processing individual records, the CC node processes the CDR input file, and each CDR in the CDR input file is converted to a NAR. In the CC node, any run-time errors during the CDR data processing are handled through the NPL rule file, and the CDR input file is renamed to sourceFilename.error. The records that are already processed by the CC node and sent to the next node cannot be rolled back.

You can configure file-level transaction for nodes, only if:

Every node in the node chain is configured to use file-level transaction.
The node chains contain a single flow without any splits.

When processing CDR input files, Offline Mediation Controller performs the following tasks:

Generates a transaction ID and associates the ID with the input file.
Does one of the following:
- If the transaction is successful, completes the transaction and renames the original CDR input file to sourceFilename.processedFileSuffix.
  
  where:
  - sourceFilename is the name of the original CDR input file.
  - processedFileSuffix is the file pattern suffix configured in the CC node to rename the CDR input file post processing. The default is .done.
- If the transaction is rejected, renames the CDR input file to sourceFilename.reject and moves the file to the OMC_home/reject directory. These rejected files are made available for suspense handling.
Provides statistics related to the file-level transaction. The statistics are displayed in the Transaction Details table in the Node Performance screen.

Configuring the File-Level Transaction Threshold

When processing input files, you can configure the file-level transaction threshold in the node manager configuration file. The file-level transaction threshold determines what percentage of records in a file can be suspended before the entire input file is rejected. This is an optional parameter. The default is 100%.

For example, if you have configured the file-level threshold to be at 40%, if a file contains 10 records, the file will be rejected when the fourth error record is encountered.

To configure the file-level transaction threshold:

Stop the node manager.
Open the OMC_home/bin/nodemgr.cfg file in a text editor.
Add the following entry to the file:
```
FILE_LEVEL_TRANSACTION_THRESHOLD 'value'
```
where value specifies the threshold percentage.

For example:
```
FILE_LEVEL_TRANSACTION_THRESHOLD '50'
```
Save and close the file.
Start the node manager.

Viewing the File-Level Transaction Performance

To view the file-level transaction performance:

Log on to Offline Mediation Controller Administration client.

The Node Hosts & Nodes (logical view) screen appears.
From the Administrative Function list, select Node Performance.
In the Mediation Hosts table, select the mediation host that you want to monitor.

The Transaction Details table displays the file-level transactions currently running under the selected mediation host.

Improving EP Performance by Using Multi-Threaded Processing

By default, an EP node processes records in sequential order, using a single processing thread. When you use a single-threaded EP node to enrich the data records with a multi-threaded collection cartridge (CC) node and a multi-threaded distribution cartridge (DC) node, the records processing is slower. When multi-threading is enabled for Record Processing EP nodes, multiple files are processed in parallel, reducing the processing time.

Improving CC Node Processing Performance with Multi-Threaded Processing

By default, file-based CC node and file-based DC node input files are processed in sequential order, using one processing thread. When multi-threading is enabled for file-based CC and DC nodes, multiple input files are processed in parallel, reducing the processing time.

When multi-threading is enabled, the order is not maintained. If you require your file-based CC nodes and file-based DC nodes to output files in the order that the files are read and processed, even when multi-threading is enabled, select the Enable Ordering check box.

If Multi Threaded and Enable Ordering are enabled, the maximum number of records set in the Max Records Per File field in the File Output node tab is disregarded. Files retain the original number of records received by the ASCII DC node.

Improving Record Archive File Performance with Bulk Read and Write Processing

You can configure to cache the bulk processing of network accounting record (NAR) data. By default, each intermediate NAR archive file is created and read line by line by each processing node. When the Enable bulk read/write check box is selected in a node configuration, each NAR archive file is read and stored in memory for processing by the respective node, and the NAR output files are written into memory before writing to the cache output file, reducing the processing time.

The bulk read and write function:

Reads and stores each intermediate NAR archive file as a whole into memory for processing by the respective node.
Writes the NAR output file as a whole into memory, until either the maximum number of NARs for a file is reached or the idle write time has expired, before writing the output data to a cache file.

Bulk read and write operations process and pass data records using memory, reducing the processing time.

Note:

If you have system memory constraints, use the Enable bulk read/write function only for your critical path node chains.

Discarding Records

You can use the following EP nodes to discard records:

Use the Discarding EP node to discard records. You can discard records based on a value or by date.
Use the Record Filter EP node to delete records according to multiple selection criteria. Filtering is based on:
- CDR Type: G-CDR, S-CDR, M-CDR, S-SMO-CDR, S-SMT-CDR·
- List of APNs
- List of SMS-Cs
  
  The Record Filter EP node also suppresses certain optional fields if they are not required.

3 Configuring Node Chains

Configuring Node Chain Input

Configuring Input from BRM

Configuring Node Chain Output

Managing Sequencing

About Record-Level Sequencing

About File-Level Sequencing

Customizing the Sequencing File Name Syntax

Managing Duplicate Records

About Removing Duplicate CDR Files

About Removing Duplicate Records

About Aggregation

About Time-Based Aggregation of CDRs

About Volume-Based Aggregation of CDRs

About Mapping and Enrichment

Record Enhancement for BRM Charging

Filtering Records Based on Data

Restoring Excluded Records

Adding Data to Records

Routing Records Based on Data

File and Record Processing Options

Processing CDRs at File Level

Configuring the File-Level Transaction Threshold

Viewing the File-Level Transaction Performance

Improving EP Performance by Using Multi-Threaded Processing

Improving CC Node Processing Performance with Multi-Threaded Processing

Improving Record Archive File Performance with Bulk Read and Write Processing

Discarding Records