This chapter provides an overview of the roaming incollect and outcollect processes for generating and processing roaming usage data for NRTRDE (Near Real-Time Roaming Data Exchange)
Note:
The NRTRDE feature replaces the High Usage Reports (HUR) feature.Before reading this document, you should be familiar with the following:
Pipeline rating. See "About Pipeline Rating" in BRM Configuring Pipeline Rating and Discounting.
Rating roaming events. See "About Rating Roaming Events".
In NRTRDE, roaming usage data records are transferred in TD35 file format. A TD35 record is structurally similar to a TAP (Transfer Account Procedure) record and contains basic call details (without the charging information), to allow HPMN (Home Public Mobile Network) operators to quickly analyze the usage data. The actual charge information is still exchanged using TAP records.
Generally, the Visited Public Mobile Network (VPMN) operator generates the TD35 files and sends them to the HPMN operator. The HPMN operator processes the TD35 files and analyzes the roaming usage data to determine any fraudulent activity.
Roaming partners exchange roaming usage data in near real time by sending the data in TD35 file format. Pipeline Manager handles TD35 files as follows:
TD35 files are created during roaming outcollect processing by the outcollect rating pipeline. See "About Generating Roaming Usage Data for NRTRDE" for more information.
TD35 files are processed during roaming incollect processing by the NRTRDE processing pipeline. See "About Processing Roaming Usage Data for NRTRDE" for more information.
Figure 14-1 Generating Roaming Usage Data (Outcollect Processing)
Figure 14-1 shows a high-level overview of how roaming usage data is generated for NRTRDE during roaming outcollect processing:
CDRs arrive at the splitter pipeline.
The outcollect splitter pipeline converts incoming call data records (CDRs) into event data record (EDR) format and separates the EDRs into home subscribers' EDRs and visiting subscribers' roaming EDRs.
Home subscribers' EDRs are sent to the normal rating pipeline to be rated and to the Rated Event Loader.
Visiting subscribers' roaming EDRs are sent to the outcollect rating pipeline to be rated. The outcollect rating pipeline rates the visiting subscribers' roaming EDRs and generates TAP files and TD35 files for each roaming partner. A copy of the visiting subscribers' roaming EDRs is sent to the outcollect settlement pipeline from where they are sent to the rated event loader.
The rated event loader send the data to be recorded in the BRM database.
See "About Processing Visiting Subscribers' Roaming Usage" and "Setting Up Pipeline Manager for NRTRDE" for more information.
Figure 14-2 Generating Roaming Usage Data (Incollect Processing)
The following is a high-level overview of how roaming usage data is processed for NRTRDE during roaming incollect processing (shown in Figure 14-2):
Incoming TD35 files are sent to the NRTRDE processing pipeline.
The input module uses NRTRDE input grammar to convert the TD35 records into EDR format and performs file and record validations based on the TD35 specifications.
For each TD35 record that fails validation, an NRTRDE error record is created in the Pipeline Manager database. Information stored in the error records is used in generating NRTRDE Error reports. See "About NRTRDE Reports".
For each TD35 file processed, a file processing record is created in the Pipeline Manager database. Information stored in the file processing records is used in generating NRTRDE File Delivery reports. See "About NRTRDE Reports".
During roaming incollect processing, the NRTRDE processing pipeline creates NRTRDE error records and NRTRDE file processing records in the Pipeline Manager database. The information stored in these records is used to generate NRTRDE reports. See "Creating NRTRDE Reports" for more information.
Important:
You must have Roaming Manager installed and configured before you configure Pipeline Manager for NRTRDE. See "Installing TAP Roaming Manager" and "Setting Up Roaming for TAP" for more information.Roaming usage data is transferred in TD35 file format between network operators for NRTRDE. To set up Pipeline Manager for NRTRDE, do the following:
Create NRTRDE tables in the BRM database. See "Creating NRTRDE Tables in the BRM Database" for more information.
Set up Pipeline Manager for generating roaming usage data. See "Setting Up Pipeline Manager for Generating Roaming Usage Data for NRTRDE" for more information.
Set up Pipeline Manager for processing roaming usage data. See "Setting Up Pipeline Manager for Processing Roaming Usage Data for NRTRDE" for more information.
NRTRDE File Delivery and NRTRDE Error reports use the following tables located in the BRM database:
NRTRDE_FILES
NRTRDE_ERRORS
NRTRDE_SEQUENCES
Create these tables in your BRM database by manually running each of the SQL commands in the NRTRDE_Tables.sql script. This script is located in the Pipeline_ home/database/Oracle/Scripts directory, where Pipeline_ home is the directory where the Pipeline Manager is installed.
Note:
Before you set up Pipeline Manager for generating roaming usage date for NRTRDE, you must set up Pipeline Manager for roaming outcollect processing. See the discussion of setting up Pipeline Manager for TAP Outcollect processing in the BRM documentation.TD35 files are generated during roaming outcollect processing by the outcollect rating pipeline. To configure roaming outcollect processing for TD35 file generation, do the following:
Limit the number of records in a TD35 file. See "Limiting the Number of Records by Configuring a Batch Controller" for more information.
Configure a sequence generation for NRTRDE files. See "Configuring a Sequence Generation for NRTRDE Files" for more information.
Include descriptions for NRTRDE stream format. See "Including NRTRDE Stream Format Descriptions" for more information.
Provide the required EDRs to roaming partner NRTRDE output streams. See "Providing Required EDRs to Roaming Partner NRTRDE Output Streams" for more information.
Configure an NRTRDE output stream for each roaming partner in the outcollect rating pipeline. See "Configuring NRTRDE Output Stream for Roaming Partners" for more information.
One of the requirements for NRTRDE is to limit the number of records in a TD35 file to a maximum of 1000 records.
When you have files with a large number of records, use the SplitSol42.sh script to split the files into multiple files and limit the number of records to a maximum of 1000 per TD35 file. You must run SplitSol42.sh on the output files generated by the outcollect splitter pipeline.
See "Controlling Batch Operations" in BRM System Administrator's Guide.
Configure a batch controller to invoke the SplitSol42.sh script as in the following example:
SplitSol42.sh source_directory target_directory
where
source_directory is the source directory
target_directory is the input directory of the outcollect rating pipeline.
SplitSol42.sh splits the files in source_directory and puts the new files in target_directory. If a source file contains less than 1000 records, SplitSol42.sh moves the file to the target_directory.
Configure sequence generation for NRTRDE files as follows:
Using Pricing Center, define a sequence generator for each roaming partner by entering a name and a unique sequence key. See the description on defining a sequence generation in BRM Pricing Center Online Help.
Configure each sequencer by editing the SequencerPool registry entries of the roaming registry (Pipeline_home/conf/roaming.reg), where Pipeline_home is the directory where Pipeline Manager is installed.
Set the SequencerInstance name to the name of the sequence generator defined in the (step 1) Pricing Center.
Set the SequencerType registry entry to Generation.
The following sample configuration shows two sequence generators for HPMN operators hpmn01 and hpmn02.
SequencerPool { SEQ_GEN_NRTRDEOUT_hpmn01 { Source = Database Controller { SequencerType = Generation ReuseGap = True SequenceLength = 7 DatabaseConnection = ifw.DataPool.Login } } SEQ_GEN_NRTRDEOUT_hpmn02 { Source = Database Controller { SequencerType = Generation ReuseGap = True SequenceLength = 7 DatabaseConnection = ifw.DataPool.Login } } }
Edit the outcollect rating pipeline DataDescription registry entries to include the NRTRDE stream format description as follows:
DataDescription { StreamFormats { SOL42 = ./formatDesc/Formats/Solution42/SOL42_V670_REL_FORINPUT.dsc TAP3 = ./formatDesc/Formats/TAP3/TAP3_v12_Blocks.dsc NRTRDE2 = ./formatDesc/Formats/TAP3/NRTRDE2_v01_Blocks.dsc SUSPENSE_CREATE_OUTPUT = ./formatDesc/Formats/SuspenseHandling/SuspendedUsageCreation.dsc SUSPENSE_UPDATE_OUTPUT = ./formatDesc/Formats/SuspenseHandling/SuspendedUsageUpdate.dsc } InputMapping { SOL42 = ./formatDesc/Formats/Solution42/SOL42_V670_REL_InMap.dsc } OutputMapping { SUSPENSE_CREATE_OUTPUT = ./formatDesc/Formats/SuspenseHandling/SuspendedUsageCreationMapping.dsc SUSPENSE_UPDATE_OUTPUT = ./formatDesc/Formats/SuspenseHandling/SuspendedUsageUpdateMapping.dsc } }
The ISC_NRTRDE_EventSplit iScript is provided for use by roaming outcollect processing to duplicate and route EDRs to the corresponding roaming partner NRTRDE output streams based on the SOURCE_NETWORK EDR field.
Configure ISC_NRTRDE_EventSplit in the outcollect rating pipeline. For example,
NRTRDE_EventSplit { FileName = ./iScriptLib/iScriptLib_Roaming/ISC_NRTRDE_EventSplit.isc NRTRDE_STREAM_PATTERN = NRTRDEOutput }
See the description for ISC_NRTRDE_EventSplit in BRM Configuring Pipeline Rating and Discounting.
Configure an NRTRDE output stream for each roaming partner in the outcollect rating pipeline. For each roaming partner, configure the OUT_GenericStream module registry entries as follows:
Set the Grammar registry entry to the NRTRDE output grammar description file.
Grammar = ./formatDesc/Formats/TAP3/NRTRDE2_v01__OutGrammar.dsc
Set the Sequencer registry entry to the name of the sequence generator for generating the sequence number for this roaming partner. See "Configuring a Sequence Generation for NRTRDE Files", described earlier.
Set the Sender registry entry to the VPMN ID sending the TD35 file.
Configure the EXT_OutFileManager registry entries to specify the output file information. See "EXT_OutFileManager" in BRM Configuring Pipeline Rating and Discounting.
The following example shows output stream configurations for HPMN operators hpmn01 and hpmn02:
hpmn01NrtrdeOutput { ModuleName = OUT_GenericStream ... Module { Grammar = ./formatDesc/Formats/TAP3/NRTRDE2_v01_OutGrammar.dsc DeleteEmptyStream = False Sequencer = SEQ_GEN_NRTRDEOUT_hpmn01 Sender = PORTL Recipient = hpmn01 OutputStream { ModuleName = EXT_OutFileManager Module { OutputPath = ./data/outcollect/nrtrdeout/hpmn01 OutputPrefix = NRPORTLhpmn01 TempPrefix = tmptest_hpmn01_ TempDataPath = ./data/outcollect/nrtrdeout/hpmn01 TempDataPrefix = test.hpmn01.tmp. TempDataSuffix = .data UseInputStreamName = [0,0] SequencerPrefix = "" AppendSequenceNumber = True } } } } hpmn02NrtrdeOutput { ModuleName = OUT_GenericStream ... Module { Grammar = ./formatDesc/Formats/TAP3/NRTRDE2_v01_OutGrammar.dsc DeleteEmptyStream = False Sequencer = SEQ_GEN_NRTRDEOUT_hpmn02 Sender = PORTL Recipient = hpmn02 OutputStream { ModuleName = EXT_OutFileManager Module { OutputPath = ./data/outcollect/nrtrdeout/hpmn02 OutputPrefix = NRPORTLhpmn02 TempPrefix = tmptest_hpmn02_ TempDataPath = ./data/outcollect/nrtrdeout/hpmn02 TempDataPrefix = test.hpmn02.tmp. TempDataSuffix = .data UseInputStreamName = [0,0] SequencerPrefix = "" AppendSequenceNumber = True } } } }
TD35 files are processed during roaming incollect processing by the NRTRDE processing pipeline.
To configure the NRTRDE processing pipeline, do the following:
Note:
For an example of NRTRDE processing pipeline configuration, see Pipeline_home/conf/roaming.reg, where is Pipeline_home the directory where Pipeline Manager is installed.Configure the NRTRDE processing pipeline DataDescription registry entries by setting the stream format as shown in this code sample:
DataDescription { StreamFormats { NRTRDE2 = ./formatDesc/Formats/TAP3/NRTRDE2_v01_Blocks.dsc } InputMapping { } OutputMapping { } }
Configure the NRTRDE processing pipeline input processing as follows:
Set the Grammar registry entry for the INP_GenericStream input module to the NRTRDE input grammar description file with the following command:
Grammar = ./formatDesc/Formats/TAP3/NRTRDE2_v01_InGrammar.dsc
Configure the EXT_InFileManager registry entries to specify information about the NRTRDE input file. See "EXT_InFileManager" in BRM Configuring Pipeline Rating and Discounting.
The following example shows input processing for the NRTRDE processing pipeline.
InputModule { ModuleName = INP_GenericStream Module { DefaultOutput = DevNull Grammar = ./formatDesc/Formats/TAP3/NRTRDE2_v01_InGrammar.dsc InputStream { ModuleName = EXT_InFileManager Module { InputPath = ./data/incollect/nrtrdein/in InputPrefix = NR #InputPrefix = test_ #InputSuffix = .edr DonePath = ./data/incollect/nrtrdein/done DonePrefix = test_NRTRDEInProcessingPipeline_ DoneSuffix = .done ErrorPath = ./data/incollect/nrtrdein/error ErrorPrefix = test_NRTRDEInProcessingPipeline_ ErrorSuffix = .err TempPrefix = temp_NRTRDEInProcessingPipeline_ Replace = True InputDirEmptyTimeout = 10 } } } }
Configure the ISC_NrtrdeHeaderValidation_v2_01 iScript. For more information on ISC_NrtrdeHeaderValidation_v2_01, see BRM Configuring Pipeline Rating and Discounting.
Configure the ISC_NRTRDE_ErrorReport iScript. For more information on ISC_NRTRDE_ErrorReport, see BRM Configuring Pipeline Rating and Discounting.
Roaming NRTRDE reports include the following:
Use the NRTRDEReportGen64 utility to generate the "NRTRDE Delivery Report" and the "NRTRDE Error Report".
To run the NRTRDE reports, you use the NRTRDEReportGen64 utility. You can generate the reports for a specific VPMN operator or for all VPMNs. See NRTRDEReportGen64.
BRM provides the following iScripts for use with NRTRDE:
ISC_NRTRDE_EventSplit. This iScript is used by roaming outcollect processing to duplicate and route EDRs to the corresponding roaming partner NRTRDE output streams based on the SOURCE_NETWORK EDR field. For more information on the ISC_NRTRDE_EventSplit iScript, see BRM Configuring Pipeline Rating and Discounting.
ISC_NrtrdeHeaderValidation_v2_0. This iScript is used during roaming incollect processing by the NRTRDE processing pipeline. ISC_NrtrdeHeaderValidation_v2_01 validates the information in the header record of the TD35 file based on the TD35 specifications. For more information on the ISC_NrtrdeHeaderValidation_v2_01 iScript, see BRM Configuring Pipeline Rating and Discounting.
ISC_NRTRDE_ErrorReport. This iScript is used during roaming incollect processing by the NRTRDE processing pipeline. It collects the validation errors in the EDRs and creates error records in the Pipeline Manager database. ISC_NRTRDE_ErrorReport also collects NRTRDE file processing information and creates file processing records in the Pipeline Manager database. The information stored in the validation and file processing records in the database are used for generating NRTRDE reports. For more information on the ISC_NRTRDE_ErrorReport iScript, see BRM Configuring Pipeline Rating and Discounting.
The NRTRDE File Delivery report file lists all the TD35 files processed by the HPMN. You can generate the report for a specific VPMN operator or for all VPMNs.
Each NRTRDE File Delivery report includes a header record, one or more detail records, and a trailer record.
The header record includes the following data:
ND,Version,HPMNId,VPMNId,SeqNo,CreationStartTime,PeriodStartTime,PeriodEndTime
where:
ND. NRTRDE File Delivery report identification.
Version. Report version number.
HPMNId. Home Public Mobile Network identifier.
VPMNId. Visited Public Mobile Network identifier.
SeqNo. Report file sequence number.
CreationStartTime. Report creation date and time in the format yyyymmddhhmmss.
PeriodStartTime. Report period start date and time in the format yyyymmddhhmmss.
PeriodEndTime. Report period end date and time in the format yyyymmddhhmmss.
The detail records include the following data:
FileName,ReceivedTime
where:
FileName. Name of the TD35 file in the format
NRxxxxxyyyyySeqNo:
where
NR identifies the file as containing NRTRDE information.
xxxx identifies the Sender TADIG Code in uppercase and five characters long.
yyyy identifies the Recipient TADIG Code in uppercase and five characters long.
SeqNo identifies the file sequence number.
ReceivedTime. The time the TD35 file was received by HPMN, in the format yyyymmddhhmmss, (described earlier).
The trailer record includes the following data:
END,NumberOfRecords
where,
END. End of file marker
NumberOfRecords. Number of total records in the report including, header and trailer records.
The following example shows the output of a NRTRDE File Delivery report for HPMN XYZ99 and VPMN ABC01.
ND,1.0,XYZ99,ABC01,00001,20070215020000-0500,20070214000000-0500,20070214235959-0500 NRABC01XYZ990000001,20070214010000-0500 NRABC01XYZ990000002,20070214040000-0500 NRABC01XYZ990000003,20070214070000-0500 NRABC01XYZ990000004,20070214100000-0500 NRABC01XYZ990000005,20070214130000-0500 NRABC01XYZ990000006,20070214160000-0500 NRABC01XYZ990000007,20070214190000-0500 NRABC01XYZ990000008,20070214220000-0500 END,10
NRTRDE File Delivery Report File
NRTRDEReportGen64 generates the NRTRDE File Delivery report and sends the output to a file with the following file name format:
NDxxxxxyyyyySeqNo:
where
ND identifies the file as containing NRTRDE File Delivery Report.
xxxx identifies the Sender TADIG Code in uppercase and five characters long.
yyyy identifies the Recipient TADIG Code in uppercase and five characters long.
SeqNo identifies the report file sequence number.
The NRTRDE Error report lists the TD35 records that failed the NRTRDE incollect validation process. You can generate the report for a specific VPMN (Visited Public Mobile Network) operator or for all VPMNs. You send this report to the VPMN for corrections.
Each NRTRDE Error report includes a header record, one or more detail records, and a trailer record.
The header record includes the following data:
NE,Version,HPMNId,VPMNId,SeqNo,CreationStartTime,PeriodStartTime,PeriodEndTime
where:
NE. NRTRDE Error report identification.
Version. Report version number.
HPMNId. Home Public Mobile Network identifier.
VPMNId. Visited Public Mobile Network identifier.
SeqNo. Report file sequence number.
CreationStartTime. Report creation date and time in the format yyyymmddhhmmss.
PeriodStartTime. Report period start date and time in the format yyyymmddhhmmss.
PeriodEndTime. Report period end date and time in the format yyyymmddhhmmss.
The detail records include the following data:
FileName,RecordNo,RecordType,ErrorCode
where:
FileName. Name of the TD35 file, that contains the TD35 record that failed, in the format
NRxxxxxyyyyySeqNo:
where
NR identifies the file as containing NRTRDE information.
xxxx identifies the Sender TADIG Code in uppercase and five characters long.
yyyy identifies the Recipient TADIG Code in uppercase and five characters long.
SeqNo identifies the file sequence number.
RecordNo. TD35 record number.
RecordType. The record type of RecordNo.
ErrorCode. TD35 validation error code.
The trailer record includes the following data:
END,NumberOfRecords
where,
END. End of file marker
NumberOfRecords. Number of total records in the report including, header and trailer records.
The following example shows the output of a NRTRDE Error report for HPMN XYZ99 and VPMN ABC01.
NE,1.0,XYZ99,ABC01,00001,20070215020000-0500,20070214000000-0500,20070214235959-0500 NRABC01XYZ990000004,,,400 NRABC01XYZ990000005,123,MOC,315 NRABC01XYZ990000005,145,MOC,315 NRABC01XYZ990000005,167,MTC,315 NRABC01XYZ990000005,753,GPRS,304,307,308 NRABC01XYZ990000006,,,117 END,8
NRTRDEReportGen64 generates the NRTRDE Error report and sends the output to a file with the following file name format:
NExxxxxyyyyySeqNo:
where
NE identifies the file as containing NRTRDE Error Report.
xxxx identifies the Sender TADIG Code in uppercase and five characters long.
yyyy identifies the Recipient TADIG Code in uppercase and five characters long.
SeqNo identifies the report sequence number.
Use the NRTRDEReportGen64 utility to generate the "NRTRDE Delivery Report" and the "NRTRDE Error Report".
Pipeline_home/bin
where, Pipeline_home is the directory where Pipeline Manager is installed.
NRTRDEReportGen64 database_access_library server_name user_name password database_name hpmn [vpmn]
The parameters for this utility are:
database_access_library. The database access library. For example, liboci10g6312d.a for Oracle on AIX.
server_name. The database alias of the host machine running the Pipeline Manager database.
user_name. The user name to log into the Pipeline Manager database.
password. The password associated with user_name.
database_name. The database ID of the Pipeline Manager database.
hpmn. The Home Public Mobile Network Id.
vpmn. The Visited Public Mobile Network Id. If this option is not specified, the utility generates reports for all VPMN operators.