Skip Headers
Oracle® Communications Billing and Revenue Management Setting Up Pricing and Rating
Release 7.5

E16711-09
Go to Documentation Home
Home
Go to Table of Contents
Contents
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

22 Loading Events from External Sources

This chapter describes how to load event data from event log files into the Oracle Communications Billing and Revenue Management (BRM) database to be rated. It also provides troubleshooting information for loading events.

For general information about importing events, see "About Rating Events Created by External Sources".

About Loading Events

You load data from an event log file into the BRM database by creating an event import template with the Universal Event Mapper (UE Mapper). You then load the event data manually by using the Universal Event (UE) Loader or automatically by using the Batch Controller and a batch handler.

The UE Loader is a command-line utility that reads data about service usage from the event log file, reads the event import template for instructions on loading the data, such as which opcodes to call to load the data, and loads the data into the BRM database. When the events are loaded, BRM rates the events and changes customer account balances. You can set the Batch Controller and create a UE Loader batch handler to run the UE Loader automatically (to load event data on a regular basis). For information about the Batch Controller, see "Controlling batch operations" in BRM System Administrator's Guide.

For information on creating an event import template and a technical overview of the event import process, see "About Rating Events Created by External Sources".

Figure 22-1 shows how the UE Mapper and UE Loader work to load events into the BRM database:

Figure 22-1 Loading Events into BRM Using the UE Mapper and UE Loader

Description of Figure 22-1 follows
Description of "Figure 22-1 Loading Events into BRM Using the UE Mapper and UE Loader"

Configuring the UE Loader

To configure the UE Loader, edit the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties). Follow these procedures:

Configuring Connection Information for the UE Loader

To specify connection information:

  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Follow the instructions in the file to set the standard connection information, such as the Connection Manager (CM) name and port number.

  3. Save and close the file.

Enabling UE Loader to Convert Flists to Strings

Important:

This is a mandatory configuration.

Ensure that UE Loader is configured to convert flists to strings:

  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Ensure that the infranet.flist.tostring.enable entry is set to true:

    infranet.flist.tostring.enable = true 
    

    If this entry is not present or is set to false, UE Loader might load records into the wrong accounts.

Specifying the Event Log File Location

When you run UE Loader, you specify only the name of the event log file, not the file location. You specify the file location in the Infranet.properties file.

For information about UE Loader log files, see "Troubleshooting Event Loading".

  1. Open the properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Enter the directory name in the infranet.uel.event_log_file_location entry. This is the directory that stores event log files for loading.

    When specifying the path, always use forward slashes (/), and type a slash after the last directory name.

    infranet.uel.event_log_file_location=/var/portal/7.3.1/apps/uel/ 
    
  3. Save and close the file.

Specifying Whether to Log Duplicate Event Log Files

By default, UE Loader does not notify you when a duplicate event log file is loaded into the database.

You can configure UE Loader to log an error when a duplicate event log file is loaded by using the infranet.uel.duplicate_check entry. When this entry is set to True, UE Loader creates a DuplicateFiles.txt file and adds the following for each duplicate file:

  • The duplicate event log file's name

  • The timestamp of when the duplicate event log file was processed

To log an error when duplicate event log files are loaded into the database:

  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Change the infranet.uel.duplicate_check entry to True:

    parameter infranet.uel.duplicate_check=True 
    
  3. Save and close the file.

Setting the Number of Threads for Performance

You can improve UE Loader performance by changing the maximum number of threads. Running multiple threads loads multiple events simultaneously, which gives faster performance but puts a greater load on the BRM system. The number of threads that you specify depends on your BRM system configuration, for example, the number and speed of CPUs.

Note:

The number of threads used when processing also depends on the event import template that you use to load the events. For example, if you load events based on their occurrence in the event log file, they are processed one at a time, so using multiple threads does not improve performance. For more information, see "Specifying How to Load Events".
  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Change the infranet.uel.max_threads entry, for example:

    infranet.uel.max_threads=5 
    
  3. Save and close the file.

    Tip:

    You can run multiple UE Loaders to improve performance. Each UE Loader can run multiple threads. If you have only one CM and one DM, however, there is a limit to how much you can improve performance by using multiple UE Loaders.

Setting Memory Options

Avoid memory errors with these guidelines:

  • UE Loader can process large log files, such as files with 200,000 records. The memory required to load events depends on the number of records in your event log file.

  • If you have a large number of records and the UE Loader reports an out of memory error (error code -2), change the infranet.uel.queue_size entry in the Infranet.properties file. Setting this entry causes the UE Loader to read records into memory in sizable chunks rather than reading all records into memory at once.

    Important:

    If the infranet.uel.queue_size entry in the Infranet.properties file is set to 0, all records are read into memory at once.
  • You can configure the memory available to UE Loader by modifying the -mx setting in the uel script.

Specifying How to Load Events

You can load events based on their occurrence in the event log file, or you can load them based on their account identifier. Specify the loading method when you create the event import template by using the Group events by account identifier option in UE Mapper. For more information, start Developer Center and refer to the UE Mapper Help.

Important:

If a custom bill item is rated cumulatively, UE Loader must group events by account identification for loading. For more information, see UE Mapper Help.

Configuring Log File Locations for the UE Loader

To specify locations of log files:

  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Enter directory names in the following entries:

    • To set the location of the error file that records errors found in event loading mode, use the infranet.uel.load_error_file_location entry.

    • To set the location for the record of successfully loaded events, use the infranet.uel.load_success_file_location entry.

    • To set the location for the standard BRM client error log file, use the infranet.uel.error_log_file_location entry.

    • To set the location for the file that records events you filter by event field values, use the infranet.uel.filter_log_file_location entry.

    When specifying the path, always use forward slashes (/), and type a slash after the last directory name.

    infranet.uel.error_log_file_location=/var/portal/7.3.1/apps/uel/ 
    
  3. Save and close the file.

Specifying the Maximum Load Errors to Process before Quitting

The maximum number of errors applies to errors generated from all threads, not errors per thread.

  1. Open the properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Set the infranet.uel.max_errors entry, for example:

    infranet.uel.max_errors=100 
    
  3. Save and close the file.

Specifying the Directory to Store Logged Event Records

You can set up filters so that particular records are logged to a file, whether you load them or not. See "About Discarding Event Records Based on Record Content" and "About Logging Event Records Based on Record Content".

To specify the directory to store the files that log filtered records:

  1. Open the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

  2. Set the infranet.uel.filter_log_file_location entry.

  3. Save and close the file.

Specifying the Default Date/Time Format for Timestamps

You can specify a timestamp format to support loading events from different countries that format dates and times in different ways. The timestamp format must match the date/time format in the event log file that you are importing. The timestamp format is used by fields with the data type PIN_FLDT_TSTAMP.

You can specify the date/time format for each timestamp field (fields with the data type PIN_FLDT_TSTAMP) in your event log file when you create the event import template. This is necessary if you have records that use different timestamp formats in the same event log file.

If you do not specify the date/time format for a record on a per field basis when you create the event import template, UE Loader uses the date/time format specified in its Infranet.properties file as the default format.

To set the default date/time format:

  1. Determine the date format by looking at the event log file you are importing.

  2. Open the properties file (BRM_Home/apps/uel/Infranet.properties).

  3. Change the infranet.uel.date_pattern entry to match the date format in the event log file you are importing. For example. if your records use a time format specified in a.m./p.m. hours (01 - 12), you specify hours by using a lowercase h and including the a parameter to indicate that the time is in a.m./p.m. format:

    infranet.uel.date_pattern=dd/MMM/yyyy:hh:mm:ss a zzzz 
    

    See "Specifying Time Format Syntax".

  4. Save and close the file.

Specifying Time Format Symbols

Table 22-1 shows examples of date formats:

Table 22-1 Example Date/Time Formats

Date in Event Log File Date/time Format in Infranet.properties File

26/October/2005:10:55:49 Pacific Standard Time

dd/MMM/yyyy:hh:mm:ss a zzzz

12:08 PM

h:mm a

12 o'clock PM, Pacific Daylight Time

hh 'o''clock' a, zzzz

2005.07.10 AD at 15:08:56 PDT

yyyy.MM.dd G 'at' hh:mm:ss z

Wed, July 10, '05

EEE, MMM d, ''yy

0:00 PM, PST

K:mm a, z

2005.July.10 AD 12:08 PM

yyyyy.MMMMM.dd GGG hh:mm aaa


Specifying Time Format Syntax

Each element in the time format syntax is represented by a character. For example, the year is represented by the "y" character. You control how each element is matched by the number of characters you use. For example, with the year 2001:

  • yy matches 01

  • yyyy matches 2001

There are also different symbol types, such as Text and Number. The number of characters has a different meaning for different symbol types as shown in Table 22-2:

Table 22-2 Time Format Syntax

Symbol Type Rules for Number of Characters

Text

4 characters or more uses the full word. Fewer than 4 characters presents an abbreviation. For example, for E (day of the week):

  • E = Wed

  • EEEE = Wednesday

Number

Any number of characters uses the minimum required number of digits, except for years. With years, 2 characters uses the short version, for example:

yy = 99

Text and number

3 characters or more uses text, 1 or 2 characters uses numbers. For example:

  • MMM = July

  • MM = 07


In addition, you can use an escape character to match text as shown in Table 22-3:

Table 22-3 Use of Escape Character to Match Text

Log File Date/Time Symbols

July 2 at 12:01

MMM d 'at' hh:mm


To match a single quotation mark, use double-single quotation marks as a literal, as shown in Table 22-4:

Table 22-4 Use of Quotation Marks to Match Text

Log File Date/Time Symbols

July 2 ''99

MMM d ''yy


Any characters outside the range of a-z and A-Z, such as colon (:), semicolon (;), and at symbol (@), are treated as quoted text, even if they are not enclosed in single quotation marks.

Important:

Characters within the ranges of a-z and A-Z that are not used as symbols, such as C, and are not enclosed in quotes, cause an error.

Allowing for Leniency in Time References

When a CDR contains a date value that uses a +1300 time reference, the system cannot read the date properly and returns an error. For example, the date entry might look like this: 20061202000000+1300.

To enable the system to process CDRs that use the +1300 time zone reference, set the Infranet.uel.lenient value to true in the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties) The default value is false:

Infranet.uel.lenient = true

Configuring UE Loader for International Log Files

This section provides instructions for configuring UE Loader to use international log files.

Configuring the UE Loader Infranet.properties File

UE Loader supports English, French, German, and Japanese log files. To support these languages, set the following parameters in the UE Loader Infranet.properties file (BRM_Home/apps/uel/Infranet.properties) shown in Table 22-5:

Table 22-5 Parameters in UE Loader File

Infranet.properties File Entry Description

infranet.uel.event_log_file_lang_code

Specify the language code. See "Language, Country, and Encoding Codes".

infranet.uel.event_log_file_country_code

Specify the country code. UE Loader supports all country codes that apply to each language (English, French, German, and Japanese).

infranet.uel.event_log_file_variant_code

Specify the variant code.

infranet.uel.event_log_file_encoding

Specify the encoding code. See "Language, Country, and Encoding Codes".

infranet.uel.date_pattern

Specify the date pattern for the locale. See "Specifying the Default Date/Time Format for Timestamps".


Language, Country, and Encoding Codes

Listed below are the language, country, and encoding codes for languages supported by each operating system.

Note:

UE Loader supports all country codes for each language.

Table 22-6 lists the system encoding support for English.

Table 22-6 System Encoding Support for English

Operating System Language Code Country Codes Encoding Code

Solaris

en

AU, CA, IE, NZ, ZA, GB, and US

8859-1

Linux

en

AU, CA, IE, NZ, ZA, GB, and US

None

HP-UX IA64

en

AU, CA, IE, NZ, ZA, GB, and US

ISO81

AIX

en

AU, CA, IE, NZ, ZA, GB, and US

ISO8859-1


Table 22-7 lists the system encoding for Japanese.

Table 22-7 System Encoding Support for Japanese

Operating System Language Code Country Codes Encoding Code

Solaris

ja

JP

SJIS

Linux

ja

JP

None

HP-UX IA64

ja

JP

sjis

AIX

ja

JP

IBM-943


Table 22-8 lists the system encoding support for German.

Table 22-8 System Encoding Support for German

Operating System Language Code Country Codes Encoding Code

Solaris

de

AT, DE, LU, and CH

8859-1

Linux

de

AT, DE, LU, and CH

None

HP-UX IA64

de

AT, DE, LU, and CH

ISO81

AIX

de

AT, DE, LU, and CH

ISO8859-1


Table 22-9 lists the system encoding for French.

Table 22-9 System Encoding Support for French

Operating System Language Code Country Codes Encoding Code

Solaris

fr

BE, CA, FR, LU, and CH

8859-1

Linux

fr

BE, CA, FR, LU, and CH

None

HP-UX IA64

fr

BE, CA, FR, LU, and CH

ISO81

AIX

fr

BE, CA, FR, LU, and CH

ISO8859-1


Loading Events

You can load events manually by running UE Loader from a command line or you can schedule UE Loader to run automatically. See:

About the Order of Events

If any event balance impacts are configured to start immediately and never expire, consider the order of events when you load them.

When an event is loaded, BRM checks the validity periods of the account's resource sub-balances. The sub-balance that has a start time earlier than the event's start time is the one that receives the impact. If there is no sub-balance that has an earlier start time, BRM creates a new sub-balance, sets its start time to the event's start time, and impacts that sub-balance. This can cause the creation of multiple sub-balances for the same resource when events are loaded out of order. Having multiple sub-balances for the same resource can impact performance.

For more information, see "How Resources in Validity-Based Sub-Balances Are Updated".

To prevent the creation of multiple account sub-balances for resources that start immediately and never expire, be sure to sort the events in ascending order based on their timestamp before running UE Loader.

Loading Events Manually

Use the Universal Event (UE) Loader to manually load events from data log files into the BRM database. The UE Loader is a command-line utility installed with BRM, and its executable file is in BRM_Home/apps/uel. For information on UE Loader command-line syntax, see "Universal Event Loader".

For information on loading events from data log files automatically, see "Loading Events Automatically".

Before You Begin

  • You must configure the UE Loader before you can load events. See "Configuring the UE Loader".

  • To avoid memory errors, follow the guidelines in "Setting Memory Options".

  • An event import template is required to load event data. You must know the exact name of the event import template that was created for the event log file you are loading.

  • Before loading events, you must specify the date/time format used in the event log file. You can specify the date/time format for all records in your log file by editing the UE Loader Infranet.properties file. See "Specifying the Default Date/Time Format for Timestamps". The date/time format for each record can also be specified on a per field basis when the event import template is created. If the date/time format is not specified on a per field basis, the date/time format specified in the Infranet.properties file is used.

  • Before loading events, run the UE Loader program on a test database to test the event import template and any custom opcodes that were created for loading your log file events.

Instructions

To load events manually, run the UE Loader by using this syntax:

uel -t template_name [-v] [-m parse|load] [-test] log_file_name

The optional -v verbose parameter causes UE Loader to output error messages or debug messages on the console according to the debug level you set in the Infranet.properties file.

The -m and -test modes are used more during testing of event loading. When you are loading events into a production database, you do not need to use these options.

For example, if the template name is WebServer and the event log file name is webserver.log, use this command to load events:

uel -t WebServer webserver.log 

Notes

  • Do not enter the path of the event log file. You specify the path in the UE Loader Infranet.properties file. See "Specifying the Event Log File Location".

  • The log file can use any file extension or no extension.

For information on UE Loader command-line syntax, see "Universal Event Loader".

Loading Events Automatically

You can load event log files automatically and manually. Some BRM service integration components provide a batch handler that executes UE Loader to process event log files automatically, while BRM system software provides the Batch Controller that controls the handler. You can also write a custom handler. You can load files of event records automatically on a periodic basis by configuring UE Loader, Batch Controller, and a handler.

To set up automatic event loading:

  1. Configure UE Loader. See "Configuring the UE Loader"

  2. Set up a handler to launch UE Loader.

    See the documentation of the service integration component you use for instructions on setting up the handler for that component.

  3. Set up the Batch Controller to do these tasks:

    1. Scan a configured directory for record files from an external source.

    2. Move the files to the appropriate handler for processing.

    See also "Controlling batch operations" in BRM System Administrator's Guide.

Troubleshooting Event Loading

This section explains UE Loader log files and error codes, and provides tips on troubleshooting event loading failures.

Understanding UE Loader Log Files

UE Loader records errors and successes in the appropriate log files in BRM_Home/apps/uel. Table 22-10 describes the UE Loader log files.

Table 22-10 UE Loader Log Files

Type of File Log File Name Description

Cache file

event_log_file_name_cache.xml

The cache file is an intermediate file generated by UE Loader after it parses the event log file and before it loads events into the database. UE Loader stores the results of event log parsing in the cache file, which includes:

  • The event records

  • The customer accounts associated with the events

  • A record number for each event record

The record number appears in the error log so you know which record to fix in the cache file if there are errors during event loading.

Filter log

event_log_file_name_filtered.xml

The filter log file is generated when events are loaded. It lists records you have specified to archive. Records are archived based on filters you set up in the event import template. See "About Logging Event Records Based on Record Content".

Note: The event import template specifies whether filtered records are loaded into the BRM database.

Error log

event_log_file_name_lerr.xml

The error log file is generated when events are loaded. It reports any loading errors and specifies the records in the event log file that caused the errors.

Success log

event_log_file_name_lsucc.xml

The success log file is generated when events are loaded. It lists all records that load into the database successfully.

Application log

uel.pinlog

The standard application log file contains complete descriptions of the errors generated in the other log files.

To log all errors, set the log level to 3.


When errors occur, you typically check the log files in this order:

  1. UE Loader pinlog

  2. Error file

  3. Success file

  4. Cache file

    Check the cache file for NULL account attributes if an account could not be found during event loading.

  5. Filter file

    Check the filter file if you have set up filters for event loading to see that events were loaded and logged according to the filter.

    Note:

    The name that UE Loader generates for its log files includes the extension of the event log file. For example, when the event log file is named CDR3.txt, the cache, filter, error, and success files are called:
    • CDR3.txt_cache.xml

    • CDR3.txt_filtered.xml

    • CDR3.txt_lerr.xml

    • CDR3.txt_lsucc.xml.

Understanding UE Loader Error Codes

Table 22-11 describes the UE Loader error codes that appear in the uel.pinlog, event_log_file_name_lerr.xml file, and event_log_file_name_cache.xml file:

Table 22-11 UE Loader Error Codes

Error Code Brief Description Error Description Possible Required Actions

0

NO_ERROR

All records loaded successfully.

No action needed.

-1

FAILED_HEADER_TRAILER_CHECK

The header record or trailer record check failed.

  • If the content does not match the header and trailer check, check for a file content problem.

  • Use UE Mapper to ensure that the header or trailer check is set up correctly in the event import template.

-2

NOT_ENOUGH_MEMORY

UE Loader ran out of memory.

  • Reconfigure the memory available to UE Loader by modifying the -mx setting in uel script.

  • Reconfigure the queue size in the Infranet.properties file. For details, see "Setting Memory Options".

-3

OUT_OF_DISK_SPACE

UE Loader ran out of disk space while creating log files.

For example, there was insufficient disk space in the specified directory to store UE Loader success, error, or filter log files.

Specify a location for log files that contains more disk space. See "Configuring Log File Locations for the UE Loader".

-4

TEMPLATE_ERROR

UE Loader detected a problem with the event import template.

For example, you used the wrong version of a template (a template that was created with an earlier version of UE Mapper).

To convert an event import template to the latest version, migrate the template in UE Mapper. See the UE Mapper Help for instructions.

-100

BAD_INF_CONNECTION

The BRM system is not available.

For example, the CM is down.

1

FIELD_PARSE_ERROR

An error occurred when parsing the fields in the event log file.

For example, UE Loader detected no matching string delimiter when parsing the event record data.

  1. Open the cache file.

  2. Find the record with the error code 1.

  3. Correct the error in the record.

  4. Save and close the cache file.

  5. Reload the events by using the -m load option. See "Reloading Events That Failed to Load".

2

ACCT_FORMAT_ERROR

When UE Loader constructed the input flist to pass to the opcode that finds the account, it detected an error in the data format.

Possible causes:

  • Incorrect field specification. For example, the login is specified as fields 1 through 5 when the login should be specified as fields 1 through 7.

  • Incorrect setup of the input flist mapping for finding accounts associated with events.

  • If the field specification is incorrect, open the event log file and correct the record.

  • If the input flist mapping for finding accounts is incorrect, use UE Mapper to correct the mapping in the event import template.

3

LOAD_FORMAT_ERROR

When UE Loader constructed the input flist to pass to the opcode that loads the event data into BRM, it detected an error in the data format.

Possible causes are similar to error code 2.

  1. Open the cache file.

  2. Find the record with error code 3.

  3. Correct the error in the record.

  4. Save and close the cache file.

Reload the events by using the -m load option. See "Reloading Events That Failed to Load"

4

PREV_PARSE_ERROR

When loading events, a parse error occurred.

Note: You do not receive this error if you choose to continue loading events when parse errors occur. To configure this, set the infranet.uel.load_on_parse_errors entry in the UE Loader Infranet.properties file. You can also configure a maximum number of parse errors before the UE Loader aborts.

Correct the record in the event log file.

Note: This type of error cannot be corrected in the cache file (for reloading with the -m load option) because the order of events is incorrect. The error must be corrected in the event log file.

5

TOO_MANY_LOAD_ERRORS

The number of load errors exceeded the number of errors you allow. This error is generated when a number of load format errors (code 3) or load opcode errors (code 101) occur.

Note: To set the maximum number of errors, use the infranet.uel.max_load_errors entry in the UE Loader Infranet.properties file.

See the possible actions for error codes 3 and 101.

10

COMMAND_LINE_ARG_ERROR

The command line syntax you used to run UE Loader is incorrect.

See the "Universal Event Loader" utility page for the correct syntax.

11

FILE_NOT_FOUND_ERROR

The event log file you specified in the UE Loader command line cannot be found.

  • Ensure that the location of the event log file is specified correctly in the Infranet.properties file.

  • Use the correct name for the event log file in the UE Loader command.

100

ACCT_NOT_FOUND

The opcode that finds accounts associated with event data could not find an account.

Possible causes:

  • The account does not exist in BRM.

  • The account exists in BRM, but the login or service specified is incorrect.

  • The input flist mapping for finding accounts associated with event data is incorrect.

  • Correct the record in the event log file.

  • Use UE Mapper to ensure that the correct service type associated with the event is specified in the event import template.

  • If the input flist mapping for finding accounts is incorrect, use UE Mapper to correct the mapping in the event import template.

101

LOAD_OPCODE_ERROR

The opcode that loads event data into BRM could not load the data.

  • Open the cache file and correct the record.

  • If the input flist mapping for loading event data is incorrect, use UE Mapper to correct the mapping in the event import template.

111

The template61.dtd file is missing.

The default UE Loader template file (template61.dtd) is not present in the directory from which the UE Loader utility is run.

Copy the template file to the directory from which you run UE Loader. The default template, template61.dtd, is in BRM_Home/apps/uel.


Troubleshooting Tips

Some common causes of failure to load event records into the BRM database:

  • Incorrect connection settings in your UE Loader configuration file.

    See the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties) for instructions on setting each entry.

  • Incorrect format of the event data in your event log files.

    UE Loader expects the records in your event log file to use the format defined in the event import template.

  • Incorrect setup of the event import template used to import the event data.

    UE Loader reads the event import template to obtain instructions for loading the events, so the event import template must specify the correct mapping for the records in your log file.

  • Incorrect time format of the event data.

Specifying the Maximum Load Errors to Process Before Quitting

If you choose to continue loading events when parse errors occur, you can specify the number of errors to allow before the UE Loader quits. To set this maximum number of load errors, use the infranet.uel.max_load_errors entry in the UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

Troubleshooting Connectivity Errors

If you have trouble connecting to the database, ensure that you are using the correct host name, and that the port number, database number, and host name are specified correctly in your UE Loader properties file (BRM_Home/apps/uel/Infranet.properties).

Troubleshooting Preprocessing Errors

The UE Loader reports two types of preprocessing errors:

  • Errors found when reading the event log file.

    These errors include the record number, line number, and error description. These errors are typically the result of the UE Loader finding an unexpected type of data, such as unmatched literal string indicators.

  • Errors in the custom opcode that identifies the account that generated the event.

Some preprocessing errors can be fixed by editing the event import template. Errors in the template are typically fixed by a developer by using the UE Mapper application in Developer Center.

In some cases, an event log file includes a record that has not been generated correctly by the event log file source. In this case, edit the event log file manually to fix the record before loading it.

Troubleshooting Event Loading Errors

Event loading errors include the record number and contents of the record as read from the intermediate cache file. To troubleshoot loading errors, open the uel.pinlog file and read the contents of the error buffer returned by the BRM opcode.

Loading errors are usually the result of the following:

  • Errors in the BRM system, such as an offline Data Manager (DM).

  • Errors in the event import template. Errors in the template are typically fixed by a developer familiar with BRM programming concepts and techniques.

Testing Event Importing

During testing of event importing, note that:

  • UE Loader does not use the time set by the pin_virtual_time utility. If the event start and end times are not obtained from the log file, UE Loader uses the system time.

  • You may want to load a specific event log file more than once.

    If you load events from an event log file successfully, UE Loader generates the log files for it, such as the success log file and the error log file. These log files must be deleted before you can reload events from the same event log file.

Reloading Events That Failed to Load

If events fail to load into the BRM database, UE Loader logs the records in an error log file called event_log_file_name_lerr.xml. Errors are also recorded in the event_log_file_name_cache.xml.

  1. Find the record number of the event record(s) that failed to load by looking in the event_log_file_name_lerr.xml file.

  2. Find the event records that failed to load in the event_log_file_name_cache.xml file by using the record number(s) you found in the error log file.

  3. Fix the errors in the event records.

  4. Reload the events by running UE Loader with the -m load parameter:

    uel -t template_name -m load event_log_file_name
    

UE Loader reads the files generated in the previous run, and ignores previously loaded events.

For more information on the cache file and error file, see "Understanding UE Loader Log Files".

Improving UE Loader Performance

You can improve UE Loader performance by adjusting the number of UE Loader threads and processes and by modifying entries in the UE Loader Infranet.properties file.

Important:

Optimal performance configuration for UE Loader depends on your system configuration such as the number and speed of CPUs and the size of your hard drive and cache memory. To find the UE Loader settings that work best for your system, you must test different combinations. For example, vary the number of threads and processes and test with different UE Mapper templates, record distribution, and queue sizes.

You can improve UE Loader performance by doing one or more of the following:

  • Increase the number of threads running in UE Loader.

    Increase the number of threads in the infranet.uei.max_threads entry in the Infranet.properties file. For more information, see "Setting the Number of Threads for Performance".

    Important:

    If the loading order of records is important, for example, when loading session events, use multiple threads only if you configure UE Mapper to load CDRs grouped by account ID.
  • Increase the number of UE Loader processes.

    You can run several UE Loader processes at once and load a file with each process. Do this by splitting the input file into several files before loading.

    Even better performance can be achieved by running several UE Loader processes using multiple threads in each process.

  • Limit the number of records loaded into memory.

    Increase the queue size in the infranet.uel.queue_size entry in the Infranet.properties file. The queue_size entry specifies the number of records to load simultaneously. The queue size is shared by multiple threads, which have concurrent access to the queue. For more information, see "Setting Memory Options".

  • Configure UE Loader to load CDRs grouped by account ID.

    Select Group events by account identifier when setting up your template in UE Mapper. For more information, start Developer Center and see UE Mapper Help.

  • Load CDRs in the order they occur when using multiple threads.

    Select Group events by account identifier in UE Mapper and ensure that all CDRs from the same account are loaded in order (ordered by time).

Retrieving Data about Events You Load

BRM stores information about batches of events that you load in a batch/gel object. This object contains information such as the event log file name and the date and time loaded.

Note:

You can display the event object in Event Browser.