21 About Sending Account Data to Pipeline Manager

This chapter describes Oracle Communications Billing and Revenue Management (BRM) Account Synchronization Manager and how account synchronization works. Anyone who installs, configures, or administers Account Synchronization Manager should read this document.

Before installing Account Synchronization Manager, you should be familiar with how a BRM wireless system works. See the following sections in BRM Telco Integration:

• About Integrating Wireless Services

• Overview of BRM Wireless Services Installation

About Account Synchronization

Account Synchronization Manager synchronizes customer and service data with pipeline rating data.

Account Synchronization Manager enables Pipeline Manager to act on events that occur in BRM based on the event type. For example, when account information changes, such as when products are purchased or account status is changed, the account information must be updated in the Pipeline Manager database so that service usage events can be rated properly. The Account Synchronization DM sends the updated account information to Pipeline Manager, enabling Pipeline Manager to rate events using the updated information.

You set up the following modules to receive events through account synchronization:

• DAT_Listener: All events are retrieved by this module first. This module passes the events to the other modules that are set up to receive them.

• DAT_AccountBatch: This module receives account update information to ensure that Pipeline Manager uses the most current account information when rating events.

• DAT_BalanceBatch: This module receives balance update information to ensure that Pipeline Manager uses the most current balance information when rating events.

• DAT_Discount: This module receives discount balance update information to ensure that Pipeline Manager uses the most current discount balance configuration when rating events.

How Account Synchronization Works

The Account Synchronization DM notifies Pipeline Manager when certain account information changes (for example, when a customer service representative (CSR) adds, cancels, or modifies an account or when an adjustment such as a cycle fee is applied to an account balance). The Account Synchronization DM sends information about updates through a database queue, and Pipeline Manager retrieves the information from the queue. For more information about the queue, see "About the Database Queues".

Note:

When an account is created, the Account Synchronization DM notifies Pipeline Manager about the new account, but the account details (such as the account's products, discounts, and other account information) are not loaded into Pipeline Manager memory until the first call data record (CDR) is rated for the account. This is because Pipeline Manager does not require the account details until the account has some usage that must be rated.

The Account Synchronization DM can send any event data to Pipeline Manager that it is configured to send. Pipeline modules register to receive events through the DAT_Listener module. When a module receives a business event, it can either use the enqueued event data for processing or access the BRM database for the most current data.

All business events sent to Pipeline Manager are stored in the database queue. If Pipeline Manager terminates, the Account Synchronization DM continues to send events to the queue. When Pipeline Manager is restarted, it retrieves the events from the queue. Likewise, if Account Synchronization Manager terminates, Pipeline Manager continues to retrieve events already in the queue.

If billing is being run while Pipeline Manager is starting, events are queued while the DAT modules are being initialized. Events are not processed until the DAT models are ready.

Figure 21-1 shows the data flow from event notification to Pipeline Manager update:

Figure 21-1 Data Flow from Event Notification to Pipeline Manager

Description of ''Figure 21-1 Data Flow from Event Notification to Pipeline Manager''

The following actions take place when account synchronization is performed:

1. An event listed in the Account Synchronization event notification list (pin_notify_ifw_sync) is generated, triggering a call to the opcode associated with that event in the list. See "About Event Notification" in BRM Developer's Guide.

2. One of the following actions occurs:

   • If the event is associated with the PCM_OP_IFW_SYNC_PUBLISH_EVENT opcode, it is passed to the PCM_OP_IFW_SYNC_POL_PUBLISH_EVENT policy opcode for modification. The policy opcode passes the event, along with any modifications, to the Payload Generator External Module (EM).

     See "About the Account Synchronization Opcodes".

   • If the event is not associated with PCM_OP_IFW_SYNC_PUBLISH_EVENT, it is sent directly to the Payload Generator EM.

3. The Payload Generator EM collects events in its payload until they compose a complete business event.

   See "About the EAI Framework".

4. When the business event is complete, the Payload Generator EM sends it to the Account Synchronization DM.

   See "About the Account Synchronization DM".

5. The Account Synchronization DM sends the business event to a database queue.

   See "About the Database Queues".

6. DAT_Listener retrieves the business event from the database queue and sends the event to pipeline modules that are registered for that event.

   See "About the DAT_Listener Module".

7. The module uses the event as needed.

About the EAI Framework

The Account Synchronization DM works with the Enterprise Application Integration (EAI) framework. You use the EAI framework to define business events for account synchronization, capture the BRM events that make up the business events, and send the completed business events to the Account Synchronization DM.

The Account Synchronization EAI framework consists of the following components:

• BRM event notification

  The Account Synchronization event notification list (pin_notify_ifw_sync) contains all the BRM events that make up the business events defined in the Account Synchronization payload configuration file (payloadconfig_ifw_sync.xml). In the list, each event is associated with an opcode. When a listed event is generated, its associated opcode is called to pass the event to the Payload Generator EM. See "About Event Notification" in BRM Developer's Guide and "Configuring Event Notification for Account Synchronization".

• The Payload Generator EM

  This module notifies the Account Synchronization DM when a business event occurs that requires some action such as an account update. Default business events for account synchronization are defined in the payload configuration file. See "Configuring the EAI Payload for Account Synchronization".

  The payloadconfig_ifw_sync.xml file includes a set of default business events. If you create custom business events, use the DAT_BalanceBatch CustomEvents registry entry to enable Pipeline Manager to be notified of those events. For information on creating business events, see "Defining Business Events" in BRM Developer's Guide.

The Payload Generator EM notifies the Account Synchronization DM when business events, such as account creation, product purchase, account status change, and account balance adjustment occur. As soon as the Payload Generator EM collects all the BRM events that belong to a specific business event, it sends the business event payload to the Account Synchronization DM.

Although the Account Synchronization DM relies on the EAI framework, you do not need to install EAI Manager separately. All necessary EAI files are included with Account Synchronization Manager.

For information about using the Payload Generator EM and the payload configuration file, see the following EAI Manager topics in BRM Developer's Guide:

• About BRM Business Events

• About Publishing Additional Business Events

• About the Payload Configuration File Syntax

About the Account Synchronization Opcodes

Account synchronization uses a set of opcodes to modify the content of certain BRM events that make up business events before the business events are sent to the Account Synchronization DM.

To enable this, the BRM events are associated with the PCM_OP_IFW_SYNC_PUBLISH_EVENT opcode in your system's event notification list. When these BRM events occur, PCM_OP_IFW_SYNC_PUBLISH_EVENT is called to pass them to the PCM_OP_IFW_SYNC_POL_PUBLISH_EVENT policy opcode for processing.

You can use the policy opcode to customize certain aspects of the BRM events based on your business needs. See "Modifying Business Events before Sending Them to Pipeline Manager".

After the Account Synchronization opcodes process the BRM events, they pass them to the EAI framework publishing opcode, which compiles them into business events to publish to Pipeline Manager.

Tip:

Not all BRM events that make up Account Synchronization business events must be passed through the Account Synchronization opcodes. If you do not need to customize the BRM events, associate them with the EAI framework publishing opcode, PCM_OP_PUBLISH_GEN_PAYLOAD (number 1301), in your system's event notification list instead. This opcode is internal to BRM. See "Configuring Event Notification for Account Synchronization".

About the Account Synchronization DM

The Account Synchronization DM sends business events to the database queue. You define how the DM connects to the database queue and which business events to send to the queue through two configuration files:

• The Account Synchronization DM configuration file (pin.conf) specifies how to connect to the queuing database schema.

  In a multischema system, to send events to multiple queues in different schemas, you must grant each source schema user execute permission for acct_sync from the target schema (see "Granting Execute Permission for acct_sync"). The Account Synchronization DM connects to only one database schema and uses schema qualifications to send events to queues in other database schemas.

• The ifw_sync_queuenames file specifies the business events to send to your database queues.

  See "Mapping Business Events to Database Queues".

  Note:

  If your system contains multiple schemas, you must install and configure an Account Synchronization DM for the primary schema. Optionally, you can also install an Account Synchronization DM for each secondary schema.

When the Account Synchronization DM receives a business event, it does the following:

1. Determines which queue to send the event to by checking the ifw_sync_queuenames file.

2. Enqueues the event in the appropriate queue.

3. Sets the event in the queue to a READY state.

   For information, see "About Event Status Flags".

About Disconnecting the Account Synchronization DM from the Queue

You can prevent the Account Synchronization DM from enqueuing events to the database queue by using the pin_ctl utility. This enables you to make changes to the queuing database schema without affecting the account synchronization process.

You control the connection between the Account Synchronization DM and the database queue by using the pin_ctl utility. See "Disconnecting and Reconnecting the Account Synchronization DM to the Queue".

For information about keeping Pipeline Manager online when you tune or shut down the queuing database, see "About Disconnecting DAT_Listener from the Queue".

About the Database Queues

Account synchronization uses a persistent database queue to pass business events from the Account Synchronization DM to Pipeline Manager. The queue enables account synchronization to pass events asynchronously, so BRM and Pipeline Manager are not required to be running at the same time.

Creating Database Queues

The Account Synchronization DM installer automatically creates a default queue in a specified database schema. If your system requires multiple queues, you must create additional queues by manually running the pin_ifw_sync_oracle utility. See "pin_ifw_sync_oracle".

Important:

To avoid system errors, do not run Pipeline Manager while you run the pin_ifw_sync_oracle utility.

When you create a queue, you must decide the following:

• The database schema in which to create the queue.

  You can create the queue in your BRM database, in the Pipeline Manager database, or in its own separate database.

• The tablespace in which to create the queue.

  You can create the queue in an existing tablespace or in its own separate tablespace.

  Note:

  For optimal performance in production systems, create the queue in its own tablespace in the Pipeline Manager database.

• The number of queues to create.

  The number of queues to create depends on your system's configuration. Each Pipeline Manager instance connects to its own database queue. Therefore, if your system contains only one instance of Pipeline Manager, you create only one database queue; if your system contains two instances of Pipeline Manager, you create two database queues.

  In multischema systems, each BRM database schema has one corresponding instance of Pipeline Manager and one corresponding database queue. See "Configuring Account Synchronization for Multiple Database Schemas".

About Event Status Flags

Events in the Oracle Advanced Queuing (AQ) queue are set to the following states:

• READY indicates that the event has not been dequeued and processed by DAT_Listener.

• PROCESSED indicates that the event was dequeued by DAT_Listener. The Oracle Queue Monitor process (QMn) removes the event from the queue after a configurable amount of time.

You can check the status of events in your queue by running a report. See "Generating Queue Reports".

About the DAT_Listener Module

Pipeline Manager uses DAT_Listener to retrieve business events from the database queue. When Pipeline Manager is running, DAT_Listener continuously checks the queue for events to process.

When DAT_Listener finds an event to process, it does the following:

1. Dequeues the event from the database queue.

2. Sets the event in the queue to a PROCESSED state.

3. Determines where to route the event by using the Pipeline Manager registry.

4. Sends the event to the appropriate pipeline module.

On Oracle AQ systems, DAT_Listener can control whether Pipeline Manager processes business events or CDRs by interleaving the two processes. You can configure DAT_Listener for concurrent or interleaved processing. See "About Controlling the Business Event Backlog".

About Using Multiple Threads to Dequeue Events

By default, DAT_Listener uses one thread for dequeuing events. All events in the queue are processed in the order in which they are queued.

You can generate additional threads by using the DAT_Listener EventThreadAllocation registry entry:

• You can generate separate threads for processing different business events. This can enhance performance when the type of business event, such as RecycleRequest business events, takes longer to process.

• You can generate additional threads for the same type of business event. This can further enhance performance when there are multiple business events of the same type that are queued in succession.

A new thread can begin processing an event as soon as processing for the previous event has begun.

For example, the following registry entry will generate four threads: one thread for RecycleRequest business events, two threads for OpenNewActgCycle business events, and one default thread (for which no entry is required) for all other types of events:

EventThreadAllocation
{
  RecycleRequest = 1
  OpenNewActgCycle = 2
}
  

Based on this registry configuration, if the following events are queued in this order:

1. RecycleRequest #1

2. CycleForward #1

3. OpenNewActgCycle #1

4. OpenNewActgCycle #2

5. OpenNewActgCycle #3

6. OpenNewActgCycle #4

7. CycleForward #2

8. RecycleRequest #2

9. CycleForward #3

They are processed during account synchronization in this order:

1. The RecycleRequest thread starts processing RecycleRequest event #1.

2. The default thread starts processing CycleForward event #1.

3. Two OpenNewActgCycle threads process OpenNewActgCycle events #1 through #4, in order.

   For example, thread 1 processes event #1 and thread 2 processes event #2. Whichever thread finishes first processes event #3, and so on.

4. When processing starts for OpenNewActgCycle event #4, the default thread starts processing CycleForward event #2 (assuming it has completed CycleForward event #1).

5. When processing starts for CycleForward event #2, the RecycleRequest thread starts processing RecycleRequest event #2 (assuming it has completed RecycleRequest event #1).

6. When processing starts for RecycleRequest event #2, the default thread starts processing CycleForward event #3 (assuming it has completed CycleForward event #2).

You can allocate only one thread for recycle requests.

Dequeuing in Batches

Dequeuing business events from the Account Synchronization queue takes longer than sending them to Pipeline Manager. To shorten the time it takes to process business events, DAT_Listener dequeues business events in batches. It then sends one event at a time to Pipeline Manager.

DAT_Listener retrieves events in batches only for events that require account synchronization, such as account data changes. These are the most frequent types of business events sent through the queue. Other events—for example, those that invoke a process such as a recycling a job—continue to be dequeued one at a time.

If a business event in a batch of dequeued events contains an error, all events in the batch are rolled back in BRM and Pipeline Manager is shut down. Shutting down Pipeline Manager clears the batch of business events from the pipeline memory. After you correct the error and stop and restart Pipeline Manager, BRM resends the business events to Pipeline Manager.

About Disconnecting DAT_Listener from the Queue

You can keep Pipeline Manager online when you tune or shut down the queuing database by disconnecting DAT_Listener from the queue. You can then reconnect the module to the queue after you finish making changes to the database.

When you disconnect DAT_Listener, it performs the following actions:

1. Waits for the total counter of active threads to become 0.

2. Disconnects each active thread:

   • If a thread is currently processing an event, DAT_Listener waits for the event to finish, inactivates the thread, and decrements the counter.

   • If a thread is not currently processing an event, DAT_Listener inactivates the thread and decreases the counter.

3. When the counter becomes 0, DAT_Listener wakes up from the wait state.

   It returns to the caller that all connections between it and the database queue have been terminated.

When you reconnect DAT_Listener, it reconnects to the queuing database and activates all threads.

You control the connection between DAT_Listener and the database queue by using DAT_Listener semaphores. See "Disconnecting and Reconnecting DAT_Listener to the Queue".

About Account Synchronization in a Multischema System

When you use a multischema system, you set up an instance of Pipeline Manager for each database schema. On one instance of Pipeline Manager, you configure the multischema account router module (see "FCT_AccountRouter" in BRM Configuring Pipeline Rating and Discounting). This module keeps track of which database schema each account belongs to.

To set up account synchronization for multiple database schemas, you install and configure an Account Synchronization DM on the primary BRM installation machine and, optionally, on your secondary BRM installation machines. You also do the following:

• Add a database queue for each BRM database schema.

  Each Account Synchronization DM connects to only one database schema and uses schema qualifications to put event messages in queues in other database schemas.

• Configure an instance of Pipeline Manager for each BRM database schema.

  Each instance of Pipeline Manager has a DAT_Listener module that retrieves events for that instance. You specify the events for each instance in the DAT_Listener registry.

  Note:

  It is recommended that you install an instance of the Account Synchronization DM and have a corresponding instance of Pipeline Manager for every BRM database schema, but it is not required. How you configure multiple database schemas depends on your business needs and should be determined by your system and database administrators. See "BRM Installation Overview" and "Installing a Multischema System".

Figure 21-2 shows the data flow from the Account Synchronization DM to multiple instances of Pipeline Manager:

Figure 21-2 Data Flow from Account Synchronization DM to Pipeline Manager Instances

Description of ''Figure 21-2 Data Flow from Account Synchronization DM to Pipeline Manager Instances''

To set up account synchronization for a multischema system, see "Configuring Account Synchronization for Multiple Database Schemas".

About the Payload Configuration File

The Account Synchronization DM is an EAI publisher. All EAI publishers in your BRM system use the same payload configuration file. This file defines how EAI applications accumulate and format published information. The name and location of this file is specified in the infranet.eai.configFile entry in the EAI properties file (BRM_Home/sys/eai_js/Infranet.properties).

The Account Synchronization DM includes a payload configuration file (payloadconfig_ifw_sync.xml). This file defines the business events that the Account Synchronization DM uses to update data in Pipeline Manager.

If your BRM system does not already have an EAI-based publisher, the Account Synchronization installation program sets the infranet.eai.configFile entry to point to the Account Synchronization payload configuration file.

If your BRM system already has an EAI-based publisher installed, the Account Synchronization installation program automatically merges the Account Synchronization payload configuration file with the existing payload configuration file. The merged file is named payloadconfig_MergedWithIfw_sync.xml, and the infranet.eai.configFile entry is updated accordingly. The names and contents of the original files used to create the merged file are not changed by the Account Synchronization installation program.

For more information, see "Specifying the Default Payload Configuration File".

Caution:

In rare cases, it may not be possible to use the merged payload configuration file. This can happen when entries in the two original payload configuration files have conflicting definitions. You must check for conflicts between the two original files before running the Account Synchronization DM. See "Checking for Conflicts in EAI Payload Configuration Files".

About Controlling the Business Event Backlog

When you use Pipeline Manager for batch rating, the Account Synchronization database queue can become backlogged with business events waiting to be processed. This can delay account synchronization and slow Pipeline Manager performance, causing a greater number of CDRs to be suspended.

To control the number of events waiting to be processed, you can configure DAT_Listener to interleave CDR and business event processing. For information about how interleaved processing works, see "About Interleaved Processing". For information about configuring DAT_Listener for interleaved processing, see "Configuring Interleaved Processing".

Why Event Backlog Occurs

Event backlog can occur for the following reasons:

• When Pipeline Manager modules update account data, they lock the accounts they are updating in each transaction. When more than one transaction must access the same account, the transactions must wait until the account is unlocked. If the account stays locked for a relatively long time, business events can become backlogged while they wait for processing.

• When business events arrive that change account information during CDR processing, Pipeline Manager must update its memory before continuing to rate events in the CDR. This requires accessing the database, which adds to the number of times the accounts are locked. With concurrent business event and CDR processing, there could be multiple updates required, further reducing overall Pipeline Manager performance.

• When an account is currently being billed, Pipeline Manager must receive a business event notifying it that billing is complete before it can process new event data records (EDRs) for the account. If a billing business event is waiting to be processed behind a backlog of other events, there might be many new EDRs for the account that require suspending until the billing event is processed. Each time a suspended EDR is recycled, it creates a new business event that is added to the account synchronization queue.

About Interleaved Processing

You can configure Pipeline Manager to process CDRs and business events exclusively rather than concurrently. With interleaved processing, business event and CDR processing are interleaved so that multiple business events can be processed before rating the CDRs, and multiple CDRs can be rated before updating Pipeline Manager memory. This balances the event backlog and improves processing time by decreasing the number of times Pipeline Manager must access the database and lock accounts. Exclusive processing reduces the likelihood that you will need to recycle events due to delayed synchronization or to stop Pipeline Manager to catch up on business event processing.

DAT_Listener controls whether business events or CDRs are processed based on the following criteria:

• How many business events are waiting to be processed in the Account Synchronization queue.

• The time that Pipeline Manager has been currently processing either business events or CDRs.

You can use the first or both of these criteria, depending on how much control you want over the interleaving process. See "About Setting Processing Thresholds".

You specify the thresholds for event processing in the DAT_Listener module registry. See "Configuring Interleaved Processing".

Assuming both the number of events and the processing time are configured, DAT_Listener performs the following actions to control interleaved processing:

1. Periodically checks the number of business events waiting to be processed in the Account Synchronization queue and tracks the amount of time that Pipeline Manager has been currently processing either business events or CDRs.

2. Stops pipeline CDR processing when the number of business events in the queue reaches the maximum threshold or when the maximum CDR processing time is reached, whichever comes first.

   DAT_Listener stops Pipeline Manager by sending a stop command to the pipeline controller. This command suspends CDR processing but allows business event processing to proceed.

3. Dequeues business events and sends them to pipelines for processing.

4. Stops dequeuing business events when the number of business events in the queue reaches the minimum threshold or when the maximum business event processing time is reached, whichever comes first.

5. Restarts Pipeline Manager CDR processing by sending a start command to the pipeline controller.

The Account Synchronization DM continues to send business events to the queue while Pipeline Manager processes CDRs.

When tracking the number of business events in the queue, DAT_Listener counts only the business events that have a state of READY. All other business events are not considered.

About Setting Processing Thresholds

For basic process interleaving, you set the minimum and maximum business event thresholds. For more control over CDR and business event interleaving, you can also set the maximum CDR and business event processing times.

For example, if you typically process fewer events in the middle of the night, it might take longer to reach the specified maximum number of business events in the queue. Account synchronization will be delayed during this time and you might have a higher incident of CDRs that need suspending or rerating. In this case, you limit the amount of time spent on CDR processing. When this time threshold is reached, DAT_Listener switches to business event processing even if the number of events in the queue has not reached the maximum.

You might want to limit business event processing time if there are certain times when many business events accumulate in the queue (for example, when you shut down Pipeline Manager for an extended period). When you stop and restart Pipeline Manager, rather than waiting until the minimum business event threshold is reached, Pipeline Manager switches to CDR processing when the maximum business event process time is reached.

To set the processing thresholds, see "Configuring Interleaved Processing".

Note:

DAT_Listener does not check whether there are CDRs waiting to be processed. It checks only for business events and processing times. Therefore, if there are no CDRs to process, DAT_Listener waits until either the CDR processing time threshold or the maximum number of business events is reached before switching to business event processing. If this wait period is too long, you can update the DAT_Listener registry to switch from CDR to business event processing. See "Switching to Business Event Processing When No CDRs Are Waiting".