14 Improving BRM Performance

This chapter provides information on evaluating and improving the performance of your Oracle Communications Billing and Revenue Management (BRM) system.

Before you read this chapter, you should be familiar with BRM system architecture. See "BRM System Architecture" in BRM Concepts.

For information on troubleshooting BRM, see "Resolving Problems in Your BRM System".

For information on tuning Pipeline Manager, see "Optimizing Pipeline Manager Performance".

Monitoring Performance

You can use BRM diagnostics tools to monitor performance. For example:

• You can monitor opcode latency.

• You can monitor event data record (EDR) throughput in a pipeline.

See "About Monitoring BRM".

Improving Connection Manager Performance

For information about Connection Managers (CMs), see "The Business Process Tier" in BRM Concepts.

To improve CM performance, see the following topics:

• Increasing CM Login Performance

• Load Balancing CMs

• Improving Performance for Loading Large Price Lists

• Improving Real-Time Rating Performance

• Improving the Performance of your Multithreaded Applications

• Logging Noncurrency Events

• Specifying the Number of Connections to CMs

• Setting the CM Time Interval between Opcode Requests

Increasing CM Login Performance

To increase CM login performance, see the following topics:

• Using CM Proxy to Allow Unauthenticated Log On

• Turning Off Session-Event Logging

• Turning Off the Checking of Logons and Passwords

Using CM Proxy to Allow Unauthenticated Log On

Use CM Proxy to provide unauthenticated connections to BRM, typically for providing connections for incoming mail messages. Connections made through CM Proxy do not require you to log on, which increases performance.

Caution:

Only connections that do not require authentication should use CM Proxy.

To increase security, you can restrict the types of operations performed by CM Proxy by specifying the opcodes that perform allowed operations.

To use CM Proxy:

1. Open the CM Proxy configuration file (BRM_home/sys/cm_proxy/pin.conf).

2. Configure CM Proxy according to the guidelines in that file, and save the file.

   The following are some of the more important entries:

   • Use the oplist entry to specify the opcodes that can be performed by CM Proxy.

   • Use the allowed entry to specify the hosts that can use CM Proxy.

   • Use the queue manager entries to manage front-end and back-end connections. See "Improving Data Manager and Queue Manager Performance".

   • Use the standard connection entries to connect to a CM or CMMP (Connection Manager Master Process). See "About Connecting BRM Components".

3. Start CM Proxy. See "Starting and Stopping the BRM System".

4. Open the configuration file for each application you want to use CM Proxy. See "Locations of Configuration and Properties Files".

5. Change the cm_ptr entry to point to the machine running CM Proxy and change the login_type entry to 0 (no login name or password required).

Turning Off Session-Event Logging

By default, the CM writes a session-event storable object for each client connection, but you can suppress the creation of these session storable objects if you do not need these records of logins. You can turn off session-event logging for some CMs for better performance while keeping this feature for other CMs dedicated to applications that require session objects, such as the mail and terminal servers.

To turn off session-event logging:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Change the value for the login_audit entry to 0 and ensure that the entry is not commented.

3. Stop and restart the CM. See "Starting and Stopping the BRM System".

Turning Off the Checking of Logons and Passwords

When an application tries to log on to BRM, the CM verifies the service specified by the application, asks for a login name, and verifies the login password. To improve performance, set up the CM to not ask for a login name or password. See "Configuring the CM to Verify Application Logins with the Service Only".

To turn off login name and password verification:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Change the value for the cm_login_module entry to read:

   - cm   cm_login_module   ./cm_login_null.extension
   

   where extension is the file type specific to your operating system: so for Solaris, Linux, or HP-UX IA64; or a for AIX. For example:

   - cm   cm_login_module   ./cm_login_null.so
   

3. Stop and restart the CM. See "Starting and Stopping the BRM System".

Load Balancing CMs

You can use two methods for balancing the load among multiple CMs:

• You can use a CMMP to provide additional reliability and to balance the load among multiple CMs. See "Using Connection Manager Master Processes".

• You can provide a simple failover system for connections to the CM by giving the application a list of CMs on the system. If the first CM in the list is unavailable, the application tries the next CM.

  For example:

  - nap  cm_ptr  ip  cm_host1  11960
  - nap  cm_ptr  ip  cm_host1  11961
  

  If the CMs are on separate machines:

  - nap  cm_ptr  ip  cm_host1  11960
  - nap  cm_ptr  ip  cm_host2  11960
  

  Tip:

  You can point to multiple CMMPs rather than to individual CMs. If the CM provided by the first CMMP is not available, the application asks for a CM from the second CMMP.

  For example:

  - nap  cm_ptr  ip  CMMP_host1  11959
  - nap  cm_ptr  ip  CMMP_host2  11959
  

Improving Performance for Loading Large Price Lists

If you have a large price list, you can improve performance in the following ways:

• Cache pricing data, such as G/L IDs and resource IDs. In a test environment where you modify your price list often, caching pricing data improves performance because there is no need to load price reference objects every time you commit the price list to the database.

  Important:

  Pricing data is created every time the CM starts. Whenever the pricing data is changed in the database, the CM must be stopped and restarted to place the new information into the cache.

  In a production system where you rarely modify your price list, you do not need to cache pricing data. This reserves the CM cache for other uses and eliminates the need to stop and restart the CM to update the cache if you change the price list.

• Turn off event logging for price list creation events.

  Note:

  When you turn off event logging, BRM still stores audit trail information for products, deals, and plans; however, there will not be an event log of when the price plans were modified and who modified them.

To improve loading performance:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the cache_references_at_start entry. The default is 1 (cache data).

   - fm_price cache_references_at_start 1
   

3. Edit the fm_price_prod_provisioning_cache entry. The default entries are usually sufficient. For more information, see the instructions in the configuration (pin.conf) file.

   - cm_cache fm_price_prod_provisioning_cache 100, 102400, 13
   

4. Edit the fm_price_cache_beid entry. The default entries are usually sufficient. For more information, see the instructions in the configuration (pin.conf) file.

   - cm_cache fm_price_cache_beid 200, 524288, 32
   

5. Edit the log_price_change_event entry. The default is 0 (events are not logged).

   - fm_price log_price_change_event  0
   

6. Edit the fm_offer_profile_cache entry. The default entries are usually sufficient. For more information, see the instructions in the configuration (pin.conf) file.

   - cm_cache fm_offer_profile_cache 20, 102400, 13
   

   Important:

   For policy-driven charging, the fm_offer_profile_cache entry must be present in the pin.conf file.

   See "Policy-Driven Charging" in BRM Setting Up Pricing and Rating.

7. Save the file.

8. Stop and restart the CM. See "Starting and Stopping the BRM System".

Improving Real-Time Rating Performance

You can improve real-time rating performance by doing the following:

• Changing the Precision of Rounded and Calculated Values

• Setting the Interval for Checking for Price List Changes

• Setting the Interval for Updating Zone Maps

• Filtering the ERAs Considered during Rating and Discounting

• Enabling and Disabling the Caching of Customized Products

• Configuring the Maximum Number of Products and Discounts Cached

Changing the Precision of Rounded and Calculated Values

To improve performance, you can change the precision of rounded values and of values calculated by real-time rating. You change the precision by adding or modifying entries in the CM pin.conf file:

• To change the precision of rounded values, add or change the rating_quantity_rounding_scale entry. The value of this entry determines the number of digits to the right of the decimal place for rounding quantities. The default is 8.

• To change the precision of calculated values, add or change the rating_max_scale entry. The value of this entry determines the number of digits to the right of the decimal place that are used. The default is 10.

  Important:

  You must stop and restart the CM after you change these values. See "Starting and Stopping the BRM System".

Setting the Interval for Checking for Price List Changes

You can set the interval at which BRM checks for changes to the price list. If you change the price list frequently, you may want to use a shorter interval. If your price list is less volatile, you can increase the interval.

To change the interval:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the following entry:

   - fm_rate refresh_product_interval 3600
   

   The value of this entry determines the interval in seconds. The default is 3600.

3. Save the file.

4. Stop and restart the CM. See "Starting and Stopping the BRM System".

Setting the Interval for Updating Zone Maps

To specify how frequently BRM checks for changes to zone maps and updates them in the database:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the following entry:

   - fm_zonemap_pol update_interval 3600
   

   The value of this entry determines the interval in seconds. The default is 3600.

3. Save the file.

4. Stop and restart the CM. See "Starting and Stopping the BRM System".

Filtering the ERAs Considered during Rating and Discounting

By default, real-time rating checks for both account-level extended rating attributes (/profile/acct_extrating object) and service-level ERAs (/profile/serv_extrating object) when it searches for rating and discounting criteria. You can improve real-time rating performance by filtering the types of ERAs that BRM considers when it searches for rating and discounting criteria. For example, you can configure BRM to search for service-level ERAs only or to omit the ERA search altogether.

You can specify the types of ERAs to consider by modifying a field in the rating instance of the /config/business_params object.

You modify the /config/business_params object by using the pin_bus_params utility. See "pin_bus_params" in BRM Developer's Guide.

To specify the ERA types:

1. Use the following command to create an editable XML file from the rating instance of the /config/business_params object:

   pin_bus_params -r BusParamsRating bus_params_rating.xml 
   

   This command creates the XML file named bus_params_rating.xml.out in your working directory. If you do not want this file in your working directory, specify the full path as part of the file name.

2. Search the XML file for following line:

   <EnableEras>serviceAndAccount</EnableEras>
   

3. Change serviceAndAccount to one of the following:

   • account: Limits the rating and discounting criteria search to account-level ERAs by retrieving only the /profile/acct_extrating object.

   • service: Limits the rating and discounting criteria search to service-level ERAs by retrieving only the /profile/serv_extrating object.

   • disabled: Omits ERAs from the rating and discounting criteria. Because neither object is retrieved, this option provides the best performance.

     Caution:

     BRM uses the XML in this file to overwrite the existing rating instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM rating configuration.

4. Save the file.

5. Change the file name from bus_params_rating.xml.out to bus_params_rating.xml.

6. Use the following command to load the change into the /config/business_params object:

   pin_bus_params bus_params_rating.xml 
   

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

7. To verify that all fields are correct, you can display the /config/business_params object by using Object Browser or by using the robj command with the testnap utility.

   For more information, see the following topics in BRM Developer's Guide:

   • Using testnap

   • Reading Objects by Using Object Browser

8. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

9. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Enabling and Disabling the Caching of Customized Products

When you use advanced customization to create customized products, BRM uses the customized products for rating. You can control whether customized products are cached for use by the real-time rating engine.

• If you choose not to cache customized products (the default setting), the real-time rating engine retrieves customized product data from the database during rating. This slows rating performance but minimizes the memory impact of customized products.

• If you choose to cache customized products, the CM size grows as customized products are created. Because the products are cached in memory, however, rating performance is increased.

You enable product caching by changing the EnableTailormadeCache field in the rating instance of the /config/business_params object from 0 to 1. You can disable caching by changing the field back to 0.

You modify the /config/business_params object by using the pin_bus_params utility. See "pin_bus_params" in BRM Developer's Guide.

To enable caching of customized products:

1. Use the following command to create an editable XML file from the rating instance of the /config/business_params object:

   pin_bus_params -r BusParamsRating bus_params_rating.xml 
   

   This command creates the XML file named bus_params_rating.xml.out in your working directory. If you do not want this file in your working directory, specify the full path as part of the file name.

2. Search the XML file for following line:

   <EnableTailormadeCache>0</EnableTailormadeCache>
   

3. Change 0 to 1.

   Caution:

   BRM uses the XML in this file to overwrite the existing rating instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM rating configuration.

4. Save the file.

5. Change the file name from bus_params_rating.xml.out to bus_params_rating.xml.

6. Use the following command to load the change into the /config/business_params object:

   pin_bus_params bus_params_rating.xml 
   

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

7. To verify that all fields are correct, you can display the /config/business_params object by using Object Browser or by using the robj command with the testnap utility.

   For information on using testnap, see "Using testnap" in BRM Developer's Guide.

   For information on how to use Object Browser, see "Reading Objects by Using Object Browser" in BRM Developer's Guide.

8. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

9. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Configuring the Maximum Number of Products and Discounts Cached

Products and discounts that are used during the real-time rating process are automatically stored in the CM cache. This improves rating performance, but, over time, it can consume a large amount of memory. To prevent the CM cache from growing too large, you can set a maximum number of products and discounts that can be stored in the CM cache.

• When you set the maximum to a nonzero value, BRM prevents the real-time rating engine from storing more than the specified number of products and discounts in CM cache. When the maximum number is reached, BRM flushes 10% of the products and discounts from cache that have been used the least.

  Note:

  The maximum number of products and discounts that should be stored in CM cache depends on the your business needs.

• When you set the maximum to zero, BRM does not regulate the number of products and discounts stored in the CM cache. This is the default.

You configure the maximum number of products and discounts that can be cached by configuring the ProductsDiscountsThreshold field in the rating instance of the /config/business_params object.

To set a maximum number of products and discounts that can be cached:

1. Use the following command to create an editable XML file from the rating instance of the /config/business_params object:

   pin_bus_params -r BusParamsRating bus_params_rating.xml 
   

   This command creates the XML file named bus_params_rating.xml.out in your working directory. If you do not want this file in your working directory, specify the full path as part of the file name.

2. Search the XML file for following line:

   <ProductsDiscountsThreshold>0</ProductsDiscountsThreshold>
   

3. Change 0 to the maximum number of products and discounts that you would like stored in cache. The default value of 0 specifies to not regulate the number of products and discounts in the CM cache.

   Caution:

   BRM uses the XML in this file to overwrite the existing rating instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM rating configuration.

4. Save the file.

5. Change the file name from bus_params_rating.xml.out to bus_params_rating.xml.

6. Use the following command to load the change into the /config/business_params object:

   pin_bus_params bus_params_rating.xml 
   

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

7. To verify that all fields are correct, you can display the /config/business_params object by using Object Browser or by using the robj command with the testnap utility.

   For more information, see the following topics in BRM Developer's Guide:

   • Using testnap

   • Reading Objects by Using Object Browser

8. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

9. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Improving the Performance of your Multithreaded Applications

You can improve the performance of multithreaded applications by controlling the thread load and monitoring the thread activity.

Controlling Thread Load on Your Multithreaded Application

The number of child threads allowed is controlled by the children entry in the application pin.conf file. You can set the number of threads manually by editing the pin.conf file or automatically by using the pin_mta_monitor monitoring utility.

To improve performance, you can limit the number of child threads. For example, you can limit the number of threads during the business day to prevent excessive load on the network and increase the number of threads during off-peak hours.

To control the number of threads for an MTA by using pin_mta_monitor:

1. Go to the directory from which you run the MTA.

2. Start the pin_mta_monitor utility using the following command:

   pin_mta_monitor mta_application
   

   where mta_application is the MTA that pin_mta_monitor tracks.

3. The (mon)> prompt appears. Provide the number by which you want to increase or decrease the number of threads that mta_application uses as the value for number in the appropriate command (press the spacebar and then press Enter).

   • To increase the number of threads that mta_application uses, enter:

     (mon)> t+number
     

   • To decrease the number of threads that mta_application uses, enter:

     (mon)> t-number
     

For example, the following commands increase the thread pool of pin_inv_export by two threads:

pin_mta_monitor pin_inv_export
(mon)> t+2

Monitoring the Thread Activity of Your Multithreaded Application

Use the pin_mta_monitor utility to monitor the thread activity of your MTA.

Note:

After entering a command at the monitor prompt, press the spacebar once and then press Enter. If you do not enter a space after the command, the utility does not run the command.

To monitor the thread activity of an MTA:

1. Go to the directory from which you run the MTA.

2. Start the pin_mta_monitor utility using the following command:

   pin_mta_monitor mta_application
   

   where mta_application is the MTA that pin_mta_monitor tracks. For example, the following command is used to monitor the activity of pin_inv_export:

   pin_mta_monitor pin_inv_export
   

3. The (mon)> prompt appears.

   To print the thread activity of the mta_application to stdout:

   (mon)> p
   

   The following is a sample output of the thread activity that pin_mta_monitor prints for pin_inv_export:

   Required:2, cmd:print
   Thread_id:2, Working POID:0.0.0.1 /invoice 221084 0, Flag:0
   Thread_id:3, Working POID:0.0.0.1 /invoice 220860 0, Flag:0
   

4. To stop printing the thread activity of the mta_application to stdout:

   (mon)> q
   

Logging Noncurrency Events

By default, BRM is configured to give maximum account-creation performance by logging only events that have a balance impact associated with a currency. To log noncurrency events at the expense of system performance, change the creation_logging entry in the CM configuration file (pin.conf).

By default, these events are logged:

• /event/billing/charge

• /event/billing/deal

• /event/billing/product

By default, these events are not logged:

• /event/customer/billinfo

• /event/customer/nameinfo

• /event/customer/login

• /event/customer/password

• /event/customer/status

• /event/billing/limit

To log noncurrency events:

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the creation_logging entry:

   - fm_cust creation_logging 1
   

3. Save and close the file.

The new value becomes effective immediately and applies to the next account created. You do not need to restart the CM to enable this entry.

Specifying the Number of Connections to CMs

The cm_max_connects entry in the CM configuration file (pin.conf) tells the CM how many client applications can connect simultaneously. If client applications are having trouble connecting to the database, you can increase the number of connections. Performance degrades when too many CMs are running on the same machine, depending on the system load.

The maximum number of connections is 1000.

Note:

Normally, cm_max_connects is commented out. This type of connection is best handled by CM Proxy. See "Using CM Proxy to Allow Unauthenticated Log On".

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the cm_max_connects entry:

   - cm   cm_max_connects  300
   

3. Save and close the file.

4. Stop and restart the CM.

   See "Starting and Stopping the BRM System".

Setting the CM Time Interval between Opcode Requests

When a client application requests a connection, the CM spawns a child process to handle the connection. The child process or thread communicates with the client application by receiving requests in the form of opcodes.

By default, the child process or thread waits infinitely for each opcode request, but you can set a time interval after which the child process or thread terminates if no request has arrived.

1. Open the CM configuration file (BRM_home/sys/cm/pin.conf).

2. Edit the cm_timeout entry.

   - cm cm_timeout 3
   

   The value of this entry specifies the time interval in minutes.

3. Save and close the file.

4. Stop and restart the CM. See "Starting and Stopping the BRM System".

Using Connection Manager Master Processes

The Connection Manager Master Process (CMMP) routes connections from a single BRM client to multiple CMs. See "About Connection Manager Master Processes (CMMPs)" in BRM Concepts.

To specify which CMs to route connections to, you list the CMs in the CMMP configuration file. (See "Setting Up a CMMP".) You can also specify how the CMMP chooses a CM:

• By default, the CMMP chooses a CM randomly from the list of CMs.

• You can specify that the CMMP choose CMs sequentially from the first entry to the last, and then back to the first (known as round-robin selection).

The CMMP does not validate whether the CMs it points to are running. If the CMMP sends a transaction to a nonfunctioning CM, the client can tell that the CM is offline and tries to connect to a different CM or CMMP. Therefore, you should include multiple cm_ptr entries to the same CMMP or to more than one CMMP.

Because it is the client's responsibility to ensure a connection to a CM, there must be at least one entry in the CMMP pin.conf file for each CM to which it is connected.

Sample CMMP Configuration

In Figure 14-1:

• There are two CMMP host machines, CMMP_host1 and CMMP_host2.

• There are four CM host machines, CM_host1, CM_host2, CM_host3, and CM_host4.

• Each CM points to its own Data Manager (DM).

Figure 14-1 Sample CMMP Configuration

Description of ''Figure 14-1 Sample CMMP Configuration''

Client configuration file

Using the connection parameters shown below, the clients attempt to connect to all CMs through both CMMPs:

- nap cm_ptr ip CMMP_host1 11959
- nap cm_ptr ip CMMP_host1 11959
- nap cm_ptr ip CMMP_host1 11959
- nap cm_ptr ip CMMP_host1 11959
- nap cm_ptr ip CMMP_host2 11959
- nap cm_ptr ip CMMP_host2 11959
- nap cm_ptr ip CMMP_host2 11959
- nap cm_ptr ip CMMP_host2 11959

CMMP configuration files

The following redirect entries are required for client-to-CM connectivity. The parameters should be the same for both CMMP configuration files:

- cm redirect - CM_host1 11960
- cm redirect - CM_host2 11960
- cm redirect - CM_host3 11960
- cm redirect - CM_host4 11960

If the client and the host are on different domains, the redirect entry must include the full host and domain name or IP address for the CMMP Port.

- cm redirect - CM_host1.example.com 11960
- cm redirect - CM_host2.example.com 11960
- cm redirect - CM_host3.example.com 11960
- cm redirect - CM_host4.example.com 11960

- cm redirect -  198.51.100.1 11960
- cm redirect -  198.51.100.2 11960
- cm redirect -  198.51.100.3 11960
- cm redirect -  198.51.100.4 11960

If the CMs are on the same host, the port numbers must be unique:

- cm redirect - CM_host1 11961
- cm redirect - CM_host1 11962
- cm redirect - CM_host1 11963
- cm redirect - CM_host1 11964

Note:

Ensure you also set the - cm cmmp_algorithm parameter.

Setting Up a CMMP

To set up a CMMP:

1. Open the CMMP configuration file (BRM_home/sys/cmmp/pin.conf).

2. Configure the CMMP according to the guidelines in the file:

   • Use the redirect entry to list the CMs on your system. For example:

     - cm redirect - CM_host1 11960
     - cm redirect - CM_host2 11960
     - cm redirect - CM_host3 11960
     - cm redirect - CM_host4 11960
     

   • Use the cmmp_algorithm entry to specify whether the CMMP chooses CMs randomly (the default) or sequentially.

3. Save and close the file.

4. Start the CMMP. See "Starting and Stopping the BRM System".

5. Open the configuration file for each application you want to use CMMP. See "Locations of Configuration and Properties Files".

6. Change the cm_ptr entry in the client application pin.conf file to point to the machine running the CMMP.

7. Save and close each configuration file you change.

Improving Data Manager and Queue Manager Performance

For information about Data Managers (DMs), see "The Data Management Tier" in BRM Concepts.

To improve DM performance, see the following topics:

• About Queuing-Based Processes

• Configuring DM Front Ends and Back Ends

• Configuring Multiple Pipelines in the DM to Improve Performance

• Setting DM Shared Memory Size

• Reducing Resources Used for Search Queries

• Load Balancing DMs

• Optimizing Memory Allocation during Database Searches

• Improving BRM Performance during Database Searches

• Increasing DM CPU Usage

• Examples of DM Configurations

About Queuing-Based Processes

Queuing improves system performance by lowering the number of connections to the database. This reduces the number of processes and therefore reduces the system load required to handle the connections.

Queuing is used in two different types of system components:

• The RADIUS Manager, CM Proxy, and Web Interface daemons use queuing to connect incoming client connections to CMs. In this case, queuing reduces the number of client connections to CMs.

• All DMs use queuing internally. Front-end processes pass requests and data through a queue to back-end processes. In this case, queuing reduces the number of connections to the database.

CMs and Connection Manager Master Processes (CMMPs) do not use queuing.

Figure 14-2 shows an example of where queuing takes place in the BRM system architecture. Queuing occurs in two locations. In this example, connections are queued in the RADIUS Manager and the in the database DM.

Figure 14-2 BRM Queuing Locations

Description of ''Figure 14-2 BRM Queuing Locations''

Example of Queuing in a Client-to-CM Connection

Figure 14-3 shows a daemon running on a system. Front-end processes pass the connections to a shared-memory queue where the connections wait for available back ends. The back ends connect to CMs.

Figure 14-3 CM Client Connection Queuing

Description of ''Figure 14-3 CM Client Connection Queuing''

Configuring DM Front Ends and Back Ends

You configure DM performance by specifying the number of front-end and back-end processes and the amount of shared memory the DM uses.

Note:

Queue Manager (QM) components, such as LDAP Manager, use the same types of configuration entries, but they have different names. For example, instead of dm_max_fe, the entry is named qm_max_fe. The functionality is the same.

Use the following DM and QM pin.conf entries to tune performance:

• dm_n_fe: Specifies the number of DM front-end processes.

• dm_n_be: Specifies the maximum number of DM back-end processes.

• dm_max_per_fe: Specifies the maximum number of connections for each front end.

• dm_trans_be_max: Specifies the maximum number of back ends that can be used for processing.

• dm_init_be_timeout: Specifies the time, in seconds, that the DM waits for the DM back-end startup process to finish.

• dm_trans_timeout: Specifies the time in minutes that DM back-end processes wait for the next opcode in a transaction.

To change one or more of these parameters:

1. Open the DM configuration file (BRM_home/sys/dm_oracle/pin.conf).

2. Change the configuration entry associated with the parameter. For tuning guidelines, see the topics following this procedure. For the syntax of each configuration entry, follow the guidelines in the configuration file.

3. Save and close the file.

4. Stop and restart the DM. See "Starting and Stopping the BRM System".

   Important:

   Besides configuring the number of connections for best performance, remember to keep the number of connections within the terms of your database license agreement.

Ratio of Front Ends to Back Ends

Oracle recommends that the total number of front ends (specified in the dm_n_fe and dm_max_per_fe entries) should be two to four times the number back ends (specified in the dm_n_be entry).

In this example, the total number of front ends is 64 (4 times 16), which is 4 times the number of back ends.

- dm dm_n_fe 4
- dm dm_max_per_fe 16
- dm dm_n_be 16

Providing Enough Front-End Connections

If connection errors occur between the CM and DM, increase the values in the dm_n_fe and dm_max_per_fe entries. If there are not enough front ends, BRM reports an error. For example:

DMfe #3: dropped connect from 194.176.218.1:45826, too full
W Thu Aug 06 13:58:05 2001 dmhost dm:17446 dm_front.c(1.47):1498

Check the dm_database.log and dm_database.pinlog files for errors.

You must have enough DM front-end connections (number of processes times the number of connections for each process) to handle the expected number of connections from all of the CMs. Otherwise, you will see errors when the applications cannot connect to BRM.

Connections might be required for the following:

• One connection for each CM Proxy thread.

• One connection for each web interface thread, plus one additional connection if customers create accounts with a web interface.

• One connection for each billing application thread plus one additional connection for the master search thread.

• Two connections for each instance of Customer Center.

The maximum number of connections each front-end process can handle depends on the activity of the connection and, on multi-processor machines, the processor speed. For intensive connections, such as a heavily utilized terminal server, a front end might be able to handle only 16 connections. For intermittent connections, such as through certain client tools, a single front end can handle 256 or 512 connections. For systems that use a combination of these activities (for example, real-time processing with some client tool activity), you can configure an intermediate value for the maximum connections per front end.

For a given number of connections, if you have too many front ends (too few connections for each front end), the DM process uses too much memory and there is too much context switching. Conversely, if you have too few front ends (too many connections for each front end), the system performs poorly.

Determining the Required Number of Back Ends

You configure the number of back ends to get maximum performance from your system, depending on the workload and the type of BRM activity. Here are some guidelines for various activities:

• Authentication/authorization: For processing terminal server requests, which consist of many single operations without an explicit transaction, size the number of back ends to handle the traffic and leave the percentage of back ends available for transactions at the default value (50%).

  For example:

  -dm dm_n_be 48
  -dm dm_trans_be_max 24
  

  Normally, however, you configure the DM to perform a variety of tasks.

• Account creation: This activity uses one transaction connection for a long time and a second regular connection intermittently. You must provide two back ends for each of the accounts you expect to be created simultaneously. Your system might lock up if you do not have enough back ends. You can leave the percentage of back ends available for transactions at the default.

  For example:

  -dm dm_n_be 48
  -dm dm_trans_be_max 46
  

  The example above enables you to have 23 account creation sessions active simultaneously.

• Billing: Because all billing operations are transactions, ensure there is at least one back end capable of handling transactions for each billing program thread, plus one additional back end for the master thread searches.

  For example:

  -dm dm_n_be 24
  -dm dm_trans_be_max 22
  

  The example above enables you to have approximately 20 billing sessions (children) active simultaneously.

In general, if you need rapid response times, reduce the number of transactions waiting to be processed by adding more back ends, devoting a larger number of them to transactions, or both. For example, try increasing the number of back ends to 3 to 4 times the number of application processes. For performance, dedicate at least 80% of the back ends to processing transactions. For heavy updating and inserting environments, especially when billing is running, dedicate all but two of the back ends to transaction processing.

For example:

-dm dm_n_fe 4
-dm dm_max_per_fe 16
-dm dm_n_be 24
-dm dm_trans_be_max 22

If you configure too many back ends, the DM process uses too much memory and there is too much context switching. Conversely, if you have too few back ends, the system performs poorly and the network is overloaded as terminal servers retry the connection.

Note:

If there are not enough DM back ends, BRM may stop responding without reporting an error message.

On small BRM systems, where you might use a single DM for multiple activities, you can calculate the peak requirements for a combination of those activities and size the back ends accordingly. For example, you might need 32 connections for authentication and authorization and another 8 for the web interface. If you run billing at hours when the rest of the system is relatively quiet, you do not need additional back ends.

Note:

The number of back ends is independent of the number of front ends. That is, front ends are not tied to particular back ends because requests are transferred via the shared memory queue.

To help gauge the correct number of back ends, monitor database utilization. If it is under-utilized, you can increase the number of back ends.

Determining the Maximum Number of Back Ends Dedicated to Transactions

The maximum number of back ends dedicated to transactions (specified in the dm_trans_be_max entry) should be at least 80% of the number of back ends specified in the dm_n_be entry. For heavy transaction loads, such as when running billing, use a value that is 2 less than the dm_n_be entry. For example:

- dm dm_n_be   48
- dm dm_trans_be_max   46

Note:

You cannot specify more transaction back ends than there are total back ends.

Setting the DM Time Interval between Opcode Requests

By default, the DM back-end processes wait an infinite amount of time for each opcode request, but you can set a time interval after which the DM back-end terminates if no opcode request has arrived. The following DM pin.conf entry specifies the maximum amount of time to wait, in minutes, for an opcode call before aborting the transaction:

- dm dm_trans_timeout 4

Note:

To have DM back-end processes wait forever, set this entry to 0.

Setting How Long the DM Waits for the Background Startup Process to Finish

The DM back-end startup process connects to the BRM database and initializes the BRM data dictionary into DM memory. By default, the DM waits 60 seconds for the DM back-end startup process to finish before timing out. You can modify how long the DM waits by adding the dm_init_be_timeout entry to the DM pin.conf file.

- dm dm_init_be_timeout 60

Note:

This entry is not included in the default DM pin.conf file, so you must add it manually.

Configuring Multiple Pipelines in the DM to Improve Performance

By default, the front-end processes in the DMs write requests to a pipeline and the back-end processes listen to the pipeline and pick up the request. When all the back-end processes listen to a single pipeline, resources are wasted and performance might become slow. To improve performance, configure multiple pipelines in the DM, with each pipeline serving a set of back ends. The front-end processes send requests to the pipelines using the round-robin method. The back-end processes listen to the pipelines in order. For example, the first set of back-end processes listens to pipeline 1, the second set of processes listens to pipeline 2, and so on.

To configure multiple pipelines, include the following entry in the DM pin.conf file:

-dm dm_n_op_fifo nn

where nn is the number of pipelines. The number of your back-end processes, specified in the dm_n_be entry, must be a multiple of the pipelines you configure. The default is 1.

Tip:

Start by configuring one pipeline for every six back-end processes and adjust the number of pipelines according to your requirements.

Setting DM Shared Memory Size

BRM queuing increases system performance by lowering the number of connections to the database. This reduces the number of processes, which reduces the system load required to handle the connections. All DMs use shared memory for internal queuing. Front-end processes pass connections through a shared-memory queue to back-end processes.

To specify DM shared memory, you use the following entries in the DM configuration file (pin.conf):

• dm_shmsize: Specifies the size of the shared memory segment, in bytes, that is shared between the front ends and back ends. The maximum allowed value of dm_shmsize in the DM's pin.conf file is 274877905920 bytes (256 GB) and must be a multiple of 8192.

• dm_bigsize: Specifies the size of shared memory for ”big” shared memory structures, such as those used for large searches (with more than 128 results) or for PIN_FLDT_BUF fields larger than 4 KB.

  The maximum allowed value of dm_bigsize in the Data Manager's (DM's) pin.conf file is 206158429184 bytes (192 GB). The value of dm_bigsize should always be set less than the value of dm_shmsize and must be a multiple of 1024.

To specify DM shared memory:

1. Open the DM configuration file (BRM_home/sys/dm_oracle/pin.conf).

2. Change the configuration entry associated with each parameter. For tuning guidelines, see the discussions following this procedure. For the syntax of each configuration entry, follow the guidelines in the configuration file.

   Note:

   You may have to increase the shmmax kernel parameter for your system. It should be at least as large as the dm_shmsize entry in the DM configuration file on any computer running a DM. Otherwise, the DM will not be able to attach to all of the shared memory it might require and BRM will fail to process some transactions. See your vendor-specific system administration guide for information about how to tune the shmmax parameter.

3. Save and close the file.

4. Stop and restart the DM. See "Starting and Stopping the BRM System".

   Note:

   Besides configuring the number of connections for best performance, remember to keep the number of connections within the terms of your database license agreement.

Determining DM Shared Memory Requirements

The amount of shared memory required by a DM depends on:

• Number of front ends: Each front end takes about 32 bytes of shared memory for its status block.

• Number of connections per front end: Each connection to a front end takes at least one 8-KB block of shared memory.

• Number of back ends: Each back end takes about 32 bytes of shared memory for its status block.

• Size and type of DM operations: Most of the shared memory used is taken by DM operations, and particularly by large searches. For example:

  • Running the pin_ledger_report utility.

  • Running searches that return large numbers or results.

  • Using large zone maps. Allocate 1 MB of memory in the dm_bigsize entry for every 3000 lines in a zone map.

  Operations that read objects, read fields, or write fields and involve a large BUF field can also be significant, but they are rare. Normal operations take 0 to 16 KB above the 8-KB-per-connection overhead.

You can also reduce the requirements for shared memory by using the PCM_OP_STEP_SEARCH opcode instead of the PCM_OP_SEARCH opcode.

You should monitor the shared memory usage and the transaction queues for each DM. See "Monitoring DM Shared Memory Usage" and "Monitoring DM Transaction Queues".

How BRM Allocates Shared Memory for Searches

The dm_shmsize entry sets the total size of the shared memory pool. The dm_bigsize entry sets the size of the portion of the shared memory reserved for ”big” shared memory structures. Therefore, the memory available to front ends, back ends, and normal (not ”big”) operations is the value of the dm_shmsize entry minus the value of the dm_bigsize entry.

For example, with these entries, the shared memory available to normal operations is 25165824:

- dm dm_shmsize  33554432
- dm dm_bigsize  8388608

Note:

The value for dm_shmsize must be a multiple of 1024. The value of dm_bigsize must be a multiple of 8192.

To allocate memory for a search, BRM uses regular shared memory until the search returns more than 128 results. At that point, BRM reallocates the search to use the memory set aside for ”big” structures. When allocating this type of memory, BRM doubles the size of the initial memory requirement in anticipation of increased memory need.

For example, consider a search that returns the POIDs of accounts that need billing. For 100,000 accounts, the memory allocated to the search is as follows:

• Memory used by ”big” structures: 3.2 MB.

  The 3.2 MB figure is derived by taking the size of a POID and the anticipated number of accounts read in a billing application and then doubling the amount of memory as a safety margin.

  100,000 x 16 x 2 = 3,200,000 (3.2 MB), which is rounded up to a multiple of 8192. For example, dm_bigsize would be set to 3203072 or 391 x 8192.

  As a general rule, dm_shmsize should be approximately 4 to 6 times larger than dm_bigsize.

• Memory used by normal structures: 4 MB.

  This memory is allocated for the following:

  • 2 MB for the result account POIDs (100,000 accounts x 20-byte chunks).

  • 2 MB for the POID types (100,000 accounts x 20-byte chunks).

• Total memory use: 7.2 MB.

Shared Memory Guidelines

DM shared memory is limited to 512 MB. Billing applications, internet telephony, and searching can affect DM shared memory requirements. It is usually best to start with a lower amount of shared memory to keep system resource usage minimal.

Shared memory for database servers can be from 512 MB for medium scale installations to several GB or more for the largest installations, depending upon activities. Some experimentation is necessary because more than 1 GB may not provide a performance increase, especially if there is a lot of update activity in the BRM database.

This example shows Solaris 2.6 kernel tuning parameters (for /etc/system) for the database server:

set bufhwm=2000
set autoup=600
set shmsys:shminfo_shmmax=0xffffffff
set shmsys:shminfo_shmseg=32
set semsys:seminfo_semmns=600
set semsys:seminfo_semmnu=600
set semsys:seminfo_semume=600
set semsys:seminfo_semmsl=100
forceload:drv/vxio
forceload:drv/vxspec

Note:

This example of a Solaris kernel configuration essentially sets the maximum shared memory limit to infinity. When this setting is used, the system can allocate as much RAM as required for shared memory.

Reducing Resources Used for Search Queries

You can increase performance for search queries that retrieve objects with multiple rows from the database (for example, account searches for multiple customers) by setting the value of the dm_in_batch_size entry in the DM configuration file (pin.conf).

BRM interprets the value of dm_in_batch_size as the number of matching rows to retrieve in one search. When you start a search, BRM executes n+1 searches, where n is the number of searches performed to retrieve the number of rows set in dm_in_batch_size. For example, if dm_in_batch_size is set to 25 and the search retrieved 100 matching rows, five searches were performed ([25 x 4]+1). The default setting is 80, indicating that BRM executes two searches to retrieve up to 80 matching rows. The maximum value is 160.

To preserve resources, you set the value in dm_in_batch_size to correlate to the size of the data set being searched. To increase performance when searching large data sets, you increase the number of retrieved rows in dm_in_batch_size. The larger the value set in dm_in_batch_size, the more resources are used to perform the search query. For example, if a typical user search query returns 10 rows from the database and dm_in_batch_size is set to 100, more resources than necessary are being used to complete the search.

Load Balancing DMs

The dm_pointer entry in the CM configuration file (pin.conf) tells the CM which DM to connect to. Having pointers to several DMs provides reliability because the system will switch to another DM if one DM fails.

You can ensure a more even load among the available DMs by adding several identical pointers to each DM, even if the DMs are on the same machine. When a CM receives a connection request, it chooses one of the pointers at random. Or, you can increase the load on a particular DM by increasing the relative number of pointers to that DM.

For example, if you have two DMs and you want to ensure that most activity goes to one with the most powerful hardware, make three or four pointers to that DM and only one or two to the other DM. When new child CM processes or threads are created, more of them are configured to point to the first DM:

- cm  dm_pointer  0.0.0.1  ip  127.0.0.1 15950
- cm  dm_pointer  0.0.0.1  ip  127.0.0.1 15950
- cm  dm_pointer  0.0.0.1  ip  127.0.0.1 15950
- cm  dm_pointer  0.0.0.1  ip  127.0.0.3 11950

Optimizing Memory Allocation during Database Searches

You can configure the Oracle DM to optimize memory allocation during database searches by using the extra_search entry in the DM configuration file. When this entry is set, the Oracle DM performs an extra search in the BRM database to calculate the number of database objects meeting the search criteria and then allocates the optimal amount of memory for the results.

Important:

Performing the extra search slows database search performance.

To optimize memory allocation by performing an extra search:

1. Open the Oracle DM configuration file (BRM_home/sys/dm_oracle/pin.conf).

2. Change the extra_search entry to 1:

   - dm extra_search 1
   

3. Save and close the file.

4. Stop and restart the Oracle DM. See "Starting and Stopping the BRM System".

Improving BRM Performance during Database Searches

Oracle databases can access tables that have nonbitmap indexes by performing an internal conversion from ROWIDs to bitmap and then from bitmap back to ROWIDs. This internal conversion process can significantly decrease BRM performance when a large number of rows are queried.

To increase search performance, Oracle recommends that you prevent the database from using bitmap access paths for nonbitmap indexes. To do so, add the following parameter to your database's init.ora file or spfile, and then restart your database:

_b_tree_bitmap_plans=false

Increasing DM CPU Usage

If the CPU usage on a DM machine reaches 75% over a 60-second average, increase the CPU capacity by using a faster CPU, adding CPUs, or adding another machine to run the same type of DM.

Examples of DM Configurations

These examples show DM pin.conf file settings used with a variety of multiple CPU configurations. These examples are intended as guidelines; your settings depend on your system resources and workload.

Example 1: BRM 16-CPU database server configuration

The example depicted in Table 14-1 shows a BRM system that uses:

• A 16x450 MHz CPU database server.

• Four 6x450 MHz CPU CM/DM/EM systems.

  Note:

  The dm_shmsize entry is set to 64 MB to handle a larger billing load.

  Table 14-1 Example 1 DM Configuration

  Daemon/program	pin.conf entry	Value
  dm_oracle	dm_n_fe	6
  dm_oracle	dm_n_be	22
  dm_oracle	dm_max_per_fe	16
  dm_oracle	dm_trans_be_max	20
  dm_oracle	dm_shmsize	67108864
  dm_oracle	dm_bigsize	1048576

  
  

Example 2: BRM 36-CPU database server configuration

The example shown in Table 14-2 shows a BRM system that uses:

• A 36x336 MHz CPU database server.

• Four 4x400 MHz CPU CM/DM/EM systems.

  Table 14-2 Example 2 DM Configuration

  Daemon/program	pin.conf entry	Value
  dm_oracle	dm_n_fe	4
  dm_oracle	dm_n_be	24
  dm_oracle	dm_max_per_fe	16
  dm_oracle	dm_trans_be_max	22
  dm_oracle	dm_shmsize	20971520
  dm_oracle	dm_bigsize	6291456

  
  

Improving Interprocess Communication (IPC) Performance

By default, CM and DM processes communicate through AF_INET sockets. You can increase your system's interprocess communication (IPC) performance by configuring it to use AF_UNIX sockets between CMs and DMs that reside on the same machine and AF_INET sockets between CMs and DMs that reside on separate machines.

Both socket types are described in Table 14-3.

Table 14-3 IPC Socket Types

Socket type	Description
AF_UNIX	Provides communication through a local socket file that the DM creates each time it starts. Note: If the DM finds a socket file when it starts, it deletes the existing file and creates a new one. These sockets provide the fastest IPC performance but can be used only by processes located on the same machine.
AF_INET	Provides communication through an IP address. These sockets are slower than AF_UNIX sockets, but they allow communication between processes on separate machines.

To improve IPC performance by configuring your system to use both AF_UNIX and AF_INET sockets, perform these steps:

1. On machines containing both a CM and a DM, set the following entry in your CM configuration file (BRM_home/sys/cm/pin.conf):

   - cm dm_pointer 0.0.0.1 local BRM_home/sys/dm/dm_port
   

   If your CM also connects to DMs on other machines, such as in a multischema system, add the following entry:

   Note:

   Ensure you add a dm_pointer entry for each DM to which the CM connects.

   - cm dm_pointer 0.0.0.x ip HostName PortNumber
   

   where

   • HostName is the associated DM machine's host name or IP address.

   • PortNumber is the associated DM's port number. The default port number is 12950.

   For an example of how to configure this file for multischema systems, see "Sample Configuration Settings for a Multischema System".

2. On machines containing both a CM and a DM, set the following entries in your DM configuration file (BRM_home/sys/dm/pin.conf):

   - dm dm_port PortNumber
   - dm dm_local_socket_name BRM_home/sys/dm/dm_port
   

   where

   • PortNumber is the DM's port number. The default port number is 12950.

3. On machines containing a DM but no CM, set the following entry in your DM configuration file:

   - dm dm_port PortNumber
   

   where PortNumber is the DM's port number. The default number is 12950.

4. Save and close all files.

Sample Configuration Settings for a Multischema System

This section shows how to set your CM and DM configuration files for the following sample multischema system shown in Figure 14-4:

Figure 14-4 Sample Multischema System

Description of ''Figure 14-4 Sample Multischema System''

CM pin.conf file

- cm dm_pointer 0.0.0.1 local BRM_home/sys/dm/dm_port
- cm dm_pointer 0.0.0.2 ip HostName2 PortNumber2
- cm dm_pointer 0.0.0.3 ip HostName3 PortNumber3

DM1 pin.conf

- dm dm_port PortNumber1
- dm dm_local_socket_name BRM_home/sys/dm/dm_port

DM2 pin.conf

- dm dm_port PortNumber2

DM3 pin.conf

- dm dm_port PortNumber3

Improving Performance by Disabling Unused Features

You can improve performance by disabling features that you do not use. To do so, you edit /config/business_params objects. See "pin_bus_params" in BRM Developer's Guide.

Table 14-4 lists the features enabled by /config/business_params objects that most heavily impact performance.

Table 14-4 BRM Features that Heavily Impact Performance

Group	Name	Description	Default Value
activity	LightWeightAuthorization	Enables and disables lightweight authorization. See "Using Lightweight Authorization" in BRM Telco Integration.	Disabled
invoicing	SubARItemsIncluded	Indicates whether subordinate account invoices include A/R items. See "About Invoicing for Hierarchical Account Groups" in BRM Designing and Generating Invoices.	A/R items are included
billing	CacheResidencyDistinction	Enables and disables object cache residency distinction. See "About Convergent BRM Systems".	Disabled
billing	EnableARA	Specifies whether Revenue Assurance is enabled for out-of-cycle billing. See "Configuring Bill Now, On-Demand Billing, and Auto-Triggered Billing to Collect Revenue Assurance Data" in BRM Collecting Revenue Assurance Data.	Disabled
billing	GeneralLedgerReporting	Enables and disables general ledger reporting. If disabled, the journal is not generated. See "Enabling and Disabling General Ledger Collection in BRM" in BRM Collecting General Ledger Data.	Enabled
rating	EnableEras	Specifies how to enable ERAs: 0 = No profiles 1 = Accounts only 2 = Services only 3 = All See "Filtering the ERAs Considered during Rating and Discounting".	All service and account profiles
rating	EnableGlobalChargeSharing	Enables and disables global charge sharing. See "About Global Charge Sharing Groups" in BRM Managing Accounts Receivable.	Disabled
rating	EnableTailormadeCache	Specifies if tailor-made products must be maintained in the rating cache. When enabled, tailor-made products are stored in cache. See "Enabling and Disabling the Caching of Customized Products".	Disabled
multi-bal	BalanceMonitoring	Enables and disables balance monitoring. See "About Balance Monitoring" and "Enabling Balance Monitoring in BRM" in BRM Managing Accounts Receivable.	Disabled
multi-bal	LockConcurrency	Indicates the concurrency of object locking. Possible values are: Normal (0): Locks the account object. High (1): More concurrency of locking with greater granularity of which balance group to lock. This setting is best for performance. See "Disabling Granular Object Locking" in BRM Developer's Guide.	High
rerate	BatchRatingPipeline	Specifies whether the batch-rating pipeline is active during rerating. See "Specifying Whether the Batch Rating Pipeline Is Enabled" in BRM Setting Up Pricing and Rating.	Enabled
subscription	GetRatePlanFromCache	Specifies whether to retrieve rate plans from the cache during product purchase. See "Improving Performance in Retrieving Product Details During Product Purchase".	Enabled

Tuning Billing Performance by Using Configuration Files

To tune billing performance, you edit entries in the Performance Entries section of the configuration file that is used by all billing utilities (BRM_home/apps/pin_billd/pin.conf). You can set values for the following:

• In case of certain BRM search operations that return large amounts of data, you can filter and find accounts in resource sharing groups. See "Filtering Search Results".

• The number of child threads (or processes) that the billing utilities use. See "Tuning the Number of Children for Billing Utilities".

• The number of accounts to cache in system memory before BRM processes the accounts. For example, if you specify 50000, BRM finds and caches 50,000 accounts before processing them. See "Tuning the Account Cache Size for Billing Utilities (fetch_size)".

• How many accounts the child threads can find and move into the cache simultaneously. This ensures that the DM memory is not overloaded. See "Tuning the Batch Size for Billing Utilities (per_batch)".

• The number of accounts sent in batches to the credit card processing service. See "Tuning the Batch Size for the pin_collect Utility".

• The number of retries in case a deadlock occurs. See "Specifying the Number of Retries in a Deadlock".

• How accounts cached in system memory are distributed to child threads. See "Rearranging Accounts to Improve Billing Performance".

In the configuration file, entries preceded by -pin_billd apply to all billing utilities. Entries preceded by a specific utility name override the general entry for that utility. In this example, the children and fetch_size entries are used by all billing utilities, but the pin_bill_accts and pin_collect utilities have different values for the per_batch entry.

- pin_billd children 5
- pin_billd fetch_size 10000
- pin_billd per_batch 2500
- pin_bill_accts per_batch 1000
- pin_collect per_batch 200

The children, fetch_size, and per_batch entries are standard configuration options for multithreaded applications. For information about these entries, see "Configuring Your Multithreaded Application" in BRM Developer's Guide.

See the comments in the configuration file (BRM_home/apps/pin_billd/pin.conf) for more information.

For billing performance issues, see "Tuning Memory for Billing". For billing performance issues not related to the configuration file, see "Additional Issues Related to Billing Performance".

Filtering Search Results

Some BRM operations, such as searching for group members or searching for items to include on an invoice, can return large amounts of data and cause the Data Manager to fail. In this case, use the following configuration file entries to use a step search to find accounts in resource sharing groups:

• group_members_fetch: Use this entry to search for members of the group sharing object when the parent group contains many members. The parameter is specified as - fm_bill group_members_fetch n, where n is the size of pool (memory in bytes) for the step search. The default value is 0.

• group_children_fetch: Use this entry to search for child accounts in groups when the parent group contains many members. The parameter is specified as - fm_bill group_children_fetch n, where n is the size of pool (memory in bytes) for the step search. The default value is 0.

• item_fetch_size: Use this entry when searching for items. The parameter is specified as - fm_bill item_fetch_size n, where n is the size of pool (memory in bytes) for the step search. The default value is 0.

To filter search results:

1. Open the Connection Manager (CM) configuration file (BRM_home/sys/cm/pin.conf) in a text editor.

2. Uncomment or enter the following lines, as needed:

   - fm_bill group_members_fetch n
   - fm_bill group_children_fetch n
   - fm_bill item_fetch_size n
   

   where n is the size of pool (memory in bytes) for the step search.

3. Save and close the file.

Tuning the Number of Children for Billing Utilities

By default, a billing utility uses five child threads to process accounts. You can increase the number in the children entry to get better billing performance when the database server remains under-utilized even though you have a large number of accounts. If you increase the number of children beyond the optimum, performance suffers from context switching. This is often indicated by higher system time with no increase in throughput.

Billing performance is best when the number of children is nearly equal to the number of DM back ends and most back ends are dedicated to processing transactions. For information on adjusting the number of DM back ends, see "Configuring DM Front Ends and Back Ends".

To tune the number of children:

1. Open the billing utilities configuration file (BRM_home/apps/pin_billd/pin.conf).

2. In the Performance Entries section, edit the children entry:

   - pin_billd children 5
   

3. Save and close the file.

Tuning the Account Cache Size for Billing Utilities (fetch_size)

If there is enough memory, make the value in the fetch_size entry equal to or greater than the number of accounts to be billed. If memory is limited, you can use a fetch_size entry of 10,000 and a per_batch entry of 2500.

Tip:

For best performance, use a fetch_size value that is a multiple of the per_batch value.

To tune the account cache size:

1. Open the billing utilities configuration file (BRM_home/apps/pin_billd/pin.conf).

2. In the Performance Entries section, edit the fetch_size entry:

   - pin_billd fetch_size 10000
   

3. Save and close the file.

Tuning the Batch Size for Billing Utilities (per_batch)

When processing a batch of accounts before moving them into the cache, the billing utility waits for all children to finish processing before getting another batch of accounts. Therefore, a single account that requires a long processing time can reduce performance because the idle child threads must wait for one child thread to finish processing the account. To minimize this loss in productivity, you can increase the batch size. If you increase the batch size too much, you will run out of memory.

To tune the batch size:

1. Open the billing utilities configuration file (BRM_home/apps/pin_billd/pin.conf).

2. In the Performance Entries section, edit the per_batch entry:

   - pin_billd per_batch 2500
   

3. Save and close the file.

Tuning the Batch Size for the pin_collect Utility

The pin_collect utility uses the per_batch entry differently than other billing utilities use it:

• For most billing utilities, the per_batch entry specifies the number of accounts that BRM finds and caches in system memory.

• For the pin_collect utility, the per_batch entry specifies the number of accounts sent in batches to the credit card processing service.

For best performance, the value in the per_batch entry for the pin_collect utility should be much smaller than the per_batch entry for the rest of the billing utilities. Use a value that is equal to the per_batch entry that applies to the pin_bill_accts entry divided by the value of the children entry. (The pin_bill_accts value can be set either by a generic -pin_billd entry in the billing utility configuration file or by a specific -pin_bill_accts entry.)

For example, if the per_batch entry for pin_bill_accts is 3000, and you use 10 child threads, the per_batch entry for the pin_collect utility should be 300.

To tune the batch size for the pin_collect utility:

1. Open the billing utility configuration file (BRM_home/apps/pin_billd/pin.conf).

2. In the Performance Entries section, add a per_batch entry for pin_collect:

   - pin_collect per_batch 500
   

3. Save and close the file.

Specifying the Number of Retries in a Deadlock

A deadlock occurs when two or more database sessions have some data locked, and each database session requests lock on the data that the other session has already locked. The database sessions wait on each other to release the lock on the data. Billing utilities, such as pin_bill_accts and pin_inv_accts, retry the operation in a deadlock. By default, the number of retries is set to 20, but you can change that number. Retries are attempted with no time delay in between. You cannot configure time delay between retries. For information on the deadlock error, see "Reference Guide to BRM Error Codes".

To specify the number of times to retry an operation when a deadlock occurs:

1. Open the billing utilities configuration file (BRM_home/apps/pin_billd/pin.conf).

2. Uncomment the - pin_bill_accts deadlock_retry_count entry.

3. Edit the deadlock_retry_count entry:

   - pin_bill_accts  deadlock_retry_count  20
   

4. Save and close the file.

Rearranging Accounts to Improve Billing Performance

Billing utilities fetch accounts and cache them to system memory in the same sequence in which they are stored in the BRM database.

Note:

The number of accounts fetched from the database is determined by the fetch_size entry in the billing utilities configuration file.

Each account in memory is then distributed to individual child threads (or processes) for billing. This behavior may slow billing performance because of database contention.

You can sometimes improve billing performance by rearranging accounts in system memory before distributing the accounts to child threads for processing by using the delta_step entry in the billing utility configuration file.

When a value is specified for this parameter, the billing utilities rearrange accounts cached in system memory based on the parameter value specified. Generally, Oracle database retrieves the accounts in the order in which they are found in the database, grouped according to their physical location on the disk.

For example, we might have 100 accounts to bill and 10 threads, as well as 10 database blocks (A, B, C, D, E, F, G, H, I, and J) that each contain 10 accounts. The database returns a list that looks like this: A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, and so on for each block. When BRM starts its threads, each of the 10 threads gets the next available account to process, and the mapping might look like this:

Thread 1  - A1
Thread 2  - A2
Thread 3  - A3
Thread 4  - A4
Thread 5  - A5
Thread 6  - A6
Thread 7  - A7
Thread 8  - A8
Thread 9  - A9
Thread 10 - A10

When a thread finishes processing an account, it takes the next available account from the list, and processing continues until all accounts have been processed. As a result, all threads at any given time may be accessing accounts in the same database blocks and vying for the same resources.

You can change the order in which these accounts are processed using the delta_step parameter. For example, to rearrange accounts by selecting and placing every tenth account cached in system memory, and then distribute these accounts to threads for billing, set this entry to 10:

pin_billd delta_step 10

The thread mapping, instead of proceeding one account at a time, would look something like this:

Thread 1  - A1
Thread 2  - B1
Thread 3  - C1
Thread 4  - D1
Thread 5  - E1
Thread 6  - F1
Thread 7  - G1
Thread 8  - H1
Thread 9  - I1
Thread 10 - J1

The delta_step parameter makes it possible for each thread to be working on data from a different area of the database, reducing contention for the same resources and improving billing performance.

You can determine the optimal setting for the delta_step parameter by testing the billing processes and monitoring their performance. By default, this parameter is set to 0, which means that the accounts cached in system memory are not rearranged before distribution.

To rearrange accounts in system memory:

1. Open the billing utility configuration file (BRM_home/apps/pin_billd/pin.conf).

2. In the Performance Entries section, edit the delta_step entry:

   - pin_billd delta_step 10
   

3. Save and close the file.

Improving Performance in Retrieving Purchased Offerings for a Bill Unit

You can improve billing performance while retrieving purchased products and discounts for a bill unit (/billinfo object) from the database, by specifying the batch size of the number of services to search at a time in the MaxServicesToSearch parameter in the /config/business_params object for subscription.

To specify the batch size of the number of services to search:

1. Create an editable XML file for the subscription parameter instance by using the following command:

   pin_bus_params -r -c "Subscription" bus_params_subscription.xml
     
   

   This command creates the XML file bus_params_subscription.xml.out in your working directory. If you do not want this file in your working directory, specify the full path as part of the file name.

2. Open the bus_params_subscription.xml.out file in a text editor.

3. In the BusParamsSubscription section, specify the batch size of the number of services to search by changing the value of the MaxServicesToSearch tag as follows:

   <BusParamsSubscription>
   ...
      <MaxServicesToSearch>5</MaxServicesToSearch>
   </BusParamsSubscription>
     
   

   Note:

   The default value is 5. The minimum allowed value is 1.The maximum allowed value depends on the length of the POIDs involved in the search and the search criteria. Depending on the length of the POIDs involved in the search, if you increase the value of MaxServicesToSearch and the length of the select statement is more than 2048 characters, the database select statement could fail.

4. Save the file as bus_params_subscription.xml.

5. Go to the BRM_home/sys/data/config directory and load the change into the /config/business_params object by using the following command:

   pin_bus_params bus_params_subscription.xml 
   

   Note:

   To run the utility from a different directory, see "pin_bus_params" in BRM Developer's Guide.

6. Read the object with the testnap utility or the Object Browser to verify that all fields are correct.

   See "Reading an Object and Fields" in BRM Developer's Guide.

7. Stop and restart the Connection Manager (CM).

   See "Starting and Stopping the BRM System".

8. Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter.

   For more information, see "pin_multidb".

Tuning Memory for Billing

If you change the children and per_batch entries for billing utilities, you should also review the dm_shmsize configuration entry for the DM. You can estimate the amount of shared memory required by using the following formulas:

• To run billing (pin_bill_accts): The amount of shared memory required, in kilobytes, is equal to 0.45 multiplied by the number specified in the per_batch entry.

• To run credit-card collection (pin_collect): The amount of shared memory required, in kilobytes, is equal to the number specified in the children entry multiplied by the number specified in the per_batch entry.

Additional Issues Related to Billing Performance

• To reduce system load, you can split large billing runs into smaller billing runs. See "Reducing Billing Run Loads" in BRM Configuring and Running Billing.

• You can improve performance of the pin_bill_accts utility by excluding accounts that do not need to be billed. For more information, see "About Suspending Billing of Accounts and Bills" in BRM Configuring and Running Billing.

• When you run the pin_collect utility, you can improve performance by archiving the temporary transmission logs created by the DM for the credit card processing service. See "Maintaining Transmission Logs for Billing Transactions" in BRM Configuring and Running Billing.

• You can create billing-related indexes just before you run billing and then delete them when billing finishes. You can add these tasks to the billing scripts. See "Editing the Billing Scripts" in BRM Configuring and Running Billing and "Removing Unused Indexes".

How the Number of Events Affects Billing

The number of events has no effect on billing or invoicing except in the following cases:

• If you defer tax calculations, billing performance is slower.

• If you create detailed invoices, performance is slower. A telephone usage invoice is an example of a detailed invoice; a fixed-fee cable subscription invoice is an example of a nondetailed invoice.

Tuning Billing Performance by Using Business Parameters

You can improve the billing performance by doing the following:

• Improving Performance by Skipping Previous Total Unpaid Bill for Open Item Accounting Type

• Improving Trial Billing Performance by Enabling General Ledger Collection

• Excluding Searches on Closed Offerings

• Excluding Searches on Overridden Products

Improving Performance by Skipping Previous Total Unpaid Bill for Open Item Accounting Type

You can configure BRM to skip the previous total unpaid bill for the open item accounting type when calculating the current bill by setting the PerfAdvancedTuningSettings performance tuning business parameter. Using this business parameter, you can reduce the time taken for calculating the current bill for the open item accounting type.

To skip previous total unpaid bill for open item accounting type:

1. Use the following command to create an editable XML file from the billing instance of the /config/business_params object:

   pin_bus_params -r BusParamsBilling bus_params_billing.xml
   

   This command creates the XML file named bus_params_billing.xml.out in your working directory. To place this file in a different directory, specify the full path name for the file. For more information on this utility, see "pin_bus_params" in BRM Developer's Guide.

2. Open the bus_params_billing.xml.out file.

3. Search for the following line:

   <PerfAdvancedTuningSettings>0</PerfAdvancedTuningSettings>
   

4. Add 8 to the existing value to skip the previous unpaid bill. For example,

   <PerfAdvancedTuningSettings>8</PerfAdvancedTuningSettings>
   

   For more information on the parameter values, see the PerfAdvancedTuningSettings parameter description in Table 45-3, "Billing business_params Entries".

   Caution:

   BRM uses the XML in this file to overwrite the existing billing instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM subscription configurations.

   Note:

   Setting the PerfAdvancedTuningSettings business parameter will impact all the bill units associated with the open item accounting type. To impact specific bill units of an open item accounting type, see "Improving Performance by Skipping Previous Total for Open Item Accounting Type When Calculating the Current Bill".

5. Save the file as bus_params_billing.xml.

6. Go to the BRM_home/sys/data/config directory which includes support files used by the pin_bus_params utility.

7. Use the following command to load this change into the appropriate /config/business_params object.

   pin_bus_param_ pathToWorkDirectory/bus_params_billing.xml
   

   where pathToWorkingDirectory is the directory in which bus_params_billing.xml resides.

   Note:

   To execute it from a different directory, see the description for pin_bus_params in BRM Developer's Guide.

8. Read the object with the testnap utility or Object Browser to verify that all fields are correct.

   For more information on reading objects by using the Object Browser, see BRM Managing Customers. For instructions on using the testnap utility, see BRM Developer's Guide.

9. Stop and restart the Connection Manager. See "Starting and Stopping the BRM System".

10. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Improving Trial Billing Performance by Enabling General Ledger Collection

You can configure BRM to enable the general ledger (G/L) collection for trial billing by setting the PerfAdvancedTuningSettings business parameter to 64. Using this value, you can collect the G/L data required for creating trial invoices.

To enable the G/L collection for trial billing:

1. Use the following command to create an editable XML file from the billing instance of the /config/business_params object:

   pin_bus_params -r BusParamsBilling bus_params_billing.xml
   

   This command creates the XML file named bus_params_billing.xml.out in your working directory. To place this file in a different directory, specify the full path name for the file. For more information on this utility, see "pin_bus_params" in BRM Developer's Guide.

2. Open the bus_params_billing.xml.out file.

3. Search for PerfAdvancedTuningSettings.

4. Add 64 (0x40, set Bit 6) to the existing value of PerfAdvancedTuningSettings.

   For example, if the existing value is 0, change it to 64 (0+64):

   <PerfAdvancedTuningSettings>64</PerfAdvancedTuningSettings>
   

   For more information on the parameter values, see the PerfAdvancedTuningSettings parameter description in Table 45-3, "Billing business_params Entries".

   Caution:

   BRM uses the XML in this file to overwrite the existing billing instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM subscription configurations.

5. Save the file as bus_params_billing.xml.

6. Go to the BRM_home/sys/data/config directory which includes support files used by the pin_bus_params utility.

7. Use the following command to load this change into the appropriate /config/business_params object.

   pin_bus_param_ pathToWorkDirectory/bus_params_billing.xml
   

   where pathToWorkingDirectory is the directory in which bus_params_billing.xml resides.

   Note:

   To execute it from a different directory, see the description for pin_bus_params in BRM Developer's Guide.

8. Read the object with the testnap utility or Object Browser to verify that all fields are correct.

   For more information on reading objects by using the Object Browser, see BRM Managing Customers. For instructions on using the testnap utility, see BRM Developer's Guide.

9. Stop and restart the Connection Manager. See "Starting and Stopping the BRM System".

10. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Excluding Searches on Closed Offerings

By default, BRM retrieves active, inactive, and closed offerings during the billing process. However, most of the time, BRM does not use the data from the closed offerings.

You can configure BRM to retrieve only the active and inactive offerings by disabling the CancelledOfferingsSearch parameter in the subscription instance of the /config/business_params object.

You modify the /config/business_params object by using the pin_bus_params utility. See "pin_bus_params" in BRM Developer's Guide.

To disable searches on closed offerings:

1. Use the following command to create an editable XML file from the subscription instance of the /config/business_params object:

   pin_bus_params -r BusParamsSubscription bus_params_subscription.xml

   This command creates the XML file named bus_params_subscription.xml.out in your working directory. To place this file in a different directory, specify the full path name for the file. For more information on this utility, see "pin_bus_params" in BRM Developer's Guide.

2. Locate the CancelledOfferingsSearch entry in the bus_params_subscription.xml.out file.

3. Set the value of CancelledOfferingsSearch to disabled, if necessary:

   <CancelledOfferingsSearch>disabled</CancelledOfferingsSearch>

4. Save this updated file as bus_params_subscription.xml.

5. Load the modified XML file into the appropriate /config/business_params object in the BRM database.

   pin_bus_params bus_params_subscription.xml

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

6. Read the object with the testnap utility or Object Browser to verify that all fields are correct.

   For more information on reading objects by using the Object Browser, see BRM Managing Customers. For instructions on using the testnap utility, see BRM Developer's Guide.

7. Stop and restart the Connection Manager. See "Starting and Stopping the BRM System".

8. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Excluding Searches on Overridden Products

By default, BRM retrieves the overridden products during the billing process.

If you are not using tailor-made products, you can configure BRM to ignore and not retrieve the overridden products by disabling the TailormadeProductsSearch parameter in the subscription instance of the /config/business_params object.

You modify the /config/business_params object by using the pin_bus_params utility. See "pin_bus_params" in BRM Developer's Guide.

To disable searches on overridden products:

1. Use the following command to create an editable XML file from the subscription instance of the /config/business_params object:

   pin_bus_params -r BusParamsSubscription bus_params_subscription.xml

   This command creates the XML file named bus_params_subscription.xml.out in your working directory. To place this file in a different directory, specify the full path name for the file. For more information on this utility, see "pin_bus_params" in BRM Developer's Guide.

2. Locate the TailormadeProductsSearch entry in the bus_params_subscription.xml.out file.

3. Set the value of TailormadeProductsSearch to disabled, if necessary:

   <TailormadeProductsSearch>disabled</TailormadeProductsSearch>

4. Save this updated file as bus_params_subscription.xml.

5. Load the modified XML file into the appropriate /config/business_params object in the BRM database.

   pin_bus_params bus_params_subscription.xml

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

6. Read the object with the testnap utility or Object Browser to verify that all fields are correct.

   For more information on reading objects by using the Object Browser, see BRM Managing Customers. For instructions on using the testnap utility, see BRM Developer's Guide.

7. Stop and restart the Connection Manager. See "Starting and Stopping the BRM System".

8. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Tuning Billing Performance by Using Business Profiles

To tune billing performance, you set entries in the business profile associated with the bill unit.

For general information about business profiles, see ”Managing Business Profiles” in BRM Managing Customers.

Improving Performance by Skipping Previous Total for Open Item Accounting Type When Calculating the Current Bill

By default, the previous unpaid bill is calculated even for an open item accounting type when calculating the current bill for a bill unit (/billinfo object), but the final bill does not include the previous unpaid amount. This is a time-consuming process as it involves checking all the previous bill items.

You can skip the previous total unpaid bill for the open item accounting type when calculating the current bill by setting the Skip_Prev_Total business profile key to improve the BRM performance. For making this performance improvement for all bill units system-wide, see "Improving Performance by Skipping Previous Total Unpaid Bill for Open Item Accounting Type".

To skip the previous total for open item accounting type when calculating the current bill:

1. Open the pin_business_profile.xml file in an XML editor or a text editor.

   By default, the file is in the BRM_home/sys/data/config directory.

2. Set the following business profile key value to Yes:

   <NameValue key="Skip_Prev_Total" value="Yes"/>
   

   When the value is set to Yes, previous unpaid bill amount is not calculated for the current bill.

   When the value is set to No, previous unpaid bill is calculated for the current bill. This is the default behavior when the entry is not in the file.

3. Save and close the file.

4. Run the following command, which loads the updated contents of the pin_business_profile.xml file into the BRM database:

   load_pin_business_profile -v pin_business_profile.xml
   

5. Read the object with the testnap utility or Object Browser and verify your changes.

   For general instructions on using testnap, see "Using testnap" in BRM Developer's Guide.

   For information on how to use Object Browser, see "Reading Objects by Using Object Browser" in BRM Developer's Guide.

6. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

For general information about business profiles, see ”Managing Business Profiles” in BRM Managing Customers.

Improving Performance by Using Multiple Item Configurations

By default, BRM uses the same item-tag-to-item-type mapping (item configuration) for all bill units in the system. The item configuration is used to assign bill items to events during the rating process; it also specifies which bill items are pre-created at the beginning of each billing cycle. For more information about the item configuration, see ”Creating Custom Bill Items” in BRM Configuring and Running Billing.

Using the same item configuration for all bill units can degrade performance by unnecessarily pre-creating bill items for certain types of bill units. For example, the default item configuration might pre-create cycle forward and cycle arrears items, which are normally used to track charges for postpaid bill units but are typically not required for prepaid bill units. Pre-creating such items for prepaid bill units needlessly consumes database storage space and takes more time to process during billing.

To generate fewer bill items, you can create multiple item configurations in your system and then assign the appropriate item configuration to each bill unit.

To assign an item configuration to a bill unit:

1. Create the appropriate item configuration for the bill unit.

   To create multiple item configurations, see ”Setting Up BRM to Assign Custom Bill Items to Events” in BRM Configuring and Running Billing.

2. Open the pin_business_profile.xml file in an XML editor or a text editor.

   By default, the file is in the BRM_home/sys/data/config directory.

3. In the business profile associated with the bill unit, add the following key-value pair if it does not already exist, and set the value to the name of the appropriate item configuration:

   <NameValue key="Item_Configuration" value="Item_Configuration_Name" />
   

   Where Item_Configuration_Name is the name of the item configuration to which you want to assign the bill unit.

   The name of the default item configuration is Default.

   Item configuration names are specified in the PIN_FLD_NAME field of /config/item_tags and /config/item_types objects. A pair of those objects with matching PIN_FLD_NAME values exists for each item configuration defined in your system. For more information, see ”Setting Up BRM to Assign Custom Bill Items to Events” in BRM Configuring and Running Billing.

   Caution:

   Modifying the business profile affects all the bill units associated with the business profile.

4. Save and close the file.

5. Run the following command, which loads the updated contents of the pin_business_profile.xml file into the BRM database:

   load_pin_business_profile -v pin_business_profile.xml
   

6. Read the object with the testnap utility or Object Browser and verify your changes.

   For general instructions on using testnap, see "Using testnap" in BRM Developer's Guide.

   For information on how to use Object Browser, see "Reading Objects by Using Object Browser" in BRM Developer's Guide.

7. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

For more information about business profiles, see "Managing Business Profiles" in BRM Managing Customers.

Improving Performance by Skipping Billing-Time Tax Calculation

BRM can calculate taxes during real-time rating, pipeline batch rating, or billing. See "Choosing When to Calculate Taxes" in BRM Calculating Taxes.

Calculating taxes at billing time (deferred taxation) can be a time-consuming process that involves searching all events and items in a billing cycle. It is triggered by regular billing, Bill Now, Bill on Demand, and trial billing.

In some cases, such as for prepaid accounts, taxes typically do not need to be calculated at billing time. If your system is configured to calculate billing-time taxes, you can improve billing performance by skipping billing-time tax calculation for individual bill units when it is not needed.

Important:

• To skip billing-time tax calculation for a bill unit, you configure the bill unit's business profile. Modifying the business profile affects all bill units associated with the business profile.

• If a bill unit hierarchy is configured to have billing-time taxes calculated for the entire hierarchy when the paying parent bill unit is billed, do not configure the paying parent bill unit to skip billing-time tax calculation. BRM ignores the skip billing-time tax calculation setting for nonpaying bill units in such hierarchies.

To skip billing-time tax calculation for individual bill units:

1. Open the pin_business_profile.xml file in an XML editor or a text editor.

   By default, the file is in the BRM_home/sys/data/config directory.

2. In a business profile associated with bill units whose taxes you do not want to calculate at billing time, add the following key-value pair if it does not already exist, and set the value to Yes:

   <NameValue key="Skip_Deferred_Taxation" value="Yes" />
   

   When the value is set to Yes, taxes are not calculated for the bill unit at billing time.

   When the value is set to No, taxes are calculated for the bill unit at billing time. This is the default behavior when the entry is not in the file.

3. Save and close the file.

4. Run the following command, which loades the updated contents of the pin_business_profile.xml file into the BRM database:

   load_pin_business_profile -v pin_business_profile.xml
   

5. Read the object with the testnap utility or Object Browser and verify your changes.

   For general instructions on using testnap, see "Using testnap" in BRM Developer's Guide.

   For information on how to use Object Browser, see "Reading Objects by Using Object Browser" in BRM Developer's Guide.

6. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

Tuning Invoicing Performance

Invoice utilities are multithreaded applications (MTAs) and use a similar set of configuration entries as the billing utilities, including children, fetch_size, and per_step. For information about these entries, see "Configuring Your Multithreaded Application" in BRM Developer's Guide.

Not all the invoice utilities use the entries in the same way, so you can configure them individually. To specify an entry for a particular utility, replace the generic name - pin_mta with the name of the specific utility. For example, you might have these two entries for fetch_size:

- pin_mta          fetch_size 30000
- pin_inv_accts    fetch_size 50000

See the comments in the configuration file (BRM_home/apps/pin_inv/pin.conf) for more information.

Setting the Number of Children for Invoice Utilities

Because the invoice utilities work faster than the billing utilities, the number of children for invoicing can be up to 50% more than for billing or credit card processing.

You must tune the DM configuration file dm_shmsize entry to handle the number of children. A typical value for the dm_shmsize entry is the size of an invoice (in bytes) multiplied by the number specified in the children entry.

To set the number of children for invoice utilities:

1. Open the invoice utilities configuration file (BRM_home/apps/pin_inv/pin.conf).

2. In the Performance Parameters section, edit the children entry:

   - pin_mta children 2500
   

3. Save and close the file.

Tuning the Account Cache Size for Invoice Utilities (fetch_size)

If enough memory is available, set the value of the fetch_size entry to the number of accounts that must be invoiced. The fetch_size value should be a multiple of the number specified in the per_step entry.

To change the account cache size for invoice utilities:

1. Open the invoice utilities configuration file (BRM_home/apps/pin_inv/pin.conf).

2. In the Performance Parameters section, edit the fetch_size entry:

   - pin_mta fetch_size 4000
   

3. Save and close the file.

Setting the Batch Size for Invoice Utilities (per_step)

The recommended value for the per_step entry is 100 times the number specified in the children entry. Too high a value for the per_step entry can overload the DM memory.

To set the batch size for invoice utilities:

1. Open the invoice utilities configuration file (BRM_home/apps/pin_inv/pin.conf).

2. In the Performance Parameters section, edit the per_step entry:

   - pin_mta per_step 250000
   

3. Save and close the file.

Optimizing Invoicing Performance

To improve invoicing performance, you can set the inv_perf_features flag in the Connection Manager's configuration file (pin.conf) to enable or disable specific optimizations during PCM_OP_IN_MAKE_INVOICE opcode processing.

- fm_inv inv_perf_features 0x00000000 

This flag is a bitmask and each bit position represents a performance optimization that can be turned on or off. By default, it is set to 0x00000000 (no optimizations are enabled).

Table 14-5 lists the bit values and their significance:

Table 14-5 Invoicing Performance Bit Values

Value	Description
0x00000001	Selects all event types. By default, only events that are configured in the /config/invoice_events object are selected.
0x00000002	Selects only the following event types: /event/billing/adjustment/account /event/billing/adjustment/event /event/billing/adjustment/item /event/billing/adjustment/tax_event /event/billing/cycle/tax /event/billing/payment/cash /event/billing/payment/cc /event/billing/payment/check /event/billing/payment/dd /event/billing/payment/payorder /event/billing/payment/postalorder /event/billing/payment/wtransfer /event/billing/product/fee/cancel /event/billing/product/fee/cycle /event/billing/product/fee/cycle/cycle_arrear /event/billing/product/fee/cycle/cycle_forward_bimonthly /event/billing/product/fee/cycle/cycle_forward_monthly /event/billing/product/fee/cycle/cycle_forward_quarterly /event/billing/product/fee/cycle/cycle_forward_semiannual /event/billing/product/fee/purchase /event/billing/refund/cash /event/billing/refund/cc
0x00000003	/event/billing/refund/check /event/billing/refund/dd /event/billing/refund/payorder /event/billing/refund/postalorder /event/billing/refund/wtransfer /event/billing/reversal/cc /event/billing/reversal/check /event/billing/reversal/dd /event/billing/reversal/payorder /event/billing/reversal/postalorder /event/billing/reversal/wtransfer /event/billing/settlement/item /event/billing/writeoff/account /event/billing/writeoff/bill /event/billing/writeoff/item /event/billing/writeoff/tax_account /event/billing/writeoff/tax_bill /event/billing/writeoff/tax_item /event/session/dialup
0x00000004	Creates the invoices but does not write them to the database. The bill object is updated with the invoice information.
0x00000008	Invoicing passes the input flist by reference. By default, invoicing creates a copy of the input flist. For large invoice input flists, setting this flag saves memory.
0x00000010	Events with no balance impacts are retained. If this flag is not set, events with no balance impacts are dropped from the final invoice.
0x00000400	Specifies to keep balance impacts for sponsored events in the member accounts.

To enable multiple optimizations, you can OR the bits. For example, to select hard coded list of event types and to not write invoices to the database, set the flag to 0x00000006.

Improving Performance in Retrieving Product Details During Product Purchase

By default, BRM retrieves product details from the rating cache during product purchase. However, reading a large number of rate plans for retrieving product details can consume a lot of time. This slows down product purchase.

To speed up the product purchase process, you can configure BRM to directly retrieve the product details from the database instead of retrieving it from the rating cache during product purchase. You can do this by setting the GetRatePlanFromCache field in the subscription instance of the /config/business_params object to disabled.

Note:

Setting the GetRatePlanFromCache field to disabled will impact the billing performance.

To improve performance in retrieving product details during product purchase:

1. Use the following command to create an editable XML file from the subscription instance of the /config/business_params object:

   pin_bus_params -r BusParamsSubscription bus_params_subscription.xml 
   

   This command creates the XML file named bus_params_subscription.xml.out in your working directory. If you do not want this file in your working directory, specify the full path as part of the file name.

2. Search the XML file for the following line:

   <GetRatePlanFromCache>enabled</GetRatePlanFromCache>
   

3. Change enabled to disabled.

   Caution:

   BRM uses the XML in this file to overwrite the existing rating instance of the /config/business_params object. If you delete or modify any other parameters in the file, these changes affect the associated aspects of the BRM rating configuration.

4. Save the file.

5. Change the file name from bus_params_subscription.xml.out to bus_params_subscription.xml.

6. Use the following command to load the change into the /config/business_params object:

   pin_bus_params bus_params_subscription.xml 
   

   You should execute this command from the BRM_home/sys/data/config directory, which includes support files used by the utility. To execute it from a different directory, see "pin_bus_params" in BRM Developer's Guide.

7. To verify that all fields are correct, you can display the /config/business_params object by using Object Browser or by using the robj command with the testnap utility.

   For information on using testnap, see "Using testnap" in BRM Developer's Guide.

   For information on how to use Object Browser, see "Reading Objects by Using Object Browser" in BRM Developer's Guide.

8. Stop and restart the CM. For more information, see "Starting and Stopping the BRM System".

9. (Multischema systems only) Run the pin_multidb script with the -R CONFIG parameter. For more information, see "pin_multidb".

Improving Data Processing Performance

A significant amount of time can be consumed in parsing SQL statements that read or write in the database. Adjusting an account, rating an event, and many other BRM activities require several steps to read and write data.

SQL statement-handle caching increases the speed at which statements are parsed.

Note:

Statement-handle caching is available with Oracle databases only.

How Statement-Handle Caching Works

An application sends an opcode through the CM to a DM, which maps each PCM operation to one or more dynamic SQL statements. For each such statement, the relational database management system (RDBMS)—such as Oracle Database—parses the statement, executes it, and fetches the results.

Oracle database maintains a cache of the most frequent SQL queries. It uses soft parsing to shortcut its process of deciphering these statements, thus saving time in retrieving the requested data from the actual database. It then sends the data back through the DM to the application.

BRM generates and repeats a finite set of SQL statement forms. If the caching of statement handles is not enabled, the DM always parses the statement before each execution. With caching enabled, BRM maintains its most recently used statement handles within the DM, freeing the RDBMS from spending its time on soft parsing.

How to Use the Statement-Handle Cache

The stmt_cache_entries entry in the Oracle DM configuration file (BRM_home/sys/dm_oracle/pin.conf) controls the statement-handle cache. The entry can be one of these two values:

• A value of 0 disables the cache.

• The default value of 1 means that the DM maintains 32 entries in each statement-handle cache for each back-end thread or process.

See the configuration file for more information.

The statement-handle caching performance feature requires a large value for the open_cursors setting in the initSID.ora database configuration file. See "Configuring Oracle Databases" in BRM Installation Guide.

Note:

If your Oracle database and DM both reside on computers with extraordinarily large memory resources, you might be able to cache more statement handles. Consult with Oracle for advice before attempting to cache more statement handles. It can be dangerous to exceed 32 entries because two Oracle database parameters must be increased along with stmt_cache_entries to prevent system failure.

Managing Database Usage

Performance of your BRM system is affected by the number of events in the database. You can limit which types of events are recorded in the database, which saves space and improves performance.

It is essential that all ratable events (events with a balance impact) be recorded in the database. You do this by including them in the pin_event_map file (BRM_home/sys/data/pricing/example) when you set up your price list. This is enforced by Pricing Center and the Price List Facilities Module (FM) opcodes.

However, many nonratable events may not need to be recorded. For example, event objects that are recorded during account registration, deal purchase, and product purchase require no further updating.

BRM provides a utility and file for excluding events from being recorded in the database. The load_pin_event_record_map utility loads the pin_event_record_map file (BRM_home/sys/data/config/pin_event_record_map), in which you specify the event types to exclude.

Note:

• By default, if an event type is not listed in the pin_event_record_map file, it is recorded.

• Event notification can still be performed based on excluded events because it is triggered even by events that are configured not to be recorded. See "Using Event Notification" in BRM Developer's Guide.

Ratable events that are mapped in the pin_event_map file should not be added to the pin_event_record_map file. If you specify a ratable event type, the event record map is ignored when the ratable event occurs.

Caution:

Excluding events from being recorded can cause applications that use the event data to return incorrect results. Ensure the events you exclude are not being used by any other application before you load the event record file.

To exclude events from being recorded:

1. Open the BRM_home/sys/data/config/pin_event_record_map file.

2. List the events you want to exclude and set their record flag value to 0. For example, to not record folds that have no balance impacts, enter

   /event/billing/cycle/fold:  0
   

   Note:

   The file includes the option to enable recording of listed events. You can use this option under special circumstances to record events that are normally not recorded.

3. Save and close the file.

4. Use the load_pin_event_record_map utility to load the file.

5. Verify that the file was loaded by using Object Browser or the robj command in the testnap utility to display the /config/event_record_map object. (See "Reading an Object and Writing Its Contents to a File" in BRM Developer's Guide.)

Rebuilding Indexes

Indexes can become large and unbalanced, which reduces performance. To increase performance, rebuild the most heavily used indexes regularly. You can quickly rebuild indexes at any time; for example, you might want to rebuild some indexes before running billing. See "Rebuilding Indexes" in BRM Installation Guide.

Removing Unused Indexes

By default, BRM installation creates indexes for all features. However, if you do not use some features, you can delete their associated indexes.

See your database documentation for information about finding unused indexes. For example, on an Oracle database, turn on the Oracle tracing facility while BRM is running. This produces an output trace file, which you use as input to the Oracle TKPROF utility. The TKPROF utility creates a file that lists the access paths for each SQL command. These access paths include indexes.

Tip:

You can also use the Oracle tracing facility to find missing indexes.

BRM Account and Rating Performance Considerations

Certain aspects of basic BRM functionality can affect performance; other aspects have no effect:

• The number of accounts does not affect performance.

• There is no performance difference when using different payment methods, such as invoice or credit card.

• BRM client applications, such as Customer Center, have little impact on system performance.

• Cycle events are rated faster than usage events.

• Performance decreases when accounts own a large number of products.

• Performance may decrease when the database contains a large number of ratable usage metrics (RUMs). In addition to computing the RUM for the specific event, BRM computes the RUMs for all base types of that event. For example, if you configured a RUM for the /a/b/c/d event, BRM also computes RUMs configured for the /a/b/c and /a/b events. You can increase performance by removing unused RUMs from your database.

Tuning Multithreaded Workloads

Usually, increasing the number of threads that an application or process can use increases performance. However, too many threads can result in too much context switching between threads, which can decrease performance.

To determine the optimum number of threads, increase the number of threads and watch the CPU utilization. If adding threads increases system time, adding threads is not helping performance.

CM and DM RAM and Swap Guidelines

Each CM system should have at least 128 MB of RAM. Too much more than 256 MB might not add additional performance unless there are a large number of connections to the CM.

DM RAM can be lower than 512 MB for smaller installations. For larger installations with eight or more DM CPUs on a single SMP box, 512 MB to 1 GB is recommended. Heavy usage of certain business policies and large searches can greatly increase CM memory requirements.

A typical requirement for swap on a CM/DM system is two to three times the amount of RAM, but this depends upon the operating system and the number of processes running on the CM/DM system.

Hardware Guidelines

• Use large disk controller RAM size.

• Maximize the processor cache size on the database and DM servers. For best performance, use caches that are at least 1 MB.

• Usually, performance is best when there are approximately one and a half times as many CPUs on the CM/DM systems as on the database server system.

• The total number of CM CPUs (regardless of the number of CMs) is approximately 25% - 33% of the total number of CPUs on the database server.

• CM and DM systems need less RAM than the database system.

• CM and DM systems have very low disk requirements. Disks are needed only for the operating system, BRM software, and swap space. Two 9-GB disks on each CM or DM system is usually enough, as long as these systems are not being used for non-BRM workloads.

  Two disks are recommended so that access to temp space, the operating system, swap, and BRM code are not bottlenecked on a single disk.

Improving Network Performance

Any kind of network connection that supports TCP/IP supports BRM (for example, local area network, virtual private network, and PPP).

The network bandwidth needs are relatively simple. BRM has an OLTP footprint (as opposed to Decision Support). Hence, transactions are normally small and network traffic will consist of smaller packets. The only requirement for network connectivity is TCP/IP. The real constraint is to have enough bandwidth between the DM systems and the database server; otherwise, the network might become a bottleneck even under moderate loads.

• Use 100BaseT or FDDI between each DM system and the database server. 10 Mbit Ethernets have too little bandwidth to handle heavy DM-to-database-server traffic. To minimize any collisions, the DM-to-database connections should use a separate physical connection (or a switch) and each of these should be a separate network.

• For switch-connected systems, connect all systems in the BRM configuration by using a single switch. For the largest configurations, use gigabit connections.

• When the test systems have multiple network cards, verify that the operating system network routing tables are configured to avoid bottlenecks. By default (depending on the system), output from the database server system may go through one LAN card, even though you have several configured, and even though input comes in through the multiple cards from different DMs. Examine and fix the routing tables as necessary. Ensure that each DM-to-database-server connection is explicit so all traffic between the two machines goes through the single dedicated physical path. The most common environment where you might find this problem is when there are multiple paths between the same sets of systems.

• For best performance, ensure all the systems in the testing environment are connected to the same hub with no intermediate hops.

Troubleshooting Poor Performance

When troubleshooting poor performance, first consider the following:

• Under-configured hardware.

• Inefficient table layout.

• Database bottlenecks.

• Inefficient custom application code.

• Repeated runtime errors resulting from configuration problems.

In addition, you can look for different problems depending on whether CPU utilization is high or low.

Low Performance with High CPU Utilization

If performance is low and CPU utilization is high, or if there are performance spikes, there is probably a configuration or indexing issue. Check the following:

• Hardware limitations.

• Table/volume layout.

• Spin count is too high.

• Lack of proper indexes. This can appear as very high CPU utilization with no other apparent problems except for a high number of processes. Find which columns are being accessed in the operation being performed and ensure that they are properly indexed.

• Not enough database buffers.

• Swapping.

• Kernel parameters too low.

Low Performance with Low CPU Utilization

If performance is low and CPU utilization is low, check for a bottleneck between different system tiers (for example, between the DM and the database).

• Use the database monitoring tools to analyze the performance of the database system.

• Use SQL tracing and timing to check for inefficient application code.

• Check for an under-configured BRM system, which could be one of the following:

  • CM Proxy with a low number of children.

  • RADIUS Manager with a low number of threads.

  • DMs with a low number of back ends.

  • System logging level is too high.

Monitor the DM system utilization and Oracle system utilization and tune the number of DM back ends accordingly. A good starting point for DM back-end numbers is eight times the number of processors.

For more information, see "Improving Data Manager and Queue Manager Performance".

Quick Troubleshooting Steps

• Run quick timing tests by using the testnap utility with op_timing turned on to ping each CM and DM (with the PCM_OP_TEST_LOOPBACK opcode). If the operations are relatively slow, it indicates a problem in the basic configuration.

• Run the system with a log level of DEBUG on the CM and DM and analyze log files.

• Check for network collisions and usage data.

• Check if you have logging (debugging) turned on in the CM. Logging is good for troubleshooting, but it should not be turned on in a production environment because it reduces performance.

• Performance parameters in pin.conf files should be large enough to handle the load. The most likely problems are in the DM entries.

• Check if you have enough DM back ends to handle your transaction load.

• Try putting tables and indexes on different disks.

• Check the size of redo and rollback logs and database configuration parameters.

• Send a few kill -USR1 commands to the DMs and CMs that seem to be having problems. This causes them to dump their state to the BRM error log files. Snapshots should be up to 20 minutes apart. These log files may contain information that indicates the nature of the problem.

• Turn on SQL tracing and analyze query plans. Look for full table scans. Ensure that indexes are on the appropriate columns for the query being run. Especially verify for any customizations.

• Turn on the timed_statistics parameter. Look for unusually long execution times for SQL commands.

• Monitor hardware activity:

  • On HP-UX IA64, AIX, Linux, and Oracle Solaris systems, use vmstat, netstat, and sar.

  • HP-UX IA64 has an additional tool called Glance.

  • Drill down to the storage device level by using sar with the -d parameter. This should help you find the source of the problem.

    Note:

    If the file systems are configured from logical volumes that are comprised of physical disks, different file systems could be sharing the same underlying disk. It is important to unravel who owns what to isolate potential contention (waiting on I/O).

• Problems such as intermittent daemon failures can be indicated by core files. Try the following command to locate them:

  % find BRM_home -name core -exec file {} \;
  

  If there are no core files, try turning on maximal debugging. You do not want to do this for very long, especially on a production system, because the log files fill up rapidly.

  % pin_ctl stop cm
  % setenv CMAP_DEBUG to 0x1331f3
  % setenv CM_DEBUG to 0x0001
  % setenv cm_loglevel to 3
  % pin_ctl start cm
  

  System level tracing can also be useful:

  # ps -ef | grep cm
  # truss -p cm_pid
  

About Benchmarking

To determine the best possible performance for your system, you must identify the desired transaction capacity at each tier and ensure that the system handles several times that capacity. The maximum capacity threshold or the transaction capacity threshold can be determined by running benchmark scenarios.

The primary goal is to achieve full utilization of the database system. This is best accomplished by measuring system performance as the following operations are carried out:

• Increase the load to get maximum throughput.

• Increase the number of DM back ends to get maximum RDBMS utilization for a given workload.

• Increase database utilization for a given number of DM back ends.

• Slowly reduce the load to keep the same performance but with faster response time.

• Multiple iterations of the above steps.

The general process for benchmarking is:

1. Create a workload on the system. The best choice to do this is by running your own program. The second best choice is to use the ITM-C workloads.

2. Measure the results. Most programs need a ramp-up period of 200 seconds to reach a steady-state condition during which actual measurements should take place. Running the program for 10 to 20 minutes should produce the same results as running the program for hours except for the amount of disk space used. If you run the system for a long time, indexes might become imbalanced and must be rebuilt, especially before billing.

   Use monitoring tools for the systems on which BRM is running to determine system load and identify performance issues. Be sure to turn monitoring tools off for your final test runs. When you start the system, turn on level 3 debugging in all BRM processes and ensure that there are no error messages while running the benchmark programs. When there are no more errors, turn off logging.

3. Monitor the hardware, operating system, and BRM utilization.

BRM Performance Diagnosis Checklist

When troubleshooting performance, use this checklist to help you look for problems. For more information, see the following:

• Monitoring and Maintaining Your BRM System

• Resolving Problems in Your BRM System

You can also use this checklist to gather information that Support needs when diagnosing trouble tickets. If you submit a performance issue to technical support, you should also include the following:

• All applicable error log files (for example, log files—or portions of log files—for the CM, DM, and client applications).

• Operating system settings such as maximum shared memory segment size and number of processes. Provide a full list:

  • Oracle Solaris: /etc/system

  • HP-UX IA64: /stand/system

  • Linux: /etc/sysctl.conf

  • AIX: Enter the following command to get a list of all parameters:

    lsattr -E -l sys0
    

• Administrative tools for managing systems:

  • HP-UX IA64: sam

  • Oracle Solaris: Admintool

  • Linux: Webmin and Easilix

  • AIX: smit

• The pin.conf files for CMs, DMs, and clients such as RADIUS Manager.

• For Oracle database, the init.ora file.

Describe the Problem

• What part of the system is experiencing the problem?

• What operation or application is running (for example, billing or credit card processing)?

• What is the actual and expected performance?

• What appears to be the problem?

• What are the error messages?

Describe the Configuration

Provide Support information about the following configurations:

• Hardware Configuration

• Operating System Configuration

• BRM Configuration

• Network Configuration

• Database Server Configuration

• Oracle Database Configuration

Hardware Configuration

For each system in the configuration:

• What is the manufacturer, model, number, and types of CPUs, and amount of RAM?

• What is the swap size?

For the database server system:

• What is the RAID level?

• How many disks, and what is their size?

• How are logical volumes configured?

Operating System Configuration

• What is the operating system version?

• What are the operating system settings for maximum shared memory segment size, number of processes, and so forth?

• Which patches have been applied?

BRM Configuration

• Which release of BRM are you using?

• Which systems do the following components run on? Which systems have multiple components, and which components run on multiple systems? How are the pin.conf files configured?

  • CMMP

  • CM Proxy

  • CM

  • DM

  • Optional managers such as RADIUS Manager

  • BRM client applications

  • Custom applications

  • Billing utilities

• Which BRM operations are slow?

• Which PCM_OPs are those slow operations associated with? (This can be found by using log level 3.)

• What is the estimated number of accounts in the database?

• What is the average number of products per account?

• What is the largest quantity of products owned by one account?

• What percentage of accounts use which payment method (for example, credit card or invoice)?

• What is the estimated number of events in the database?

Network Configuration

• How are the systems connected (for example, 10BaseT or 100BaseT)?

• Are separate networks used for each DM database connection?

Database Server Configuration

• What are the index and data file sizes?

• What are the database hot spots?

• What is the disk layout?

• What is the assignment of tablespaces to logical devices?

• Are disk volumes used?

• Are redo logs on their own disk?

Oracle Database Configuration

• What is the Oracle database version?

• How is the init.ora file configured?

  The following init.ora Oracle database parameters are particularly important.

  • db_block_buffers

  • shared_pool_size

  • use_aysnc_io

  • db_block_size

  • max_rollback_segments

  • processes

  • dml_locks

  • log_buffer

  Compare how your parameters are configured to those in the example BRM performance configurations.

  • Does the SGA roughly equal half the physical RAM?

  • What are the sizes and number of rollbacks?

  • Is check-pointing or archiving enabled?

  • Index and table fragmentation?

  • Number of extents, next extent size?

  • Run the query select index_name from user_indexes to view indexes. Check the indexes vs. columns in the WHERE clause.

  • Which optimizer option is being used (CHOOSE or RULE)?

Describe the Activity

• Are there any messages in any error logs (CM, DM, application)?

• Are there any operating system or database system error messages?

• Are there any bad blocks?

• Are you using any nonstandard resources, custom code (especially in the CM), or debugging aids such as writing log records to files that might result in contention or bottlenecks?

• Is there enough free swap space?

• What is the CPU utilization on servers used for BRM processes?

• Database system:

  • What are I/Os per disk per second, size of disk queues, disk service time, and percent of time waiting for I/O?

  • What is the CPU utilization on the database system?