Skip Headers
Oracle® Communications Order and Service Management System Administrator's Guide
Release 7.2.2

E35414-02
Go to Documentation Home
Home
Go to Table of Contents
Contents
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

10 Monitoring and Managing OSM

This chapter describes how to monitor and manage the Oracle Communications Order and Service Management (OSM) system using the Oracle WebLogic Server Console.

About Monitoring and Managing OSM

Oracle WebLogic Server requires a set of interrelated resources, such as database connections, Java Messaging Service (JMS) queues, execution threads, transactions, and system memory to work together in order to provide the functionality required of OSM.

You use the WebLogic Server Console to manage these resources, including tasks such as starting and stopping servers, balancing the load on servers or connection pools, selecting and monitoring the configuration of resources, detecting and correcting problems, monitoring and evaluating system performance, and making sure that OSM is correctly deployed to the target servers.

The WebLogic Server Console is a Web-based application that allows system administrators, support staff, and others to monitor and manage the OSM application remotely.

See the Oracle WebLogic documentation for more information.

Accessing the WebLogic Server Administration Console

To access the Oracle WebLogic Server Administration console do one of the following:

When started, the WebLogic Server Console prompts for a password. This should be the password for a user that is a member of the Administrators group in WebLogic. One such user is the WebLogic administration user that was created when the domain was created. By default, the name of this user is weblogic.

After you have successfully logged in, the WebLogic console Home window is displayed.

Using the WebLogic Console to Determine the Status of the OSM Application

After you have logged into the WebLogic console, you can access information about the status of the WebLogic servers and OSM deployments.

To access the status of the OSM server and deployments:

  1. If you are not in the Home window of the WebLogic Console, click the Home icon in the upper left part of the window.

  2. In the Environment subsection of the Domain Configurations section, click Servers.

    The Summary of Servers window is displayed. Server statuses are contained in the Health column of the table.

  3. Click the Home icon in the upper left part of the window.

  4. In the Your Deployed Resources subsection of the Domain Configurations section, click Deployments.

    The Summary of Deployments window is displayed. Deployment statuses are contained in the Health column of the table.

    Note:

    If any of the deployments are not in the status that you expected, you can use the buttons on this window to start and stop individual deployments if necessary.

Managing Log Files

The following section details how to manage log files using the WebLogic Server Console.

Reading Log Files

You can view the OSM log files by opening the WebLogic Server Console.

To view the logs:

  1. In the left pane of the console, click Diagnostics/Log Files.

  2. From the Log Files table in the right pane, choose ServerLog, and then click View.

    The resulting page displays the latest entries in the log file. To view a specific, click to select the row, and then click View.

Configuring the Log View

OSM logs everything that happens in the system, which can make the logs file quite large. Using the WebLogic Server Console, you can configure the log views to show only the most recent messages.

To configure the view of the logs:

  1. Log in to WebLogic Server Console.

  2. From the log files page, click Customize this table.

  3. Choose the appropriate filter and view options, then click Apply. To add an option to the Chosen column, highlight it in the Available column, then click the arrow that points to the Chosen column. To deselect the option, reverse this action.

Log Size and Rotation

It is important to be able to manage your log file maximum size and rotations to prevent the logs from filling up your disk resources.

To manage your log settings:

  1. Log in to WebLogic Server Console.

  2. Click Environment/Servers/server.

    This displays the General Configuration page for the selected server.

  3. Click the Logging tab.

    The resulting page displays the logging settings for the selected server.

  4. Modify the Rotation file size to 25,000 kilobytes and be sure to limit the number of Files to retain. Modify these settings for both tabs, General and HTTP.

Managing Error Message Volume and Logging Levels

OSM uses Log4j to generate and manage the system log messages. Through the use of Log4j, you can develop and maintain a logging strategy that minimizes the overall impact of logging operations on the application's resources. It does this by letting you control the volume of log messages generated.

You also have the ability, when necessary, to dynamically change, temporarily, the level and detail of the logged messages. This feature helps you to, for example, increase the level and detail of logged messages to help analyze performance problems within a production environment.

To read more about Log4j, refer to the Apache web site a:

http://logging.apache.org/log4j

Severity Levels

To control the volume of log messages generated and written to the output destinations (the console and the WebLogic log file, which are referred to as Appenders by Log4j), you assign severity levels to the various areas of the application that generate their own discrete messages (these areas are referred to as Categories by Log4.

The following is an ascending list of the severity levels, starting with the least severe:

  • Debug: (least severe) designates fine-grained informational events that are most useful to debug an application

  • Info: designates informational messages that highlight the progress of the application at coarse-grained level

  • Warning: designates potentially harmful situations

  • Error: designates error events that might still allow the application to continue running

  • Fatal: (most severe) designates very severe error events that presumably leads the application to abort

If an event occurs inside a given category that triggers a message below the severity level assigned to the category (that is, less severe than the assigned level), Log4j does not generate the message.

Example 1

If you assigned the severity level Warning to a given category, Log4j does not generate any messages for that category that are flagged in the OSM code as Debug or Info level messages. Log4j will, however, generate all messages that are flagged as Warning, Error, and Fatal.

You can further control the number of messages written to the output destinations, or appenders, by also assigning a severity level to them. When you assign a severity level to an appender, it rejects messages below that severity level, even if Log4j passes the message to it.

Example 2

If you configure the console to the Warning severity level, and one of the categories is configured with the Info level, the console will not display the Info message, even though Log4j generates the message and passes it on to the console, because the message's severity level falls below the threshold for which the appender is configured. If, however, that same category later generates an Error level message, the console accepts and displays the message, because it carries a severity level equal to or higher than the console's threshold.

By default, the console and the WebLogic log file accepts all error messages, from the least severe to the most severe.

Displaying Log Levels

To display log levels:

  1. Log in to WebLogic Server Console.

  2. In the left pane of the Console, expand Diagnostics and select Log Files.

  3. In the Log Files table, select the radio button next to the name of the log you want to view, and then click View.

    The page displays the latest contents of the log file; up to 500 messages in reverse chronological order. The messages at the top of the window are the most recent messages that the server has generated.

    The log viewer does not display messages that have been rotated into archive log files.

    The page displays the log file entry.

Configuring the Severity Levels

There are two methods by which you can configure the severity levels:

  • log4j.xml: The log4j.xml file resides within the omslogging.jar of the OMS.ear file. Using this method you can change the default severity settings. This requires you to stop, then restart the servers to take effect. See "Configuring the log4j.xml File", below.

  • log4jAdmin: The log4jAdmin web page lets you change the severity levels while the servers are running. The changes take effect immediately (this method is temporary; the system reverts to the default levels you established in the log4j.xml file when you restart the server). See "Configuring Log Levels Temporarily".

    Note:

    To use this method, your Web Client log-in ID must have OMS_log_manager permissions.

Configuring the log4j.xml File

Use this method to set up the default severity levels. To change these levels later, you must stop the server, modify the log4j.xml file, then restart the server.

If you need to modify the logging levels, you should use the log4jAdmin web page, as described in "Configuring Log Levels Temporarily".

There are two sections of the log4j.xml file that you should look at when configuring this file:

  • Appenders: This section defines the output destination for the messages. At installation, the Appenders section contains two entries, as follows:

    • Console: From this entry you control the level of messages that the WebLogic console accepts.

    • WebLogic: From this entry you control the level of messages that the WebLogic log file accepts.

  • Categories: This section contains references to all of the OSM categories that generate messages and gives you the ability to control the level of messages they generate.

To configure the log4j.xml file:

  1. Unpack the oms.ear file. Copy oms.ear from domain_home/bin on the server to OSM_home\SDK\Customization.

  2. Configure the unpackOMS.bat file, also located in OSM_home\SDK\Customization, with the correct location of JAVA_HOME for your environment.

  3. Run the unpackOMS.bat file. This creates the osm-ejb\logging folder, where you will find the log4j.xml file

  4. Use a text editor to open log4j.xml.

  5. Go to the Console entry in the Appenders section. To do this, search for the following string, which is at the top of the Appenders section:

    <!-- Append messages to the console -->
    

    The Console entry governs what level of messages are written to the console.

  6. If necessary, change the threshold level. By default, it is configured to DEBUG, allowing the console to display all messages sent to it. If you want to restrict the number of messages displayed, change the threshold entry to the severity level appropriate for your installation (see "Severity Levels" above, for a description of the severity levels).

    In the example below, the level is changed from DEBUG to INFO.

    Before

    <param name="Threshold" value="DEBUG"/>
    

    After

    <param name="Threshold" value="INFO"/>
    
  7. Go to the WebLogic's Log File entry in the Appenders section. To do this, search for the following:

    <!-- Append messages to the weblogic's log file-->
    

    The weblogic log file entry governs what level of messages are written to the WebLogic log file.

  8. If necessary, change the threshold level as described in step 6.

  9. Go to the Categories section. To do this, go to the top of the file and search for the string:

    category name
    

    This takes you to the first category entry in the file.

  10. Review each of the categories in this section, changing the severity level where necessary.

    In the example below, the level is changed from INFO to WARNING.

    Before

    <category name="org.jboss.system">
    <priority value="info"/>
    </category>
    

    After

    <category name="org.jboss.system">
    <priority value="warning"/>
    </category>
    
  11. When you finish updating the categories, save and close the log4j.xml file.

  12. Repack the oms.ear file. At the command prompt, run packOMS.bat.

  13. Re-deploy the oms.ear file.

Configuring Log Levels Temporarily

Use log4jAdmin web page to check the current logging levels or to change the logging levels dynamically.

Note:

You can use this method only to change the severity levels of the Categories. To change the Appender level (the logfile output, for example, console or file), you must reconfigure the log4j.xml file (see "Configuring the log4j.xml File" for an explanation of the Categories, the Appenders, and how to configure the log4j.xml file).

The changes you make to the logging severity levels using this method are temporary; they are not written to the log4j.xml file. When you restart the server, the logging levels return to those that are configured in the log4j.xml file.

Note:

To perform the following procedures, your log-in ID must belong to the OMS_log_manager group. This set of permissions can be granted through the WebLogic Server console.

Checking the current logging levels

You can use the Filter Loggers feature to check the logging level of specific categories or sub components. If you know the name of the category or sub-component that you want to check, you can use the filter to display only that category, or related, categories.

To check the current logging level:

  1. To open the log4jAdmin web page, enter the following path in the browser's address line (Note: the URL is case sensitive):

    http://localhost:7001/OrderManagement/control/log4jAdmin
    
  2. Enter the beginning of the name, or a part of the name, in the Filter Loggers field

  3. Click either:

    • Begins With (if filtering on the beginning of the name)

    • Contains (if filtering on just a part-name)

  4. The list displays the categories and subcategories that match the entry in the Filter Loggers field.

    Note:

    To change the actual logging level from the list that you get from the Filter Loggers, use the following procedure.

To change the logging levels dynamically:

  1. Open the log4jAdmin web page, enter the following path in the browser's address line (Note: the URL is case sensitive):

    http://<host:port>/OrderManagement/admin/log4jAdmin
    
  2. Scan down the entries in the left-hand column of the page. This column contains a list of the categories and their related sub-components. Find the category or sub-component for which you want to change the logging level.

    Example

    com.mslv.oms.automation (category name)

    or

    com.mslv.oms.automation.AutomationDispatcher (sub-component name)

    Note:

    If you know the name of the category or sub-component that you want to change, you can use the filter at the top of the page to display just that, or related, categories. Enter the beginning of the name, or a part of the name, in the Filter Loggers: field, then click Begins With (if filtering on the beginning of the name) or Contains (if just filtering on a part-name).

    To show specific category and sub-component names, at least one corresponding order should have been previously submitted to OSM. For example, to show sub-components beginning with "oracle.communications.ordermanagement.requestprocessor", at least one instance of the corresponding SalesOrder10000DeliverEBM.xml sample order should have been previously submitted.

  3. Scan across the row to the severity levels. The level that currently is selected is highlighted in a different color from the other levels and appears in the Effective Level column.

  4. Click the level to which you want to make the change. To:

    • Change an entire Category: Click the category name

    • Change just the sub-component: Click the sub-component name

    The change takes place immediately.

    Note:

    In order for the severity level you select in the log4jAdmin web page to work, the "Minimum severity to log" value in WebLogic Server must be set to the same or a lower severity level.
  5. Repeat step 4 for as many categories or sub-components that you need to change.

  6. When you finish making the changes, close the page.

Monitoring Performance

The WebLogic Server Console provides a real-time view of system performance.

You can access the performance monitor by using the following procedure.

To access the performance monitor:

  1. Start the WebLogic Server Console.

  2. Click Environment/Servers/server. This displays the General Configuration page for the selected server.

  3. Click Monitoring.

  4. Click the Health tab to view the health status for all OSM related sub-systems. If the status is not OK, review the reason and, if required, access the server log for more information.

    Health status severity levels are shown in the bottom left pane of the console under System Status. (OK, Warn, Overloaded, Critical, Failed)

  5. Click the Performance tab to view JVM memory utilization statistics for the server.

    If the memory usage statistics are high, you must allocate more memory to the Java runtime by increasing the -Xms and -Xmx parameter values. The file that contains this parameter depends on the operating system you are using and the WebLogic Server to which the OSM server is deployed. An example is shown below.

    Example:

    Open the file ${DOMAIN_HOME}/bin/setDomainEnv.sh and provide the following parameter values:

    MEM_ARGS="-Xms2048m -Xmx2048m"
    

    The more memory that you allow for the Java runtime, the faster OSM will run under high loads.

  6. Click the Threads tab to monitor thread activity for the server. Important columns to monitor are Queue Length and Pending User Request Count. A count of zero is optimal, meaning no user requests are stuck or waiting to be processed.

    If any of the counts are unusually high in the pool, go to the second table to troubleshoot the individual threads.

  7. Click the Workload tab to monitor the Work Managers configured for the server. If the Pending Requests count is not zero, you should access the server log file for more information or perform a thread dump.

  8. Click the JDBC tab to monitor the database pool connections configured for the server. The installer creates a maximum capacity of 15 connections for the connection pool.

    If the Active Connections High Count is 15 and the Active Connections Average Count is half of that, you may need to increase the number of connections. See "Managing Database Connections" for more information.

  9. Click the JTA tab to monitor transaction activity on the server. There should be no Roll Back statistics in the summary; if so, refer to the server log file for more information.

Managing Database Connections

In OSM, database connections are managed through database pools that are set up in WebLogic.

The database pool connections can be configured through the WebLogic Server Console by clicking Services/JDBC/Data Sources/oms_pool/Connection Pool.

Use the parameters on the Connection Pool page to modify and fine tune your connection pool settings as the need arises.

Using JMS Queues to Send Messages

OSM uses JMS Queues and Topics which are both JMS Destinations. Queues follow a point-to-point communication paradigm, while Topics follow the publish and subscribe paradigm.

Note:

In an OSM clustered environment, you must use JMS queues as a JMS destination to receive JMS events. Do not use JMS topics in an OSM clustered environment.

When OSM sends data to an external system, such as UIM or ASAP, it does so by sending JMS messages to the appropriate JMS request queue of an external system.

If the external system is not processing the requests from OSM, the queues get backlogged. It is important to be able to monitor the size of the JMS queues in order to know whether or not they are backing up.

To monitor the JMS queues:

  1. Login to WebLogic Administration Console

    Click Services/Messaging/JMS Servers/oms_jms_server.

    The General Configuration page is displayed.

  2. Click the Monitoring tab then click Active Destinations.

    A list of active destinations targeted to the server is displayed.

    Note:

    The default view of the table does not contain the Consumers column. We recommend that you customize the table using Customize link to include this column, along with any other customizations you may want to make.

The Consumers column defines the current number of listeners on the destination. If a destination does not have any listeners, then the external system will not receive the messages.

The Messages Current column defines the current number of unprocessed messages in the JMS destination. A large number (for example, 10,000) in this destination is a problem. It means that the messages are not getting processed, or that the messages are getting processed but errors are occurring and the messages are getting put back on the destination.

When OSM is first installed, the following JMS destinations are present:

Monitoring the Event Queue

The destination oms_events is the JMS destination to which OSM events are sent. OSM events are sent when tasks change states, or when notifications occur.

The number of consumers for the oms_events is determined by which plug-ins are configured. If plug-ins are configured, the number of consumers must not be 0.

If there is a problem with automation plug-ins getting invoked, check the consumers queue and the messages queue.

If the consumers queue is less than the number of plug-ins, the plug-ins are not configured correctly. Check the OSM_home\SDK\Samples\DatabasePlugin\map\ automationmap.xml file and make sure that all of the plug-ins have been deployed.

If the messages queue keeps getting larger, the plug-ins may not be committing the transactions during processing of the events. Verify the plug-in code and check the log files.

Sending Data to External Systems Using Plug-Ins

If there are external systems deployed to the same WebLogic instance as OSM, when you monitor the JMS destinations, watch for the following.

Note:

The important columns are Consumers, Messages, and Messages Received.

If the number in the messages column for these queues continues to grow, the external system may not be processing the messages sent by OSM. You must check to see if the external system is working properly.

If the number of consumers for the queues is 0, such as UIMrequestQueue, the external system may not have configured its listeners properly. Check to see if the external system is configured properly.

Updating the JMS Redelivery Configuration Settings

When the Order-to-Activate cartridges are installed, the Redelivery Delay Override and Redelivery Limit WebLogic parameters are set during installation to 7000 msec and 10, respectively. However, different values may be more effective for your OSM environment depending on your usage of the system.

If you encounter timing related issues for message delivery on JMS queues, there are a number of WebLogic settings that you can modify to resolve the issue. These values are set on every JMS queue through the WebLogic Service Console. From Home, select JMS Modules, and then select oms_jms_module to modify the following settings:

  • Redelivery Delay Override: Delay in milliseconds before rolled back or recovered messages are redelivered. This value overrides the Redelivery Delay setting.

  • Redelivery Limit: The number of times to attempt to redeliver a message.

  • Time-to-Deliver: Delay in milliseconds before a sent message is visible at the target destination. Typical values for this setting are 100 through 700.

To find the best values for these parameters, start with initial values less than 7000 msec for the Redelivery Delay Override, 10 for the Redelivery Limit and 100 for the Time-to-Deliver parameter and increase them slightly until no occurrences of errors are observed. The actual values you finalize on will depend on your particular implementation of OSM. See the Oracle WebLogic documentation for complete details on these parameters.

About OSM and XA Support

The OSM database does not support XA transactions because the Oracle thin-client driver used for JDBC connections does not support XA. However, the OSM WebLogic Server configuration uses an XA emulation feature in order to get a two-phase commit across JMS/JDBC automation transactions.

Even though OSM uses a non-XA driver for database transactions, external XA resources can still participate in transactions. For example, JMS bridges can be XA-enabled for an outside application, but the OSM side of the transaction will still use the non-XA emulated two-phase commit. Note that this also applies to JMS queues which support Application Integration Architecture (AIA) cartridges.

Overriding the Internet Explorer Language in the OMS Web Clients

If the Internet Explorer installation that is used to access the OMS web clients is set to a language other than English, and this language matches one of the properties files included in the oms.ear file, the Web client prompts appear in the non-English language.

The language used in the Web clients is controlled by the resources.properties file. Additional language property files in the oms.ear file include:

To remove support for a non-English language, unpack the oms.ear file, remove the corresponding properties file, repack the oms.ear file and redeploy it.

For example, if the browser is set to use the Czech language, and the resources_cs.properties file exists in the oms.ear file, the Web client prompts appear in Czech. Removing the resources_cs.properties file causes the Web client prompts to appear in English, even though the browser language setting is still configured to the Czech language.