Oracle Enterprise Manager Administrator's Guide Release 2.1 A75683-01 |
|
The Event system allows you to monitor your network for specific conditions, such as loss of service or lack of storage, that may occur in your managed environment. You select tests to run on managed targets (databases, nodes, listeners, or other services), then set the threshold parameters for which you want to be notified. You can share events with other administrators, in addition to being able to notify specific administrators when an event condition occurs. For some event tests, you can also choose to execute a fixit job that automatically corrects the problem.
The following topics are discussed in this chapter:
The Event system allows you to efficiently monitor a large system. Using the Event system and Intelligent Agents, you can effectively monitor any number of databases, nodes, or other services 24 hours a day, and be alerted when a problem or specific condition is detected. You can also pinpoint only the services you wish to monitor. The Event system can be extended to include other third-party applications that detect events independent of the Intelligent Agents.
In the Event system, event settings are stored based on the administrator registering the event. This allows administrators of large systems to customize their event systems to their preferences and tasks. Administrators receive messages for events for which they have been selected to receive notifications by other administrators.
The Event system includes the following processes:
You need to create and register events, which are simply a group of event tests that you want to run on your managed systems. Oracle Enterprise Manager includes a variety of predefined event tests that you can use when creating events. The event tests are grouped by service type, for instance:
You can create events using the predefined event tests that have been installed with Oracle Enterprise Manager. See "Event Categories and Types" on page 5-6 for more information.
The events are created with information entered in the Event property sheet. You determine parameters such as the destination that is monitored, the specific tests to perform, the frequency that the event test is executed, and whether other administrators can share the events. See "Permissions" on page 1-16 for more information. Some event tests have parameters with threshold values that you can customize for your system. See "Event Parameters Page" on page 5-29 for more information.
Events are registered, or submitted, to specific destinations, such as nodes, listeners, or databases. The status of a registered event is viewed in the Registered page of the Event pane.
The event scripts are executed on nodes with the permissions of the Intelligent Agent. However, some of the database event tests, such as Continued Rows, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Preferred Credentials" on page 1-18 for more information.
The Intelligent Agent is responsible for detecting when a specific event condition has occurred. The Intelligent Agent first notifies a Management Server, which in turn notifies interested administrators either through the Oracle Enterprise Manger Console, or by external means such as Email or Paging.
The Management Server is responsible for registering event information with the appropriate Intelligent Agents on nodes in the network. You determine the frequency that an Intelligent Agent checks an event. See "Frequency" on page 5-27 for details on setting the frequency interval for an event. An exception to this is the Up/Down (node) event test, which is checked at an interval set by the system itself. See "Fault Management Event Tests" on page 5-6 for more information on this event test.
When an alert condition occurs, the Intelligent Agent is responsible for notifying the Management Server. Each event is logged in the repository and can be viewed and acknowledged in the Alerts page of the Console. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.
Events can consist of multiple event tests. If any one of these tests identify a specified condition, the event is triggered and a notification is sent to the Console. If enhanced notification is configured for your system, paging and/or email notifications are sent.
Event notification occurs as follows:
Enterprise Manager administrators can be notified in various ways, such as electronic mail or paging, depending on the administrator's setups and permissions. You need to set up the notification services and determine the administrators that need to be notified for the events. See "Event Permissions Page" on page 5-30 and "Permissions" on page 1-16 to determine the administrators that receive notifications. See "Notification" on page 1-12 and "Schedule" on page 1-15 to determine how and when an administrator is notified.
If you plan to notify administrators with email or paging, you need to make sure the following are set up properly:
All events return values and some events produce output messages. The events return color values depending on the severity of the event. These severity levels are determined by parameters you set for the event tests during event creation. The colors are displayed on the event severity flag that is located:
The colors of the event severity flags are:
If an UpDown event test is specified for a particular event, and the node to which the event is assigned goes down, the event will return an "unknown" severity (gray flag). The original event severity will be restored when the node comes back up. At least one UpDown event test (node, listener, database, etc.) must be specified for an event in order for the "unknown" severity to be returned. Events not containing an UpDown event test will not provide any node down notification.
Some events, such as Probe and User Blocks events, do not return a warning value because the warning threshold parameter is not used. The event has either occurred or not occurred.
When an event occurs, you need to correct the problem. In some cases, you can create a fixit job that responds to specific event conditions. These situations are noted in the online help for Oracle events.
In other cases, the solution requires careful attention by a system administrator. For example, space management event conditions may require an administrator to increase space requirements and resource management conditions may require an administrator to adjust initialization parameters.
For information on how to correct event conditions, refer to the appropriate documentation. To correct Oracle database problems, refer to the Oracle Server Administrator's, Tuning, and Reference Guides. For network problems, refer to the Oracle networking guides for your system.
The Oracle event tests for the database, listener, and node destination types are grouped into categories:
Only the UpDown event tests are included with Oracle Enterprise Manager. Additional advanced events are available with the optional Oracle Diagnostics Pack.
See the online help for Oracle predefined event tests and "Oracle Event Tests" on page 5-34 for information on events and their parameters. All the Node events are supported on Unix and Windows NT platforms. For other platforms, see your platform-specific documentation.
This category of event tests monitors for catastrophic conditions on the system, such as a database, node, or listener is down. Immediate action must be taken by the administrator. Examples of event tests available in this category include:
Most of the fault management event tests do not require any threshold values because the event test only checks whether the service is up or down. For the Alert event test, the event test checks whether error messages are written into the database alert log file.
The UpDown event tests are provided with the Enterprise Manager base product. These event tests check whether a database, listener, or node is available. With the UpDown event test for databases or listeners, you can use the Startup Database or Startup Listener task as a fixit job to re-start the database or listener. To avoid executing that job when the database or listener is brought down intentionally, you need to remove the event registration.
This category of event tests track possible space problems, such as running out of space on a disk or archive device. Examples of space management event tests in this category include:
To check for space management events, set a threshold on the free space left. For example, set an alert if the free space on a disk falls below a specific number of bytes. In order to properly choose the threshold value, you need to know the characteristics of the tablespaces. For example, you would want to know whether the tablespaces contain online transaction processing (OLTP) tables or decision support tables. The former usually has a very fast growth rate, while the latter almost never grows.
This category of event tests track possible resource problems, such as exceeding datafile or lock limits. Examples of resource management event tests in this category include:
To check for resource management events, set a threshold on the percentage of a resource used. For example, you can set an alert if the percentage of the datafile resource used is greater than a specified value.
This category of event test monitors the system for performance problems, such as excessive disk input/output or library cache miss rate. Examples of events in this category include:
To check for performance management events, set a threshold on a system value. For example, you can set an alert if the library cache miss rate is greater than a specific value. The set of threshold values is system specific, depending on the hardware platform, number of users, and other factors.
Unsolicited event tests are event tests that have been initiated outside the Enterprise Manager Event system. An event is considered unsolicited if it is raised by a process other than the Oracle Intelligent Agent, but is running on the same node as the Intelligent Agent. These events are usually provided by third-party software. Creating an unsolicited event allows you to integrate and monitor third-party events. Essentially, there are two phases to setting up an unsolicited event:
To register interest in an unsolicited event, check the Unsolicited Event Test box in the Event General page and complete the information in the succeeding property sheet pages. Select the Unsolicited event in the Test page and complete the Parameters pages. Information on how to fill out the Parameters page is discussed in the next section. After completing the unsolicited event, you can save and submit the event. See "Creating, Modifying, and Registering an Event" on page 5-24 for more information.
Because unsolicited events originate outside the Event system, you may wish to screen only for specific external events. The Parameters page for the Unsolicited Event Test allows you to filter unsolicited events based upon the following field entries:
This is the four-part name of the event in the form:
/vendor/product/category/name
You can enter any character strings but all four parts and the forward slashes (/) are required. The eventname is assumed to be in 7-bit ASCII, so that it never changes regardless of platform or language. The name of the event that fires must match the value specified in this parameter field in order for the unsolicited event to fire.
Enter a "*" if you do not want to filter on an event name.
An object can be any service the unsolicited event is monitoring, such as a database, listener, or node name. Important: The object must be a service that the Intelligent Agent has already discovered.
Enter a wildcard character ( "*") if you want to receive unsolicted events from any type of object. Or, you can use the percent sign "%" for partial filtering. For example, ORCL% for events fired on objects whose name starts with "ORCL."
Enter the numeric severity level (-1, 1, 2) corresponding to the severity of the event you want ot filter through.
Severity levels map to different flags in the Console Event Pane:
Flag Color | Severity | Severity Level |
---|---|---|
Green |
Cleared |
Severity -1 |
Yellow |
Warning |
Severity 1 |
Red |
Critical or Alert |
Severity 2 |
Message is a quoted text string that is displayed in the Console, such as "File not found." If a message filter is specified, then the event is passed only if the message matches what is specified in the message filter.
You can specify wildcard characters as part of the filter i.e. Message Filter : ERROR%. In this case the unsolicited event should fire on anything with a message that starts with "ERROR:"
To raise unsolicited events, users have a choice of a command-line interface (oemevent executable) or an OraTcl verb (orareportevent). The related syntax is as follows:
oemevent [event_name] [object_name] [severity] [message] orareportevent [event_name] [object_name] [severity] [message]
where event_name is the name of the event that triggered the unsolicited event. object_name is the name of the object that the event is monitoring, severity is the level of severity for the event, and message is the text string displayed in the Enterprise Manager Console. For additional details, please refer to the Intelligent Agent User's Guide.
Note that the severity is specified as a string in oemevent and as an integer in orareportevent. Also, note that the event_name must be a four-part string of the form /a/b/c/d, where the different elements may be used to organize the event test within a hierarchy of event tests. For example, /myevents/node/files/filefound may be an event test you developed. It relates to nodes, more specifically space on nodes, and it monitors for the existence of a particular file.
For an example of using OraTcl to trigger a third-party event, see Tcl Script Examples in the Run Tcl job task. For information on OraTcl and event scripts, see the Oracle Intelligent Agent User's Guide.
Typically, unsolicited events are evaluated and raised by third-party software. Enterprise Manager allows you to implement unsolicted events through the Job system and Tcl. You create a Tcl job and submit it as a periodic job. The Tcl contains logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Since the job is submitted as a periodic job, the underlying test is evaluated periodically like all regular Enterprise Manager event tests. Using techniques like the ones scribed below, users of Enterprise can implement event monitoring specific to their environments.
Example 1: Simple Job Raising an Unsolicited EventIt is possible to submit a job with an imbedded OS command task that executes the oemevent program and passes the program all necessary arguments. All users that have registered for the event raised by oemevent, will receive the event notification. The event has to be known to the administrator submitting the job that raises it.
However, the job that raises the event may contain enough logic to evaluate the underlying test and decide whether it needs to raise the event and at what severity level. Such a job may be submitted as a periodic job so that the underlying test is evaluated periodically, which is similar to regular OEM event tests.
Unsolicited events (their logic actually) are evaluated in their own process and within the proper OS security envelope. For this reason, no longer pose security or robustness threats to the system. What we have introduced here is a procedure where one needs to submit a job in order to monitor for an event.
Consider as an example how to implement an event test which triggers when a particular file is found. Lets call this event /myevents/node/files/filefound. The following Tcl script needs to be submitted as the job:
# event name set event_name /myevents/node/files/filefound # filename to look for comes at the first (and only) argument set file_name [lindex $argv 0] # check for the file, and if it's found trigger the event as critical if { [file exists $file_name] } { orareportevent $event_name $oramsg(oraobject) 2 "$file_name found" }
In order to receive this event, a user needs to register an event with the Unsolicited test selected and configured to filter in events with name /myevents/node/files/filefound. This event should be registered against a node and will trigger against it. The message attached to the event occurrence will contain the values of all parameters passed into orareportevent.
Although this event is fairly straightforward, there are two things wrong::
Any other scripting language or executable program can also be used to implement the logic of an unsolicited event test. However, Tcl is preferrable because it allows platform-independent implementation and the fact that the code may be sent from the Enterprise Manager Console `on-demand' without requiring anything to be installed on the Intelligent Agent side.
Example 2: Unsolicited Events with the Proper LifecycleAs with regular Enterprise Manager events, unsolicited events could be triggered only once per condition detected and could clear automatically if the condition that triggers the event is no longer met. Events adhering to this operational pattern are said to have a "proper lifecycle."
Typically, scripts that implement unsolicited events are composed of two basic parts:
The following Tcl script illustrates this two-part script implementation, as well as a technique that allows proper event lifecycle.
#--------------------------------------------------------------------- # # Tcl Procedure # orareportevent1 # # Purpose: # Trigger an unsolicited event only previous state is different # # Arguments: # - event_name: event test to trigger # - severity: new severity # - message: message to be attached to the event report # #--------------------------------------------------------------------- proc orareportevent1 {event_name severity message} { # define a 'lock' that its contents define the previous event status # and figure out the event state during the previous execution global oramsg append event_lock [tempdir] "/" $oramsg(jobid) ".el" if { [file exists $event_lock] } { set f [open $event_lock r] gets $f previous_severity close $f } else { set previous_severity -1 } # if event test state has changed, trigger the event at new severity if { $previous_severity != $severity } { orareportevent $event_name $oramsg(oraobject) $severity $message if { $severity == -1 } { rmfile $event_lock } else { set f [open $event_lock w] puts $f $severity close $f } } } #--------------------------------------------------------------------- # # Event Test Name: # /myevents/node/files/filefound # # Purpose: # Monitor for the existence of a particular file # The test triggers at warning level if the file exists, but # at critical level if the file is larger than the specified # value # # Arguments: # - filename to look for # - critical file size # #--------------------------------------------------------------------- set event_name /myevents/node/files/filefound set file_name [lindex $argv 0] set critical_filesize [lindex $argv 1] if { [file exists $file_name] } { # if the file exists calculate its size in Kilobytes set file_size [expr [file size $file_name] / 1024] if { $file_size > $critical_filesize } { # if file is larger than the critical value, trigger as critical orareportevent1 $event_name 2 "Size: $file_size Kb" } else { # if file is smaller than the critical value, trigger as warning orareportevent1 $event_name 1 "Filesize: $file_size Kb" } } else { # if file in no longer there, clear the event orareportevent1 $event_name -1 "File does not exist" }Example 3: An Unsolicited Event Script that Accesses the Oracle Database
This section contains an example of unsolicited event tests. In this case, the test evaluation involves connecting to an Oracle instance and performing some SQL against it.
This example checks the size of a particular table in the database and triggers the event when a set threshold is crossed. There is a warning value and a critical value. The size of the table is measured by counting the number of its rows.
#--------------------------------------------------------------------- # # Tcl Procedure # orareportevent1 # # Purpose: # Trigger an unsolicited event only previous state is different # # Arguments: # - event_name: event test to trigger # - severity: new severity # - message: message to be attached to the event report # #--------------------------------------------------------------------- proc orareportevent1 {event_name severity message} { # define a 'lock' that its contents define the previous event status # and figure out the event state during the previous execution global oramsg append event_lock [tempdir] "/" $oramsg(jobid) ".el" if { [file exists $event_lock] } { set f [open $event_lock r] gets $f previous_severity close $f } else { set previous_severity -1 } # if event test state has changed, trigger the event at the new severity if { $previous_severity != $severity } { orareportevent $event_name $oramsg(oraobject) $severity $message if { $severity == -1 } { rmfile $event_lock } else { set f [open $event_lock w] puts $f $severity close $f } } } #--------------------------------------------------------------------- # # Event Test Name: # /myevents/database/space/tablesize # # Purpose: # Monitor the size of a particular database table # The test triggers at warning level when the warning threshold # is crossed and at critical level when the critical threshold # is crossed # # Arguments: # - table name # - critical threshold # - warning threshold # - username/password for conneting to target (optional) # #--------------------------------------------------------------------- set event_name /myevents/database/space/tablesize set table_name [lindex $argv 0] set critical_threshold [lindex $argv 1] set warning_threshold [lindex $argv 2] if { $argc == 4 } { set connect [format "%s@%s" [lindex $argv 3] $oramsg(oraobject)] } else { set connect [format "%s/%s@%s" $SMP_USER $SMP_PASSWORD $oramsg(oraobject)] } if {[catch {oralogon $connect} lda]} { append msg "Cannot connect to target." "\n" $oramsg(errortxt) orafail $msg } if {[catch {oraopen $lda} cur]} { append msg "Cannot connect to target." "\n" $oramsg(errortxt) oralogoff $lda orafail $msg } set sql [format "select count(*) from %s" $table_name] if {[catch {orasql $cur $sql}]} { append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt) oraclose $cur oralogoff $lda orafail $msg } if {[catch {orafetch $cur} row]} { append msg "Cannot execute SQL against the target." "\n" $oramsg(errortxt) oraclose $cur oralogoff $lda orafail $msg } set current_tablesize [lindex $row 0] if { $current_tablesize > $critical_threshold } { orareportevent1 $event_name 2 "Table:$table_name #rows:$current_tablesize" } elseif { $current_tablesize > $warning_threshold } { orareportevent1 $event_name 1 "Table:$table_name #rows:$current_tablesize" } else { orareportevent1 $event_name -1 "Table:$table_name #rows:$current_tablesize" }
A number of OraTcl verbs were used in this script. Refer to the Intelligent Agent User's Guide for details on OraTcl verbs. Note that the preferred credentials, specified in the console, are available to the script writer via the SMP_USER and SMP_PASSWORD Tcl global variables. For jobs against a database, the values of those variables are set to the username and password specified as preferred credentials for that database. Notice that this script allows for an optional overwrite of the preferred credentials via an optional forth input argument.
The Event pane contains the following pages:
You can switch between the pages by clicking the tab of each page. The rows in any page can be sorted on any column by clicking the column heading. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event pane.
The Event pane can be hidden or shown by selecting Show/Hide Event Display in the Console View menu. You can also hide or show the pane by clicking on the flag icon in the Console toolbar.
The Alerts page displays event tests that have been triggered.
Severity of the event occurrence: critical (red), warning (yellow), clear (green), or unknown (gray).
Name of the event.
Target where the event was triggered.
Time and date of the event occurrence.
Administrators assigned to work on the event occurrence.
Administrator who owns the event.
To view details of an event that has occurred, double-click on the event to display the Event Viewer property sheet. You can enter notes on the nature and progress of the event condition. Note: Comments entered into the log are viewable/editable by admins with the Modify permission. After you have reviewed an event, you can move it to the History page. See "Event Viewer" on page 5-22 for more information.
The Event History page displays a history of events that have occurred and have been moved to History by an administrator or cleared by an Intelligent Agent. The Event History page displays the same columns as the Alerts page.:
The History page is refreshed automatically each time you move between the History page and the Alerts or Registered page. However, to refresh the event history list while currently viewing the History pane, you must click the Refresh History icon located at the bottom left of the event History page.
To clear all entries in the History page, click the Clear History icon located next to the Refresh History icon.
The Registered page displays the events that have been registered, or submitted, to monitor test conditions on any network objects. The Registered page contains the following information:
Name of the event.
Target where the event was monitored.
Type of event destination: database, node, listener
Current registration status of the event: Registered, Registration Pending, De-Registration Pending, and Registration Failed.
Administrator who owns the event.
Note: Under certain circumstances, an event will remain in a Registration Pending state. There are two probable causes:
The Event menu allows you to set up event and administrator information. This menu also provides options to register, track, and view specific events. Menu options are enabled depending on the items selected in the Event pane. See Figure 5-1, "Event Menu and Pane" for an illustration of the Event menu.
When you register or remove an event, there is usually a slight delay while the Intelligent Agent processes the request.
Displays the Event property sheet and allows you to create the definition of a new event. See "Creating, Modifying, and Registering an Event" on page 5-24 for more information.
Displays the definition of an existing event. See "Creating, Modifying, and Registering an Event" on page 5-24 for more information.
Acknowledges the selected event in the Alerts page.
Copy the selected event in the Event pane to the Event Library.
Removes the selected event from the Event pane. This menu option only appears when an event test is selected in the Registered page.
Moves the selected event in the Alerts page to Event History page of the Event pane.
Clears the contents of the Event History page to a file.
Saves the contents of the Event History page to a text file.
Displays the Event Library dialog. See "Event Library Dialog" on page 5-21 for more information.
Registers, or submits, the selected event to a destination. This action monitors the destination, or network object, with the event tests in the event. This menu option is used to retry a registration that has failed.
If you select an item in the Event pane with the right mouse button, the context-sensitive menu for that item appears. This menu is a subset of the Event menu plus selection-specific menu options.
The Event Library dialog displays the events that have been created and saved to the Event Library. The advantage to using the Event Library is that both events and any associated target information can be stored, copied, or modified in the library for future use. When you create an event, you have the option of submitting, saving to the Event Library, or submitting and saving to the Event Library.
This dialog contains the following information:
Name of the event.
Description of the event.
Administrator who created the event.
Select an event and click Edit to display the property sheet for the event. The property sheet allows you to view and modify the events. Editing most event parameters can only be done via the Event Library dialog box.
Several predefined event tests have been installed with Oracle Enterprise Manager. These appear in the Tests page of the Event property sheet. You can add these tests to an event. The tests include:
Only the UpDown tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.
To view the specific tests assigned to an event, double-click on the event in the Event Library dialog and view the Test page of the Event property sheet. See the online help for Oracle events and "Oracle Event Tests" on page 5-34 for information on Oracle events and their parameters.
The Event Viewer property sheet displays details on a selected event in the History or Alerts page. To view details of an event, double-click on an event to display the property sheet. You can enter optional comments in the Log page. Once cleared, events are automatically moved to the History page. The pages of the Event Viewer include:
The Event Viewer General page displays statistics and owner information on a selected event. You can display the property sheet for the event with the Show Event Definition option. The following statistics are displayed on the General page:
Destination of the event.
Destination type, such as database, listener, or node.
Time of last update.
Administrator that created the event.
List of administrators that are receiving notifications for this event.
Event test that is performed.
Severity of the event occurrence: critical, warning, clear, or unknown.
Time and date of the event occurrence.
Message generated from the alert.
The Event Viewer Log page displays an entry whenever an event is moved to history. An event can be moved manually with the Move to History menu option or automatically when the severity of the event changes.
The Log page also allows comments to be entered on a selected event. Any administrator with permissions to modify the event can add comments in this page. You enter comments in the text box and select the Apply or OK button to add the comment.
The information displayed in the Log page includes:
Text input field allowing you to add comments.
Comment that has been entered for this event.
Administrator that entered the comment.
Time and date when the comment was entered.
The Event Viewer Notification Details displays details of email and paging notifications sent for a selected event. The information displayed in Details page includes:
Administrator that was notified.
Time and date of the notification.
Email address or pager number.
Status of the notification, indicating whether the notification was sent, is pending, or has failed.
Events include the destination type and the event information that you want to monitor. Events can consist of multiple event tests. To create or modify an event:
Modification of event parameters can only be carried out by editing the event from the Event Library dialog.
If you registered an event, the Intelligent Agent for a destination processes the event and the event appears in the Registered page of the Event pane. Each destination service is listed separately with the event.
There is usually a slight delay between the time the event is registered and the actual notification by the Intelligent Agent.
When threshold values are exceeded for the tests in an event, the event appears in the Alerts page of the Event pane. This notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:
Cases where an event notification is Unknown (gray flag) indicate the target or node where the event is registered is unavailable.
Do not register an UpDown event (included in the Oracle DB Fault event) against the database or node where the repository table is stored. The communication between the Intelligent Agent and Management Server can interfere with the repository database connection. If the repository database or node is down, the event cannot notify the Management Server.
The property sheet for creating a new event is the same as the property sheet for modifying an event, except that the event name field is read-only. See Figure 5-2, "Event General Page" for an illustration of the Event property sheet.
You can also use events that have been installed with Oracle Enterprise Manager. See "Event Categories and Types" on page 5-6 for more information.
On the General page, you determine the set name, destination type, description, and whether this event should monitor third-party events.
Enter an event name.
Select the destination type you want to monitor from the pull-down list. The types include Database, Listener, Node, or other service that is integrated into the Console.
Enter a description or comment for the event
A fixit job is designed to correct a problem. For example, you may want the Intelligent Agent to run a job to restart a database when the database instance has shut down unexpectedly. Fixit jobs have been created with the Job system and have been designated as fixit jobs. The jobs must be submitted and running on the same destination that the event is set on. You must select destinations to make fixit jobs available in the pull-down list. See "General Page" on page 4-13 for more information.
Fixit job options are:
To turn off a fixit job after an event has been registered, you must remove the event registration, create a new event, and register it.
Each event must use a unique fixit job on each destination where the event is registered. Also, when a single Intelligent Agent is monitoring multiple databases at a destination, create a separate event and fixit job for each database.
Determine the frequency that you want the test to run on the selected destinations. The frequency determines how often the event condition is checked. For example, if the frequency is set to 30 seconds, the event condition is checked every 30 seconds. To ensure timely notification, set the frequency to the smallest interval possible. To set the frequency:
See "Event Notifications" on page 5-4 for details on the notification frequency.
Check this box to monitor unsolicited events. See "Unsolicited Event Tests" on page 5-8 for more information.
On the Tests page, you determine the event tests that you want to perform. Event test are arranged hierarchically in a tree list for ease of viewing and selection. As with the Console Navigator, you can expand and compress entries in the tree list.
Select the event tests in the list you want to perform in this event, then click on the << (Add) button to move the events to the Selected Events list.
Select the event tests in the list you want to remove from this event, then click on the >> (Remove) button.
The fixit job, frequency, and parameter settings are entered in the Event Information section of the Parameters page of the Event property sheet. The settings and types of parameters vary according to the event test selected. Some event tests do not have parameters. See the online help for Oracle events and "Oracle Event Tests" on page 5-34 for information on tests and their parameters.
The parameters for an event are displayed when the event is selected in the Selected Tests list. The parameters vary according to the event selected. Some events do not have parameters.
You can accept the default values or change the values for the parameters. To enter parameter values for an event, you can:
Filtering is used in events such as Chunk Small and Maximum Extents. Examples of filters are = 'SYSTEM', LIKE '%SMP%', and IN ('SYSTEM', 'TOOLS'). Note that the quotes are single quotes. Use uppercase to match the case of the database object names. If you enter a filter value that does not select any objects or is an incorrect value, the event fails.
Determine the permissions that you want to assign to the event with the Permissions Page. This allows other administrators to view or modify the event. Notifications can be assigned with this page.
The levels of permission that you can assign to an Enterprise Manager administrator are:
Does not allow the administrator to view this event anywhere.
Allows the administrator to view the event, inspect event properties, and receive notifications.
Allows the administrator to modify the event's log (See "Event Viewer" on page 5-22), and edit the event's properties except those reserved for Full permission.
Allows the administrator to delete the event, modify permissions for other administrators, and change the ownership of the event.
Allows the administrator to enhance event notifications on the objects through paging or email. Other notifications will be routed to that particular administrator's Console. Notify permission cannot be assigned if the administrator's permission level is set to None.
Any permissions assigned on this page supersede any administrator default permissions. See "Permissions" on page 1-16 for more information.
When checked, permits external notification (SNMP traps) for third-party event tests at the destination where the Intelligent Agent is located.
Show Notification Schedule displays the notification schedule for the event. You can view the administrators that are notified by moving the cursor over any time block in the grid. To add or remove notifications for an administrator, display the context menu (press the right mouse button) on any time block. The context menu provides options for adding and removing recipients of the notifications.
The Event Progress page displays when you edit an event from the Registered page of the Events pane. This page provides the current registration status for the event selected: Registered, Registration Failed, or Registration Pending. In addition, the destination and time and date when registration was attempted is shown.
When the Progress page is displayed, it shows only the status for the selected event. If the selected event is registered, or had been submitted for registration on other destinations, you can view the status of this event for those destinations by selecting the desired target from the Destination pull-down list. The status of the event displays for that destination. To view the status of this event for all destinations simultaneously, select <All>.
The following options are available on the Progress page:
Select the destination of the event you want to view from the pull-down list. Select <All> for all destinations for which this event has either been registered or failed to be registered.
Status for the event: Registered, Registration Pending, or Registration Failed.
Network destination for the event.
Time and date the event was submitted for registration.
Displays the Event Status Message dialog. This button is active only when event registration has failed, thus generating an error message.
Saves the contents of the list to a text file.
In the following example, a new event for monitoring extents in a database is created and registered at several databases. When creating an event, the General, Tests, Parameters, and Permissions Pages are completed to define the event. You need to register, or submit, an event to run event tests on specific destinations in the network environment.
Note: The Maximum Extents event test is only available with the optional Diagnostics Pack. The standard Up/Down event tests provided with Enterprise Manager do not require parameter settings. |
When an event is submitted, each destination is validated and the Intelligent Agent for a destination processes the event. If the registration is successful, the event appears in the Registered page of the Event pane. Each destination database is listed separately with the event.
If an event condition occurs or threshold values are exceeded for an event test, a notification is sent to the Alerts page of the Console Event pane. If administrators have been selected to be notified by email or paging, an email or page is sent. The event notification changes the color of the severity flag for the event in the Alerts page. If the destination database icon is displayed in the Group pane, the flag on the icon changes color. The colors and their meaning are:
After an event condition is fixed, the event is cleared automatically. You can also clear an event by acknowledging and moving the event to the Event History page.
Oracle Enterprise Manager allows you to specify administrators that are notified when a particular event condition occurs. Each administrator can be associated with an email ID and/or a pager number. When using a paging service or email notification, each administrator can be assigned responsibility for specific systems at specific days and times.
For more information on setting up Oracle Enterprise Manager administrators, see "Managing Enterprise Manager Administrators" on page 1-7. For information on configuring and email and paging services, see the Oracle Enterprise Manager Configuration Guide.
This section lists the Event system event tests with their parameters and return values. See "Event Parameters Page" on page 5-29 for information on entering parameter values. A list of event tests with numeric pager event Ids is also provided. See "Numeric Pager Job/Event Ids" on page 5-36 for more information.
Event tests are specified for database, listener, and node services. The event tests are also divided into fault, space, resource, and performance management categories. Only the UpDown event tests are included with Oracle Enterprise Manager. Additional advanced event tests are available with the optional Oracle Diagnostics Pack.
The event test scripts are written in the Tool Command Language (Tcl) enhanced with Oracle Tcl commands (OraTcl). For information on using Tcl and OraTcl to write event scripts, see Oracle Enterprise Manager Application Developer's Guide.
Some of the event tests require special tables in the database. For example, the catblock.sql script needs to be run to use the User Blocks event test. See the Oracle Enterprise Manager Configuration Guide for scripts that need to be executed against the database that will be monitored.
Some of the database event tests, such as Chain Row, require access to system tables and require additional permissions. You need to set up preferred credentials for the monitored database with an administrator that has system privileges. See "Preferred Credentials" on page 1-18 for more information.
This category of event test tracks severe problems that require immediate attention.
This event test checks whether the database being monitored is running. If this event test is triggered, other database events are not ignored.
none
The Startup Database job task can be set up as a fixit job for automatically correcting the problem.
If the listener serving a database is down, this event test may be triggered because the Intelligent Agent uses the listener to communicate with the database.
This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the Intelligent Agent on a node can be accessed from the Console via periodic "heartbeating" between the Management Server and the Intelligent Agent. Since this event does not actually register with the Intelligent Agent, no fixit jobs can be associated with it. If the defaults are accepted, this event will trigger within five minutes of the Intelligent Agent becoming inaccessible and will clear within three minutes of the Intelligent Agent becoming accessible again.
none
This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the listener on the node being monitored is available.
none
The Startup Listener job task can be set up as a fixit job for automatically correcting the problem.
The Oracle Data Gatherer Alert and UpDown events are both Fault Management events for the node destination type. This category of events monitors for catastrophic conditions on the system. Immediate action needs to be taken by the administrator.
This event checks whether the Intelligent Agent data gathering service on a node can be accessed from the Console. If the Intelligent Agent data gathering service is down, this event is triggered.
none
The Event Management System provides paging services that notify an administrator with a page when an event has occurred. Alphanumeric pagers provide a brief text message identifying the event. Numeric pagers provide the numeric pager event Ids to identify the event.
For job notifications, you will receive a 6 digit number. The first 3 digits indicate the job-id. The last 3 digits indicate job status.
For event notifications you will receive the event ID with the status code.
The event status and page status codes are listed as follows:
100 = Job Started
200 = Job Completed
300 = Job Failed
400 = Job Deleted
500 = Event Cleared
600 = Event Warning
700 = Event Alert
800 = Event Node Down
900 = Event Unknown
1000 = Event Assignee Changed
If JJJ= job id then,
JJJ100 => job started for job# JJJ JJJ200 => job completed for job# JJJ JJJ300 => job failed for job# JJJ
For example, if 001100 is displayed on your numeric pager, then you know that job id 001 just started. If 001300 appears, then you know job id 001 just failed.
If EE= event ID then,
EE500 => event cleared for event # EE
For example, if 7500 is displayed, you know that event number 7 had a status change and the status is now cleared.
The following procedures will assist you and/or Oracle Support if problems are encountered with the Event subsystem.
If you need to contact Oracle Support, you must provide specific information on the status of your Oracle Enterprise Manager repository. To assist you, a SQL script has been provided that extracts pertinent Repository data and generates a debug log file.
To generate the log file, perform the following procedure:
$ORACLE_HOME/sysman/log.
connect <repository name>/<repository password>
@$ORACLE_HOME/sysman/admin/evtdbg.sql
If you submit jobs or events to a node running an Intelligent Agent, and the Intelligent Agent unexpectedly goes out of service and is not restarted, you will be unable to delete active jobs and events from the Oracle Enterprise Manager repository: Jobs and events will remain in a "pending deletion" state. Consequently, the node itself cannot be removed.
To remove these "stale" entries from the Oracle Enterprise Manager repository:
$ oemctrl stop oms <username>/<password>
$ sqlplus <user_name>/<user_pwd>@<repository_database>
You may alternatively use Server Manager.
execute smp_maintenance.delete_operations_on_node('<target nodename>');
The node name should be identical to what appears in the Console Navigator (it may include the domain name as well).
$ oemctrl start oms
This will remove all currently active jobs and events against the specified node. No historical data will be affected. The node itself will not be deleted. To remove the node, log on to the Management Server through the Console and remove the node from the Navigator.
|
![]() Copyright © 2000 Oracle Corporation. All Rights Reserved. |
|