This chapter covers the following topics:
Blackouts and Notification Blackouts (notification blackouts) help you maintain monitoring accuracy during target maintenance windows by providing you with the ability to suspend various Enterprise Manager monitoring functions for the duration of the maintenance period. For example, when bringing down targets for upgrade or patching, you may not want that downtime included as part of the collected metric data or have it affect a Service Level Agreement (SLA).
Blackout/Notification Blackout functionality is available from both the Enterprise Manager console as well as via the Enterprise Manager command-line interface (EMCLI).
Blackouts allow you to suspend monitoring on one or more targets in order to perform maintenance operations. Blackouts, also known as patching blackouts, ensure that the target is not changed during the period of the blackout so that a maintenance operation on the actual target will not be affected. During this period, the Agent does not perform metric data collection on the target and no notifications will be raised for the target. Blackouts will allow Enterprise Manager jobs to run on the target during the blackout period by default. Optionally, job runs can be prevented during the blackout period.
A blackout can be defined for individual target(s), a group of multiple targets that reside on different hosts, or for all targets on a host. The blackout can be scheduled to run immediately or in the future, and to run indefinitely or stop after a specific duration. Blackouts can be created on an as-needed basis, or scheduled to run at regular intervals. If, during the maintenance period, the administrator discovers that he needs more (or less) time to complete his maintenance tasks, he can easily extend (or stop) the blackout that is currently in effect. Blackout functionality is available from both the Enterprise Manager console as well as via the Enterprise Manager command-line interface (EMCLI). EMCLI is often useful for administrators who would like to incorporate the blacking out of a target within their maintenance scripts.
Why use blackouts?
Blackouts allow you to collect accurate monitoring data. For example, you can stop data collections during periods where a managed target is undergoing routine maintenance, such as a database backup or hardware upgrade. If you continue monitoring during these periods, the collected data will show trends and other monitoring information that are not the result of normal day-to-day operations. To get a more accurate, long-term picture of a target's performance, you can use blackouts to exclude these special-case situations from data analysis.
Blackout Access
Enterprise Manager administrators that have at least Blackout Target privileges on all Selected Targets in a blackout will be able to create, edit, stop, or delete the blackout.
In case an administrator has at least Blackout Target privileges on all Selected Targets (targets directly added to the blackout), but does not have Blackout Target privileges on some or all of the Dependent Targets, then that administrator will be able to edit, stop, or delete the blackout. For more information on Blackout access, see "About Blackouts Best Effort".
Notification Blackouts are solely for suppressing the notifications on targets during the Notification Blackout duration. The Agent continues to monitor the target under Notification Blackout and the OMS will show the actual target status along with an indication that the target is currently under Notification Blackout. Events will be generated as usual during a Notification Blackout. Only the event notifications are suppressed.
The period of time under which the target is in Notification Blackout is not used to calculate the target's Service Level Agreement (SLA).
To place a target under Notification Blackout, you need to have at least Blackout Target privilege on the target.
There are two types of Notification Blackouts:
Maintenance Notification Blackout: The target is under a planned maintenance and administrators do not want to receive any notifications during this period. Since the target is brought down deliberately for maintenance purposes, the Notification Blackout duration should not be considered while calculating the availability percentage and SLA. In this scenario, an administrator should create a maintenance Notification Blackout.
Notification-only Notification Blackout: The target is experiencing an unexpected down time such as a server crash. While the administrator is fixing the server, they do not want to receive alerts as they are already aware of the issue and are currently working to resolve it. However, the availability percentage computation should consider the actual target status of the Notification Blackout duration and the SLA should be computed accordingly. In this scenario, the administrator should create a Notification-only Notification Blackout.
By default, when a Notification Blackout is created, it is a maintenance Notification Blackout (the Under Maintenance option will be selected by default and the administrator will need to select the Non-maintenance option in order to create a regular Notification-only Notification Blackout.
A Notification Blackout can be defined for individual target(s), a group of multiple targets that reside on different hosts, or for all targets on a host. The Notification Blackout can be scheduled to run immediately or in the future, and to run indefinitely or stop after a specific duration. Notification Blackouts can be created on an as-needed basis, or scheduled to run at regular intervals. If, during the maintenance period, the administrator discovers that he needs more (or less) time to complete his maintenance tasks, he can easily extend (or stop) the Notification Blackout that is currently in effect. Notification Blackout functionality is available from both the Enterprise Manager console as well as via the Enterprise Manager command-line interface (EMCLI). EMCLI is often useful for administrators who would like to incorporate the blacking out of a target within their maintenance scripts.
Notification Blackout Access
Enterprise Manager administrators that have at least Blackout Target privileges on all Selected Targets in a Notification Blackout will be able to create, edit, stop, or delete the Notification Blackout.
In case an administrator has at least Blackout Target privileges on all Selected Targets (targets directly added to the Notification Blackout), but does not have Blackout Target privileges on some or all of the Dependent Targets, then that administrator will be able to edit, stop, or delete the Notification Blackout.
Blackouts allow you to collect accurate monitoring data. For example, you can stop data collections during periods where a managed target is undergoing routine maintenance, such as a database backup or hardware upgrade. If you continue monitoring during these periods, the collected data will show trends and other monitoring information that are not the result of normal day-to-day operations. To get a more accurate, long-term picture of a target's performance, you can use blackouts to exclude these special-case situations from data analysis.
Blackouts/Notification Blackouts allow you to suspend monitoring on one or more managed targets.
To create a Blackout/Notification Blackout:
From the Enterprise menu, select Monitoring and then Blackouts and Notification Blackouts.
From the table, click Create. Blackout and Notification Blackout selection dialog displays.
Choose either Blackout or Notification Blackout and click Create. The Create Blackout/Notification Blackout page displays.
When creating a Notification Blackout, you will also be able to specify the type of Notification Blackout via the Maintenance Window options.
Under maintenance. Target downtime is excluded from Availability(%) calculations.
Non-maintenance. Any target downtime will impact Availability(%) calculations.
Enter the requisite parameters for the new blackout/Notification Blackout and then click Submit.
Blackouts allow you to suspend monitoring on one or more managed targets.
To edit a Blackout:
From the Enterprise menu, select Monitoring and then Blackouts and Notification Blackouts.
If necessary, use the Search and display options to show the blackouts you want to change in the blackouts table.
Select the desired Blackout/Notification Blackout. Details are displayed. click Edit. The Edit Blackout/Notification Blackout page displays.
Make the desired changes and click Submit.
Note: Enterprise Manager also allows you to edit blackouts after they have already started.
To view information and current status of a blackout:
From the Enterprise menu, select Monitoring and then Blackouts and Notification Blackouts.
If necessary, you can use the Search and display options to show the blackouts you want to view in the blackouts table.
Select the desired Blackout/Notification Blackout. Details are displayed.
Viewing Blackouts from Target Home Pages
For most target types, you can view a Blackout/Notification Blackout information from the target home page for any target currently under Blackout/Notification Blackout. The Blackout/Notification Blackout Summary region provides pertinent Blackout/Notification Blackout status information for that target.
Viewing Blackout/Notification Blackouts from Groups and Systems Target Administration Pages
For Groups and Systems, you can view Blackout/Notification Blackout information about the number of active/scheduled Blackouts/Notification Blackouts on a group/system and its member targets.
When managing a large number of targets, the number of completed Blackouts/Notification Blackouts, or those Blackouts/Notification Blackouts that have been ended by an administrator can become quite large. Removing these ended Blackouts/Notification Blackouts facilitates better search and display for current Blackouts/Notification Blackouts.
To purge ended Blackouts/Notification Blackouts from Enterprise Manager:
From the Enterprise menu, select Monitoring and then Blackouts and Brownouts.
Use the search criteria to filter for the desired targets.
From the Show drop-down menu, select History.
In the table, select the ended Blackouts/Notification Blackouts you want to remove and click Delete. The Delete Blackout/Brownout confirmation page appears.
Click Delete to complete the purge process.
You can control blackouts from the Oracle Enterprise Manager 13c Cloud Control Console or from the Enterprise Manager command line utility (emctl). However, if you are controlling target blackouts from the command line, you should not attempt to control the same blackouts from the Cloud Control console. Similarly, if you are controlling target blackouts from the Cloud Control console, do not attempt to control those blackouts from the command line.
From the command line, you can perform the following blackout functions:
Starting Immediate Blackouts
Stopping Immediate Blackouts
Checking the Status of Immediate Blackouts
Note:
When you start a blackout from the command line, any Enterprise Manager jobs scheduled to run against the blacked out targets will still run. If you use the Cloud Control Console to control blackouts, you can optionally prevent jobs from running against blacked out targets.To use the Enterprise Manager command-line utility to control blackouts:
Change directory to the AGENT_HOME/bin directory (UNIX) or the
AGENT_INSTANCE_HOME\bin directory (Windows).
Enter the appropriate command as described in Table 4-1, "Summary of Blackout Commands".
Note:
When you start a blackout, you must identify the target or targets affected by the blackout. To obtain the correct target name and target type for a target, see Chapter 27, "Administering Enterprise Manager Using EMCTL Commands."Table 4-1 Summary of Blackout Commands
Use the following examples to learn more about controlling blackouts from the Enterprise Manager command line:
To start a blackout called "bk1" for databases "db1" and "db2," and for Oracle Listener "ldb2," enter the following command:
$PROMPT> emctl start blackout bk1 db1 db2 ldb2:oracle_listener -d 5 02:30
The blackout starts immediately and will last for 5 days 2 hours and 30 minutes.
To check the status of all the blackouts on a managed host:
$PROMPT> emctl status blackout
To stop blackout "bk2" immediately:
$PROMPT> emctl stop blackout bk2
To start an immediate blackout called "bk3" for all targets on the host:
$PROMPT> emctl start blackout bk3 -nodeLevel
To start an immediate blackout called "bk3" for database "db1" for 30 minutes:
$PROMPT> emctl start blackout bk3 db1 -d 30
To start an immediate blackout called "bk3" for database "db2" for five hours:
$PROMPT> emctl start blackout bk db2 -d 5:00
The Blackouts Best Effort feature allows you to create blackouts on aggregate targets, such as groups or systems, for which you do not have Blackout Target (or Higher) privileges on all members of the aggregate target.
Here, an Enterprise Manager administrator has Blackout Target privilege on an aggregate target but do not have OPERATOR privilege on its member/associated targets. You should ideally create a Full Blackout on this aggregate target. When defining the blackout, you are allowed to select any member target, even those member targets for which you have no Blackout Target privileges.
When the blackout actually starts, Enterprise Manager checks privileges on each member target and only blackout those on which you have Blackout Target( or Higher) privileges. This automated privilege check and target blackout selection is Enterprise Manager's "best effort" at blacking out the aggregate target.
The Blackout Best Effort functionality is targeted towards the creation of blackouts on targets of any aggregate type, such as Group, Hosts, Application Servers, Web Applications, Redundancy Groups, or Systems.
All targets the blackout creator has Blackout Target (or higher) privilege on will be displayed in the first step of Create/Edit Blackout Wizard. Once the blackout creator selects an aggregate type of target to be included in the Blackout Definition, this Blackout is "Full Blackout" by default.
The creator has the option of choosing the Blackout to run on ”All Current” or ”Selected” Targets, by selecting the appropriate values from the List box. Only when the "Full Blackout" option is chosen, will Blackout Best Effort affect targets for which the creator does not have Blackout Target (or higher) privileges.
Consider 3 targets T1,T2 and T3 (all databases). A Group G1 contains all these 3 targets.
User U1 has OPERATOR privilege on T1,T2 and G1. User U1 has VIEW privilege on T3.
User U1 creates a scheduled full blackout on target G1. Scheduled implies that the blackout will start at a later point in time.
At the time of blackout creation, the tip text Needs Blackout Target privilege, see Tip below the table would be shown beside target T3.
When this blackout starts, if by that time User U1 has been granted OPERATOR privileges on target T3, then target T3 would also be under blackout. Otherwise only targets T1, T2 and G1 will be under blackout.