Investigate Real-time Observations

As previously described, observations are the actions that were seen on a host or target that were configured to be monitored through real-time monitoring rules. Each distinct user action results in one observation.

Observations can have one of many audit statuses. The basic audit status "unaudited" means that the observation was detected, there just is no indication that this action was good or bad. The authorized status means that some review has happened for the observation and it should be treated as expected to occur (it was a good change). The unauthorized status means that this observation has been reviewed and has been found to be against policy. This may result in either a corrective fix, a change to policy, or a compensating control being put in place. The audit status for observations can be automatically set by a rule so that all observations triggered by the rule get a default audit status. The status can also be set manually through the UI reports discussed below. The most advanced capability involves integrating with a Change Management Request server through a Cloud Control connector to automatically determine on a per-observation basis if that action was supposed to happen.

The following sections provide additional details regarding real-time monitoring observations:

View Observations

There are four key ways to see what real-time monitoring observations have occurred in your environment:

The first three observation screens are available from the Enterprise menu by selecting Compliance, then selecting Real-time Observations. This page that lets you choose which of the three reports to look at and also shows any Management Agent warnings related to configuration of Real-time monitoring rule configuration. These warnings are reported from the Management Agent and could impact observations from being delivered to the Cloud Control server. If you are missing observations that are expected, review these warnings and address any configuration issues that is causing them.

Viewing Observations By Systems

When observations occur, they can be marked as authorized or unauthorized automatically. This provides one way you to find observations that are important for you to look into. However, if a rule is not configured to reconcile observations with a change management server, it can be difficult to find the observations that are important to you through only an attribute search. Being able to view observations by business application (generic systems) and drilling down into observation details allows you to discover where there may be issues that should be investigated regardless of the observation's audit status.

Typically, IT managers and line of business owners must identify when unwanted configuration drift occurs in their business applications. By browsing observations by systems, you can easily see which changes affect specific business applications. Observations can be filtered by whether they are authorized, unauthorized, unaudited or both. They can also be filtered by time.

This begins with you choosing one or more business applications and being able to see the relative counts of observations. This report starts at the business application level (generic systems) because an IT manager and compliance auditor may not know what a target is used for. A business application is modeled in Cloud Control as a generic system.

If you are more technical, you still may want to start at this business application level if this is the business application you are working on.

To view observations by systems, follow these steps:

From the Enterprise menu, select Compliance, then select Real-time Observations.
Click Browse Observations by System Targets.

Cloud Control displays the Select Root Target(s) page that lists the Target Name for each system target. There is also a link for all targets not belonging to a system target.
You can begin viewing a report for a given system target by selecting one or more system targets and clicking on the View Details for Selected Systems button.

You will see counts for each system target selected by the time range selected. For instance if you are looking at the monthly time range, each column in the table will represent one day from the month. The count will be the count of observations for that day and system target.

Click on the system target name to drill down and show the counts by each target that comprises the system target. You can continue to click on the links in the first column of the table to drill down until you get to the entities that had observations (for example: file names, process names, user accounts, and so on).

Clicking on the count displays a screen that shows the actual observations that occurred during that time period.

Viewing Observations By Compliance Framework

The ability to view observations as they relate to a compliance standard structure is something that is typically done by a non-technical role such as an IT Manager, Line of Business Owner, Compliance Manager, or Executive.

You can identify some set of Compliance Frameworks that reflect the IT compliance framework that the organization follows. Observations can be filtered by whether they are authorized, unauthorized, unaudited or both. They can also be filtered by time.

To view observations by Compliance Framework, follow these steps:

From the Enterprise menu, select Compliance, then select Real-time Observations.

Clicking on the count displays a screen that shows the actual observations that occurred during that time period.
Click Browse Observations by Compliance Frameworks.

Cloud Control displays the Select Compliance Frameworks page that lists each defined Compliance Framework.
You can begin viewing a report for a given framework by selecting one or more frameworks and clicking on the View Details for Selected Frameworks button.

You will see counts for each framework selected by the time range selected. For instance if you are looking at the monthly time range, each column in the table will represent one day from the month. The count will be the count of observations for that day and framework.Click on the framework name to drill down and show the counts by each second-level framework folder that is in the selected framework. You can continue to click on the links in the first column of the table to drill down until you get to the entities that had observations (for example: file names, process names, user accounts, and so on.
Clicking on the count displays a screen that shows the actual observations that occurred during that time period.

This drill-down capability provided by these screens makes it easy for you to easily find where observations are occurring. When you have an environment with tens of thousands of targets across hundreds of business applications, it is impossible to view observations simply using a table and search unless you know exactly the search conditions they are looking for. In a matter of an hour, with this large of an environment even with little activity, there can be thousands of observations.

Viewing Observations By Search

For cases when the two browse by screens cannot provide the best view of what observations have happened in your environment, Cloud Control also provides a search capability to find observations.

To search observations, follow these steps:

From the Enterprise menu, select Compliance, then select Real-time Observations.
Click Search Observations.

Cloud Control displays the Search observation page which has search filters on the top half of the page and search results on the bottom half
You can set any number of filters in the search area. You can also click on the Add Fields button to add any fields that are available in the search results table.
With the options available in search, you can find observations performed over a time range, by a specific user, against a specific target, changes to a specific entity, and so on. Nearly every use case for finding observations can be solved using a combination of search fields.

Viewing Details of an Incident

Observations are logically bundled together based on the compliance standard rule, target and user that performed the action. This bundling is discussed in more detail in Creating a Real-time monitoring Rule section.

When one or more observations of a bundle are unauthorized, the bundle is considered to be in violation. This violation will lead to an event being created in Cloud Control Incident Management. The event name will be based on the message field defined in the real-time monitoring rule. When viewing this event in the incident management UI, several fields will show details of the bundle; the target type, entity type, number of observations in the bundle, observations by audit status, and so on. You can click on the Update Audit Status link to go to the bundle observations page.

This Observations page shows the list of observations in the observation bundle for this event. You can filter on various attributes for each observation, including but not limited to the authorized/unauthorized status, user, time, and so on.

Operations on Observations During Compliance Evaluation

The following sections describe how a real-time monitoring observation's audit status can be adjusted and how notifications can help in evaluating compliance results.

Manually Setting an Observation As Authorized Or Not Authorized

Any time a user is viewing the details of a real-time observation, the user can change the audit status for the observation. You can override the audit status of an observation if you investigate the user action and determine that the activity should have resulted in a different audit status. Based on the real-time monitoring rule, all observations will either have a pre-set audit status or will have an audit status determined by an integration with a Change Request Management server. The available audit statuses are:

Unaudited: No evaluation has happened to determine if the observation was good or bad.
Authorized: The observation has been determined to be good, some action that was desired to occur.
Unauthorized: The observation has been determined to be bad, some action that was not wanted.
Unauthorized-Cleared: The observation had previously been determined to be bad, some action that was not wanted, but it has been handled through a fix, a policy change, or a compensating control and has now been cleared.

To change the audit status of an observation, view the observation from either of the browse by UI pages, the observation search page, or the incident manager UI. Select the observation and click Update Audit Status. A popup will come up allowing you to select the new audit status and a comment describing the reason for the status change. The history of all audit status changes is maintained for each observation.

If the Cloud Control instance is using the Change Request Management server connector for integration, there are some special considerations:

If you change an unauthorized observation into an authorized observation, then you have the option of entering a change request ID that is known to authorize the change. This change request ID should match a request that already exists in your change request management system. You can also enter a comment. If a change request ID is provided, then the change request is annotated with the change just as if the system had automatically authorized it. If an incident had been created for the observation bundle, then the event/incident is updated with the new number of unauthorized observations.
If you change an authorized observation into an unauthorized or unaudited observation, any annotations that were made to any change requests are rolled back. If there was already an incident raised for the observation bundle, then the annotation is changed to update the number of unauthorized observations in the incident. If this is the first unauthorized observation in a group, then an event is created an incident is raised. You can provide a comment for the change.
When you manually set the observation to be authorized and enter a change request ID and the rule has change management integration enabled, no attributes of the change request are compared with the observation. The change request is simply updated with the observation details.
When rolling back annotations in the change management server, the observation annotations are marked as rolled-back instead of actually removing the annotation. This occurs to avoid user confusion not knowing possibly why the annotations were removed. Also, if the observation later becomes authorized again, the rolled-back marking can simply be removed to bring the annotation back.

Notifying a User When an Observation Occurs

If a compliance standard rule is created and you do not use change management reconciliation with the rule, then there will be no automated authorized/unauthorized check done on the observations. You can specify for this rule that each observation bundle should result in informational event being generated for the observation bundle. Details on how to configure this is in the section Creating a Real-time Monitoring rule.

The event will have a notation. From the Incident Management console the user can look at events and incidents. When looking at a single event, there is a link available to see the observations associated with this observation bundle's event. Each observation bundle can only have one event. If at least one observation in the bundle is unauthorized, then the bundle is considered to be in violation which results in the event being generated.

Since this notification does not require user intervention or follow-up action, it is treated as informational. If at a later time, someone changes one of these unaudited observations into an authorized or unauthorized one, a new informational event for the unaudited observations will not be re-delivered. It is delivered only once for the observation bundle. However if one of the observations is manually set to unauthorized, then a violation is raised for the entire observation bundle.

When at least one observation in a bundle is in an unauthorized state, a violation is created. That violation becomes an event in the Incident Manager Console. Use the Incident Manager feature to set up a notification. For more information about this, on the Incident Manager page, click on the online help link, Setting Up Notifications With Rules under the Setting Up Notifications section under Getting Started.

Notifying a User When an Authorized Observation Occurs

When an authorized observation occurs, it is not typical for you to receive a notification on these observations since the activity that caused the observation was expected. If you are using change management reconciliation, you have an option to annotate the authorizing change request with the observation details. The updates to the change request is one way customers can learn of authorized activity. You can set filters in their change management system to let them know that a change request has had authorized activity against it.