Skip Headers
Oracle® Real User Experience Insight User's Guide
12c Release 1 for Linux x86-64

Part Number E26360-03
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

12 Controlling the Reporting of Monitored Traffic

This chapter describes how the reporting of monitored traffic can be fine optimized to meet your information requirements. This includes the specification of the cookie technologies used within your network environment, use of named Web server and client groups, as well as a number of advanced facilities, such as rule ordering and data retention policies.

12.1 Viewing a Traffic Summary

You can open an overview of the monitored network traffic by selecting System, then Status, and then Data processing. A list of the processing systems within your RUEI deployment is displayed. An example is shown in Figure 12-1.

Figure 12-1 Data Processing Status Window

Description of Figure 12-1 follows
Description of "Figure 12-1 Data Processing Status Window"

Click the required system to view information about hits, pages, and session processing, as well as the system load. An example is shown in Figure 12-2.

Figure 12-2 Data Processing Dialog

Description of Figure 12-2 follows
Description of "Figure 12-2 Data Processing Dialog"

Note the Available resource usage (%) item on the Performance tab indicates the current processing level. If this approaches 100%, it means a lag in the processing of data is starting to occur, and it is no longer possible to process data in real time.

Be aware that because this facility is based on application logic, non-application traffic (such as suites, services, and SSOs), are not represented in the displayed reports.

Important:

In order for RUEI to correctly report on monitored traffic, it is strongly recommended that you regularly review this traffic summary. If necessary, review the RUEI configuration accordingly. For example, add additional cookie technologies. In addition, if the system is unable to track sessions, proper tracking of user flows will also not be available because user flow reporting requires session tracking.

12.2 Specifying the Cookie Technology

In order to accurately monitor your Web environment, RUEI needs to know and understand the cookie technology your Web site is using. This will either be a standard technology (such as ASP or ColdFusion), or a custom implementation. In the case of the latter, you will need to provide the system with information about it. Note that you can define a maximum of five cookie technologies for use when monitoring.

To specify your cookie technology, do the following:

  1. Select Configuration, then Applications, and then Session tracking. Note that this option is only available to Administrators. The currently defined cookie settings are displayed. An example is shown in Figure 12-3.

    Figure 12-3 Session Tracking Window

    Description of Figure 12-3 follows
    Description of "Figure 12-3 Session Tracking Window"

  2. Click Add new cookie or an existing cookie definition. A dialog similar to the one shown in Figure 12-4 appears.

    Figure 12-4 Add New Cookie Dialog

    Description of Figure 12-4 follows
    Description of "Figure 12-4 Add New Cookie Dialog"

  3. Select the cookie technology used in your Web environment from the Cookie type menu. If you are using a non-standard technology, select "(custom)".

  4. If you selected "(custom)", you are required to specify the name of the cookie used by your organization. Note that you can specify wildcard characters (*) as part of the cookie name.

    Note:

    Cookie names are case sensitive.
  5. If you select "(URL argument)", you are required to specify the name of the URL argument used by your organization. The use of URL arguments in session tracking is fully explained in Appendix B, "Cookie Structures". When ready, click Save.

Any changes made to this setting are applied after a short interval (typically, 5 - 10 minutes), and are then visible within the Reporter system shortly after this.

Multiple Configured Cookies

When multiple configured cookies are found in the same hit, RUEI merges all hits with the multiple cookie values into a single user session. For example, consider the situation in which three hits have following cookie values:

hit1: CookieA=123;
hit2: CookieB=321;CookieA=123;
hit3: CookieB=321;

In this case, all three hits will be regarded as belonging to the same user session.

12.2.1 Implementing JavaScript Cookie Generation

As mentioned earlier, session tracking is based on cookies. However, in certain circumstances, a cookie may not be suitable or available. For example, consider the following situations:

  • The cookie changes with every hit (for instance, this is the case with ObSSOCookie).

  • The path set within the cookie only covers part of the application.

  • The privacy policies configured on the Web server disable the use of cookies.

If no suitable cookie is available for session tracking, it is recommended that you implement a client-side cookie mechanism using JavaScript.

Configuring a Client-Side Cookie Mechanism

Do the following:

  1. Add code similar to the following to the appropriate login page:

    <SCRIPT 
    LANGUAGE="JavaScript">if(document.cookie.indexOf('track=')==-1){document.cookie
    ='track='+parseInt(Math.random()*2147418112)+new 
    Date().getTime()+';path=/;domain='+document.location.host.substring( 
    document.location.host.lastIndexOf('.',  
    document.location.host.lastIndexOf('.') - 1)) ;}</SCRIPT>
    

    Note that the above code is for informational purposes only. You may need to modify it to meet your specific requirements.

  2. Select Configuration, then Applications, and then Session tracking. Click Add new cookie. The dialog shown in Figure 12-5 appears.

    Figure 12-5 Add New Cookie Dialog

    Description of Figure 12-5 follows
    Description of "Figure 12-5 Add New Cookie Dialog"

  3. Select the cookie technology (custom) from the Cookie type menu, and specify the appropriate cookie name. In the above JavaScript code, this is track. Note that the name should match that specified in the login page JavaScript code, and should only contain alphanumeric characters. In addition, it is recommended that the cookie name is restricted to no more than 10 characters in order to minimize header sizes. When ready, click Save.

Verifying the Cookie Configuration

To verify that your cookie configuration is being tracked correctly, do the following:

  1. Clear all cookies in the browser.

  2. (Re)login to the monitored application.

  3. Perform a number of page views.

  4. Logout out of the monitored application.

  5. Wait for at least 10 minutes.

  6. Open the RUEI Reporter environment, and select Browse data, open the All sessions group, select Session diagnostics, and locate the recorded session (by user ID or time). You can filter on applications.

  7. Open the session and verify that there where more page views than just the login page. This verifies that the session ID is preserved after the login.

12.2.2 Specifying the Fallback Session Tracking Mechanism

If you do not specify a cookie technology, then (by default) a combination of the client network and client browser is used to track sessions. However, in the event that this is not suitable for your environment, the client IP address can be used as an alternative tracking mechanism.

To specify the fallback session tracking mechanism, do the following:

  1. Select Configuration, then Applications, and then Session tracking. The currently defined cookie settings are displayed. Click the currently defined session tracking fallback mechanism. The dialog shown in Figure 12-6 appears.

    Figure 12-6 Set Session Tracking Fallback Dialog

    Description of Figure 12-6 follows
    Description of "Figure 12-6 Set Session Tracking Fallback Dialog"

  2. Use the Tracking mechanism menu to specify if a client network and browser combination should be used (the default), or the client IP address.

    When ready, click Save. Any change you make takes effect immediately.

Which Fallback Session Tracking Mechanism Should be Selected?

When considering which fallback mechanism to use, a general rule is that external-facing applications should use the default network/browser combination, while internal-facing applications should use client IP address. In the case of multiple users behind the same proxy server, the use of the default fallback mechanism is recommended. However, be aware this will result in all such users being recorded in one single session. The use of the client IP address mechanism is generally recommended in the following circumstances:

  • All users have a unique IP address. Note that for each application, you can specify if the client IP address should be retrieved from the TCP packet or a specific HTTP request header. This is described in Appendix P, "Monitoring NATed Traffic".

  • The organization enforces the use of a normalized browser. That is, a standard browser (such as Internet Explorer or Mozilla Firefox), with a standard version and plug-ins.

  • Some (or all) of the monitored applications are partially implemented in Java. Oracle E-Business Suite (EBS) is an example of such an application architecture. For these applications, the use of the client IP address mechanism prevents both Java and client requests appearing in the same reported session.

Important:

The accurate specification of the cookie technologies used within your Web site is strongly recommended to ensure the accurate reporting of your network traffic.

In addition, you should that the cookie specified to track visitor sessions is not blinded. If it is, session creation based on the cookie will fail.

12.3 Defining Named Web Server Groups

You can use the Named servers facility to obtain more detailed insight into server usage within your monitored Web sites. This facility allows you to assign ranges of server IP addresses to a Web server group, and to individual Web servers. For example, a server group could be a department or data center, and the server name refers to specific Web servers within that group. In this way, you can easily identify the location of specific Web servers when problems (such as failed pages) occurred. The Named server facility is only available to users with Full IT user access (see Table 14-2).

Viewing Server Information

The Web server information collected during monitoring can be viewed in the Data Browser via the All pages, Key pages, All functions, Failed functions groups, Failed URLs, Failed pages, and Slow URLs groups. The server IP identifies the specified IP addresses, and the server group refers to the group name. By zooming into a server group, you can view the individual Web server names that comprise the group. Zoom in again, and you can view the individual IP addresses assigned to that Web server.

Sources for Named Server Identification

When reporting named servers, the server's IP address is, by default, fetched from the IP packet. However, when the Web server is placed in front of a NAT device, it may be more useful for the IP address to be obtained from a specific HTTP header, cookie, or other item. For this reason, you can specify a hierarchy of sources from which it should be obtained. Similar source schemes can also be specified for group and server names. Note that you can also define fallback identification that should be used when these schemes fail to yield a value.

12.3.1 Defining Named Server Identification Sources

To define how your Web servers should be identified and reported, do the following:

  1. Select Configuration, then General, and then Named servers. Select one of the following options:

    • IP address sources: use this option to specify the sources from which server IP addresses should be obtained.

    • Named sources: use this option to specify the sources from which server names should be obtained.

    • Group sources: use this option to specify the sources from which server group names should be obtained.

    • Identification fallback: use this option to specify the IP address ranges, groups, and names that should be used for server identification when they cannot be obtained from their defined sources. The use of this option is described in Section 12.2.2, "Specifying the Fallback Session Tracking Mechanism".

  2. Click Add new source. A dialog similar to the one shown in Figure 12-7 appears.

    Figure 12-7 Add Server IP Address Source Dialog

    Description of Figure 12-7 follows
    Description of "Figure 12-7 Add Server IP Address Source Dialog"

  3. Use the Source type and Source value fields to specify the identification scheme that should be used for the selected item. The available options are explained in Table 12-1. When ready, click Save.

    Table 12-1 Identification Options

    Option Description

    Header in request

    The identification item should be taken from a specified request header.

    Header in response

    The identification item should be taken from a specified response header.

    Cookie

    The identification item should be taken from a specified cookie element.

    SSL Client certificate name

    The identification item should be taken from the SSL client certificate. This option is not available for IP addresses.

    Literal value

    Specifies a literal value for the identification item. This option is not available for IP addresses.


  4. Once created, you can use the ruling facility to specify additional matching rules that should be used to refine the newly created definition. This is described in Section 8.2.3, "Using the Ruling Facility".

  5. It is important to understand that the defined sources for a server's identification are evaluated in the order in which they appear in the list. If necessary, you can use the Move up and Move down options within an item's context menu to modify its position within the list.

12.3.2 Defining the Server Identification Fallback Scheme

You can define a server identification scheme that should be used if information about a server cannot be obtained from its specified sources. This is referred to as server identification fallback. To define a identification fallback scheme, do the following:

  1. Select Configuration, then General, then Named servers, and then Identification fallback. Click Add new server. The dialog shown in Figure 12-8 appears.

    Figure 12-8 Add Named Server Fallback Identification Dialog

    Description of Figure 12-8 follows
    Description of "Figure 12-8 Add Named Server Fallback Identification Dialog"

  2. Specify a range of IP addresses (or a specific IP address) within a netmask, and the associated Web server and group names. When ready, click Save.

Important:

The specified IP address/netmask combination must be unique. In addition, if duplicate IP addresses with different netmasks are specified, the more specific one is used for reporting purposes.

Uploading a List of Fallback Identifications

Optionally, you can click Upload to merge a list of fallback identification definitions with those that are currently defined. The file must contain only one entry per line, and the information for each server (as shown in Figure 12-8) must be tab-separated. Note that any definition in the merged file for an already defined server identification overwrites its existing definition.

12.4 Defining Named Client Groups

In some instances, you want to be able to enhance the information associated with visitor IP addresses. This is especially useful when monitoring Intranet traffic and you want to be able to use your own client classification.

To use this facility, do the following:

  1. Select Configuration, then General, and then Named clients. The currently defined named servers are listed. Click Add. This option is only available to users with IT Full level access. The dialog shown in Figure 12-9 appears.

    Figure 12-9 Add Named Client Dialog

    Description of Figure 12-9 follows
    Description of "Figure 12-9 Add Named Client Dialog"

  2. Use the fields within the dialog to specify a range of IP addresses or a specific IP address within a netmask, the client, and their associated group (for example, company department). When ready, click Save.

Important

The specified IP address/netmask combination must be unique. In addition, if duplicate IP addresses with different netmasks are specified, the more specific one is used for reporting purposes.

12.4.1 Uploading a List of Named Clients

Optionally, you can click Upload to merge a list of named clients with those that are currently defined. The file must contain only one entry per line, and the information for each client (shown in Figure 12-9) must be tab-separated. Note that any definition in the merged file for an already defined named client overwrites its existing definition.

Any changes made to your defined named client groups are applied after a short interval (typically, 5-10 minutes), and are then visible within the Reporter system shortly after this.

12.4.2 Viewing Named Client Group Information

The visitor information can be viewed within the Data Browser via the named client view (within the Failed URLs, Failed pages, Key pages, Slow URL, All sessions, All functions, and Failed functions groups).

12.5 Ignoring Failed URL Hits

Hit failures are recorded in the failed URL group. Because hit failures can occur for a wide variety of reasons, you can control what is recorded. For example, it is unlikely that you want incidents related to remote robot searches to be recorded. Do the following:

  1. Select Configuration, then General, then Advanced settings, then URL handling, and then Ignore failed URLs. Note that this option is only available to Administrators. The dialog shown in Figure 12-10 appears.

    Figure 12-10 Ignore Failed URLs Dialog

    Description of Figure 12-10 follows
    Description of "Figure 12-10 Ignore Failed URLs Dialog"

  2. Specify any file names that should be ignored within the failed URL view. That is, they should not be seen as errors. Note that any directory information within file name definitions are ignored, and the defined files are also removed from the listed object URLs. Click Add to define a new file name that should be ignored. Click the Remove icon to the right of a defined file name to delete it from the list of files to be ignored.

    Upon installation, two default files, robots.txt and favicon.ico, are automatically configured. When ready, Click Save. Any changes to this setting are applied after 10 minutes. A short period after this time, the changes you have specified are visible in the Reporter interface.

12.6 Filtering Arguments in the Page URL Dimension

You can control whether you want all, some, or no URL arguments recorded within the lowest level page URL dimension. Do the following:

  1. Select Configuration, then General, then Advanced settings, then Page URL handling, and then Page URL argument filtering. Note that this option is only available to Administrators. The dialog shown in Figure 12-11 appears.

    Figure 12-11 Page URL Argument Filtering

    Description of Figure 12-11 follows
    Description of "Figure 12-11 Page URL Argument Filtering"

  2. Use the Argument filter menu to select the appropriate filter. The default is “allow-all”. That is, record all arguments. When ready, click Next.

  3. If you selected the "allow-some" filter, the next dialog requires you specify which arguments should be recorded. Separate multiple arguments with an ampersand (&) symbol. When ready, click Next.

The new setting is applied after 10 minutes. Shortly after this time, the changes you have specified are visible in the Reporter interface.

Note:

It is recommended that you make use of this facility if session or other random arguments are included in your page URLs. Otherwise, the content of page-based views (such as all pages or failed URLs) can become very large.

12.7 Controlling Session Reporting

Within RUEI, session information is reported within the All sessions group. Here, information about a visitor session is available appropriately five minutes after the start of a session. By default, a visitor session is considered terminated if the visitor has been inactive for longer than the defined session idle time (by default, 60 minutes).

In order to optimize the reporting of sessions, the Session idle time advanced setting is available to specify the period (in minutes) of inactivity after which a visitor session is regarded as terminated. The default is 60 minutes.

Important:

Because of the impact this setting can have on the performance of your installation, as well as the accuracy of the reported data, it is strongly recommended that you only change it under guidance from Customer Support.

Specifying Session Settings

In order to specify the idle time that should used when reporting sessions, do the following:

  1. Select Configuration, then General, then Advanced settings, then Session processing, and then Session idle time. The dialog shown in Figure 12-12 appears.

    Figure 12-12 Change Session Reporting Dialog

    Description of Figure 12-12 follows
    Description of "Figure 12-12 Change Session Reporting Dialog"

  2. Specify, in minutes, the period of visitor inactivity after which the session should be regarded as terminated. The default is 60 minutes. When ready, click Save.

Any change you make to this setting takes effect within five minutes.

12.8 Controlling Rule Ordering Within RUEI

By default, the order in which application, SSO profile, suite, and service filters are matched within RUEI is determined by the level of detail specified in the definition. That is, the definitions with the most information specified for them are applied first. However, sometimes you may want to modify the order in which filters are applied.

For example, you want to monitor network traffic for the domain "shop.oracle.com". You have defined two applications: one for the domain "shop*", and one for the domain "*oracle*. Because the string "*oracle*" is longer than the string "shop*", it is applied first. However, you want page identification for the "shop*" domain to take priority. You can use the rule ordering facility to override the default rule matching order, and specify the order in which pages for the required domains should be applied.

Note:

It is recommended you use the default rule ordering, and that you define your applications, SSO profiles, suites, and services with sufficient information for them to be mutually exclusive.

To use the rule ordering facility, do the following:

  1. Click the Configuration tab, select the Configuration menu option, and then the option Edit ruling orders. Note this option is only available to users with Full IT access permissions. The dialog shown in Figure 12-13 appears.

    Figure 12-13 Edit Rule Ordering

    Description of Figure 12-13 follows
    Description of "Figure 12-13 Edit Rule Ordering"

  2. Use the Automatic rule ordering check box to specify whether the rule ordering is automatically derived from the currently defined applications, SSO profiles, suites, and services. As explained earlier, by default, the definitions with the most information specified for them are applied first. This is check box is automatically unchecked if you use the Up and Down controls to specify the order in which the rules should be applied. If you re-check it, the filter ordering is automatically reset to the default.

    Note any changes you make are immediately put into effect. When ready, click Close.

    Important:

    Be aware that if you modify the default rule ordering, and then define a new application, SSO profile, suite, or service, its associated filter is immediately placed at the bottom of the current rule ordering. Therefore, you should always review the rule ordering after the creation of new filters.

12.9 Specifying Data Retention Policies

The availability of specific data within the Data Browser, as well as reports based on that data, depends on the amount of available disk space on the Collector and Reporter systems, as well as the amount of database space available on the Reporter system. This is illustrated in Figure 12-14.

Figure 12-14 Data Retention Across Collector and Reporter Systems

Description of Figure 12-14 follows
Description of "Figure 12-14 Data Retention Across Collector and Reporter Systems"

Data gathered during monitoring is first written to log files, stored on the Collector system. These files are copied to, and processed by, the Reporter to populate the database that holds the multi-dimensional data structure viewable through the Data Browser and reports. These temporary log files are automatically removed from the Collector system after three days, and from the Reporter system (by default) after seven days. Note that data masking options (described in Section 13.6, "Masking User Information") can be specified to omit the logging of sensitive information.

The log files are used to created the Full session replay (FSR) data store. These files are regularly filtered to create the Error page replay (EPR) data store. The EPR files only contain information about failed events (that is, failed pages, objects, and function calls). Both the FSR and EPR data is held on the Collector system.

The size of the database user quota for the Reporter system is configurable during installation. By default, it is set to 200 GB. It is important to understand that data is consolidated when it is no longer required by the Reporter's defined retention policy. For example, by default, daily information about the last 32 days is retained. Daily information older than this is consolidated into the monthly information. Similarly, monthly information is consolidated into yearly information.

By default, RUEI keeps information on a daily, monthly, and yearly levels for 32 days, 13 months, and five years, respectively. Hence, for example, the oldest daily information will be dropped after 32 days. In addition, temporary log files are kept on the file system for approximately seven days. Be aware that a new RUEI installation will grow quickest during the first 32 days. After that time, the growth rate will slow. Of course, the growth rate depends on monitored traffic levels.

By default, information about failed URLs, pages, and service calls is kept for 15 days. If available, it can be viewed via the Session diagnostics replay facility (described in Section 4.1, "Introduction").

The settings described in the rest of this section allow you to optimize the disk and database utilization of your RUEI installation to meet your operational requirements.

12.9.1 Defining Reporter Retention Policies

To specify the data retention policies used by the Reporter system, do the following:

  1. Select Configuration, then General, then Advanced settings, and then Reporter data retention policy. A screen similar to the one shown in Figure 12-15 appears.

    Figure 12-15 Reporter Data Retention Policy Panel

    Description of Figure 12-15 follows
    Description of "Figure 12-15 Reporter Data Retention Policy Panel"

  2. Select the required setting. The settings shown in Table 12-2 are available.

    Table 12-2 Reporter Data Retention Policy Settings

    Setting Description

    Maximum database Size

    Specifies (in gigabytes) the maximum amount of data allowed to be stored in the selected database. The default is 200 GB. Note that you will need to specify the database SYSTEM user password to change this setting.

    Failed event data retention

    Specifies the period for which information about failed URLs, pages, and service calls is available. The default is for the last 15 days. If information is not available in the Session diagnostics replay facility, you may need to review this setting. Note this setting is linked to the Error Page Replay (EPR) storage size setting (described in Section 13.8, "Defining Collector Data Retention Policies"). If you intend to increase the Failed event data retention setting, it is recommended that you also increase the EPR storage size setting in order to facilitate this. Note also this setting has a high impact on disk space usage, and any change to it should be carefully considered in terms of anticipated network traffic.

    Session diagnostics retention

    Specifies the maximum period for which session diagnostics information is available. This facility is described in Section 4.1, "Introduction". The default is the last seven days, and the minimum is the last two days. This setting has an impact on database and disk space usage. Note that the reported database usage is not included in the reported disk space usage.

    Enriched data exchange retention

    Specifies the maximum period for which information is available via the Enhanced data exchange facility (see Appendix S, "Enriched Data Export Facility"). The default is for the last seven days. That is, data is available for the last six days, plus the current day. The maximum retention period depends on the available database storage capacity.

    Be aware that this setting is applied on the full-day boundary (at approximately midnight). Therefore, if you decrease the number of days (for example, from seven to five), the setting change will take effect within 15 minutes, and the data for days five and six will be purged from the database.

    Note that if set to one day, the previous day's data is deleted at approximately midnight, and only a limited amount of information is available for the current day. Therefore, in order to have access to the previous day's data after midnight, a retention period of at least two days must be specified.

    KPI data exchange retention

    Specifies the maximum period for which KPI information is available via the Enhanced KPI data exchange facility (see Appendix S, "Enriched Data Export Facility"). The default is the last 365 days. The maximum retention period depends on the available database storage capacity.

    Daily data retention

    Specifies the period for which daily information is available. The default is the last 32 days. Be aware that the daily retention setting ensures that there is never a daily group whose monthly group has already been deleted. Therefore, the maximum period for which daily data may be kept depends on the monthly setting. If you want to increase the number of days beyond the indicated permitted range, you should first increase the monthly data retention setting.

    Monthly data retention

    Specifies the period for which monthly information is available. The default is the last 13 months. Be aware that the monthly retention setting ensures that there is never a monthly group whose yearly group has already been deleted. Therefore, the maximum period for which monthly data may be kept depends on the yearly setting. If you want to increase the number of months beyond the indicated permitted range, you should first increase the yearly data retention setting.

    Yearly data retention

    Specifies the period for which yearly information is available. The default is the last five years. The minimum setting depends on the monthly setting.

    Maximum data group size

    Specifies the maximum size to which data groups within the Reporter database are permitted to grow. This setting is described in Section 12.9.3, "Setting the Maximum Size of the Failed Groups". Note that this setting does not apply to groups within Processing Engine databases.


    A dialog similar to the one shown in Figure 12-16 appears.

    Figure 12-16 Change Data Retention

    Description of Figure 12-16 follows
    Description of "Figure 12-16 Change Data Retention"

  3. Use the dialog's control to specify the retention policy for the selected option.

    The projected database utilization (based on monitored traffic levels) is indicated. Information about disk space utilization is available within the dialog boxes for individual settings. Be aware that these estimates are based on the highest usage within the deployed processing systems. For example, if an item has a usage within the Reporter of 50%, and 20% within a Processing Engine, the higher value is used to report usage and availability.

    For most settings, you can click Calculate to see the effect of your selection on database or disk space usage, as applicable.

    When ready, click Save. Note that changes to disk space allocations take effect after approximately 10 minutes, while changes to database allocations only take effect after midnight.

  4. Optionally, click the DB usage tab for an indication of the total database space (in gigabytes) currently used for an item, and the proportion this represents of the database's maximum permitted size. An example is shown in Figure 12-17.

    Figure 12-17 Database Usage Indication

    Description of Figure 12-17 follows
    Description of "Figure 12-17 Database Usage Indication"

Note:

It is recommended that if you want to increase the amount of data kept, you start with the low-level data retention setting and work towards the high-level data retention setting. If you want to decrease the amount of data kept, start with the high-level data retention setting, and work towards the low-level data retention setting.

Calculating Required Days, Months, and Years

When specifying the high, medium, and low-level data retention settings, it is important to understand the dependency between stored days, months, and years. Use the following rules to calculate the required settings:

  • A month is assumed to have 30 days. The number of months that must be stored for a specified period of days is the number of days divided by 30 (rounded up to the next hole integer), plus one. For example, 33 days would require 33/30 (1.1 rounded up to 2), plus 1. Hence, three months.

  • The number of required years for a specified period of months is the number of months divided by 12 (rounded up to the next whole integer). For example, 11 months would require one year, while 13 months would require two years.

For example: 900 days, 31 months, and 3 years.

Note that configuration of Collector data retention policies is described in Section 13.8, "Defining Collector Data Retention Policies".

12.9.2 Setting the Maximum Data Group Size

This section provides a detailed explanation of how the default maximum Data Browser group sizes within the Reporter database can be increased to provide more accurate reporting of monitored traffic. It is strongly recommended that you carefully review the information presented before modifying the default settings.

Condensation

In order to optimize performance, individual groups are not permitted to grow to unlimited sizes. Each main group table has a maximum permitted size. By default, this is 600 MB. The size of the Failed URLs, Failed services, Failed pages, and Slow URLs groups are controlled by a maximum number of rows that can be added to the group's main database table during a 5-minute period. By default, this is 5000 rows. This fully described in Section 12.9.3, "Setting the Maximum Size of the Failed Groups". There is no limit for session diagnostics information.

The maximum size of a group is managed by condensing. This is the process by which the number of rows within the group table are reduced by moving the least used data to a "other" row. For example, rows within the client-browser/version dimension might contain a small number of rows for rarely seen Mozilla Firefox versions. In this case, the number of rows within the table is reduced by replacing these with a single "other(firefox)" row.

You might consider increasing the maximum size of the groups if the information you require is regularly being condensed. For example, required individual page names or user IDs are only reported as "other". Broadly speaking, this approach is recommended in the case of high levels of network traffic.

Important

If the maximum group size is set to more than 150% (that is, 900 MB), it is strongly recommended that the database is configured as a remote database. The maximum recommended group size is 300%Foot 1  (that is, 2.0 GB). Be aware that response times to Data Browser queries and dashboard updates may take longer to process if the maximum group size is increased.

It is important to note that custom dimensions have a significant impact on the condensing process. Incorrect configuration of custom dimensions can result in excessive condensing. For example, product names (or similar highly selective attributes) should not be configured as custom dimensions because that will skew the condensing process. Specifically, each page/product combination becomes a separate candidate for condensing, and only high frequency combinations will remain visible after condensing.

Reviewing the Effects of Changing the Maximum Group Size

After increasing the maximum group size setting, it is strongly recommended that you carefully review the effect of the change on the performance of the Reporter system. You should wait at least a full day, and select System, then Status, and the Data processing. Review the system load and determine whether the Reporter system can handle the additional processing overhead that increasing the maximum group size setting represents.

Important:

Increasing the data group sizes can have a significant impact on overall system performance. Therefore, any changes to data group sizes should be carefully planned, and performed in manageable steps.

12.9.3 Setting the Maximum Size of the Failed Groups

As explained earlier, the Failed URLs, Failed services, and Failed pages groups do not use the maximum group size setting. Instead, their size is controlled through the event_max_fail setting. This specifies the maximum number of rows that can added to the group's main database table during a 5-minute period. By default, this is 5000 rows. For the Slow URLs group, the event_max_slow setting is used, and specifies the number of the slowest URLs that are recorded within each 5-minute period. By default, this is 5000 rows.

Note that if you change the event_max_fail or the event_max_slow setting, you should also review the daily_max_fail setting. This specifies the maximum number of rows that the groups' tables can contain. This is derived from the formula 288 * event_max_fail. The default, is 1.4 million rows.

To modify the above settings, issue the following commands:

$ sqlplus /@$RUEI_DB_TNSNAME
SQL> update UXS_CONFIG set VALUE='10000' where NAME='event_max_fail';
SQL> update UXS_CONFIG set VALUE='4320000' where NAME='daily_max_fail';

Note that the event_max_fail setting is limited to a maximum of 10,000 rows.

12.10 Controlling the Reporting of the Current Period

By default, information about the current (incomplete) period is always shown within selected periods that extend to the present time. In graphical visualizations, this is indicated with a dotted line. An example is shown in Figure 12-18.

Figure 12-18 Example of Incomplete Period Reporting

Description of Figure 12-18 follows
Description of "Figure 12-18 Example of Incomplete Period Reporting"

Specifying When Incomplete Periods Should be Reported

To specify when incomplete periods should be reported, do the following:

  1. Select Configuration, then General, then Advanced settings, then Data visualization, and then Current period reporting. The dialog shown in Figure 12-19 appears.

    Figure 12-19 Current Period Reporting Dialog

    Description of Figure 12-19 follows
    Description of "Figure 12-19 Current Period Reporting Dialog"

  2. Select the visualization scheme to be used when reporting the current period. The options shown in Table 12-3 are available.

    Table 12-3 Visualization Options

    Options Description

    Enabled

    Specifies that all incomplete periods should be reported within all graphical visualizations, as well as value lists, reports, and exports. This is the default.

    Disabled

    Specifies that no incomplete period should ever be reported.

    Specified

    Specifies that incomplete periods should only be reported within the specific visualizations. Note that the "Value list" option covers not only value lists within the Data Browser, but also reports and exports.


    When ready, click Save. Any change you make to this setting takes effect immediately.

12.11 Specifying KPI and SLA Reporting Precision

KPI and SLA values are reported to a certain level of precision. By default, this is two decimal places. However, you are free to modify this to reflect your reporting requirements. Do the following:

  1. Select Configuration, then General, then Advanced settings, and then KPI and SLA precision reporting. Select the item whose reporting you want to modify. For example, SLA success. A dialog similar to the one shown in Figure 12-20 appears.

    Figure 12-20 Edit Precision Reporting Dialog

    Description of Figure 12-20 follows
    Description of "Figure 12-20 Edit Precision Reporting Dialog"

  2. Specify the number of decimal places to which the selected item should be reported. When ready, click Save. Any change to these settings takes effect immediately.

12.12 Setting System-Wide Preferences

As explained in Section 1.5, "Customizing Your Environment", users can customize the formatting settings used in their sessions. They can specify the characters used for the decimal point indicator and the thousand separator, and the date format that should be used. Administrators can also specify defaults for these settings on a system-wide basis by selecting System, then Maintenance, and then Formatting preferences.

12.13 Modifying Client Location Reporting

The client location information reported by RUEI is derived from a predefined table that specifies the geographical location associated with specific IP address and netmask combinations.

If the information held in this table does not meet your reporting requirements (that is, it is insufficient or incorrect), you can define exceptions to it that should be used when reporting. To define an IP address location exception, do the following:

  1. Select Configuration, then General, and then IP address origins. Note that this option is only available to users with IT Full level access. The currently defined IP address location exceptions are listed. Click Add to define a new exception, or click an existing one to modify it. The dialog shown in Figure 12-21 appears.

    Figure 12-21 Add IP Address Origin Dialog

    Description of Figure 12-21 follows
    Description of "Figure 12-21 Add IP Address Origin Dialog"

  2. Use the fields within the dialog to specify an exception to the predefined IP address locations. When ready, click Save.

Important

The specified IP address/netmask combination must be unique. Note that if the client IP address in the monitored traffic matches multiple IP address/netmask combinations, the one with the smallest subnet range is used for reporting purposes.

12.13.1 Uploading a List of Named Locations

Optionally, you can click Upload to merge a list of exceptions with the currently defined IP address location exceptions. The uploaded file must contain one entry per line, and information fields must be tab-separated. Note that any definition in the file for an existing exception (that is, the same IP address/netmask combination) overwrites the existing exception. In addition, each of the fields in the uploaded file must meet the requirements shown in Table 12-4.

Table 12-4 Requirements for Uploaded File

Field Requirements

IP address

Must consist of four fields separated by a period. Every field must be an integer between 0-255.

Subnet mask

Must consist of four fields separated by a period. Every field must be an integer between 0-255. Note that not all numbers between 0-255 are valid.

Country code

Must be a valid 2-character ISO 3166-1 country code (for example, "AL"). A complete list of supported country codes is available from the following location:

http://www.iso.org/iso/english_country_names_and_code_elements

Region code

This must be the code used to identify the region in the MaxMind database (http://www.maxmind.com). It is in the format countrycode-regioncode. The countrycode is always the same 2-character ISO 3166-1 code as specified for the country code. For US/Canadian IP addresses, regioncode should specify the ISO 3166-2 state/province code. For all other regions, it should be specify the FIPS 10-4 region code. For example, "US-TX" (Texas) and "CH-25" (Zurich).

City name

There are no requirements for this field other than it cannot be blank.


12.14 Controlling the Reporting of Objects as Pages

Within RUEI, forced objects are always be recorded as objects, and never as pages. This is regardless of the response time, or any errors that are reported for them. Table 12-5 shows the default file extensions used for forced objects.

Table 12-5 Default Forced Object File Extensions

Extension Extension Extension
.bmp
.class
.css

.dat

.doc
.gif
.ico
.jar
.jpeg
.jpg
.js
.mid
.mpeg
.mpg
.png
.ppt
.properties
.swf
.tif
.tiff
.xls

You can control whether objects with particular file extensions should be regarded as forced objects. Do the following:

  1. Select Configuration, then General, then Advanced settings, then URL handling, and then Forced objects. The dialog shown in Figure 12-22 appears.

    Figure 12-22 Forced Objects Dialog

    Description of Figure 12-22 follows
    Description of "Figure 12-22 Forced Objects Dialog"

  2. Specify the object file extension (without period) that should be treated as a forced object. When ready, click Add. The file extension is immediately added to the displayed list. Note that you can remove an object file extension from the list by clicking the Remove icon immediately to its right. When ready, click Save.

Note that any changes you make to the listed of defined forced object file extensions take effect immediately.



Footnote Legend

Footnote 1: Be aware that increasing the maximum group size will impact reporting performance. It is recommended that you incrementally increase the setting and verify that performance is acceptable.