Oracle Real User Experience Insight Administrator's Guide, 12c Release 2 (12.1.0.3) for Linux x86-64

12c Release 2 (12.1.0.3)

E37267-01

Contents

Title and Copyright Information

Preface

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

1 Controlling Reporting

- 1.1 Obtaining User Event Information

- 1.2 Increasing the Size of the Failed Groups

- 1.3 Increasing the Default Limits for User Flows

- 1.4 Obtaining Client IP Addresses within Desktop Virtualization Environments

- 1.5 Controlling the Maximum Session Duration and Idle Time

- 1.6 Improving Processing Concurrency

2 Configuring Collector Systems

- 2.1 Increasing Memory Availability to Collectors

- 2.2 Configuring Domain-Based Segmentation

- 2.3 Configuring the Number of NPA Threads

3 Maintaining the System

- 3.1 Disabling Modification to Administrators' Properties

- 3.2 Increasing the Linux Socket Memory Allocation Limit

- 3.3 Improving GUI Performance

- 3.4 Backing up a RUEI Deployment

- 3.4.1 Backing up RUEI Configuration Data

- 3.4.2 Backing up Session Diagnostic Data

- 3.4.3 Restoring a RUEI Deployment Backup

- 3.5 Moving RUEI Datafiles to a New Location

4 Managing the Database

- 4.1 Viewing the Status of RUEI Database Tables

- 4.2 Suspending Processing When Performing Database Maintenance

- 4.3 Enabling Online Tablespace Backups

- 4.4 Using Redo Logging

- 4.5 Improving Database Performance

- 4.6 Configuring Statistics and Facts Tables

5 Troubleshooting

- 5.1 Enabling Core Dumps for Collector Processes

- 5.2 Manually Creating Helpdesk Reports

A Third-Party Licenses

Oracle® Real User Experience Insight

Administrator's Guide

12c Release 2 (12.1.0.3) for Linux x86-64

E37267-01

September 2012

Oracle Real User Experience Insight Administrator's Guide, 12c Release 2 (12.1.0.3) for Linux x86-64

E37267-01

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Primary Author: Paul Coghlan

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

This software and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Preface

Oracle Real User Experience Insight (RUEI) provides you with powerful analysis of your network and business application infrastructure. You can monitor the real-user experience, set Key Performance Indicators (KPIs) and Service Level Agreements (SLAs), and trigger alert notifications for incidents that violate them.

Audience

This guide describes the Oracle Real User Experience Insight (RUEI) command-line interface tools and commands available for performing advanced system administration. It is intended for Administrators responsible for maintaining your RUEI deployment(s).

Using This Guide

This guide is organized as follows:

- Chapter 1, "Controlling Reporting" describes settings to optimize the reporting of monitored traffic. These include increasing the amount of information available within the Failed Data Browser groups, increasing the default user flow limits, and obtaining user event information.

- Chapter 2, "Configuring Collector Systems" describes settings to configure your Collector systems to perform domain-based segmentation, and increase the memory available to Collector processes.

- Chapter 3, "Maintaining the System" describes settings to perform various maintenance tasks, such as backing up a RUEI deployment, and improving Reporter GUI performance.

- Chapter 4, "Managing the Database" describes a number of steps necessary to perform database maintenance and facilitate backups.

- Chapter 5, "Troubleshooting" describes settings for helping Customer Support to resolve problems encountered when using RUEI.

More information

- Information on Oracle Enterprise Manager is available at the following location:

http://www.oracle.com/us/products/enterprise-manager/index.html

- Detailed technical information is available from My Oracle Support:

https://support.oracle.com

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Related Documents

For more information, see the following documents in the Oracle Real User Experience Insight (RUEI) documentation set:

- Oracle Real User Experience Insight User's Guide.

- Oracle Real User Experience Insight Installation Guide.

- Oracle Real User Experience Insight Release Notes.

The latest version of this and other RUEI books can be found at the following location:

http://www.oracle.com/technetwork/documentation/realuserei-091455.html

RUEI also provides extensive online help. Select the option Help option from the System menu, or click the Help icon within a dialog to display the online help system.

Conventions

The following text conventions are used in this document:

Convention

Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.

italic

Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.

monospace

Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.

1 Controlling Reporting

This chapter describes settings to optimize the reporting of monitored traffic. These include increasing the amount of information available within the Failed Data Browser groups, increasing the default user flow limits, and obtaining user event information.

1.1 Obtaining User Event Information

The RUEI database contains information about user events (such as when a user opens a report, consults a KPI alert log, or logs on and off). This information can be used for a wide variety of purposes, such as determining how often a particular report is opened or downloaded by users, or which is the most frequently accessed Data Browser group. In this way, you can optimize your RUEI installation to best meet the needs of your users.

The recording of user events is controlled by the user_events_enabled setting within the uxs_config table. When set to 1 (the default), user events are recorded; when set to 0, user events are not recorded.

By default, information about user events is held in the database for a maximum of 31 days. This is controlled by the db_max_user_events entry within the uxs_config table. To modify either of these settings, do the following:

Become the RUEI_USER user, and issue the following command to modify the user event retention setting:

execsql config_set_value processor db_max_user_events days

where days specifies the maximum number of days for which user event information should be stored. Note that this setting has an impact on database usage.

User Event Table Structure

The UXS_USER_EVENTS table, shown in Table 1-1, contains user event information.

Table 1-1 UXS_EVENTS Table

Column

Type

Description

ID

NUMBER

Unique ID used to identify the user event.

STAMP

TIMESTAMP

Time (in UTC format) when event was performed by user.

USERNAME

VARCHAR2 (255 BYTE)

Logon name of user.

CODE

NUMBER

This is an event code.

EVENT

VARCHAR2 (4000 BYTE)

Brief description of the event.

Event Codes and Descriptions

The possible CODE events and their associated descriptions are shown in Table 1-2.

Table 1-2 UXS_LANG_CATALOG_DATA Table

Code

Description

0

User logon.

1

User logout.

2

Load/reload Dashboard tab.

3

Added new dashboard (%1%s).

4

Updated dashboard (%1$s).

5

Removed dashboard (%1$s).

6

Load/reload Report tab.

7

View report (%1$s).

8

Load/reload preview report (%1$s).

9

Save report (%1$s).

10

Save report as new (%1$s).

11

Download report as PDF (%1$s).

12

Download report as CSV (%1$s).

13

Download report as TSV (%1$s).

14

Download report as XLS (%1$s).

15

Download report as XML (%1$s).

16

Add report to Favorites (%1$s).

17

Remove report from Favorites (%1$s).

18

Toggle report %1$s mailing (%2$s).

19

Remove report from %1$s mailing (%2$s).

20

Send %1$s mailing now.

21

Load/reload Browse tab.

22

Select graph (%1$s).

23

Select graph category (%1$s).

24

Select group (%1$s).

25

Load/reload diagnostics.

26

Browse report (%1$s).

27

Load/reload KPI overview tab (%1$s).

28

Load/reload KPI overall alert log.

29

Show KPI specific alert log (%1$s).

30

Load/reload KPI correlation (%1$s).

31

User %1$s has been added.

32

User %1$s has been removed.

33

Application %1$s has been added.

34

Application %1$s has been removed.

35

Service %1$s has been added.

36

Service %1$s has been removed.

37

Suite %1$s has been added.

38

Suite %1$s has been removed.

39

Collector profile %1$s has been added.

40

Collector profile %1$s has been removed.

41

Collector %1$s has been registered in profile %2$s.

42

Collector %1$s from profile %2$s has been unregistered.

43

Collector %1$s in profile %2$s has been restarted.

44

Collector %1$s in profile %2$s has been disabled.

45

Collector %1$s has been moved to profile %2$s.

46

Traffic filter in profile %1$s has been changed to %2$s.

47

VLAN filter in profile %1$s has been changed to %2$s.

48

Port numbers (%1$s) in profile %2$s has been added.

49

Port numbers (%1$s) in profile %2$s has been removed.

50

The IP filter (%1$s) has been added in profile %2$s.

51

The IP filter (%1$s) has been removed from profile %2$s.

52

User account %1$s has been enabled.

53

User account %1$s has been disabled.

54

User account %1$s has been locked.

55

User account %1$s has been unlocked.

56

Maximum login attempt reached for user account %1$s.

57

The password for user %1$s has been expired.

58

The initial password for user %1$s has expired.

59

The minimum password length has been changed to %1$s.

60

The maximum password duration has been changed to %1$s.

61

Remove report (%1$s).

62

URL prefix %1$s with action: %2$s has been added.

63

URL prefix %1$s with action: %2$s has been removed.

64

URL prefix %1$s with action: %2$s has been updated.

65

Default replay action has been changed to %1$s.

66

Replay IP range action has been changed to %1$s.

67

Replay IP range %1$s has been added.

68

Replay IP range %1$s has been removed.

69

Replay all IP ranges have been removed.

70

Replay IP range %1$s has been changed.

71

Replay IP ranges uploaded.

72

%1$s action with source value: %2$s and action: %3$s has been added.

73

%1$s action with source value: %2$s and action: %3$s has been removed.

74

%1$s action source value: %2$s and action: %3$s has been updated.

75

Default action for %1$s has changed to %2$s.

1.2 Increasing the Size of the Failed Groups

As explained in section 12.10 of the Oracle Real User Experience Insight User's Guide, the Failed URLs, Failed services, and Failed pages groups do not use the maximum group size setting. Instead, their size is controlled through the event_max_fail setting. This specifies the maximum number of rows that can added to the group's main database table during a 1-minute period. By default, this is 1000 rows. For the Slow URLs group, the event_max_slow setting is used, and specifies the number of the slowest URLs that are recorded within each 1-minute period. By default, this is 1000 rows.

Note that if you change the event_max_fail or the event_max_slow setting, you should also review the daily_max_fail setting. This specifies the maximum number of rows that the groups' tables can contain. This is derived from the formula 1440 * event_max_fail. The default, is 1.4 million rows.

To modify the above settings, issue the following commands:

execsql config_set_value processor event_max_fail 10000
execsql config_set_value processor daily_max_fail 4320000

Note that the event_max_fail setting is limited to a maximum of 10,000 rows.

Before starting the procedure described below, you should do the following:

- Confirm that more than 1000 error pages are actually reported for a 1-minute period within the All sessions group.

- Ensure that replay viewer functionality is enabled. To check this, select Configuration, then Security, and then Replay logging policy, and then click the Default replay action setting. Select the "Complete logging" option.

Important

Before changing the default of 1000 error pages, you should consider the following:

- Carefully consider whether you actually need to increase this limit. Typically, if a high number of error pages are reported within a 1-minute period, it is unlikely that they refer to different problems. Hence, having a large number of recordings for the same page errors will probably not help with root-cause analysis.

- Increasing the limit imposes a considerable I/O overhead on both the Reporter and Collector systems. Therefore, you should carefully consider the limits of these systems before modifying the default limit.

- Each group within the Data Browser has a maximum size. This is 1.5 times its "condense limit" (as specified by the cube_max_size option in the UXS_CONFIG table). The effect of trying to merge more than 5000 error pages within a 5-minute period can be that the system stops merging data at some point during the day. Obviously, the more error pages that are encountered, the sooner the Data Browser group will become full. Note you can diagnose this in the error log file (RUEI_DATA/processor/log/error.log) by searching for errors containing the string "wg_failpg_dy_*" starting with the string "no merge:".

- The event_max_fail settings is used not only by the Failed pages group, but also by the Failed URLs and Failed services groups.

1.3 Increasing the Default Limits for User Flows

The default maximum number of steps that can be defined within a user flow is 15. This can be modified via the txn_max_steps setting. The default maximum number of user flows that can be defined is 200. This can be modified via the txn_max_trans setting. To change either setting, do the following:

1. Logon to the Reporter system as the RUEI_USER user.

2. Issue the following commands:

execsql config_set_value processor txn_max_steps steps
execsql config_set_value processor txn_max_trans flows

where:

- steps specifies the new maximum number of steps allowed with user flows.

- flows specifies the new maximum number of user flows that can be defined.

Important

Be aware that increasing either default maximum carries a performance overhead. In addition, if the maximum number of steps within user flows is significantly increased, the graphical reporting of user flows (such as the Flow status and Flow transitions) may become difficult to read.

1.4 Obtaining Client IP Addresses within Desktop Virtualization Environments

By default, the client IP address is obtained from the IP header packet sent from the client. The IP packet contains, among other things, the numerical source and destination address of the packet. If RUEI has been placed after a NAT device (such as a load balancer), you can configure RUEI to look in a specified header (set by the NAT device) rather than the IP packet. The procedure to do this is described in section P.2 of the Oracle Real User Experience Insight User's Guide. However, if monitored clients are using a desktop virtualization environment (such as a Citrix server), the IP address of the server is returned as the client IP address.

Important:
The following procedure requires you to have a correct user-client IP address mapping.

In order to configure RUEI to report a preferred client IP address, do the following:

1. Create a file containing a list of the IP address range(s) that you want to be remapped. Each range must be specified using the format 10.1.1.0/24. It is recommended that you call the file ip-map-ranges-file.tsv. For example:

RANGE
169.254.0.0/16
172.16.0.0/12

2. Create a tab-separated file containing a list of the required user IDs and client IP addresses. It is recommended that you call the file ip-map-users-file.tsv. For example:

USER_ID\tCLIENT_IP
JohnSmith\t10.10.10.50
FredWhite\t10.10.10.51
SteveBrown\t10.10.10.52

Note that in the above example \t indicates a tab character. Ensure that both files do not contain any leading or trailing characters, and no lines containing only whitespace or special characters (such as /n or /r).

3. Logon to the RUEI Reporter system as the RUEI_USER user.

4. Import the two created files onto a suitable location on the RUEI Reporter system.

5. Execute the import-ip-map script (located in the RUEI_DATA/processor/bin directory) using the following command:

import-ip-map -r ip-map-ranges-file -u ip-map-users-file

where ip-map-ranges-file and ip-map-users-file are the two files created and imported above.

Any reporting changes made by this facility take effect within appropriately 5 minutes.

Restoring Default functionality

To restore default client IP address reporting, create two files containing only column headers and repeat the above procedure.

1.5 Controlling the Maximum Session Duration and Idle Time

By default, a visitor session is regarded as terminated if the visitor has been inactive for longer than 60 minutes. This is controlled through the session_idle_time setting. In addition, the default number of hours that user IDs and custom dimensions are remembered for a session is 12 hours. This is controlled through the max_age_session setting.

Lowering the session_idle_time setting will increase Reporter system performance in terms of CPU utilization. It has no impact on memory usage. However, be aware that a drawback of lowering this setting is that identified visitors returning within the specified session idle time will be reported as anonymous.

You should consider lowering the max_age_session setting when the Reporter system does not have enough memory and starts to swap. Be aware that when this setting is lowered, and the monitored traffic contains mostly long sessions, user IDs can be lost. This setting should not be set lower than the session_idle_time setting.

Use the following commands to obtain a setting's current value:

execsql config_get_value processor session_idle_time
execsql config_get_value processor max_age_session

Use the following commands to modify a setting's value:

execsql config_set_value processor session_idle_time idle_time
execsql config_set_value processor max_age_session max_age

where

- idle_time specifies the number of seconds of visitor inactivity after which the session is considered terminated.

- max_age specifies the maximum number of hours after which session information is cleared from memory.

1.6 Improving Processing Concurrency

By default, 3 threads are used on the Reporter system for traffic processing. It is controlled by the lookup_threads setting. Performance improvement can be obtained (through additional concurrency in processing) by increasing this setting. An indication that this setting is too low is the following internal error appearing in the Event log:

Processing backlog larger than %d minutes, restarting logr (the backlog will be skipped).

It means that the Reporter system cannot keep up with the processing of the arriving data.

Use the following command to obtain the setting's current value:

execsql config_get_value processor lookup_threads

Use the following command to modify the setting's value:

execsql config_set_value processor lookup_threads threads

where threads specifies the number of threads available for use by the Reporter system. This setting should not be higher than the number of cores available on the Reporter system.

2 Configuring Collector Systems

This chapter describes settings to configure your Collector systems to perform domain-based segmentation, and increase the memory available to Collector processes.

2.1 Increasing Memory Availability to Collectors

By default, the Collector process (panther) is assigned 30% of available system memory within a single-server installation. Within a remote Collector installation, the Collector process is assigned 70% of available memory. To set the memory available to the Collector process, use the following command:

execsql config_set_profile_value wg profile config MaxMemoryUsage replace setting

where:

- profile specifies the name of the Collector profile that needs to be updated.

- setting is the percentage of system memory available to the Collector process. Note that percentage sign must not be specified with the setting. It is recommended that you specify a percentage not higher than 90%. If the Collector process has to share resources with other software running on the system, a maximum setting of 80% is more appropriate.

Collector Profile Name

Note that the required Collector profile name can either be obtained via the Reporter GUI (select Configuration, then Security, and then Collector profiles), or by executing the following command:

execsql config_get_profiles wg

2.2 Configuring Domain-Based Segmentation

To configure RUEI to filter (segment) monitored traffic based on domain names, do the following:

1. Select Configuration, then Security, then Network filters, and select the required Collector profile. Ensure that the Packet capture menu specifies the "Specified domains" option for each required Collector profile.

2. Create, modify, or delete the required rows in the wg__domain_segments database table. The table has the following format:

ID        Priority   Domain   Profile_ID   Traffic_segment
1000      10         *.nl          2            1|1
1100      8          *.be          2            1|2
1150      3          *.oracle.*    2            1|1
1200      1          *.com         2            3|4

where:

- The ID column represents a unique identifier for each row in the table.

- The Priority column represents the order in which the filters are applied. The filters with the highest priority numbers are applied first, and those with the lowest are resolved last. Hence, in the above example, monitored traffic relating to the domain myshop.oracle.com would be filtered as *.oracle.* 1|1, and not the *.com 3|4 filters. Also, all domain traffic with the country code nl is monitored, while only the first half of the data stream should be monitored for domains with the country code be.

- The Domain column contains the actual filter value where * can be used as a wildcard.

- The Profile_ID column relates to the ID of the Collector profile for which the filters should apply. This ID can be found in wg__cprofiles.

- The Traffic_segment column contains the segment which should be used for the specified filter. You can specify up to 128 parts. For example, 34|128 will take the 34th segment out of 128.

3. To view the currently defined network filters, logon to the Reporter system as the RUEI_USER user, and issue the following command:

sqlplus /@RUEI_DB_TNSNAME
select id, prio, domain, profile_id, traffic_segment from wg__domain_segments order by prio;

4. To insert a row into the table, issue the following command:

insert into wg__domain_segments (id, prio, domain, profile_id, traffic_segment) values (wg__domain_segments_seq.nextval, 1, '*.nl', 2, '1|2');

5. To delete a row from the table, issue the following command:

delete from wg__domain_segments where id=1;

6. To alter a filter's priority, issue the following command:

update wg__domain_segments set prio=100 where id=2;

2.3 Configuring the Number of NPA Threads

By default, a Collector system use three threads for Network Protocol Analysis (NPA). This is controlled by the NumNPAThreads setting. Collector performance can be improved by increasing this setting.

If not currently defined, this number can be increased by using the following command:

execsql config_set_profile_value wg profile config NumNPAThreads add 4

If previously defined, it can be modified by using the following command:

execsql config_set_profile_value wg profile config NumNPAThreads replace 5

where profile specifies the name of the Collector profile that needs to be updated.

3 Maintaining the System

This chapter describes settings to perform various maintenance tasks, such as backing up a RUEI deployment, and improving Reporter GUI performance.

3.1 Disabling Modification to Administrators' Properties

By default, users with Administrator permissions can change the properties of other Administrators, as well as create and delete Administrator user accounts. If this is not consistent with your security requirements, you can disable this functionality by issuing the following commands:

execsql config_set_value wi_core user_mgmt_admin_edit_admins 0

3.2 Increasing the Linux Socket Memory Allocation Limit

The underlying Linux socket interface used by the Collector for monitoring traffic has a memory allocation limit of 20KB. This limit can be exceeded when a large number of network filters (or VLAN definitions) are configured. If so, the following error is reported in the Event log:

linux.c, 326,cap_dev_set_filter()]: setsockopt(): Cannot allocate memory

In order to increase this limit, do the following:

1. Logon to the required Collector system as the root user.

2. Issue the following command to increase the underlying limit:

/sbin/sysctl -w net.core.optmem_max=65535

3. To make this setting persistent across reboots, add the following line to the /etc/sysctl.conf file:

net.core.optmem_max=65535

3.3 Improving GUI Performance

Within the Reporter user interface, the performance of queries (such as refreshing a dashboard or retrieving data within the Data Browser) is heavily influenced by the specified Degree of Parallelism (DOP) setting. This regulates the maximum number of parallel queries that may be made to the database. By default, this is two. In the case of deployments where the Reporter system has substantially more cores than this default, or where a dedicated database server is being used, a considerable user interface performance improvement can be realized by increasing the DOP setting.

The DOP is controlled by the db_gui_dop entry within the uxs_config table. Upon installation, this entry does not exist in the database. Do the following:

1. Logon to the Reporter system, become the RUEI_USER user, and issue the following command:

# sqlplus /@uxinsight

2. Issue the following command to obtain the setting's current value:

execsql config_get_value wi_core db_gui_dop

Use the following command to change the setting's value:

execsql config_set_value wi_core db_gui_dop dop

where dop specifies the degree of parallelism used for queries within the Reporter interface. Note that this should be less than the number of cores within the database system.

3.4 Backing up a RUEI Deployment

RUEI does not provide dedicated database backup and recovery functionality. Instead, it relies on standard Oracle database functionality. This is described in the Oracle Database Backup and Recovery User's Guide, available at the following location:

http://docs.oracle.com/cd/B28359_01/backup.111/b28270/toc.htm

Important

Regardless of the backup method you use, it is strongly recommended that you first stop RUEI data processing. Unless you do so, the integrity of the backed up data cannot be guaranteed. To do so, issue the following command as the RUEI_USER user:

project -stop wg

Be advised that this procedure may take several minutes, and any data being processed at the time of the stop command will be lost. However, traffic monitoring continues, and is written to log files that will be committed to the database once processing is resumed.

After backup creation, processing can be restarted with the following command:

project -start wg

3.4.1 Backing up RUEI Configuration Data

In addition to the database, RUEI configuration data should also be backed up. The procedure described below extracts configuration data from both the database as well as the file system, and writes it to the file system where it can be picked up for further backup to a suitable storage device.

1. Logon to the Reporter system as the RUEI_USER user, and issue the following command:

project -save --all

By default, this stores backup data to the RUEI_DATA/processor/backup. An alternate location can be specified using the –file directive. For example, to store to the location /tmp/backup, use the following command:

project -save --file /tmp/backup --all

2. To restore an earlier backup, issue the following command:

project -restore --file /tmp/backup --all

3.4.2 Backing up Session Diagnostic Data

One of the major strengths of RUEI is its ability to diagnose individual user sessions for slow performance or problem pages. This functionality relies on log files that are stored outside of the RUEI database. In order to allow access to Session Diagnostics functionality, this data also needs to be available during a restore. Backup the contents of the RUEI_DATA/processor/data directory.

Replay content is the data required to replay error pages or the full content of a session. Backup of this data depends on your requirements. That is, if there is a need to replay session content on a regular basis. Replay content can be easily backed up from the file system. The relevant directories are $APPSENSOR_HOME/*/REPLAY. The default location is RUEI_DATA/collector/wg/REPLAY. Note that the entire directory (and all sub-directories) should be backed up.

Note that the directories indicated above must be backed for each required Collector system. In a distributed environment, that means that the backup may have to be performed on multiple systems.

3.4.3 Restoring a RUEI Deployment Backup

To restore a RUEI deployment from scratch, do the following:

1. Install the RUEI software. The procedure do this is fully described in the Oracle Real User Experience Insight Installation Guide.

2. Restore the database content following the instructions in Oracle Database Backup and Recovery User's Guide for the selected backup approach.

3. Restore the RUEI configuration information using the following command:

project -restore --all backup-file-location

where backup-file-location specifies the location of the backed-up data.

4. Restore the RUEI Session Diagnostics information by restoring the contents of the RUEI_DATA/processor/data directory.

5. For each required Collector system, restore the replay content to the location $APPSENSOR_HOME/*/REPLAY. Note that the Collector must be stopped before performing a restore. To stop the Collector, issue the following command as the RUEI_DATA user:

appsensor stop wg

To restart the Collector, issue the following command as the RUEI_USER user:

appsensor start wg

3.5 Moving RUEI Datafiles to a New Location

You may need to move the database datafiles to a new location. For example, because the current mount point or directory is running out of space. Note that the following procedure assumes that the database is running on the Reporter system, and the default installation paths are being used. This is fully described in the Oracle Real User Experience Insight Installation Guide.

Do the following:

1. Logon to the Reporter system as the RUEI_USER user.

2. Stop the database and processing by issuing the following commands:

project -stop wgexit/etc/init.d/oracledb stop

3. Prepare the new mount using the following commands:

mkdir -p /oradata/ux/
chown oracle:oinstall -R /oradata

4. Copy the datafiles as the oracle user by issuing the following commands:

cd /u01/app/oracle/oradata
mv ux/* /oradata/ux
rm -f ux
ln -s /oradata/ux ux

5. Restart the database and processing by issuing the following commands:

# /etc/init.d/oracledb start
# su - RUEI_USER$
project -start wg

4 Managing the Database

This chapter describes a number of settings necessary to perform database maintenance and facilitate backups.

4.1 Viewing the Status of RUEI Database Tables

In the event of a database crash, objects may become corrupted. Typically, this reveals itself with ORA-00376 and similar errors reported in the Event Log. It is recommended that you carefully review the information in the Knowledge Base article at the following location:

https://support.us.oracle.com/oip/faces/secure/km/DocumentDisplay.jspx?id=1303180.1

In particular, ensure that the indicated tablespaces are set to force logging. You can use the following command to view the status of the database tables:

cop stats %period

where period indicates the required year (2012), month (201203), or day (20120326). The command output appears as follows:

STRUCTURE      PRESENTATION   DATA ROWS         DATA SIZE hash  data dims lvls  pres view   data    desc      data     desc   cube name------ ---- ---- ----  ---- ----  ------  ------   -------  -------  ----------fTq7vQ  19   11   22   133  156        0       2    0.1 MB   0.1 MB  wg_keypage_mo_201203u7q+3g   9    4    8    13    7        -     470    0.6 MB   0.1 MB  wg_kpi_mo_201203PMocAw  22    9   17   159  174        -   16960   19.0 MB   4.0 MB  wg_page_mo_201203K/p4ww  12   12   29   123  104        0       0    0.1 MB   0.1 MB  wg_service_mo_2012031S2Ggg  10   19   29    79   90        -     247    2.0 MB   0.1 MB  wg_slowurl_mo_201203lZRuxg  29    5   10   279   61        0       0    0.1 MB   0.1 MB  wg_trasta_mo_201203yuY0aQ  29   11   20   153  204        -     343    2.0 MB   0.1 MB  wg_visit_mo_201203

Note that if the Data column contains a zero value, or there a large number of zeros or dashes, this would indicate corrupted database tables. In this case, you should use the script described in the following Knowledge Base article to restore the database:

https://support.us.oracle.com/oip/faces/secure/km/DocumentDisplay.jspx?id=556733.1

In addition, it is recommended that you issue the following commands to force an update of the RUEI configuration and template tables:

makedatabase @
modr -fn all

4.2 Suspending Processing When Performing Database Maintenance

When performing maintenance on the database, it is recommended that you manually stop RUEI processing for the time that the database is down to prevent the reporting of error messages to show. Do the following:

1. Use SSH to logon to the Reporter system as the RUEI_USER user.

2. Issue the following command to stop processing:

project -stop wg

3. Ensure that the following processes are no longer running before bringing down the database: qjob, logr, and rsynclogdird. If necessary, use the kill command to stop them.

4. After completion of database maintenance, restart processing by issuing the following command:

project -start wg

4.3 Enabling Online Tablespace Backups

As of version 12.1.0.3, the USERS and UXCONF tablespaces within new installations are set to force logging mode. Previously, the default mode was nologging. The upgrade procedure does not change your database's current setting. However, be aware that changing the tablespace mode to force logging can considerably increase disk I/O.

By default, the database does not support online backups. In order to do so, the database's noarchivelog mode needs to be changed, and a number of operations changed from nologging mode to force logging mode. Do the following:

1. Logon to the database system as the oracle user:

2. Stop all processing by issuing the following commands:

source /etc/ruei.conf
su - $RUEI_USER
project -stop wg
killall logmsgd
killall qjobd
killall rsynclogdird

3. Ensure that the $RUEI_DB_INST setting specifies the RUEI database.

4. Change the database to archivelog mode by issuing the following commands:

. oraenv
sqlplus / as sysdba
shutdown immediate
startup mount
alter database archivelog;
alter database open;

5. Issue the following commands to set the required operations to force logging mode:

alter tablespace USERS force logging;
alter tablespace UXCONF force logging;

6. Configure and schedule the online backup.

See the Oracle Backup and Recovery User's Guide for further information. It is available at the following location:

http://www.oracle.com/pls/db112/portal.portal_db?selected=14

4.4 Using Redo Logging

By default, redo logging of the RUEI database is disabled. If this is enabled for the complete database, very large redo log archives can be created. Therefore, if you want to use redo logging as part of your backup strategy, you need to make a number of configuration changes. Do the following:

1. Logon to the database system as the oracle user.

2. Issue the following commands to set the required logging options in the RUEI database table spaces:

sqlplus / as sysdba
SQL> alter tablespace USERS force logging;
SQL> alter tablespace UXCONF force logging;
SQL> alter tablespace UXSTAT no force logging;
SQL> alter tablespace UXTEMP no force logging;

Note that the UXSTAT and UXTEMP tablespaces are not set to force logging because they are not relevant to the backup and restore process because they only contain intermediate data.

4.5 Improving Database Performance

By default, the degree of parallelism used for core-related queries in the database is 1. This is controlled by the db_core_dop setting. Increasing the number available can improve database performance. However, this setting should never be set to a number higher than the amount of cores available from the database server. This setting utilizes the DOP features of the Oracle database. It has no functional impact other than potentially making data processing run faster.

Use the following command to obtain the setting's current value:

execsql config_get_value processor db_core_dop

Use the following command to modify the setting's value:

execsql config_set_value processor db_core_dop dop

where dop specifies the degree of parallelism used for queries in processing.

4.6 Configuring Statistics and Facts Tables

Additional statistics and facts database tables are automatically created to facilitate RUEI integration with Oracle Enterprise Manager. These tables can significantly increase the level of database I/O activity and storage requirements. The creation of these tables is controlled by the appliance_fact_enabled and appliance_statistics_enabled setting:

Use the following commands to obtain the settings' current values:

execsql config_get_value appliance fact_enabled
execsql config_get_value appliance_statistics_enabled

Use the following commands to modify the settings' value:

execsql config_set_value appliance fact_enabled value
execsql config_set_value appliance statistics_enabled value

where value specifies whether the setting should be enabled (1) or disabled (0).

Important

Be aware that disabling either of the above settings will prevent the correct operation of an Oracle Enterprise Manager deployment with a RUEI integration.

5 Troubleshooting

This chapter describes settings for helping Customer Support to resolve problems encountered when using RUEI.

5.1 Enabling Core Dumps for Collector Processes

By default, in the event of a Collector instance crashing, no core dump is generated. This is for security reasons because the Collector may be monitoring encrypted (SSL) traffic. However, some customer issues can only be resolved by Customer Support if a core dump is made available. In order to ensure the creation of core dumps, do the following:

1. Issue the following command as the RUEI_DATA user on the system on which the Collector instance is running:

ulimit -c unlimited

2. Edit the APPSENSOR_HOME/wg/config/config.cfg file, and modify the value of CoreSize setting to -1.

3. Restart the Collector by issuing the following command as the RUEI_DATA user:

appsensor restart wg

When core dumps are enabled, stack trace extracts are stored in the APPSENSOR_HOME/core_dir directory. Note that RUEI automatically cleans up any core dumps in the APPSENSOR_HOME directory every night at 2:30 AM. In addition, be aware that if core dumps are regularly generated, the file system may start filling up. Therefore, it is recommended that the default configuration is restored as soon as the required core dumps have been harvested.

5.2 Manually Creating Helpdesk Reports

When contacting Customer Support, it is strongly recommended that a Helpdesk report file is created and uploaded to the Service Request (SR). This file contains extended system information that is extremely useful to Customer Support when handling any issues that are reported. This file can be created by selecting System, then Maintenance, and then Helpdesk report.

If the Reporter user interface, the Helpdesk report can be created manually by doing the following:

1. Logon to the Reporter system as the RUEI_USER user.

2. Issue the following commands:

source /etc/ruei.conf
project -save --all

3. Fetch the generated .tgz file from the location as indicated by the command output.

4. Upload the file to the appropriate SR.

A Third-Party Licenses

This appendix contains licensing information about certain third-party products included with this version of RUEI. Unless otherwise specifically noted, all licenses herein are provided for notice purposes only.

The sections in this appendix describe the following third-party licenses:

- Apache Software License, Version 2.0

- OpenSSL

- PHP

- Java Runtime Environment

- The MIT License (MIT)

Apache Software License, Version 2.0

Apache License

Version 2.0, January 2004

http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.

"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.

"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.

"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.

"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.

"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).

"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.

"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."

"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.

2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.

4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:

- You must give any other recipients of the Work or Derivative Works a copy of this License; and

- You must cause any modified files to carry prominent notices stating that You changed the files; and

- You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and

- If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.

5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.

6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.

7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.

8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.

9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS

APPENDIX: How to apply the Apache License to your work.

To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.

Copyright [yyyy] [name of copyright owner]

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

OpenSSL

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org).

Copyright © 1998-2007 The OpenSSL Project. All rights reserved.

Copyright © 1995-1998 Eric Young (eay@cryptsoft.com). All rights reserved.

THIS SOFTWARE IS PROVIDED BY THE OPENSSL PROJECT "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OPENSSL PROJECT BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

PHP

Copyright © 1999-2006 The PHP Group. All rights reserved.

This product includes PHP software, freely available from http://php.net/software/.

"THIS SOFTWARE IS PROVIDED BY THE PHP DEVELOPMENT TEAM "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE PHP DEVELOPMENT TEAM BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."

Java Runtime Environment

Oracle is required to provide the following notice as part of the license terms for the Sun JRE that the field of use for the JRE is: (i) general purpose desktop computers, laptops and servers, and (ii) embedded systems (by way of example: embedded applications, cell phones, PDAs, TV devices, digital set top boxes, telematics devices and home gateway devices), provided that the Java Runtime Environment (JRE) is licensed only to run Licensee applications, middleware and database products and the JRE is not licensed to directly run any third party applications. This shall not be understood to prevent third party applications from indirectly and incidentally utilizing the JRE, but only as such is required to enable other Licensee Product functionality.

After installing the JRE, the complete terms of the Sun Microsystems license agreement are available in the file /usr/java/jre1.5.0_18/THIRDPARTYLICENSEREADME.txt.

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Copyright © 2012, Oracle and/or its affiliates. All rights reserved.