This chapter describes settings to optimize the reporting of monitored traffic. These include increasing the amount of information available within the Failed Data Browser groups, increasing the default user flow limits, and obtaining user event information.
The RUEI database contains information about user events (such as when a user opens a report, consults a KPI alert log, or logs on and off). This information can be used for a wide variety of purposes, such as determining how often a particular report is opened or downloaded by users, or which is the most frequently accessed Data Browser group. In this way, you can optimize your RUEI installation to best meet the needs of your users.
The recording of user events is controlled by the user_events_enabled setting within the C_config table. When set to 1 (the default), user events are recorded; when set to 0, user events are not recorded.
By default, information about user events is held in the database for a maximum of 31 days. This is controlled by the db_max_user_events entry within the C_config table. To modify either of these settings, do the following:
Become the RUEI_USER user, and issue the following command to modify the user event retention setting:
execsql config_set_value processor db_max_user_events days
where days specifies the maximum number of days for which user event information should be stored. Note that this setting has an impact on database usage.
The C_USER_EVENTS table, shown in Table 1-1, contains user event information.
| Column | Type | Description |
|---|---|---|
|
|
NUMBER |
Unique ID used to identify the user event. |
|
|
TIMESTAMP |
Time (in UTC format) when event was performed by user. |
|
|
VARCHAR2 (255 BYTE) |
Logon name of user. |
|
|
NUMBER |
This is an event code. |
|
|
VARCHAR2 (4000 BYTE) |
Brief description of the event. |
The possible CODE events and their associated descriptions are shown in Table 1-2.
Table 1-2 C_LANG_CATALOG_DATA Table
| Code | Description |
|---|---|
|
0 |
User logon. |
|
1 |
User logout. |
|
2 |
Load/reload Dashboard tab. |
|
3 |
Added new dashboard (%1%s). |
|
4 |
Updated dashboard (%1$s). |
|
5 |
Removed dashboard (%1$s). |
|
6 |
Load/reload Report tab. |
|
7 |
View report (%1$s). |
|
8 |
Load/reload preview report (%1$s). |
|
9 |
Save report (%1$s). |
|
10 |
Save report as new (%1$s). |
|
11 |
Download report as PDF (%1$s). |
|
12 |
Download report as CSV (%1$s). |
|
13 |
Download report as TSV (%1$s). |
|
14 |
Download report as XLS (%1$s). |
|
15 |
Download report as XML (%1$s). |
|
16 |
Add report to Favorites (%1$s). |
|
17 |
Remove report from Favorites (%1$s). |
|
18 |
Toggle report %1$s mailing (%2$s). |
|
19 |
Remove report from %1$s mailing (%2$s). |
|
20 |
Send %1$s mailing now. |
|
21 |
Load/reload Browse tab. |
|
22 |
Select graph (%1$s). |
|
23 |
Select graph category (%1$s). |
|
24 |
Select group (%1$s). |
|
25 |
Load/reload diagnostics. |
|
26 |
Browse report (%1$s). |
|
27 |
Load/reload KPI overview tab (%1$s). |
|
28 |
Load/reload KPI overall alert log. |
|
29 |
Show KPI specific alert log (%1$s). |
|
30 |
Load/reload KPI correlation (%1$s). |
|
31 |
User %1$s has been added (%2$s, disabled: %3$d, locked: %4$d, admin: %5$d, sec officer: %6$d). |
|
32 |
User %1$s has been removed. |
|
33 |
Application %1$s has been added. |
|
34 |
Application %1$s has been removed. |
|
35 |
Service %1$s has been added. |
|
36 |
Service %1$s has been removed. |
|
37 |
Suite %1$s has been added. |
|
38 |
Suite %1$s has been removed. |
|
39 |
Collector profile %1$s has been added. |
|
40 |
Collector profile %1$s has been removed. |
|
41 |
Collector %1$s has been registered in profile %2$s. |
|
42 |
Collector %1$s from profile %2$s has been unregistered. |
|
43 |
Collector %1$s in profile %2$s has been restarted. |
|
44 |
Collector %1$s in profile %2$s has been disabled. |
|
45 |
Collector %1$s has been moved to profile %2$s. |
|
46 |
Traffic filter in profile %1$s has been changed to %2$s. |
|
47 |
VLAN filter in profile %1$s has been changed to %2$s. |
|
48 |
Port numbers (%1$s) in profile %2$s has been added. |
|
49 |
Port numbers (%1$s) in profile %2$s has been removed. |
|
50 |
The IP filter (%1$s) has been added in profile %2$s. |
|
51 |
The IP filter (%1$s) has been removed from profile %2$s. |
|
52 |
User account %1$s has been enabled. |
|
53 |
User account %1$s has been disabled. |
|
54 |
User account %1$s has been locked. |
|
55 |
User account %1$s has been unlocked. |
|
56 |
Maximum login attempt reached for user account %1$s. |
|
57 |
The password for user %1$s has been expired. |
|
58 |
The initial password for user %1$s has expired. |
|
59 |
The minimum password length has been changed to %1$s. |
|
60 |
The maximum password duration has been changed to %1$s days. |
|
61 |
Remove report (%1$s). |
|
62 |
URL prefix %1$s with action: %2$s has been added. |
|
63 |
URL prefix %1$s with action: %2$s has been removed. |
|
64 |
URL prefix %1$s with action: %2$s has been updated. |
|
65 |
Default replay action has been changed to %1$s. |
|
66 |
Replay IP range action has been changed to %1$s. |
|
67 |
Replay IP range %1$s has been added. |
|
68 |
Replay IP range %1$s has been removed. |
|
69 |
Replay all IP ranges have been removed. |
|
70 |
Replay IP range %1$s has been changed. |
|
71 |
Replay IP ranges uploaded. |
|
72 |
%1$s action with source value: %2$s and action: %3$s has been added. |
|
73 |
%1$s action with source value: %2$s and action: %3$s has been removed. |
|
74 |
%1$s action source value: %2$s and action: %3$s has been updated. |
|
75 |
Default action for %1$s has changed to %2$s. |
|
76 |
User account %1$s has been renamed to %2$s |
|
78 |
User account %1$s password has been changed |
|
79 |
User account %1$s has been set as administrator |
|
80 |
User account %1$s has been unset as administrator |
|
81 |
User account %1$s has been set as security officer |
|
82 |
User account %1$s has been unset as security officer |
|
83 |
The initial password duration has been changed to %1$s days |
|
84 |
The number of allowed login attempts has been changed to %1$s |
|
85 |
The SSL key (%1$s) valid from %2$s to %3$s in profile %4$s has been added |
|
86 |
The SSL key (%1$s) valid from %2$s to %3$s in profile %4$s has been removed |
|
87 |
SSL certifcate masking in profile %1$s has been changed to %2$s |
|
88 |
KPI %1$s (%2$s) has been added |
|
89 |
KPI %1$s (%2$s) has been removed |
|
90 |
KPI %1$s (%2$s) has been updated |
|
91 |
KPI category %1$s has been removed |
|
92 |
KPI category %1$s has been renamed to %2$s |
|
93 |
Naming scheme of %1$s has been updated |
|
94 |
Loading satisfaction of %1$s has been updated |
|
95 |
System account has been set to not expire |
|
96 |
System account has been set to expire |
The Failed URLs, Failed services, and Failed pages groups do not use the maximum group size setting. Instead, their size is controlled through the event_max_fail setting. This specifies the maximum number of rows that can added to the group's main database table during a 1-minute period. By default, this is 1000 rows. For the Slow URLs group, the event_max_slow setting is used, and specifies the number of the slowest URLs that are recorded within each 1-minute period. By default, this is 1000 rows.
Note that if you change the event_max_fail or the event_max_slow setting, you should also review the daily_max_fail setting. This specifies the maximum number of rows that the groups' tables can contain. This is derived from the formula 1440 * event_max_fail. The default, is 1.4 million rows.
To modify the above settings, issue the following commands:
execsql config_set_value processor event_max_fail 10000 execsql config_set_value processor daily_max_fail 4320000
Note that the event_max_fail setting is limited to a maximum of 10,000 rows.
Before starting the procedure described below, you should do the following:
Confirm that more than 1000 error pages are actually reported for a 1-minute period within the All sessions group.
Ensure that replay viewer functionality is enabled. To check this, select Configuration, then Security, and then Replay logging policy, and then click the Default replay action setting. Select the "Complete logging" option.
Before changing the default of 1000 error pages, you should consider the following:
Carefully consider whether you actually need to increase this limit. Typically, if a high number of error pages are reported within a 1-minute period, it is unlikely that they refer to different problems. Hence, having a large number of recordings for the same page errors will probably not help with root-cause analysis.
Increasing the limit imposes a considerable I/O overhead on both the Reporter and Collector systems. Therefore, you should carefully consider the limits of these systems before modifying the default limit.
Each group within the Data Browser has a maximum size. This is 1.5 times its "condense limit" (as specified by the cube_max_size option in the C_CONFIG table). The effect of trying to merge more than 5000 error pages within a 5-minute period can be that the system stops merging data at some point during the day. Obviously, the more error pages that are encountered, the sooner the Data Browser group will become full. Note you can diagnose this in the error log file (RUEI_DATA/processor/log/error.log) by searching for errors containing the string "wg_failpg_dy_*" starting with the string "no merge:".
The event_max_fail settings is used not only by the Failed pages group, but also by the Failed URLs and Failed services groups.
The default maximum number of steps that can be defined within a user flow is 15. This can be modified via the txn_max_steps setting. The default maximum number of user flows that can be defined is 200. This can be modified via the txn_max_trans setting. To change either setting, do the following:
Logon to the Reporter system as the RUEI_USER user.
Issue the following commands:
execsql config_set_value processor txn_max_steps steps execsql config_set_value processor txn_max_trans flows
where:
steps specifies the new maximum number of steps allowed with user flows.
flows specifies the new maximum number of user flows that can be defined.
Be aware that increasing either default maximum carries a performance overhead. In addition, if the maximum number of steps within user flows is significantly increased, the graphical reporting of user flows (such as the Flow status and Flow transitions) may become difficult to read.
By default, the client IP address is obtained from the IP header packet sent from the client. The IP packet contains, among other things, the numerical source and destination address of the packet. If RUEI has been placed after a NAT device (such as a load balancer), you can configure RUEI to look in a specified header (set by the NAT device) rather than the IP packet. The procedure to do this is described in Section 2 of the Monitoring NATed Traffic Appendix of the Oracle Real User Experience Insight User's Guide. However, if monitored clients are using a desktop virtualization environment (such as a Citrix server), the IP address of the server is returned as the client IP address.
The following important points need to be considered:
In desktop virtualization environments, you connect to the Internet using a browser running on the Desktop Virtualization Server (citrix for example) rather than on the client machine. RUEI sees the IP from the Virtualization server and not the real originating client IP from the user. However, RUEI provides mapping of user-id to client-ip to provide some way of reporting on the real originating client IP. You can upload this mapping, but note that this has limited functionality.
The map-ranges file contains the originating server IP ranges from which the user-id to client-ip mapping is done.
The map-users file contains the user-id to real originating client-IP. For example: A set of Citrix Servers have IP addresses in the ranges 10.0.1.2 - 10.0.1.254 (10.0.1.0/24). Citrix Clients connecting to the Citrix Server have IP-addresses for example in the range 192.168.1.2 to 192.168.1.254 (192.168.1.0/24). Users on these Citrix clients are using a web-application monitored by RUEI. In order to configure RUEI to report on the real client-ip instead of the Citrix Server IP, the following configuration is used:
RANGE 10.0.1.0/24 USER_ID\tCLIENT_IP JohnSmith\t192.168.1.10 FredWhite\t192.168.1.10 SteveBrown\t192.168.1.10
Whenever a session with a client-ip (the Citrix Server IP) within one of the ip-ranges in the RANGE file is found, RUEI will attempt to map the user-name from that session to a real-client ip (the citrix-client-pc of the user) using the USER_ID-CLIENT_IP mapping file.
So any functionality or reporting ( for example, Client Network views in the data browser ) in RUEI that depends on the client-ip will use the mapped client-ip. If no match is found in the USER_ID\tCLIENT_IP mapping file, the original client-ip retrieved from TCP/IP layer or from configured header will be used.
Important:
Any user having a client IP in the map-ranges file, but where the user id is not in the map-users file, is not mapped. Pages requested by that user are reported with IP "unknown".In order to configure RUEI to report a preferred client IP address, do the following:
Create a file containing a list of the IP address range(s) that you want to be remapped. Each range must be specified using the format 10.1.1.0/24. It is recommended that you call the file ip-map-ranges-file.tsv. For example:
RANGE 169.254.0.0/16 172.16.0.0/12
Create a tab-separated file containing a list of the required user IDs and client IP addresses. It is recommended that you call the file ip-map-users-file.tsv. For example:
USER_ID\tCLIENT_IP JohnSmith\t10.10.10.50 FredWhite\t10.10.10.51 SteveBrown\t10.10.10.52
Note that in the above example \t indicates a tab character. Ensure that both files do not contain any leading or trailing characters, and no lines containing only whitespace or special characters (such as /n or /r).
Logon to the RUEI Reporter system as the RUEI_USER user.
Import the two created files onto a suitable location on the RUEI Reporter system.
Execute the import-ip-map script (located in the RUEI_DATA/processor/bin directory) using the following command:
import-ip-map -r ip-map-ranges-file -u ip-map-users-file
where ip-map-ranges-file and ip-map-users-file are the two files created and imported above.
Any reporting changes made by this facility take effect within appropriately 5 minutes.
Restoring Default functionality
To restore default client IP address reporting, create two files containing only column headers and repeat the above procedure.
By default, a visitor session is regarded as terminated if the visitor has been inactive for longer than 60 minutes. This is controlled through the session_idle_time setting. In addition, the default number of hours that user IDs and custom dimensions are remembered for a session is 12 hours. This is controlled through the max_age_session setting.
Lowering the session_idle_time setting will increase Reporter system performance in terms of CPU utilization. It has no impact on memory usage. However, be aware that a drawback of lowering this setting is that identified visitors returning within the specified session idle time will be reported as anonymous.
You should consider lowering the max_age_session setting when the Reporter system does not have enough memory and starts to swap. Be aware that when this setting is lowered, and the monitored traffic contains mostly long sessions, user IDs can be lost. This setting should not be set lower than the session_idle_time setting.
Use the following commands to obtain a setting's current value:
execsql config_get_value processor session_idle_time execsql config_get_value processor max_age_session
Use the following commands to modify a setting's value:
execsql config_set_value processor session_idle_time idle_time execsql config_set_value processor max_age_session max_age
where
idle_time specifies the number of seconds of visitor inactivity after which the session is considered terminated.
max_age specifies the maximum number of hours after which session information is cleared from memory.
By default, 3 threads are used on the Reporter system for traffic processing. It is controlled by the lookup_threads setting. Performance improvement can be obtained (through additional concurrency in processing) by increasing this setting. An indication that this setting is too low is the following internal error appearing in the Event log:
Processing backlog larger than %d minutes, restarting logr (the backlog will be skipped).
It means that the Reporter system cannot keep up with the processing of the arriving data.
Use the following command to obtain the setting's current value:
execsql config_get_value processor lookup_threads
Use the following command to modify the setting's value:
execsql config_set_value processor lookup_threads threads
where threads specifies the number of threads available for use by the Reporter system. This setting should not be higher than the number of cores available on the Reporter system.
Note that a separate setting is available to control the performance of the Reporter user interface, and is described in Section 3.3, "Improving GUI Performance".