1 Controlling Reporting
Obtaining User Event Information
The RUEI database contains information about user events (such as when a user opens a report, consults a KPI alert log, or logs on and off). This information can be used for various purposes, such as determining how often a particular report is opened or downloaded by users, or which is the most frequently accessed Data Browser group. By collecting and analyzing this information, you can optimize your RUEI installation to more accurately meet user requirements.
The recording of user events is controlled by the user_events_enabled
setting, in the C_config
table. When set to 1 (the default), user events are recorded; when set to 0, user events are not recorded.
By default, information about user events is stored in the database for a maximum of 31 days. This is controlled by the db_max_user_events
entry in the C_config
table. To modify either of these settings, do the following:
-
Get access as a
RUEI_USER
user -
Run the following command to modify the user event retention setting:
execsql config_set_value processor db_max_user_events
days
where, days
specifies the maximum number of days for which user event information should be stored. This setting has an impact on database usage.
User Event Table Structure
The following table lists the user
event information.
Table 1-1 C_EVENTS Table
Column | Type | Description |
---|---|---|
|
NUMBER |
Unique ID used to identify the user event. |
|
TIMESTAMP |
Time (in UTC format) when the event was performed by user. |
|
VARCHAR2 (255 BYTE) |
Login name of user. |
|
NUMBER |
The event code. |
|
VARCHAR2 (4000 BYTE) |
Description of the event. |
Event Codes and Descriptions
The following tables lists the possible CODE
events and their associated descriptions.
Table 1-2 C_LANG_CATALOG_DATA Table
Code | Description |
---|---|
0 |
User logon. |
1 |
User logout. |
2 |
Load/reload Dashboard tab. |
3 |
Added dashboard (%1%s). |
4 |
Updated dashboard (%1$s). |
5 |
Removed dashboard (%1$s). |
6 |
Load/reload Report tab. |
7 |
View report (%1$s). |
8 |
Load/reload preview report (%1$s). |
9 |
Save report (%1$s). |
10 |
Save report as new (%1$s). |
11 |
Download report as PDF (%1$s). |
12 |
Download report as CSV (%1$s). |
13 |
Download report as TSV (%1$s). |
14 |
Download report as XLS (%1$s). |
15 |
Download report as XML (%1$s). |
16 |
Add report to Favorites (%1$s). |
17 |
Remove report from Favorites (%1$s). |
18 |
Toggle report %1$s mailing (%2$s). |
19 |
Remove report from %1$s mailing (%2$s). |
20 |
Send %1$s mailing now. |
21 |
Load/reload Browse tab. |
22 |
Select graph (%1$s). |
23 |
Select graph category (%1$s). |
24 |
Select group (%1$s). |
25 |
Load/reload diagnostics. |
26 |
Browse report (%1$s). |
27 |
Load/reload KPI overview tab (%1$s). |
28 |
Load/reload KPI overall alert log. |
29 |
Show KPI specific alert log (%1$s). |
30 |
Load/reload KPI correlation (%1$s). |
31 |
User %1$s has been added (%2$s, disabled: %3$d, locked: %4$d, admin: %5$d, sec officer: %6$d). |
32 |
User %1$s has been removed. |
33 |
Application %1$s has been added. |
34 |
Application %1$s has been removed. |
35 |
Service %1$s has been added. |
36 |
Service %1$s has been removed. |
37 |
Suite %1$s has been added. |
38 |
Suite %1$s has been removed. |
39 |
Collector profile %1$s has been added. |
40 |
Collector profile %1$s has been removed. |
41 |
Collector %1$s has been registered in profile %2$s. |
42 |
Collector %1$s from profile %2$s has been unregistered. |
43 |
Collector %1$s in profile %2$s has been restarted. |
44 |
Collector %1$s in profile %2$s has been disabled. |
45 |
Collector %1$s has been moved to profile %2$s. |
46 |
Traffic filter in profile %1$s has been changed to %2$s. |
47 |
VLAN filter in profile %1$s has been changed to %2$s. |
48 |
Port numbers (%1$s) in profile %2$s has been added. |
49 |
Port numbers (%1$s) in profile %2$s has been removed. |
50 |
The IP filter (%1$s) has been added in profile %2$s. |
51 |
The IP filter (%1$s) has been removed from profile %2$s. |
52 |
User account %1$s has been enabled. |
53 |
User account %1$s has been disabled. |
54 |
User account %1$s has been locked. |
55 |
User account %1$s has been unlocked. |
56 |
Maximum login attempt reached for user account %1$s. |
57 |
The password for user %1$s has been expired. |
58 |
The initial password for user %1$s has expired. |
59 |
The minimum password length has been changed to %1$s. |
60 |
The maximum password duration has been changed to %1$s days. |
61 |
Remove report (%1$s). |
62 |
URL prefix %1$s with action: %2$s has been added. |
63 |
URL prefix %1$s with action: %2$s has been removed. |
64 |
URL prefix %1$s with action: %2$s has been updated. |
65 |
Default replay action has been changed to %1$s. |
66 |
Replay IP range action has been changed to %1$s. |
67 |
Replay IP range %1$s has been added. |
68 |
Replay IP range %1$s has been removed. |
69 |
Replay all IP ranges have been removed. |
70 |
Replay IP range %1$s has been changed. |
71 |
Replay IP ranges uploaded. |
72 |
%1$s action with source value: %2$s and action: %3$s has been added. |
73 |
%1$s action with source value: %2$s and action: %3$s has been removed. |
74 |
%1$s action source value: %2$s and action: %3$s has been updated. |
75 |
Default action for %1$s has changed to %2$s. |
76 |
User account %1$s has been renamed to %2$s. |
78 |
User account %1$s password has been changed. |
79 |
User account %1$s has been set as administrator. |
80 |
User account %1$s has been unset as administrator. |
81 |
User account %1$s has been set as security officer. |
82 |
User account %1$s has been unset as security officer. |
83 |
The initial password duration has been changed to %1$s days. |
84 |
The number of allowed login attempts has been changed to %1$s. |
85 |
The SSL key (%1$s) valid from %2$s to %3$s in profile %4$s has been added. |
86 |
The SSL key (%1$s) valid from %2$s to %3$s in profile %4$s has been removed. |
87 |
SSL certifcate masking in profile %1$s has been changed to %2$s. |
88 |
KPI %1$s (%2$s) has been added. |
89 |
KPI %1$s (%2$s) has been removed. |
90 |
KPI %1$s (%2$s) has been updated. |
91 |
KPI category %1$s has been removed. |
92 |
KPI category %1$s has been renamed to %2$s. |
93 |
Naming scheme of %1$s has been updated. |
94 |
Loading satisfaction of %1$s has been updated. |
95 |
System account has been set to not expire. |
96 |
System account has been set to expire. |
Increasing the Size of the Failed Groups
The Failed URLs, Failed services, and Failed pages groups do not use the maximum group size setting. Instead, their size is controlled through the event_max_fail
setting. This specifies the maximum number of rows that can added to the group's main database table during a 1-minute period. By default, this is 1000 rows. For the Slow URLs group, the event_max_slow
setting is used, and specifies the number of the slowest URLs that are recorded within each 1-minute period. By default, this is 1000 rows.
If you change the event_max_fail
or the event_max_slow
setting, you should also review the daily_max_fail
setting. This specifies the maximum number of rows that the groups' tables can contain. This is derived from the formula 1440 * event_max_fail
. By default, this is 1.4 million rows.
To modify the size settings, run the following commands
execsql config_set_value processor event_max_fail 10000 execsql config_set_value processor daily_max_fail 4320000
The event_max_fail
setting is limited to a maximum of 10,000 rows.
Pre-requisites:
-
Confirm that more than 1000 error pages are actually reported for a 1-minute period within the All sessions group.
-
Ensure that replay viewer functionality is enabled. To verify, select Configuration >Security >Replay logging policy>Default replay action setting, and then select Complete logging option.
Note:
Before changing the default of 1000 error pages, you should consider the following:
-
Consider whether you need to increase this limit. Typically, if a high number of error pages are reported within a 1-minute period, it is unlikely that they refer to different problems. Hence, having a large number of recordings for the same page errors might not help with root-cause analysis.
-
Increasing the limit imposes a considerable I/O overhead on both the Reporter and Collector systems. Therefore, you should carefully consider the limits of these systems before modifying the default limit.
-
Each group within the Data Browser has a maximum size. This is 1.5 times its "condense limit" (as specified by the
cube_max_size
option in theC_CONFIG
table). The effect of trying to merge more than 5000 error pages within a 5-minute period can be that the system stops merging data at some point during the day. The more error pages that are encountered, the sooner the Data Browser group will become full. You can diagnose this in the error log file (RUEI_DATA
/processor/log/error.log
) by searching for errors containing the string "wg_failpg_dy_*" starting with the string "no merge:". -
The
event_max_fail
settings is used by both the Failed pages and Failed URLs, and Failed services groups.
Increasing the Default Limits for User Flows
The default maximum number of steps that can be defined within a user flow is 15. This can be modified by using the txn_max_steps
setting. The default maximum number of user flows that can be defined is 200. This can be modified by using the txn_max_trans
setting. To change either setting, do the following:
-
Login to the Reporter system as the
RUEI_USER
user. -
Run the following commands:
execsql config_set_value processor txn_max_steps
steps
execsql config_set_value processor txn_max_transflows
where:
-
steps
specifies the new maximum number of steps allowed with user flows. -
flows
specifies the new maximum number of user flows that can be defined.
-
Note:
Increasing the default maximum values might result in a performance overhead. In addition, if the maximum number of steps within user flows is significantly increased, the graphical reporting of user flows (such as the Flow status and Flow transitions) may become difficult to read.
Obtaining Client IP Addresses within Desktop Virtualization Environments
By default, the client IP address is obtained from the IP header packet sent from the client. The IP packet contains, among other things, the numerical source and destination address of the packet. If RUEI has been placed after a NAT device (such as a load balancer), you can configure RUEI to look in a specified header (set by the NAT device) rather than the IP packet. For more information on this procedure, see Monitoring NATed Traffic in the Oracle Real User Experience Insight User's Guide. However, if monitored clients are using a desktop virtualization environment (such as a Citrix server), the IP address of the server is returned as the client IP address.
The following important points need to be considered:
-
In desktop virtualization environments, you connect to the Internet using a browser running on the Desktop Virtualization Server (citrix for example) rather than on the client machine. RUEI sees the IP from the Virtualization server and not the real originating client IP from the user. However, RUEI provides mapping of user-id to client-ip to provide some way of reporting on the real originating client IP. You can upload this mapping, but this has limited functionality.
-
The map-ranges file contains the originating server IP ranges from which the user-id to client-ip mapping is done.
-
The map-users file contains the user-id to real originating client-IP. For example: A set of Citrix Servers have IP addresses in the ranges 10.0.1.2 - 10.0.1.254 (10.0.1.0/24). Citrix Clients connecting to the Citrix Server have IP-addresses for example in the range 192.168.1.2 to 192.168.1.254 (192.168.1.0/24). Users on these Citrix clients are using a web-application monitored by RUEI. In order to configure RUEI to report on the real client-ip instead of the Citrix Server IP, the following configuration is used:
RANGE 10.0.1.0/24 USER_ID\tCLIENT_IP JohnSmith\t192.168.1.10 FredWhite\t192.168.1.10 SteveBrown\t192.168.1.10
-
When a session with a client-ip (the Citrix Server IP) within one of the ip-ranges in the RANGE file is found, RUEI will attempt to map the user-name from that session to a real-client ip (the citrix-client-pc of the user) by using the
USER_ID-CLIENT_IP
mapping file.So, any functionality or reporting (for example, Client Network views in the data browser ) in RUEI that depends on the client-ip will use the mapped client-ip. If no match is found in the
USER_ID\tCLIENT_IP
mapping file, the original client-ip retrieved from TCP/IP layer or from configured header will be used.
Note:
Any user having a client IP in the map-ranges file, but where the user id is not in the map-users file, is not mapped. Pages requested by that user are reported with IP "unknown".
To configure RUEI to report a preferred client IP address, do the following:
-
Create a file containing a list of the IP address range(s) that you want to be remapped. Each range must be specified using the format 10.1.1.0/24. It is recommended that you call the file
ip-map-ranges-file.tsv
, this file can also contain IPv6 based CIDR notation. For example:RANGE 169.254.0.0/16 172.16.0.0/12
-
Create a tab-separated file containing a list of the required user IDs and client IP addresses. It is recommended that you call the file
ip-map-users-file.tsv
, this file can also contain IPv6 based CIDR notation. For example:USER_ID\tCLIENT_IP JohnSmith\t10.10.10.50 FredWhite\t10.10.10.51 SteveBrown\t10.10.10.52
In the above example
\t
indicates a tab character. Ensure that both files do not contain any leading or trailing characters, and no lines containing only whitespace or special characters (such as/n
or/r)
. -
Logon to the RUEI Reporter system as the
RUEI_USER
user. -
Import the two created files onto a suitable location on the RUEI Reporter system.
-
Run the
import-ip-map
script (located in theRUEI_DATA
/processor/bin
directory) by using the following command:import-ip-map -r
ip-map-ranges-file
-uip-map-users-file
Where,
ip-map-ranges-file
andip-map-users-file
are the two files to be created and imported.Any reporting changes made by this facility take effect within appropriately 5 minutes.
Restoring Default functionality
To restore default client IP address reporting, create two files containing only column headers, and repeat the previous procedure.
Controlling the Maximum Session Duration and Idle Time
By default, a visitor session is regarded as terminated if the visitor has been inactive for longer than 60 minutes. This is controlled through the session_idle_time
setting. In addition, the default number of hours that user IDs and custom dimensions are remembered for a session is 12 hours. This is controlled through the max_age_session
setting.
Lowering the session_idle_time
setting will increase Reporter system performance in terms of CPU utilization. It has no impact on memory usage. However, a drawback of lowering this setting is that the identified visitors returning within the specified session idle time will be reported as anonymous.
You should consider lowering the max_age_session
setting when the Reporter system does not have enough memory and starts to swap. When this setting is lowered, and the monitored traffic contains mostly long sessions, user IDs can be lost. This setting should not be set lower than the session_idle_time
setting.
Use the following commands to obtain a setting's current value:
execsql config_get_value processor session_idle_time execsql config_get_value processor max_age_session
Use the following commands to modify a setting's value:
execsql config_set_value processor session_idle_timeidle_time
execsql config_set_value processor max_age_sessionmax_age
Where:
-
idle_time
specifies the number of seconds of visitor inactivity after which the session is considered terminated. -
max_age
specifies the maximum number of hours after which session information is cleared from memory.
Improving Processing Concurrency
By default, 3 threads are used on the Reporter system for traffic processing. It is controlled by the lookup_threads
setting. Performance improvement can be obtained (through additional concurrency in processing) by increasing the value of this setting. An indication that this setting is too low is the following internal error appearing in the Event log:
Processing backlog larger than %d minutes, restarting logr (the backlog will be skipped).
It means that the Reporter system cannot keep up with the processing of the arriving data.
Use the following command to obtain the setting's current value:
execsql config_get_value processor lookup_threads
Use the following command to modify the setting's value:
execsql config_set_value processor lookup_threads threads
Where, threads
specifies the number of threads available for use by the Reporter system. This setting must not be higher than the number of cores available on the Reporter system.