Oracle® Real User Experience Insight Installation Guide Release 6.5.0 for Linux x86-64 Part Number E17370-02 |
|
|
View PDF |
This appendix provides a detailed explanation of how the default maximum Data Browser group sizes can be increased to provide more accurate reporting of monitored traffic. It is strongly recommended that you carefully review the information presented before modifying the default settings.
In order to optimize performance, individual groups are not permitted to grow to unlimited sizes. Each main group table has a maximum permitted size. By default, this is 300 MB. For session diagnostics information, this limit is 500 MB.
The maximum size of a group is managed by condensing. This is the process by which the number of rows within the group table are reduced by moving the least used data to a "other" group. For example, rows within the client-browser/version dimension might contain a small number of rows for rarely seen Mozilla Firefox versions. In this case, the number of rows within the table is reduced by replacing these with a single "other-firefox" row.
You might consider increasing the maximum size of the groups if the information you require is regularly being condensed. For example, required individual page names or user IDs are only reported as "other". Broadly speaking, this approach is recommended in the case of high levels of network traffic.
If the maximum group size is set to more than 150% (that is, 450 MB), it is strongly recommended that the database is configured as a remote database. The maximum supported group size is 300% (that is, 900 MB). Be aware that response times to Data Browser queries may take longer to process if the maximum group size is increased.
It is important to note that custom dimensions have a significant impact on the condensing process. Incorrect configuration of custom dimensions can result in excessive condensing. For example, product names (or similar highly selective attributes) should not be configured as custom dimensions because that will skew the condensing process. Specifically, each page/product combination becomes a separate candidate for condensing, and only high frequency combinations will remain visible after condensing.
To determine the current data group sizes, issue the following command as the defined RUEI_USER user:
cop stats %date_filter
where date_filter
specifies the date for which you want to view group sizes. Note that if you specify today's date, because the groups are still growing, the reported sizes may not represent their final sizes. Therefore, it is recommended that you issue the query as late in the day as possible in order to most accurately determine their final sizes. If you specify a previous day's date (for example, yesterday), the reported group sizes reflect their compressed sizes. A group's compressed size needs to be approximately doubled to determine its uncompressed size.
An example command and sample output would appear as follows:
cop stats %20091110 DATA ROWS DATA SIZE data desc data desc cube name ---- ------ ------- ------- ---------- 705532 565699 300.0 MB 30.0 MB wg_visit_dy_20091110 (uncompressed) 918028 48727 303.0 MB 4.0 MB wg_trans_dy_20091110 (uncompressed) 319163 1043110 169.0 MB 96.0 MB wg_slowurl_dy_20091110 (uncompressed) 937011 222760 439.0 MB 11.0 MB wg_sesdiag_dy_20091110 (uncompressed) 005411 005320 2.1 MB 0.3 MB wg_service_dy_20091110 (uncompressed) 834063 126196 315.0 MB 20.0 MB wg_page_dy_20091110 (uncompressed) 019008 001026 4.0 MB 5.0 MB wg_kpi_dy_20091110 (uncompressed) 000345 004598 0.8 MB 0.6 MB wg_keypage_dy_20091110 (uncompressed) 286384 967535 168.0 MB 88.0 MB wg_failurl_dy_20091110 (uncompressed) 000378 005544 0.4 MB 0.2 MB wg_failsrv_dy_20091110 (uncompressed) 632973 2175464 363.0 MB 151.0 MB wg_failpg_dy_20091110 (uncompressed)
In the above example, it appears that the daily All pages (wg_page_dy
), and two other groups have reached their maximum limits (300 MB), and data is being condensed. Note that the Failed pages group (wg_failpg_dy
), although it has over 300 MB of data, has not reached its limit of 1.4 million rows.
To set the maximum group size, issue the following commands:
$ su – moniforce $ sqlplus /@$RUEI_DB_TNSNAME SQL> update UXS_CONFIG set VALUE='375000000' where NAME='cube_max_size';
where the value specifies (in bytes) the maximum group size. In the above example, this is 375 MB.
Note that the Failed URLs, Failed services, and Failed pages groups do not use the maximum group size setting. Instead, their size is controlled through the event_max_fail
setting. This specifies the maximum number of rows that can added to the group's main database table during a 5-minute period. By default, this is 5000 rows. For the Slow URLs group, the event_max_slow
setting is used, and specifies the number of the slowest URLs that are recorded within each 5-minute period. By default, this is 5000 rows.
Note that if you change the event_max_fail
or the event_max_slow
setting, you should also review the daily_max_fail
setting. This specifies the maximum number of rows that the groups' tables can contain. This is derived from the formula 288 * event_max_fail
. The default, is 1.4 million rows.
Example - Increasing Maximum Group Size
Consider the case in which the maximum main group table size is increased to 900 MB (that is, 300% of the default maximum size). The maximum required database storage is indicated in Table B-0.
Table B-1 Default Required Database storage
Type of data retained | Default retention policy | Maximum group size (%) | DB space required per period (GB) | Total DB space required (GB) |
---|---|---|---|---|
Failed pages/URLs/services |
15 |
300% |
4.5 |
67.5 |
Daily |
32 |
300% |
3 |
96 |
Monthly |
13 |
300% |
3 |
39 |
Yearly |
5 |
300% |
3 |
15 |
Suites |
15 |
300% |
1.5 |
22.5 |
Additional overhead |
10 |
|||
Total required DB space |
250 |
After increasing the maximum group size setting, it is strongly recommended that you carefully review the effect of the change on the performance of the Reporter system. You should wait at least a full day, and select System, then Status, and the Data processing. Review the system load and determine whether the Reporter system can handle the additional processing overhead that increasing the maximum group size setting represents.
Important:
Increasing the data group sizes can have a significant impact on overall system performance. Therefore, any changes to data group sizes should be carefully planned, and performed in manageable steps.By default, the RUEI database is limited to a maximum size of 960 GB. This should be enough to meet your operational requirements. However, if the maximum database size is ever reached, processing automatically stops. You will then need to modify the Reporter data retention policies to reflect the size of available database storage, or increase the maximum database size, before processing can continue.