| Oracle® Real User Experience Insight Installation Guide Release 6.5.0 for Linux x86-64 Part Number E17370-02 |
|
|
View PDF |
| Oracle® Real User Experience Insight Installation Guide Release 6.5.0 for Linux x86-64 Part Number E17370-02 |
|
|
View PDF |
This chapter introduces the role of Oracle Real User Experience Insight (or RUEI for short), its architecture, attachment, and deployment options.
The usage of Web applications and services continues to grow. This includes not only the use of the Internet as a marketing channel, but also Extranet-based supply chain and back-office integration, and Intranet deployment of internal applications. Increasingly, it also includes the utilization of Web services which implement clearly defined business functions. RUEI is designed for measuring, analyzing, and improving the availability and performance of all of these deployment scenarios.
Typically, RUEI is installed before the Web servers, behind a firewall in the DMZ (as shown in Figure 1-1). The data collection method is based on Network Protocol Analysis (NPA) technology. This method is 100% non-intrusive. Hence, it does not place any load on a Web server, or require installing software agents that will impact performance. In addition, it does not require any change to the current application or infrastructure. When a new application release is deployed, or when an additional Web server is added, there is no or very little change required to RUEI's monitoring environment.
When an object is requested by a visitor, RUEI sees the request and measures the time the Web server requires to present the visitor with the requested object. At this point, RUEI knows who requested the page (the client IP), which object was requested, and from which server the object was requested (server IP).
When the Web server responds and sends the requested object to the visitor, RUEI sees that response. At this point, RUEI can see whether there is a response from the server, whether this response is correct, how much time the Web server required to generate the requested object, and the size of the object. In addition, RUEI can also see whether the object was completely received by the visitor, or if the visitor aborted the download (that is, proof of delivery). Hence, RUEI can determine the time taken for the object to traverse the Internet to the visitor, and calculate the Internet throughput between the visitor and the server (that is, the connection speed of the visitor).
RUEI is based on a three layer product architecture, as shown in Figure 1-2.
The monitored data packets are processed by the following layers:
Data collection
This layer is responsible for acquiring raw data and delivering it to the Data processor. This data can be collected from multiple sources. The available attachment options are described later in this section.
Data processing
This layer converts the raw data into OLAP data sets. These comprise the multidimensional data structure that is viewable within the data browser.
Data presentation (reporter)
This layer is RUEI's analysis and reporting environment. This is a Web-based information portal that can be accessed from any selected browser. The interface between the data processing and data presentation layer is based on open db calls.
To read HTTP(S) data streams, a proprietary software module reassembles TCP/IP packet streams. Because the data collectors do not have an assigned IP number, and the software using these data collectors does not have a functional IP stack, RUEI is not able to respond to incoming traffic received on the data collectors. This makes RUEI "invisible" to the monitored networks, and completely secure.
Note:
Because of the non-intrusive way in which RUEI collects data, it is not possible for it to request retransmission in the event of an error on the measurement port.Data collection can be configured to log encrypted data. To facilitate this, a copy of the Web server's private SSL keys needs to be set up in the data collector. In addition, RUEI can be configured to omit logging of sensitive data in the arguments of POST requests of forms or content; so called data masking (or blinding).
RUEI supports the use of both copy portsFoot 1 and TAPsFoot 2 for monitoring network traffic (10/100 Mbps and 1/10 Gbps Ethernet connections are supported). Copy ports and TAPs are available for copper or fibre-based network infrastructures. While both devices allow non-intrusive monitoring of network traffic, there are differences between these two connection options. These are highlighted in the rest of this section.
A copy port is a switch that starts to build up a Layer 2 forwarding table on the basis of the source MAC address of the different packets that the switch receives. After this forwarding table is built, the switch forward traffic that is destined for a MAC address directly to the corresponding port.
For example, after the Web server MAC in Figure 1-3 is learned, unicast traffic from the browser to the Web server is only forwarded to the Web server port. Therefore, the Collector does not see this traffic.
Figure 1-3 Network Connection Using a Copy Port
In the configuration shown in the lower part of Figure 1-3, the Collector is attached to a port that is configured to receive a copy of every packet that the browser sends and receives. This port is called a copy port. Copy ports can copy traffic from any or all data ports to a single unused port and prevents bi-directional traffic on the port to protect against backflow or traffic into the network.
Be aware that activating a copy port on a switch can have a performance impact. Typically, copy ports support a wide range of configuration options, and for further information about these options you should consult your switch documentation or contact the vendor.
TAPs can be placed between any two network devices (such as routers and firewalls). Any monitoring device connected to a TAP receives the same traffic as if it were in-line, including all errors. This is achieved through the TAP duplicating all traffic on the link, and forwarding it to the monitoring port(s). The example shown in Figure 1-4 illustrates a typical TAP deployment for one Collector.
Figure 1-4 Network Monitoring Using a TAP
Unlike copy ports, in the event of power failure, TAPs continue to allow data to flow between network devices. TAP devices are available for copper or fibre-based infrastructures. Generally, the use of TAPs is considered preferable because they can be easily deployed when and where required, but without reconfiguration of switches or engineers needing to re-cable a network link.
Note:
Broadly speaking, there are three types of TAPs: network, regeneration, and aggregation TAPs. RUEI supports the use of network and regeneration TAPs. The use of aggregation TAPs is not recommended because they can lose data, and do not provide an acceptable level of accuracy. However, the deployment of multiple Collectors, or the connection of multiple links directly to one Collector, is available for the aggregation of data from multiple streams. In addition, be aware that when capturing data with a network-TAP device, the use of cascaded TAP configurations is not supported.A RUEI system can be configured in two different ways: as a Reporter, or as a Collector. Each installation option is reviewed in the following sections.
Reporter
Here, the Reporter provides a browser-based interface to the collected data. After processing, this data is stored in an Oracle database. This database can reside locally (that is, on the Reporter system), or on a remote database server.
Note that each Reporter installation also contains a local Collector instance. The Reporter can either be configured to just process information gathered by this local Collector (this is a single-server configuration similar to the one shown in Figure 1-5), or can (optionally) be configured to receive information from additional Collector installations. Note the local Collector instance on the Reporter system can also be disabled if not required.
Collector
If a RUEI system is installed as a Collector, it submits the data it gathers to a Reporter system. Multiple Collectors can be attached to the same Reporter. Split server #1 in Figure 1-5 is an example of a single Collector split-server configuration, while split server #2 is an example of a split-server configuration using two Collectors. Note that a direct network connection is required between the Collector(s) systems and the Reporter system.
As mentioned earlier, the data available via the Reporter system is stored in an Oracle database. This database can reside locally on the Reporter system, or on a remote database server (such as a database cluster).
The use of a remote database server provides a number of potential advantages over a locally installed database. In particular, it offers easier integration with existing security and back-up policies, as well as improved performance through the use of dedicated servers.
Currently, RUEI 6.5 supports the Oracle 11g database. Note the Oracle 10g database is not supported.
The use of multiple Collectors may be considered when there is a need to monitor very high levels of data traffic. In addition, this deployment also provides the possibility of enhanced security. For example, by placing the Collector(s) outside the office network, while placing the Reporter system within the network.
Split-server configuration #1 in Figure 1-5 shows an example of a typical DMZ installation. The Collector is located in the DMZ, and the Reporter is within the server network environment. Note that the local Collector instance is disabled. Split-server configuration #2 shows an example of a deployment consisting of two Collectors. This could, for example, be used between two data centers (both monitoring the DMZ), where one data center acts as a failover for the other.
Split-server configuration #3 shows an example of a deployment in which both data lines are monitored in the same reporting environment. It also features the use of offloading to a database on a separate server. Note this deployment assumes that the traffic on each line is mutually exclusive. It also shows an example of a deployment used for security reasons. While the traffic from Web servers A and B are monitored and reported, the traffic from Web server C is not. This is also the reason why the Collectors are not placed above the switch.
For security reasons, it is recommended that access to the Reporter system is restricted to trusted IP ranges. Similarly, you may want to locate the Reporter system inside the internal network to maximize its security. The Collector's data gathering ports should be in the DMZ.
The required minimum system specifications for the selected configuration (as explained in Installation and Deployment Options) are described in the following sections.
Network Cards
It is recommended that you carefully consider the selection of network cards for your infrastructure. Depending on the connection option you selected in Connection Options, both copper and fibre-based network cards may be required. If necessary, consult both your network and systems management teams.
Note:
For more information about required and recommended system specifications, please contact Customer Support.Table 1-1 Single-Server System Minimum Requirements
| Element | Requirements |
|---|---|
|
CPU |
64-bit Intel or AMD dual-CPU, dual-core processor (> 2 G Hz) or equivalent. |
|
Memory |
16 GB. |
|
Disk space |
|
|
Network interfaces |
When using a network-TAP deviceFoot 3 , a minimum of three network interfaces is required:
|
|
GSM modem (optional) |
Optional support for a GSM modem to send text messages messages. The modem needs to be either GSM07.05 or GSM07.07 compatible. It can be connected through a serial or USB port. If USB is used, RUEI uses the first available port (ttyUSB0). Alternative methods of sending text messages are available (http/e-mail). |
Footnote 1 To ensure acceptable performance of the RUEI installation, it is recommended to use high performance disk systems, with a minimum supported I/O rate of 70 MB/s. When monitoring high volumes of traffic, more powerful disk systems may be required. (Hardware) RAID-5, RAID-10, or equivalent storage configurations are strongly recommended.
Footnote 2 This may need to be increased if Enriched data exchange is enabled.
Footnote 3 When capturing data with a network-TAP device, the use of cascaded TAP configurations is not supported.
Table 1-2 Reporter System Minimum Requirements
| Element | Requirements |
|---|---|
|
CPU |
64-bit Intel or AMD dual-CPU, dual-core processor (> 2 G Hz) or equivalent. |
|
Memory |
16 GB. |
|
Disk space |
|
|
Network interfaces |
A minimum of 1 network interface is required. |
|
GSM modem (optional) |
Optional support for a GSM modem to send text messages. The modem needs to be either GSM07.05 or GSM07.07 compatible. It can be connected through a serial or USB port. If USB is used, RUEI uses the first available port (ttyUSB0). Alternative methods of sending text messages are available (http/e-mail). |
Footnote 1 To ensure acceptable performance of the RUEI installation, it is recommended to use high performance disk systems, with a minimum supported I/O rate of 70 MB/s. When monitoring high volumes of traffic, more powerful disk systems may be required. (Hardware) RAID-5, RAID-10, or equivalent storage configurations are strongly recommended.
Footnote 2 This may need to be increased if Enriched data exchange is enabled.
The requirements for Collector systems are shown in Table 1–3.
Table 1-3 Collector System Minimum Requirements
| Element | Requirement |
|---|---|
|
CPU |
64-bit Intel or AMD dual-core processor or equivalent. |
|
Memory |
8 GB. |
|
Disk space |
Minimum 200 GB HDD free space. |
|
Network interfaces |
When using a network-TAPFoot 1 device, a minimum of three network interfaces are required:
When using a network-copy port, a minimum of two network interfaces are required:
|
Footnote 1 Capturing data with a network-TAP device prevents the use of a cascaded TAPs configuration.
Footnote 2 For up and down stream traffic. Note that the use of TAPs that integrate up and down stream traffic on one line is not recommended.
Important:
Please note that an Intel (or compatible) 64-bit platform is a strict requirement for both the hardware and the operating system in all deployment scenarios.This section presents a best practices framework within which to optimize your RUEI deployment. It is recommended that you carefully review the following information.
Planning Your Deployment
It is important that the nature of the monitored network environment is clearly understood before deciding upon your RUEI deployment strategy. This includes not only the basic network connectivity, ports, addressing, and physical device requirements, but also a sound understanding of the monitored applications.
Moreover, before deploying RUEI, the basic traffic flows within the network must have been identified. This should include information about average and peak volumes of traffic. Any physical deployment requirements (such as space limitations, distances, power planning, rack space and layout, or cabling) should also have been identified.
You can use the checklist presented in Appendix E, "Installation Checklist" to capture much of this information.
Forms-Based Traffic
If you are planning to monitor Forms-based traffic (with the Oracle E-Business accelerator package), be aware that the memory requirements may be higher than those outlined in Hardware Requirements. This is especially the case in deployments with heavy levels of Forms traffic. In this case, you should consider a split-server deployment.
Full Session Replay
If you are planning to make use of the Full Session Replay facility, you may need to configure additional storage capacity. This is explained in Full Session Replay Storage Requirements.
Encrypted Traffic
If a significant level of the monitored traffic is encrypted, this can increase the CPU overhead. In this case, it is recommended that you consider configuring additional CPUs or, alternatively, a split-server deployment.
Very High Levels of Traffic
When very high levels of traffic are being monitored (that is, more than 10 million page views per day), you should consider a split-server deployment. Alternatively, consider the use of a remote database. The latter has the effect of significantly reducing (by up to 30%) the CPU overhead on the Reporter system. Monitored environments with more than 20 million page views per day should consider the use of both a split-server deployment and a remote database.
As explained in the Oracle Real User Experience Insight User's Guide, the availability of specific data within the Data Browser, as well as reports based on that data, depends on the Reporter data retention policies defined for your RUEI installation. By default, RUEI retains information on daily, monthly, and yearly levels for 32 days, 13 months, and 5 years, respectively. In addition, information about failed pages, URLs, and services is retained for 15 days. The maximum amount of database storage required to maintain the default data retention policies is shown in Figure 1-4.
Table 1-4 Default Required Database Storage
| Type of data retained | Default retention policy | DB space required per period (GB) | Total DB space required (GB) |
|---|---|---|---|
|
Failed pages/URLs/services |
15 |
1.5 |
22.5 |
|
DailyFoot 1 |
32 |
1 |
32 |
|
Monthly |
13 |
1 |
13 |
|
Yearly |
5 |
1 |
5 |
|
SuitesFoot 2 |
15 |
0.5Foot 3 |
7.5 |
|
Additional overhead |
10 |
||
|
Total required DB space |
90 |
Footnote 1 This includes the All pages, All sessions, All functions, All transactions, Key pages, and URL diagnostics groups.
Footnote 2 Suites use the Daily retention policy setting.
Footnote 3 0.5 GB is required per day for each configured suite type.
Be aware that, in addition to the database storage required for each retained type of data, appropriately 10 GB of database storage is also required for other purposes. This includes KPIs, SLAs, and processing requirements. In Table 1-4, it is assumed that one suite type is configured.
If you modify the default Reporter data retention policies, it is recommended that you use the Calculate facility to see the effect your new retention policy would have on required database storage. Note that the projected database utilization is based on previous database utilization, rather than maximum database usage.
The default amount of database storage available to the Reporter is 200 GB, and for most deployments, this will meet your operational requirements.
Example - Increasing the Number of Groups
Consider the following situation. You have decided to retain information on daily, monthly, and yearly levels for 90 days, 24 months, and 5 years, respectively, and failed pages, URLs, and services information should be retained for 90 days. For the purposes of this example, it is assumed that only one suite type is configured. The maximum amount of database storage required to maintain this data is shown in Figure 1-5.
Table 1-5 Required Database storage
| Type of data retained | Default retention policy | DB space required per period (GB) | Total DB space required (GB) |
|---|---|---|---|
|
Failed pages/URLs/services |
90 |
1.5 |
135 |
|
Daily |
90 |
1 |
90 |
|
Monthly |
24 |
1 |
24 |
|
Yearly |
5 |
1 |
5 |
|
Suites |
90 |
0.5 |
45 |
|
Additional overhead |
10 |
||
|
Total required DB space |
264 |
Maximum Data Browser Group Sizes
In addition to modifying the data retention policy settings, you can also modify the maximum size to which Data Browser groups are allowed to grow before data is condensed. This is fully explained in Appendix B, "Setting the Maximum Data Group Size".
If you are planning to make use of the Full Session Replay facility, you may need to configure additional storage capacity available to the Collector system. This should be a separate device (not a partition of the Collector server's existing hard drive), and made accessible to the RUEI file system. The procedure to do this, together with guidance on storage requirements, is described in the rest of this section. Note that this procedure must be repeated for each Collector for which full session replay information is required.
Configuring Additional Storage for Full Session Replay
To configure the additional required storage, do the following:
Mount the device. For example, under /mnt/external_storage.
Temporarily stop the Collector by issuing the following command:
appsensor stop wg
Move the $APPSENSOR_HOME/wg/REPLAY directory to the new device. In the above example, this is /mnt/external_storage, and the result is that the replay files are now located in the /mnt/external_storage/REPLAY directory.
Create a symbolic link from /mnt/external_storage/REPLAY to $APPSENSOR_HOME/wg/REPLAY.
Restart the Collector by issuing the following command:
appsensor start wg
Calculate the required storage capacity. To do so, multiple the average number of daily page views by the average page size. Then, multiple this number by the number of days you wish full session replay data to be retained. Use Table 1-6 as guidance.
Table 1-6 Full Session Replay Storage Estimates
| Low page weight (~10 Kb) | Medium page weight (~50 Kb) | High page weight (~100 Kb) | ||||
|---|---|---|---|---|---|---|
|
Page views per day (millions) |
Size per day (GB) |
Disk I/O (MB/sec) |
Size per day (GB) |
Disk I/O (MB/sec) |
Size per day (GB) |
Disk I/O (MB/sec) |
|
0.5 |
5 |
0.1 |
25 |
0.3 |
50 |
0.6 |
|
2 |
20 |
0.2 |
100 |
1.2 |
200 |
2.3 |
|
5 |
50 |
0.6 |
250 |
2.9 |
500 |
5.8 |
|
10 |
100 |
1.2 |
500 |
5.8 |
1000 |
11.6 |
|
20 |
200 |
2.3 |
1000 |
11.6 |
2000 |
23.1 |
|
50 |
500 |
5.8 |
2500 |
28.9 |
5000 |
57.9 |
Important:
Table 1-6 is intended for guidance only. It is strongly recommended that you regularly review average page sizes and daily page views, and adjust the required storage size as necessary.Select Configuration, then General, then Advanced settings, and then Collector data retention policy. Click the Full session replay storage size (GB) setting. Specify (in gigabytes) the required storage size. Note that the maximum that can be specified is 100 TB. When ready, click Save.
When calculating the amount of RAM required by your RUEI installation, it is recommended that you consider the following:
For the Reporter system, each million visitor sessions per day requires 256 MB. Hence, 3 million visitor sessions per day would require 768 MB. In addition, each million page views requires 100 MB - 256 MB. Note that exact amount depends the length of monitored URLs, average session duration, and the number of events (such as custom dimensions).
For each Collector system, each 10,000 hits requires 200 MB, and each 1000 SSL connections require 1 MB. In addition, up to 600 MB of network traffic can be buffered before individual TCP sessions start to be dropped. Up to 600 MB should also be assumed for content checks (such as XPath queries and error strings). Note that if you define a large number of content checks, or specify that they contain NLS character sets, the memory required may increase.
The following GNU/Linux distributions are supported:
Oracle Enterprise Linux 5.x.
RedHat Enterprise Linux 5.x.
Encrypting Sensitive Data
If sensitive data needs to be encrypted, you have the opportunity to encrypt your entire disk configuration during the disk partitioning phase of the Enterprise Linux installation procedure. For more information, see Set up Disk Partitioning.
All server system clocks should be synchronized via NTP using UDP port 123.
Support DNS information requests over TCP and UDP port 53.
Support reports and e-mail alerts using TCP port 25.
Support SNMP traps on request from an SNMP Manager using UDP port 161/162.
The RUEI user interface is accessible over HTTPS port 443.
In the case of a remote database setup, access to TCP port 1521 is required between the Reporter and remote database server.
Each remote Collector system should be accessible by the Reporter system over TCP port 22. It is recommended all other ports be blocked.
Collector-Reporter Bandwidth
The amount data transferred between a remote Collector and the Reporter system largely depends on the type and level of network application traffic monitored by RUEI. In addition, the configuration of RUEI (such as defined functional errors, content checks, and page naming schemes) also influences the size of Collector files that need to be transferred to the Reporter system.
At peak times, the amount of data that needs to be transferred will be higher than during low traffic periods. Note that the exact amount of the data transmission from a remote Collector to the Reporter system can only be determined after the actual RUEI deployment.
For an initial deployment, the following simple rule can be used: each 5 million daily page views will result in a peak transfer of approximately 125 MB at peak time, and approximately 1 GB per day. Hence, typically only a few percent of the actual monitored traffic will be stored by a Collector and transferred to the Reporter. When you want or need to minimize this data transfer, it is recommended that you minimize the amount of monitored HTTP traffic which is not required by RUEI. For example, by using a subnet or VLAN-filtered network.
The workstations that will access the RUEI user interface must have one of the following browsers installed:
Mozilla Firefox 3.0.
Internet Explorer 6 SP2.
Internet Explorer 7.
Note that JavaScript must be enabled. No other browser plug-ins are required.
In addition, the workstation should have a screen resolution of 1024 * 768 (or higher).
Important:
Ensure that any pop-up blocker within the browser has been disabled.RUEI uses AJAX to enhance its user interaction. Internet Explorer relies on the MSXML control to facilitate AJAX. The AJAX dependencies can trigger a security warning when using strict security settings.
Internet Explorer 6 does not properly support transparent images in the PNG format. RUEI uses a well know fix (AlphaImageLoader) for this problem which relies on DirectX. If you are experiencing browser crashes with IE 6, you may need to update your version of DirectX. The PNG fix can trigger a security warning when using strict security settings.
The process of preparing the server system, and installing the RUEI software, has the following phases:
Install the Linux operating system. This is described in Chapter 2, "Installing the Linux Operating System".
Install all prerequisites for RUEI. This is described in Chapter 3, "Installing the Required Components".
Install the RUEI software. This is described in Chapter 4, "Installing the RUEI Software". The procedure for upgrading an existing 5.x or 6.0.x installation to release 6.5 is also described.
Perform an initial configuration of the RUEI installation. This is described in Chapter 6, "Configuring RUEI".
Install and configure the Oracle HTTP server. Note that this is only required if you intend to use the Oracle Single Sign-on (SSO) service to authenticate RUEI users. This is described in Chapter 7, "Installing and Configuring the Oracle HTTP Server".
The procedure to install the required components depends on whether you are installing a Reporter or Collector system, and whether the Reporter system should use a locally installed database or a remote one. Figure 1-6 shows the recommended installation path.
Footnote Legend
Footnote 1: Copy ports are also known as Switched Port Analyzer (SPAN) ports which is a feature of Cisco switches.
|
 Copyright © 2010, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|
