Configuring Collector Systems

Increasing Memory Availability to Collectors

By default, the Collector process (panther) is assigned 30% of available system memory within a single-server installation. Within a remote Collector installation, the Collector process is assigned 70% of available memory.

To set the memory available to the Collector process, run the following command:

execsql config_set_profile_value profile config MaxMemoryUsage replace setting

Where:

profile specifies the name of the Collector profile that needs to be updated.
setting is the percentage of system memory available to the Collector process. The percentage sign must not be specified with the setting. It is recommended that you specify a percentage not higher than 90%. If the Collector process has to share resources with other software running on the system, a maximum setting of 80% is more appropriate.

To obtain the required Collector profile name on Reporter GUI, select Configuration > Security, and then Collector profiles, or run the following command:

execsql config_get_profiles

Configuring Domain-Based Segmentation

To configure RUEI to filter (segment) monitored traffic based on domain names, do the following:

Select Configuration >Security >Network filters, and select the required Collector profile. Ensure that the Packet capture menu specifies the Specified domains option for each of the required Collector profile.
Create, modify, or delete the required rows in the c_domain_segments database table. The table has the following format:
```
ID        Priority   Domain   Profile_ID   Traffic_segment
1000      10         *.nl          2            1|1
1100      8          *.be          2            1|2
1150      3          *.oracle.*    2            1|1
1200      1          *.com         2            3|4
```
Where:
- The ID column represents a unique identifier for each row in the table.
- The Priority column represents the order in which the filters are applied. The filters with the highest priority numbers are applied first, and those with the lowest are resolved last. In the example, monitored traffic relating to the domain myshop.oracle.com would be filtered as *.oracle.* 1|1, and not the *.com 3|4 filters. Also, all domain traffic with the country code nl is monitored, while only the first half of the data stream should be monitored for domains with the country code be.
- The Domain column contains the actual filter value where * can be used as a wildcard.
- The Profile_ID column relates to the ID of the Collector profile for which the filters should apply. This ID can be found in c_cprofiles.
- The Traffic_segment column contains the segment which should be used for the specified filter. You can specify up to 128 parts. For example, 34|128 will take the 34th segment out of 128.
To view the currently defined network filters, logon to the Reporter system as the RUEI_USER user, and run the following command:
```
sqlplus /@RUEI_DB_TNSNAME
select id, prio, domain, profile_id, traffic_segment from c_domain_segments order by prio; 
```

To insert a row into the table, run the following command:

insert into c_domain_segments (id, prio, domain, profile_id, traffic_segment) values (c_domain_segments_seq.nextval, 1, '*.nl', 2, '1|2');

To delete a row from the table, run the following command:
```
delete from c_domain_segments where id=1;
```
To alter a filter's priority, run the following command:
```
update c_domain_segments set prio=100 where id=2; 
```

Configuring the Forms Socket Mode Timeout

By default, the Forms socket mode setting is set to 10 minutes. To view it, run the following command:

execsql config_get_profile_value System forms FormsSocketTimeout

To alter it, run the following command:

execsql config_set_profile_value System forms FormsSocketTimeout replace 600

Configuring Collector Performance Settings

The collector can use multi-threading to perform more traffic analysis tasks. Along with other scaling settings, this feature can be used to let the collector take advantage of the increasing amounts of CPU cores and memory installed in modern hardware configurations. This results in a more efficient usage of available hardware resources and increased performance.

The collector contains a pipeline of different thread types. Different thread types perform different functions, and it is important to increase the thread count for the specific thread type that is experiencing high load. Load per thread type can be viewed in the collector status, performance section.

Configuring the RX_RING Buffer

The RX_RING buffer is used to send network traffic from the kernel to the collector process. Configuring this buffer to contain more frames increases collector resilience against small fluctuations in the amount of incoming traffic. Configuring this buffer to contain fewer frames reduces memory usage, and may be required to configure Jumbo frames.

There are three collector configuration parameters:

CaptureLength: The CaptureLength setting specifies the maximum size of a single packet to be captured. This parameter can be found in the UI at the following location: Configuration-->Security-->Jumbo frames
CaptureFrames: The CaptureFrames setting specifies the maximum number of packets that fit in the RX_RING buffer.
CaptureBufferMaxMemoryUsage: The CaptureBufferMaxMemoryUsage setting limits the total size of the buffer.

The actual memory used is based on the lower of (CaptureLengh * CaptureFrames) and CaptureBufferMaxMemoryUsage. The number of frames stored may be automatically adjusted down further – most kernels do not support more than 4GB of reserved memory for the RX_RING buffer. The following examples show the commands used to update these settings.

Examples

execsql config_set_profile_value profile config CaptureLength replace 65536
execsql config_set_profile_value profile config CaptureFrames replace 65536
execsql config_set_profile_value profile config CaptureBufferMaxMemoryUsage replace 4095mb

Replace the word profile with the correct profile name. Instructions on how to obtain the profile name can be found at the bottom of this page.

NumATMThreads

The 'atm' threads handle HTTP stream parsing. By default, there is 1 ATM thread.

Note:

If your monitored traffic contains servlet-mode oracle forms traffic, do not increase the number of ATM threads.

HTTPPrestartApts

The 'apt' threads handle HTTP content parsing, including content scan and xpath scanning. By default, there is one APT thread per ATM, and additional threads start automatically. There can be a maximum of 64 threads per ATM.

Examples

If the collector system has a minimum of 12 cores and at least 32GB of RAM, run the following command as the RUEI_USER on the reporter system:

execsql config_set_profile_value profile http HTTPPrestartApts replace 4
execsql config_set_profile_value profile config NumATMThreads replace 2
execsql config_set_profile_value profile config CaptureBufferMaxMemoryUsage replace 3gb

Where, profile is the name of the Collector profile that needs to be updated.

If the collector system has a minimum of 24 cores and at least 32GB of RAM, enter the following command as the RUEI_USER on the reporter system:

execsql config_set_profile_value profile http HTTPPrestartApts replace 4
execsql config_set_profile_value profile config NumATMThreads replace 4
execsql config_set_profile_value profile config CaptureBufferMaxMemoryUsage replace 3gb

Where, profile is the name of the Collector profile that needs to be updated.

Obtaining the Profile Name

To view the profile name on RUEI console, go to Configuration> Security, and then select Collector Profiles or run the following command:

execsql config_get_profiles