Managing Disk Space Usage

When you install Oracle SES, various components of Oracle SES consume server space. The different components that consume space include:

Destination Path (ORACLE_BASE): The root directory or location where Oracle SES is installed.
Data storage location: The directory or location where Oracle SES stores its data. The data includes internal database files and crawler log files among others. This location should not be within the ORACLE_HOME directory.
Cache directory location: Location or directory to store cached data. By default the cached data is stored within the database. You can however specify a different location to store this data using the Crawler Configuration page.
Storage areas: Users can create storage areas on a need basis.
Crawler log location: Location or directory to store crawler logs. By default, the crawler logs are stored within the data storage location. You can however specify a different location to store the logs using the Crawler Configuration page.

You can set space usage quota for these components using the Administration API. Among the different components, data storage location consumes the most disk space over time. To address this, Oracle SES sets a default space quota for this component during installation. The default space quota for the data storage location is the initial space allocated by Oracle SES during installation, plus half of the available free disk space at the time of installation. For example, if at the time of installation, the available free disk space is 350 GB, then Oracle SES allocates 175 GB of disk space to data storage location.

Note that you can use the storageArea API to modify the default space quota for data storage location. Using this API, you can remove the quota, in which case the data storage location can utilize the entire free disk space. However, if Oracle SES uses up the entire disk space, then the crawler fails and the Oracle SES instance crashes. It is not possible to recover from this state even if you clear up space. Hence, Oracle recommends that you do not remove the preset quota.

Oracle SES calculates the space usage of all the storage components on a periodic basis. You can define the frequency of the periodic checks. For example you can set it to be an hourly, daily, or a weekly task depending on your usage. Additionally, you can calculate the space usage at any time by calling the spaceCalculator Administration API.

A variety of tasks such as crawling, index optimization, metadata backup, and auto merge among others consume space. When the space usage reaches 80% of the defined space usage quota, Oracle SES raises a warning. When the usage exceeds the quota, Oracle SES raises an alert and performs the following operations:

It immediately stops all crawler activities. Note that some crawler activities cannot be stopped immediately. Therefore, there is likely to be a slight delay before all crawler activities are stopped.
It allows other tasks to run till completion, and then disables them.
It disables all the pre scheduled crawler activities.

After you clear up the space or increase the space usage quota, you can resume all the disrupted activities including the stopped On-Demand crawler schedules and other disabled tasks. You must call the Administration API resumeAllSpaceConsumingTasks. If you do not call this API, then Oracle SES cannot restart any of the stopped activities, even if you create free space.

To perform space management tasks, you must use the Administration API. See Oracle Secure Enterprise Search Administration API Guide for more information.