5.8 Using Automatic Maintenance for an Index
Instead of manually managing synchronization tasks for your indexes, you can automate CTX_DDL.SYNC_INDEX
operations using automatic maintenance.
5.8.1 About Automatic Maintenance
Indexes with automatic maintenance are synchronized in the background without any user intervention.
Overview
Index maintenance is the process of updating index data structures (in-memory and on-disk) as a result of performing DML operations.
Automatic maintenance is the default method for synchronizing Oracle Text CONTEXT
and search indexes (Oracle Text, JSON, and XML search indexes) that are created in Oracle Database 23ai and later releases.
Both the automatic maintenance and synchronization (SYNC
) methods involve processing pending updates, inserts, and deletes to the base table. However, the automatic maintenance and SYNC
specifications are orthogonal. Automatic maintenance uses an asynchronous maintenance framework to perform SYNC
operations in the background, and provides the following capabilities:
-
Eliminates time-based or manual SYNC operations:
In an automatic maintenance mode, IRnn background processes automatically perform index maintenance operations in an optimal manner. This feature internally determines an optimal synchronization interval (based on the DML arrival) and automatically schedules background
SYNC
operations, as required. You cannot override the automatically determined intervals.For detailed information about this background mechanism process, see Asynchronous Maintenance Framework.
-
Reduces the frequency of background jobs:
Background processes maintain indexes rather than the database scheduler. The background mechanism breaks each
CTX_DDL.SYNC_INDEX
operation into separate events (sync stages) and launches each event only when needed. -
Provides the default maintenance configuration:
These indexes do not require you to configure a
SYNC
type or set any synchronization interval. By default, indexes are configured with a combination of automatic maintenance andSYNC (MANUAL)
. No otherSYNC
settings are compatible with these indexes.Note that the
SYNC (MANUAL)
behavior is different in this mode. Unlike the regularSYNC (MANUAL)
type (where you must manually synchronize an index), hereCTX_DDL.SYNC_INDEX
is automatically called in the background at optimal intervals.
Why and When to Use Automatic Maintenance?
Oracle recommends that you use automatic maintenance in cases where sync requirements for indexes are not clear or you want to synchronize a large number of indexes in an optimal manner.
In addition to reducing the administrative tasks of managing your indexes, the benefit of using this framework is that it automatically determines when a background SYNC
operation needs to be performed, by tracking the DML queue. It also provides more control over the frequency of different background jobs running at any given time, instead of creating independent jobs for each index or index partition per pluggable database (PDB). As a result, automatic maintenance helps in reducing the workload on database resources, eliminates scheduling conflicts, and enhances query performance.
With SYNC (EVERY)
, which also enables automatic background synchronization, you must manually specify sync interval using interval-string
. Although SYNC (EVERY)
allows you to explicitly control the synchronization interval, automatic maintenance provides an efficient usage of database resources especially when supporting multiple PDBs. In addition, SYNC (EVERY)
may result in excessive launching of background sync jobs, based on the user's estimate of how frequently new index data may arrive.
What is Manual Maintenance?
Manual maintenance is a non-automatic maintenance mode that provides the pre-release 23ai synchronization behavior.
In a manual maintenance mode, you can specify SYNC
types, such as SYNC MANUAL
, SYNC EVERY interval-string
, or SYNC ON COMMIT
. The MAINTENANCE MANUAL
index parameter sets your indexes to manual maintenance.
After upgrading to a new release, existing indexes continue to use the previously specified method of synchronization. For example, after upgrading to Oracle Database 23ai, existing indexes are set to manual maintenance with the previously specified SYNC
settings. If you did not specify any SYNC
setting before the upgrade, then the index uses the default SYNC
type. That is, SYNC MANUAL
for Oracle Text CONTEXT
indexes and SYNC ON COMMIT
for JSON and XML search indexes. If required, you can manually enable automatic maintenance for such indexes.
How to Configure the MAINTENANCE Parameter?
The MAINTENANCE
parameter controls the maintenance type (mode) for your index. You can set the MAINTENANCE
parameter globally and not per partition. This means that the maintenance type specified for an index applies to all index partitions.
-
MAINTENANCE AUTO
(default): Sets your indexes to automatic maintenance.By default, you do not need to configure automatic maintenance while creating an index. This example creates a JSON search index that has the default behavior (noPARAMETERS
clause):CREATE SEARCH INDEX po_search_idx ON j_purchaseorder (po_document) FOR JSON;
This example creates a JSON search index by explicitly specifyingMAINTENANCE AUTO
using thePARAMETERS
clause:CREATE SEARCH INDEX po_search_idx ON j_purchaseorder (po_document) FOR JSON PARAMETERS ('MAINTENANCE AUTO');
-
MAINTENANCE MANUAL
: Sets your indexes to manual maintenance.This example disables automatic maintenance on a new JSON search index by specifyingMAINTENANCE MANUAL
using thePARAMETERS
clause:CREATE SEARCH INDEX po_search_idx ON j_purchaseorder (po_document) FOR JSON PARAMETERS('MAINTENANCE MANUAL');
For detailed information about configuring these parameters, see Enabling and Disabling Automatic Maintenance.
You can switch between the automatic and manual maintenance modes using ALTER INDEX
. This command alters only the synchronization options, and thus you do not need to rebuild the index. When set to manual maintenance, if you do not explicitly specify any SYNC
type, then the index uses the default SYNC
type.
5.8.2 Requirements and Restrictions for Automatic Maintenance
Review these requirements and restrictions (such as database compatibility, supported parameter combinations, and supported indexes) when using the automatic maintenance mode.
-
Database compatibility for the asynchronous maintenance framework is Oracle Database 21.0.0.0 and later.
-
The combination of automatic maintenance with the following parameters is not supported:
-
FAST_QUERY
-
ASYNCHRONOUS_UPDATE
-
TRANSACTIONAL
-
SYNC (ON COMMIT)
andSYNC (EVERY)
Running any of the preceding combinations results in an error, prompting you to use a compatible mode for your index.
-
-
Shadow indexes do not support automatic maintenance.
-
Sharded search indexes do not support automatic maintenance.
If you explicitly use the
MAINTENANCE AUTO
keyword to specify automatic maintenance (usingCREATE SEARCH INDEX
) or switch toMAINTENANCE AUTO
(usingALTER INDEX
), then anORA-2049
orORA-1591
error occurs.If you do not use the
MAINTENANCE AUTO
keyword to create or alter sharded search indexes, then aMAINTENANCE MANUAL
index is created with the followingSYNC
andOPTIMIZE
preferences:-
SYNC
:The specified
SYNC
type is used.If the
SYNC
type is not specified, then the default synchronization method is used. The default synchronization method for Oracle Text search indexes is set toSYNC (EVERY "freq=minutely; interval=3")
(indexes are synchronized every 3 minutes). The default synchronization method for JSON search indexes and XML search indexes is set toSYNC (On Commit)
. -
OPTIMIZE
:The specified
OPTIMIZE
type is used.If the
OPTIMIZE
type is not specified, then the default optimize method is used. The default optimize method is set toOPTIMIZE (MANUAL)
for all search indexes.
-
5.8.3 Asynchronous Maintenance Framework
In an automatic maintenance mode, IRnn background processes perform index maintenance operations, which provides a better scalability of background jobs and enhances query performance.
List of Maintenance Events
Each SYNC
operation consists of separate events (stages) that can concurrently run in the background.
Event | Description |
---|---|
(Sync-M) |
Reads the $B catalog table to find the next DocID, and allocates DocID to each ROWID. Next, it reads the contents of the $C commit journal, sorts it on ROWID, and decides which DocIDs must be removed or added. Next, it performs deletes and inserts on the $K mapping table. Then, it adds the removed DocIDs to the $N garbage collection table. Finally, it deletes all the rows read from $C. During a failure, these events are retried and also broadcasted to other Oracle Real Application Clusters (Oracle RAC) nodes. |
(Sync-MT) |
Specifies the Sync-M timeout events, generated on a commit. These are processed only by timeout actions and not by interrupt actions. During a timeout, the Sync-MT events are converted into the Sync-M events. |
(Sync-R) |
Assigns DocID ranges generated by Sync-M as During a failure, these events are retried and also broadcasted to other RAC nodes. |
(Sync-S) |
Schedules Sync-P based on the number of |
(Sync-P) |
Generates postings lists that contain new index data. Sync-P starts by getting a On Oracle RAC systems, concurrent events are also broadcasted to other nodes for running as the During a failure, the Sync-S events are scheduled with increased iteration. On Oracle RAC systems, the failed Sync-PC and |
(Sync-PS) |
Runs Sync-P serially. The postings are generated in SGA batches. |
(Sync-PC) |
Runs Sync-P concurrently. Can schedule multiple ranges to run concurrently and independently without contention. |
(Sync-W) |
Writes SGA batches of postings lists to disk. Sync-W events can run concurrently with itself. However, these events cannot run with Sync-W events are never broadcasted to other RAC nodes because they process SGA batches, which are local to the node that generated them. |
(Sync-C) |
Cleans up |
(Sync-I) |
Inspects index and index partitions (checks the $C and $B tables) to find if any events are missing. During a failure, the Sync-I events are not retried and are also not broadcasted to other RAC nodes. |
|
Schedules the Sync-I events for each index or index partition and the EClean events for each pluggable database (PDB). The Scheduler schedules the actual Monitor event, which is further processed by the Monitor worker. |
(EStat) |
Terminal event that is processed by Writer workers (Sync-W events). It writes event stats for all completed events in a PDB. |
(EClean) |
Cleans up persisted event stats (older than the PDB-specific threshold) from the dictionary. |
(Opti-ST) |
Prevents dealing with large postings lists gaps that are crated after the order completion of the Sync-W events. |
(Opti-S) |
Determines the maximum DocID for which there are no gaps in postings lists due to the out-of-order processing of the The Opti-S event aggregates $G token counts that are stored in $B by the Sync-W events, and it schedules the Opti-M event when the aggregate count reaches a user-specified threshold. |
(Opti-M) |
Terminal event that does not perform the actual |
Background Mechanism Process
The main Scheduler background process checks for the workload from all indexes at predefined intervals (that is, every 3 seconds by default). It then assigns the workload to a Worker background process, which reads events one at a time and processes them based on the event type. The index is synchronized immediately after the Worker process runs CTX_DDL.SYNC_INDEX
. Apart from the Scheduler and Worker processes, the Monitor background process helps in recovering lost events.
CTX_DDL.SYNC_INDEX
call performs the following steps in an order:
-
Resets all waiting events for an index or index partition:
When an event fails, it adds the event to the Waiting Queue with progressively increasing delay. Even after the issue is corrected, the event continues to wait for the current delay to elapse. To track such delays, it schedules retry events at an incremented level. This means that the Scheduler process first moves a retry event into an Event Queue. From the Event Queue, the Scheduler moves it into the Waiting Queue, then to the Ready Queue, and finally allocates the event to the Worker process.
In addition to incremental levels, each event has a retry iteration. On long retries (that is, when a retry event is not the same as the original event), it increments the iteration instead of the level and initializes the level to the iteration.
CTX_DDL.SYNC_INDEX
can force immediate re-execution of the event, which moves all relevant Waiting Queue events to the Event Queue and also resets the level and iteration. If the event fails again, then it restarts from the starting incremental level or iteration. -
Performs Sync-M in the foreground:
CTX_DDL.SYNC_INDEX
waits for the background maintenance to finish. However, instead of waiting for all events to finish, it calls Sync-M in the foreground and gets the maximum DocID that is allocated. It uses this DocID to ignore all future events. -
Schedules other stages of
SYNC
in the background:CTX_DDL.SYNC_INDEX
posts the Scheduler process so that it can immediately start processing pending events. -
Waits for the completion of background processing:
The waiting is controlled by the
locking
parameter when it is set toCTX_DDL.LOCK_WAIT
. For all other values,CTX_DDL.SYNC_INDEX
returns after completing Sync-M.The values of the
memory
,parallel_degree
,maxtime
, anddirect_path
parameters are ignored.If some background events are delayed or cannot complete, then
CTX_DDL.SYNC_INDEX
returns ORA-30608 and logs an error message in the catalog views.
Differences in SYNC Behavior Between Automatic Maintenance and Manual Maintenance
Compare the differences in the synchronization behavior between automatic maintenance and manual maintenance, and how different events are processed during a CTX_DDL.SYNC_INDEX
operation:
Behavior | Automatic Maintenance | Manual Maintenance |
---|---|---|
Background mechanism |
In an automatic maintenance mode, background processes maintain indexes. The background mechanism breaks each For example, Sync-S launches Sync-P to pick up new index data only when Sync-R generates |
In a manual maintenance mode, the DBMS_SCHEDULER background jobs maintain indexes. The background mechanism implements all sync events (Sync-M, Sync-R, and Sync-P) together as a single SYNC operation.
|
|
These indexes are preconfigured with a combination of automatic maintenance and Unlike the regular The other |
Running |
Catalog views |
|
|
Related Topics
5.8.4 Enabling and Disabling Automatic Maintenance
CONTEXT
and search indexes. Learn how to explicitly specify automatic maintenance while creating an index, or disable it to override the default behavior and enable SYNC
instead. 5.8.5 Switching between Automatic and Manual Maintenance
You can use ALTER INDEX
to switch between MAINTENANCE AUTO
and MAINTENANCE MANUAL
, without rebuilding the index. While changing modes, you must specify compatible Maintenance type and SYNC
type combinations.
Guidelines for Switching between Modes
-
MAINTENANCE AUTO
is supported withSYNC (MANUAL)
. By default, all indexes with automatic maintenance are specified with this combination. -
A combination of
MAINTENANCE AUTO
andSYNC ON COMMIT
orSYNC (EVERY)
is not supported. If you want to specifyMAINTENANCE AUTO
for indexes that also useSYNC (ON COMMIT)
orSYNC (EVERY)
, then you must first set such indexes toSYNC (MANUAL)
. -
Static dictionary view
CTX_USER_INDEXES
contains information about existing Oracle TextCONTEXT
and search indexes for the current user. For example, this query lists theSYNC
and Maintenance types for an Oracle Text index set toMAINTENANCE AUTO
:SQL> SELECT IDX_NAME, IDX_SYNC_TYPE, IDX_MAINTENANCE_TYPE FROM CTX_USER_INDEXES; IDX_NAME IDX_SYNC_TYPE IDX_MAINTENANCE_TYPE ----------------------- ---------------------------- ---------------------- DOCIDX MANUAL AUTO
Switching Indexes to Automatic Maintenance
This table provides ALTER INDEX
examples with various SYNC
types, while altering your index from MAINTENANCE MANUAL
to MAINTENANCE AUTO
:
Setting | Maintenance and SYNC Type | Example |
---|---|---|
to
|
|
|
to
|
|
|
to
|
|
|
Switching Indexes to Manual Maintenance
This table provides ALTER INDEX
examples with various SYNC
types, while altering your index from MAINTENANCE AUTO
to MAINTENANCE MANUAL
:
Setting | Maintenance and SYNC Type | Example |
---|---|---|
to
|
|
The default SYNC type for a CONTEXT index is MANUAL .
|
to
|
|
|
to
|
|
|
Related Topics
5.8.6 Monitoring Maintenance Events and Errors
The SYS
and CTXSYS
users can query catalog views to monitor the status of all background maintenance events for indexes with automatic maintenance.
CTX
and dynamic performance views to know the status of all background maintenance events that are complete or delayed.
Querying Data and Status about Maintenance Events
-
CTX_BACKGROUND_EVENTS
:This Oracle Text view displays historical information about the execution of events for the
SYS
orCTXSYS
user. -
CTX_USER_BACKGROUND_EVENTS
:This Oracle Text view displays historical information about the execution of events for the current user, based on the index owner.
-
V$TEXT_WAITING_EVENTS
:This dynamic performance view displays historical information about events that are delayed or cannot complete due to errors or contentions.
For example, you can query the index object number and name, index owner number and name, base table (or table partition) object number and name, type of event (such as SYNC-Postings
, SYNC-Mapping
, SYNC-Ranges
, and so on), status of the event (such as successful, running, failed, and so on), status of the retry iterations (such as retry delays, waiting time, and so on), elapsed time since the event started waiting, logged error messages, and so on.
Handling Errors
-
Some errors are transient and do not reproduce on a retry. Such error types do not require user intervention.
-
Some failed events may automatically succeed after a retry. If a retry event does not succeed, then try restarting the event from another event.
For example, when
SYNC-Postings
(Sync-P) fails after a retry, you can restartSYNC-Scheduler
(Sync-S) so that the system can schedule a serial or concurrent operation. A successful completion of Sync-S clears the queue for Sync-P, and Sync-P immediately runs at a starting level without overloading the system. -
If an event does not succeed even after periodic retries, then contact your database administrator.
-
To limit the load on the system due to periodic retries, the delays between successive retries may progressively increase.
To track such delays, all retry events are scheduled at an incremented level. This means that the Scheduler process first moves a retry event into an Event Queue. From the Event Queue, the Scheduler process moves it into the Waiting Queue, then to the Ready Queue, and finally allocates the event to the Worker process.
In addition to incremental levels, each event has a retry iteration. On long retries (that is, when a retry event is not the same as the original event), the iteration is incremented instead of the level and the level is initialized to the iteration.
-
On Oracle Real Application Clusters (Oracle RAC) systems, there may be errors that occur only on some nodes but not on others. In such cases, an event may successfully complete only when the event is sent to the other nodes.
Related Topics