6.3.2.2 Monitoring XRMEM Cache Using Database Statistics and Wait Events

Note:

The availability of a specific statistic or wait event is subject to the version of Oracle Database being used.

The following table describes database statistics that are useful for monitoring XRMEM cache. The statistics are available in various dynamic performance views, including V$SYSSTAT, and may be displayed in the Global Activity Statistics or Instance Activity Statistics section of an AWR report.

Statistic	Description
cell RDMA reads	The number of successful XRMEM cache read requests using RDMA
cell RDMA reads eligible	The number of database reads that meet basic eligibility criteria for RDMA
cell RDMA read hash table probes	The total number of RDMA hash table probes issued to determine the presence of data in XRMEM cache. Each eligible read is usually associated with a hash table probe.
cell RDMA reads issued	The total number of RDMA reads issued to retrieve data from XRMEM cache. An RDMA read is usually issued after each successful hash table probe.
cell RDMA probe failures - hash table buffer allocation failed cell RDMA probe failures - IPCDAT metadata allocation failed cell RDMA probe failures - IPCDAT errors	The total number of RDMA hash table probes that failed. Each statistic covers a specific failure reason. The sum of these failures accounts for most of the difference between cell RDMA read hash table probes and cell RDMA reads issued. Some RDMA hash table probe failures are normal while XRMEM cache is being initialized. However, frequent failures or a large number of failures may indicate a problem requiring further investigation.
cell RDMA read failures - lease expired cell RDMA read failures - client registration errors	The total number of RDMA reads that failed. Each statistic covers a specific failure reason. The sum of these failures accounts for most of the difference between cell RDMA reads issued and cell RDMA reads. Normally, some RDMA read failures may occur. However, frequent failures or a large number of failures may indicate a problem requiring further investigation.
cell RDMA reads rejected - ineligible	The number of database reads that failed basic eligibility criteria for RDMA
cell xrmem cache read hits	The number of non-RDMA read requests processed by cellsrv resulting in a XRMEM cache hit

The following table describes database wait events that are useful for monitoring XRMEM cache. The wait events are visible in various dynamic performance views, including V$SESSION, V$SYSTEM_EVENT and V$SESSION_EVENT, and may be displayed in the Wait Event sections of the AWR report.

Wait Event	Description
cell list of blocks physical read	This wait event occurs during recovery or during buffer pre-fetching (rather than performing multiple single-block reads). It is used to monitor database blocks that must be changed as part of recovery and are read in parallel for the database. In V$SESSION, records associated with this event include additional parameters: P1 contains the relevant storage server hash number, which corresponds to V$CELL.CELL_HASHVAL. P2 identifies the disk hash number for the grid disk that contains the data, which corresponds to V$ASM_DISK.HASH_VALUE. P3 specifies the number of bytes processed during the I/O read operation. This wait event is equivalent to db file parallel read for a cell.
cell list of blocks read request	This is a placeholder wait event associated with cell list of blocks physical read, which is visible only during the wait period. After the wait event ends, the placeholder is typically converted to cell list of blocks physical read.
cell multiblock physical read	This wait event represents the time taken to perform all the I/Os for a multi-block database read. In V$SESSION, records associated with this event include additional parameters: P1 contains the relevant storage server hash number, which corresponds to V$CELL.CELL_HASHVAL. P2 identifies the disk hash number for the grid disk that contains the data, which corresponds to V$ASM_DISK.HASH_VALUE. P3 specifies the number of bytes processed during the I/O read operation. This wait event is equivalent to db file scattered read for a cell.
cell multiblock read request	This is a placeholder wait event associated with cell multiblock physical read, which is visible only during the wait period. After the wait event ends, the placeholder is typically converted to cell multiblock physical read.
cell single block physical read	This wait event represents the time taken to perform a single block database I/O, equivalent to db file sequential read for a cell. This wait event does not include I/O from Exadata Smart Flash Cache, I/O from XRMEM cache, or database I/O using a Remote Direct Memory Access (RDMA) read. In V$SESSION, records associated with this event include additional parameters: P1 contains the relevant storage server hash number, which corresponds to V$CELL.CELL_HASHVAL. P2 identifies the disk hash number for the grid disk that contains the data, which corresponds to V$ASM_DISK.HASH_VALUE. P3 specifies the number of bytes processed during the I/O read operation.
cell single block physical read: RDMA	This wait event represents the time taken to perform a single block database I/O using a Remote Direct Memory Access (RDMA) read. In V$SESSION, records associated with this event include additional parameters: P1 contains the relevant storage server hash number, which corresponds to V$CELL.CELL_HASHVAL. P2 identifies the disk hash number for the grid disk that contains the data, which corresponds to V$ASM_DISK.HASH_VALUE. P3 specifies the number of bytes processed during the I/O read operation. This wait event was introduced in the May 2022 Oracle Database release updates and is present in Oracle Database versions 19.15.0.0.220419, 21.6.0.0.220419, and later. Previously, cell single block physical read included these waits.
cell single block physical read: xrmem cache	This wait event represents the time taken to perform a single block database I/O from XRMEM cache. Effective use of XRMEM cache coincides with extremely low latency for this wait event, which is typical for operations using RDMA. In V$SESSION, records associated with this event include additional parameters: P1 contains the relevant storage server hash number, which corresponds to V$CELL.CELL_HASHVAL. P2 identifies the disk hash number for the grid disk that contains the data (not the cache location), which corresponds to V$ASM_DISK.HASH_VALUE. P3 specifies the number of bytes processed during the I/O read operation.
cell single block read request	This is a placeholder wait event associated with a single block database I/O that is visible only during the wait period. After the wait event ends, the placeholder is converted to the appropriate wait event, which is typically one of the cell single block physical read events.
cell interconnect retransmit during physical read	This wait event appears during retransmission for an I/O of a single-block or multiblock read. The cell hash number in the P1 column in the V$SESSION_WAIT view is the same cell identified for cell single block physical read and cell multiblock physical read events. The P2 column contains the subnet number to the cell, and the P3 column contains the number of bytes processed during the I/O read operation.