- Exascale User's Guide
- Oracle Exadata Exascale System Administration
- Administer the Storage Devices in an Exascale Storage Pool
- Proactively Replacing a Storage Device
2.6.2 Proactively Replacing a Storage Device
You may need to proactively replace a storage device that has not failed.
For example, you may choose to replace a device after Exadata detects a temporary performance anomaly (temporary confinement), or you may need to replace a functioning device for any other reason.
The following steps outline the procedure to proactively replace a storage device:
- Prepare the storage device for replacement.
Use the CellCLI
ALTER PHYSICALDISK
command to prepare the system for replacement of the specified storage device. You can choose whether or not to maintain redundancy throughout the device replacement process.-
If you maintain redundancy throughout the device replacement process, Exascale moves data off any affected pool disk and rebalances the associated storage pool. This option is generally recommended and provides the best protection from any other failure that could occur while replacing the device. However, it comes with the cost of moving the affected data and rebalancing the storage pool.
To choose this option, add the
MAINTAIN REDUNDANCY
clause to theALTER PHYSICALDISK ... DROP FOR REPLACEMENT
command. For example, you can use the following command to prepare storage device0:5
for replacement while maintaining redundancy:CellCLI> alter physicaldisk 0:5 drop for replacement maintain redundancy
-
If you choose not to maintain redundancy throughout the device replacement process, Exascale only performs a quick redundancy check before the device is taken offline. The redundancy check ensures that another copy of data is available for any affected pool disk. This option allows for quicker device replacement using reduced redundancy. However, it comes with an increased risk of data loss or a storage pool outage if another failure occurs while replacing the device. Consequently, this option is only recommended where data loss can be tolerated (for example, a non-critical test system) or where the increased risk associated with reduced redundancy is mitigated by some other means (for example, the data is replicated using Oracle Data Guard).
To choose this option, use the
ALTER PHYSICALDISK ... DROP FOR REPLACEMENT
command without theMAINTAIN REDUNDANCY
clause. For example, you can use the following command to prepare storage device0:5
for replacement without maintaining redundancy:CellCLI> alter physicaldisk 0:5 drop for replacement
-
- Ensure that the storage server Do Not Service LED is not lit.
- Ensure that the device is ready for removal.
-
If the failed device is a hard disk drive (HDD) or flash drive located in one of the hot-swappable drive bays in the front of the server, ensure that the blue OK to Remove LED on the device is lit before removing the device.
-
If the failed device is a hot-swappable flash card contained inside the server, ensure that the power LED on the flash card is not lit before removing the device. Starting with Exadata Storage Server X7-2, all storage server models contain hot-swappable flash cards.
-
- Remove the failed storage device and install the replacement.
See the associated server hardware guide for additional details about physical hardware replacement.
- Wait for the server to recognize the replaced device.
When you physically replace a hot-swappable storage device, it may take a few minutes for the server to recognize the new device.
- Confirm the status of the replacement device.
Use the CellCLI
LIST PHYSICALDISK
command to confirm that the status of the replacement device isnormal
.For example:
CellCLI> list physicaldisk 0:5 detail name: 0:5 deviceName: /dev/sdi diskType: HardDisk enclosureDeviceId: 0 luns: 0_5 makeModel: "WDC W7222A520ORA022T" physicalFirmware: A7B0 physicalInsertTime: 2023-09-01T12:00:25-07:00 physicalInterface: sas physicalSerial: 75X8RD physicalSize: 20.009765625T slotNumber: 5 status: normal
- Monitor the storage pool rebalance operation.
After the storage device is replaced, the Exadata cell disks and grid disks that existed on the original device are re-created on the new device. If any grid disk is designated as an Exascale pool disk, it is added to the storage pool and the storage pool is automatically rebalanced.
Use the ESCLI
lsstoragepooloperation
command to monitor the storage pool rebalance operation. - Confirm that Exascale is using the
replacement device.
Use the ESCLI
lspooldisk
command and examine thestatus
attribute.Initially, as the replacement device comes online, the pool disk status is briefly set to
BEING ADDED
. However, the status value transitions toONLINE
as Exascale reintegrates the replacement device.