3.11 Returning the Rack to Service

Use the following procedure to ensure the new hardware is correctly configured and ready for use:

1. Verify the RDMA Network Fabric cables are connected and secure.

   • For RoCE, run the verify_roce_cables.py script, available from My Oracle Support.
   • For InfiniBand, run the /opt/oracle.SupportTools/ibdiagtools/verify-topology command.

2. Run the Oracle Exadata Database Machine HealthCheck utility using the steps described in My Oracle Support note 1070954.1.

3. Verify the instance additions using the following commands:

   srvctl config database -d dbm
   srvctl status database -d dbm

4. Check the cluster resources using the following command:

   crsctl stat res -t

5. Ensure the original configuration summary report from the original cluster deployment is updated to include all servers. This document should include the calibrate and network verifications for the new rack, and the RDMA Network Fabric cable checks.

6. Conduct a power-off test, if possible. If the new Exadata Storage Servers cannot be powered off, then verify that the new database servers with the new instances can be powered off and powered on, and that all processes start automatically.

   Note:

   Ensure the Oracle ASM disk rebalance process has completed for all disk groups. Connect to an Oracle ASM instance and issue the following command:

   SELECT * FROM gv$asm_operation;

   No rows should be returned by the command.

7. Review the configuration settings, such as the following:

   • All parallelism settings
   • Backup configurations
   • Standby site, if any
   • Service configuration
   • Oracle Database File System (DBFS) configuration, and mount points on new servers (not required for X7 and later servers)
   • Installation of Oracle Enterprise HugePage Manager agents on new database servers
   • HugePages settings

8. Incorporate the new cell and database servers into Oracle Auto Service Request (ASR).

9. Update Oracle Enterprise Manager Cloud Control to include the new nodes.