6 Servicing Oracle Private Cloud Appliance Components

This chapter explains the service procedures for Oracle Private Cloud Appliance in case a failure occurs. Optionally, you can configure the system with Oracle Auto Service Request (ASR), which generates a service request with Oracle automatically when it detects a hardware fault. Certain components of Private Cloud Appliance are customer-replaceable. These components are listed in this chapter, along with the replacement instructions.

Oracle Auto Service Request (ASR)

Oracle Private Cloud Appliance is qualified for Oracle Auto Service Request (ASR), a software feature for support purposes.

When ASR is enabled, a service request is automatically created and sent to Oracle Support Services when specific Private Cloud Appliance hardware faults occur. Both the My Oracle Support email account and the technical contact for Private Cloud Appliance are automatically notified that the service request was made. ASR expedites and simplifies the delivery of replacement hardware.

ASR detects faults in the most common hardware components, such as disks, fans, and power supplies. Oracle is continuously analyzing and improving the ASR fault rules. However, ASR does not detect all possible hardware faults, and it is not a replacement for other monitoring mechanisms. In addition, a service request might not be filed automatically in some cases, for example if a loss of connectivity to ASR occurs. Administrators should monitor their systems for faults and call Oracle Support Services if they do not receive notice that a service request has been filed automatically.

To use ASR, install and configure the software, register the Private Cloud Appliance as an ASR asset, and configure the Private Cloud Appliance to send hardware fault telemetry to Oracle Support.

For more information about ASR, see the following resources:

• Oracle Auto Service Request web page: https://www.oracle.com/servers/technologies/auto-service-request.html.

• Oracle Auto Service Request user documentation: https://docs.oracle.com/cd/E37710_01/index.htm.

Installing and Configuring ASR

ASR Prerequisites

Before you install ASR, ensure the following prerequisites are satisfied:

• You have a valid My Oracle Support account.

  If necessary, create an account at https://support.oracle.com/.

• The following are set up correctly in My Oracle Support:

  • Technical contact person at the customer site who is responsible for Private Cloud Appliance.

  • Valid shipping address at the customer site where the Private Cloud Appliance is located, so that parts are delivered to the site where they must be installed.

• The management nodes have an active outbound Internet connection using HTTPS or an HTTPS proxy.

ASR Manager (ASRM)

Download ASR Manager (ASRM) packages to a secure location that is accessible from both management nodes. ASRM must be installed on both management nodes. Both management nodes must have an active outbound Internet connection using HTTPS or an HTTPS proxy. Failover must be configured so that the ASR Manager role is always fulfilled by the management node that also has the active role.

You can register the ASRM as a stand-alone ASRM, or as a relay to another ASRM in your network, including the Oracle Advanced Support Gateway. Registering the Private Cloud Appliance as a stand-alone ASRM means it communicates directly with the Oracle backend systems through My Oracle Support, which is the standard registration method.

Installing ASR

To install and configure ASR, see the following support documents:

• How to Install Auto Service Request (ASR) on Private Cloud Appliance (PCA) 2.4.1 or Later (Doc ID 2560988.1).

• How to Install Auto Service Request (ASR) on Private Cloud Appliance (PCA) X8 (Doc ID 2560988.1).

Adding ASR Assets

When ASR is installed on your Private Cloud Appliance, log in to your My Oracle Support account and approve the Private Cloud Appliance as a new asset. See How To Manage and Approve Pending Oracle Auto Service Request (ASR) Assets In My Oracle Support (Doc ID 1329200.1).

Enabling ASR

To enable ASR, the Private Cloud Appliance components must be configured to send hardware fault telemetry to the ASR Manager (ASRM) software. See Phone Home Service.

Replaceable Components

According to Oracle's Component Replacement Policy, the replaceable components in your system are designated as either field-replaceable units (FRUs) or customer-replaceable units (CRUs).

• A part designated as a FRU must be replaced by an Oracle-qualified service technician.

• A part designated as a CRU can be replaced by a person who is not an Oracle-qualified service technician.

Servicing instructions included in this Oracle Private Cloud Appliance Administration Guide are focused primarily on CRUs. For FRU replacement, please contact Oracle.

Rack Components

The following table lists the replaceable components of the Oracle Private Cloud Appliance rack.

Note:

For the current list of replacement parts and their manufacturing part numbers, refer to the Oracle Private Cloud Appliance components list in the Oracle System Handbook.

You access the Oracle System Handbook using this link: https://support.oracle.com/handbook_private/.

Click Current Systems, then click your generation of Oracle Private Cloud Appliance Hardware to open the main product page in the System Handbook.

Table 6-1 Replaceable Oracle Private Cloud Appliance Rack Components

Component Description	Hot-Swap
Oracle Rack Cabinet 1242:	No
Jumper Cable C13-C14, 2m	Yes
Ethernet Cable, Category 5/5E, 10ft, Black	Yes
Ethernet Cable, Category 5/5E, 10ft, Blue	Yes
Ethernet Cable, Shielded, Category 5E, 1m, Grey	Yes
Ethernet Cable, Category 5, 8ft, Black	Yes
Ethernet Cable, Category 5, 8ft, Green	Yes
Ethernet Cable, Category 5, 8ft, Yellow	Yes
Active Optical Cable, Blue, 3m	Yes
10Gbps QSFP to QSFP Cable, Passive Copper, 3m	Yes
QSFP28 Cable, 30AWG, Passive Copper, 3m	Yes
QSFP28 Cable, 30AWG, Passive Copper, 1m	Yes
1U/2U Screw-Mount Slide Rail Kit	No
1U/2U Cable Management Arm (Snap-in)	No
Power Distribution Units (PDUs):
15KVA Single-Phase PDU, North America	Yes
15KVA Three-Phase PDU, North America	Yes
15KVA Three-Phase PDU, International	Yes
22KVA Single-Phase PDU, North America	Yes
22KVA Single-Phase PDU, International	Yes
24KVA Three-Phase PDU, North America	Yes
24KVA Three-Phase PDU, International	Yes

For rack-level component servicing instructions, see Servicing the Oracle Private Cloud Appliance Rack System.

Oracle Server X9-2 Components

The following table lists the replaceable components of the Oracle Server X9-2 compute nodes.

Note:

For the current list of replacement parts and their manufacturing part numbers, refer to the Oracle Private Cloud Appliance components list in the Oracle System Handbook.

You access the Oracle System Handbook using this link: https://support.oracle.com/handbook_private/.

Click Current Systems, then click your generation of Oracle Private Cloud Appliance Hardware to open the main product page in the System Handbook.

Table 6-2 Replaceable Oracle Server X9-2 Components

Component Description	Hot-Swap
Motherboard Assembly	No
Fan Modules	Yes
PCI Risers	No
PCI Cards	No
Type A269.2 1200 Watt AC Input Power Supply	Yes
(2) Thirty-two-core Intel Xeon P-8358 processors (2.6 GHz), 250W	No
CPU Heatsink	No
Front LED Indicator Module	No
Disk Backplane Assembly	No
Internal M2 flash SSDs	No
DDR4 DIMM, 32GB	No
DDR4 DIMM, 64GB	No
Temperature sensor	No
Dual port 100Gbps Ethernet OCP V3 Network Interface Card	No
(2) 240GB M.2 SATA boot devices configured as RAID 1 mirror	No
System Battery	No
NVMe Cables	No

For Oracle Server X9-2 component servicing instructions, see Servicing a Compute Node.

Oracle Server X8-2 Components

The following table lists the replaceable components of the Oracle Server X8-2 compute nodes.

Note:

For the current list of replacement parts and their manufacturing part numbers, refer to the Oracle Private Cloud Appliance components list in the Oracle System Handbook.

You access the Oracle System Handbook using this link: https://support.oracle.com/handbook_private/.

Click Current Systems, then click your generation of Oracle Private Cloud Appliance Hardware to open the main product page in the System Handbook.

Table 6-3 Replaceable Oracle Server X8-2 Components

Component Description	Hot-Swap
Motherboard Assembly	No
Quad Counter Rotating Fan Module	Yes
1-Slot PCI Express Riser Assembly	No
2-Slot PCI Express Riser Assembly	No
Type A266 800/1200 Watt AC Input Power Supply	Yes
Sixteen-core Intel Xeon G-5218 processor (2.3 GHz), 125W	No
Twenty-four-core Intel Xeon P-8260 processor (2.4 GHz), 165W	No
CPU Heatsink	No
2.5" Disk Cage Front Indicator Module	No
8-Slot 2.5" Disk Backplane Assembly	No
1.2TB - 10000 RPM SAS-3 Disk Assembly with 1 bracket	Yes
DDR4 DIMM, 32GB	No
DDR4 DIMM, 64GB	No
Dual port 100Gbps Ethernet PCI Express 3.0 Host Channel Adapter (CX-5)	No
Dual port 32Gbps Fibre Channel PCI Express 3.0 Host Bus Adapter (optional component)	No
8-Port 12Gbps SAS-3 RAID PCI Express HBA	No
System Battery	No
Cable kit	No

For Oracle Server X8-2 component servicing instructions, see Servicing a Compute Node.

Oracle ZFS Storage Appliance ZS7-2 Components

The following table lists the replaceable components of the Oracle ZFS Storage Appliance ZS7-2.

Note:

For the current list of replacement parts and their manufacturing part numbers, refer to the Oracle Private Cloud Appliance components list in the Oracle System Handbook.

You access the Oracle System Handbook using this link: https://support.oracle.com/handbook_private/.

Click Current Systems, then click your generation of Oracle Private Cloud Appliance Hardware to open the main product page in the System Handbook.

Table 6-4 Replaceable Oracle ZFS Storage Appliance ZS7-2 Components

Component Description	Hot-Swap
Oracle ZFS Storage Appliance ZS7-2 Storage Head:	NA
2.3GHz Intel 18-Core Xeon G-6140, 140W	No
Pre-greased CPU Heatsink	No
64GB DDR4 DIMM	No
7.68TB SAS-3 Disk Assembly	Yes
1.2TB - 10000 RPM SAS-3 Disk Assembly	Yes
Fortville dual PCIe 40Gb Ethernet Adapter	No
2.5" Disk Cage Front Indicator Module	No
12-Slot 2.5" Disk Backplane Assembly	No
Interlock Cable, 125mm	No
Cable Kit	No
Dual Counter Rotating Fan Module	Yes
System Board Assembly	No
3V lithium coin cell battery	No
Type A266 800/1200 Watt AC Input Power Supply	Yes
Cluster Heartbeat Assembly	No
8-Port 12Gbps SAS HBA	No
4x4 Port 12Gbps SAS-3 PCI Express HBA	No
Oracle Storage DE3-24C Disk Shelf:	NA
580 Watt AC Input Power Supply	Yes
12Gbps SAS-3 I/O Controller Module	Yes
4RU Chassis Assembly with Midplane	No
36-Pin Mini SAS3 HD Cable, SFF-8644 to SFF-8644, 3M	Yes
DE3-24C Mounting Rail Kit	No
14TB - 7200 RPM SAS-3 Disk Drive Assembly	Yes
200GB SAS-3 Solid State Drive Assembly	Yes

For Oracle ZFS Storage Appliance ZS7-2 component servicing instructions, see Servicing the Oracle ZFS Storage Appliance ZS7-2.

Preparing Oracle Private Cloud Appliance for Service

This section describes safety considerations and prerequisites for component replacement procedures.

Safety Precautions

For your protection, observe the following safety precautions when servicing your equipment:

• Follow all standard cautions, warnings, and instructions marked on the equipment and described in the following documents:

  • The printed document Important Safety Information for Sun Hardware Systems (7063567)

  • The Oracle Private Cloud Appliance Safety and Compliance Guide

• Follow the safety guidelines described in the Oracle Private Cloud Appliance Installation Guide:

• Follow the electrostatic discharge safety practices as described in this section.

• Disconnect all power supply cords before servicing components.

Electrostatic Discharge Safety

Devices that are sensitive to electrostatic discharge (ESD), such as motherboards, PCIe cards, drives, processors, and memory cards require special handling.

Caution:

Equipment Damage

Take antistatic measures and do not touch components along their connector edges.

• Use an antistatic wrist strap.

  Wear an antistatic wrist strap and use an antistatic mat when handling components such as drive assemblies, boards, or cards. When servicing or removing rack node components, attach an antistatic strap to your wrist and then to a metal area on the chassis. Then disconnect the power cords from the component. Following this practice equalizes the electrical potentials between you and the component.

  An antistatic wrist strap is not included in the Oracle Private Cloud Appliance shipment.

• Use an antistatic mat.

  Place ESD-sensitive components such as the motherboard, memory, and other PCB cards on an antistatic mat.

  The following items can be used as an antistatic mat:

  • Antistatic bag used to wrap an Oracle replacement part

  • An ESD mat (orderable from Oracle)

  • A disposable ESD mat (shipped with some replacement parts or optional system components)

Servicing the Oracle Private Cloud Appliance Rack System

This section provides instructions to service replaceable components (CRUs/FRUs) in the appliance rack. Before starting any service procedure, read and follow the guidelines in Preparing Oracle Private Cloud Appliance for Service.

Powering Down Oracle Private Cloud Appliance (When Required)

Some service procedures may require you to power down the Oracle Private Cloud Appliance. Perform the following steps to manually power down the system.

Caution:

Whenever a hardware system must be powered down, make sure that the virtual machines hosted by that system are shut down first. If you power down the appliance with running virtual machines, these will be in an error state when the system is returned to operation.

For details, consult the Oracle VM Manager User's Guide.

• Stop Virtual Machines

• Stop Server

Shutting Down the Oracle VM Environment

1. Log in to Oracle VM Manager and open the Servers and VMs tab.

2. Using the navigation tree, select each virtual machine and click Stop to shut it down gracefully.

   If the applications hosted by your VMs require the services and machines to be shut down in a particular order, respect those requirements just like you would with physical machines.

   Once the VMs have been shut down, you can proceed to power off the compute nodes.

3. Using the navigation tree, select each compute node and click Stop Server to shut it down gracefully.

4. Using SSH and an account with superuser privileges, log into the active management node at the management virtual IP address. Stop Oracle VM Manager by entering the command service ovmm stop.

Powering Down the System for Service

1. If, at this point, any compute nodes have not shut down properly, press the Power button on the running compute nodes in order to shut them down gracefully.

2. Press the Power button on the management nodes in order to shut them down gracefully.

   Once the servers are powered off, you can proceed to power off the storage appliance.

3. Press the Power button on the storage server heads attached to the chassis of the storage device.

4. Toggle the rack Power switches to the Off position.

   Note:

   The Ethernet switches do not have power switches. They power off when power is removed, by way of the power distribution unit (PDU) or at the breaker in the data center.

Returning the System to Operation After Service or Unplanned Outage

1. Toggle the power distribution unit (PDU) circuit breakers of both PDUs to the On position.

2. Wait at least two minutes to allow the PDUs to complete their power-on sequence.

   The Ethernet switches are powered on with the PDUs.

3. Press the Power button on the storage server heads.

   Wait approximately two minutes until the power-on self-test completes, and the Power/OK LED on the front panel lights and remains lit.

4. Press the Power button on the management nodes.

   The management node that completes booting first assumes the active role.

   Note:

   Compute nodes do not power on automatically like the internal ZFS Storage Appliance, switches and other components. Make sure that the management nodes and internal storage are up and running, then manually power on the compute nodes.

5. When the management nodes are up, press the Power button on the compute nodes.

   Caution:

   The compute node ILOM policy for automatic power-on is disabled, and must remain disabled, to prevent a server from booting prematurely and disrupting the correct boot order of the appliance components.

   When all compute nodes are up, verify the status of all system components in Oracle VM Manager.

   If no components are in error state, the appliance is ready to resume normal operation.

Service Procedures for Rack System Components

For parts that are not hot-swappable, power down the Oracle Private Cloud Appliance before starting the service procedure. Generally speaking, hot-swappable components can be serviced without specific additional steps.

Table 6-5 Service Instructions for Rack System Components

Replaceable Part(s)	Oracle Server X8-2 Instructions	Oracle Server X9-2 Instructions
Power cables	NA	NA
Ethernet cables	NA	NA
Cable management arms (CMAs) (Oracle-qualified service technician only)	Remove the Cable Management Arm Install the Cable Management Arm	Remove the Cable Management Arm Install the Cable Management Arm
Slide rails (Oracle-qualified service technician only)	To service the slide rails, the server must be removed from the rack. For instructions, refer to: Remove the Server From the Rack Reinstall the Server Into the Rack For slide rail installation instructions, refer to the section Attach the Slide-Rails. To remove the slide rails, reverse the installation steps.	To service the slide rails, the server must be removed from the rack. For instructions, refer to: Remove the Server From the Rack Reinstall the Server Into the Rack For slide rail installation instructions, refer to the section Attach the Slide-Rails. To remove the slide rails, reverse the installation steps.

Servicing a Compute Node

This section provides instructions to service replaceable components in a supported Oracle Server X8-2 or Oracle Server X9-2 compute node. Before starting any service procedure, read and follow the guidelines in Preparing Oracle Private Cloud Appliance for Service.

Powering Down a Compute Node for Service (When Required)

If you need to execute a service procedure that requires the compute node to be powered down, follow these instructions:

Placing a Compute Node Into Maintenance Mode

Before a compute node compute node can be powered down, it must be placed into maintenance mode from within Oracle VM Manager. As a result, all virtual machines running on the compute node are automatically migrated to other servers in the Oracle VM server pool, if they are available. Information on maintenance mode is provided in the Oracle VM Manager User's Guide section entitled Edit Server.

1. Log in to the Oracle VM Manager Web UI.

   For details, refer to the section "Logging in to the Oracle VM Manager Web UI".

   1. Enter the following address in a Web browser: https://manager-vip:7002/ovm/console.

      Replace manager-vip with the virtual IP address, or corresponding host name, that you have configured for your management nodes during installation.

   2. Enter the Oracle VM Manager user name and password in the respective fields and click OK.

2. In the Servers and VMs tab, select the Oracle VM Server in the navigation pane. Click Edit Server in the management pane toolbar.

   The Edit Server dialog box is displayed.

3. Select the Maintenance Mode check box to place the Oracle VM Server into maintenance mode. Click OK.

   The Oracle VM Server is in maintenance mode and ready for servicing.

4. When the compute node is ready to rejoin the Oracle VM server pool, perform the same procedure and clear the Maintenance Mode check box.

Powering Down the System

These steps briefly describe the procedure. For detailed instructions, refer to:

• For Oracle X8-2 Servers: Perparing the Server for Component Replacement
• For Oracle X9-2 Servers: Preparing the Server for Component Replacement

1. Power down the server gracefully whenever possible.

   The easiest way is to press and quickly release the Power button on the front panel.

2. Perform immediate shutdown only if the system does not respond to graceful power-down tasks.

   Caution:

   An immediate power down might corrupt system data, therefore, only use this procedure to power down the server after attempting the graceful power down procedure.

3. Disconnect the power cables from the server.

4. Extend the server to the service position.

5. All service operations can be performed while the server is in the service position.

Returning the System to Operation

These steps briefly describe the procedure. For detailed instructions, refer to:

• For Oracle X8-2 Servers: Returning the Server to Operation
• For Oracle X9-2 Servers: Returning the Server to Operation

1. If the top cover was removed to service a component, reinstall the top cover on the server.

2. If the server was removed, reinstall it into the rack.

3. Return the server to its normal operational position in the rack, making sure the CMA is correctly installed.

4. Reconnect data cables and power cords.

5. Power on the server.

Service Procedures for Compute Node Components

For parts that are not hot-swappable, power down the compute node before starting the service procedure. If the server is in use in the Oracle VM environment, place it in maintenance mode first. This protects your virtual infrastructure against data corruption, and allows it to remain in service as long as the configuration of your environment allows it.

Generally speaking, hot-swappable components can be serviced without specific additional steps for Oracle Private Cloud Appliance. Follow the applicable procedure in the Service Manual. The following tables provide links to each service procedure and indicates whether parts are hot-swappable or require the component to be taken offline and powered down.

Table 6-6 Service Procedures for Oracle Server X8-2 Components

Replaceable Part(s)	Hot-Swap	URL
Storage drives	Yes	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gquak.html
Fan Modules	Yes	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gquhg.html
Power supplies	Yes	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gqunc.html
DIMMs (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gqvkr.html
PCI Express risers (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gqvft.html
PCI Express cards (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gqvjk.html
Battery	No	https://docs.oracle.com/en/servers/x86/x8-2/service-manual/gqviw.html

Table 6-7 Service Procedures for Oracle Server X9-2 Components

Replaceable Part(s)	Hot-Swap	URL
Storage drives	Yes	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-storage-drives-cru.html
Fan Modules	Yes	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-fan-modules-cru.html
Power supplies	Yes	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-power-supplies-cru.html
Internal M.2 SSDs	No	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-internal-m-2-flash-ssds-cru.html
DIMMs (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-dimms-cru.html
PCI Express risers (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-pcie-risers-cru.html
PCI Express cards (Oracle-qualified service technician only)	No	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-pcie-cards-cru.html
Battery	No	https://docs.oracle.com/en/servers/x86/x9-2/service-manual/servicing-battery-cru.html

Servicing the Oracle ZFS Storage Appliance ZS7-2

This section provides instructions to service replaceable components (CRUs/FRUs) in the Oracle ZFS Storage Appliance ZS7-2. Before starting any service procedure, read and follow the guidelines in Preparing Oracle Private Cloud Appliance for Service.

Powering Down the Oracle ZFS Storage Appliance ZS7-2 for Service (When Required)

If you need to execute a service procedure that requires the Oracle ZFS Storage Appliance ZS7-2 to be powered down, follow these instructions:

Powering Down the Storage Head/Controller

Because the storage controllers are clustered, there is no loss of access to storage when one controller is powered down for service. Performing a graceful shutdown ensures that data is saved and not corrupted, and that resources are assigned to the other controller in the storage head cluster. Power down a controller for component replacement using one of the following methods:

• Log in to the UI by using the server's IP address in the appliance management network:

  1. In your browser, enter https://ipaddress:215.

  2. Log in as root, using the system-wide Oracle Private Cloud Appliance password.

  3. Click the Power icon on the left side under masthead.

• Alternatively, SSH in to the storage appliance as root, and enter the command maintenance system poweroff.

If graceful shutdown as described above is not possible, use the power button:

• Use a pen or non-conducting pointed object to press and release the Power button on the front panel.

• SSH or use a serial connection to log in to the service processor (SP), and then issue the command stop /SYS.

• If the server did not respond, initiate an emergency shutdown. Press and hold the Power button for at least four seconds until the Power/OK status indicator on the front panel flashes, indicating that the storage controller is in standby power mode. To completely remove power, disconnect the AC power cords from the rear panel of the storage controller.

  Caution:

  An emergency shutdown causes all applications and files to be closed abruptly without saving. You might corrupt or lose system data, or lose the server configuration (the resources assigned to it) during an immediate power down.

Note:

Powering down the disk shelf is not required

All replaceable components in the disk shelf are hot-swappable. The disk shelf itself does not need to be powered down for the replacement of defective components.

However, do not remove a component if you do not have an immediate replacement. The disk shelf must not be operated without all components in place.

Powering on the Storage Appliance

Caution:

The disk shelf must not be operated without all components in place.

1. Connect any storage head power and data cables you removed to service a component.

2. Power on the server by pressing the Power button on the front panel.

   If you are not physically located at the system, use either of these ILOM methods instead:

   • Log in to the Oracle ILOM web interface.

     Click Host Management > Power Control, and in the Actions list click Power On.

   • Log in to the Oracle ILOM command-line interface (CLI).

     At the CLI prompt, type the following command: start /System.

3. When the controller is powered on and the power-on self-test (POST) code checkpoint tests have completed, the green Power/OK status indicator on the front panel lights and remains lit.

4. If you performed a graceful shutdown earlier, return resources to the server that was just serviced.

   1. Log into the web UI for the server that was not serviced.

   2. Go to Configuration > Cluster.

   3. Click Failback.

   Note:

   For information about configuring the clustered servers and attached disk shelves, see the "Oracle ZFS Storage System Administration Guide" for the appropriate software release.

Service Procedures for Oracle ZFS Storage Appliance ZS7-2 Components

For parts that are not hot-swappable, power down the Oracle ZFS Storage Appliance ZS7-2 before starting the service procedure.

Attention:

If you need to execute a service procedure that interrupts the connection between virtual machines and their virtual disks, shut down the virtual machines in Oracle VM Manager prior to servicing the storage hardware. Disconnecting a running virtual machine from its disks may cause data corruption.

Generally speaking, hot-swappable components can be serviced without specific additional steps for Oracle Private Cloud Appliance. Follow the applicable procedure in the Service Manual. The following table provides links to each service procedure and indicates whether parts are hot-swappable or require the component to be taken offline and powered down.

Table 6-8 Service Procedures for Oracle ZFS Storage Appliance ZS7-2 Components

Replaceable Part(s)	Hot-Swap	URL
Storage head hard drives	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbno.html#scrolltoc
Disk shelf drives	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/goxds.html#scrolltoc
Fan modules	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbxa.html#scrolltoc
Storage head power supplies	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbon.html#scrolltoc
Disk shelf power supplies	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/goxbs.html#scrolltoc
Memory modules (Oracle-qualified service technician only)	No	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbou.html#scrolltoc
PCI Express cards (Oracle-qualified service technician only)	No	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbnz.html#scrolltoc
Battery	No	https://docs.oracle.com/cd/F13758_01/html/F13771/gtbwl.html#scrolltoc
Disk shelf I/O modules (Oracle-qualified service technician only)	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/goxeo.html#scrolltoc
Disk shelf SIM boards (Oracle-qualified service technician only)	Yes	https://docs.oracle.com/cd/F13758_01/html/F13771/goxef.html#scrolltoc

Servicing Cisco Nexus 9336C-FX2 Switch Components

This section provides instructions to service replaceable components (CRUs/FRUs) in an Cisco Nexus 9336C-FX2 Switch. Before starting any service procedure, read and follow the guidelines in Preparing Oracle Private Cloud Appliance for Service.

For parts that are not hot-swappable, power down the Cisco Nexus 9336C-FX2 Switch before starting the service procedure.

Caution:

Management, storage, VM and external network connectivity may be affected while the Cisco Nexus 9336C-FX2 Switch or an I/O module is out of service. Please take the necessary precautions.

Caution:

When replacing the entire switch assembly, begin by saving the configuration from the existing component, so that you can restore the configuration after replacement.

Generally speaking, hot-swappable components can be serviced without specific additional steps for Oracle Private Cloud Appliance. Follow the applicable procedure in the Service Manual. The following table provides links to each service procedure and indicates whether parts are hot-swappable or require the component to be taken offline and powered down.

Table 6-9 Service Procedures for Cisco Nexus 9336C-FX2 Switch Components

Replaceable Part(s)	Hot-Swap	URL
Power supplies (Oracle-qualified service technician only)	Yes	https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/hw/n9336cfx2_hig/guide/b_n9336cFX2_nxos_hardware_installation_guide/b_n9336cFX2_nxos_hardware_installation_guide_chapter_0101.html#concept_65E9CCDC546846709DF28AA295965D5C
Fan modules (Oracle-qualified service technician only)	Yes	https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/hw/n9336cfx2_hig/guide/b_n9336cFX2_nxos_hardware_installation_guide/b_n9336cFX2_nxos_hardware_installation_guide_chapter_0101.html#concept_62AF8E25C1F744AD9B2534999E521EE9

Servicing Cisco Nexus 9348GC-FXP Switch Components

This section provides instructions to service replaceable components (CRUs/FRUs) in an Cisco Nexus 9348GC-FXP Switch. Before starting any service procedure, read and follow the guidelines in Preparing Oracle Private Cloud Appliance for Service.

For parts that are not hot-swappable, power down the Cisco Nexus 9348GC-FXP Switch before starting the service procedure.

Note:

The switches are configured in high availability pairs, so if one switch fails, network traffic continues, although it may be degraded. If more than one switch is being serviced at one time, you should prepare your environment for disruption to the network connectivity.

Attention:

Internal Ethernet connectivity is affected while the component is out of service. Please take the necessary precautions.

Caution:

When replacing the entire switch assembly, begin by saving the configuration from the existing component, so that you can restore the configuration after replacement.

Generally speaking, hot-swappable components can be serviced without specific additional steps for Oracle Private Cloud Appliance. Follow the applicable procedure in the component documentation. The following table provides links to each service procedure and indicates whether parts are hot-swappable or require the component to be taken offline and powered down.

Table 6-10 Service Procedures for Cisco Nexus 9348GC-FXP Switch Components

Replaceable Part(s)	Hot-Swap	URL
Power supplies	Yes	https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/hw/n9348gcfxp_hig/guide/b_c9348gc-fxp_nxos_mode_hardware_install_guide/b_c9348gc-fxp_nxos_mode_hardware_install_guide_chapter_0101.html#concept_65E9CCDC546846709DF28AA295965D5C
Fan module	Yes	https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/hw/n9348gcfxp_hig/guide/b_c9348gc-fxp_nxos_mode_hardware_install_guide/b_c9348gc-fxp_nxos_mode_hardware_install_guide_chapter_0101.html#concept_62AF8E25C1F744AD9B2534999E521EE9