Skip Headers
Oracle® Database Appliance Service Manual
Release 2.10 for Linux x86-64

Part Number E23342-18
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF

4 Servicing CRUs That Require Server Node Power Off

These sections describe how to service customer-replaceable units (CRUs) for the Oracle Database Appliance that require you to power off the server node. All of these CRUs are located on the server nodes.


Note ‐  The servicing of hot-swap components such as the HDDs, SDDs, power supplies, and fans is described in Chapter 3, Servicing CRUs That Do Not Require Server Power Off.

Servicing Oracle Database Appliance X3-2/X4-2 CRUs

This section describes how to service customer-replaceable units (CRUs) that require you to power off the server.


Note ‐  All of the CRUs on the storage shelf and the storage expansion shelf can be replaced while the power is on. See Oracle Database Appliance X3-2/X4-2 Storage Shelf Hot-Swappable Components for storage shelf component replacement procedures.

Note ‐  The links in the following table take you to instructions outside of the Oracle Database Appliance library to the Sun Server X3-2 and Sun Server X4-2 documentation libraries. Use your browser's Back button to return to the Oracle Database Appliance library.
ODA X3-2
ODA X4-2

Servicing Oracle Database Appliance (Original Version) CRUs

This section includes information on how to service the following Oracle Database Appliance (original version) CRUs:

Server Node CRU Component Locations

Each server node (also called a system controller) in Oracle Database Appliance (original version) includes the following replaceable components.

image:Figure showing the locations of the server node replaceable components.
Callout
Description
1
DDR3 Memory DIMMs (12)
2
PCIe Cards Mounted on PCIe Riser
3
PCIe Riser
4
Internal HBA PCIe Card
5
Battery

Servicing Memory Modules (DIMMs) (CRU)

Each server node in Oracle Database Appliance (original version) is provisioned with twelve 1333 MHz or 1600 MHz DDR3, 8GB dual rank DIMMs, for a total of 96GB.

The server node supports a maximum memory speed of 1333 MHz, even if 1600 MHz DIMMs are installed.

Before replacing a DIMM see the following:

DIMM and CPU Physical Layout

Each processor in Oracle Database Appliance (original version) has six DIMM slots numbered D0 through D5. D0 is the closest to the processor and D5 is the farthest.

Each slot should contain a 1333 MHz or 1600 MHz DDR3, 8GB dual rank DIMM. No other configurations are supported.

The server node supports a maximum memory speed of 1333 MHz, even if it is configured with 1600 MHz DIMMs.

If the system contains a mixture of 1333 MHz and 1600 MHz DIMMs, then P0 and P1 should be configured the same, so that both sides are symmetrical. For example, if you install only two 1600 MHz DIMMs, they should go into P0, D0 and P1, D0.

The following picture shows the physical layout of the DIMMs and CPUs. When viewing the server node from the front, CPU0 (P0) is on the right.

image:Figure shows DIMM physical layout on the server node.

Each processor socket supports three independent memory channels labeled channel A, B, and C, where channel A is closest to the processor socket. Each channel supports two DIMM slots, is controlled by an independent memory controller, and operates independent of the other channels.

Remove Faulty DIMMs


Note ‐ The DDR3 memory modules (DIMMs) are customer-replaceable units (CRUs) and do not require an Oracle-qualified service person for replacement.
  1. Prepare the server node for service.
    1. Attach an antistatic wrist strap.

      See Performing Electrostatic Discharge and Antistatic Prevention Measures.

    2. Remove the server node from the system.

      Power is removed from the server node when you remove it from the system. See Removing Server Nodes From the System Chassis.

    3. Remove the server node top cover.

      See Removing the Server Node Top Cover.

  2. Identify the faulty DIMM(s).
    1. To identify the location of a faulty DIMMs, press the Fault Remind button on the motherboard.

      Note ‐ When the Fault Remind button is pressed, the Fault Remind Power Good LED lights green to indicate that there is sufficient voltage present in the fault remind circuit to light the fault LEDs. If this LED fails to light when you press the Fault Remind button, the capacitor powering the fault remind circuit has lost its charge. This can happen if the Fault Remind button is pressed for a long time with fault LEDs lit or if the server node has been out of the chassis for more then 15 minutes.

      image:Figure showing location of the Fault Remind button and the Fault Remind Power Good LED.
      Callout
      Description
      1
      Fault Remind Button
      2
      Fault Remind Power Good LED (green)
    2. Note the location of faulty DIMMs.

      The amber DIMM fault LEDs light if the adjacent DIMM is faulty.


      image:Figure showing the location of the memory DIMM fault LEDs.
      • If the DIMM fault LED is off, the DIMM is operating properly.
      • If the DIMM fault Required LED is on (amber), the DIMM is faulty and should be replaced.
  3. To remove the faulty DIMM(s), do the following:
    1. Rotate both DIMM slot ejectors outward as far as they go.

      The ejectors push the DIMM partially up.

    2. Carefully lift the DIMM straight up to remove it from the socket.
      image:This figure shows the DIMM socket release and alignment.
      Callout
      Description
      1
      DIMM connector slot
      2
      DIMM connector key
      3
      DIMM ejector lever
  4. Replace each faulty DIMM with either another DIMM of the same rank size.

    For DIMM replacement instructions, see Install DIMMs.


    Graphic showing caution icon

    Caution  ‐ Do not operate the system with an empty DIMM slot. If you remove a DIMM, you must replace it before starting the server.


Install DIMMs

  1. Unpack the replacement DIMMs and place them on an antistatic mat.
  2. Ensure that the replacement DIMMs match the system requirements.

    The system supports only 1333 MHz or 1600 MHz DDR3, 8GB, dual rank DIMMs obtained from Oracle.

    For the best performance, if you combine 1333 MHz and 1600 MHz DIMMs in a single system, they should be configured so that P0 and P1 have the same mixture, in the same DIMM slots. For example, in a 1333 MHz system, if you replace P0, D0 with a 1600 MHz DIMM, you should also replace P1, D0 with a 1600 MHz DIMM.

    The server node supports a maximum memory speed of 1333 MHz, even if 1600 MHz DIMMs are installed.

  3. Ensure that the ejector tabs are in the open position.
  4. Align the replacement DIMM with the connector.

    Align the notch in the DIMM with the key in the connector. The notch ensures that the DIMM is oriented correctly.


    image:Figure showing how to install a DIMM.
  5. Push the DIMM into the connector until the ejector tabs lock the DIMM in place.

    If the DIMM does not easily seat into the connector, verify that the notch in the DIMM is aligned with the key in the connector as shown above. If the notch is not aligned, damage to the DIMM might occur.

  6. Repeat Step 3 through Step 5 until all replacement DIMMs are installed.
  7. Return the server node to operation.
    1. Install the server node top cover.

      See Installing the Server Node Top Cover.

    2. Install the server node into the system.

      See Installing a Server Node Into the System Chassis.

    3. Power on the server node.

      See Powering On Oracle Database Appliance (Original Version).

  8. Clear the DIMM faults.
    1. Log in to the server node Oracle ILOM CLI as root.
    2. To view server node faults, type the following command:

      -> show /SP/faultmgmt

      Oracle ILOM lists all known faults. For example:

      SP/faultmgmt
      Targets:
      0 (/SYS/MB/P0/D2)
      Properties:
      Commands:
      cd
      show
    3. To clear the DIMM fault, type the following command:

      -> set /SYS/MB/Px/Dn clear_fault_action=true

    For example, to clear a fault on the DIMM in location D2 on CPU0:

    -> set /SYS/MB/P0/D2 clear_fault_action=true
    Are you sure you want to clear /SYS/MB/P0 (y/n)? y
    Set 'clear_fault_action' to 'true'

Error Correction and Parity Protection

The server node's processor provides parity protection on its internal cache memories, and error-correcting code (ECC) protection of the data. The server node detects and posts the following errors to the Oracle ILOM event log:

Advanced ECC corrects up to 4 bits in error on nibble boundaries, as long as they are all in the same DRAM. If a DRAM fails, the DIMM continues to function.

To clear a fault once the faulty DIMM has been replaced, type the following in the Oracle ILOM CLI:

-> set clear_fault_action=true

For more information on clearing DIMM faults, see the Oracle Integrated Lights Out Manager (Oracle ILOM) 3.0 Documentation Collection: http://www.oracle.com/goto/ILOM/docs.

Servicing PCIe Risers (CRU)

PCIe cards in Oracle Database Appliance (original version) are installed on a vertical riser. You must remove the riser to remove and replace a PCIe card. Also, you must remove the PCIe riser when replacing the motherboard.

This section includes:


Graphic showing electrical caution icon

Caution  ‐  These procedures require that you handle components that are sensitive to static discharge. This sensitivity can cause the component to fail. To avoid damage, ensure that you follow antistatic practices as described inPerforming Electrostatic Discharge and Antistatic Prevention Measures.


Remove PCIe Riser From Server Nodes

  1. Prepare the server node for service.
    1. Attach an antistatic wrist strap.

      See Performing Electrostatic Discharge and Antistatic Prevention Measures.

    2. Remove the server node from the system.

      Power is removed from the server node when you remove it from the system. See Removing Server Nodes From the System Chassis.

    3. Remove the server node top cover.

      See Removing the Server Node Top Cover.

  2. If you are servicing a PCIe card, locate its position on the riser.
  3. Disconnect any data cables connected to the cards on the PCIe riser being removed.

    To disconnect a cable from a PCIe card, press the latch, push in toward the connector, and then pull out to remove the cable.

    Note the location of all cables for reinstallation later.

  4. Ensure that the latch adjacent to the location where PCIe riser is installed is in the open (down) position [1].

    If the latch is in the closed position, it prevents you from removing the PCIe riser.

  5. Loosen the three green captive Phillips screws that secure the PCIe riser to the server node [1].

    Access these screws from the rear of the server node.


    image:Figure showing how to remove the PCIe riser.
  6. Loosen the green captive Phillips screw that secures the PCIe riser to the motherboard [2].
  7. To lift the PCIe riser out of the server node, grasp the front-most side plastic tab with one hand and the rear of the riser with the other hand, and lift it straight up and out of the server node [2].
  8. Place the PCIe riser and any cards attached to it on an antistatic mat.

Install PCIe Riser in Server Nodes

  1. Before installing the PCIe riser, ensure that the latch adjacent to the location where PCIe riser is to be installed is in the open (down) position [2].
  2. Install the PCIe riser.
    1. Ensure that any PCIe cards installed in the PCIe riser are properly seated, and that the card's locking bracket is flush with the recess in the riser and is engaging the locking tab.

      For instructions for installing a PCIe card into a PCIe riser, see Install PCIe Card in PCIe Riser.

    2. Lower the PCIe riser and any cards attached to it, into the server node [1].
    3. Push the PCIe riser down until its connector is seated into the connector on the motherboard.
    4. Tighten the green captive Phillips screw that secures the riser to the motherboard [1].
      image:Figure showing how to install the PCIe riser.
    5. Tighten the three green captive Phillips screws that secure the PCIe riser to the chassis [2].

      Access these screws from the rear of the server node.

  3. Return the server node to operation.
    1. Install the server node top cover.

      See Installing the Server Node Top Cover.

    2. Install the server node into the system.

      See Installing a Server Node Into the System Chassis.

    3. Reconnect any data cables to the PCIe card connectors on the system rear panel.
    4. Power on the server node.

      See Powering On Oracle Database Appliance (Original Version).

  4. Clear PCIe riser faults.
    1. Log in to the Oracle ILOM CLI as root.
    2. Type:

      -> show /SP/faultmgmt

      Oracle ILOM lists all known faults. For example:

      SP/faultmgmt
      Targets:
      0 (/SYS/MB/RISER)
      Properties:
      Commands:
      cd
      show
    3. To clear PCIe riser faults, use the set command. For example:
      -> set /SYS/MB/RISER clear_fault_action=true
      Are you sure you want to clear /SYS/MB/RISER (y/n)? y
      Set 'clear_fault_action' to 'true'

Servicing PCIe Cards (CRU)

This section describes servicing PCIe cards in Oracle Database Appliance (original version).

PCIe Card Configuration

The Oracle Database Appliance (original version) PCIe expansion system includes one standalone low-profile slot on the motherboard and a PCIe riser with three slots. These are provisioned as follows:


Graphic showing caution icon

Caution  ‐ If you replace a PCIe card, ensure the replacement is identical to the card you are replacing.


Remove PCIe Card From PCIe Riser

  1. Prepare the server node for service.
    1. Attach an antistatic wrist strap.

      See Performing Electrostatic Discharge and Antistatic Prevention Measures.

    2. Remove the server node from the system.

      Power is removed from the server node when you remove it from the system. See Removing Server Nodes From the System Chassis.

    3. Remove the server node top cover.

      See Removing the Server Node Top Cover.

  2. Locate the PCIe card in the PCIe riser that you want to remove.

    See the "Oracle Database Appliance (Original Version) Back Panel" section of the Oracle Database Owner's Guide for the location of the PCIe slots.

  3. If necessary, make a note of where the PCIe cards are installed.
  4. Disconnect all data cables from the PCIe card.

    To disconnect a cable from the PCIe card, press the latch, push in toward the connector, and then pull out to remove the cable.

    Note the location of all cables for reinstallation later.

  5. Remove the PCIe riser.

    See Remove PCIe Riser From Server Nodes.

  6. Remove the PCIe card from the riser.
    1. Carefully disconnect the PCIe card from the riser connector and remove it from the riser board connector.
      image:Figure showing how to remove a PCIe card from a PCIe riser.
    2. Place the PCIe card on an antistatic mat.

Install PCIe Card in PCIe Riser

  1. Unpack the replacement PCIe card and place it on an antistatic mat.
  2. Locate the proper PCIe slot for the card you are replacing.
  3. Remove the PCIe riser.

    See Remove PCIe Riser From Server Nodes.

  4. Insert the PCIe card into the correct slot on the riser.
    1. Press the PCIe card until the locking bracket on the PCIe card is flush with the recessed area on the PCIe riser.
    2. Slide the locking bracket down until it engages the PCIe riser locking tab.
      image:Figure showing how to install a PCIe card into a PCIe riser.
      Callout
      Description
      1
      Recessed area on the PCIe riser
      2
      PCIe riser locking tab
      3
      PCIe card locking bracket
  5. Install the PCIe riser.

    See Install PCIe Riser in Server Nodes.

  6. Reconnect any cables to the PCIe card that you unplugged during the removal procedure.
  7. Return the server node to operation.
    1. Install the server node top cover.

      See Install Server Node Top Cover.

    2. Install the server node into the system.

      See Install Server Node Into System Chassis.

    3. Reconnect any data cables to the PCIe card connectors on the system rear panel.
    4. Power on the server node.

      See Powering On Oracle Database Appliance (Original Version).

  8. Clear the PCIe card fault.
    1. Log in to the server node as root using the Oracle ILOM CLI.
    2. To view server node faults, type the following command to list all known faults on the system:

      -> show /SP/faultmgmt

      Oracle ILOM lists all known faults, for example:

      SP/faultmgmt
      Targets:
      0 (/SYS/MB/RISER/PCIE2)
      Properties:
      Commands:
      cd
      show
    3. To clear PCIe riser faults, use the set command. For example:
      -> set /SYS/MB/RISER clear_fault_action=true
      Are you sure you want to clear /SYS/MB/RISER (y/n)? y
      Set 'clear_fault_action' to 'true'
    4. To clear a PCIe card fault, use the set command. For example, to clear a fault on PCIe 2:
      -> set /SYS/MB/RISER/PCIE2 clear_fault_action=true
      Are you sure you want to clear /SYS/MB/RISER/PCIE2 (y/n)? y
      Set 'clear_fault_action' to 'true'

Remove Primary HBA PCIe Card From Motherboard

  1. Prepare the server node for service.
    1. Attach an antistatic wrist strap.

      See Performing Electrostatic Discharge and Antistatic Prevention Measures.

    2. Remove the server node from the system.

      Power is removed from the server node when you remove it from the system. See Removing Server Nodes From the System Chassis.

    3. Remove the server node top cover.

      See Removing the Server Node Top Cover.

  2. Remove the primary PCIe card.
    1. Locate the internal PCIe card [1].

      The internal PCIe card is located next to the rear-mounted disk drives.


      image:Figure showing how to remove the internal PCIe card.
    2. Disconnect the data cables from the PCIe card [1].

      There are two data cables that connect the PCIe card to the server node connector that connects to disk drive backplane.

      Note the location of all cables for reinstallation later.

      To disconnect the cables from the internal PCIe card, press the latch, push in toward the connector, and then pull out to remove the cable.

    3. Grasp the PCIe card and lift it up and out of the server node [2].
    4. Place the PCIe card on an antistatic mat.

Install Primary HBA PCIe Card in Motherboard

  1. Unpack the replacement PCIe card and place it on an antistatic mat.
  2. Install the internal PCIe card.
    1. Locate the connector for the internal PCIe card [1].

      The internal PCIe connector is located next to the rear-mounted disk drives.


      image:Figure showing how to install the internal PCIe card.
    2. Align the notch on the PCIe card with the key on the motherboard connector [1].
    3. Align the rear of the PCIe card with the support bracket located on the rear of the server node [1].
    4. Press down on the PCIe to seat it into the connector [1].

      Graphic showing caution icon

      Caution  ‐ If the notch and key are not aligned, damage to the PCIe card might occur.



      Note ‐ To install the primary HBA into the HBA slot on the motherboard, a special PCI metal bracket is required. The standard low-profile metal bracket or full-height metal bracket must be removed from the PCIe card and replaced with the unique metal bracket that is provided with the system.
    5. Reconnect the cables to the PCIe card that you disconnected during the removal procedure [2].
  3. Return the server node to operation.
    1. Install the server node top cover.

      See Install Server Node Top Cover.

    2. Install the server node into the system.

      See Install Server Node Into System Chassis.

    3. Power on the server node.

      See Powering On Oracle Database Appliance (Original Version).

  4. Clear the primary HBA card fault.
    1. Log in to the server node as root using the Oracle ILOM CLI.
    2. To view server node faults, type the following command to list all known faults on the server node:

      -> show /SP/faultmgmt

      Oracle ILOM lists all known faults, for example:

      SP/faultmgmt
      Targets:
      0 (/SYS/MB/HBA)
      Properties:
      Commands:
      cd
      show
    3. To clear the internal PCIe card fault, use the set command. For example:
      -> set /SYS/MB/HBA clear_fault_action=true
      Are you sure you want to clear /SYS/MB/HBA (y/n)? y
      Set 'clear_fault_action' to 'true'

Servicing the Server Node Battery (CRU)

The Oracle Database Appliance (original version) battery maintains the date and time for the Oracle ILOM service processor when the server node is removed from the system and, therefore, is powered off. If the server node fails to maintain the proper time when powered off, replace the battery.


Graphic showing electrical caution icon

Caution  ‐ These procedures require that you handle components that are sensitive to static discharge. This sensitivity can cause the component to fail. To avoid damage, ensure that you follow antistatic practices as described in Performing Electrostatic Discharge and Antistatic Prevention Measures.


Remove Battery

  1. Prepare the server node for service.
    1. Attach an antistatic wrist strap.

      See Performing Electrostatic Discharge and Antistatic Prevention Measures.

    2. Remove the server node from the system.

      Power is removed from the server node when you remove it from the system. See Removing Server Nodes From the System Chassis.

    3. Remove the server node top cover.

      See Removing the Server Node Top Cover.

  2. Remove the battery.
    1. Place you finger between the battery and the server node side wall and push the battery away from the side wall.
    2. Twist the battery (either clockwise or counter-clockwise) and lift it out.
      image:Figure showing how to remove the battery.

Install Battery

  1. Unpack the replacement battery.
  2. Press the new battery into the battery holder with the positive side (+) facing the server node side wall.
  3. If the service processor is configured to synchronize with a network time server using the Network Time Protocol (NTP), the Oracle ILOM clock is reset as soon as the server node is powered on and connected to the network. Otherwise, proceed to the next step.
  4. If the service processor is not configured to use NTP, you must reset the Oracle ILOM clock using the Oracle ILOM CLI or the web interface.

    For instructions, see the Oracle Integrated Lights Out Manager (Oracle ILOM) 3.0 Documentation Collection.

  5. Return the server node to operation.
    1. Install the server node top cover.

      See Install Server Node Top Cover.

    2. Install the server node into the system.

      See Install Server Node Into System Chassis.

    3. Power on the server node.

      See Powering On Oracle Database Appliance (Original Version).