3 Readying Oracle Database Appliance for Deployment

Complete these tasks to prepare to deploy Oracle Database Appliance.

Topics:

About Interconnect Cabling

Interconnect is reserved for Oracle Grid Infrastructure (GI) and Oracle Relational Database Management System (RDBMS).

Oracle GI includes Oracle Clusterware, Oracle Automatic Storage Management (Oracle ASM), and Oracle Restart. Even if you do not use Oracle Real Application Clusters (RAC) , Oracle Database Appliance uses the interconnect for RAC ASM and Oracle GI.

Note:

Do not use interconnect for other applications.

When you power on Oracle Database Appliance for the first time, the system automatically defines your public network interface based on the interconnect. If you use the InfiniBand cards, then the InfiniBand network is used for the interconnect. If you ordered the configuration that contains the 10GbE SFP+ (fiber) cards instead of InfiniBand cards, then the onboard 10GBase-T (Copper) ports are used for the interconnect.

Attaching Supplied Cables for Oracle Database Appliance

If you are using Oracle Database Appliance with a single storage shelf, attach the interconnect cables.

Topics:

Attaching Cables for Oracle Database Appliance X6-2-HA

Use the diagram in this topic to understand how to attach cable connections between ports on Oracle Database Appliance X6-2-HA.

Single Shelf

The back panel of each node contains three PCIe generation 3 terminal slots, with two sockets in each slot. The PCIe slots are labeled X PCIe3, where X is the PCIe slot number.

The following figure illustrates the cable connections between ports on Oracle Database Appliance X6-2-HA in a single storage shelf. The cables are color-coded. Match the colors of the labels at the ends of each cable with the colored line shown in the following illustration. Also match the cable colors to the background colors of the socket identification labels.

Description of GUID-77BE2C8C-0FD2-458E-81CF-AC81892FBEA4-print.eps follows
Description of the illustration GUID-77BE2C8C-0FD2-458E-81CF-AC81892FBEA4-print.eps

Table 3-1 Description of Callouts for Cabling Oracle Database Appliance X6-2-HA with a Single Storage Shelf

Callout Number Description

1

Green InfiniBand Cable

  • From: Node 0, PCIe slot 1, PORT2

  • To: Node 1, PCIe slot 1, PORT2

2

Yellow InfiniBand Cable

  • From: Node 0, PCIe slot 1, PORT1

  • To: Node 1, PCIe slot 1, PORT1

3

Dark blue SAS cable:

  • From: Node 0, PCIe slot 2, SAS0

  • To: Storage shelf, top I/O port 0

4

Light blue SAS cable:

  • From: Node 0, PCIe slot 3, SAS1

  • To: Storage shelf, bottom I/O port 0

5

Dark red SAS cable:

  • From: Node 1, PCIe slot 2, SAS1

  • To: Storage shelf, top I/O port 1

6

Light red SAS cable:

  • From: Node 1, PCIe slot 3, SAS0

  • To: Storage shelf, bottom I/O port 1

Cabling Oracle Database Appliance to Connect to Fiber Public Networks (Optional)

You can configure Oracle Database Appliance X6-2-HA to use 10GbE SFP+ (fiber) connections for the public network if you ordered the configuration that contains 10GbE SFP+ (fiber) cards instead of InfiniBand cards.

To use fiber connections between Oracle Database Appliance and your public network, you must cable the interconnect by using the green and yellow copper ports (onboard ports net0 and net1). The green and yellow Category 6 (Cat 6) interconnect cables for these ports ship with the base system.

The following describes how to connect the ports:

  • Plug the green Cat 6 cable into the onboard NET 1 port on each node, as shown in callout 1 in the figure.

  • Plug the yellow Cat 6 cable into the onboard NET 0 port on each node, as shown in callout 2 in the figure.

Description of GUID-97CC8FEB-9086-42D7-B43C-BA9FEE5CA5B0-default.png follows
Description of the illustration GUID-97CC8FEB-9086-42D7-B43C-BA9FEE5CA5B0-default.png

The following sections show the cabling options for 10 GbE SFP+ PCI cards.

Fiber Cables

For optical cables, you must purchase either Short Range (SR) or Long Range (LR) SFP+ transceivers for each of the network ports, and then plug in the appropriate optical cable. Currently, Oracle sells both the SR and LR SFP+ transceivers. In addition to these transceivers, you must purchase the appropriate LC-LC terminated fiber optic cables from a third-party vendor.

Name Part Number

10 GbE Transceiver SR (SFP+)

X2129A-N

10 GbE Transceiver LR (SFP+)

X5562A-Z

Copper Cables

You can purchase the following copper cables from Oracle. These cables have built-in SFP+ connectors:

Name Length Part Number

TwinAx 1m

1m

X2130-1M-N

TwinAx 3m

3m

X2130-3M-N

TwinAx 5m

5m

X2130-3M-N

Attaching a Storage Expansion Shelf to Oracle Database Appliance

Review these topics to install and cable a storage expansion shelf for Oracle Database Appliance.

Topics:

About Attaching and Cabling Storage Shelves

Review these options and best practices for adding storage shelves for Oracle Database Appliance X6-2-HA.

Storage Shelf Options for Oracle Database Appliance X6-2-HA

Oracle Database Appliance X6-2-HA can be shipped with one or two storage shelves. You can obtain the second shelf at a later time to double your storage capacity.

If you originally deployed Oracle Database Appliance with one storage shelf, then you can add a storage expansion shelf at any time without having to shut down your databases or applications.

Best Practice Guidelines for Adding Storage Expansion Shelves

Oracle recommends that you add a storage expansion shelf when you have relatively little activity on your databases. When the system discovers the new storage, Oracle ASM automatically rebalances the disk groups. The rebalance operation may degrade database performance until the operation completes.

Caution:

Review cabling instructions to ensure that you have carried out cabling correctly. Incorrect connections can cause data loss when adding a storage expansion shelf to Oracle Database Appliance with existing databases.

Do not attach cables to Ethernet ports in the I/O modules labeled "SERVICE ONLY" when cabling the system. These ports are intended for use by service engineers only.

How to Cable a Storage Expansion Shelf

The following figure shows you how to insert the required cables into the storage expansion shelf of Oracle Database Appliance. The cables for the two nodes and the original storage shelf are included in the illustration for reference.

The cables are color-coded. Match the colors of the labels at the ends of each cable with the colored line shown in the illustration. Also match the cable colors to the background colors of the socket identification labels. You can also use the callouts in the diagram to identify the cables and ports. Because all cables with the same terminations are interchangeable, you can ignore the color coding. The back panel of each node contains three PCIe generation 3 terminal slots, with two sockets in each slot. The PCIe slots are labeled X PCIe3, where X is the PCIe slot number. Review the callout table that follows the figure for more information.

Description of GUID-D76DBBCE-A5E7-4B9E-BAAF-DA3BB7C7C510-print.eps follows
Description of the illustration GUID-D76DBBCE-A5E7-4B9E-BAAF-DA3BB7C7C510-print.eps

The cables identified with callout numbers in the preceding figures are included with each shipped Oracle Database Appliance storage expansion shelf. These cables are all black, but have colored labels at each end that match the references to cable colors in the text and the label colors on the back panels.

Table 3-2 Description of Callouts for Cabling Oracle Database Appliance Storage Expansion Shelf

Callout Number Description

7

Dark blue SAS cable:

  • From: Node 1, PCIe slot 2, SAS0

  • To: Storage expansion shelf, top I/O port 0

8

Light blue SAS cable:

  • From: Node 1, PCIe slot 3, SAS1

  • To: Storage expansion shelf, bottom I/O port 0

9

Dark red SAS cable:

  • From: Node 0, PCIe slot 2, SAS1

  • To: Storage expansion shelf, top I/O port 1

10

Light red SAS cable:

  • From: Node 0, PCIe slot 3, SAS0

  • To: Storage expansion shelf, bottom I/O port 1

Caution:

Do not attach cables to Ethernet ports in the I/O modules labeled "SERVICE ONLY" when cabling the system. These ports are intended for use by service engineers only.

Installing and Cabling a Storage Expansion Shelf

Use this procedure to install and cable a storage expansion shelf for Oracle Database Appliance

  1. Place the storage expansion shelf below your Oracle Database Appliance, if possible, or else close enough to connect the provided cables.
    The storage expansion shelf normally sits at the bottom of Oracle Database Appliance, beneath the storage shelf. However, because racks should always be provisioned from the bottom up, that space may be unavailable. In that case, to avoid re-rack mounting the entire system, you can position the storage expansion shelf above the server nodes, or in a different, but adjacent, rack.
  2. Install the extension storage shelf in exactly the same manner as the original storage shelf. To review information about installing optional components for Oracle Database Appliance, see the section "Optional Component Installation" in Chapter 4 of Oracle Database Appliance Owner’s Guide.
  3. Use the figures in this topic to assist you to cable the storage expansion shelf for your Oracle Database Appliance model.

    Caution:

    Incorrect connections can cause data loss when adding a storage expansion shelf to Oracle Database Appliance with existing databases.
  4. Attach the supplied power cords. Refer to the section "Attaching Power Cords and Initializing Components" to obtain information and see figures showing how to attach power cords.

Attaching Peripheral Devices

Complete this task if you have direct access to Oracle Database Appliance and you intend to use a locally connected monitor, keyboard and mouse.

Oracle Database Appliance is not equipped with human-computer interface devices, such as a monitor or keyboard. If you want to log in locally, instead of through a network, then you must attach interface devices.

Attach a monitor to the graphics card port, and attach a keyboard and a mouse to the USB ports. Refer to the figure and table to identify the ports.

Description of GUID-EAD47912-576E-4634-BE60-261AC9575675-print.eps follows
Description of the illustration GUID-EAD47912-576E-4634-BE60-261AC9575675-print.eps

In the figure, callout 1 identifies the ports for the keyboard and mouse. Callout 2 identifies the monitor port.

Table 3-3 Peripheral Device Connections for Oracle Database Appliance

Callout Number Description

1

USB ports for the keyboard and mouse

2

Graphics card port for the monitor

First Startup of Oracle Database Appliance

Use this procedure to start up either a newly-installed Oracle Database Appliance, or to start up the appliance after you power it down.

To ready Oracle Database Appliance for the powering on the first time, you need to attach all of the required power cords and confirm that initialization completes successfully. You can then start up the system by pushing the power button once.

Topics:

Attaching Power Cords and Initializing Components

Attach power cords for Oracle Database Appliance.

Caution:

When you plug in power cords, ensure that the electrical outlets providing the power are grounded before plugging in the power cords.

If you use only a single AC circuit, then connect both power cords for each component to that circuit. If you want to maintain N+1 power supply redundancy, then use two separate AC circuits. Connect one power cord from each AC circuit into each component.

For more information about cabling with the supplied Cable Management Arm, refer to Oracle Database Appliance Owner’s Guide.

Powering On Oracle Database Appliance the First Time

Use this procedure the first time you power on Oracle Database Appliance.

Note:

After you connect power cords, the green SP OK light-emitting diode (LED) lights blink for a few minutes, and then turn to steady ON. The cooling fans also may turn on. However, these events do not indicate that the system is started. You must complete all of the steps in this section to turn on the appliance properly. Read through the entire section to understand the sequence required for initial system startup. (Optional) Enter task prerequisites here.
  1. Turn on the attached storage shelf, or shelves, before powering on the server nodes.
  2. After turning on the storage in the rear of each storage shelf, push the recessed power button to turn on the server nodes.

    Note:

    The fully-initialized state is indicated by the green SP OK LEDs staying steadily lit. If any of the green LED lights on the server are still blinking, then wait for them to complete their initialization steps.

    The figure is an image of the front of the power panel, see the following callout table to identify the power button and the system initialization status indicator lights.

    Description of GUID-78161D04-C945-48EE-BC33-C9199891A9F2-print.eps follows
    Description of the illustration GUID-78161D04-C945-48EE-BC33-C9199891A9F2-print.eps

    Table 3-4 Description of Callouts for Powering On Oracle Database Appliance

    Callout Function
    1 Recessed On/Off power button, which is located on the front panel of the appliance. Push this button only once. Do not repeatedly push the power button.
    2 SP OK LED light, located on the front panel of the appliance.
    3 Green Power OK LED, located on the front panel. These lights must be in the steady ON position (Green light does not blink) before you log on to the system.
  3. Wait for Oracle Database Appliance to complete startup.

    Note:

    Do not repeatedly push the power buttons. Startup can take several minutes to complete.

    Oracle Database Appliance is ready for use when the green Power OK LEDs (callout 3) on the front of the system remains steadily on.

Configuring Oracle Integrated Lights Out Manager

Configure Oracle Integrated Lights Out Manager (Oracle ILOM) to manage Oracle Database Appliance independent of the operating system.

Oracle ILOM provides alternate ways to restart and troubleshoot Oracle Database Appliance.

You must set up the following items first to configure Oracle ILOM:

  • A name and IP address

  • A password to replace the default Oracle ILOM password

  • Access to a management network, using an assigned netmask

  • An Ethernet cable connected from the NET MGT port to the management network

In the default configuration, Dynamic Host Configuration Protocol (DHCP) is enabled in Oracle ILOM and the DHCP server automatically assigns network settings. To determine the IP address or host name assigned by the DHCP server, use the network tools provided with the DHCP server.

If you do not use DHCP, then use the custom option in Oracle Database Appliance Manager Configurator to assign IP addresses and host names to Oracle ILOM when you deploy your database.

See the Oracle Database Appliance Owner's Guide for details about the port used for ILOM on your platform.

Note:

If you have not enabled DHCP, then you must complete Oracle Database Appliance configuration to access Oracle ILOM.

To connect to the Oracle ILOM, use one of the following two methods:

  1. Log in using a web interface by completing these steps:

    1. Using a client system's browser, enter the IP address or host name assigned by DHCP into the browser address field and press Enter.

    2. At the login page, enter the default user name, root and the default password, changeme.

      The Oracle ILOM web interface appears.

  2. Log in using a command line interface (CLI) by completing these steps:

    1. Using a client system, establish a secure shell (SSH) connection by entering the following on the command line:

      ssh -l root sp_ip_address

      where sp_ip_address is the IP address assigned by DHCP.

    2. Enter the default user name, root, and the default password, changeme.

      The Oracle ILOM CLI prompt appears.

Configuring an Initial Network Connection

Configure a temporary network configuration framework to build your network information during deployment.

To manage your Oracle Database Appliance deployment across a network, you must configure an initial network connection. Use this network to transfer deployment software to Oracle Database Appliance. When you power on Oracle Database Appliance X6-2-HA for the first time, the system automatically defines your public network interface based on the interconnect. The InfiniBand network is used for the interconnect if InfiniBand cards are used. If InfiniBand cards are replaced with 10 GbE SFP+ (fiber) cards, then the onboard 10GBase-T (Copper) ports are used for the interconnect.

If you have physical access to your Oracle Database Appliance, then you can use a USB storage device to transfer deployment software to the appliance. In that case, you do not need to set up an initial network configuration.

Note:

The initial network configuration is temporary. It is replaced during the final image deployment.

  1. Log in to Oracle Database Appliance as root, using the password welcome1.

    On Bare Metal installations, log in to a console.

    On Oracle Database Appliance Virtualized Platforms, log in to Dom0.

  2. On Bare Metal installations, if you have one or more storage shelves, then run the command oakcli validate with the storageTopology option to perform a validity check of cabling.

    For example:

    oakcli validate -c storageTopology

    On Virtualized Platform installations, you perform this check as part of the Oracle Software deployment process.

  3. Run the command oakcli configure firstnet to configure the initial network.

    As needed, refer to the network configuration information that you previously collected in preparation for deployment.

    To use the global option, accept the default value in the first prompt, and then provide the required data for the remaining prompts. You do not need to enter anything to accept the default values that are shown in square brackets at the end of prompts.

    Use the following example to help you to respond to prompts. In this example, default values are always selected. Values that you need to provide are shown in italic font, with the exception of the net1 gateway address; the program obtains the gateway IP.

    In the preceding example, the prompt for the net1 gateway address includes a default value, shown as gateway-IP-address. The program derives this gateway address using the network information you provided for the other IP addresses. Accept this value, unless your network administrator provides an alternative gateway address that is different from the default that the appliance command-line interface detects:

    oakcli configure firstnet
Configure the network for the node(s)(local, global) [global]:
The network configuration for both nodes:
Domain Name:  my-organization-domain-name
DNS Server(s):  Primary Dns Server:  my-primary-DNS-server
                Secondary Dns Server:  my-secondary-DNS-server
                Tertiary Dns Server:  my-tertiary-DNS-server
Node Name       Host Name
0               my-Node-0-host-name
1               my-Node-1-host-name
Choose the network interface to configure (net1, net2, net3, net4) [net1]:
Configure DHCP on net1 (yes/no) [no]:
You have chosen static configuration on net1
Enter the IP address for net1 on Node 0: my-Node-0-IP-address
Enter the IP address for net1 on Node 1: my-Node-1-IP-address
Netmask for net1: your-netmask
Gateway Address for net1 [gateway-IP-address]: 
Plumbing the IPs now on Node 0 …
::::::::::::::::::::::::::::::::::::::::::::::::::
Plumbing the IPs now on Node 1 …
::::::::::::::::::::::::::::::::::::::::::::::::::

Caution:

Oracle recommends using the oakcli configure firstnet command only one time on Oracle Database Appliance. Subsequent use after configuring the initial network can cause unpredictable changes to your network settings.