3 Readying Oracle Database Appliance for Deployment
Complete these tasks to prepare to deploy Oracle Database Appliance.
Topics:
- About Interconnect Cabling
Interconnect is reserved for Oracle Grid Infrastructure (GI) and Oracle Relational Database Management System (RDBMS). - Attaching Supplied Cables for Oracle Database Appliance
If you are using Oracle Database Appliance with a single storage shelf, attach the interconnect cables. - Attaching a Storage Expansion Shelf to Oracle Database Appliance
Review these topics to install and cable a storage expansion shelf for Oracle Database Appliance. - Attaching Peripheral Devices
Complete this task if you have direct access to Oracle Database Appliance and you intend to use a locally connected monitor, keyboard and mouse. - Attaching Power Cords and Initializing Components
Attach power cords for Oracle Database Appliance. - Powering On Oracle Database Appliance the First Time
Use this procedure to power on Oracle Database Appliance. - Defining Your Public Network Interface Type
Understand how you can define the public network interface type before deploying the appliance. - Configuring Oracle Integrated Lights Out Manager
Configure Oracle Integrated Lights Out Manager (Oracle ILOM) to manage Oracle Database Appliance independent of the operating system.
About Interconnect Cabling
Interconnect is reserved for Oracle Grid Infrastructure (GI) and Oracle Relational Database Management System (RDBMS).
Oracle GI includes Oracle Clusterware, Oracle Automatic Storage Management (Oracle ASM), and Oracle Restart. Even if you do not use Oracle Real Application Clusters (RAC) , Oracle Database Appliance uses the interconnect for RAC ASM and Oracle GI.
Note:
Do not use interconnect for other applications.When you power on Oracle Database Appliance for the first time, the system automatically defines your public network interface based on the interconnect. If you use the InfiniBand cards, then the InfiniBand network is used for the interconnect. If you ordered the configuration that contains the 10GbE SFP+ (fiber) cards instead of InfiniBand cards, then the onboard 10GBase-T (Copper) ports are used for the interconnect.
Parent topic: Readying Oracle Database Appliance for Deployment
Attaching Supplied Cables for Oracle Database Appliance
If you are using Oracle Database Appliance with a single storage shelf, attach the interconnect cables.
Topics:
- Attaching Cables for Oracle Database Appliance X5-2
Use the diagram in this topic to understand how to attach cable connections between ports on Oracle Database Appliance model X5-2 with a single storage shelf: - Attaching Cables for Oracle Database Appliance Models X4-2
Use the diagram in this topic to understand how to attach cable connections between ports on Oracle Database Appliance model X4-2. - Cabling Oracle Database Appliance to Connect to Fiber Public Networks (Optional)
You can configure X5-2 appliances to use fiber connections for the public network.
Parent topic: Readying Oracle Database Appliance for Deployment
Attaching Cables for Oracle Database Appliance X5-2
Use the diagram in this topic to understand how to attach cable connections between ports on Oracle Database Appliance model X5-2 with a single storage shelf:
Figure 3-1 Interconnects and Cables for a Single Storage Shelf on Oracle Database Appliance X5-2
Description of "Figure 3-1 Interconnects and Cables for a Single Storage Shelf on Oracle Database Appliance X5-2"
Table 3-1 Description of Callouts for Cabling Oracle Database Appliance X5-2 With Single Storage Shelf
Callout Number | Description |
---|---|
1 |
Green InfiniBand Cable (Oracle Database Appliance X5-2)
|
2 |
Yellow InfiniBand Cable (Oracle Database Appliance X5-2)
|
3 |
Dark blue SAS cable:
|
4 |
Light blue SAS cable:
|
5 |
Dark red SAS cable:
|
6 |
Light red SAS cable:
|
Note:
The back panel of each node contains three PCIe generation 3 terminal slots, with two sockets in each slot. The PCIe slots are labeled X PCIe3, where X is the PCIe slot number.
Parent topic: Attaching Supplied Cables for Oracle Database Appliance
Attaching Cables for Oracle Database Appliance Models X4-2
Use the diagram in this topic to understand how to attach cable connections between ports on Oracle Database Appliance model X4-2.
The following figure illustrates the cable connections between ports on Oracle Database Appliance model X4-2 in a single storage shelf.
Figure 3-2 Interconnects and Cables for Single Storage Shelf on Oracle Database Appliance X4-2
Description of "Figure 3-2 Interconnects and Cables for Single Storage Shelf on Oracle Database Appliance X4-2"
Table 3-2 Description of Callouts for Cabling Oracle Database Appliance X4-2 With Single Storage Shelf
Callout Number | Description |
---|---|
1 |
Green-labelled twinax cable (Oracle Database Appliance X4-2):
|
2 |
Yellow-labeled twinax cable (Oracle Database Appliance X4-2)
|
3 |
Dark blue SAS cable:
|
4 |
Light blue SAS cable:
|
5 |
Dark red SAS cable:
|
6 |
Light red SAS cable:
|
Caution:
Do not attach cables to ethernet ports in the I/O modules labeled "SERVICE ONLY" when cabling the system. These ports are intended for use by service engineers only. You must use Oracle Appliance Manager to manage Oracle Database Appliance storage.
Note:
The back panel of each node contains three PCIe generation 3 terminal slots, with two sockets in each slot. The PCIe slots are labeled X PCIe3, where X is the PCIe slot number.
Parent topic: Attaching Supplied Cables for Oracle Database Appliance
Cabling Oracle Database Appliance to Connect to Fiber Public Networks (Optional)
You can configure X5-2 appliances to use fiber connections for the public network.
For Oracle Database Appliance X5-2 installations, if you want to use fiber connections between Oracle Database Appliance and your public network, then you must cable the interconnect by using the green and yellow copper ports (onboard ports net0
and net1
). The green and yellow Category 6 (Cat 6
) interconnect cables for these ports ship with the base system.
For Oracle Database Appliance X5-2, if you want to use a fiber port connection, then you must replace the InfiniBand cards with 10 GbE SFP+ (fiber) cards. Fiber ports do not exist in the base configuration for X5-2. You must re-image Oracle Database Appliance X5-2 after you replace the PCIe cards. After you replace the cards, Oracle Database Appliance auto-detects the network configuration changes.
Caution:
If you replace the InfiniBand interconnect cards on Oracle Database Appliance X5-2, then you disable the Oracle Database Appliance in-memory fault tolerance feature. Also, interconnect bandwidth is reduced from 40 GB to 10 GB. If you have Oracle RAC databases, then you may experience slower intercommunication speeds, depending on the system workload.
In the following figure, callouts 1 and 2 show how to cable the interconnect between Node 1 and Node 0 using the green and yellow copper ports (onboard ports net0
and net1
) to use a fiber public network. Note that the illustration shows the rear panel of an Oracle Database Appliance X4-2. The rear panel of the Oracle Database Appliance X5-2 is slightly different, but the interconnect cabling is the same.
Figure 3-3 Interconnects When Using Fiber Connections to the Public Network on Oracle Database Appliance Models X5-2
Description of "Figure 3-3 Interconnects When Using Fiber Connections to the Public Network on Oracle Database Appliance Models X5-2"
The following sections show the cabling options for 10 GbE SFP+ PCI cards.
Fiber Cables
For optical cables, you must purchase either Short Range (SR) or Long Range (LR) SFP+ transceivers for each of the network ports, and then plug in the appropriate optical cable. Currently, Oracle sells both the SR and LR SFP+ transceivers. In addition to these transceivers, you must purchase the appropriate LC-LC terminated fiber optic cables from a third-party vendor.
Name | Oracle Sun Part Number |
---|---|
10 GbE Transceiver SR (SFP+) |
X2129A-N |
10 GbE Transceiver LR (SFP+) |
X5562A-Z |
Copper Cables
You can purchase the following copper cables from Oracle Sun. These cables have built-in SFP+ connectors:
Name | Length | Oracle Sun Part Number |
---|---|---|
TwinAx 1m |
1m |
X2130-1M |
TwinAx 3m |
3 m |
X2130-3M |
TwinAx 5m |
5m |
X2130-5M |
Parent topic: Attaching Supplied Cables for Oracle Database Appliance
Attaching a Storage Expansion Shelf to Oracle Database Appliance
Review these topics to install and cable a storage expansion shelf for Oracle Database Appliance.
Topics:
- About Attaching and Cabling Storage Shelves
Review these options and best practices for adding storage shelves for Oracle Database Appliance X3-2, X4-2 and X5-2. - Installing and Cabling a Storage Expansion Shelf
Use this procedure to install and cable a storage expansion shelf for Oracle Database Appliance
Parent topic: Readying Oracle Database Appliance for Deployment
About Attaching and Cabling Storage Shelves
Review these options and best practices for adding storage shelves for Oracle Database Appliance X3-2, X4-2 and X5-2.
Storage Shelf Options for Oracle Database Appliance X3-2, X4-2 and X5-2
Oracle Database Appliance models X3-2, X4-2, and X5-2 can be shipped with one or two storage shelves. You can obtain the second shelf at a later time to double your storage capacity.
If you originally deployed Oracle Database Appliance with one storage shelf, then you can add a storage expansion shelf at any time without having to shut down your databases or applications.
Best Practice Guidelines for Adding Storage Expansion Shelves
Oracle recommends that you add a storage expansion shelf when you have relatively little activity on your databases. When the system discovers the new storage, Oracle ASM automatically rebalances the disk groups. The rebalance operation may degrade database performance until the operation completes.
Caution:
Review cabling instructions to ensure that you have carried out cabling correctly. Incorrect connections can cause data loss when adding a storage expansion shelf to Oracle Database Appliance with existing databases.
Do not attach cables to Ethernet ports in the I/O modules labeled "SERVICE ONLY" when cabling the system. These ports are intended for use by service engineers only.
Installing and Cabling a Storage Expansion Shelf
Use this procedure to install and cable a storage expansion shelf for Oracle Database Appliance
-
Place the storage expansion shelf below your Oracle Database Appliance, if possible, or else close enough to connect the provided cables.
The storage expansion shelf normally sits at the bottom of Oracle Database Appliance, beneath the storage shelf. However, because racks should always be provisioned from the bottom up, that space may be unavailable. In that case, to avoid re-rack mounting the entire system, you can position the storage expansion shelf above the server nodes, or in a different, but adjacent, rack.
-
Install the extension storage shelf in exactly the same manner as the original storage shelf.
-
Use the figures in this topic to assist you to cable the storage expansion shelf for your Oracle Database Appliance model.
Caution:
Incorrect connections can cause data loss when adding a storage expansion shelf to Oracle Database Appliance with existing databases.
-
Attach the supplied power cords. Refer to the section "Attaching Power Cords and Initializing Components" to obtain information and see figures showing how to attach power cords.
Figures Showing How to Cable a Storage Expansion Shelf
The following figure shows you how to insert the required cables into the storage expansion shelf of Oracle Database Appliance. The cables for the two nodes and the original storage shelf are included in the illustration for reference.
The cables are color-coded. Match the colors of the labels at the ends of each cable with the colored line shown in the following illustration. Also match the cable colors to the background colors of the socket identification labels.
You can also use the callouts in the diagrams and the callout table that follows the diagrams to identify the cables and ports. Because all cables with the same terminations are interchangeable, you can ignore the color coding.
Note:
The back panel of each node contains three PCIe generation 3 terminal slots, with two sockets in each slot. The PCIe slots are labeled X PCIe3, where X is the PCIe slot number. Review the callout table that follows the figures for more information.
Figure 3-4 Cables for Storage Expansion Shelf on Oracle Database Appliance X5-2
Description of "Figure 3-4 Cables for Storage Expansion Shelf on Oracle Database Appliance X5-2"
The cables identified with callout numbers in the preceding figures are included with each shipped Oracle Database Appliance storage expansion shelf. These cables are all black, but have colored labels at each end that match the references to cable colors in the text and the label colors on the back panels.
Table 3-3 Description of Callouts for Cabling Oracle Database Appliance Storage Expansion Shelf
Callout Number | Description |
---|---|
7 |
Dark blue SAS cable:
|
8 |
Light blue SAS cable:
|
9 |
Dark red SAS cable:
|
10 |
Light red SAS cable:
|
Caution:
Do not attach cables to Ethernet ports in the I/O modules labeled "SERVICE ONLY" when cabling the system. These ports are intended for use by service engineers only.
See Also:
Attaching Power Cords and Initializing Components
To review information about installing optional components for Oracle Database Appliance, see the section "Optional Component Installation" in Chapter 4 of Oracle Database Appliance Owner’s Guide.
Attaching Peripheral Devices
Complete this task if you have direct access to Oracle Database Appliance and you intend to use a locally connected monitor, keyboard and mouse.
Oracle Database Appliance is not equipped with human-computer interface devices, such as a monitor or keyboard. If you want to log in locally, instead of through a network, then you must attach interface devices.
Attach a monitor to the graphics card port, and attach a keyboard and a mouse to the USB ports. Refer to the figure and table to identify the ports.
In the figure, callout 1 identifies the ports for the keyboard and mouse. Callout 2 identifies the monitor port.
Table 3-4 Peripheral Device Connections for Oracle Database Appliance
Callout Number | Description |
---|---|
1 |
USB ports for the keyboard and mouse |
2 |
Graphics card port for the monitor |
Parent topic: Readying Oracle Database Appliance for Deployment
Attaching Power Cords and Initializing Components
Attach power cords for Oracle Database Appliance.
Caution:
Before plugging in the power cords, ensure that the electrical outlets providing the power are grounded.
If you use only a single AC circuit, then connect both power cords for each component to that circuit. If you want to maintain N+1 power supply redundancy, then use two separate AC circuits. Connect one power cord from each AC circuit into each component.
For more information about cabling with the supplied Cable Management Arm, refer to Oracle Database Appliance Owner’s Guide.
Parent topic: Readying Oracle Database Appliance for Deployment
Powering On Oracle Database Appliance the First Time
Use this procedure to power on Oracle Database Appliance.
Before starting to power on the appliance, make sure that all of the required power cables are attached to both Oracle Database Appliance nodes, and to all storage shelves, if they are included with your appliance.
Note:
After you connect power cords, the green SP OK light-emitting diode (LED) lights blink for a few minutes, and then turn to steady ON. The cooling fans also may turn on. However, these events do not indicate that the system is started. You must complete all of the steps in this section to turn on the appliance properly. Read through the entire section to understand the sequence required for initial system startup.
Parent topic: Readying Oracle Database Appliance for Deployment
Defining Your Public Network Interface Type
Understand how you can define the public network interface type before deploying the appliance.
For Oracle Database Appliance X5-2 and X6-2 High-Availability non-InfiniBand deployments, the system by default, configures the fiber cards as public network, and copper cards as private network.
Optionally, before provisioning the appliance, if you want to switch the public network to copper, then follow these steps:
- Log into both nodes as
root
. - Configure the network.
For bare metal deployments, run the command on both nodes:
# /opt/oracle/dcs/bin/odacli configure-network -publicNet copper
For Virtualized Platform deployments, run the command on both nodes:# oakcli configure network -publicNet copper
For Oracle Database Appliance X4-2, the system by default, configures the copper cards as public network, and fiber cards as private network.
Optionally, before provisioning the appliance, if you want to switch the public network to fiber, then follow these steps:
- Log into both nodes as
root
. - Configure the network.
For bare metal deployments, run the command on both nodes:
# /opt/oracle/dcs/bin/odacli configure-network -publicNet fiber
For Virtualized Platform deployments, run the command on both nodes:# oakcli configure network -publicNet fiber
Parent topic: Readying Oracle Database Appliance for Deployment
Configuring Oracle Integrated Lights Out Manager
Configure Oracle Integrated Lights Out Manager (Oracle ILOM) to manage Oracle Database Appliance independent of the operating system.
Oracle ILOM provides alternate ways to restart and troubleshoot Oracle Database Appliance.
You must set up the following items first to configure Oracle ILOM:
-
A name and IP address
-
A password to replace the default Oracle ILOM password
-
Access to a management network, using an assigned netmask
-
An Ethernet cable connected from the NET MGT port to the management network
In the default configuration, Dynamic Host Configuration Protocol (DHCP) is enabled in Oracle ILOM and the DHCP server automatically assigns network settings. To determine the IP address or host name assigned by the DHCP server, use the network tools provided with the DHCP server.
If you do not use DHCP, then use the custom option in Oracle Database Appliance Manager Configurator to assign IP addresses and host names to Oracle ILOM when you deploy your database.
Note:
If you have not enabled DHCP, then you must complete Oracle Database Appliance configuration to access Oracle ILOM.
To connect to the Oracle ILOM, use one of the following two methods:
-
Log in using a web interface by completing these steps:
-
Using a client system's browser, enter the IP address or host name assigned by DHCP into the browser address field and press Enter.
-
At the login page, enter the default user name,
root
and the default password,changeme
.The Oracle ILOM web interface appears.
-
-
Log in using a command-line interface (CLI) by completing these steps:
-
Using a client system, establish a secure shell (SSH) connection by entering the following on the command line:
ssh -l root sp_ip_address
where
sp_ip_address
is the IP address assigned by DHCP. -
Enter the default user name,
root
, and the default password,changeme
.The Oracle ILOM CLI prompt appears.
-
Troubleshooting the ILOM Configuration
If the ILOM is not working, then ensure that the ILOM NIC is enabled and that ipv4
is enabled.
You can configure the ILOM with the IPMI tool or through the BIOS. The following describes how to use the IPMI tool:
-
Open the IPMI tool.
# ipmitool –I open sunoem cli
-
Go to
SP/network
and verify that the port is enabled:cd /SP/network
-
If the port is not enabled, then enable the port.
set state=enabled
-
In
SP/network
, verify thatipv4
is enabled. -
If
ipv6
is enabled, then disableipv6
and enableipv4
# ipmitool -I open sunoem cli "set /SP/network/ipv6 state=disabled" # ipmitool -I open sunoem cli "set /SP/network/ipv4 state=enabled"
Parent topic: Readying Oracle Database Appliance for Deployment