1 Multirack Cabling Overview

This chapter describes the Exalogic Machine multirack cabling service.

It contains the following sections:

• Section 1.1, "About the Multirack Cabling Service"

• Section 1.2, "Multirack Cabling Scenarios"

• Section 1.3, "Important Notes Before You Begin"

• Section 1.4, "Hardware Components of Exalogic Machine"

• Section 1.5, "Exalogic Machine Rack Layout"

• Section 1.6, "Port Usage in InfiniBand Switches"

• Section 1.7, "Multirack Cabling Tasks"

1.1 About the Multirack Cabling Service

The multirack cabling service allows customers to extend the capability of their Exalogic rack systems to beyond just a single rack. The multirack cabling service allows customers to cable up to eight Exalogic machine full racks together, without requiring any external InfiniBand switches.

It is assumed that all racks are adjacent to each other, in a side-by-side configuration. If racks are not side-by-side, then customers may need to purchase longer cables for the extended connections. Placing the racks in any location other than side-by-side may likely require the purchase of additional 10–meter cables. Cable lengths across computer room aisles can vary from site to site, and may increase the difficulty of determining the cable lengths that may need to be purchased.

1.2 Multirack Cabling Scenarios

Table 1-1 lists multirack cabling scenarios for Exalogic machines:

Table 1-1 Exalogic Multirack Cabling Scenarios

Multiple Exalogic Machines	Exalogic Machine with Oracle Exadata Database Machine	Additional Switch Required (Yes/No)
An Exalogic machine quarter rack with another Exalogic machine quarter rack	An Exalogic machine quarter rack with an Oracle Exadata Database Machine	No
An Exalogic machine quarter rack with an Exalogic machine half rack	An Exalogic machine half rack with an Oracle Exadata Database Machine quarter rack	No
An Exalogic machine quarter rack with an Exalogic machine full rack	An Exalogic machine full rack with an Oracle Exadata Database Machine quarter rack	No
An Exalogic machine full rack with another Exalogic machine full rack	An Exalogic machine quarter rack with an Oracle Exadata Database Machine half rack	No
An Exalogic machine half rack with another Exalogic machine half rack	An Exalogic machine quarter rack with an Oracle Exadata Database Machine full rack	No
Four Exalogic machine full racks	An Exalogic machine half rack with an Oracle Exadata Database Machine half rack	No
Eight Exalogic machine full racks	An Exalogic machine full rack with an Oracle Exadata Database Machine full rack	No
Six Exalogic machine full racks with two Exalogic machine quarter racks	Two Exalogic machine full racks with two Oracle Exadata Database Machine full racks	No
n/a	Four Exalogic machine full racks with four Oracle Exadata Database Machine full racks	No
n/a	Any combination of eight racks of Exalogic machine and Oracle Exadata Database Machine, with a maximum of two Oracle Exadata Database Machine quarter racks and a maximum of two Exalogic machine quarter racks	No
More than eight Exalogic machine full racks	A combination of more than eight Exalogic machine full racks and Oracle Exadata Database Machine full racks	Yes Such multirack cabling scenarios require additional switches,such as such as an M9-648 port modular switch.

1.3 Important Notes Before You Begin

The current release of this guide discusses how to perform multirack cabling tasks in the following scenarios:

• Connecting an Exalogic machine full rack to another Exalogic machine full rack together on the same InfiniBand fabric, as described in Chapter 5, "Connecting Multiple Exalogic Machines"

• Connecting an Exalogic machine full rack to an Oracle Exadata Database Machine full rack together on the same InfiniBand fabric, as described in Chapter 6, "Connecting Exalogic and Exadata Machines"

• Connecting an Exalogic machine quarter rack to an Oracle Exadata Database Machine quarter rack together on the same InfiniBand fabric, as described in Chapter 6, "Connecting Exalogic and Exadata Machines"

Note:

More multirack configuration scenarios will be added to the future revisions of this guide.

For various multirack connectivity diagrams, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide.

1.4 Hardware Components of Exalogic Machine

See the Oracle Exalogic Elastic Cloud Machine Owner's Guide.

1.5 Exalogic Machine Rack Layout

See the Oracle Exalogic Elastic Cloud Machine Owner's Guide.

1.6 Port Usage in InfiniBand Switches

All configurations of the Exalogic machine (full, half, and quarter) include Sun Network QDR InfiniBand Gateway Switches. Only Exalogic machine full rack and half rack configurations include a Sun Datacenter InfiniBand Switch 36.

This section discusses the port usage in the following InfiniBand switches in multirack configuration scenarios:

• Port Usage in NM2-GW

• Port Usage in NM2-36P

1.6.1 Port Usage in NM2-GW

Figure 1-1 illustrates the port usage in each of the Sun Network QDR InfiniBand Gateway Switches (NM2-GW).

Figure 1-1 Port Usage/Connection Groups for NM2-GW in Multirack Scenarios

Description of "Figure 1-1 Port Usage/Connection Groups for NM2-GW in Multirack Scenarios"

1.6.2 Port Usage in NM2-36P

illustrates the port usage in the Sun Datacenter InfiniBand Switch 36 (NM2-36P), which is included in Exalogic machine full rack and half rack configurations.

Figure 1-2 Port Usage/Connection Groups for NM2-36P in Multirack Scenarios

Description of "Figure 1-2 Port Usage/Connection Groups for NM2-36P in Multirack Scenarios"

1.7 Multirack Cabling Tasks

Before cabling an Exalogic machine rack for multiple rack operation, you must know the system configuration and obtain all the prerequisite information for system installation. It is assumed that all Exalogic machine racks have been properly installed and verified. If you are connecting your Exalogic machine to an Oracle Exadata Database Machine, ensure that both Exalogic machine and Oracle Exadata Database Machine are installed and verified.

Procedures in this guide assume that the racks are next to each other.

Before you begin installing an Exalogic machine, perform pre–installation planning. Complete the checklists in the Oracle Exalogic Elastic Cloud Machine Owner's Guide. Verify the site space, access route, power, grounding, air flow, and network requirements that are provided in the Oracle Exalogic Elastic Cloud Machine Owner's Guide.

Install Exalogic machine racks if the cabling service is performed as part of an initial installation. To install an Exalogic machine, follow the instructions in the Oracle Exalogic Elastic Cloud Machine Owner's Guide. Ensure that you read the entire guide before beginning installation, and follow all safety precautions.

Perform the following multirack cabling tasks, in order, after completing the installation process.

• Review safety considerations.

• Perform preliminary tasks.

  • Section 3.1, "Preparing a Multirack Cabling Plan"

  • Section 3.2, "Verifying Host Names and Addresses"

  • Section 3.3, "Inventorying Cables"

  • Section 3.4, "Gathering Tools"

• Perform cabling tasks.

  • Section 4.1, "Shutting Down Affected Exalogic Machines and Oracle Exadata Database Machines"

  • Section 4.2, "Handling Inter-Switch Cables"

  • Section 4.3, "Extracting Spare InfiniBand Cables"

  • Section 4.4, "Unbundling Inter-Switch Cables"

  • Section 4.5, "Applying Cable and Rack Labels"

  • Section 4.6, "Connecting Cables"

  • Section 4.7, "Bundling Cables"

  • Section 4.8, "Inspecting Cables"

  • Section 4.9, "Verifying Multirack Systems"