Oracle® Exalytics

In-Memory Machine Owner's Guide

Exalytics X2-4, X3-4, X4-4, and X5-4 Release 2.0.0.0

E63850-02

July 2015

Oracle Exalytics In-Memory Machine Owner's Guide Exalytics X2-4, X3-4, X4-4, and X5-4 Release 2.0.0.0

E63850-02

Copyright © 2015, Oracle and/or its affiliates. All rights reserved.

Primary Author: Chakravarthy Tenneti

Contributor: Paul Joyce, Vasu Murthy, Joe Moriarty, Guy Shaked, Dharnidhar Gundu, Gururaj BS, Paul Dorian, Rekha Kamath, Shynitha K S, Showvik Roy Chowdhuri, Nisha Singh, and Kumar Dhanagopal

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Contents

Preface

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

1 Getting Started

- 1.1 Tools and Equipment Needed

- 1.2 Installation Tasks

- 1.3 Opening the Box

- 1.3.1 Server Box Contents

- 1.4 ESD Precautions

- 1.5 Oracle Exalytics In-Memory Machine Description

- 1.5.1 Front Panel Features

- 1.5.2 Back Panel Features

- 1.6 Connecting the Cables

- 1.7 Connecting the Power Cords

2 Introduction

- 2.1 About Oracle Exalytics In-Memory Machine

- 2.2 Hardware Configuration

- 2.3 Assigning PCIe Slots

- 2.4 Key Features

- 2.5 Environmental Specifications

- 2.6 Safety and Compliance Information

3 Installing Oracle Exalytics In-Memory Machine Into a Rack

- 3.1 Task Roadmap

- 3.2 Reviewing Rack Compatibility

- 3.3 Before You Begin

- 3.4 Disassembling the Slide-Rails

- 3.5 Installing the Mounting Brackets Onto Oracle Exalytics In-Memory Machine

- 3.6 Attaching the Slide-Rail Assemblies to the Rack

- 3.7 Installing Oracle Exalytics In-Memory Machine into the Slide-Rail Assemblies

- 3.8 Installing the Cable Management Arm

- 3.9 Verifying the Operation of the Slide Rails and CMA

4 Adding Oracle Exalytics In-Memory Machine to Your Network

- 4.1 Before You Begin

- 4.1.1 Switch Requirements

- 4.1.2 Cable Requirements

- 4.2 Overview of Network Ports

- 4.3 Transceiver and Optical Modules for 10 GbE Connectivity

- 4.4 Universal Host Bus Adapter (HBA) for Fibre Channel (FC) and Fibre Channel over Ethernet (FCoE) Connectivity

- 4.4.1 Installing the Universal Host Bus Adapter

- 4.5 Default ETH0 IP Address

- 4.6 Connecting Exalytics In-Memory Machine to Your Network

- 4.7 Configuring the Network for Oracle Exalytics In-Memory Machine

- 4.7.1 Overview

- 4.7.1.1 InfiniBand

- 4.7.1.2 10 GbE Interfaces

- 4.7.1.3 1 GbE Interfaces

- 4.7.1.4 Host Name

- 4.7.1.5 Time

- 4.7.1.6 Domain Name Server (DNS)

- 4.7.2 State of the Oracle Exalytics In-Memory Machine Before Running the Script

- 4.7.3 Running the exalytics_configure_network.sh Script

- 4.7.4 Verifying the Configuration

- 4.8 What Next?

5 Storage Configuration

- 5.1 Overview of Disk Partitions

- 5.2 Connecting Exalytics to a Storage Area Network

- 5.2.1 Before You Begin

- 5.2.1.1 Cable Requirements

- 5.2.1.2 Host Bus Adapter Features and Specifications

- 5.2.2 Preparing for Storage Area Network (SAN) Configuration

- 5.2.3 Connecting the Optical Cable

- 5.2.4 Installing Diagnostic Support for the Red Hat/SUSE Linux OS (Optional)

- 5.2.5 Verifying the Connectivity

- 5.2.6 Completing the Setup

- 5.3 For Additional Help

6 Using Oracle Integrated Lights Out Manager (ILOM)

- 6.1 ILOM Overview

- 6.1.1 ILOM Interfaces

- 6.2 Important Notes Before You Begin

- 6.3 Connecting to ILOM via the Network

- 6.3.1 Connecting to the CLI

- 6.3.2 Connecting to the Web GUI

- 6.3.3 Launching a Remote KVM Session

- 6.4 Connecting to ILOM via a Serial Connection

- 6.5 Configuring Network Access

- 6.5.1 Configuring ILOM IP Addresses Manually

7 Connecting Oracle Exalytics In-Memory Machine to Oracle Exadata Database Machine

- 7.1 Understanding InfiniBand Card, Cable, and Switch Firmware Requirements

- 7.1.1 InfiniBand Card

- 7.1.2 InfiniBand Cables

- 7.1.3 InfiniBand Switches in the Oracle Exadata Database Machine

- 7.2 Physically Connecting the Network

- 7.2.1 Physical Layout Considerations

- 7.2.2 Scenario 1: Single Exalytics In-Memory Machine with Oracle Exadata Database Machine

- 7.2.3 Scenario 2: Clustered Exalytics In-Memory Machine with Oracle Exadata Database Machine

- 7.3 Choosing a Subnet Manager

- 7.4 Configuring the InfiniBand Connection

- 7.5 Monitoring the InfiniBand Fabric

- 7.6 Checking Network Health

- 7.7 Testing the Extended Network Connectivity

8 Connecting Oracle Exalytics In-Memory Machine to Oracle Big Data Appliance

- 8.1 Hardware Requirements

- 8.2 Software Requirements

- 8.2.1 Exalytics Version Details

- 8.2.2 BDA Version Details

- 8.3 Physical Connectivity

- 8.4 InfiniBand Connection Validation (Layer-2)

- 8.5 IPoIB Configuration (Layer-3)

- 8.6 IPoIB Validation

9 Installing Auto Service Request (ASR) Software

- 9.1 About Oracle Auto Service Request (ASR)

- 9.2 Recommended Configuration

- 9.3 Before You Begin

- 9.4 Installing ASR Manager on a Standalone System

- 9.5 Activating Exalytics Assets Manually

- 9.6 Viewing and Verifying ASR Assets ASR Manager

- 9.7 Configuring Telemetry Sources

- 9.8 Enabling Exalytics Machine Assets in My Oracle Support

- 9.9 Sending Test Alerts

- 9.10 Fault Coverage

A Troubleshooting

- A.1 Troubleshooting the Setup

- A.2 Additional Troubleshooting Information

- A.3 Locating the Machine Serial Number

Preface

This guide describes how to set up and commission the Oracle Exalytics In-Memory Machine in your data center. It includes information about hardware operations and site planning, as well as physical, electrical, and environmental specifications.

Note:
All hardware-related specifications in this guide are based on information for a typical deployment provided by Oracle at the time this guide was written. Oracle is not responsible for hardware problems that may result from following the average specifications in this document. For detailed information about preparing your site for Oracle Exalytics In-Memory Machine deployment, consult your hardware specification.

This guide applies to Oracle Exalytics X2-4, X3-4, X4-4, and X5-4 Release 2 (2.0).

This preface contains the following sections:

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

Audience

This guide is intended for Oracle Exalytics In-Memory Machine customers and those responsible for data center site planning.

It is assumed that the readers of this manual have knowledge of the following:

- System administration concepts

- Hardware and networking concepts

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.

Related Documents

For more information, see the following documents:

- Oracle Fusion Middleware 11g Release 1 (11.1.1.6.0) Documentation Library

- Oracle Business Intelligence 11g Release 1 (11.1.1.6.0) Documentation

- Oracle Exalytics In-Memory Machine Installation and Administration Guide for Linux

Conventions

The following text conventions are used in this document:

Convention

Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.

italic

Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.

monospace

Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.

1 Getting Started

This chapter describes how to get started with your Oracle Exalytics In-Memory Machine.

It discusses the following topics:

- Section 1.1, "Tools and Equipment Needed"

- Section 1.2, "Installation Tasks"

- Section 1.3, "Opening the Box"

- Section 1.4, "ESD Precautions"

- Section 1.5, "Oracle Exalytics In-Memory Machine Description"

- Section 1.6, "Connecting the Cables"

- Section 1.7, "Connecting the Power Cords"

1.1 Tools and Equipment Needed

To install the system, you need the following tools:

- No. 2 Phillips screwdriver

- ESD mat and grounding strap

- Pencil, stylus, or other pointed device, for pushing front panel buttons

You also need a system console device, such as one of the following:

- A laptop or Sun workstation

- ASCII terminal

- Terminal server

1.2 Installation Tasks

This section summarizes a list of tasks that you must perform to install the Oracle Exalytics In-Memory Machine.

Table 1-1 Server Installation Tasks

Step

Description

For More Information

1

Unpack the machine and any optional components ordered for the Oracle Exalytics In-Memory Machine from the shipping containers.

See Opening the Box

2

Review electrostatic damage (ESD) precautions before installing the Oracle Exalytics In-Memory Machine.

See ESD Precautions

2

Install the Oracle Exalytics In-Memory Machine into a rack using slide-rails.

See Installing Oracle Exalytics In-Memory Machine Into a Rack

3

Connect cables to the server.

See Connecting the Cables

4

Connect power cords.

See Connecting the Power Cords

1.3 Opening the Box

Carefully open the shipping box. Unpack all server components from the packing cartons.

After opening the box, verify the server box contents and any optional components.

1.3.1 Server Box Contents

The server box should contain the following items:

- Oracle Sun Server X4-4

- Power cord, packaged separately with country kit

- Rackmount kit containing rack rails and installation instructions

- (Optional) Oracle Sun Server X4-4 Server Documentation and Media Kit, including the following:

- License and safety documentation

- Tools and Drivers DVD (includes drivers and additional software), Oracle Hardware Installation Assistant CD, and SunVTS CD

1.4 ESD Precautions

Electronic equipment is susceptible to damage by static electricity. Use a grounded antistatic wrist strap, foot strap, or equivalent safety equipment to prevent electrostatic damage (ESD) when you install or service the Oracle Exalytics In-Memory Machine.

Caution:
To protect electronic components from electrostatic damage, which can permanently disable the system or require repair by authorized service technicians, place components on an antistatic surface, such as an antistatic discharge mat, an antistatic bag, or a disposable antistatic mat. Wear an antistatic grounding strap connected to a metal surface on the chassis when you work on system components.

1.5 Oracle Exalytics In-Memory Machine Description

This section shows the connectors and buttons on the front and back panels of the Oracle Exalytics In-Memory Machine.

It contains the following topics:

- Front Panel Features

- Back Panel Features

1.5.1 Front Panel Features

Figure 1-1 shows the front panel of the Oracle Exalytics In-Memory Machine.

Figure 1-1 Oracle Exalytics In-Memory Machine Front Panel

Table 1-2 describes the front panel buttons and LED indicators.

Table 1-2 Figure Legend

Number

Description

1

Locator LED/Locator button: white

2

Service Action Required LED: amber

3

Main Power/OK LED: green

4

Power button

5

SP OK/Fault LED: green/amber

6

Service Action Required LEDs (3) for Fan Module (FAN), Processor (CPU) and Memory: amber

7

Power Supply (PS) Fault (Service Action Required) LED: amber

8

Over Temperature Warning LED: amber

9

USB 2.0 connectors (2)

10

DB-15 video connector

11

Hard disk drive 0

12

Hard disk drive 1

13

Hard disk drive 2

14

Hard disk drive 3

15

Hard disk drive 4

16

Hard disk drive 5

1.5.2 Back Panel Features

Figure 1-2 and Figure 1-3 shows the back panel of the Oracle Exalytics In-Memory Machine X2-4, and Oracle Exalytics In-Memory Machine X3-4, respectively.

Figure 1-2 Oracle Exalytics In-Memory Machine X2-4 Back Panel

Figure 1-3 Oracle Exalytics In-Memory Machine X3-4 Back Panel

Table 1-3 describes the back panel buttons and LED indicators.

Table 1-3 Figure Legend

Number

Description

1

Power supply unit 0 status indicator LEDs:

Service Action Required: amber

DC OK: green

AC OK: green or amber

2

Power supply unit 0 AC inlet

3

Power supply unit 1 status indicator LEDs:

Service Action Required: amber

DC OK: green

AC OK: green or amber

4

Power supply unit 1 AC inlet

5

System status LEDs:

Power/OK: green

Attention: amber

Locate: white

6

PCIe card slots 0–4

7

Cluster card slot

8

Network (NET) 10/100/1000 ports: NET0–NET3

9

USB 2.0 connectors (2)

10

PCIe card slots 5–9

11

Service processor (SP) network management (NET MGT) port

12

Serial management (SER MGT)/RJ-45 serial port

13

DB-15 video connector

Figure 1-4 Oracle Exalytics In-Memory Machine X5-4 and X4-4 Back Panel

Table 1-3 describes the back panel buttons and LED indicators on the Oracle Exalytics In-Memory Machine X5-4 and X4-4 Back Panel.

Table 1-4 Figure Legend for Oracle Exalytics In-Memory Machine X5-4 and X4-4 Back Panel

Number

Description

1

Power supply unit 0 status indicator LEDs:

Service Action Required: amber

DC OK: green

AC OK: green or amber

2

Power supply unit 0 AC inlet

3

Power supply unit 1 status indicator LEDs:

Service Action Required: amber

DC OK: green

AC OK: green or amber

4

Power supply unit 1 AC inlet

5

System status LEDs:

Power/OK: green

Attention: amber

Locate: white

6

PCIe card slots 1–6

7

Network (NET) 10/100/1000 ports: NET0–NET3

8

USB 2.0 connectors (2)

9

PCIe card slots 7–11

10

Service processor (SP) network management (NET MGT) port

11

Serial management (SER MGT)/RJ-45 serial port

12

DB-15 video connector

1.6 Connecting the Cables

Connect cables to the Oracle Exalytics In-Memory Machine as follows:

1. Connect an Ethernet cable to the Gigabit Ethernet (NET0) connectors as needed for operating system support. See the part 8 in Figure 1-2.

2. (Optional) If you want to interact with the system console directly, connect any external devices, such as a mouse and keyboard, to the server's USB connectors, or a monitor to the DB-15 video connector. See parts 9 and 13 in Figure 1-2.

3. If you plan to configure network IP address for Oracle Integrated Lights Out Manager (ILOM), connect an Ethernet cable to the Ethernet port labeled SER MGT. See the part 12 in Figure 1-2.

4. If you plan to access the ILOM command-line interface (CLI) using the serial management port, connect a serial null modem cable to the RJ-45 serial port labeled SER MGT. See the part 12 in Figure 1-2.

1.7 Connecting the Power Cords

Connect the power cords to the Oracle Exalytics In-Memory Machine as follows:

1. Connect two grounded server power cords to grounded electrical outlets.

2. Connect the two server power cords to the AC connectors on the back panel of the server.

When power is connected, the server boots into standby power mode. In standby power mode, the SP OK/Fault LED flashes, but the main Power/OK LED remains off. After a few minutes, the main Power/OK LED slowly flashes the standby pattern, indicating the service processor (SP) is working. Note that the server is not initialized or powered on yet.

Figure 1-5 Front Panel Controls

Figure Legend

1 - Power/OK LED

2 - Power button

3 - SP OK/Fault LED

2 Introduction

This chapter describes the features and hardware components of an Oracle Exalytics In-Memory Machine.

It contains the following topics:

- Section 2.1, "About Oracle Exalytics In-Memory Machine"

- Section 2.2, "Hardware Configuration"

- Section 2.3, "Assigning PCIe Slots"

- Section 2.4, "Key Features"

- Section 2.5, "Environmental Specifications"

- Section 2.6, "Safety and Compliance Information"

2.1 About Oracle Exalytics In-Memory Machine

Oracle Exalytics In-Memory Machine is an engineered solution that includes a memory-centric hardware platform, proven in-memory technology from Oracle TimesTen, and the breadth of Oracle Business Intelligence Foundation Suite functionality. Oracle Exalytics In-Memory Machine enables Oracle Business Intelligence software users to gain quick insight, make better decisions, and take immediate actions. For system administrators, the pre-engineered system simplifies the process of setting up and maintaining an enterprise-level, high-performance Business Intelligence implementation. It eliminates the risk associated with procuring the system infrastructure from multiple vendors. Oracle's technology is designed to scale seamlessly from small departmental installations to large scale enterprise Business Intelligence deployments.

2.2 Hardware Configuration

Table 2-1 describes the configuration for each of the hardware components of the Oracle Exalytics In-Memory Machine.

Table 2-1 Oracle Exalytics In-Memory Machine Hardware

Component

Description

Core Components

Compute Server

For X5-4: Oracle Server X5-4

For X4-4: Sun Server X4-4

For X3-4 and earlier: One Sun Fire X4470 M2 server

Power Supply

Two power supply units: 1U A239A 2000-watt AC power supply

System Processor

For X5-4: Four CPUs: E7-8895v3, 18C, 2.6 GHz, 155W, 44MB Cache, 8 GT/s QPI, Intel Turbo Boost Technology, Intel HT Technology Processor with Heatsink. X-Option. RoHS

For X4-4: Four CPUs: E7-8895v2, 10C, 2.8 GHz, 155W, 37.5MB Cache, 8 GT/s QPI, Intel Turbo Boost Technology, Intel HT Technology Processor with Heatsink. X-Option. RoHS

For X3-4 and earlier: Four CPUs: E7-4870, 10C, 2.4 GHz, 130W, 30MB Cache, 6.4 GT/s QPI, Intel Turbo Boost Technology, Intel HT Technology Processor with Heatsink. X-Option. RoHS

System Memory

For X5-4: Sixty-four dual in-line memory modules: 32 GB, DDR3 LRDIMM-1600 (total system memory: 2 TB per server, upgradable to 3 TB)

For X4-4: Sixty-four dual in-line memory modules: 32 GB, DDR3 LRDIMM-1600 (total system memory: 2 TB per server)

For X3-4: Sixty-four dual in-line memory modules: 32 GB, DDR3 RDIMM-1066 (total system memory: 2 TB per server)

For X2-4: Sixty-four dual in-line memory modules: LV DIMM, 16 GB, DDR3-1066, 1.35V (total system memory: 1 TB per server)

SAS Host Bus Adapter (HBA)

For X5-4 and X4-4: One Sun Storage 6 Gb SAS PCIe RAID host bus adapter

For X3-4 and earlier: One SAS-2 8-port internal (2x4), 6 Gb, RAID PCIe Gen 2 host bus adapter

InfiniBandHost Channel Adapter (HCA)

For X5-4 and X4-4: One Sun Dual Port QDR Infiniband host channel adapter for PCI Express Gen 3; Infiniband CX3

For X3-4 and earlier: One 2-port QDR InfiniBand host channel adapter

Ethernet Network Interface Card (NIC)

For X5-4: Four 10Gbps Ethernet ports based on two Intel 82599 10GbE controllers

For X4-4: None

For X3-4 and earlier: One Dual-Port 10 Gigabit-Ethernet PCIe 2.0 Copper/Fiber SFP+ card

Fiberchannel Card

For X5-4 and X4-4: Sun Storage 16 Gb Fiber Channel PCIe Universal HBA, QLogic

For X3-4 and earlier: One 8Gb Dual-Port Fiber Channel PCI-Express card

Hard Disk Drives (HDD)

For X5-4: Six 2.5 inch 1.2 TB 10K RPM SAS3 front-accessible, hot-swappable, drive bays (total HDD memory: 7.2 TB per server)

For X4-4: Six 2.5 inch 1.2 TB 10K RPM SAS2 hard disk drives (total HDD memory: 7.2 TB per server)

For X3-4: Six 2.5 inch 900 GB 10K RPM SAS2 hard disk drives (total HDD memory: 5.4 TB per server)

For X2-4: Six 2.5 inch 600 GB 10K RPM SAS2 hard disk drives (total HDD memory: 3.6 TB per server)

Cooling Fans

Six hot-swappable, redundant fans at chassis front (top-loading); redundant fans in each power supply

Flash cards

For X5-4: Three 1.6 TB NVMe PCIe Cards (4.8 TB total)

For X4-4: Three 800 GB Sun FlashAccelerator F80 PCIe cards

For X3-4: Six 400 GB Sun FlashAccelerator F40 PCIe cards

Accessories

Rack Mount Slide Rail Kit

One tool-less rack-mount slide-rail kit

Cable Management Arm (CMA)

One cable management arm

Ports and I/O Slots on the Server

USB 2.0 Ports

Two front, two rear, and one internal

VGA Ports

One front and one rear high-density DB-15 video port

The rear VGA port supports VESA Device Data Channel for monitor identification.

PCI Express 2.0 I/O Slots

Ten PCI Express 2.0 slots that accommodate low-profile PCIe cards.

- Slots 0 and 9: x4 electrical interface

- Slots 1, 2, 4, 6, 7, and 8: x8 electrical interface

- Slots 3 and 5: x8 or x16 electrical interface (x16 connector)

Ethernet Ports

For X5-4 and X4-4: The four ports that can be used for either 10GbE or for slower 1GbE connections

Intel Ethernet Controller X540

For X3-4 and earlier: Four 10/100/1000 RJ-45 GbE ports on rear panel

Each Network Interface Controller (NIC) supports Intel QuickData Technology, Intel I/OAT, VMDq, PCI-SIG SR-IOV, IPSec offload, and LinkSec.

2.3 Assigning PCIe Slots

Table 2-2 describes the assignment of PCIe cards in the slots on the Oracle Exalytics In-Memory Machine X4-4.

Table 2-2 Oracle Exalytics In-Memory Machine PCIe Card Slots in Exalytics X5-4

PCIe Card

Slot Number for X5-4

Sun Flash Accelerator F80 800GB eMLC PCI-E 2.0 Low Profile Adapter

10,7,6

Aspen-INT: Sun Storage 12 Gb SAS PCIe HBA; Internal

8

Sun Dual Port QDR Infiniband Host Channel Adapter for PCI-Express Gen 3; Infiniband CX3

4

Sun Storage 16 GB Fiber Channel PCIe Universal HBA, QLogic

11

Table 2-3 describes the assignment of PCIe cards in the slots on the Oracle Exalytics In-Memory Machine X4-4.

Table 2-3 Oracle Exalytics In-Memory Machine PCIe Card Slots in Exalytics X4-4

PCIe Card

Slot Number for X4-4

Sun Flash Accelerator F80 800GB eMLC PCI-E 2.0 Low Profile Adapter

10,7,6

Sun Storage 6 Gb SAS PCIe RAID HBA; Internal

2

Sun Dual Port QDR Infiniband Host Channel Adapter for PCI-Express Gen 3; Infiniband CX3

4

Sun Storage 16 GB Fiber Channel PCIe Universal HBA, QLogic

11,8

Table 2-4 describes the assignment of PCIe cards in the slots on the Oracle Exalytics In-Memory Machine X2-4 and X3-4.

Table 2-4 Oracle Exalytics In-Memory Machine PCIe Card Slots in Exalytics X3-4 and X2-4

PCIe Card

Slot Number for X2-4

Slot Number for X3-4

8Gb Dual-Port Fiber Channel PCI-Express Card

0

0

Dual-Port 10 Gigabit-Ethernet PCIe 2.0 Copper/Fiber SFP+

2

9

SAS-2 8-port internal (2x4), 6Gb, RAID PCIe Gen 2 HBA

4

4

2-port QDR InfiniBand HCA

6

6

Sun FlashAccelerator F40 PCIe Card, 400GB eMLC

NA

Slots: 1,2,3,5,7,8

2.4 Key Features

Oracle Exalytics In-Memory Machine includes the following features:

- Complete, pre-assembled, and certified system for maximum performance

- Best platform for end-to-end consolidation of large-scale application deployments

- Ready to deploy

- Linearly scalable

- Enterprise-ready

- Enterprise-level software and hardware support

2.5 Environmental Specifications

Table 2-5 describes the environmental specifications for the Oracle Exalytics In-Memory Machine.

Table 2-5 Environmental Specifications for Oracle Exalytics In-Memory Machine

Component

Description

Environment

- Operating temperature: 5°C to 35°C (41°F to 95°F) at sea level; 5°C to 31°C (41°F to 88°F) at altitude

- Non-operating temperature: -40°C to 68°C (-40°F to 154°F)

- Operating relative humidity: 10% - 90% relative humidity, non-condensing

- Acoustic noise: LwAd: 8.9 B (idle and operating, room temperature), 8.9 B (maximum ambient); LpAm: 75 dBA (bystander position, maximum ambient)

Power

- AC power: 200 - 240 VAC (2 or 4 CPUs)

- Power consumption measurements with CPUs at maximum utilization:

- For X5-4 and X4-4: 1600W

- For X3-4: 1467W

- For X2-4: 1317W

- Active Idle (operating systems booted and stabilized, yet running at minimal utilization):

- For X5-4 and X4-4: 723W

- For X3-4: 723W

- For X2-4: 663W

Regulations

- Safety: IEC 60950, UL/CSA 60950. CB Scheme with all country differences

- RFI/EMI: FCC CFR 47 Part 15 Class A, EN 55022 Class A, EN 61000-3-2, EN 61000-3-3, EN 300 386

- Immunity: EN 55024, EN 300 386

Certifications

- Safety: UL/cUL, UL DEMKO, CE, BSMI, CCC, GOST-R, S-Mark

- EMC: CE, FCC, VCCI, ICES, C-Tick, MIC, CCC, GOST-R, BSMI Class A

- Other: Complies with WEEE Directive (2002/96/EC) and RoHS Directive (2002/95/EC)

Physical Specifications

- Weight:
For X5-4 and X4-4: 40 kg (88 lb.) maximum

For X3-4: 44 kg (97 lb.) maximum

For X2-4: 38.5 kg (85 lb.) maximum

- Height: 129.85 mm (5.1 in.)

- Width: 436.5 mm (17.2 in.)

- Depth: 732 mm (28.8 in.)

Mounting Option

- Tool-less rack-mounting slide-rail kit

- Cable management arm

- Two 5m InfiniBand cables

2.6 Safety and Compliance Information

For safety and compliance information about your Oracle Exalytics In-Memory Machine X4-4, see Table 2-6, Table 2-7, and Table 2-8.

Table 2-6 Product Safety Specifications Data Sheet Content

RegulationsFoot 1 Foot 2

- Product Safety: UL/CSA 60950-1, EN 60950-1, IEC 60950-1 CB Scheme with all country differences

- RFI/EMC:

- Immunity:

CertificationsFoot 3

North America (NRTL), European Union (EU), International CB Scheme

European Union Directives

2006/95/EC (73/23/EEC) Low Voltage Directive,

Footnote 1 For additional information to the latest standard, contact your sales representative.

Footnote 2 Other country regulations may apply.

Footnote 3 Other country certifications may apply.

Table 2-7 Electromagnetic Compatibility (EMC) Standards/Regulations

EN55022

CNS-13438 Class A (Taiwan)

FCC CFR 47 CFR 15B Class A (Code of Federal Regulations, Title 47, Part 15, Subpart B Class A (United States)

ICES-003 (A) NMB-3 (A) Class A (Canada)

ETSI EN300386 (V1.6.1)

VCCI: 2008 Class A (Japan)

KN 22 RRL Public Notice 2009-9 (Dec. 21, 2009

EN55024 (Immunity)

EN61000-3-2

EN61000-3-3

Restriction of Hazardous Substances (RoHS) Directive 2011/65/EU

Waste Electrical and Electronic Equipment (WEEE) Directive 2002/96/EC

EMC Directive 2004/108/EC

Table 2-8 Certification Markings on product

Safety Marks: UL

Regulatory Marks: CE, FCC, EAC, RCM, VCCI, BSMI, MSIP, ICES-003 (A) NMB-3 (A), WEEE symbol, China RoHS Symbol, India BIS Registration Declaration.

For safety and compliance information about your Oracle Exalytics In-Memory Machine X2-4 and X3-4, see the following documents:

- Important Safety Information for Sun Hardware Systems – Printed document included in the ship kit.

- Sun Fire X4470 M2 Server Safety and Compliance Guide – Online at Oracle's Sun documentation site.

3 Installing Oracle Exalytics In-Memory Machine Into a Rack

This chapter describes how to optionally place your Oracle Exalytics In-Memory Machine into a rack using the rail assembly in the rackmount kit. The rail assembly is included in your Oracle Exalytics In-Memory Machine package.

Note:
A rack is not shipped with the Oracle Exalytics In-Memory Machine. You should use an existing four-post rack that meets the standards listed in Table 3-1.

This chapter contains the following topics:

- Section 3.1, "Task Roadmap"

- Section 3.2, "Reviewing Rack Compatibility"

- Section 3.3, "Before You Begin"

- Section 3.4, "Disassembling the Slide-Rails"

- Section 3.5, "Installing the Mounting Brackets Onto Oracle Exalytics In-Memory Machine"

- Section 3.6, "Attaching the Slide-Rail Assemblies to the Rack"

- Section 3.7, "Installing Oracle Exalytics In-Memory Machine into the Slide-Rail Assemblies"

- Section 3.8, "Installing the Cable Management Arm"

- Section 3.9, "Verifying the Operation of the Slide Rails and CMA"

3.1 Task Roadmap

Installing your Oracle Exalytics In-Memory Machine into a four-post rack using the slide-rail and cable management arm options involves the following steps:

1. Reviewing Rack Compatibility

2. Disassembling the Slide-Rails

3. Installing the Mounting Brackets Onto Oracle Exalytics In-Memory Machine

4. Attaching the Slide-Rail Assemblies to the Rack

5. Installing Oracle Exalytics In-Memory Machine into the Slide-Rail Assemblies

6. Installing the Cable Management Arm

7. Verifying the Operation of the Slide Rails and CMA

3.2 Reviewing Rack Compatibility

Verify that your rack is compatible with the slide-rail and cable management arm (CMA) options. The optional slide-rails are compatible with a wide range of equipment racks that meet the standards listed in.

Table 3-1 Rack Compatibility for Oracle Exalytics In-Memory Machine

Item

Requirement

Structure

Four-post rack (mounting at both front and rear). Two-post racks are not compatible.

Rack horizontal opening and unit vertical pitch

Conforms to ANSI/EIA 310-D-1992 or IEC 60927 standards. Only M6 tapped or 9.5 mm square are supported.

Distance between front and rear mounting planes

Minimum 610 mm and maximum 915 mm

(24 inches to 36 inches).

Clearance depth in front of front mounting plane

Distance to front cabinet door is at least 25.4 mm

(1 inch).

Clearance depth behind front mounting plane

Distance to rear cabinet door is at least 900 mm (35.5 inches) with the cable management arm, or 770 mm (30.4 inches) without the cable management arm.

Clearance width between front and rear mounting planes

Distance between structural supports and cable troughs is at least 456 mm (18 inches).

Machine dimensions

Depth: (not including PSU handle): 732 mm (28.82 inches)

Width: (not including ears): 436.5 mm (17.19 inches)

Height: 129.85 mm (5.11 inches)

3.3 Before You Begin

Before installing Oracle Exalytics In-Memory Machine into a rack, read the following cautions and important notes:

- Always load equipment into a rack from the bottom up, so that the rack will not become top-heavy and tip over. Deploy your rack's anti-tip bar to prevent the rack from tipping during equipment installation.

- If the Oracle Exalytics In-Memory Machine is installed in a closed or multi-unit rack assembly, the operating ambient temperature of the rack environment might be greater than room ambient temperature. Therefore, consideration should be given to installing the equipment in an environment compatible with the maximum ambient temperature (Tma) specified for the Oracle Exalytics In-Memory Machine.

- Installation of the equipment in a rack should be such that the amount of air flow required for safe operation of the equipment is not compromised.

- Mounting of the equipment in the rack should be such that a hazardous condition is not achieved due to uneven mechanical loading.

- Consideration should be given to the connection of the equipment to the supply circuit and the effect that overloading of the circuits might have on over-current protection and supply wiring. Appropriate consideration of equipment nameplate power ratings should be used when addressing this concern.

- Reliable earthing of rackmounted equipment should be maintained. Particular attention should be given to supply connections other than direct connections to the branch circuit (for example, use of power strips).

- Slide-rail mounted equipment must not be used as a shelf or a work space.

3.4 Disassembling the Slide-Rails

To disassemble the slide-rails before the installation, complete the following steps:

1. Unpack the slide-rails.

2. Locate the slide-rail lock at the front of the slide-rail assembly, as shown in Figure 3-1.

3. Press and hold the slide-rail lock toward the direction of the arrow, while you pull the mounting bracket out of the slide-rail assembly until it reaches the stop, as shown in Figure 3-1.

4. Push the mounting bracket release button toward the front of the mounting bracket (Figure 3-1), and simultaneously withdraw the mounting bracket from the slide-rail assembly.

Figure 3-1 Disassembling the Slide-Rails Before Installation

Table 3-2 Figure Legend

Number

Description

1

Slide-rail lock

2

Mounting bracket release button

3.5 Installing the Mounting Brackets Onto Oracle Exalytics In-Memory Machine

You must install the mounting brackets onto the Oracle Exalytics In-Memory Machine before you can rackmount it.

To install the mounting brackets, complete the following steps:

1. Position a mounting bracket against the chassis so that the slide-rail lock is at the Oracle Exalytics In-Memory Machine front, and the five keyhole openings on the mounting bracket are aligned with the five locating pins on the side of the chassis, as shown in Figure 3-2.

Note:
The mounting brackets are identical and can be installed on either side of the chassis.

Figure 3-2 Installing the Mounting Brackets

2. With the heads of the five chassis locating pins protruding though the five keyhole openings in the mounting bracket, pull the mounting bracket toward the front of the chassis until the mounting bracket clip locks into place with an audible click, as shown in Figure 3-2.

3. Verify that the rear locating pin has engaged the mounting bracket clip.

4. Repeat to install the remaining mounting bracket on the other side of the Oracle Exalytics In-Memory Machine.

3.6 Attaching the Slide-Rail Assemblies to the Rack

The slide rail assemblies support only racks with 9.5-mm square holes and M6 round holes. All other racks, including those racks with 7.2-mm, M5, or 10-32 mounting holes, are not supported. See your rack documentation for information about the size of its rail holes.

To attach the slide-rail assemblies to the rack, complete the following steps:

1. (Optional) If you need to move the rack with the Oracle Exalytics In-Memory Machine installed, it is recommended that you attach the slide-rail assembly with mounting screws and cage nuts.

Note:
Insert the cage nuts prior to performing the next step. See the Rail Rackmount Kit Overview and Information card for instructions on inserting these cage nuts. This card is included with the rack kit.

2. Position a slide-rail assembly in your rack, so that the slide-rail assembly front bracket is on the outside of the front rack post and the slide-rail assembly rear bracket is on the inside of the rear rack post, as shown in Figure 3-3.

3. Align the slide-rail assembly mounting pins with the front and rear rack post mounting holes. Then lock the assembly into place by pushing the assembly toward the rear of the rack until the mounting pins engage the rack, as shown in Figure 3-4.
You will hear an audible click when the mounting pins engage the rack.

Note:
The slide assembly mounting pins accommodate either 9.5 mm square mounting holes or M6 round mounting holes. No other mounting hole sizes are supported.

Figure 3-3 Installing Slide-Rail Assemblies Onto the Rack

Figure 3-4 Slide-Rail Mounting Pins

4. (Optional) If you choose to attach the slide-rail assembly with mounting screws and cage nuts, insert the M6 mounting screws through both front and rear slide-rail brackets and rack posts, then secure them with the caged nuts. See Figure 3-5.

Figure 3-5 Attach Slide-Rail Assembly to a Rack

5. Repeat Step 2 through Step 4 for the remaining slide-rail assembly.

6. If available, extend the anti-tip bar at the bottom of the rack, as shown in Figure 3-6. Refer to your rack documentation for instructions.

Figure 3-6 Extending the Anti-tip Bar

Caution:
If your rack does not have an anti-tip bar, the rack could tip over.

3.7 Installing Oracle Exalytics In-Memory Machine into the Slide-Rail Assemblies

Use this procedure to install the Oracle Exalytics In-Memory Machine chassis, with mounting brackets, into the slide-rail assemblies that are mounted to the rack.

Caution:
This procedure requires a minimum of two persons because of the weight of the Oracle Exalytics In-Memory Machine. Attempting this procedure alone could result in equipment damage or personal injury.
Always load equipment into a rack from the bottom up so that the rack will not become top-heavy and tip over. Extend your rack's anti-tip bar to prevent the rack from tipping during equipment installation.

To install Oracle Exalytics In-Memory Machine into the slide-rail assemblies, complete the following steps:

1. Push the slide-rails into the slide-rail assemblies in the rack as far as possible.

2. Raise the Oracle Exalytics In-Memory Machine so that the rear ends of the mounting brackets are aligned with the slide-rail assemblies that are mounted in the rack, as shown in Figure 3-7.

3. Insert the mounting brackets into the slide-rails, then push the server into the rack until the mounting brackets encounter the slide-rail stops (approximately 12 inches, or 30 cm).

Caution:
When inserting the Oracle Exalytics In-Memory Machine into the slide-rail, ensure that both the top and bottom mounting lips of the mounting brackets are inserted into the slide-rail. The Oracle Exalytics In-Memory Machine should slide forward and backward easily if correctly installed. If the unit does not slide easily, ensure that each mounting lip is inserted properly. If the mounting brackets are not inserted properly, the unit may fall when removing it from the rack.

Figure 3-7 Inserting the Oracle Exalytics In-Memory Machine with Mounting Brackets into the Slide-Rails

4. Simultaneously push and hold the green slide-rail release buttons (Figure 3-1) on each mounting bracket while you push the Oracle Exalytics In-Memory Machine into the rack (Figure 3-8). Continue pushing until the slide-rail locks (on the front of the mounting brackets) engage the slide-rail assemblies. You will hear an audible click.

Figure 3-8 Sliding the Oracle Exalytics In-Memory Machine Into the Rack

Caution:
Verify that the Oracle Exalytics In-Memory Machine is securely mounted in the rack and that the slide-rail locks are engaged with the mounting brackets before continuing.

3.8 Installing the Cable Management Arm

The cable management arm (CMA) is an optional assembly that you can use to route the server cables in the rack.

To install the optional CMA, complete the following steps:

1. Unpack the CMA parts.

2. Take the CMA to the back of the equipment rack and ensure that you have adequate room to work around the back of the Oracle Exalytics In-Memory Machine.

Note:
References to left or right in this procedure assume that you are facing the back of the equipment rack.

3. Remove tape to separate the parts of the CMA.

4. Insert the CMA's mounting bracket connector into the right slide-rail until the connector locks into place with an audible click, as shown in Figure 3-9.

Figure 3-9 Inserting the CMA Mounting Bracket Into the Back of the Right Slide-Rail

Table 3-3 Figure Legend

Number

Description

1

CMA mounting bracket

2

Right slide-rail

5. Insert the right CMA slide-rail connector into the right slide-rail assembly until the connector locks into place with an audible click, as shown in Figure 3-10.

Figure 3-10 Inserting CMA Slide-Rail Connector Into the Back of the Right Slide-Rail

Table 3-4 Figure Legend

Number

Description

1

CMA slide-rail connector

2

Right slide-rail

6. Insert the left CMA slide-rail connector into the left slide-rail assembly until the connector locks into place with an audible click, as shown in Figure 3-11.

Figure 3-11 Inserting CMA Slide-Rail Connector Into the Back of the Left Slide-Rail

Table 3-5 Figure Legend

Number

Description

1

CMA slide-rail connector

2

Left slide-rail

7. Install and route cables to your Oracle Exalytics In-Memory Machine, as required.

8. If required, attach the cable hook and loop straps to the CMA, and press them into place to secure the cables, as shown in Figure 3-12.

Note:
Cable hooks and loop straps are preinstalled on the CMA. Perform the procedure in this step if you need to reinstall cable hooks and straps on the CMA.

Figure 3-12 Installing CMA Cable Straps

Table 3-6 Figure Legend

Number

Description

1

CMA cable strap

2

CMA arm

3.9 Verifying the Operation of the Slide Rails and CMA

Use the following procedure to ensure that the slide-rails and CMA are operating correctly.

Note:
Two persons are recommended for this procedure: one to move the Oracle Exalytics In-Memory Machine in and out of the rack, and one to observe the cables and CMA.

1. Slowly pull the Oracle Exalytics In-Memory Machine out of the rack until the slide-rails reach their stops.

2. Inspect the attached cables for any binding or kinks.

3. Verify that the CMA extends fully from the slide-rails.

4. Push the Oracle Exalytics In-Memory Machine back into the rack, as described in the following sub-steps.
When the Oracle Exalytics In-Memory Machine is fully extended, you must release two sets of slide-rail stops to return the Oracle Exalytics In-Memory Machine to the rack:

- The first set of stops are levers, located on the inside of each slide-rail, just behind the back panel of the Oracle Exalytics In-Memory Machine. Push in both green levers simultaneously and slide the Oracle Exalytics In-Memory Machine toward the rack. The Oracle Exalytics In-Memory Machine will slide in approximately 18 inches (46 cm) and stop. Verify that the cables and the CMA retract without binding before you continue.

- The second set of stops are the slide-rail release buttons, located near the front of each mounting bracket (Figure 3-1). Simultaneously push both of the green slide-rail release buttons, and push the Oracle Exalytics In-Memory Machine completely into the rack until both slide-rail locks engage.

5. Adjust the cable straps and CMA, as required.

4 Adding Oracle Exalytics In-Memory Machine to Your Network

This chapter describes how to connect the Oracle Exalytics In-Memory Machine to your existing network. It also describes how to configure the Oracle Exalytics In-Memory Machine using the exalytics_configure_network.sh script.

Note:
You must download the exalytics_configure_network.sh script from the edelivery site. Create directories named exalytics/bin in the /opt directory on your Oracle Exalytics In-Memory Machine and copy the script to this location. Run the script from this location.

It contains the following topics:

- Section 4.1, "Before You Begin"

- Section 4.2, "Overview of Network Ports"

- Section 4.3, "Transceiver and Optical Modules for 10 GbE Connectivity"

- Section 4.5, "Default ETH0 IP Address"

- Section 4.6, "Connecting Exalytics In-Memory Machine to Your Network"

- Section 4.7, "Configuring the Network for Oracle Exalytics In-Memory Machine"

- Section 4.8, "What Next?"

4.1 Before You Begin

Before connecting the Oracle Exalytics In-Memory Machine to your data center's existing network, ensure that the following prerequisites are satisfied:

- Switch Requirements

- Cable Requirements

4.1.1 Switch Requirements

Before connecting the Oracle Exalytics In-Memory Machine to your data center's network, complete the following preparatory tasks:

- Identify the network switch or connector that you will use to connect the Oracle Exalytics In-Memory Machine to your network.

- Identify the IP subnet and the address range associated with the network switch.

4.1.2 Cable Requirements

The following Ethernet cables are included in the Oracle Exalytics In-Memory Machine package:

- One 10-feet (CAT5/CAT5E) red cable (part number: 530-4433-01)

- One 10-feet (CAT5/CAT5E) black cable (part number: 530-4435-01)

You can use these cables to connect the Oracle Exalytics In-Memory Machine to your data center's network.

4.2 Overview of Network Ports

Figure 4-1 shows the network ports available on the Oracle Exalytics In-Memory Machine X4-4.

Figure 4-1 Network Ports on the Back of the Oracle Exalytics In-Memory Machine X5-4 and X4-4

Table 4-1 lists out the ports called out in Figure 4-1.

Table 4-1 Back Panel of Oracle Exalytics In-Memory Machine X4-4

Call out

Description

1

Power supply unit (PSU) 0 indicator panel

2

PSU 0 AC inlet

3

PSU 1 indicator panel

4

PSU 1 AC inlet

5

System status indicator panel

6

PCIe card slots 1–6

7

Network (NET) 10 GbE ports: NET0–NET3

8

USB 2.0 ports (2)

9

PCIe card slots 7–11

10

Service processor (SP) network management (NET MGT) port

11

Serial management (SER MGT)/RJ-45 serial port

12

DB-15 video port

Figure 4-2 and Figure 4-3 show the network ports available on the Oracle Exalytics In-Memory Machine X3-4, and Oracle Exalytics In-Memory Machine X2-4, respectively.

Figure 4-2 Network Ports on the Back of the Oracle Exalytics In-Memory Machine X3-4

Figure 4-3 Network Ports on the Back of the Oracle Exalytics In-Memory Machine X2-4

In Figure 4-3 and Figure 4-2, the four 10/100/1000 Gb Ethernet ports located on the rear of the Oracle Exalytics In-Memory Machine are shown in a red circle of dashed line. They are named NET 0, NET 1, NET 2, and NET 3.

Note:
Although you can use any of these Gb Ethernet ports, Oracle recommends that you use NET 0.

The ILOM 10/100 Ethernet port is shown in a blue circle of solid line. This port is named NET MGT.

4.3 Transceiver and Optical Modules for 10 GbE Connectivity

To use the 10 GbE network ports, you must provide Small Form-factor Pluggable Plus (SFP+) transceivers and/or cables (either optical or copper) to connect the network ports to your existing 10 GbE network infrastructure. The Oracle Exalytics In-Memory Machine does not ship with SFP+ transceivers or 10 GbE cables. You must purchase them separately.For copper cables, you can purchase the following cables from Oracle. These cables have built-in SFP+ transceivers, and there is no need to purchase separate SFP+ transceivers for the network ports on the Oracle Exalytics In-Memory Machine.

Table 4-2 Cables

Name

Length

Oracle Part Number

TwinAx 1m

1m

X2130-1M

TwinAx 3m

3m

X2130-3M

TwinAx 5m

5m

X2130-5M

TwinAx 10m

10m

X2130-10M

For optical cables, you must purchase SFP+ transceivers (either Short Range (SR) or Long Range (LR)) for each of the network ports, and then plug in the appropriate optical cable. Currently Oracle sells both the SR and LR SFP+ transceivers.

Table 4-3 Transceivers

Name

Part Number

10 GbE Transceiver SR (SFP+)

X2129A-N

10 GbE Transceiver LR (SFP+)

X5562A-Z

In addition to these transceivers, you must purchase the appropriate LC-LC terminated fiber optic cables from a third party vendor.

4.4 Universal Host Bus Adapter (HBA) for Fibre Channel (FC) and Fibre Channel over Ethernet (FCoE) Connectivity

Use the Universal Host Bus Adapter to connect to a FC and FCoE networks. The universal HBA is a configurable board that enables you to change its operating protocol mode from a two-port 16 Gb FC HBA to a two-port 10 GbE Fibre Channel over Ethernet (FCoE) Converged Network Adapter. The universal HBA has four possible configurations:

- 10 GbE FCoE Copper — This configuration provides twin-ax copper cable connectivity and 10 GbE FCoE HBA functionality. No optical transceiver modules are installed in, nor provided with, this configuration of the universal HBA.

- 16 Gb FC SW (shortwave) Optical — This configuration requires installing 16 Gb FC shortwave optical transceiver modules in the SFP+ connector of the universal HBA, enabling 16 Gb Fibre Channel HBA functionality.

- 10 GbE FCoE SR (short-range) Optical — This configuration requires installing 10 GbE short-range optical transceiver modules in the SFP+ connectors of the universal HBA, enabling 10 GbE FCoE Converged Network Adapter functionality.

- 16 Gb FC LW (longwave) Optical—This configuration requires installing 16Gb FC long-range optical transceiver modules in the SFP+ connectors of the universal HBA, enabling 16 Gb FC HBA functionality.

4.4.1 Installing the Universal Host Bus Adapter

To install the Universal Host Bus Adapter for connectivity, complete the following steps:

1. Prepare for installation.
If you choose to configure the universal HBA so that the HBA is connected to SFP+ optical transceiver modules, you must install the optical transceiver modules before you install the universal HBA into the system. See Preparing for Installation for the detailed steps.

2. Install the Universal HBA hardware. See Installing the Hardware for detailed steps.

3. Install the Universal HBA software. See Software Installation for detailed steps.

4.5 Default ETH0 IP Address

When an Exalytics In-Memory Machine is delivered, the default IP address that is configured for the eth0 (physical NET0 port) interface of the machine might vary. It can be any IP address in the class C subnet 192.168.1.0.

To determine the eth0 IP address of an Exalytics In-Memory Machine, connect to the ILOM of that machine through SSH using the default IP address 192.168.1.200. After logging in, run the following command to view the network configuration information:

show /SP/network

4.6 Connecting Exalytics In-Memory Machine to Your Network

To connect the Oracle Exalytics In-Memory Machine to your data center's network, complete the following steps:

1. Attach one end of the network cable to a Gb Ethernet port, such as NET 2 (see Figure 4-3).

2. Connect the other end of the cable to a network connector or switch, if the network is already set up in your building; or to a hub, for a client/server network.

3. Ensure that you have a laptop with functional USB and network ports.

4. Ensure that you have a Category 5E patch cable of maximum length 25 feet and a serial cable of maximum length 15 feet.

5. Connect the network port of your laptop into an unused input port in the network connector or switch. Do not connect to any management ports on the network connector or switch.

6. If you have not booted the operating system on your laptop, start the operating system now.

- If you are using the Windows operating system on your laptop, do the following:

1. Go to Control Panel > Network Connections. Select your wired network adapter in the list of network connections, right-click and select Properties. The network properties screen is displayed.

2. Click the General tab, and select Internet Protocol (TCP/IP). Click Properties. The Internet Protocol (TCP/IP) Properties screen is displayed.

3. Select the Use the following IP address: option, and enter a static IP address for your laptop. Although a default gateway is not necessary, enter the same IP address in the Default Gateway field. Click OK to exit the network connections screen.

Note:
This static IP should be on the same subnet and address range as the network on which your data center's network switch or connector resides.

- If you are using a Linux operating system on your laptop, do the following:

1. Log in as a root user.

2. At the command prompt, enter the following command to display the network devices, such as ETH0, attached to the Oracle Exalytics In-Memory Machine:
# ifconfig -a

The list of network devices or adapters attached to the Oracle Exalytics In-Memory Machine is displayed.

3. To set up the desired network interface, run the ifconfig command at the command prompt, as in the following example:
# ifconfig eth0 192.0.2.10 netmask 255.255.255.0 up

In this example, the ifconfig command assigns the IPv4 address 192.0.2.10, with a network mask of 255.255.255.0, to the eth0 interface.

4.7 Configuring the Network for Oracle Exalytics In-Memory Machine

This section describes how to use the exalytics_configure_network.sh script, to configure the network for Oracle Exalytics In-Memory Machine.

It contains the following topics:

- Overview

- State of the Oracle Exalytics In-Memory Machine Before Running the Script

- Running the exalytics_configure_network.sh Script

- Verifying the Configuration

4.7.1 Overview

The exalytics_configure_network.sh script is used to configure the Oracle Exalytics In-Memory Machine after the initial operating system image is installed.

This script verifies the following:

- When started, the script verifies that the user is root. Users other than root are not allowed to run the script.

- It verifies that the Oracle Exalytics In-Memory Machine is installed with a 10 GbE card. If the card is not installed or the drivers are not configured, the script stops its execution.

- It verifies connectivity to all gateways and hostnames configured in the session.

You can run the script to configure the following multiple interfaces:

- InfiniBand

- 10 GbE Interfaces

- 1 GbE Interfaces

- Host Name

- Time

- Domain Name Server (DNS)

Note:
You cannot configure Fiber Channel using the exalytics_configure_network.sh script.

4.7.1.1 InfiniBand

The exalytics_configure_network.sh script optionally configures the InfiniBand network interface by bonding two InfiniBand interfaces. This interface is for IPoIB communication (bond0) intended for Oracle Exalytics In-Memory Machine's connectivity with Exalogic machine or Oracle Exadata Database Machine. You must specify an IP address and netmask for the script. A gateway address for this InfiniBand interface is not necessary.

When you configure InfiniBand using the exalytics_configure_network.sh script, the following files are updated:

/etc/sysconfig/network-scripts/ifcfg-ib*
/etc/sysconfig/network-scripts/ifcfg-bond0
/etc/modprobe.conf

Note:
The script keeps a copy of these files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.1.2 10 GbE Interfaces

The exalytics_configure_network.sh script verifies whether bonding is required on the 10 GbE interfaces. If you opt for bonding, the script configures the bonded interfaces using bond1 on all 10 GbE interfaces. For bond1 interface, the script requires an IP address, netmask, and gateway address.

If you do not opt for a bonded interface, the script verifies whether you want to configure each interface separately. You are prompted to enter an IP address, netmask, and gateway address for each individual interface configured.

When you configure 10 GbE interfaces using the exalytics_configure_network.sh script, the following files are updated:

/etc/sysconfig/network-scripts/ifcfg-eth*
/etc/sysconfig/network-scripts/ifcfg-bond1
/etc/modprobe.conf

Note:
The script keeps a copy of the interface files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.1.3 1 GbE Interfaces

The exalytics_configure_network.sh script verifies whether bonding is required on the 1 GbE interfaces. If you opt for bonding, the script configures the bonded interfaces using bond2 on all 1 GbE interfaces. For bond2 interface, the script requires an IP address, netmask, and gateway address.

If you do not opt for a bonded interface, the script verifies whether you want to configure each interface separately. You are prompted to enter an IP address, netmask, and gateway address for each individual interface configured.

When you configure 1 GbE interfaces using the exalytics_configure_network.sh script, the following files are updated:

/etc/sysconfig/network-scripts/ifcfg-eth*
/etc/sysconfig/network-scripts/ifcfg-bond2
/etc/modprobe.conf

Note:
The script keeps a copy of the interface files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.1.4 Host Name

The exalytics_configure_network.sh script optionally configures a host name and a domain name for your Oracle Exalytics In-Memory Machine. You cannot configure a domain name without configuring a host name.

The configured host name maps to an IP address in the following sequence:

- The Last IP configured in this session.

- If no IP configured, it will take the IP from first active interface from: bond1, bond2, …, eth5.

- If no IP is found, then no IP is configured in the /etc/hosts file.

When you configure host name using the exalytics_configure_network.sh script, the following files are updated:

/etc/sysconfig/network
/etc/hosts

Note:
The script keeps a copy of the interface files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.1.5 Time

The exalytics_configure_network.sh script optionally configures the time zone using the tzselect script. This script guides you to select a continent, country, and city using numbered menu selection. The script verifies whether a network time server should be configured. If you choose to configure a server, the script prompts you to enter the server name and other required information.

If you do not opt for time server configuration, the script prompts you to set time and date.

For network time server configuration, the script modifies the /etc/ntp.conf file, and it starts the ntpd daemon and adds it to start at boot.

For time zone selection, the script creates a new soft link to /etc/localtime.

Note:
The script keeps a copy of the interface files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.1.6 Domain Name Server (DNS)

The exalytics_configure_network.sh script optionally configures one or more domain name servers that you specify.

When you configure domain name servers using the exalytics_configure_network.sh script, the following file is updated:

/etc/resolv.conf

Note:
The script keeps a copy of the interface files in the /tmp/exalytics.save/ directory and creates a flat backup for the files that may be modified.

4.7.2 State of the Oracle Exalytics In-Memory Machine Before Running the Script

Before running the exalytics_configure_network.sh script, your Oracle Exalytics In-Memory Machine is not configured with the following:

- 10 GbE and 1 GbE network interfaces

- Optional InfiniBand network interfaces

- Host name, domain name, and time zone

- Time server and domain name servers

4.7.3 Running the exalytics_configure_network.sh Script

Run the exalytics_configure_network.sh script, and choose your configuration option, as described in Table 4-4.

Table 4-4 exalytics_configure_network.sh Properties

Question/Parameter

Description

Do you want to configure InfiniBand for Exadata connection? [y/n) [n]

Enter y if you want to configure InfiniBand for connecting Oracle Exalytics In-Memory Machine to Oracle Exadata Database Machine.

Enter bond0 IP address

Enter a bond0 IP address. For example, 192.168.21.12.

Use xx.xxx.xx.xx for bond0 ip address (y/n) [y]

Verify the bond0 IP address you specified, and enter y.

Enter bond0 netmask address

Enter a bond0 netmask address. For example, 255.255.255.0.

Use xx.xxx.xxx.xx for bond0 netmask (y/n) [y]

Verify the bond0 netmask address you specified, and enter y.

Do you want to configure 10G network(y/n) [n]

Depending on your configuration option, enter y or n.

Configure bonding on eth0 eth1? (y/n) [n]

Depending on your configuration option, enter y or n.

Configure eth0 Interface? (y/n) [n]

Depending on your configuration option, enter y or n.

Enter eth0 IP address

Enter an eth0 IP address. For example, 10.10.10.10.

Use xx.xxx.xx.xx for eth0 ip address (y/n) [y]

Verify the eth0 IP address you specified, and enter y.

Enter etho0 netmask address

Enter a netmask for the eth0 interface. For example, 255.255.248.0.

Use xx.xxx.xxx.xx for eth0 netmask (y/n) [y]

Verify the eth0 netmask you specified, and enter y.

Enter eth0 gateway address

Enter the eth0 gateway address. For example, 10.20.30.40.

Use xx.xxx.xxx.xx for eth0 gateway (y/n) [y]

Verify the eth0 gateway address you specified, and enter y.

Configure eth1 Interface? (y/n) [n]

Depending on your configuration option, enter y or n.

Disable eth1

Depending on your configuration option, enter y or n.

Do you want to configure 1G network (y/n) [y]

Depending on your configuration option, enter y or n.

Configure bonding on eth2 eth3 eth4 eth5? (y/n) [n]: n

Depending on your configuration option, enter y or n.

Configure eth2 Interface? (y/n) [n]: y

Depending on your configuration option, enter y or n.

Enter eth2 IP address

Enter an eth2 IP address. For example, 10.10.10.11.

Use xx.x.x.xx for eth2 ip address (y/n) [y]: y

Verify the eth2 IP address you specified, and enter y.

Enter eth2 netmask address

Enter a netmask for the eth2 interface. For example, 255.255.255.254.

Use xxx.xxx.xxx.xxx for eth2 netmask (y/n) [y]: y

Verify the eth2 netmask address you specified and then enter y.

Enter eth2 gateway address

Enter an eth2 gateway address. For example, 10.1.1.1.

Use xx.x.x.x for eth2 gateway (y/n) [y]: y

Verify the eth2 gateway address you specified, and enter y.

Configure eth3 Interface? (y/n) [n]: n

Depending on your configuration option, enter y or n.

Disable eth3

Depending on your configuration option, enter y or n.

Configure eth4 Interface? (y/n) [n]: n

Depending on your configuration option, enter y or n.

Disable eth4

Depending on your configuration option, enter y or n.

Configure eth5 Interface? (y/n) [n]: n

Depending on your configuration option, enter y or n.

Disable eth5

Depending on your configuration option, enter y or n.

Do you want to configure hostname (y/n) [n]: y

Depending on your configuration option, enter y or n.

Enter hostname

Enter a hostname for the machine. For example, myhost.

Use xxxxx for host_name (y/n) [y]: y

Verify the hostname you specified, and enter y.

Enter domainname:mydomain

Enter a domain name. For example, mydomain.

Use xxxxx in for domainname (y/n) [y]: y

Verify the domain name you specified, and enter y.

Do you want to configure timezone (y/n) [y]: y

Depending on your configuration option, enter y or n.

Please identify a location so that time zone rules can be set correctly.

Please select a continent or ocean.

Enter your location. For example, if your location is Americas, enter 2.

Please select a country.

Enter your country. For example, if your country is United States, enter 47.

Please select one of the following time zone regions

Enter your time zone. For example, if your time zone is Pacific time, enter 21.

The following information has been given:         United States         Pacific Time Therefore TZ='America/Los_Angeles' will be used. Local time is now:      Thu Oct 20 07:19:36 PDT 2005. Universal Time is now:  Thu Oct 20 14:19:36 UTC 2005. Is the above information OK?

If the information is correct, enter 1.

Here is that TZ value again, this time on standard output so that you can use the /usr/bin/tzselect command in shell scripts: do you want to configure time server  (y/n) [y]: y

If you want to configure a time server, enter y.

Enter network time server:10.162.39.1 Use 10.162.39.1 for timeserver (y/n) [y]: y Do you want to configure DNS(s) (y/n) [y]: y Enter DNS: 10.162.39.57 Use 10.162.39.57 for DNS (y/n) [y]: y configure another DNS? (y/n) [n]: n

If you want to configure one or more domain name servers, enter y.

Commit changes? (y/n) [n]: y

To confirm your configuration options and values, enter y.

4.7.4 Verifying the Configuration

After the exalytics_configure_network.sh script runs successfully, you can verify the network configuration for your Oracle Exalytics In-Memory Machine machine as follows:

- Ping 10 GbE network addresses to verify the 10 GbE network interfaces

- Ping 1 GbE network addresses to verify the 1 GbE network interfaces

- Run the ibstat command on the Oracle Exalytics In-Memory Machine to verify the InfiniBand network interface

- Ping the gateway addresses

- Ping an IP address of a machine that is in the same subnet as Oracle Exalytics In-Memory Machine for each of the interfaces

- Verify the log file (/var/log/init-exalytics/init-exalytics.log)

- Run the ifconfig -a command to verify the network configuration

4.8 What Next?

After configuring network for the Oracle Exalytics In-Memory Machine, you can proceed to install the Oracle Business Intelligence software on the machine. For more information, see the Oracle Exalytics In-Memory Machine Installation and Administration Guide for Linux.

For information about installing Oracle Essbase, see Oracle Hyperion Enterprise Performance Management System Installation and Configuration Guide.

5 Storage Configuration

This chapter describes the storage configuration for the Oracle Exalytics In-Memory Machine.

It contains the following topics:

- Section 5.1, "Overview of Disk Partitions"

- Section 5.2, "Connecting Exalytics to a Storage Area Network"

- Section 5.3, "For Additional Help"

5.1 Overview of Disk Partitions

Table 5-1 lists the disk partitions configured on the Oracle Exalytics In-Memory Machine.

Table 5-1 Disk Partitions

Partition

Capacity

Hardware RAID

Mount Point

Description

boot

99 MB

RAID1

/boot

Boot partition

base operating system, logs, and temp space

X5-4: 670 GB

X4-4: 670 GB

X3-4: 670 GB

X2-4: 447 GB

RAID1

/

Partition for the base operating system, logs, and temp space

Software, data files, and so on

X5-4: 2.4 TB

X4-4: 2.4 TB

X3-4: 2.4 TB

X2-4: 1.797 TB

RAID5

The /u01 disk partition is created and available on this sdb disk.

Data partition

Note:
In addition to the hard-disk storage described in Table 5-1, X3-4 machines have six 400 GB Sun FlashAccelerator F40 PCIe cards (total memory: 2.4 TB RAW) and X4-4 machines have three 800 GB Sun FlashAccelerator F80 PCIe cards (total memory:2.4 TB RAW), mounted on /u02.
For information about configuring the Flash cards, refer to the "Configuring Flash and Replacing a Defective Flash Card" topic in the Oracle Exalytics In-Memory Machine Installation and Administration Guide for Linux.

5.2 Connecting Exalytics to a Storage Area Network

Starting with Oracle Exalytics Release 1 Patchset 1 (1.0.0.1), the Storage Area Network (SAN) feature is supported. This section describes the procedure for connecting Exalytics to a Storage Area Network.

Connecting Exalytics to a Storage Area Network involves the following steps:

- Before You Begin

- Preparing for Storage Area Network (SAN) Configuration

- Connecting the Optical Cable

- Installing Diagnostic Support for the Red Hat/SUSE Linux OS (Optional)

- Verifying the Connectivity

- Completing the Setup

5.2.1 Before You Begin

Before connecting the Oracle Exalytics In-Memory Machine to your data center's existing network, ensure that the following prerequisites are satisfied:

- Cable Requirements

- Host Bus Adapter Features and Specifications

5.2.1.1 Cable Requirements

Fiber-optic media, short-wave, multimode fiber (400-M5- SN-S)

5.2.1.2 Host Bus Adapter Features and Specifications

For more information, see "HBA Features and Specifications" in the StorageTek 8 Gb FC PCI-Express HBA Installation Guide Installation Guide For HBA Models SG-XPCIE1FC-QF8-Z, SG-PCIE1FC-QF8-Z, SG-XPCIE1FC-QF8-N, SG-PCIE1FC-QF8-NandSG-XPCIE2-QF8-Z, SG-PCIE2FC-QF8-Z, SG-XPCIE2-QF8-N, SG-PCIE2FC-QF8-N.

5.2.2 Preparing for Storage Area Network (SAN) Configuration

To prepare Oracle Exalytics In-Memory Machine for SAN Configuration, complete the following steps:

1. Configure the system to enable loading of the following kernal module at boot:

- qla2xxx.ko

- dm-multipath.ko

2. Initialize the /etc/multipath.conf file by running the following command:

/sbin/mpathconf

3. Enable multipath startup during an operating system boot, by running the following commands as the root user:

chkconfig multipathd on
chkconfig --level 345 multipathd on
service multipathd start

5.2.3 Connecting the Optical Cable

To connect the optical cable, complete the following steps:

Note:
The HBA does not allow normal data transmission on an optical link unless it is connected to another similar or compatible Fibre Channel product (that is, multimode to multimode).

1. Connect the fiber-optic cable to an LC connector on the HBA.

Figure 5-1 Connecting the Optical Cable: Dual Port HBA in Oracle Exalytics In-Memory Machine X2-4

Figure 5-2 Connecting the Optical Cable: Dual Port HBA in Oracle Exalytics In-Memory Machine X3-4

2. Connect the other end of the cable to the FC device.

After the optical cable is connected to the HBA, you can power on the system.

5.2.4 Installing Diagnostic Support for the Red Hat/SUSE Linux OS (Optional)

For more information, see "Diagnostic Support for the Red Hat/SUSE OS" in the StorageTek 8 Gb FC PCI-Express HBA Installation Guide Installation Guide For HBA Models SG-XPCIE1FC-QF8-Z, SG-PCIE1FC-QF8-Z, SG-XPCIE1FC-QF8-N, SG-PCIE1FC-QF8-NandSG-XPCIE2-QF8-Z, SG-PCIE2FC-QF8-Z, SG-XPCIE2-QF8-N, SG-PCIE2FC-QF8-N.

5.2.5 Verifying the Connectivity

To verify the connectivity, complete the following steps:

1. Check the status of the multipath device as follows:

[root@node1 ~]# service multipathd status
multipathd (pid  xxxx) is running...

2. Run multipath -ll command to view the detailed information of the multipath devices as shown in the following example:

# multipath -ll
mpath2 (3600144f0d153d65600004f90462c0003) dm-3 SUN,Sun Storage 7410
[size=100G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 7:0:0:0 sdc 8:32  [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 8:0:0:0 sde 8:64  [active][ready]
mpath3 (3600144f0d153d65600004f90464f0004) dm-4 SUN,Sun Storage 7410
[size=200G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
\_ 7:0:0:2 sdd 8:48  [active][ready]
\_ round-robin 0 [prio=1][enabled]
\_ 8:0:0:2 sdf 8:80  [active][ready]

In the above example, mpath2 and mpath3 are the two LUNs configured in the storage device.

5.2.6 Completing the Setup

Complete the following steps:

1. Mount the LUNs to a mount point:

# mount /dev/mapper/mpathXX <mount_point>

Note:
The multipath LUNs are available as block devices at: /dev/mapper/mpathXX.

2. Add the mount point to /etc/fstab to auto mount during an operating system boot.

5.3 For Additional Help

For additional help or information regarding the storage configuration, contact the Oracle support personnel.

6 Using Oracle Integrated Lights Out Manager (ILOM)

This chapter describes how to set up and access Oracle Integrated Lights Out Manager (ILOM) for managing the Oracle Business Intelligence Machine.

It contains the following topics:

- Section 6.1, "ILOM Overview"

- Section 6.2, "Important Notes Before You Begin"

- Section 6.3, "Connecting to ILOM via the Network"

- Section 6.4, "Connecting to ILOM via a Serial Connection"

- Section 6.5, "Configuring Network Access"

6.1 ILOM Overview

Oracle Integrated Lights Out Manager (ILOM) provides advanced service processor (SP) hardware and software that you can use to manage and monitor your Oracle Oracle Exalytics In-Memory Machine.

ILOM enables you to actively manage and monitor your Oracle Oracle Exalytics In-Memory Machine independently of the operating system state, providing you with a reliable Lights Out Management (LOM) system.

With ILOM, you can proactively:

- Learn about hardware errors and faults as they occur

- Remotely control the power state of your compute node

- View the graphical and non-graphical consoles for the host

- View the current status of sensors and indicators on the system

- Determine the hardware configuration of your system

- Receive generated alerts about system events in advance via IPMI PETs, SNMP Traps, or E-mail Alerts.

The ILOM service processor (SP) runs its own embedded operating system and has a dedicated Ethernet port, which together provide out-of-band management capability. In addition, you can access ILOM from the operating system (Oracle Linux) installed on the Oracle Oracle Exalytics In-Memory Machine. Using ILOM, you can remotely manage your compute node as if you were using a locally attached keyboard, monitor, and mouse.

ILOM automatically initializes as soon as power is applied to your compute node. It provides a full-featured, browser-based web interface and has an equivalent command-line interface (CLI).

6.1.1 ILOM Interfaces

ILOM supports multiple interfaces for accessing its features and functions. You can choose to use a browser-based web interface or a command-line interface.

Web Interface

The web interface provides an easy-to-use browser interface that enables you to log in to the SP, then to perform system management and monitoring.

Command-Line Interface

The command-line interface enables you to operate ILOM using keyboard commands and adheres to industry-standard DMTF-style CLI and scripting protocols. ILOM supports SSH v2.0 and v3.0 for secure access to the CLI. Using the CLI, you can reuse existing scripts with Sun systems, and automate tasks using familiar interfaces.

6.2 Important Notes Before You Begin

You require at least one static IP address for Service Processor (ILOM access).

Note the following:

- As a general convention, root is used as the default user name.

- The default value of the Service Processor (SP ILOM) password is changeme.

6.3 Connecting to ILOM via the Network

Under normal circumstances, you can access ILOM via the network. You can use the Ethernet connection method to connect to the ILOM. You must know the ILOM IP address. The default ILOM IP address is 192.168.1.200 (netmask 255.255.255.0). You can use either the CLI or the web interface to access ILOM. Alternatively, you can launch a remote KVM session to access ILOM.

Note:
You can use this connection method only if ILOM IP addresses are configured and can be accessed over the network. The network must be connected to the NET0 port (with sideband) of the Oracle Oracle Exalytics In-Memory Machine.
Oracle recommends that you use this connection method to access ILOM for the Oracle Oracle Exalytics In-Memory Machine.

This section discusses the following topics:

- Connecting to the CLI

- Connecting to the Web GUI

- Launching a Remote KVM Session

6.3.1 Connecting to the CLI

1. Start your SSH client, such as PuTTY.

2. In the Host Name (or IP address) field, enter the ILOM IP address of your Oracle Oracle Exalytics In-Memory Machine. The default ILOM IP address is 192.168.1.200.

3. Ensure that SSH is chosen as the Connection Type.

4. Type the user name and the password, when prompted. The default user name is root, and the default password is changeme.
The CLI command prompt is displayed.

6.3.2 Connecting to the Web GUI

1. In the address bar of a web browser, type the ILOM IP address of an Oracle Oracle Exalytics In-Memory Machine. The default ILOM IP address is 192.168.1.200.
Press Enter.

The login screen is displayed.

2. Enter the user name and the password. The default user name is root, and the default password is changeme.

3. Click Log In.
The web GUI is displayed.

6.3.3 Launching a Remote KVM Session

To access the ILOM console for your Oracle Oracle Exalytics In-Memory Machine that is connected to ILOM, do the following:

1. Ensure that pop-up blockers are disabled in your browser before you launch the remote console.

2. In the address bar of a browser, type the ILOM IP address of your Oracle Oracle Exalytics In-Memory Machine.

3. Log in to ILOM using the default user name and password (root and changeme). After login, the ILOM home page is displayed.

4. Click the Remote Control tab, and click Launch Remote Console.
The remote console is displayed.

6.4 Connecting to ILOM via a Serial Connection

You can connect to ILOM via a serial connection if you are unable to access ILOM via the network due to any of the following problems:

- Misconfiguration of the network

- Misconfiguration of ILOM IP addresses

Note:
You can reconfigure network access after connecting to ILOM via a serial connection. For more information, see Configuring Network Access.

To connect to ILOM using a serial connection, complete the following steps:

1. Attach a serial cable from a terminal or a PC running terminal emulation software to the SER MGT port of your Oracle Oracle Exalytics In-Memory Machine. The cable should be of length 15 feet or less.

2. Verify that your terminal or laptop is operational.

3. Configure the terminal device or the terminal emulation software to use the following settings:

- 8N1: eight data bits, no parity, one stop bit

- 9600 baud (default, but can be set to any standard rate up to 115200)

- Disable software flow control (XON/XOFF)

- Disable hardware control

4. Verify that power is supplied to either PSU.
If there is power applied to either PSU, then ILOM will be functional regardless of the power state of compute nodes.

5. Press Enter on the terminal device. A connection between the terminal device and the ILOM is established.
The ILOM login prompt is displayed.

6. Log in to the CLI using the default user name and the password (root and changeme).
The ILOM default command prompt is displayed.

6.5 Configuring Network Access

You must configure network access after connecting to the ILOM using a serial connection or an Ethernet connection.

Note:
Oracle recommends that you use an Ethernet connection if reconfiguration of network access becomes necessary.

This section contains the following topics:

- Using a Serial Connection

- Using the Ethernet Connection

Using a Serial Connection

1. Connect to the ILOM using a serial connection, as described in Connecting to ILOM via a Serial Connection.

2. Run the following commands at the ILOM command prompt:
cd /SP/network

set pendingipdiscovery=static

set pendingipaddress=<ip_address>

set pendingipnetmask=<ip_netmask>

set pendingipgateway=<ip_gateway>

set commitpending=true

Using the Ethernet Connection

1. Connect to the ILOM using the Ethernet connection, as described in Connecting to ILOM via the Network.

2. Run the following commands at the ILOM command prompt:
cd /SP/network

set pendingipdiscovery=static

set pendingipaddress=<ip_address>

set pendingipnetmask=<ip_netmask>

set pendingipgateway=<ip_gateway>

set commitpending=true

Using ipmitool Commands When SP Network Information Is Lost

If the service processor (SP) network information gets lost, you can use an SSH shell to connect to the ETH0 interface of the operating system on the compute node and run appropriate ipmitool commands to reconfigure network settings, as in the following example:

/opt/ipmitool/bin/ipmitool sunoem cli 'set /SP/users/rootpassword=somepasswd' somepasswd

This example resets the ILOM root password.

Note:
For more information about ipmitool commands and options, see the ipmitool man page at the following URL:
http://sourceforge.net/p/ipmitool/wiki/coding_standards/

6.5.1 Configuring ILOM IP Addresses Manually

If ILOM IP addresses get misconfigured, you can configure ILOM IP addresses manually. Under normal circumstances, you do not need to configure ILOM IP addresses manually.

To configure IP addresses manually, complete the following steps:

1. Obtain a free static IP address from the network to which the Exalytics machine is connected.

2. Configure the IP address and sideband management by using a serial port connection as follows:

1. Establish a serial connection to the SP of a compute node by inserting a serial cable into the SER MGT port on the rear of the compute node.

2. In the terminal window, configure the following settings:
8N1: Eight data bit, no parity, one stop bit

9600 Baud

Disable hardware control

Disable software control

3. At the login prompt, enter root as the user name, and changeme as the password.

4. From your present working directory, use the cd command to move to the /SP/network directory, as shown in Figure 6-1.

Figure 6-1 Configuring ILOM IP Using Serial Port Connection

5. Run the following commands at the command prompt:
cd /SP/network

set pendingipdiscovery=static

set pendingipaddress=<ip_address>

set pendingipnetmask=<ip_netmask>

set pendingipgateway=<ip_gateway>

set commitpending=true

7 Connecting Oracle Exalytics In-Memory Machine to Oracle Exadata Database Machine

This chapter describes how to connect an Oracle Exalytics In-Memory Machine and an Oracle Exadata Database Machine together on the same InfiniBand fabric.

It contains the following topics:

- Section 7.1, "Understanding InfiniBand Card, Cable, and Switch Firmware Requirements"

- Section 7.2, "Physically Connecting the Network"

- Section 7.3, "Choosing a Subnet Manager"

- Section 7.4, "Configuring the InfiniBand Connection"

- Section 7.5, "Monitoring the InfiniBand Fabric"

- Section 7.6, "Checking Network Health"

- Section 7.7, "Testing the Extended Network Connectivity"

7.1 Understanding InfiniBand Card, Cable, and Switch Firmware Requirements

This section contains the following topics:

- InfiniBand Card

- InfiniBand Cables

- InfiniBand Switches in the Oracle Exadata Database Machine

7.1.1 InfiniBand Card

Each Oracle Oracle Exalytics In-Memory Machine includes an InfiniBand card, such as the 2-port QDR InfiniBand Host Channel Adapter (HCA).

This InfiniBand card is installed in the Oracle Oracle Exalytics In-Memory Machine in its PCIe Slot 6.

7.1.2 InfiniBand Cables

You may use any spare 5-meter InfiniBand cables included in your Oracle Exadata Database Machine. If you need assistance, contact Oracle support personnel. You need these cables for connecting your Oracle Oracle Exalytics In-Memory Machine to an Oracle Exadata Database Machine on the same InfiniBand fabric.

7.1.3 InfiniBand Switches in the Oracle Exadata Database Machine

Oracle Exadata Database Machine includes two 36-port InfiniBand leaf switches (Sun Datacenter QDR InfiniBand Switch 36). These two switches located in the middle of the rack are referred to as leaf switches. A spine switch is included in unit 1 of Oracle Exadata Database Machine Full Rack and Half Rack configurations.

Note:
In this document, the leaf switch located in the lower unit of the Oracle Exadata Database Machine rack is referred to as leaf 1. The leaf switch located above leaf 1 in the Oracle Exadata Database Machine rack is referred to as leaf 2.
For more information about Oracle Exadata Database Machine's rack layout and hardware components, see the Oracle Exadata Database Machine Owner's Guide.

7.2 Physically Connecting the Network

This section describes the following topics:

- Physical Layout Considerations

- Scenario 1: Single Exalytics In-Memory Machine with Oracle Exadata Database Machine

- Scenario 2: Clustered Exalytics In-Memory Machine with Oracle Exadata Database Machine

7.2.1 Physical Layout Considerations

It is assumed that the rack containing the Oracle Oracle Exalytics In-Memory Machine and the Oracle Exadata Database Machine rack are placed side by side in the data center. Consider the length of the connecting InfiniBand cables when placing the racks in your data center.

You can place up to two Oracle Exalytics Machines in an Oracle Exadata Database Machine rack.

7.2.2 Scenario 1: Single Exalytics In-Memory Machine with Oracle Exadata Database Machine

Figure 7-1 shows how a single (non-clustered) Oracle Oracle Exalytics In-Memory Machine is connected to an Oracle Exadata Database Machine rack.

Figure 7-1 Non-Clustered Oracle Exalytics In-Memory Machine Connected to Oracle Exadata Database Machine

For high availability purposes, connect the two ports of the InfiniBand card, which is installed in the Oracle Oracle Exalytics In-Memory Machine, to separate leaf switches of the Oracle Exadata Database Machine. For example, as shown in Figure 7-1, you can connect port 1 of the Oracle Oracle Exalytics In-Memory Machine's InfiniBand card to the leaf 1 switch (Sun Datacenter InfiniBand Switch 36 installed in the lower unit of the Oracle Exadata Database Machine). In addition, you should connect port 2 of the Oracle Oracle Exalytics In-Memory Machine's InfiniBand card to the leaf 2 switch (Sun Datacenter InfiniBand Switch 36 installed in the upper unit of the Oracle Exadata Database Machine). Port 1 can function as the active port, and port 2 can function as the passive port.

Note:
On the leaft switches of the Oracle Exadata Database Machine, the following free ports can be used:
Ports 5B, 6A, 6B, 7A, 7B, and 12A

7.2.3 Scenario 2: Clustered Exalytics In-Memory Machine with Oracle Exadata Database Machine

Figure 7-2 shows how a cluster of Oracle Oracle Exalytics In-Memory Machines is connected to an Oracle Exadata Database Machine rack.

Figure 7-2 Clustered Oracle Exalytics In-Memory Machine Connected to Oracle Exadata Database Machine

For high availability purposes, connect the active port of the InfiniBand card included in each of the Oracle Oracle Exalytics In-Memory Machines to the leaf 1 switch (Sun Datacenter InfiniBand Switch 36 installed in the lower unit of the Oracle Exadata Database Machine). In addition, you should connect the passive port of the InfiniBand card included in each of the Oracle Oracle Exalytics In-Memory Machines to the leaf 2 switch (Sun Datacenter InfiniBand Switch 36 installed in the upper unit of the Oracle Exadata Database Machine). Port 1 can function as the active port, and port 2 can function as the passive port.

Note:
On the leaft switches of the Oracle Exadata Database Machine, the following free ports can be used:
Ports 5B, 6A,6B,7A,7B, and 12A

7.3 Choosing a Subnet Manager

Each InfiniBand network requires a Subnet Manager (SM). You can use the SM running on one of the Oracle Exadata Database Machine's leaf switches in your fabric.

For more information about using the InfiniBand switch, see the following documents:

- Sun Datacenter InfiniBand Switch 36 User's Guide

- Sun Datacenter InfiniBand Switch 36 Command Reference

7.4 Configuring the InfiniBand Connection

To configure the InfiniBand connection between Exalytics and Exadata, do the following:

1. Enable automatic RDS kernel module loading on Exalytics.

1. Edit the /etc/ofed/openib.conf file and set SDP_LOAD=yes and RDS_LOAD=yes.

2. Reboot the nodes.

2. Create InfiniBand Listener on Exadata compute nodes.

1. Edit the /etc/ofed/openib.conf file and set SDP_LOAD=yes and RDS_LOAD=yes.

2. Edit the /etc/ofed/libsdp.conf file and find the following lines:

use both server * :
use both client * :

Change these lines to read as follows:

use tcp server * *:*
use tcp client * *:*

use tcp indicates that TCP is the default protocol and SDP will be used from specific applications using their local configuration.

3. Edit the /etc/modprobe.conf file and add this line at the end:

options ib_sdp sdp_zcopy_thresh=0 recv_poll=0

4. Reboot each compute node.

5. Edit the /etc/hosts file on each Exalytics machine and Exadata Database Machine compute node to add the virtual IP addresses for use on the InfiniBand network, for example, Exadata Database Machine quarter rack (2 compute nodes). Ensure that these IP addresses are not in use on the private network.

192.168.10.21 dm01db01-ibvip.mycompany.com dm01db01-ibvip
192.168.10.22 dm01db02-ibvip.mycompany.com dm01db02-ibvip

6. On one of the Exadata Database Machine compute nodes, as the root user execute the following commands to create a network resource for the InfiniBand network, and add the starting virtual IP addresses for each compute node on it. Make appropriate substitutions for the network and subnet mask in use on your private network (it may not be 192.168.10.0/255.255.255.0). Ask your admin or use ipcalc, based on the network mask to calculate the network.

srvctl add network -k 2 -S 192.168.10.0/255.255.255.0/bondib0
srvctl add vip -n dm01db01 -A dm01db01-ibvip/255.255.255.0/bondib0 -k 2
srvctl add vip -n dm01db02 -A dm01db02-ibvip/255.255.255.0/bondib0 -k 2
srvctl start vip -i dbm01db01-ibvip
srvctl start vip -i dbm01db02-ibvip

7. On one of the Exadata Database Machine compute nodes, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME) execute the following command:

srvctl add listener -l LISTENER_IB -k 2 -p TCP:1522/SDP:1522

8. On each Exadata Database Machine compute node, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME), create or append to tnsnames.ora in GRID_HOME/network/admin with the following new entries (note that the *IBREMOTE and *IBLOCAL entries will be slightly different on each node to reference the proper remote and local VIP names—you cannot copy the entries from one node to another without some modifications):

DBM =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01-scan)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dbm)
))
DBM_IB =
(DESCRIPTION =
(LOAD_BALANCE=on)
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01db01-ibvip)(PORT = 1522))
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01db02-ibvip)(PORT = 1522))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dbm)
))
LISTENER_IBREMOTE =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01db02-ibvip.mycompany.com)(PORT = 1522))
))
LISTENER_IBLOCAL =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01db01-ibvip.mycompany.com)(PORT = 1522))
(ADDRESS = (PROTOCOL = SDP)(HOST = dm01db01-ibvip.mycompany.com)(PORT = 1522))
))
LISTENER_IPLOCAL =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = dm0101-vip.mycompany.com)(PORT = 1521))
))
LISTENER_IPREMOTE =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = dm01-scan.mycompany.com)(PORT = 1521))
))

9. On one Exadata Database Machine compute node, connect to the database instance as sysdba:

SQLPLUS> alter system set listener_networks='((NAME=network2)
(LOCAL_LISTENER=LISTENER_IBLOCAL)(REMOTE_LISTENER=LISTENER_IBREMOTE))',
'((NAME=network1)(LOCAL_LISTENER=LISTENER_IPLOCAL)
(REMOTE_LISTENER=LISTENER_IPREMOTE))' scope=both;

10. On one Exadata Database Machine compute node, as oracle user (or owner of the Grid Infrastructure ORACLE_HOME) restart the listener named LISTENER_IB by running the following commands:

srvctl stop listener -l LISTENER_IB
srvctl start listener -l LISTENER_IB

3. On the Exalytics machine, edit sys.odbc.ini file to make an entry that points to the Exadata DB host:

OracleNetServiceName=DWH.ESS.LOCAL_IB

7.5 Monitoring the InfiniBand Fabric

You can monitor many aspects of the InfiniBand fabric by running InfiniBand commands on the Sun Datacenter InfiniBand Switch 36, which is included in the Oracle Exadata Database Machine.

For more information about these commands, see the Sun Datacenter InfiniBand Switch 36 Command Reference.

7.6 Checking Network Health

After establishing InfiniBand network connectivity, you can verify the status of the basic InfiniBand network connectivity with rds-ping, rds-info -l by using the ibstat command on the Exalytics machine.

The ibstat command displays the basic status, as in the following example:

# ibstat

CA 'mlx4_0'
CA type: MT26428
Number of ports: 2
Firmware version: 2.7.8130
Hardware version: b0
Node GUID: 0x0021280001cf577e
System image GUID: 0x0021280001cf5781
Port 1:
State: Active
Physical state: LinkUp
Rate: 40
Base lid: 106
LMC: 0
SM lid: 54
Capability mask: 0x02510868
Port GUID: 0x0021280001cf577f
Link layer: IB
Port 2:
State: Active
Physical state: LinkUp
Rate: 40
Base lid: 95
LMC: 0
SM lid: 54
Capability mask: 0x02510868
Port GUID: 0x0021280001cf5780
Link layer: IB

The physical state of connected ports is shown as LinkUp.

7.7 Testing the Extended Network Connectivity

After verifying the basic InfiniBand network connectivity, you can run the ibhosts, ibswitches, and ibnetdiscover commands on the switch CLI.

The ibhosts command displays host nodes in the InfiniBand fabric, the ibswitches command displays the InfiniBand switch nodes in the fabric, and the ibnetdiscover command discovers the InfiniBand topology.

For more information such monitoring and diagnostic commands, see the Sun Datacenter InfiniBand Switch 36 Command Reference.

8 Connecting Oracle Exalytics In-Memory Machine to Oracle Big Data Appliance

This chapter describes how to connect Oracle Exalytics In-Memory Machine and an Oracle Big Data Appliance together on the same InfiniBand fabric.

It contains the following sections:

- Section 8.1, "Hardware Requirements"

- Section 8.2, "Software Requirements"

- Section 8.3, "Physical Connectivity"

- Section 8.4, "InfiniBand Connection Validation (Layer-2)"

- Section 8.5, "IPoIB Configuration (Layer-3)"

- Section 8.6, "IPoIB Validation"

8.1 Hardware Requirements

Oracle Big Data Appliance has three in-built InfiniBand switches. Use the two NM2-GW leaf switches to connect the Oracle Exalytics machine. The Oracle Exalytics machine has one InfiniBand HCA card, but it does not include any InfiniBand switch or any instance of InfiniBand Subnet Manager.

To connect Oracle Exalytics to Oracle Big Data Appliance, you need a pair of InfiniBand cables depending on the length requirements at the datacenter. If the total distance between Exalytics and Oracle Big Data Appliance leaf switch ports is less than 10 meters, you can use copper cables. If the length is more than 10 meters, you must use fiber optic cable for InfiniBand with appropriate transceivers.

8.2 Software Requirements

The base images of both Oracle Big Data Appliance and Oracle Exalytics include software support for InfiniBand connectivity.

8.2.1 Exalytics Version Details

To view software details of the Oracle Exalytics machine, run the following commands:

Note:
From the following commands, use imageinfo command for 1.0.0.2 systems and earlier. Use exalytics_imageinfo command for 1.0.0.3 systems and later.

[root@exalytics root]# exalytics_imageinfo
Exalogic 1.0.0.5 (build:r)

Image version : 1.0.0.5
Image build version :
Creation timestamp : Wed 19 Jun 2013 12:26:49 PM BST
Kernel version : 2.6.32-100.23.1.el5
Image activated : 2013-09-03 20:49:13 -0400
Image status : SUCCESS

[root@exalytics root]# ofed_info | head -1
BXOFED-1.5.2-1.3.8000:

[root@exalytics root]# lsmod | grep ipoib
ib_ipoib 67547 1
ib_cm 32459 2 rdma_cm,ib_ipoib
ib_sa 36903 5 rdma_ucm,rdma_cm,mlx4_vnic,ib_ipoib,ib_cm
ib_core 61642 14 rdma_ucm,rds_rdma,ib_sdp,rdma_cm,iw_cm,ib_uverbs,ib_umad,mlx4_vnic,mlx4_ib,ib_mthca,ib_ipoib,ib_cm,ib_sa,ib_mad
ipv6 286932 222 ib_sdp,ib_addr,ib_ipoib,bonding

[root@exalytics root]# lsmod | grep rds
rds_tcp 9460 0
rds_rdma 72457 1
rds 91561 2 rds_tcp,rds_rdma
rdma_cm 29058 3 rdma_ucm,rds_rdma,ib_sdp
ib_core 61642 14 rdma_ucm,rds_rdma,ib_sdp,rdma_cm,iw_cm,ib_uverbs,ib_umad,mlx4_vnic,mlx4_ib,ib_mthca,ib_ipoib,ib_cm,ib_sa,ib_mad
[root@exalytics root#

8.2.2 BDA Version Details

To view software support of the Oracle Big Data Appliance, run the following commands:

[root@bda10 ~]# imageinfo
Big Data Appliance Image Info

IMAGE_CREATION_DATE : Tue Aug 20 23:36:32 PDT 2013
IMAGE_LABEL : BDA_2.2.1_LINUX.X64_130820
IMAGE_VERSION : 2.2.1
KERNEL_VERSION : 2.6.32-200.21.1.el5uek
BDA_RPM_VERSION : bda-2.2.1-1
OFA_RPM_VERSION : ofa-2.6.32-200.21.1.el5uek-1.5.5-4.0.55.4
JDK_VERSION : jdk-1.6.0_51-fcs
HADOOP_VERSION : 2.0.0-cdh4.3.0

[root@bda10 ~]# ofed_info | head -1
OFED-IOV-1.5.5-1.0.0120:

[root@bda10 ~]# lsmod | grep ipoib
ib_ipoib 67931 1
ib_cm 35714 2 rdma_cm,ib_ipoib
ipv6 290974 5 bonding,cnic,ib_sdp,ib_addr,ib_ipoib
ib_sa 39126 6 rdma_ucm,rdma_cm,ib_ipoib,ib_cm,mlx4_vnic,mlx4_ib
ib_core 66876 12 rdma_ucm,ib_sdp,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_uverbs,ib_umad,mlx4_vnic,mlx4_ib,ib_sa,ib_mad

[root@bda10 ~]# lsmod | grep rds
rds 94503 0
[root@bda10 ~]#

8.3 Physical Connectivity

On both GW switches of Oracle Big Data Appliance, four ports are available for external connections-5A, 5B, 6A, 6B. Oracle recommends that you use ports 5A on both leaf switches. If these ports are not available, use any other available port on the leaf switches.

Note:
Never connect a host (HCA) system to the spine switch inside a rack.

If the Oracle Big Data Appliance rack is connected to other Oracle Engineered Systems, port 5-6[A|B] may not be available. In such a case, it is recommended to use any other available port on the InfiniBand leaf switch of the Oracle Big Data Appliance rack. If no free ports are available, contact Oracle Support.

Figure 8-1 shows how to connect an Exalytics to a standalone BDA rack.

Figure 8-1 Physical Connection

8.4 InfiniBand Connection Validation (Layer-2)

Once the cables are connected and you can see green LEDs on the ports, you must validate the InfiniBand connection.

1. Log in to the Oracle Big Data Appliance compute node or Exalytics machine.

2. From the Linux root shell, run the ibhosts command to verify if the newly connected (Exalytics/Oracle Big Data Appliance) machine is visible.

3. On the Exalytics machine, run the ibstat command to see the local ports' status. If an active Subnet Manager is present, then the ports should be Active.

[root@exalytics ~]# ibstat

CA 'mlx4_0'
CA type: MT26428
Number of ports: 2
Firmware version: 2.7.8130
Hardware version: b0
Node GUID: 0x0021280001efe984
System image GUID: 0x0021280001efe987
Port 1:
State: Active
Physical state: LinkUp
Rate: 40
Base lid: 209
LMC: 0
SM lid: 187
Capability mask: 0x02510868
Port GUID: 0x0021280001efe985
Link layer: IB
Port 2:
State: Active
Physical state: LinkUp
Rate: 40
Base lid: 73
LMC: 0
SM lid: 187
Capability mask: 0x02510868
Port GUID: 0x0021280001efe986
Link layer: IB

[root@exalytics ~]# sminfo

sminfo: sm lid 187 sm guid 0x10e00c16c0c0a0, activity count 124743184 priority 14 state 3 SMINFO_MASTER

4. Run the following command to make sure all the links are at 4X QDR:

[root@exalytics ~]# ibdiagnet -ls 10 -lw 4x

8.5 IPoIB Configuration (Layer-3)

Make sure that a unique IP address is available in the IPoIB subnet used in the Oracle Big Data Appliance rack. Never reuse an IP address that is already assigned, or allocated for Big Data Appliance or other purposes in this subnet. To identify a unique and available IP address with associated subnet mask, refer to the Oracle Big Data Appliance configuration..

To identify IPoIB network information from an existing Oracle Big Data Appliance compute node, run the following command:

[root@bda10 ~]# egrep '(IPADDR|NETMASK|NETWORK|BROADCAST)' /etc/sysconfig/network-scripts/ifcfg-bondib0

IPADDR=192.168.41.10
NETMASK=255.255.248.0
NETWORK=192.168.40.0
BROADCAST=192.168.47.255

Using the information obtained, you can request for a free and unique IP address belonging to Oracle Big Data Appliance's IPoIB subnet.

To find out the CIDR prefix, run the ipcalc command as follows:

[root@bda10 ~]# ipcalc -p 192.168.41.10 255.255.248.0
PREFIX=21

Note that in the example, the possible IP address host range is from 192.168.40.1 to 192.168.47.254. In the following step, IP 192.168.44.100/21 will be used as an IP address for Exalytics' IPoIB bond0 interface.

Run the exalytics_configure_network.sh script on Exalytics as root. When prompted, configure the IPoIB (InfiniBand) network interface. It is a bonded interface named bond0 using ib0 and ib1.

[root@exalytics root]# /opt/exalytics/bin/exalytics_configure_network.sh
Network Configuration Script Version exalytics-scripts-1.0.0.5ps3-52
Do you want to configure InfiniBand for Exadata connection? (y/n) [n]: y
Enter bond0 IP address:192.168.44.100
Use 192.168.44.162 for bond0 ip address (y/n) [y]:
Enter bond0 netmask address:255.255.248.0
Use 255.255.248.0 for bond0 netmask (y/n) [y]:

8.6 IPoIB Validation

To validate, ping an Oracle Big Data Appliance node's IPoIB address from Exalytics. If the ping is successful, you can use the connection for applications. You can test the application usage between the Exalytics and BDA. It may be necessary to install appropriate plugins and/or clients for Hadoop on Exalytics Machine. For further information, refer to product owner's guide.

9 Installing Auto Service Request (ASR) Software

This chapter describes how to install Auto Service Request (ASR) for Exalytics machines.

Note:
Starting with the Oracle Exalytics Release 1 Patchset 1 (1.0.0.1), the Auto Service Request (ASR) feature is supported. For more information, see the chapter "New Features for Exalytics Machine" in the Oracle Exalytics In-Memory Machine Installation and Administration Guide for Linux.

It contains the following topics:

- Section 9.1, "About Oracle Auto Service Request (ASR)"

- Section 9.2, "Recommended Configuration"

- Section 9.3, "Before You Begin"

- Section 9.4, "Installing ASR Manager on a Standalone System"

- Section 9.5, "Activating Exalytics Assets Manually"

- Section 9.6, "Viewing and Verifying ASR Assets ASR Manager"

- Section 9.7, "Configuring Telemetry Sources"

- Section 9.8, "Enabling Exalytics Machine Assets in My Oracle Support"

- Section 9.9, "Sending Test Alerts"

- Section 9.10, "Fault Coverage"

9.1 About Oracle Auto Service Request (ASR)

Auto Service Request (ASR) is a secure, scalable, customer-installable software feature of warranty and Oracle Support Services that provides auto-case generation when common hardware component faults occur. ASR is designed to enable faster problem resolution by eliminating the need to initiate contact with Oracle Support Services for common hardware component failures, reducing both the number of phone calls needed and overall phone time required. ASR also simplifies support operations by using electronic diagnostic data. Easily installed and deployed, ASR is completely controlled by you, the customer, to ensure security. ASR is applicable only for component faults. Not all component failures are covered, though the most common components (such as disk, fan, and power supplies) are covered.

Note:
ASR is not a replacement for other monitoring mechanisms, such as SMTP and SNMP alerts, within your data center. It is a complementary mechanism that expedites and simplifies the delivery of replacement hardware. ASR should not be used for downtime events in high-priority systems. For high-priority events, contact Oracle Support Services directly.

9.2 Recommended Configuration

The recommended configuration is to install the ASR Manager on a machine outside of Exalytics, on a Management only network away from the Data network. The supported operating systems are Oracle Linux 5.3 and above, and Solaris 10 Update 5 and above.

Note:
ASR Manager should not to be on the same subnet as the Bonded Data network interfaces.
ETH0 must be on a separate subnet that does not overlap the Bonded Data network.

9.3 Before You Begin

Before installing ASR, ensure that the following conditions are met:

- Ensure that you have access to My Oracle Support (http://support.oracle.com) and confirm your contact information is correct and current.

- Make sure that all of your Exalytics machine assets have a contact assigned and that the contact is correct and current.

- Identify and designate a system, outside of Exalytics, to serve as ASR Manager.

- Retrieve the IP address of your ASR Manager.

- Identify and verify ASR assets.

- Ensure connectivity to the Internet using HTTPS.

- Download and install the Oracle Exalytics Configuration Utility patch and ASR utility, on the Exalytics host machine.

Note:
Starting with Oracle Exalytics Release 1 Patchset 3 (1.0.0.3), the Oracle Exalytics Configuration Utility patch and the ASR utility are included in the base image.
Steps 1 through 4 are not applicable for users using Oracle Exalytics Release 1 Patchset 3 (1.0.0.3) or higher version.

If you are using Oracle Exalytics Release 1 Patchset 3 (1.0.0.3) or higher version, you must start at step 5.

To download and install the Oracle Exalytics Configuration utility patch and ASR utility, perform the following actions:

1. Download and install the Oracle Exalytics Configuration Utility patch (1.0.0.4) and ASR utility for Oracle Exalytics Release 1, Patchset 1 (ex_x2-4_config_utils_linux_x86_64.zip file) from the Oracle Software Delivery Cloud Web site at the following link:
http://edelivery.oracle.com

2. Unzip the file into a temporary directory.
The zip file contains the files that are listed in Table 9-1.

Table 9-1 Files in the Oracle Exalytics Configuration Utility Zip File

File Name

Notes

exalytics_configure_network.sh

Oracle Exalytics network configuration script

SUN-HW-TRAP-MIB.mib

ASR (Auto Service Request) on Oracle Exalytics

bda_mon_hw_asr.pl

ASR mail script on Oracle Exalytics

common_subs.pm

ASR on Oracle Exalytics

setup_asr_ilom.pl

ASR on Oracle Exalytics

3. Create the following directories:

- mkdir -p /opt/exalytics

- mkdir -p /opt/exalytics/asr

4. Copy the file exalytics_configure_network.sh to /opt/exalytics/bin. Copy the following files to the /opt/exalytics/asr directory:

- bda_mon_hw_asr.pl

- setup_asr_ilom.pl

- common_subs.pm

- SUN-HW-TRAP-MIB.mib

Note:
Before copying the file, ensure that you back up the existing file as exalytics_configure_network.sh_V1 first.

5. Execute the Oracle Exalytics network configuration script to configure the network interfaces.

#cd /opt/exalytics

#./exalytics_configure_network.sh

- Complete the following steps to configure ASR:

1. Go to the /opt/exalytics/asr directory, and run the following commands to add executive permissions on the two Perl files:
chmod +x bda_mon_hw_asr.pl

chmod +x setup_asr_ilom.pl

2. Add the following line to the /etc/inittab file:
emh1:345:respawn:/opt/exalytics/asr/bda_mon_hw_asr.pl -server

3. At the command line, execute the init q command to initialize and start the bda_mon server.

9.4 Installing ASR Manager on a Standalone System

You must install ASR Manager on a standalone system (a machine outside of Exalytics running Oracle Linux, as described in the section "Install Software" in the Oracle Auto Service Request Installation and Operations Guide.

9.5 Activating Exalytics Assets Manually

If you need to activate HOST and ILOM for Exalytics manually, complete the following steps in the ASR Manager:

1. Login as the root user in your ASR Manager. Start the ASR menu by running the following command:

#cd /opt/SUNWswasr/bin
#./asr

2. Activate ASR on Exalytics host machine by running the following command:
asr> activate_asset -i IP_address_of_exalytics_host_machine

or

asr> activate_asset -h host_name_exalytics_machine

3. Activate ILOM for Exalytics individually by running the following command:
asr> activate_asset -i ILOM_IP_address_of_compute_node

or

asr> activate_asset -h ILOM_host_name_of_exalytics_machine

9.6 Viewing and Verifying ASR Assets ASR Manager

View and verify ASR Asset (Exalytics machine components whose ASR is active) by running the following command from the ASR Manager system:

asr> list_asset - lists all assets associated with this ASR Manager

For example:

xx.xxx.xxx.xx exalytics 1234567890 Enabled SNMP FMA SUN FIRE X4470 M2 SERVER x86/x64 System

xx.xxx.xxx.xxx exalytics-sp 1234567890 Enabled SNMP ILOM SUN FIRE X4470 M2 SERVER

9.7 Configuring Telemetry Sources

You must configure telemetry sources for ILOM and Exalytics host machine to forward their telemetry to the ASR Manager. Scripts are provided to configure both ILOM and the Host. The scripts are found in /opt/exalytics/asr. These scripts should be executed on the Exalytics host machine.

Complete the following steps:

1. Login to the Exalytics Host as the root user.

2. Configure ILOM telemetry by running the following command:

#cd /opt/exalytics/asr
#./setup_asr_ilom.pl -i <IP Address Of ASR Manager>

3. Configure Exalytics host machine telemetry by running the following command:

#cd /opt/exalytics/asr
#./bda_mon_hw_asr.pl -set_snmp_subscribers type=asr,host=<IP Address Of ASR Manager>,fromip=<IP Address of Exalytics Host>

9.8 Enabling Exalytics Machine Assets in My Oracle Support

Complete the following steps to enable Exalytics machine assets:

1. Log in to My Oracle Support (https://support.oracle.com).

2. In the My Oracle Support Dashboard, click the More... tab. Then select Settings from the menu.

3. In the Settings pane on the left of the window, select Assets (located under the Administrative submenu). For your Support Identifier, select ASR Status from the sort filter, then select Pending from the All Statuses drop-down menu.

4. You can approve a Pending asset in two ways:

- Activate via Asset Toolbar
Click the asset's serial number to display a toolbar with following options:

- Assign Contact - select this option to assign a contact to the asset. Only users associated with the support identifier can be a contact.

- Change Address - select this option to update the asset's physical location.

- Approve - for assets that are Pending, click Approve to enable ASR for the asset.

- Other Actions - you can also perform other actions such as Activate (for an inactive asset), Deactivate (for an active asset), or Deny (for a pending request).

- Activate via Asset Details
At the bottom of the Asset pane, there is a "Show Asset Details" option. Click an asset's serial number and then the Show Asset Details to update information about the asset. Required fields for ASR asset activation are indicated by an asterisk (*).

For more information, see the Knowledge Management article ID 1329200.1 on My Oracle Support. To obtain this article on My Oracle Support, complete the following steps:

1. Log in to My Oracle Support (https://support.oracle.com).

2. Click the Knowledge tab.

3. In the Knowledge Base section, click the Search & Browse tab.

4. In the Enter search terms option, enter 1329200.1 and click Search. The article "How to activate ASR within My Oracle Support" is displayed.

9.9 Sending Test Alerts

To send test alerts, complete the following:

1. Login to the Exalytics Host as the root user.

2. Send Test Alerts for ILOM telemetry by running the following command:
#ipmitool sunoem cli "set /SP/alertmgmt/rules/1 testrule=true"

3. Send Test Alerts for Exalytics host machine by running the following command:

#./opt/exalytics/asr/bda_mon_hw_asr.pl -validate_snmp_subscriber -type asr

You will receive an e-mail notification for the test alerts.

Note:
Oracle highly recommends to send test alerts from the ILOM, and the Exalytics host machine.

9.10 Fault Coverage

For more information, see Oracle ASR ILOM Fault Coverage.

A Troubleshooting

This appendix provides basic information about troubleshooting your Oracle Oracle Exalytics In-Memory Machine.

It contains the following topics:

- Section A.1, "Troubleshooting the Setup"

- Section A.2, "Additional Troubleshooting Information"

- Section A.3, "Locating the Machine Serial Number"

A.1 Troubleshooting the Setup

This section contains information to help you troubleshoot minor machine problems.

If you experience problems while setting up your Oracle Oracle Exalytics In-Memory Machine, refer to the troubleshooting information in Table A-1.

Table A-1 Troubleshooting Procedures

Problem

Possible Solution

Oracle Exalytics In-Memory Machine does not power on when the front panel Power button is pressed.

Keep notes on the following situations in case you need to call service:

- Is the Power/OK LED flashing or constantly illuminated on the front of the system? (Ensure that the power cord is connected to the system and to a grounded power receptacle.)

- Does the wall outlet have power? Test by plugging in another device.

- Does the monitor sync within five minutes after power on? (The green LED on the monitor stops flashing and remains illuminated.)

- Has the system fully booted the ILOM SP? (Note that the system will block Power button presses until the ILOM SP is fully booted.)

Oracle Exalytics In-Memory Machine appears to be in standby power mode, but the Power/OK LED does not blink.

The Power/OK LED only blinks when all Oracle Exalytics In-Memory Machine components are in standby power mode. A tape drive might be connected to your server. Because tape drives do not enter standby power mode, the Power/OK LED does not blink.

Hung or frozen server: No response from mouse or keyboard or any application.

Try to access your system from a different machine on the network:

1. On another system, type: ping IP_address_of_server.

2. If a response is returned, then try logging in to the Oracle Exalytics In-Memory Machine using either telnet, ssh, or rlogin.

3. If you successfully log in, list the running processes using the ps command.

4. Kill any processes that appear unresponsive or should not be running, by using the kill process_ID command.

5. Check the responsiveness of the Oracle Exalytics In-Memory Machine after each process is killed.

If the this procedure does not work, power cycle the Oracle Exalytics In-Memory Machine:

1. Press the Power/OK button to power off the Oracle Exalytics In-Memory Machine and wait 20 to 30 seconds.

2. Press the Power/OK button again to power on the system.

A.2 Additional Troubleshooting Information

For additional troubleshooting information, see the Sun Fire X4470 M2 Server Service Manual (821-0703) and the Oracle x86 Servers Diagnostics Guide (820-6750).

A.3 Locating the Machine Serial Number

You might need to have the serial number of your Oracle Oracle Exalytics In-Memory Machine when you ask for service on your system. Record this number for future use. Use one of the following methods to locate your machine's serial number:

- On the front panel of the Oracle Oracle Exalytics In-Memory Machine, look at the bottom ledge (near the center) to locate the machine's serial number. For an illustration of the Oracle Oracle Exalytics In-Memory Machine front panel, see Section 1.5.1, "Front Panel Features".

- Locate the yellow Customer Information Sheet (CIS) attached to your Oracle Oracle Exalytics In-Memory Machine packaging. This sheet includes the serial number.

- From ILOM, enter the show/SYS command, or go to the System Information tab in the ILOM browser interface.

Use Table A-2 to collect information that you might need to communicate with Oracle support personnel.

Table A-2 System Information Required for Support

System Configuration Information Required

Your Information

Service contract number

System model

Operating environment

System serial number

Peripherals attached to the system

Email address and phone number for you and a secondary contact

Street address where the system is located

Administrator password

Summary of the problem and the work being done when the problem occurred

Other useful information

IP address

Oracle Exalytics In-Memory Machine name (system host name)

Network or internet domain name

Proxy server configuration

Oracle Legal Notices

Copyright Notice

Copyright © 1994-2015, Oracle and/or its affiliates. All rights reserved.

Trademark Notice

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

License Restrictions Warranty/Consequential Damages Disclaimer

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

Warranty Disclaimer

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

Restricted Rights Notice

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

Hazardous Applications Notice

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Third-Party Content, Products, and Services Disclaimer

This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

Alpha and Beta Draft Documentation Notice

If this document is in preproduction status:

This documentation is in preproduction status and is intended for demonstration and preliminary use only. It may not be specific to the hardware on which you are using the software. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to this documentation and will not be responsible for any loss, costs, or damages incurred due to the use of this documentation.