JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle SuperCluster M6-32 HTML Owner’s Guide
search filter icon
search icon

Document Information

Using This Documentation

Product Documentation Library

Feedback

Overview

Determining SuperCluster M6-32 Configurations

Configuration Flowchart

Determine the Number of Compute Servers

Determine the Number of DCUs in Each Compute Server

Determine the Number of CMUs in Each DCU

CMU Overview

Determine the Amount of Memory in Each DCU

Determine the PDomain Configuration on Each Compute Server

Determine the LDom Configuration for Each PDomain

Determining the Best Configuration for Your Situation

Understanding PDomain Configurations

Understanding High Redundancy

Allocating CPU Resources for LDoms

Allocating Memory Resources for LDoms

Understanding PCIe Cards and Slots for LDoms

Understanding Storage for LDoms

Understanding the Hardware

Understanding SuperCluster M6-32

Identifying SuperCluster M6-32 Components

Understanding DCUs

DCU Overview

Understanding DCU Configurations

Understanding Half-Populated DCU Root Complexes

Understanding Fully-Populated DCU Root Complexes

Understanding PDomains

PDomains Overview

PDomain Guidelines

Extended Configuration PDomain Overview

Understanding Extended Configuration PDomains

Base Configuration Overview

Understanding Base Configuration PDomains

Understanding LDoms

Understanding Compute Server Hardware and Networks

Understanding Domains

Understanding LDom Configurations for Extended Configuration PDomains

Understanding LDom Configurations for Base Configuration PDomains

Understanding Clustering Software

Cluster Software Overview

Cluster Software for the Database Domain

Cluster Software for the Oracle Solaris Application Domains

Understanding System Administration Resources

Oracle ILOM Overview

Understanding Platform-Specific Oracle ILOM Features

Oracle Solaris OS Overview

OpenBoot Overview

Oracle ILOM Remote Console Plus Overview

Oracle Hardware Management Pack Overview

Time Synchronization and NTP Service

SNMP Service

Multidomain Extensions to Oracle ILOM MIBs

LDAP/SSL

Active Directory

Hardware Installation

Installing SuperCluster M6-32

Hardware Installation Overview

Weight Specifications

Hardware Installation Task Overview

Hardware Installation Documents

Preparing the Site (Storage Rack and Expansion Racks)

Prepare the Site for the Racks

Storage Rack Components

Physical Specifications

Reviewing Power Requirements

Preparing for Cooling

Preparing the Network

Network Topology

Network Infrastructure Requirements

Compute Server Default Host Names and IP Addresses

Install Cable Drops

Prepare DNS

Cabling SuperCluster M6-32

Compute Server Network Components

Storage Rack Network Components

Cable the ZFS Storage Appliance

ZFS Appliance Power Cord Connection Reference

ZFS Storage Appliance Cabling Reference

Cable IB Switches

Leaf Switch 1 Cabling Reference

Leaf Switch 2 Cabling Reference

IB Switch-to-Switch Cabling Reference

Cable the Ethernet Management Switch

Ethernet Management Switch Cabling Reference

Connect SuperCluster M6-32 to the Facility Networks

Connecting Expansion Racks

Expansion Rack Overview

Expansion Rack Components

Expansion Rack Layouts

Installing Expansion Racks

One Expansion Rack Cabling

Two Expansion Racks Cabling

Three Expansion Racks Cabling

Four Expansion Racks Cabling

Five Expansion Racks Cabling

Six Expansion Racks Cabling

Seven Expansion Racks Cabling

Expansion Rack Default IP Addresses

Understanding Internal Cabling (Expansion Rack)

Administration

Understanding SuperCluster Software

Identify the Version of SuperCluster Software

SuperCluster Tools

Controlling SuperCluster M6-32

Cautions

Power On SuperCluster M6-32

Powering Off SuperCluster M6-32 Gracefully

Power Off SuperCluster M6-32 in an Emergency

Monitoring SuperCluster M6-32 (OCM)

OCM Overview

Access OCM Documentation

Monitoring the System With ASR

ASR Overview

ASR Resources

ASR Installation Overview

Configure ASR on the Compute Servers (Oracle ILOM)

Configure SNMP Trap Destinations for Storage Servers

Configure ASR on the ZFS Storage Appliance

Configuring ASR on the Compute Servers (Oracle Solaris 11)

Approve and Verify ASR Asset Activation

Tuning SuperCluster M6-32

ssctuner Overview

Monitor ssctuner Activity

View Log Files

Change ssctuner Properties and Disable Features

Install ssctuner

Enable ssctuner

Configuring CPU and Memory Resources (osc-setcoremem)

osc-setcoremem Overview

Minimum and Maximum Resources (Dedicated Domains)

Supported Domain Configurations

Plan CPU and Memory Allocations

Display the Current Domain Configuration (osc-setcoremem)

Display the Current Domain Configuration (ldm)

Change CPU/Memory Allocations (Socket Granularity)

Change CPU/Memory Allocations (Core Granularity)

Park Cores and Memory

Access osc-setcoremem Log Files

View the SP Configuration

Revert to a Previous CPU/Memory Configuration

Remove a CPU/Memory Configuration

Obtaining the EM Exadata Plug-in

Confirm System Requirements

Known Issues With the EM Exadata Plug-in

Configuring the Exalogic Software

Exalogic Software Overview

Prepare to Configure the Exalogic Software

Enable Domain-Level Enhancements

Enable Cluster-Level Session Replication Enhancements

Configuring Grid Link Data Source for Dept1_Cluster1

Configuring SDP-Enabled JDBC Drivers for Dept1_Cluster1

Create an SDP Listener on the IB Network

Administering Oracle Solaris 11 Boot Environments

Advantages to Maintaining Multiple Boot Environments

Create a Boot Environment

Mount to a Different Build Environment

Reboot to the Original Boot Environment

Create a Snapshot of a Boot Environment

Remove Unwanted Boot Environments

Administering DISM

DISM Restrictions

Disable DISM

Administering Storage Servers

Monitor Write-through Caching Mode

Shut Down or Reboot a Storage Server

Drop a Storage Server

Glossary

Index

Glossary

A

Application Domain

A domain that runs Oracle Solaris and client applications.

ASMM

Automatic shared memory management.

ASR

Auto Service Request. A feature of Oracle or Sun hardware that automatically opens service requests when specific hardware faults occur. ASR is integrated with MOS and requires a support agreement. See also MOS.

B

base configuration PDomain

A SuperCluster M6-32 configuration comprised of two or four PDomains, where one DCU is associated with each PDomain. A base configuration PDomain can reside on a single compute server or can be split across two compute servers. See also compute server, DCU, extended configuration PDomain, and PDomain.

C

CFM

Cubic feet per minute.

Cisco Catalyst Ethernet switch

Provides the SuperCluster M6-32 management network. Referred to in this documentation using the shortened name “Ethernet management switch.” See also Ethernet management switch.

CMP

Chip multiprocessing. Each CMU contains 2 CMP processors. The compute server can contain a maximum of 32 CMPs.

CMU

CPU memory unit. Each CMU in the compute server contains two CMPs and two sets of DIMM slots.

COD

Capacity on Demand.

compute server

Shortened name for the SPARC M6-32 server, a major component of SuperCluster M6-32. See also SPARC M6-32 server.

D

Database Domain

The domain that contains the SuperCluster M6-32 database.

DB

Oracle Database.

DCM

Domain configuration management. The reconfiguration of boards in PDomains for Enterprise-class systems. See also PDomain.

DCU

Domain configurable unit. The smallest building block for PDomains. Each DCU in the compute server contains two or four CMUs and one IOU. See also PDomain.

dedicated domain

A SuperCluster LDom category that includes the domains configured at installation time as either a Database Domain or an Application Domain (running the Oracle Solaris 10 or Oracle Solaris 11 OS). Dedicated domains have direct access to the 10GbE NICs and IB HCAs (and Fibre Channel cards, if present). See also Database Domain and Application Domain.

DHCP

Dynamic Host Configuration Protocol. Software that automatically assigns IP addresses to clients on a TCP/IP network. See also TCP.

DIMM

Dual in-line memory module.

DISM

Dynamic intimate shared memory.

E

EECS

Oracle Exalogic Elastic Cloud software.

EMS

Express module SAS. Each EMS contains two 10GBASE-T network connections and provides access to four hard drives on the compute server.

EPO switch

Emergency power-off switch.

ESD

Electrostatic discharge.

Ethernet management switch

Shortened name for the Cisco Catalyst Ethernet switch. See also Cisco Catalyst Ethernet switch.

expansion rack

Shortened name for optional Oracle Exadata Storage Expansion Racks (up to 17) that can be added to SuperCluster M6-32. See also Oracle Exadata Storage Expansion Rack.

extended configuration PDomain

A SuperCluster M6-32 configuration comprised of two PDomains, where two DCUs are associated with each PDomain. An extended configuration PDomain can reside on a single compute server or can be split across two compute servers. See also, base configuration PDomain, compute server, DCU, and PDomain.

F

FAN

Fast application notification event.

FCoE

Fibre Channel over Ethernet.

FM

Fan module.

FMA

Fault management architecture. A feature of Oracle Solaris servers that includes error handlers, structured error telemetry, automated diagnostic software, response agents, and messaging.

FRU

Field-replaceable unit.

fully-populated DCU configuration

A configuration where each DCU in the compute servers contains four CMUs. See also DCU and half-populated DCU configuration.

G

GB

Gigabyte. 1 gigabyte = 1024 megabytes.

GbE

Gigabit Ethernet.

GNS

Grid Naming Service.

H

half-populated DCU configuration

A configuration where each DCU in the compute servers contains two CMUs. See also DCU and fully-populated DCU configuration.

HCA

Host channel adapter.

HDD

Hard disk drive. In Oracle Solaris OS output, HDD can refer to hard disk drives or SSDs.

I

I/O Domain

If you have Root Domains, you create I/O Domains with your choice of resources at the time of your choosing. The I/O Domain Creation tool enables you to assign resources to I/O Domains from the CPU and memory repositories, and from virtual functions hosted by Root Domains. When you create an I/O Domain, you assign it as a Database Domain or Application Domain running the Oracle Solaris 11 OS. See also Root Domain.

IB

InfiniBand.

IB switch

Shortened name for the Sun Datacenter InfiniBand Switch 36. See also leaf switch, spine switch, and Sun Datacenter InfiniBand Switch 36.

ILOM

See Oracle ILOM.

IOU

I/O unit. The compute server contains up to 4 IOUs, one for each DCU. Each IOU supports up to 16 PCIe slots, 8 10GBASE-T ports on 4 EMS modules, and 8 drives.

IPMI

Intelligent Platform Management Interface.

IPMP

IP network multipathing.

iSCSI

Internet Small Computer System Interface.

K

KVMS

Keyboard video mouse storage.

L

LDom

Logical domain. A virtual machine comprising a discrete logical grouping of resources that has its own operating system and identity within a single computer system. LDoms are created using Oracle VM Server for SPARC software. See also Oracle VM Server for SPARC.

leaf switch

Two of the IB switches are configured as leaf switches, the third is configured as a spine switch. See also IB switch.

M

MIB

Management information base.

MOS

My Oracle Support.

N

NET MGT

The network management port on an SP. See also SP.

NIC

Network interface card.

NUMA

Nonuniform memory access.

O

OBP

OpenBoot PROM. Firmware on SPARC servers that enables the server to load platform-independent drivers directly from devices, and provides an interface through which you can boot the compute server and run low-level diagnostics.

OCM

Oracle Configuration Manager.

ONS

Oracle Notification Service.

Oracle ASM

Oracle Automatic Storage Management. A volume manager and a file system that supports Oracle databases.

Oracle Exadata Storage Expansion Rack

Optional expansion racks (in full, half, or quarter configurations) that can be added to SuperCluster M6-32 systems that require additional storage. Referred to in this documentation using the shortened name “expansion rack.” See also expansion rack.

Oracle ILOM

Oracle Integrated Lights Out Manager. Software on the SP that enables you to manage a server independently from the operating system. See also SP.

Oracle Solaris OS

Oracle Solaris operating system.

Oracle SuperCluster

Refers to all Oracle SuperCluster models.

Oracle SuperCluster M6-32

Full name of this SuperCluster model. Referred to in this documentation using the shortened name “SuperCluster M6-32.” See also SuperCluster M6-32.

Oracle SuperCluster M6-32 storage rack

Full name of first storage rack that contains the storage servers, ZFS storage appliance, IB switches, and Ethernet management switch. Referred to in this documentation using the shortened name “storage rack.” See also storage rack.

Oracle VM Server for SPARC

SPARC server virtualization and partitioning technology. See also LDom.

Oracle VTS

Oracle Validation Test Suite. An application, preinstalled with Oracle Solaris, that exercises the system, provides hardware validation, and identifies possible faulty components.

Oracle XA

Oracle's implementation of the X/Open distributed transaction processing XA interface that is included in Oracle DB software.

Oracle ZFS ZS3-ES storage appliance

Located in the storage rack, it provides SuperCluster M6-32 with shared storage capabilities. Referred to in this documentation using the shortened name “ZFS storage appliance.” See also ZFS storage appliance.

OS

Operating system.

P

parked resources

CPU and memory resources that are set aside in the CPU and memory repositories. You assign parked resources to I/O Domains with the I/O Domain Creation tool.

PCIe

Peripheral Component Interconnect Express.

PDomain

Physical domain. Each PDomain on the compute server is an independently configurable and bootable entity with full hardware domain isolation for fault isolation and security purposes. See also compute server, DCU, and SSB.

PDomain-SPP

The lead SPP of a PDomain. The PDomain-SPP on the compute server manages tasks and provides rKVMS service for that PDomain. See also PDomain.

PDU

Power distribution unit.

PF

Physical function. Functions provided by physical I/O devices, such as the IB HCAs, 10GbE NICs, and any Fibre Channel cards installed in the PCIe slots. Logical devices, or virtual functions (VFs), are created from PFs, with each PF hosting 32 VFs.

POST

Power-on self-test. A diagnostic that runs when the compute server is powered on.

PS

Power supply.

PSDB

Power system distribution board.

PSH

Predictive self healing. An Oracle Solaris OS technology that continuously monitors the health of the compute server and works with Oracle ILOM to take a faulty component offline if needed.

Q

QMU

Quarterly maintenance update.

QSFP

Quad small form-factor, pluggable. A transceiver specification for 10GbE technology.

R

RAC

Real Application Cluster.

RCLB

Runtime connection load balancing.

rKVMS

Remote keyboard video mouse and storage.

root complex

CMP circuitry that provides the base to a PCIe I/O fabric. Each PCIe I/O fabric consists of the PCIe switches, PCIe slots, and leaf devices associated with the root complex.

Root Domain

A logical domain that is configured at installation time. Root Domains are required if you plan to configure I/O Domains. Root Domains host PFs from which I/O Domains derive VFs. The majority of Root Domain CPU and memory resources are parked for later use by I/O Domains.

S

SAS

Serial attached SCSI.

SATA

Serial advance technology attachment.

scalability

The ability to increase (or scale up) processing power in a compute server by combining the server's physical configurable hardware (see also DCU) into one or more logical groups (see also PDomain).

SCAN

Single Client Access Name. A feature used in RAC environments that provides a single name for clients to access any Oracle Database running in a cluster. See also RAC.

SDP

Session Description Protocol.

SER MGT

The serial management port on an SP. See also SP.

SFP and SFP+

Small form-factor pluggable standard. SFP+ is a specification for a transceiver for 10GbE technology.

SGA

System global area.

SMF

Service Management Facility.

SNEEP

Serial number in EEPROM.

SNMP

Simple Management Network Protocol.

SP

Service processor. A processor, separate from the host, that monitors and manages the host no matter what state the host is in. The SP runs Oracle ILOM, which provides remote lights out management. In SuperCluster M6-32, SPs are located on the compute servers, storage servers, ZFS storage appliance controllers, and IB switches. See also Oracle ILOM.

SPARC M6-32 server

A major component of SuperCluster M6-32 that provides the main compute resources. Referred to in this documentation using the shortened name “compute server.” See also compute server.

spine switch

One of the SuperCluster M6-32 IB switches that is configured as a spine switch. See also IB switch and leaf switch.

SPP

Service processor proxy. One SPP in the compute server is assigned to manage each PDomain. SPPs monitor environmental sensors and manage the CMUs, memory controllers, and DIMMs within the DCU. See also PDomain-SPP.

SR-IOV Domain

Single-Root I/O Virtualization Domain -- A SuperCluster logical domain category that includes Root Domains and I/O Domains. This category of domains support single-root I/O virtualization. See also I/O Domain and Root Domain.

SSB

Scalability switch board in the compute server.

SSD

Solid state drive.

STB

Oracle Services Tool Bundle.

storage rack

Shortened name for the Oracle SuperCluster M6-32 storage rack that contains the storage servers. See also Oracle SuperCluster M6-32 storage rack.

storage server

Storage servers in SuperCluster M6-32.

Sun Datacenter InfiniBand Switch 36

Interconnects SuperCluster M6-32 components on a private network. Referred to in this documentation using the shortened name “IB switch.” See also IB switch, leaf switch, and spine switch.

SuperCluster M6-32

Shortened name for Oracle SuperCluster M6-32. See also Oracle SuperCluster M6-32.

T

TCP

Transmission Control Protocol.

TNS

Transparent Network Substrate.

TPM

Trusted platform module.

U

UPS

Uninterruptible power supply.

V

VAC

Voltage alternating current.

VF

Virtual function. Logical I/O devices that are created from PFs, with each PF hosting 32 VFs.

VIP

Virtual IP.

VLAN

Virtual local area network.

VNET

Virtual network.

W

WWN

World Wide Name.

X

XA

See Oracle XA.

Z

ZFS

A file system with added volume management capabilities. ZFS is the default file system in Oracle Solaris 11.

ZFS storage appliance

Shortened name for Oracle ZFS Storage ZS3-ES storage appliance. See also Oracle ZFS ZS3-ES storage appliance.

ZFS storage controller

Servers in the Oracle ZFS ZS3-ES storage appliance that manage the storage appliance. See also ZFS storage appliance.