JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle SuperCluster M6-32 HTML Owner’s Guide
search filter icon
search icon

Document Information

Using This Documentation

Product Documentation Library

Feedback

Overview

Determining SuperCluster M6-32 Configurations

Configuration Flowchart

Determine the Number of Compute Servers

Determine the Number of DCUs in Each Compute Server

Determine the Number of CMUs in Each DCU

CMU Overview

Determine the Amount of Memory in Each DCU

Determine the PDomain Configuration on Each Compute Server

Determine the LDom Configuration for Each PDomain

Determining the Best Configuration for Your Situation

Understanding PDomain Configurations

Understanding High Redundancy

Allocating CPU Resources for LDoms

Allocating Memory Resources for LDoms

Understanding PCIe Cards and Slots for LDoms

Understanding Storage for LDoms

Understanding the Hardware

Understanding SuperCluster M6-32

Identifying SuperCluster M6-32 Components

Understanding DCUs

DCU Overview

Understanding DCU Configurations

Understanding Half-Populated DCU Root Complexes

Understanding Fully-Populated DCU Root Complexes

Understanding PDomains

PDomains Overview

PDomain Guidelines

Extended Configuration PDomain Overview

Understanding Extended Configuration PDomains

Base Configuration Overview

Understanding Base Configuration PDomains

Understanding LDoms

Understanding Compute Server Hardware and Networks

Understanding Domains

Understanding LDom Configurations for Extended Configuration PDomains

Understanding LDom Configurations for Base Configuration PDomains

Understanding Clustering Software

Cluster Software Overview

Cluster Software for the Database Domain

Cluster Software for the Oracle Solaris Application Domains

Understanding System Administration Resources

Oracle ILOM Overview

Understanding Platform-Specific Oracle ILOM Features

Oracle Solaris OS Overview

OpenBoot Overview

Oracle ILOM Remote Console Plus Overview

Oracle Hardware Management Pack Overview

Time Synchronization and NTP Service

SNMP Service

Multidomain Extensions to Oracle ILOM MIBs

LDAP/SSL

Active Directory

Hardware Installation

Installing SuperCluster M6-32

Hardware Installation Overview

Weight Specifications

Hardware Installation Task Overview

Hardware Installation Documents

Preparing the Site (Storage Rack and Expansion Racks)

Prepare the Site for the Racks

Storage Rack Components

Physical Specifications

Reviewing Power Requirements

Preparing for Cooling

Preparing the Network

Network Topology

Network Infrastructure Requirements

Compute Server Default Host Names and IP Addresses

Install Cable Drops

Prepare DNS

Cabling SuperCluster M6-32

Compute Server Network Components

Storage Rack Network Components

Cable the ZFS Storage Appliance

ZFS Appliance Power Cord Connection Reference

ZFS Storage Appliance Cabling Reference

Cable IB Switches

Leaf Switch 1 Cabling Reference

Leaf Switch 2 Cabling Reference

IB Switch-to-Switch Cabling Reference

Cable the Ethernet Management Switch

Ethernet Management Switch Cabling Reference

Connect SuperCluster M6-32 to the Facility Networks

Connecting Expansion Racks

Expansion Rack Overview

Expansion Rack Components

Expansion Rack Layouts

Installing Expansion Racks

One Expansion Rack Cabling

Two Expansion Racks Cabling

Three Expansion Racks Cabling

Four Expansion Racks Cabling

Five Expansion Racks Cabling

Six Expansion Racks Cabling

Seven Expansion Racks Cabling

Expansion Rack Default IP Addresses

Understanding Internal Cabling (Expansion Rack)

Administration

Understanding SuperCluster Software

Identify the Version of SuperCluster Software

SuperCluster Tools

Controlling SuperCluster M6-32

Cautions

Power On SuperCluster M6-32

Powering Off SuperCluster M6-32 Gracefully

Power Off SuperCluster M6-32 in an Emergency

Monitoring SuperCluster M6-32 (OCM)

OCM Overview

Access OCM Documentation

Monitoring the System With ASR

ASR Overview

ASR Resources

ASR Installation Overview

Configure ASR on the Compute Servers (Oracle ILOM)

Configure SNMP Trap Destinations for Storage Servers

Configure ASR on the ZFS Storage Appliance

Configuring ASR on the Compute Servers (Oracle Solaris 11)

Approve and Verify ASR Asset Activation

Tuning SuperCluster M6-32

ssctuner Overview

Monitor ssctuner Activity

View Log Files

Change ssctuner Properties and Disable Features

Install ssctuner

Enable ssctuner

Configuring CPU and Memory Resources (osc-setcoremem)

osc-setcoremem Overview

Minimum and Maximum Resources (Dedicated Domains)

Supported Domain Configurations

Plan CPU and Memory Allocations

Display the Current Domain Configuration (osc-setcoremem)

Display the Current Domain Configuration (ldm)

Change CPU/Memory Allocations (Socket Granularity)

Change CPU/Memory Allocations (Core Granularity)

Park Cores and Memory

Access osc-setcoremem Log Files

View the SP Configuration

Revert to a Previous CPU/Memory Configuration

Remove a CPU/Memory Configuration

Obtaining the EM Exadata Plug-in

Confirm System Requirements

Known Issues With the EM Exadata Plug-in

Configuring the Exalogic Software

Exalogic Software Overview

Prepare to Configure the Exalogic Software

Enable Domain-Level Enhancements

Enable Cluster-Level Session Replication Enhancements

Configuring Grid Link Data Source for Dept1_Cluster1

Configuring SDP-Enabled JDBC Drivers for Dept1_Cluster1

Create an SDP Listener on the IB Network

Administering Oracle Solaris 11 Boot Environments

Advantages to Maintaining Multiple Boot Environments

Create a Boot Environment

Mount to a Different Build Environment

Reboot to the Original Boot Environment

Create a Snapshot of a Boot Environment

Remove Unwanted Boot Environments

Administering DISM

DISM Restrictions

Disable DISM

Administering Storage Servers

Monitor Write-through Caching Mode

Shut Down or Reboot a Storage Server

Drop a Storage Server

Glossary

Index

Change CPU/Memory Allocations (Socket Granularity)

Perform this procedure on each compute node to change its CPU and memory resource allocation at the socket granularity level.


Note - To find out if you can perform this procedure, see Supported Domain Configurations.


The tool makes these changes:

In this example, one socket and 1 TB memory are removed from the primary domain and allocated to ssccn3-dom1.

This table shows the allocation plan (see Plan CPU and Memory Allocations).

Domain
Domain Type
Sockets Before
Sockets

After

Memory Before (GB)
Memory After (GB)
primary
Dedicated
2
1
2048
1024
ssccn3-dom1
Dedicated
2
3
2048
3072
ssccn3-dom2
Root
n/a
n/a
n/a
n/a
ssccn3-dom3
Root
n/a
n/a
n/a
n/a
Unallocated resources
45
45
4048
4048
Total resources
49
39
4144
4144
  1. Log in as superuser on the compute node's control domain.
  2. Ensure that applications are shut down and that there is no production activity.
  3. Activate any inactive domains using the ldm bind command.

    The tool does not continue if any inactive domains are present.

  4. Run osc-setcoremem to reconfigure the resources.

    Respond when prompted. Press Enter to select the default value.

    # /opt/oracle.supercluster/bin/osc-setcoremem
     
                                  osc-setcoremem
                        v2.0  built on Aug 27 2015 23:09:35
     
     
     Current Configuration: SuperCluster Fully-Populated M6-32 Base
     
     +------------------------------+-------+--------+-----------+--- MINIMUM ----+
     | DOMAIN                           | CORES | MEM GB |   TYPE    | CORES | MEM GB |
     +------------------------------+-------+--------+-----------+-------+--------+
     | primary                          |    24 |   2048 | Dedicated |     2 |     32 |
     | ssccn3-dom1                      |    24 |   2048 | Dedicated |     2 |     32 |
     | ssccn3-dom2                      |     1 |     16 |   Root    |     1 |     16 |
     | ssccn3-dom3                      |     2 |     32 |   Root    |     2 |     32 |
     +------------------------------+-------+--------+-----------+-------+--------+
     | unallocated or parked            |    45 |   4048 |    --     |    -- |   --   |
     +------------------------------+-------+--------+-----------+-------+--------+
     
     [Note] Following domains will be skipped in this session.
     
     Root Domains
     ------------
     ssccn3-dom2
     ssccn3-dom3
     
     
     CPU allocation preference:
     
            1. Socket level
            2. Core level
     
     In case of Socket level granularity, proportional memory capacity is
      automatically selected for you.
     
     Choose Socket or Core level [S or C] s
     
     
     Step 1 of 1: Socket Count
     
     primary      : specify socket count [min: 1, max: 3. default: 1] : 1
                    you chose [1] socket for primary domain
     
     ssccn3-dom1  : specify socket count [min: 1, max: 3. default: 1] : 3
                    you chose [3] sockets for ssccn3-dom1 domain
     
     
     Configuration In Progress After Socket Count Selection:
     
     +----------------------------------+----------+----------+-----------+
     | DOMAIN                           |  SOCKETS |   MEM GB |   TYPE    |
     +----------------------------------+----------+----------+-----------+
     | primary                          |     1    |     1024 | Dedicated |
     | ssccn3-dom1                      |     3    |     3072 | Dedicated |
     | *ssccn3-dom2                     |   0.083  |       16 |   Root    |
     | *ssccn3-dom3                     |   0.167  |       32 |   Root    |
     +----------------------------------+----------+----------+-----------+
     | unallocated or parked            |   3.750  |     4048 |    --     |
     +----------------------------------+----------+----------+-----------+
     
     
     Following domains will be stopped and restarted:
     
            ssccn3-dom1
     
     This configuration requires rebooting the control domain.
     Do you want to proceed? Y/N : Y 
     
     +- IMPORTANT NOTE: -+
     |  After the reboot, osc-setcoremem attempts to complete CPU, memory re-configuration. |
     |  Please check syslog and the state of all domains before using the system.           |
     |  eg.,  dmesg | grep osc-setcoremem ; ldm list | grep -v active ; date |
     +- -+
     
     All activity is being recorded in log file:
            /opt/oracle.supercluster/osc-setcoremem/log/osc-setcoremem_activity_08-28-2015_15:31:27.log
     
     Please wait while osc-setcoremem is setting up the new CPU, memory configuration.
     It may take a while. Be patient and do not interrupt.
     
     0%    10    20    30    40    50    60    70    80    90   100%
     |-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|
     
     [Info] Domain ssccn3-dom1 is taking too long to stop. Waiting ..
     [Info] Domain ssccn3-dom1 is taking too long to stop. Still waiting for the domain to stop ..
     *=====*=====*=====*=====*=====*=====*=====*=====*=====*=====*
     
    Broadcast Message from root (pts/1) on etc5mdbadm0301 Fri Aug 28 15:36:45...
    THE SYSTEM etc5mdbadm0301 IS BEING SHUT DOWN NOW ! ! !
    Log off now or risk your files being damaged
     
                    Task complete with no errors.
     
    #
  5. Check the system log and the status of all logical domains to ensure that they are in active state before proceeding with the regular activity.

    Example:

    # dmesg | grep osc-setcoremem
    Aug 28 15:43:46 etc5mdbadm0301 root[2074]: [ID 702911 user.alert] osc-setcoremem: core, memory re-configuration complete. system can be used for regular work.
  6. Verify the new resource allocation.

    You can verify the resource allocation and check for possible osc-setcoremem errors in several ways:

  7. Repeat this procedure if you need to change resource allocations on another compute node.
Related Information