Park Cores and Memory

Perform this procedure on each compute node to move CPU and memory resources from dedicated domains into logical CPU and memory repositories, making the resources available for I/O Domains.

If you are parking cores and memory, plan carefully. Once you park resources and create I/O Domains you cannot move resources back to dedicated domains.

In this example, 12 cores and 1 TB memory are parked from the primary domain, and 18 cores and 1536 GB memory are parked from the ssccn3-dom1 domain.

This table shows the allocation plan (see Plan CPU and Memory Allocations).

1. Log in as superuser on the compute node's control domain.
2. Ensure that all applications are shut down and that there is no production activity running.
3. Activate any inactive domains using the ldm bind command.

   The tool does not continue if any inactive domains are present.

4. Run osc-setcoremem to change resource allocations.

   In this example, some resources are left unallocated which parks them.

   Respond when prompted. Press Enter to select the default value.

   # /opt/oracle.supercluster/bin/osc-setcoremem
    
                                 osc-setcoremem
                       v2.0  built on Aug 27 2015 23:09:35
    
    
    Current Configuration: SuperCluster Fully-Populated M6-32 Base
    
    +--------------------------------+-------+--------+-----------+--- MINIMUM ----+
    | DOMAIN                           | CORES | MEM GB |   TYPE    | CORES | MEM GB |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | primary                          |    18 |   1536 | Dedicated |     2 |     32 |
    | ssccn3-dom1                      |    30 |   2560 | Dedicated |     2 |     32 |
    | ssccn3-dom2                      |     1 |     16 |   Root    |     1 |     16 |
    | ssccn3-dom3                      |     2 |     32 |   Root    |     2 |     32 |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | unallocated or parked            |    45 |   4048 |    --     |    -- |   --   |
    +---------------------------------+-------+--------+-----------+-------+--------+
    
    [Note] Following domains will be skipped in this session.
    
    Root Domains
    ------------
    ssccn3-dom2
    ssccn3-dom3
    
    
    CPU allocation preference:
    
           1. Socket level
           2. Core level
    
    In case of Socket level granularity, proportional memory capacity is
     automatically selected for you.
    
    Choose Socket or Core level [S or C] c
    
    
    Step 1 of 2: Core Count
    
    primary      : specify number of cores [min: 2, max: 46. default: 18] : 6
                   you chose [6] cores for primary domain
    
    ssccn3-dom1  : specify number of cores [min: 2, max: 42. default: 30] : 12
                   you chose [12] cores for ssccn3-dom1 domain
    
    
    Configuration In Progress After Core Count Selection:
    
    +--------------------------------+-------+--------+-----------+--- MINIMUM ----+
    | DOMAIN                           | CORES | MEM GB |   TYPE    | CORES | MEM GB |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | primary                          |     6 |   1536 | Dedicated |     2 |     32 |
    | ssccn3-dom1                      |    12 |   2560 | Dedicated |     2 |     64 |
    | *ssccn3-dom2                     |     1 |     16 |   Root    |     1 |     16 |
    | *ssccn3-dom3                     |     2 |     32 |   Root    |     2 |     32 |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | unallocated or parked            |    75 |   4048 |    --     |    -- |   --   |
    +---------------------------------+-------+--------+-----------+-------+--------+
    
    
    Step 2 of 2: Memory Capacity
           (must be 16 GB aligned)
    
    primary: specify memory capacity in GB [min: 32, max: 2048. default: 2048] : 512
                   you chose [512 GB] memory for primary domain
    
    ssccn3-dom1:specify memory capacity in GB [min: 64, max: 2048. default: 2048] : 1024
                   you chose [1024 GB] memory for ssccn3-dom1 domain
    
    
    Configuration In progress After Memory Capacity Selection:
    
    +--------------------------------+-------+--------+-----------+--- MINIMUM ----+
    | DOMAIN                           | CORES | MEM GB |   TYPE    | CORES | MEM GB |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | primary                          |     6 |    512 | Dedicated |     2 |     32 |
    | ssccn3-dom1                      |    12 |   1024 | Dedicated |     2 |     64 |
    | *ssccn3-dom2                     |     1 |     16 |   Root    |     1 |     16 |
    | *ssccn3-dom3                     |     2 |     32 |   Root    |     2 |     32 |
    +---------------------------------+-------+--------+-----------+-------+--------+
    | unallocated or parked            |    75 |   6608 |    --     |    -- |   --   |
    +--------------------------------+-------+--------+-----------+-------+--------+
    
    
    Following domains will be stopped and restarted:
    
           ssccn3-dom1
    
    This configuration requires rebooting the control domain.
    Do you want to proceed? Y/N : y
    
    IMPORTANT NOTE:
    +-                                                                                    -+
    |  After the reboot, osc-setcoremem attempts to complete CPU, memory re-configuration. |
    |  Please check syslog and the state of all domains before using the system.           |
    |  eg.,  dmesg | grep osc-setcoremem ; ldm list | grep -v active ; date                |
    +-                                                                                    -+
    
    All activity is being recorded in log file:
           /opt/oracle.supercluster/osc-setcoremem/log/osc-setcoremem_activity_08-28-2015_16:18:57.log 
   Please wait while osc-setcoremem is setting up the new CPU, memory configuration.
    It may take a while. Be patient and do not interrupt.
    
    0%    10    20    30    40    50    60    70    80    90   100%
    |-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|
    *=====*=====*=====*=====*=====*=====*=====*=====*=====*=====*
    
   Broadcast Message from root (pts/1) on etc5mdbadm0301 Fri Aug 28 16:22:07...
   THE SYSTEM etc5mdbadm0301 IS BEING SHUT DOWN NOW ! ! !
   Log off now or risk your files being damaged
    
                   Task complete with no errors.
    
   #

5. If the tool indicated that a reboot was needed, after the system reboots, log in as root on the compute node's control domain.
6. Verify the new resource allocation.

   You can verify the resource allocation and check for possible osc-setcoremem errors in several ways:

   • Display the Current Domain Configuration (osc-setcoremem)

   • Display the Current Domain Configuration (osc-setcoremem)

   • Access osc-setcoremem Log Files

7. Check the log file to ensure that all reconfiguration steps were successful.

   # cd /opt/oracle.supercluster/osc-setcoremem/log
   # ls (identify the name of the log file)
   # tail -17 osc-setcoremem_activity_08-28-2015_16\:18\:57.log
    
    ::Post-reboot activity::
    
    Please wait while osc-setcoremem is setting up the new CPU, memory configuration.
    It may take a while. Be patient and do not interrupt.
    
    
    Executing ldm commands ..
    
    0%    10    20    30    40    50    60    70    80    90   100%
    |-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|
    *=====*=====*=====*=====*=====*=====*=====*=====*=====*=====*
    
                   Task complete with no errors.
                   This concludes socket/core, memory reconfiguration.
                   You can continue using the system.

8. Verify the new resource allocation.

   You can verify the resource allocation and check for possible osc-setcoremem errors in several ways:

   • Display the Current Domain Configuration (osc-setcoremem)

   • Display the Current Domain Configuration (osc-setcoremem)

   • Access osc-setcoremem Log Files

   Example:

   # dmesg | grep osc-setcoremem
   Aug 28 16:27:50 etc5mdbadm0301 root[1926]: [ID 702911 user.alert] osc-setcoremem: core, memory re-configuration complete. system can be used for regular work.
    
   # ldm list
   NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
   primary          active     -n-cv-  UART    48    523008M  0.4%  0.4%  6m
   ssccn3-dom1      active     -n----  5001    96    1T       0.2%  0.2%  3m
   ssccn3-dom2      active     -n----  5002    8     16G      0.1%  0.1%  3d 36m
   ssccn3-dom3      active     -n--v-  5003    16    32G      0.1%  0.1%  3d 36m

9. Verify the parked cores.

   See Display the Current Domain Configuration (ldm):

   # ldm list-devices -p core | grep cid | wc -l
         75

10. Verify the parked memory.

    See Display the Current Domain Configuration (ldm):

    # ldm list-devices memory
    MEMORY
        PA                   SIZE
        0x3c00000000         768G
        0x84000000000        768G
        0x100000000000       1008G
        0x180000000000       1T
        0x208000000000       512G
        0x288000000000       512G
        0x300000000000       1008G
        0x380000000000       1008G

11. Repeat this procedure if you need to change resource allocations on the other compute node.

Related Information

• Supported Domain Configurations

• Plan CPU and Memory Allocations

• Display the Current Domain Configuration (osc-setcoremem)

• Display the Current Domain Configuration (ldm)

• Access osc-setcoremem Log Files