Partitions

Partitions are logical collections of nodes that work cooperatively to run programs on the Sun HPC cluster. An MPI job can run on a single partition or on the combination of a single partition and one or more nodes that are not members of any partition. MPI jobs cannot run in multiple partitions.

You must create a partition and enable it before you can run MPI programs on your Sun HPC cluster. Once a partition is created, you can configure it to meet the specific needs of your site and enable it for use.

Once a partition is created and enabled, you can run serial or parallel jobs on it. Serial programs run on a single node of a partition. Parallel programs run on any number of nodes of a partition in parallel.

The CRE performs load balancing on shared partitions. When you use mprun to execute a program on a shared partition, the CRE automatically runs it on the least-loaded nodes that satisfy any specified resource requirements.

Partitions are mutable. That is, after you create and configure a partition, you can change it if your site requirements change. You can add nodes to a partition or remove them. You can change a partition's attributes. Also, since you can enable and disable partitions, you can have many partitions defined and use only a few at a time according to current needs.

There are no restrictions on the number or size of partitions, so long as no node is a member of more than one enabled partition.

Partition Commands

Table 6-8 lists the mpadmin commands that can be used within the partition context.

Table 6-8 Partition-Level mpadmin Commands

Command	Synopsis
current partition	Set the context to the specified partition for future commands.
create partition	Create a new partition with the given name.
delete [partition]	Delete a partition.
list	List all the defined partitions.
show [partition]	Show a partition's attributes.
dump [partition]	Show the attributes of a partition.
set attribute[=value]	Set the current partition's attribute.
unset attribute	Delete the current partition's attribute.
up	Move up one level in the context hierarchy.
top	Move to the top level in the context hierarchy.
echo ...	Print the rest of the line on the standard output.
help [command]	Show information about the command command.
? [command]	Show information about the command command.

Creating Partitions

Prerequisites

• Nodes have to exist in the CRE database before you can add them to partitions.

• A node must be enabled for it to be an active member of a partition. If a node is configured as a partition member, but is not enabled, it will not participate in jobs that run on that partition.

Viewing Existing Partitions

Before creating a new partition, you might want to list the partitions that have already been created. To do this, use the list command from within the Partition context.

[node0]
Partition:: list        part0
        part1
[node0] Partition:: 

Creating a Partition

To create a partition, use the create command, followed by the name of the new partition. "Naming Partitions and Custom Attributes" discusses the rules for naming partitions.

For example:

[node0]
Partition:: create part0[node0] P(part0):: 

The create command automatically changes the context to that of the new partition.

At this point, your partition exists by name but contains no nodes. You must assign nodes to the partition before using it. You can do this by setting the partition's nodes attribute. See "Configuring Partitions" for details.

Configuring Partitions

You can configure partitions by setting and deleting their attributes using the set and unset commands. Table 6-9 shows commonly used partition attributes.

Table 6-9 Common Partitions and Their Attributes

Partition Type	Relevant Attributes	Recommended Value
Login	no_logins	not set
Login	max_total_procs	not set or set greater than
Dedicated	max_total_procs no_logins	=1 set
Serial	no_mp_tasks	set
Parallel	no_mp_task	not set

You can combine the attributes listed in Table 6-10 in any way that makes sense for your site. See "Configuring Partitions" for suggestions about how to configure your partitions.

Partitions, once created, can be enabled and disabled. This lets you define many partitions but use just a few at a time. For instance, you might want to define a number of shared partitions for development use and dedicated partitions for executing production jobs, but have only a subset available for use at a given time.

Partition Attributes

Table 6-10 lists the predefined partition attributes. To see their current values, use the mpadmin show command.

Table 6-10 Predefined Partition Attributes

Attribute	Kind	Description
enabled	Boolean	Set if the partition is enabled, that is, if it is ready to accept logins or jobs.
max_locked_mem	Value	Maximum amount of shared memory allowed to be locked down by MPI processes (in Kbytes).
max_total_procs	Value	Maximum number of simultaneously running processes allowed on each node in the partition.
min_unlocked_mem	Value	Minimum amount of shared memory that may not be locked down by MPI processes (in Kbytes).
name	Value	Name of the partition.
no_logins	Boolean	Disallow logins.
no_mp_tasks	Boolean	Disallow multiprocess parallel jobs.
nodes	Value	List of nodes in the partition.
shmem_minfree	Value	Fraction of swap space kept free for non-MPI use

enabled

Set the enabled attribute to make a partition available for use.

By default, the enabled attribute is not set when a partition is created.

max_locked_mem and min_unlocked_mem

---

Note -

You should not change these partition attributes. See "max_locked_mem and min_unlocked_mem" for a description of their effects.

---

max_total_procs

To limit the number of simultaneously running mprun processes allowed on all nodes in a partition or in all partitions, set the max_total_procs attribute in a specific node context or in the general Partition context.

[node0]
P(part0):: set max_total_procs=10[node0] P(part0):: 

You can set max_total_procs if you want to limit the load on a partition. By default, max_total_procs is unset.

The CRE does not impose any limit on the number of processes allowed on a node.

name

The name attribute is set when a partition is created. To change the name of a partition, set its name attribute to a new name.

[node0]
P(part0):: set name=part1[node0] P(part1):: 

See "Naming Partitions and Custom Attributes" for partition naming rules.

no_logins

To prohibit users from logging in to a partition, set the no_logins attribute.

[node0]
P(part1):: set no_logins[node0] P(part1):: 

no_mp_tasks

To prohibit multiprocess parallel jobs from running on a partition--that is, to make a serial partition, set the no_mp_tasks attribute.

[node0]
P(part1):: set no_mp_tasks[node0] P(part1):: 

nodes

To specify the nodes that are members of a partition, set the partition's nodes attribute.

[node0]
P(part1):: set nodes=node1[node0] P(part1):: show        set nodes = node1
        set enabled
[node0] P(part1):: 

The value you give the nodes attribute defines the entire list of nodes in the partition. To add a node to an already existing node list without retyping the names of nodes that are already present, use the + (plus) character.

[node0]
P(part1):: set nodes=+node2 node3[node0] P(part1):: show        set nodes = node0 node1 node2 node3
        set enabled
[node0] P(part1):: 

Similarly, you can use the - (minus) character to remove a node from a partition.

To assign a range of nodes to the nodes attribute, use the : (colon) syntax. This example assigns to part0 all nodes whose names are alphabetically greater than or equal to node0 and less than or equal to node3:

[node0]
P(part1):: set nodes = node0:node3[node0] P(part1):: 

Setting the nodes attribute of an enabled partition has the side effect of setting the partition attribute of the corresponding nodes. Continuing the example, setting the nodes attribute of part1 affects the partition attribute of node2:

[node0]
P(part1):: node node2[node0] N(node2):: show        set partition = part1
[node0] N(node2):: 

A node cannot be a member of more than one enabled partition. If you try to add a node that is already in an enabled partition, mpadmin returns an error message.

[node0]
P(part1):: show        set nodes = node0 node1 node2 node3
        set enabled
[node0] P(part1):: current part0[node0] P(part0):: set enabled[node0] P(part0):: set nodes=node1mpadmin: node1 must be removed from part1 before it
can be added to part0

Unsetting the nodes attribute of an enabled partition has the side effect of unsetting the partition attribute of the corresponding node.

Unsetting the nodes attribute of a disabled partition removes the nodes from the partition but does not change their partition attributes.

shmem_minfree

Use the shmem_minfree attribute to reserve some portion of the /tmp file system for non-MPI use.

For example, if /tmp is 1 Gbyte and shmem_minfree is set to 0.2, any time free space on /tmp drops below 200 Mbytes (1 Gbyte * 0.2), programs using the MPI shared memory protocol will not be allowed to run.

[node0]
P(part1):: set shmem_minfree=0.2

shmem_minfree must be set to a value between 0.0 and 1.0. When shmem_minfree is unset, it defaults to 0.1.

This attribute can be set on both nodes and partitions. If both are set, the node's shmem_minfree attribute overrides the partition's shmem_minfree attribute.

Enabling Partitions

A partition must be enabled before users can run programs on it.

Prerequisite

Before enabling a partition, you must disable any partitions that share nodes with the partition that you are about to enable.

Setting enabled

To enable a partition, set its enabled attribute.

[node0]
P(part0):: set enabled[node0] P(part0):: 

Now the partition is ready for use.

Enabling a partition has the side effect of setting the partition attribute of every node in that partition.

If you try to enable a partition that shares a node with another enabled partition, mpadmin prints an error message.

[node0]
P(part1):: show        set nodes = node1 node2 node3
        set enabled
[node0] P(part1):: current part2[node0] P(part2):: show        set nodes = node1
[node0] P(part2):: set enabledmpadmin: part1/node1: partition resource conflict

Disabling Partitions

To disable a partition, unset its enabled attribute.

[node0]
P(part0):: unset enabled[node0] P(part0):: 

Now the partition can no longer be used.

Any jobs are running on a partition when it is disabled will continue to run. After disabling a partition, you should either wait for ay running jobs to terminate or stop them using the mpkill command. This is described in the Sun HPC Cluster Runtime Environment 1.0 User's Guide.

Deleting Partitions

Delete a partition when you don't plan to use it anymore.

---

Note -

Although it is possible to delete a partition without first disabling it, you should disable the partition by unsetting its enabled attribute before deleting it.

---

To delete a partition, use the delete command in the context of the partition you want to delete.

[node0]
P(part0):: delete[node0] Partition::