test

Sun MPI 4.0 User's Guide: With CRE

Expressing More Complex Resource Requirements

Use the -R option to express complex node requirements that are not accessible via the basic options discussed above.

The -R option takes a resource requirement specifier (RRS) as an argument. The RRS is enclosed in quotation marks and provides the settings for any number of attributes that you want to use to control the selection of nodes. You combine multiple attribute settings using the logical & (AND) and | (OR) operators.

The CRE parses the attribute settings in the order in which they are listed in the RRS, along with other options you specify. The CRE merges these results with the results of an internally specified RRS that controls load balancing.

Note -

One option is an exception to this merging behavior, -j. This exception is discussed later.

The result is an ordered list of CPUs that meet the specified criteria. If you are starting a single process, the CRE starts the process on the CPU that's first in the list. If you are starting n processes, the CRE starts them on the first n CPUs, wrapping if necessary.

Note -

Unless -Ns is specified, the RRS specifies node resources but generates a list of CPUs. If -Ns is specified, the list refers only to nodes.

Specifying Resource Attributes

Table 3-3 lists predefined attributes you can include in an RRS. Your system administrator may also have defined attributes specific to your Sun HPC cluster. You can see what settings these administrator-defined attributes have with the mpinfo command.

Table 3-3 Standard RRS Attributes

Attribute 

Meaning 

cpu_idle

Percent of time that the CPU is idle. 

cpu_iowait 

Percent of time that the CPU spends waiting for I/O. 

cpu_kernel 

Percent of time that the CPU spends in the kernel. 

cpu_scale 

Performance rating of the CPU. 

cpu_swap 

Percent of time that the CPU spends waiting for swap. 

cpu_type 

CPU architecture. 

cpu_user 

Percent of time that the CPU spends running user's program. 

load1 

Node's load average for the past minute. 

load5 

Node's load average for the past 5 minutes. 

load15 

Node's load average for the past 15 minutes. 

manufacturer 

Hardware manufacturer. 

mem_free 

Nodes's available memory, in Mbytes. 

mem_total 

Node's total physical memory, in Mbytes. 

name 

Node's hostname. 

os_max_proc 

Maximum number of processes allowed on the node, including cluster daemons. 

os_arch_kernel 

Node's kernel architecture. 

os_name 

Operating system's name. 

os_release 

Operating system's release number. 

os_release_maj 

The major number of the operating system's release number. 

os_release_min 

The minor number of the operating system's release number. 

os_version 

Operating system's version. 

serial_number 

Node's serial number. 

swap_free 

Node's available swap space, in Mbytes. 

swap_total 

Node's total swap space, in Mbytes. 

The CRE recognizes two types of attributes, value and boolean.

Value-Based Attributes

Value attributes can take a literal value or a numeric value. Or, depending on the operator used, they may take no value.

% mprun -R "name = hpc-demo" a.out

% mprun -R "load5 < 4" a.out

specifies that you only want nodes whose individual load averages over the previous 5 minutes were less than 4.

% mprun -R "mem_total>>" a.out

specifies that you prefer nodes with the largest physical memory available.

Table 3-4 identifies the operators that can be used in RRS expressions.

Table 3-4 Operators Valid for Use in RRS

Operator 

Meaning 

< 

Select all nodes where the value of the specified attribute is less than the specified value. 

<= 

Select all nodes where the value of the specified attribute is less than or equal to the specified value. 

= 

Select all nodes where the value of the specified attribute is equal to the specified value. 

>= 

Select all nodes where the value of the specified attribute is greater than or equal to the specified value. 

> 

Select all nodes where the value of the specified attribute is greater than the specified value. 

!= 

Attribute must not be equal to the specified value. (Precede with a backslash in the C shell.) 

<< 

Select the node(s) that have the lowest value for this attribute. 

>> 

Select the node(s) that have the highest value for this attribute. 

The operators have the following precedence, from strongest to weakest:

unary -
*, /
+, binary -
=, !=, >=, <=, >, <, <<, >>
!
&, |
?

If you use the << or >> operator, the CRE does not provide load balancing. In the previous example, the CRE would choose the node with the most free swap space, regardless of its load. If you use << or >> more than once, only the last use has any effect--it overrides the previous uses. For example,

% mprun -R "mem_free>> swap_free>>" a.out

initially selects the nodes that have the most free memory, but then selects nodes that have the largest amount of available swap space. The second selection may yield a different set of nodes than were selected initially.

You can also use arithmetic expressions for numeric attributes anywhere. For example,

% mprun -R "load1 / load5 < 2" a.out

specifies that the ratio between the one-minute load average and the five-minute load average must be less than 2. In other words, the load average on the node must not be growing too fast.

You can use standard arithmetic operators as well as the C ?: conditional operator.

Note -

Because some shell programs interpret characters used in RRS arguments, you may need to protect your RRS entries from undesired interpretation by your shell program. For example, if you use csh, write "-R \!private" instead of "-R !private".

Boolean Attributes

Boolean attributes are either true or false. If you want the attribute to be true, simply list the attribute in the RRS. For example, if your system administrator has defined an attribute called ionode, you can request a node with that attribute: 

% mprun -R "ionode" a.out

If you want the attribute to be false (that is, you do not want a resource with that attribute), precede the attribute's name with !. (Precede this with a backslash in the C shell; the backslash is an escape character to prevent the shell from interpreting the exclamation point as a "history" escape.) For example,

% mprun -R "\!ionode" a.out

For example,

% mprun -R "mem_free > 256" a.out

specifies that the node must have over 256 megabytes of available RAM.

% mprun -R "swap_free >>" a.out

specifies that the node picked must have the highest available swap space.

Examples

Here are some examples of the -R option in use.

The following example specifies that the program must run on a node in the partition with 512 Mbytes of memory:

% mprun -p part2 -R "mem_total=512" a.out

The following example specifies that you want to run on any of the three nodes listed:

% mprun -R "name=node1 | name=node2 | name=node3" a.out

The following example chooses nodes with over 300 Mbytes of free swap space. Of these nodes, it then chooses the one with the most total physical memory:

% mprun -R "swap_free > 300 & mem_total>>" a.out

The following example assumes that your system administrator has defined an attribute called framebuffer, which is set (TRUE) on any node that has a frame buffer attached to it. You could then request such a node via the command

% mprun -R "framebuffer" a.out

The -R Option and MPRUN_FLAGS

With the exception of the -j option, specifying -R on the command line as well as in the MPRUN_FLAGS environment variable combines the two sets of values--that is, the command line does not override the environment variable settings. For example, if you have

% setenv MPRUN_FLAGS '-R "load1 < 1"'

and issue the command

% mprun -R "load5 < 1" -R "load15 < 1" a.out

this would be the same as issuing the command

% mprun -R "(load1<1) & (load5<1) & (load15<1)" a.out

This combining behavior does not happen with the -j option. When -j is specified by MPRUN_FLAGS as well as on the mprun command line, the command line use overrides the environment variable setting.