Migrating Third-Party Job Launchers

C H A P T E R 3

Sun HPC ClusterTools 6 and previous versions provided support for use with third-party job launchers, including Sun Grid Engine. The current version of SGE is now known as Sun Grid Engine. Sun HPC ClusterTools 7.1 supports Sun Grid Engine, in conjunction with ORTE (Open Run-Time Environment).

ORTE is the runtime support system for Open MPI that allows users to execute their applications in a distributed clustering environment.

Note - Sun HPC ClusterTools 6 supported the LSF job-launching software. LSF is not supported in Sun HPC ClusterTools 7.1.

How Sun Grid Engine Works With Open MPI

HPC sites use batch systems to share resources fairly and accountably, and also to guarantee that a job can obtain the resources it needs to run at maximum efficiency. To properly monitor a job’s resource consumption, the batch system must be the agent that launches the job.

Sun Grid Engine, like many other batch systems, cannot launch multiple process jobs (such as MPI applications) on its own. In Sun HPC ClusterTools 7.1, ORTE launches the multiple process jobs and sets up the environment required by Open MPI.

When Sun Grid Engine launches a parallel job in cooperation with OpenRTE, Sun Grid Engine “owns” the resulting launched processes. Sun Grid Engine monitors the resources for these processes, thereby creating a tightly integrated environment for resource accounting. OpenRTE allows users to execute their parallel applications.

Note - There is also an open source version of Grid Engine (GE) hosted on http://www.sunsource.net. Although the Sun HPC ClusterTools 7.1/Open MPI integration is developed with Sun Grid Engine, this integration should work for the open source Grid Engine as well.

Using Sun Grid Engine With ORTE

Open MPI requires that its executables are in your PATH on every node that will run them.

Note - mpirun was called mprun in previous versions of Sun HPC ClusterTools.

For example, if Open MPI was installed into /opt/SUNWhpc/HPC7.1/bin, you would add this location to your PATH as shown in the following example:

% setenv PATH /opt/SUNWhpc/HPC7.1/bin:${PATH}
% mpirun -np 4 a.out

The setenv command will prefix the PATH on both the local and remote hosts with /opt/SUNWhpc/HPC7.1/bin.

For more information about the mpirun command, its usage, and its options, refer to the Sun HPC ClusterTools 7.1 Software User’s Guide.

MCA Parameters

Open MPI provides MCA (Modular Component Architecture) parameters for use with the mpirun command. These parameters direct mpirun to perform the specified functions under Sun Grid Engine. To specify an MCA parameter, use the -mca flag with the mpirun command.

Note - In most cases, you do not need to change the values set in the gridengine MCA parameters. If you run into difficulty and want to change the values for debugging purposes, the option is available.

For more information about MCA parameters, refer to the Sun HPC ClusterTools 7.1 Software User’s Guide.

Submitting Jobs Under Sun Grid Engine Integration

To submit jobs under Sun Grid Engine integration in Sun HPC ClusterTools 7.1, you must first create a Sun Grid Engine (SGE) environment using qsub, qsh, and so on. Instructions about how to set up the parallel environment (PE) and queue in Sun Grid Engine are described in the Sun HPC ClusterTools 7.1 Software User’s Guide.

There are two ways to submit jobs under Sun Grid Engine integration: interactive mode and batch mode. These procedures are also described in the Sun HPC ClusterTools 7.1 Software User’s Guide.

In previous versions of Sun HPC ClusterTools software, integration with Sun Grid Engine required you to set an mprun.conf file and to use the mprun -x sge option. When using Sun HPC ClusterTools 7.1, neither of these steps is necessary.

More Information About Sun Grid Engine, the Open Source Grid Engine, and MCA Parameters

For more information about Sun Grid Engine, refer to the following:

Sun Grid Engine product page:

http://www.sun.com/software/gridware

Sun Grid Engine 6 Administration Guide

http://docs.sun.com/app/docs/doc/817-5677

Sun Grid Engine 6 Installation Guide

http://docs.sun.com/app/docs/doc/817-6118

Sun Grid Engine 6 User’s Guide

http://docs.sun.com/app/docs/doc/817-6117

Grid Engine project on SunSource:

http://gridengine.sunsource.net

For more information about MCA parameters, refer to the following:

Sun HPC ClusterTools 7.1 Software User’s Guide (819-7480)

Sun HPC ClusterTools 7.1 Software Release Notes (819-7478)

Open MPI FAQ

http://www.open-mpi.org/faq/?category=tuning