Sun N1 Grid Engine 6.1 Installation Guide

Additional Setup Required

The steps required to set up the Certificate Security Protocol enhanced version of the grid engine system are similar to the standard setup. You generally follow the instructions in Plan the Installation, How to Load the Distribution Files On a Workstation, How to Install the Master Host, How to Install Execution Hosts, and Registering Administration Hosts.

However, the following additional tasks are required:

Generating the Certificate Authority (CA) system keys and certificates on the master host by calling the installation script with the -csp flag

Distributing the system keys and certificates to the execution and submit hosts using a secure method such as ssh

Generating user keys and certificates automatically, after master installation

Adding new users

How to Install a CSP-Secured System

Install the grid engine software as outlined in Performing an Installation, with the following exception: use the additional flag -csp when invoking the various installation scripts.

To install a CSP-secured system, change the master host installation procedure by typing the following command and responding to the prompts from the installation script.
# ./install_qmaster -csp

To generate the CSP certificates and keys, you must supply the following information:
- Two-letter country code, for example, US for the United States
- State
- Location, such as a city
- Organization
- Organizational unit
- CA email address
As the installation proceeds, the Certificate Authority is created. A CA specific to the grid engine system is created on the master host. The directories that contain information relevant to security are as follows:
- The publicly accessible CA and daemon certificate are stored in sge-root/cell/common/sgeCA.
- The corresponding private keys are stored in /var/sgeCA/{sge_service | portSGE_QMASTER_PORT}/cell/private.
- User keys and certificates are stored in /var/sgeCA/{sge_service | portSGE_QMASTER_PORT}/cell/userkeys/$USER.

The script prompts you for site information.

Confirm whether the information you supplied is correct.

After the security-related setup of the master host sge_qmaster is finished, the script prompts you to continue with the rest of the installation procedure, as in the following example:
SGE startup script -------------------- Your system wide SGE startup script is installed as: "/scratch2/eddy/sge_sec/default/common/sgemaster" Hit Return to continue >>

Determine whether the shared file system is secure enough to hold the CSP security information.

If you think that the shared file system is secure enough, continue with the basic installation procedure as outlined in How to Install Execution Hosts.

Note –
Remember to add the -csp flag when you call the ./install_execd script for the execution host installation.

If the root user does not have write permissions in the sge-root directory on all of the machines where grid engine software will be installed, you are asked whether to install the software as the user to whom the directory belongs. If you answer yes, you must install the security-related files into that user's $HOME/.sge directory, as shown in the following example. In the following example, sgeadmin is the name of the user who owns the installation directory.
% su - sgeadmin % source sge-root/default/common/settings.csh % sge-root/util/sgeCA/sge_ca -copy % logout
After completing all remaining installation steps, refer to the instructions in How to Generate Certificates and Private Keys for Users.

If you think that the shared file system is not secure enough to hold the CSP security information in a place that can be accessed by the execution daemons, you must transfer the directory containing the daemon's private key and the random file to the execution host.

As root on the master host, type the following commands to prepare to copy the private keys to the machines you set up as execution hosts.
# umask 077 # cd / # tar cvpf /var/sgeCA/port536.tar /var/sgeCA/port536/default

As root on each execution host, use the following commands to securely copy the files.
# umask 077 # cd / # scp masterhost:/var/sgeCA/port536.tar . # umask 022 # tar xvpf /port536.tar # rm /port536.tar
Note –
On a Windows execution host, the tar utility cannot restore the ownerships and permissions. In this case, the Administrator must set the ownerships and permissions manually.

Type the following command to verify the file permissions.

# ls -lR /var/sgeCA/port536/

The output should look like the following example:

/var/sgeCA/port536/:
total 2
drwxr-xr-x   4 eddy     other        512 Mar  6 10:52 default
/var/sgeCA/port536/default:
total 4
drwx------   2 eddy     staff        512 Mar  6 10:53 private
drwxr-xr-x   4 eddy     staff        512 Mar  6 10:54 userkeys
/var/sgeCA/port536/default/private:
total 8
-rw-------   1 eddy     staff        887 Mar  6 10:53 cakey.pem
-rw-------   1 eddy     staff        887 Mar  6 10:53 key.pem
-rw-------   1 eddy     staff       1024 Mar  6 10:54 rand.seed
-rw-------   1 eddy     staff        761 Mar  6 10:53 req.pem
/var/sgeCA/port536/default/userkeys:
total 4
dr-x------   2 eddy     staff        512 Mar  6 10:54 eddy
dr-x------   2 root     staff        512 Mar  6 10:54 root
/var/sgeCA/port536/default/userkeys/eddy:
total 16
-r--------   1 eddy     staff       3811 Mar  6 10:54 cert.pem
-r--------   1 eddy     staff        887 Mar  6 10:54 key.pem
-r--------   1 eddy     staff       2048 Mar  6 10:54 rand.seed
-r--------   1 eddy     staff        769 Mar  6 10:54 req.pem
/var/sgeCA/port536/default/userkeys/root:
total 16
-r--------   1 root     staff       3805 Mar  6 10:54 cert.pem
-r--------   1 root     staff        887 Mar  6 10:54 key.pem
-r--------   1 root     staff       2048 Mar  6 10:53 rand.seed
-r--------   1 root     staff        769 Mar  6 10:54 req.pem

Install the security-related files into the admin user's $HOME/.sge directory.

If the root user does not have write permissions in the sge-root directory on all of the machines where grid engine software will be installed, you are asked whether to install the software as the user to whom the directory belongs. If you answer yes, you must install the security-related files into that user's $HOME/.sge directory, as shown in the following example. In the following example, sgeadmin is the name of the user who owns the installation directory.
% su - sgeadmin % source sge-root/default/common/settings.csh % sge-root/util/sgeCA/sge_ca -copy % logout

Continue with the grid engine software installation on the execution host by typing the following commands.

Note –
You can also use the master host for executing jobs. To do so, you must carry out the execution host installation for the master machine. However, if you use a very slow machine as master host, or if your cluster is significantly large, do not use the master host as an execution host.
# cd sge-root # ./install_execd -csp

Respond to the prompts from the installation script.

The execution host installation procedure creates the appropriate directory hierarchy required by sge_execd, and starts the sge_execd daemon on the execution host.

Create the environment variables for use with the grid engine software.

Note –
If no cell name was specified during installation, the value of cell is default.
- If you are using a C shell, type the following command:
  % source sge_root/cell/common/settings.csh
- If you are using a Bourne shell or Korn shell, type the following command:
  $ . sge_root/cell/common/settings.sh

Proceed to the next task,How to Generate Certificates and Private Keys for Users.

How to Generate Certificates and Private Keys for Users

To use the CSP-secured system, the user must have access to a user-specific certificate and private key. The most convenient method of gaining access is to create a text file identifying the users.

On the master host, create and save a text file that identifies users.

Use the format of the file myusers.txt shown in the following example. The fields of the file are UNIX_username:Gecos_field:email_address.
eddy:Eddy Smith:eddy@my.org sarah:Sarah Miller:sarah@my.org leo:Leo Lion:leo@my.org

As root on the master host, type the following command:

# sge-root/util/sgeCA/sge_ca -usercert myusers.txt

Confirm by typing the following command:

# ls -l /var/sgeCA/port536/default/userkeys

This directory listing produces output similar to the following example.

dr-x------  2 eddy  staff       512 Mar  5 16:13 eddy
dr-x------  2 sarah staff        512 Mar  5 16:13 sarah
dr-x------  2 leo   staff        512 Mar 5 16:13 leo

Tell each user listed in the file (myusers.txt in the example) to install the security-related files in their $HOME/.sge directories by typing the following commands.

% source sge-root/default/common/settings.csh
% sge-root/util/sgeCA/sge_ca -copy

Users should see the following confirmation (user eddy in the example).

Certificate and private key for user
eddy have been installed

For every grid engine software installation, a subdirectory for the corresponding SGE_QMASTER_PORT number is installed. The following example, based on the myusers.txt file, results from issuing the command preceding the output.

% ls -lR $HOME/.sge

/home/eddy/.sge:
total 2
drwxr-xr-x  3 eddy staff        512 Mar  5 16:20 port536

/home/eddy/.sge/port536:
total 2
drwxr-xr-x  4 eddy staff        512 Mar  5 16:20 default

/home/eddy/.sge/port536/default:
total 4
drwxr-xr-x  2 eddy staff        512 Mar  5 16:20 certs
drwx------  2 eddy staff        512 Mar 5 16:20 private

/home/eddy/.sge/port536/default/certs:
total 8
-r--r--r--  1 eddy staff       3859 Mar  5 16:20 cert.pem

/home/eddy/.sge/port536/default/private:
total 6
-r--------  1 eddy staff        887 Mar  5 16:20 key.pem
-r--------  1 eddy staff       2048 Mar 5 16:20 rand.seed

How to Renew Certificates

Change to $SGE_ROOT and becomeroot on the master host (assuming that $SGE_CELL is the default):
# tcsh # source $SGE_ROOT/default/settings.csh

Edit $SGE_ROOT/util/sgeCA/renew_all_certs.csh, and change the number of days that the certificates are valid:

 # extend the validity of the CA certificate by
  set CADAYS = 365
  # extend the validity of the daemon certificate by
  set DAEMONDAYS = 365
  # extend the validity of the user certificate by
  set USERDAYS = 365

Run the changed script (the default for all extension times is 365 days from the day the script is run).
# util/sgeCA/renew_all_certs.csh

Replace the old certificates against the new ones on all hosts that installed them locally (that is, under /var/sgeCA/..., see the execution daemon installation).

If users have copied certificates and keys to $HOME/.sge, they have to repeat $SGE_ROOT/util/sgeCA/sge_ca -copy to have access to the renewed certificates.

Checking Certificates

The following sections provide examples of commands related to certificates, where arch is your system architecture, as in sol-sparc64. Depending on what you want to do, type one or more of the following commands.