test

Sun S3L 3.0 Programming and Reference Guide

Chapter 7 Sun S3L Toolkit Routines

Sun S3L provides an extensive subset of auxiliary routines, referred to as the toolkit functions, which enable you to do the following.

Setting Up a Sun S3L Environment

S3L_init

Description

Before an application can start using Sun S3L functions, every process involved in the application must call S3L_init to initialize the S3L environment. S3L_init initializes the BLACS environment as well.

S3L_init also initializes the Sun MPI layer if the user has not done so. If S3L_init calls MPI_Init internally, subsequent use of S3L_exit will also result in an internal call to MPI_Finalize.

S3L_init tests the MPI library to verify that it is Sun MPI. If not, it returns an error message and terminates.

If the MPI layer is Sun MPI, S3L_init proceeds to initialize the S3L environment, the BLACS environment, and if not already initialized, the Sun MPI environment. It also enables the Prism library to access Sun S3L operations.

Syntax

The C and Fortran syntax for S3L_init are illustrated below.

C/C++ Syntax

Example 7-1 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_init()

F77/F90 Syntax

Example 7-2 

include `s3l/s3l-f.h'
include s3l/s3l_errno-f.h'
subroutine
S3L_init(ier)
    integer*4          ier

Input

S3L_init takes no input arguments.

Output

When called from a Fortran program, S3L_init returns error status in ier.

Error Handling

On successful completion, S3L_init returns S3L_SUCCESS.

S3L_init tests to see if the MPI library is Sun MPI. If not, it returns the following error message and terminates.

S3L error: invalid
MPI. Please use Sun HPC MPI.

Examples

../examples/s3l/utils/copy_array.c 
../examples/s3l/utils/copy_array.f

Related Functions

S3L_exit(3)

Leaving a Sun S3L Environment

S3L_exit

Description

When an application is finished using Sun S3L functions, it must call S3L_exit to perform various cleanup tasks associated with the current S3L environment.

S3L_exit checks to see if the S3L environment is in the initialized state--S3L_init has been called more recently than S3L_exit. If not, S3L_exit returns the error message S3L_ERR_NOT_INIT and exits.

Syntax

The C and Fortran syntax for S3L_exit are illustrated below.

C/C++ Syntax

Example 7-3 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_exit()

F77/F90 Syntax

Example 7-4 

include `s3l/s3l-f.h'
include s3l/s3l_errno-f.h'
subroutine
S3L_exit(ier)
    integer*4          ier

Input

S3L_exit takes no input arguments.

Output

When called from a Fortran program, S3L_exit returns error status in ier.

Error Handling

On successful completion, S3L_exit returns S3L_SUCCESS.

The following condition will cause S3L_exit to terminate and return the associated error value:

Examples

../examples/s3l/dense_matrix_ops/inner_prod.c 
../examples/s3l/dense_matrix_ops-f/inner_prod.f
../examples/s3l/utils/copy_array.f

Related Functions

S3L_init(3)

Declaring Parallel Arrays

The Sun S3L toolkit functions described in this section share a common purpose--they all enable you to define parallel arrays that can then be operated on by other Sun S3L routines. Each of these routines returns an S3L array handle, which the application uses to reference the parallel array in subsequent S3L calls.

Each array declaring routine is described separately below.

Sun S3L also provides a routine for declaring a sparse matrix. This routine, called, S3L_declare_sparse is described in "S3L_declare_sparse ".

S3L_declare

Description

S3L_declare creates an S3L array handle that describes an S3L parallel array . It supports calling arguments that enable the user to specify

Based on the argument-supplied specifications, a process grid size is internaly determined to distribute the array as evenly as possible.

Note -

An array subgrid is the set of array elements that is allocated to a particular process.

The axis_is_local argument specifies which array axes (if any) will be local to the process. It consists of an integer vector whose length is at least equal to the rank (number of dimensions) of the array. Each element of the vector indicates whether the corresponding axis is local or not: 1 = local, 0 = not local.

When axis_is_local is ignored, all array axes except the last will be local. The last axis will be block distributed.

For greater control over array distribution, use S3L_declare_detailed().

Upon successful completion, S3L_declare returns an S3L array handle, which subsequent S3L calls can use as an argument to gain access to that array.

Syntax

The C and Fortran syntax for S3L_declare are shown below.

C/C++ Syntax

Example 7-5 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_declare(A, rank, extents, type, axis_is_local, atype)
    S3L_array_t        *A
    int                rank
    int                *extents
    S3L_data_type      type
    S3L_boolean_t      *axis_is_local
    S3L_alloc_type     atype

F77/F90 Syntax

Example 7-6 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_declare(A, rank, extents, type, axis_is_local, atype, ier)
    integer*8           A
    integer*4          rank
    integer*4          extents(*)
    integer*4          type
    integer*4          axis_is_local(*)
    integer*4          atype
    integer*4          ier

Input

Output

S3L_declare uses the following arguments for output:

Error Handling

On successful completion, S3L_declare returns S3L_SUCCESS.

S3L_declare applies verious checks to the arrays it accepts as arguments. If an array argument fails any of these checks, the function returns an error code indicating the kind of error that was detected and terminates. See Appendix A, S3L Array Checking Errors of this manual for a list of these error codes.

In addition, the following conditions will cause S3L_declare to terminate and return the associated error value:

Notes

When S3L_USE_MMAP or S3L_USE_SHMGET is used on a 32-bit platform, the part of an S3L array owned by a single SMP cannot exceed 2 gigabytes.

When S3L_USE_MALLOC or S3L_USE_MEMALIGN64 is used, the part of the array owned by any single process can not exceed 2 gigabytes.

If these size restrictions are violated, an S3L_ERR_MEMALLOC will be returned. On 64-bit platforms, the upper bound is equal to the system's maximum available memory.

Examples

../examples/s3l/transpose/ex_trans1.c
../examples/s3l/grade-f/ex_grade.f

Related Functions

S3L_declare_detailed(3)
S3L_free(3)

S3L_declare_detailed

Description

This subroutine offers the same functionality as S3L_declare, but supports the additional input argument, addr_a, which gives the user additional control over array distribution.

Syntax

The C and Fortran syntax for S3L_declare_detailed are shown below.

C/C++ Syntax

Example 7-7 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_declare_detailed(A, addr_a, rank, extents, type, blocksizes, proc_src,
axis_is_local, pgrid, atype)
    S3L_array_t        *A
    void               *addr_a
    int                rank
    int                *extents
    S3L_data_type      type
    int                *blocksizes
    int                *proc_src
    S3L_boolean_t      *axis_is_local
    S3L_pgrid_t        pgrid
    S3L_alloc_type     atype

F77/F90 Syntax

Example 7-8 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_declare_detailed(A, addr_a, rank, extents, type, blocksizes, proc_src,
axis_is_local, pgrid, atype, ier)
    integer*8          A
    <type>             array(1)
    pointer            (addrr_a,array)
    integer*4          rank
    integer*4          extents(*)
    integer*8          type
    integer*4          *blocksizes
    integer*4          *proc_src
    integer*4          axis_is_local(*)
    integer*8          pgrid
    integer*4          atype
    integer*4          ier

where <type> is one of: integer*4, integer*8, real*4, real*8, complex*8, or complex*16.

Input

S3L_declare_detailed accepts the following arguments as input:

Output

Error Handling

On successful completion, S3L_declare_detailed returns S3L_SUCCESS.

S3L_declare_detailed applies various checks to the arrays it accepts as arguments. If an array argument fails any of these checks, the function returns an error code indicating the kind of error that was detected and terminates. See Appendix A, S3L Array Checking Errors of this manual for a list of these error codes.

In addition, the following conditions will cause S3L_declare_detailed to terminate and return the associated error value:

Notes

When S3L_USE_MMAP or S3L_USE_SHMGET is used on a 32-bit platform, the part of an S3L array owned by a single SMP cannot exceed 2 gigabytes.

When S3L_USE_MALLOC or S3L_USE_MEMALIGN64 is used, the part of the array owned by any single process cannot exceed 2 gigabytes.

If these size restrictions are violated, an S3L_ERR_MEMALLOC will be returned. On 64-bit platforms, the upper bound is equal to the system's maximum available memory.

Examples

../examples/s3l/utils/copy_array.c
../examples/s3l/utils-f/copy_array.f
../examples/s3l/utils/get_attribute.c
../examples/s3l/utils-f/get_attribute.f
../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

S3L_declare(3)
S3L_free(3)
S3L_get_process_grid(3)
S3L_set_process_grid(3)

S3L_DefineArray

Description

S3L_DefineArray associates an internal S3L array handle to a user-distributed parallel array. The array must be distributed in such a manner that it can be expressed as a block cyclic distribution. The array handle returned by S3L_DefineArray can then be used in subsequent calls by Sun MPI programs to S3L functions.

S3L_DefineArray does not allocate the memory required to store the local (process specific) part on the array. The user must allocate sufficient memory on each process to hold the local part of the parallel array before calling this function.

Syntax

The C and Fortran syntax for S3L_DefineArray are shown below.

C/C++ Syntax

Example 7-9 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
S3L_array_t
S3L_DefineArray(address, rank, type, extents, blocks,
sprocs, p_ext)
    void               address
    int                rank
    int                type
    int                *extents
    int                *blocks
    int                *sprocs
    int                *p_ext

F77/F90 Syntax

Example 7-10 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
integer*8 function
S3L_DefineArray(address,
rank, type, extents, blocks, sprocs, p_ext)
    <type>             array(1)
    integer*4          rank
    pointer            (addrr_a,array)
    integer*4          rank
    integer*4          type
    integer*4          extents(*)
    integer*4          blocks(*)
    integer*4          sprocs(*)
    integer*4          p_ext(*)

Input

S3L_DefineArray accepts the following arguments as input:

Note that process ordering within the process grid is always column major (F77 major) and that the product of the extents of the process grid must equal the total number of processes participating in the computation (that is, must equal the size of MPI_COMM_WORLD).

Error Handling

On success, S3L_DefineArray returns an S3L array handle that can be used for subsequent calls to other Sun S3L functions.

On error, it returns 0.

Examples

../examples/s3l/api
../examples/s3l/api-f

Related Functions

S3L_UnDefineArray(3)

Parallel Process Grids

S3L_set_process_grid

Description

S3L_set_process_grid allows the user to define various aspects of an internal process grid. It returns a process grid handle, which subsequent calls to other Sun S3L functions can use to refer to that process grid.

Syntax

The C and Fortran syntax for S3L_DefineArray are shown below.

C/C++ Syntax

Example 7-11 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_set_process_grid(pgrid,
rank, majorness, grid_extents, plist_length, process_list)
    S3L_pgrid_t        *pgrid
    int                rank
    int                majorness
    int                *grid_extents
    int                plist_length
    int                *process_list

F77/F90 Syntax

Example 7-12 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_set_process_grid(pgrid,
rank, majorness, grid_extents, plist_length, process_list, ier)
    integer*8          pgrid
    integer*4          rank
    integer*4          majorness
    integer*4          grid_extents(*)
    integer*4          plist_length
    integer*4          process_list(*)
    integer*4          ier

Input

S3L_set_process_grid accepts the following arguments as input:

Output

S3L_set_process_grid uses the following arguments for output:

Error Handling

On success, S3L_set_process_grid returns S3L_SUCCESS.

S3L_set_process_grid performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code.

Examples

../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

S3L_declare_detailed(3)
S3L_free_process_grid(3)

S3L_free_process_grid

Description

S3L_free_process_grid frees the process grid handle returned by a previous call to S3L_set_process_grid.

Syntax

The C and Fortran syntax for S3L_DefineArray are shown below.

C/C++ Syntax

Example 7-13 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_free_process_grid
    S3L_pgrid_t        *pgrid

F77/F90 Syntax

Example 7-14 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_free_process_grid(pgrid, ier)
    integer*8          pgrid
    integer*4          ier

Input

S3L_free_process_grid accepts the following arguments as input:

Output

S3L_free_process_grid uses the following arguments for output:

Error Handling

On success, S3L_free_process_grid returns S3L_SUCCESS.

On error, the following error code may be returned:

Examples

../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

S3L_set_process_grid(3)

Deallocating Parallel Arrays

S3L_free

Description

S3L_free deallocates the memory reserved for a parallel S3L array and undefines the associated array handle.

Note -

If memory was allocated for the array by the user rather than by S3L, S3L_free destroys the array handle, but does not deallocate the memory. This situation can arise when S3L_declare_detailed() is invoked with the atype option set to S3L_DONOT_ALLOCATE.

Syntax

The C and Fortran syntax for S3L_free are shown below.

C/C++ Syntax

Example 7-15 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_free(a)
    S3L_array_t        *a

F77/F90 Syntax

Example 7-16 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_free(a, ier)
    integer*8          a
    integer*4          ier

Input

S3L_free accepts the following argument as input:

Output

S3L_free uses the following argument for output:

Error Handling

On success, S3L_free returns S3L_SUCCESS.

On error, the following error code may be returned:

Examples

../examples/s3l/io/ex_print1.c
../examples/s3l/io-f/ex_print1.f

Related Functions

S3L_declare(3)
S3L_declare_detailed(3)

S3L_UnDefineArray

Description

S3L_UnDefineArray frees the array handle and the associated memory that were set up by a previous call to S3L_DefineArray.

Note -

S3L_UnDefineArray does not free the local (process-resident) memory, where the local part of a parallel array is stored. The user is responsible for deallocating local memory assigned to the parallel array before the parallel program exits.

Syntax

The C and Fortran syntax for S3L_UnDefineArray are shown below.

C/C++ Syntax

Example 7-17 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_UnDefineArray(a)
    S3L_array_t         a

F77/F90 Syntax

Example 7-18 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_UnDefineArray(a)
    integer*8          a

Input

S3L_UnDefineArray accepts the following argument as input:

Error Handling

S3L_UnDefineArray does not return any value.

Examples

../examples/s3l/api
../examples/s3l/api.f
../examples/s3l/array_utils

Related Functions

S3L_DefineArray(3)
S3L_declare(3)
S3L_declare_detailed(3)
S3L_free(3)

Performing Operations on S3L Parallel Arrays

The toolkit functions described in this section enable the user to apply various kinds of operations on a parallel array's elements.

S3L_array_op1

Description

S3L_array_op1 applies a predefined unary (single-operand) operation to each element of an S3L parallel array. The S3L array handle argument, a, identifies the parallel array to be operated on and the op argument specifies the operation to be performed. The value of op must be:

Syntax

The C and Fortran syntax for S3L_array_op1 are shown below.

C/C++ Syntax

Example 7-19 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_array_op1(a, op)
    S3L_array_t         a
    int                 op

F77/F90

Example 7-20 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_array_op1(a, op, ier)
    integer*8          a
    integer*4          op
    integer*4          ier

Input

S3L_array_op1 accepts the following arguments as input:

Output

S3L_array_op1 uses the following argument for output:

Error Handling

On success, S3L_array_op1 returns S3L_SUCCESS.

S3L_array_op1 performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code.

Examples

../examples/s3l/fft/ex_fft1.c
../examples/s3l/deconv-f/ex_deconv.f

Related Functions

S3L_array_op2(3)
S3L_array_scalar_op2(3)
S3L_reduce_scalar(3)

S3L_array_op2

Description

S3L_array_op2 applies the operation specified by op to elements of parallel arrays a and b, which must be of the same type and have the same distribution. The parameter op can be one of the following:

S3L_OP_MUL replaces each element in a with the elementwise product of multiplying a and b.

S3L_OP_CMUL performs the same operation as S3L_OP_MUL, except it multiplies each element in a by the conjugate of the corresponding element in b.

S3L_OP_DIV performs elementwise division of a by b, overwriting a with the integer (truncated quotient) results.

S3L_OP_MINUS performs elementwise subtraction of b from a, overwriting a with the differences.

S3L_OP_PLUS performs elementwise addition of a with b, overwriting a with the sum.

Syntax

The C and Fortran syntax for S3L_array_op2 are shown below.

C/C++ Syntax

Example 7-21 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_array_op2(a, b, op)
    S3L_array_t         a
    S3L_array_t         b
    int                 op

F77/F90 Syntax

Example 7-22 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_array_op2(a, b, op, ier)
    integer*8          a
    integer*8          b
    integer*4          op
    integer*4          ier

Input

S3L_array_op2 accepts the following arguments as input:

Output

S3L_array_op2 uses the following argument for output:

Error Handling

On success, S3L_array_op2 returns S3L_SUCCESS.

S3L_array_op2 performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/fft/ex_fft1.c
../examples/s3l/fft-f/ex_fft1.f

Related Functions

S3L_array_op1(3)
S3L_array_scalar_op2(3)
S3L_reduce_scalar(3)

S3L_array_scalar_op2

Description

S3L_array_scalar_op2 applies a binary operation to each element of an S3L array that involves the element and a scalar.

op determines which operation will be performed. It can be one of:

Syntax

The C and Fortran syntax for S3L_array_scalar_op2 are shown below.

C/C++ Syntax

Example 7-23 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_array_scalar_op2(a, scalar,
op)
    S3L_array_t         a
    void                *scalar
    int                 op

F77/F90 Syntax

Example 7-24 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_array_scalar_op2(a, scalar, op, ier)
    integer*8          a
    <type>             scalar
    integer*4          op
    integer*4          ier

where <type> is one of: integer*4, integer*8, real*4, real*8, complex*8, or complex*16.

Input

S3L_array_scalar_op2 accepts the following arguments as input:

Output

S3L_array_scalar_op2 uses the following argument for output:

Error Handling

On success, S3L_array_scalar_op2 returns S3L_SUCCESS.

S3L_array_scalar_op2 performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/fft/ex_fft1.c
../examples/s3l/fft-f/ex_fft1.f

Related Functions

S3L_array_op1(3)
S3L_array_op2(3)
S3L_reduce_scalar(3)

S3L_cshift

S3L_cshift performs a circular shift of a specified amount along a specified axis of the parallel array associated with array handle A. The argument axis indicates the dimension to be shifted, and index prescribes the shift distance.

Shift direction is upwards for positive index values and downward for negative index values.

For example, if A denotes a one-dimensional array of length n before the cshift, B denotes the same array after the cshift, and index is equal to 1, the array A will be circularly shifted upwards, as shown below:

For C Programs

B[1:n-1]=A[0:n-2], B[0]=A[n-1]

For Fortran Programs

B(2:n)=A(1:n-1), B(1)=A(n)

Syntax

The C and Fortran syntax for S3L_cshift are shown below.

C/C++ Syntax

Example 7-25 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_cshift(A, axis, index)
    S3L_array_t         A
    void                axis
    int                 index

F77/F90 Syntax

Example 7-26 

include `s3l/s3l-f.h
include `s3l/s3l_errno-f.h'
subroutine
S3L_cshift(A, axis, index,
ier)
    integer*8          A
    integer*4          axis
    integer*4          index
    integer*4          ier

Input

S3L_cshift accepts the following arguments as input:

Output

S3L_cshift uses the following argument for output:

Error Handling

On success, S3L_cshift returns S3L_SUCCESS.

S3L_cshift performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/utils/cshift_reduce.c
../examples/s3l/utils-f/cshift_reduce.f

Related Functions

S3L_reduce
S3L_reduce_axis

S3L_forall

Description

S3L_forall applies a user-defined function to elements of a parallel Sun S3L array. and sets its values accordingly. Three different function types are supported. These types are described in Table 7-1.

Table 7-1 User-Defined Function Types for S3L_forall

fn_type 

C Prototype 

Fortran Interface 

S3L_ELEM_FN1 

void user_fn(void *elem_addr); 

subroutine user_fn(a) 

   <type>  a 

   end  user_fn 

S3L_ELEM_FNN 

void user_fn(void *elem_addr, int n); 

subroutine user_fn(a,n) 

   <type> a 

   integer*4  n 

   end  user_fn 

S3L_ELEM_FN 

void user_fn(void *elem_addr,  

int *coord); 

subroutine user_fn(a, coord) 

   <type>  a 

Here, <type> is one of: integer*4, integer*8, real*4, real*8, complex*8, or complex*16 and rank is the rank of the array.

For S3L_ELEM_FN1, the user function is applied to each element in the array.

For S3L_ELEM_FNN, the user function is supplied the local subgrid address and subgrid size and iterates over subgrid elements. This form delivers the highest performance because the looping over the elements is contained within the function call.

For S3L_INDEX_FN, the user function is applied to each element in the subarray specified by the triplets argument to S3L_forall. If the triplets argument is NULL in C/C++ or has a leading value of 0 in F77/F90, the whole array is implied. The user function may involve the global coordinates of the array element; these are contained in the coord argument. Global coordinates of array elements are 0-based for C programs and 1-based for Fortran programs.

Note -

When a Fortran program uses triplets, the length of first axis of the triplets must equal the rank of the array. Failure to meet this requirement can produce wrong results or a segmentation violation.

Note -

A subgrid is the portion of the parallel array that is owned by a process. A subarray is the portion of the parallel array that is described by a lower bound, an upper bound, and a stride in each dimension.

Syntax

The C and Fortran syntax for S3L_cshift are shown below.

C/C++ Syntax

Example 7-27 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_forall(a, user_fn, fn_type, triplets)
    S3L_array_t        a
    void               (*user_fn)()
    int                fn_type
    int                triplets[rank][3]

where rank is the rank of the array.

F77/F90 Syntax

Example 7-28 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_forall(a, user_fn, fn_type, triplets, ier)
    integer*8          a
    <external>         user_fn
    integer*4          fn_type
    integer*4          triplets(rank,3)
    integer*4          ier

where rank is the rank of the array.

Input

S3L_forall accepts the following arguments as input:

inclusive lower bound inclusive upper bound stride

for each axis of the array. The stride must be positive. To apply the function to all the elements in the array, set triplets to NULL (C/C++) or to 0 (F77/F90).

Output

S3L_forall uses the following argument for output:

Error Handling

On success, S3L_forall returns S3L_SUCCESS.

S3L_forall performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/forall/ex_forall.c
../examples/s3l/forall/ex_forall2.cc
../examples/s3l/forall-f/ex_forall.f

S3L_reduce

Description

S3L_reduce performs a predefined reduction function over all elements of a parallel array. The array is described by the S3L array handle argument A. The argument op specifiesthe type of reduction operations, which can be one of the following:

Syntax

The C and Fortran syntax for S3L_reduce are shown below.

C/C++ Syntax

Example 7-29 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_reduce(A, op, res)
    S3L_array_t           A
    S3L_op_type           op
    void                  *res

F77/F90 Syntax

Example 7-30 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_reduce(A, op, res, ier)
    integer*8              A
    integer*4              op
    <type>                 res
    integer*4              ier

where <type> is one of: real*4, real*8, complex*8, or complex*16.

Input

S3L_reduce accepts the following arguments as input:

Output

S3L_reduce uses the following arguments for output:

Error Handling

On success, S3L_reduce returns S3L_SUCCESS.

S3L_reduce performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code.

Examples

../examples/s3l/utils/cshift_reduce.c
../examples/s3l/utils-f/cshift_reduce.f

Related Functions

S3L_reduce_axis(3)

S3L_reduce_axis

Description

S3L_reduce_axis applies a predefined reduction operation along a given axis of a parallel S3L array. If n is the rank (number of dimensions) of a, the result b is a parallel array of rank n-1. The argument op specifies the operation to be performed. The value of op must be one of:

Syntax

The C and Fortran syntax for S3L_reduce_axis are shown below.

C/C++ Syntax

Example 7-31 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_reduce_axis(a, op, axis,
b)
    S3L_array_t           a
    S3L_op_type           op
    int                   axis
    S3L_array_t           b

F77/F90 Syntax

Example 7-32 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_reduce_axis(a, op, axis, b, ier)
    integer*8              a
    integer*4              op
    integer*4              axis
    integer*8              b
    integer*4              ier

Input

S3L_reduce_axis accepts the following arguments as input:

Output

S3L_reduce_axis uses the following arguments for output:

Error Handling

On success, S3L_reduce_axis returns S3L_SUCCESS.

S3L_reduce_axis performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/utils/cshift_reduce.c
../examples/s3l/utils-f/cshift_reduce.f

Related Functions

S3L_reduce(3)

S3L_set_array_element, S3L_get_array_element, S3L_set_array_element_on_proc, and S3L_get_array_element_on_proc

Description

The four subroutines described in this section enable the user to alter (set) and retrieve (get) individual elements of an array. Two of these subroutines also allow the user to know which process will participate in the set/get activity.

S3L_set_array_element assigns the value stored in val to a specific element of a distributed S3L array, whose global coordinates are specified by coor. The val variable is colocated with the array subgrid containing the target element.

Note -

Because an S3L array is distibuted across a set of processes, each process has a subsection of the global array local to it. These array subsections are also referred to as array subgrids.

For example, if a parallel array is distributed across four processes, P0 - P3, and coor specifies an element in the subgrid that is local to P2, the val that is located on P2 will be the source of the value used to set the target element.

S3L_get_array_element is similar to S3L_set_array_element, but operates in the opposite direction. It assigns the value stored in the element specified by coor to val on every process. Since S3L_get_array_element broadcasts the element value to every process, upon completion, every process contains the same value in val.

S3L_set_array_element_on_proc specifies which process will be the source of the value to be assigned to the target element. That is, the argument pnum specifies the MPI rank of a particular process. The value of the variable val on that process will be assigned to the target element--that is, the element whose coordinates are specified by coor.

Note -

The MPI rank of a process is defined in the global communicator MPI_COMM_WORLD.

S3L_get_array_element_on_proc updates the variable val on the process whose MPI rank is supplied in pnum, using the element whose indices are given in coor as the source for the update.

Syntax

The C and Fortran syntax for S3L_set_array_element and its related routines are shown below.

C/C++ Syntax

Example 7-33 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_set_array_element(a, coor, val)
S3L_get_array_element(a, coor, val)
S3L_set_array_element_on_proc(a, coor, val, pnum)
S3L_get_array_element_on_proc(a, coor, val, pnum)
    S3L_array_t           a
    int                   coor
    void                  val
    int                   pnum

F77/F90 Syntax

Example 7-34 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_set_array_element(a, coor, val, ier)
S3L_get_array_element(a,
coor, val, ier)
S3L_set_array_element_on_proc(a, coor, val, pnum, ier)
S3L_set_array_element_on_proc(a, coor, val, pnum, ier)
    integer*8              a
    integer*4              coor
    <type>                 val
    integer*4              pnum
    integer*4              ier

where <type> is one of: integer*4, real*4, real*8, complex*8, or complex*16.

Input

S3L_set_array_element and S3L_get_array_element accept the following arguments as input:

Output

These functions use the following argument for output:

Error Handling

On success, these functions return S3L_SUCCESS.

In addition, the following conditions will cause these functions to return the associated error code and terminate.

Examples

../examples/s3l/utils/cshift_reduce.c
../examples/s3l/utils-f/cshift_reduce.f

S3L_zero_elements

Description

S3L_zero_elements sets to zero all elements of the S3L array whose array handle is A.

Syntax

The C and Fortran syntax for S3L_zero_elements are illustrated below.

C/C++ Syntax

Example 7-35 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_zero_elements(A)
    S3L_array_t        A

F77/F90 Syntax

Example 7-36 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_zero_elements(A, ier)
    integer*8              A
    integer*4              ier

Input

Output

This function uses the following argument for output:

Error Handling

On success, S3L_zero_elements returns S3L_SUCCESS.

S3L_zero_elements checks the arrays it accepts as argument. If the array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated code:

Examples

../examples/s3l/utils/zero_elements.c
../examples/s3l/utils-f/zero_elements.f

Extracting Information About S3L Parallel Arrays

The functions described in this section allow users to retrieve information about parallel arrays for which an array handle exists.

S3L_describe

Description

S3L_describe prints information about a parallel array or a process grid to standard output. If an array handle is supplied in argument A, the parallel array is described. If a process grid is supplied in A, the associated process grid is described. The info_node argument specifies the MPI rank of the process on which the subgrid of interest is located.

If A is an S3L array handle, the following is provided:

Note: If the entire array fits on the process specified by info_node, all parts of the S3L_describe output apply to the full array. Otherwise, some parts of the output, such as subgrid size, will apply only to the portion of the array that is on process info_node.

If A is a process grid handle, S3L_describe provides only a description of the underlying grid of processes to which data is mapped.

To determine what value to enter for info_node, run MPI_Comm_rank on the process of interest.

Syntax

The C and Fortran syntax for S3L_describe are shown below.

C/C++ Syntax

Example 7-37 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_describe(A, info_node)
    S3L_array_t        A
    int                info_node

F77/F90 Syntax

Example 7-38 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_zero_elements(A, ier)
    integer*8          A
    integer*4          info_node
    integer*4          ier

Input

S3L_describe accepts the following arguments as input:

Output

S3L_describe uses the following argument for output:

Error Handling

On success, S3L_describe returns S3L_SUCCESS.

S3L_describe performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code.

Examples

../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

MPI_Comm_rank(3)
S3L_declare(3)
S3L_declare_detailed(3)
S3L_set_process_grid(3)

S3L_get_attribute

Description

S3L_get_attribute returns a requested attribute of an S3L dense array or sparse matrix. The user specifies one of a set of predefined req_attr values and, on return, the integer value of the requested attribute is stored in attr. For attributes associated with array axes--such as the extents or blocksizes of an array--the user specifies the axis as well.

The req_attr entry must be one of the following:

For an S3L sparse matrix stored in the S3L_SPARSE_CSR format, S3L_RIDX_SGRID_ADDR returns in attr the starting address of an array containing the pointers to the beginning of each row of the local submatrix (per-process).

Note: Users must not change the data returned in attr. It is created for internal use only.

Note: User must not change the data returned in attr. It is created for internal use only.

For an S3L sparse matrix stored in the S3L_SPARSE_CSR format, S3L_RIDX_SGRID_SIZE returns in attr the size of an array containing the pointers to the beginning of each row of the local submatrix (per-process).

Syntax

The C and Fortran syntax for S3L_get_attribute are shown below.

C/C++ Syntax

Example 7-39 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_S3L_get_attribute(a, req_attr, axis, attr)
    S3L_array_t           a
    int                   req_attr
    int                   axis
    void                  *attr

F77/F90 Syntax

Example 7-40 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_get_attribute(a, req_attr, axis, attr, ier)
    integer*8          a
    integer*4          req_attr
    integer*4          axis
    <type>             attr
    integer*4          ier

where <type> is either of integer*4 type or of pointer type. When attr is an address, it should be of type pointer. In all other cases, it should be of type integer*4.

Input

S3L_get_attribute accepts the following arguments as input:

Output

S3L_get_attribute uses the following argument for output:

Error Handling

On success, S3L_get_attribute returns S3L_SUCCESS.

S3L_get_attribute performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/utils/get_attribute.c
../examples/s3l/utils-f/get_attribute.f

Related Functions

S3L_set_array_element(3)
S3L_set_array_element_on_proc(3)

Reading Data Into and Printing From S3L Parallel Arrays

The toolkit functions described in this section allow the user to read data from a file into a parallel array, to write data from a parallel array into a local file, and to print data from a parallel array to standard output.

S3L_read_array and S3L_read_sub_array

S3L_read_array causes the process with MPI rank 0 to read the contents of a distributed array from a local file and distribute them to the processes that own the parts (subgrids) of the array. The local file is specified by the filename argument.

S3L_read_sub_array reads a specific section of the array, within the limits specified by the lbounds and ubounds arguments. The strides argument specifies the stride along each axis; it must be greater than zero. The format argument is a string that specifies the format of the file to be read. It can be either "ascii" or "binary".

The values of lbounds and ubounds should refer to zero-based indexed arrays for the C interface and to one-based indexed arrays for the Fortran interface.

Syntax

The C and Fortran syntax for S3L_read_array and S3L_read_sub_array are shown below.

C/C++ Syntax

Example 7-41 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_read_array(a, filename, format)
S3L_read_sub_array(a, lbounds, ubounds, strides, filename, format)
    S3L_array_t        a
    int                *lbounds
    int                *ubounds
    int                *strides
    char               *filename
    char               *format

F77/F90 Syntax

Example 7-42 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_read_array(a, filename, format, ier)
S3L_read_sub_array(a, lbounds, ubounds, strides, filename, format, ier)
    integer*8          a
    integer*4          lbounds(*)
    integer*4          ubounds(*)
    integer*4          strides(*)
    character*1        filename(*)
    character*1        format(*)
    integer*4          ier

Input

S3L_read_array and S3L_read_sub_array accept the following arguments as input:

Output

S3L_read_array and S3L_read_sub_array use the following argument for output:

Error Handling

On success, S3L_read_array and S3L_read_sub_array return S3L_SUCCESS.

S3L_read_array and S3L_read_sub_array perform generic checking of the validity of the arrays they accept as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/io/ex_io.c
../examples/s3l/io-f/ex_io.f

Related Functions

S3L_print_array(3)
S3L_write_array(3)

S3L_print_array and S3L_print_sub_array

S3L_print_array causes the process with MPI rank 0 to print the parallel array represented by the array handle a to standard output.

S3L_print_sub_array prints a specific section of the parallel array. This array section is defined by the lbounds, ubounds, and strides arguments. lbounds and ubounds specify the array section's lower and upper index bounds. strides specifies the stride to be used along each axis; it must be greater than zero.

Note -

The values of lbounds and ubounds should refer to zero-based indexed arrays for the C interface and to one-based indexed arrays for the Fortran interface.

Syntax

The C and Fortran syntax for S3L_print_array and S3L_print_sub_array are shown below.

C/C++ Syntax

Example 7-43 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_print_array(a)
S3L_print_sub_array(a, lbounds, ubounds, strides)
    S3L_array_t        a
    int                *lbounds
    int                *ubounds
    int                *strides

F77/F90 Syntax

Example 7-44 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_print_array(a, ier)
S3L_print_sub_array(a, lbounds, ubounds, strides, ier)
    integer*8          a
    integer*4          lbounds(*)
    integer*4          ubounds(*)
    integer*4          strides(*)
    integer*4          ier

Input

S3L_print_array and S3L_print_sub_array accept the following arguments as input:

Output

S3L_print_array and S3L_print_sub_array use the following argument for output:

Error Handling

On success, S3L_print_array and S3L_print_sub_array return S3L_SUCCESS.

S3L_print_array and S3L_print_sub_array perform generic checking of the validity of the arrays they accept as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following condition will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/io/ex_print1.c
../examples/s3l/io/ex_io.c
../examples/s3l/io-f/ex_io.f

Related Functions

S3L_read_array(3)
S3L_write_array(3)

S3L_write_array and S3L_write_sub_array

S3L_write_array causes the process with MPI rank 0 to write the parallel array represented by the array handle a into a file specified by the filename argument. The file filename resides on the process with rank 0.

S3L_write_sub_array writes a specific section of the parallel array to filename. This section is defined by the lbounds, ubounds, and strides arguments. lbounds and ubounds specify the array section's lower and upper index bounds. strides specifies the stride along each axis; it must be greater than zero.

Note -

The values of lbounds and ubounds should refer to zero-based indexed arrays for the C interface and to one-based indexed arrays for the Fortran interface.

Syntax

The C and Fortran syntax for S3L_write_array and S3L_write_sub_array are shown below.

C/C++ Syntax

Example 7-45 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_write_array(a, filename, format)
S3L_write_sub_array(a, lbounds, ubounds, strides, filename, format)
    S3L_array_t        a
    int                *lbounds
    int                *ubounds
    int                *strides
    char               *filename
    char               *format

F77/F90 Syntax

Example 7-46 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_write_array(a, filename, format, ier)
S3L_write_sub_array(a, lbounds, ubounds, strides, filename, format,
ier)
    integer*8          a
    integer*4          lbounds(*)
    integer*4          ubounds(*)
    integer*4          strides(*)
    character*1        filename(*)
    character*1        format(*)
    integer*4          ier

Input

S3L_write_array and S3L_write_sub_array accept the following arguments as input:

Output

S3L_write_array and S3L_write_sub_array use the following argument for output:

Error Handling

On success, S3L_write_array and S3L_write_sub_array return S3L_SUCCESS.

S3L_write_array and S3L_write_sub_array perform generic checking of the validity of the arrays they accept as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/io/ex_io.c
../examples/s3l/io-f/ex_io.f

Related Functions

S3L_print_array(3)
S3L_read_array(3)

Copy Array

S3L_copy_array

Description

S3L_copy_array copies the contents of array A into array B, which must have the same rank, extents and data type as A.

Syntax

The C and Fortran syntax for S3L_copy_array are illustrated below.

C/C++ Syntax

Example 7-47 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_write_array(a, filename, format)
    S3L_array_t        A
    S3L_array_t        B
    char               *format

F77/F90 Syntax

Example 7-48 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_copy_array(A, B, ier)
    integer*8          A
    integer*8          B
    integer*4          ier

Input

Output

This function uses the following arguments for output:

Error Handling

On success, S3L_copy_array returns S3L_SUCCESS.

S3L_copy_array checks the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated code:

Examples

../examples/s3l/utils/copy_array.c
../examples/s3l/utils-f/copy_array.f

Converting Between ScaLAPACK Descriptors and S3L Array Handles

The functions described in this section make it possible to convert ScaLAPACK descriptors to S3L array handles and vice versa.

S3L_from_ScaLAPACK_desc

Description

S3L_from_ScaLAPACK_desc converts the ScaLAPACK descriptor and subgrid address specified by scdesc and address into an S3L array handle, which is returned in s3ldesc.

Syntax

The C and Fortran syntax for S3L_from_ScaLAPACK_desc are shown below.

C/C++ Syntax

Example 7-49 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_from_ScaLAPACK_desc(s3ldesc, scdesc, data_type, address)
    S3L_array_t        *s3ldesc
    int                *scdesc
    S3L_data_type      data_type
    void               *address

F77/F90 Syntax

Example 7-50 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_from_ScaLAPACK_desc(s3ldesc, scdesc, data_type, address, ier)
    integer*8          s3ldesc
    integer*4          scdesc(*)
    integer*4          data_type
    pointer            address
    integer*4          ier

Input

S3L_from_ScaLAPACK_desc accepts the following arguments as input:

Output

S3L_from_ScaLAPACK_desc uses the following arguments for output:

Error Handling

On success, S3L_from_ScaLAPACK_desc returns S3L_SUCCESS.

S3L_from_ScaLAPACK_desc performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

S3L_to_ScaLAPACK_desc(3)

S3L_to_ScaLAPACK_desc

Description

S3L_to_ScaLAPACK_desc converts the S3L array handle specified by s3ldesc into a ScaLAPACK array descriptor and subgrid address, which are returned in scdesc and address, respectively.

The array referred to by s3ldesc must be two-dimensional; that is, it must be a rank 2 array.

Syntax

The C and Fortran syntax for S3L_to_ScaLAPACK_desc are shown below.

C/C++ Syntax

Example 7-51 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_to_ScaLAPACK_desc(s3ldesc, scdesc, data_type, address)
    S3L_array_t        *s3ldesc
    int                *scdesc
    int                data_type
    void               **address

F77/F90 Syntax

Example 7-52 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_to_ScaLAPACK_desc(s3ldesc, scdesc, data_type, address, ier)
    integer*8          s3ldesc
    integer*4          scdesc(*)
    integer*4          data_type
    pointer            address
    integer*4          ier

Input

S3L_to_ScaLAPACK_desc accepts the following arguments as input:

Output

S3L_to_ScaLAPACK_desc uses the following argument for output:

Error Handling

On success, S3L_to_ScaLAPACK_desc returns S3L_SUCCESS.

S3L_to_ScaLAPACK_desc performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

../examples/s3l/utils/scalapack_conv.c
../examples/s3l/utils-f/scalapack_conv.f

Related Functions

S3L_from_ScaLAPACK_desc(3)

Performing Miscellaneous S3L Control Functions

This section describes three toolkit functions that support the following tasks:

S3L_thread_comm_setup

Description

S3L_thread_comm_setup establishes the appropriate internal MPI communicators and data for thread-safe operation of S3L functions. It should be called from each thread in which S3L functions will be used.

Only S3L_init can be called before S3L_thread_comm_setup.

The argument comm specifies an MPI communicator, which should be congruent with, but not identical to, MPI_COMM_WORLD.

A unique communicator must be used for each thread or set of cooperating threads. The term "cooperating threads" refers to a set of threads that will be working on the same data. For example, one thread can initialize a random number generator, obtain a setup ID, and pass this to a fellow cooperating thread, which will then use the random number generator.

In such cases, the user must ensure that the threads within a cooperating set are properly synchronized.

A unique communicator is required because S3L performs internal communications. For example, when S3L_mat_mult is called from a multithreaded program, the thread on one node needs to communicate with the appropriate thread on another node. This can be done only if a communicator that is unique to these threads has been previously defined and passed to the communications routines within S3L.

S3L_thread_comm_setup need not be invoked if S3L functions are called from only one thread in the user's program.

Note -

S3L_thread_comm_setup is useful when a user performs explicit multithreading via threads library functions. Since threads library functions are not available in F77, the F77 interface for S3L_thread_comm_setup is not provided.

Syntax

The C and Fortran syntax for S3L_thread_comm_setup are shown below.

C/C++ Syntax

Example 7-53 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_thread_comm_setup(comm)
    MPI_Comm           comm

F77/F90 Syntax

Example 7-54 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_thread_comm_setup(comm, ier)
    integer*4          comm
    integer*4          ier

Input

S3L_thread_comm_setup accepts the following argument as input:

Output

S3L_thread_comm_setup uses the following argument for output:

Error Handling

On success, S3L_thread_comm_setup returns S3L_SUCCESS.

S3L_thread_comm_setup performs generic checking of the validity of the arrays it accepts as arguments. If an array argument contains an invalid or corrupted value, the function terminates and an error code indicating which value of the array handle was invalid is returned. See Appendix A of this manual for a detailed list of these error codes.

In addition, the following conditions will cause the function to terminate and return the associated error code:

Examples

See the following C and Fortran 77 program examples for illustrations of S3L_thread_comm_setup in use: 

../examples/s3l/dense_matrix_ops/inner_prod_mt.c
../examples/s3l/dense_matrix_ops/matmult_mt.c

Related Functions

The following may be of related interest.

MPI_Comm_dup(3)
S3L_set_safety(3)
threads(3T)
Also, "Multithreaded Programming" is a relevant section in the Sun MPI
User's Guide

S3L_set_safety

Description

The S3L safety mechanism offers two types of services:

The S3L safety mechanism can be set to operate at ony one of four levels, which are described in Table 7-2.

Table 7-2 S3L Safety Levels

Safety Level 

Description 

0

Turns the safety mechanism off. Explicit synchronization and errorchecking are not performed. This level is appropriate for productionruns of code that have already been thoroughly tested. 

2

Detects potential race conditions in multithreaded S3L operations on parallel arrays. To avoid race conditions, an S3L function locks all parallel array handles in its argument list before proceeding. This safety level causes warning messages to be generated if more than one S3L function attempts to use the same parallel array at the same time. 

5

In addition to checking for and reporting level 2 errors, performsexplicit synchronization before and after each call and locates eacherror with respect to the synchronization points. This safety level isappropriate during program development or during runs for which a small performance penalty can be tolerated. 

9

Checks for and reports all level 2 and level 5 errors, as well as errorsgenerated by any lower levels of code called from within S3L. Performs explicit synchronization in these lower levels of code and locates each error with respect to the synchronization points. This level is appropriate for detailed debugging following the occurrenceof a problem. 

The S3L safety mechanism can be controlled in either of two ways:

To set the S3L safety level using the S3L_SAFETY environment variable, issue the command:

setenv S3L_SAFETY
{ 0 | 2 | 5 | 9 }

Syntax

The C and Fortran syntax for S3L_set_safety are shown below.

C/C++ Syntax

Example 7-55 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_set_safety(n)
    int                n

F77/F90 Syntax

Example 7-56 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_set_safety(n, ier)
    integer*4          n
    integer*4          ier

Input

S3L_set_safety accepts the following argument as input:

Output

S3L_set_safety uses the following argument for output:

Error Handling

On success, S3L_set_safety returns S3L_SUCCESS.

On error, the following condition will cause the function to return the associated error code and terminate:

Examples

../examples/s3l/utils/copy_array.c
../examples/s3l/utils-f/copy_array.f

Related Functions

S3L_get_safety(3)

S3L_get_safety

Description

When S3L_get_safety is called from within an application, the value it returns indicates the current setting of the S3L safety mechanism. The possible return values are listed and their meaning explained in Table 7-2.

Syntax

The C and Fortran syntax for S3L_set_safety are shown below.

C/C++ Syntax

Example 7-57 

#include <s3l/s3l-c.h>
#include <s3l/s3l_errno-c.h>
int
S3L_get_safety()

F77/F90 Syntax

Example 7-58 

include `s3l/s3l-f.h'
include `s3l/s3l_errno-f.h'
subroutine
S3L_get_safety(ier)
    integer*4          ier

Input

S3L_get_safety takes no input arguments.

Output

S3L_get_safety returns the S3L safety level. When called by a Fortran program, it uses the following argument for output:

Examples

../examples/s3l/utils/copy_array.c
../examples/s3l/utils-f/copy_array.f

Related Functions

S3L_set_safety(3)