OpenMP is a portable, parallel programming model for shared memory multiprocessor architectures, developed in collaboration with a number of computer vendors. The specifications were created and are published by the OpenMP Architecture Review Board. For more information on the OpenMP developer community, including tutorials and other resources, see their web site at:
http://www.openmp.org

The OpenMP API is the recommended parallel programming model for all Forte Developer compilers. See Chapter 4 for guidelines on converting legacy Fortran and C parallelization directives to OpenMP.

This chapter summarizes the directives, run-time library routines, and environment variables comprising the OpenMP Application Program Interfaces, as implemented by the Forte Developer Fortran 95, C and C++ compilers.

---

1.1 Where to Find the OpenMP Specifications

The material presented in this chapter is only a summary with many details left out intentionally for the sake of brevity. In all cases, refer to the OpenMP specification documents for complete details.

The Fortran 2.0 and C/C++ 1.0 OpenMP specifications can be found on the official OpenMP website, http://www.openmp.org/, and are hyper linked to the Forte Developer documentation index installed with the software, at:

file:/opt/SUNWspro/docs/index.html

---

1.2 Special Conventions Used Here

In the tables and examples that follow, Fortran directives and source code are shown in upper case, but are case-insensitive.

The term structured-block refers to a block of Fortran or C/C++ statements having no transfers into or out from the block.

Constructs within square brackets, [...], are optional.

Throughout this manual, "Fortran" refers to the Fortran 95 language and compiler, f95.

The terms "directive" and "pragma" are used interchangeably in this manual.

---

1.3 Directive Formats

Only one directive-name can be specified on a directive line.

Fortran:

Fortran fixed format accepts three directive "sentinels", free format only one. In the Fortran examples that follow, free format will be used.

C/C++:

C and C++ use the standard preprocessing directive starting with #pragma omp.

---

1.4 Conditional Compilation

The OpenMP API defines the preprocessor symbol _OPENMP to be used for conditional compilation. In addition, OpenMP Fortran API accepts a conditional compilation sentinel.

---

1.5 PARALLEL - Parallel Region Construct

The PARALLEL directive defines a parallel region, which is a region of the program that is to be executed by multiple threads in parallel.

TABLE 1-1 identifies the clauses that can appear with this construct.

---

1.6 Work-Sharing Constructs

Work-sharing constructs divide the execution of the enclosed code region among the members of the team of threads that encounter it. Work sharing constructs must be enclosed within a parallel region for the construct to execute in parallel.

There are many special conditions and restrictions on these directives and the code they apply to. Programmers are urged to refer to the appropriate OpenMP specification document for the details.

1.6.1 DO and for

Specifies that the iterations of the DO or for loop that follows must be executed in parallel.

1.6.2 SECTIONS

SECTIONS encloses a non-iterative block of code to be divided among threads in the team. Each block is executed once by a thread in the team.

Each section is preceded by a SECTION directive, which is optional for the first section.

1.6.3 SINGLE

The structured block enclosed by SINGLE is executed by only one thread in the team. Threads in the team that are not executing the SINGLE block wait at the end of the block unless NOWAIT is specified.

1.6.4 Fortran WORKSHARE

Divides the work of executing the enclosed code block into separate units of work, and causes the threads of the team to share the work such that each unit is executed only once.

There is no C/C++ equivalent to the Fortran WORKSHARE construct.

TABLE 1-1 identifies the clauses that can appear with these constructs.

---

1.7 Combined Parallel Work-sharing Constructs

The combined parallel work-sharing constructs are shortcuts for specifying a parallel region that contains one work-sharing construct.

There are many special conditions and restrictions on these directives and the code they apply to. Programmers are urged to refer to the appropriate OpenMP specification document for the details.

TABLE 1-1 identifies the clauses that can appear with these constructs.

1.7.1 PARALLEL DO and parallel for

Shortcut for specifying a parallel region that contains a single DO or for loop. Equivalent to a PARALLEL directive followed immediately by a DO or for directive. clause can be any of the clauses accepted by the PARALLEL and DO/for directives, except the NOWAIT modifier.

1.7.2 PARALLEL SECTIONS

Shortcut for specifying a parallel region that contains a single SECTIONS directive. Equivalent to a PARALLEL directive followed by a SECTIONS directive. clause can be any of the clauses accepted by the PARALLEL and SECTIONS directives, except the NOWAIT modifier.

1.7.3 PARALLEL WORKSHARE

Provides a shortcut for specifying a parallel region that contains a single WORKSHARE directive. clause can be one of the clauses accepted by either the PARALLEL or WORKSHARE directive.

There is no C/C++ equivalent.

---

1.8 Synchronization Constructs

The following constructs specify thread synchronization. There are many special conditions and restrictions regarding these constructs that are too numerous to summarize here. Programmers are urged to refer to the appropriate OpenMP specification document for the details.

1.8.1 MASTER

Only the master thread of the team executes the block enclosed by this directive. The other threads skip this block and continue. There is no implied barrier on entry to or exit from the master section.

1.8.2 CRITICAL

Restrict access to the structured block to only one thread at a time. The optional name argument identifies the critical region. All unnamed CRITICAL directives map to the same name. Critical section names are global entities of the program and must be unique. For Fortran, if name appears on the CRITICAL directive, it must also appear on the END CRITICAL directive. For C/C++, the identifier used to name a critical region has external linkage and is in a name space which is separate from the name spaces used by labels, tags, members, and ordinary identifiers.

1.8.3 BARRIER

Synchronizes all the threads in a team. Each thread waits until all the others in the team have reached this point.

1.8.4 ATOMIC

Ensures that a specific memory location is to be updated atomically, rather than exposing it to the possibility of multiple, simultaneous writing threads.

This implementation replaces all ATOMIC directives by enclosing the expression-statement in a critical section.

1.8.5 FLUSH

Thread-visible Fortran variables or C objects are written back to memory at the point at which this directive appears. The FLUSH directive only provides consistency between operations within the executing thread and global memory. The optional list consists of a comma-separated list of variables or objects that need to be flushed. A flush directive without a list synchronizes all thread-visible shared variables or objects.

1.8.6 ORDERED

The enclosed block is executed in the order that iterations would be executed in a sequential execution of the loop.

---

1.9 Data Environment Directives

The following directives control the data environment during execution of parallel constructs.

1.9.1 THREADPRIVATE

Makes the list of objects (Fortran common blocks and named variables, C named variables) private to a thread but global within the thread.

See the OpenMP specifications (section 2.6.1 in the Fortran 2.0 specifications, secton 2.7.1 in the C/C++) for the complete details and restrictions.

---

1.10 OpenMP Directive Clauses

This section summarizes the data scoping and scheduling clauses that can appear on OpenMP directives.

1.10.1 Data Scoping Clauses

Several directives accept clauses that allow a user to control the scope attributes of variables within the extent of the construct. If no data scope clause is specified for a directive, the default scope for variables affected by the directive is SHARED.

Fortran: list is a comma-separated list of named variables or common blocks that are accessible in the scoping unit. Common block names must appear within slashes (for example, /ABLOCK/).

There are important restrictions on the use of these scoping clauses. Refer to section 2.6.2 in the Fortran 2.0 specification, and section 2.7.2 in the C/C++ specification for complete details.

TABLE 1-1 identifies the directives on which these clauses can appear.

1.10.1.1 PRIVATE

private(list)

Declares the variables in the comma separated list to be private to each thread in a team.

1.10.1.2 SHARED

shared(list)

All the threads in the team share the variables that appear in list, and access the same storage area.

1.10.1.3 DEFAULT

Fortran

DEFAULT(PRIVATE | SHARED | NONE)

C/C++

default(shared | none)

Specify scoping attribute for all variables within a parallel region. THREADPRIVATE variables are not affected by this clause. If not specified, DEFAULT(SHARED) is assumed.

1.10.1.4 FIRSTPRIVATE

firstprivate(list)

Variables on list are PRIVATE. In addition, private copies of the variables are initialized from the original object existing before the construct.

1.10.1.5 LASTPRIVATE

lastprivate(list)

Variables on the list are PRIVATE. In addition, when the LASTPRIVATE clause appears on a DO or for directive, the thread that executes the sequentially last iteration updates the version of the variable before the construct. On a SECTIONS directive, the thread that executes the lexically last SECTION updates the version of the object it had before the construct.

1.10.1.6 COPYIN

Fortran

COPYIN(list)

The COPYIN clause applies only to variables, common blocks, and variables in common blocks that are declared as THREADPRIVATE. In a parallel region, COPYIN specifies that the data in the master thread of the team be copied to the thread private copies of the common block at the beginning of the parallel region.

C/C++

copyin(list)

The COPYIN clause applies only to variables that are declared as THREADPRIVATE. In a parallel region, COPYIN specifies that the data in the master thread of the team be copied to the thread private copies at the beginning of the parallel region.

1.10.1.7 COPYPRIVATE

Fortran

COPYPRIVATE(list)

Uses a private variable to broadcast a value, or a pointer to a shared object, from one member of a team to the other members. Variables in list must not appear in a PRIVATE or FIRSTPRIVATE clause of the SINGLE construct specifying COPYPRIVATE.

There is no C/C++ equivalent.

1.10.1.8 REDUCTION

Fortran

REDUCTION(operator|intrinsic:list)

operator is one of: +, *, -, .AND., .OR., .EQV., .NEQV.

intrinsic is one of: MAX, MIN, IAND, IOR, IEOR

Variables in list must be named variables of intrinsic type.

C/C++

reduction(operator:list)

operator is one of: +, *, -, &, ^, |, &&, ||

The REDUCTION clause is intended to be used on a region in which the reduction variable is used only in reduction statements. Variables on list must be SHARED in the enclosing context. A private copy of each variable is created for each thread as if it were PRIVATE. At the end of the reduction, the shared variable is updated by combining the original value with the final value of each of the private copies.

1.10.2 Scheduling Clauses

The SCHEDULE clause specifies how iterations in a Fortran DO loop or C/C++ for loop are divided among the threads in a team. TABLE 1-1 shows which directives allow the SCHEDULE clause.

There are important restrictions on the use of these scheduling clauses. Refer to section 2.3.1 in the Fortran 2.0 specification, and section 2.4.1 in the C/C++ specification for complete details.

schedule(type [,chunk])

Specifies how iterations of the DO or for loop are divided among the threads of the team. type can be one of STATIC, DYNAMIC, GUIDED, or RUNTIME. In the absence of a SCHEDULE clause, STATIC scheduling is used. chunk must be an integer expression.

1.10.2.1 STATIC Scheduling

schedule(static[,chunk])

Iterations are divided into pieces of a size specified by chunk. The pieces are statically assigned to threads in the team in a round-robin fashion in the order of the thread number. If not specified, chunk is chosen to divide the iterations into contiguous chunks nearly equal in size with one chunk assigned to each thread.

1.10.2.2 DYNAMIC Scheduling

schedule(dynamic[,chunk])

Iterations are broken into pieces of a size specified by chunk. As each thread finishes a piece of the iteration space, it dynamically obtains the next set of iterations. When no chunk is specified, it defaults to 1.

1.10.2.3 GUIDED Scheduling

schedule(guided[,chunk])

With GUIDED, the chunk size is reduced in an exponentially decreasing manner with each dispatched piece of the iterations. chunk specifies the minimum number of iterations to dispatch each time. (The size of the initial chunk of the iterations is implementation dependent; see Chapter 2.). When no chunk is specified, it defaults to 1.

1.10.2.4 RUNTIME Scheduling

schedule(runtime)

Scheduling is deferred until runtime. Schedule type and chunk size will be determined from the setting of the OMP_SCHEDULE environment variable. (Default is SCHEDULE(STATIC)

1.10.3 NUM_THREADS Clause

The Fortran OpenMP API provides a NUM_THREADS clause on the PARALLEL, PARALLEL SECTIONS, PARALLEL DO, and PARALLEL WORKSHARE directives.

There is no C/C++ equivalent.

1.10.4 Placement of Clauses on Directives

TABLE 1-1 shows the clauses that can appear on these directives and pragmas:

IF

PRIVATE

SHARED

FIRSTPRIVATE

LASTPRIVATE

DEFAULT

REDUCTION

COPYIN

COPYPRIVATE

ORDERED

SCHEDULE

NOWAIT

NUM_THREADS