2.3 Processor Binding

With processor binding, the programmer instructs the operating system that a thread in the program should run on the same processor throughout the execution of the program.

Processor binding, when used along with static scheduling, benefits applications that exhibit a certain data reuse pattern where data accessed by a thread in a parallel or worksharing region will be in the local cache from a previous invocation of a parallel or worksharing region.

From the hardware point of view, a computer system is composed of one or more physical processors. From the operating system point of view, each of these physical processors maps to one or more virtual processors onto which threads in a program can be run. If n virtual processors are available, then n threads can be scheduled to run at the same time. Depending on the system, a virtual processor may be a processor, a core, etc.

For example, the UltraSPARC T2 physical processor has eight cores, and each core can run eight simultaneous processing threads; from the Solaris OS point of view, there are 64 virtual processors onto which threads can be scheduled to run. On Solaris platforms, the number of virtual processors can be determined by using the psrinfo(1M) command. On Linux systems, the file /proc/cpuinfo provides information about available processors.

When the operating system binds threads to processors, they are in effect bound to specific virtual processors, not physical processors.

Set the SUNW_MP_PROCBIND environment variable to bind threads in an OpenMP program to specific virtual processors. The value specified for SUNW_MP_PROCBIND can be one of the following:

• The string "TRUE" or "FALSE" (or lower case "true" or "false"). For example, % setenv SUNW_MP_PROCBIND "false"

• A non-negative integer. For example, % setenv SUNW_MP_PROCBIND "2"

• A list of two or more non-negative integers separated by one or more spaces. For example, % setenv SUNW_MP_PROCBIND "0 2 4 6"

• Two non-negative integers, n1 and n2, separated by a minus ("-"); n1 must be less than or equal to n2. For example, % setenv SUNW_MP_PROCBIND "0-6"

Interpretation of the values accepted by SUNW_MP_PROCBIND appears in 2.3.3 Interpreting the Value Specified for SUNW_MP_PROCBIND

Note that the non-negative integers referred to above denote logical identifiers (IDs). Logical IDs may be different from virtual processor IDs. The difference will be explained below.

2.3.1 Virtual Processor IDs

Each virtual processor in a system has a unique processor ID. You can use the Solaris OS psrinfo(1M) command to display information about the processors in a system, including their processor IDs. Moreover, you can use the prtdiag(1M) command to display system configuration and diagnostic information.

You can use psrinfo -pv to list all physical processors in the system and the virtual processors that are associated with each physical processor.

Virtual processor IDs may be sequential or there may be gaps in the IDs. For example, on a Sun Fire 4810 with 8 UltraSPARC IV processors (16 cores), the virtual processor IDs may be: 0, 1, 2, 3, 8, 9, 10, 11, 512, 513, 514, 515, 520, 521, 522, 523.

2.3.2 Logical IDs

As mentioned above, the non-negative integers specified for SUNW_MP_PROCBIND are logical IDs. Logical IDs are consecutive integers that start with 0. If the number of virtual processors available in the system is n, then their logical IDs are 0, 1, ..., n-1, in the order presented by psrinfo(1M). The following Korn shell script can be used to display the mapping from virtual processor IDs to logical IDs.

#!/bin/ksh

NUMV= `psrinfo | fgrep "on-line" | wc -l `
set -A VID  `psrinfo | cut -f1 `

echo "Total number of on-line virtual processors = $NUMV"
echo

let "I=0"
let "J=0"
while [[ $I -lt $NUMV ]]
do
  echo "Virtual processor ID ${VID[I]} maps to logical ID ${J}"
  let "I=I+1"
  let "J=J+1"
done

On systems where a single physical processor maps to several virtual processors, it may be useful to know which logical IDs correspond to virtual processors that belong to the same physical processor. The following Korn shell script can be used with later Solaris releases to display this information.

#!/bin/ksh

NUMV= `psrinfo | grep "on-line" | wc -l `
set -A VLIST  `psrinfo | cut -f1 `
set -A CHECKLIST  `psrinfo | cut -f1 `

let "I=0"

while [ $I -lt $NUMV ]
do
  let "COUNT=0"
  SAMELIST="$I"

  let "J=I+1"

  while [ $J -lt $NUMV ]
  do
    if [ ${CHECKLIST[J]} -ne -1 ]
    then
      if [  `psrinfo -p ${VLIST[I]} ${VLIST[J]} ` = 1 ]
      then
    SAMELIST="$SAMELIST $J"
    let "CHECKLIST[J]=-1"
    let "COUNT=COUNT+1"
      fi
    fi
    let "J=J+1"
  done

  if [ $COUNT -gt 0 ]
  then
    echo "The following logical IDs belong to the same physical processor:"
    echo "$SAMELIST"
    echo " "
  fi

  let "I=I+1"
done

2.3.3 Interpreting the Value Specified for SUNW_MP_PROCBIND

If the value specified for SUNW_MP_PROCBIND is TRUE, then the threads will be bound to virtual processors in a round-robin fashion. The starting processor for the binding is determined by the runtime library with the goal of achieving best performance.

If the value specified for SUNW_MP_PROCBIND is FALSE, the threads will not be bound to any processors. This is the default setting.

If the value specified for SUNW_MP_PROCBIND is a non-negative integer, then that integer denotes the starting logical ID of the virtual processor to which threads should be bound. Threads will be bound to virtual processors in a round-robin fashion, starting with the processor with the specified logical ID, and wrapping around to the processor with logical ID 0, after binding to the processor with logical ID n-1.

If the value specified for SUNW_MP_PROCBIND is a list of two or more non-negative integers, then threads will be bound in a round-robin fashion to virtual processors with the specified logical IDs. Processors with logical IDs other than those specified will not be used.

If the value specified for SUNW_MP_PROCBIND is two non-negative integers separated by a minus ("-"), then threads will be bound in a round-robin fashion to virtual processors in the range that begins with the first logical ID and ends with the second logical ID. Processors with logical IDs other than those included in the range will not be used.

If the value specified for SUNW_MP_PROCBIND does not conform to one of the forms described above, or if an invalid logical ID is given, then an error message will be emitted and execution of the program will terminate.

Note that the number of threads created by the OpenMP runtime library, libmtsk, depends on environment variables, API calls in the user’s program, and the num_threads clause. SUNW_MP_PROCBIND specifies the logical IDs of virtual processors to which the threads should be bound. Threads will be bound to that set of processors in a round-robin fashion. If the number of threads used in the program is less than the number of logical IDs specified by SUNW_MP_PROCBIND, then some virtual processors will not be used by the program. If the number of threads is greater than the number of logical IDs specified by SUNW_MP_PROCBIND, them some virtual processors will have more than one thread bound to them.

2.3.4 Interaction with OS Processor Sets

A processor set can be specified using the psrset utility on Solaris platforms, or the taskset command on Linux platforms. SUNW_MP_PROCBIND does not take processor sets into account. If the programmer uses processor sets, then it is their responsibility to ensure that the setting of SUNW_MP_PROCBIND is consistent with the processor set used. Otherwise, the setting of SUNW_MP_PROCBIND will override the processor set setting on Linux systems, while on Solaris systems an error message will be issued.