LWPs and Scheduling Classes

As mentioned in the "Scheduling" section of the "Chapter 1, Covering Multithreading Basics", the Solaris pthreads implementation supports only the SCHED_OTHER scheduling policy. The others are optional under POSIX.

The POSIX SCHED_FIFO and SCHED_RR policies can be duplicated or emulated using the standard Solaris mechanisms. These scheduling mechanisms are described in this section.

The Solaris kernel has three classes of scheduling. The highest-priority scheduling class is Realtime (RT). The middle-priority scheduling class is system. The system class cannot be applied to a user process. The lowest-priority scheduling class is timeshare (TS), which is also the default class.

Scheduling class is maintained for each LWP. When a process is created, the initial LWP inherits the scheduling class and priority of the creating LWP in the parent process. As more LWPs are created to run unbound threads, they also inherit this scheduling class and priority.

All unbound threads in a process have the same scheduling class and priority. Each scheduling class maps the priority of the LWP it is scheduling to an overall dispatching priority according to the configurable priority of the scheduling class.

Bound threads have the scheduling class and priority of their underlying LWPs. Each bound thread in a process can have a unique scheduling class and priority that is visible to the kernel. Bound threads are scheduled with respect to all other LWPs in the system.

Thread priorities regulate access to LWP resources. By default LWPs are in the timesharing class. For compute-bound multithreading, thread priorities are not very useful. For multithreaded applications that do a lot of synchronization using the MT libraries, thread priorities become more meaningful.

The scheduling class is set by priocntl(2). How you specify the first two arguments determines whether just the calling LWP or all the LWPs of one or more processes are affected. The third argument of priocntl() is the command, which can be one of the following.

• PC_GETCID. Get the class ID and class attributes for a specific class.

• PC_GETCLINFO. Get the class name and class attributes for a specific class.

• PC_GETPARMS. Get the class identifier and the class-specific scheduling parameters of a process, an LWP with a process, or a group of processes.

• PC_SETPARMS. Set the class identifier and the class-specific scheduling parameters of a process, an LWP with a process, or a group of processes.

Use priocntl() only on bound threads. To affect the priority of unbound threads, use pthread_setprio(3T).

Timeshare Scheduling

Timeshare scheduling distributes the processing resource fairly among the LWPs in this scheduling class. Other parts of the kernel can monopolize the processor for short intervals without degrading response time as seen by the user.

The priocntl(2) call sets the nice(2) level of one or more processes. The priocntl() call also affects the nice() level of all the timesharing class LWPs in the process. The nice() level ranges from 0 to +20 normally and from -20 to +20 for processes with superuser privilege. The lower the value, the higher the priority.

The dispatch priority of time shared LWPs is calculated from the instantaneous CPU use rate of the LWP and from its nice() level. The nice() level indicates the relative priority of the LWPs to the timeshare scheduler.

LWPs with a greater nice() value get a smaller, but nonzero, share of the total processing. An LWP that has received a larger amount of processing is given lower priority than one that has received little or no processing.

Realtime Scheduling

The Realtime class (RT) can be applied to a whole process or to one or more LWPs in a process. This requires superuser privilege.

Unlike the nice(2) level of the timeshare class, LWPs that are classified Realtime can be assigned priorities either individually or jointly. A priocntl(2) call affects the attributes of all the Realtime LWPs in the process.

The scheduler always dispatches the highest-priority Realtime LWP. It preempts a lower-priority LWP when a higher-priority LWP becomes runnable. A preempted LWP is placed at the head of its level queue.

A Realtime LWP retains control of a processor until it is preempted, it suspends, or its Realtime priority is changed. LWPs in the RT class have absolute priority over processes in the TS class.

A new LWP inherits the scheduling class of the parent process or LWP. An RT class LWP inherits the parent's time slice, whether finite or infinite.

An LWP with a finite time slice runs until it terminates, blocks (for example, to wait for an I/O event), is preempted by a higher-priority runnable Realtime process, or the time slice expires.

An LWP with an infinite time slice ceases execution only when it terminates, blocks, or is preempted.

LWP Scheduling and Thread Binding

The threads library automatically adjusts the number of LWPs in the pool used to run unbound threads. Its objectives are:

• To prevent the program from being blocked by a lack of unblocked LWPs.

  For example, if there are more runnable unbound threads than LWPs and all the active threads block in the kernel in indefinite waits (such as while reading a tty), the process cannot progress until a waiting thread returns.

• To make efficient use of LWPs.

  For example, if the library creates one LWP for each thread, many LWPs will usually be idle and the operating environment is overloaded by the resource requirements of the unused LWPs.

Keep in mind that LWPs are time sliced, not threads. This means that when there is only one LWP, there is no time slicing within the process--threads run on the LWP until they block (through interthread synchronization), are preempted, or terminate.

You can assign priorities to threads with pthread_setprio(3T); lower-priority unbound threads are assigned to LWPs only when no higher-priority unbound threads are available. Bound threads, of course, do not compete for LWPs because they have their own. Note that the thread priority that is set with pthread_setprio() regulates threads access to LWPs, not to CPUs.

Bind threads to your LWPs to get precise control over whatever is being scheduled. This control is not possible when many unbound threads compete for an LWP.

In particular, a lower-priority unbound thread could be on a higher-priority LWP and running on a CPU, while a higher-priority unbound thread assigned to a lower-priority LWP is not running. In this sense, thread priorities are just a hint about access to CPUs.

Realtime threads are useful for getting a quick response to external stimuli. Consider a thread used for mouse tracking that must respond instantly to mouse clicks. By binding the thread to an LWP, you guarantee that there is an LWP available when it is needed. By assigning the LWP to the Realtime scheduling class, you ensure that the LWP is scheduled quickly in response to mouse clicks.

SIGWAITING--Creating LWPs for Waiting Threads

The library usually ensures that there are enough LWPs in its pool for a program to proceed.

When all the LWPs in the process are blocked in indefinite waits (such as blocked reading from a tty or network), the operating environment sends the new signal, SIGWAITING, to the process. This signal is handled by the threads library. When the process contains a thread that is waiting to run, a new LWP is created and the appropriate waiting thread is assigned to it for execution.

The SIGWAITING mechanism does not ensure that an additional LWP is created when one or more threads are compute bound and another thread becomes runnable. A compute-bound thread can prevent multiple runnable threads from being started because of a shortage of LWPs.

This can be prevented by calling thr_setconcurrency(3THR). While using thr_setconcurrency() with POSIX threads is not POSIX compliant, its use is recommended to avoid LWP shortages for unbound threads in some computationally intensive situations. (The only way to be completely POSIX compliant and avoid LWP shortages is to create only PTHREAD_SCOPE_SYSTEM bound threads.)

See "Thread Concurrency (Solaris Threads Only)" for more information about using the thr_setconcurrency(3T) function.

In Solaris threads, you can also use THR_NEW_LWP in calls to thr_create(3THR) to create another LWP.

Aging LWPs

When the number of active threads is reduced, some of the LWPs in the pool are no longer needed. When there are more LWPs than active threads, the threads library destroys the unneeded LWPs. The library ages LWPs--they are deleted when they are unused for a "long" time; the default is five minutes.