This chapter describes how to compile and debug multithreaded programs.
Many options are available for header files, define flags, and linking.
The following items are required to compile and link a multithreaded program. Except for the C compiler, all should come with your Solaris operating environment.
A standard C compiler
Include files:
The Solaris threads library (libthread), the POSIX threads library (libpthread), and possibly the POSIX realtime library (librt) for semaphores
MT-safe libraries (libc, libm, libw, libintl, libnsl, libsocket, libmalloc, libmapmalloc, and so on)
Certain functions, including the ones listed in Table 7-1, have different semantics in the POSIX 1003.1c standard than in the Solaris 2.4 Operating Environment release, which was based on an earlier POSIX draft. Function definitions are chosen at compile time. See the man Pages(3): Library Routines for a description of the differences in expected parameters and return values.
Table 7-1 Functions With POSIX/Solaris Semantic Differences
sigwait(2) |
|
ctime_r(3C) |
asctime_r(3C) |
ftrylockfile(3S) - new |
getlogin_r(3C) |
getgrnam_r(3C) |
getgrgid_r(3C) |
getpwnam_r(3C) |
getpwuid_r(3C) |
readdir_r(3C) |
ttyname_r(3C) |
The Solaris fork(2) function duplicates all threads (fork-all behavior), while the POSIX fork(2) function duplicates only the calling thread (fork-one behavior), as does the Solaris fork1() function.
The handling of an alarm(2) is also different: a Solaris alarm goes to the thread's LWP, while a POSIX alarm goes to the whole process (see "Per-Thread Alarms").
The include file <thread.h>, used with the -lthread library, compiles code that is upward compatible with earlier releases of the Solaris Operating Environment. This library contains both interfaces--those with Solaris semantics and those with POSIX semantics. To call thr_setconcurrency(3T) with POSIX threads, your program needs to include <thread.h>.
The include file <pthread.h>, used with the
-lpthread library, compiles code that is conformant with
the multithreading interfaces defined by the POSIX 1003.1c standard. For complete
POSIX compliance, the define flag _POSIX_C_SOURCE should
be set to a (long
) value >= 199506:
cc [flags] file... -D_POSIX_C_SOURCE=N (where N 199506L)
You can mix Solaris threads and POSIX threads in the same application, by including both <thread.h> and <pthread.h>, and linking with either the -lthread or -lpthread library.
In mixed use, Solaris semantics prevail when compiling with -D_REENTRANT and linking with -lthread, whereas POSIX semantics prevail when compiling with -D_POSIX_C_SOURCE and linking with -lpthread.
For POSIX behavior, compile applications with the -D_POSIX_C_SOURCE flag set >= 199506L. For Solaris behavior, compile multithreaded programs with the -D_REENTRANT flag. This applies to every module of an application.
For mixed applications (for example, Solaris threads with POSIX semantics), compile with the -D_REENTRANT and -D_POSIX_PTHREAD_SEMANTICS flags.
To compile a single-threaded application, define neither the -D_REENTRANT nor the -D_POSIX_C_SOURCE flag. When these flags are not present, all the old definitions for errno, stdio, and so on, remain in effect.
Compile single-threaded applications, not linked with either of the thread libraries (libthread.so.1 or libpthread.so.1), without the -D_REENTRANT flag. This eliminates performance degradation incurred when macros, such as putc(3s), are converted into reentrant function calls.
To summarize, POSIX applications that define -D_POSIX_C_SOURCE get the POSIX 1003.1c semantics for the routines listed in Table 7-1. Applications that define only -D_REENTRANT get the Solaris semantics for these routines. Solaris applications that define -D_POSIX_PTHREAD_SEMANTICS get the POSIX semantics for these routines, but can still use the Solaris threads interface.
Applications that define both -D_POSIX_C_SOURCE and -D_REENTRANT get the POSIX semantics.
For POSIX threads behavior, load the libpthread library. For Solaris threads behavior, load the libthread library. Some POSIX programmers might want to link with -lthread to preserve the Solaris distinction between fork() and fork1(). All that -lpthread really does is to make fork() behave the same way as the Solaris fork1() call, and change the behavior of alarm(2).
To use libthread, specify -lthread before -lc on the ld command line, or last on the cc command line.
To use libpthread, specify -lpthread before -lc on the ld command line, or last on the cc command line.
Do not link a nonthreaded program with -lthread or -lpthread. Doing so establishes multithreading mechanisms at link time that are initiated at runtime. These slow down a single-threaded application, waste system resources, and produce misleading results when you debug your code.
Figure 7-1 summarizes the compile options.
In mixed usage, you need to include both thread.h and pthread.h.
All calls to libthread and libpthread are no-ops if the application does not link -lthread or -lpthread. The runtime library libc has many predefined libthread and libpthread stubs that are null procedures. True procedures are interposed by libthread or libpthread when the application links both libc and the thread library.
The behavior of the C library is undefined if a program is constructed with an ld command line that includes the following incorrect fragment:
.o's ... -lc -lthread ... (this is incorrect) or .o's ... -lc -lpthread ... (this is incorrect)
For C++ programs that use threads, use the -mt option, rather than -lthread, to compile and link your application. The -mt option links with libthread and ensures proper library linking order. Using -lthread might cause your program to core dump.
The Solaris semaphore routines, sema_*(3T), are contained in the libthread library. By contrast, you link with the -lposix4 library to get the standard sem_*(3R) POSIX 1003.1c semaphore routines described in "Semaphores".
Table 7-2 shows that multithreaded object modules should be linked with old object modules only with great caution.
Table 7-2 Compiling With and Without the _REENTRANT Flag
The standard Solaris threads implementation is built upon a two-level threading model in which user-level threads are multiplexed over possibly fewer lightweight processes, or LWPs. An LWP is the fundamental unit of execution that is dispatched to a processor by the operating system. A mechanism is provided in the standard implementation for associating threads one-to-one with LWPs (the THR_BOUND and PTHREAD_SCOPE_SYSTEM flags) for writing applications with one-level semantics.
The Solaris 8 Operating Environment provides an alternate threads implementation of a one-level model in which user-level threads are associated one-to-one with LWPs. This implementation is simpler than the standard implementation and may be beneficial to some multithreaded applications. It provides exactly the same interfaces, both for POSIX threads and Solaris threads, as the standard implementation.
To link with the alternate implementation, use the following run-path -R option when linking the program.
For POSIX threads use:
cc -mt ... -lpthread ... -R /usr/lib/lwp (32-bit) cc -mt ... -lpthread ... -R /usr/lib/lwp/64 (64-bit) |
For Solaris threads use:
cc -mt ... -R /usr/lib/lwp (32-bit) cc -mt ... -R /usr/lib/lwp/64 (64-bit) |
For multithreaded programs that have been previously linked with the standard threads library, the environment variable LD_LIBRARY_PATH and LD_LIBRARY_PATH_64 can be set as follows to bind the program at run time to the alternate threads library:
LD_LIBRARY_PATH=/usr/lib/lwp LD_LIBRARY_PATH=/usr/lib/lwp:/usr/lib/lwp/64 |
Note that if an LD_LIBRARY_PATH environment variable is in effect for a secure process, then only the trusted directories specified by this variable will be used to augment the runtime linker's search rules.
When using the alternate one-level threads implementation, the library might create more LWPs than the standard implementation using unbound threads. LWPs consume operating system memory in contrast to threads, which consume only user-level memory. Thus, a multithreaded application linked with this library that creates thousands of threads would create an equal number of LWPs and might run the system out of resources required to support the application.
The following list points out some of the more frequent oversights that can cause bugs in multithreaded programs.
Accessing global memory (shared changeable state) without the protection of a synchronization mechanism.
Creating deadlocks caused by two threads trying to acquire rights to the same pair of global resources in alternate order (so that one thread controls the first resource and the other controls the second resource and neither can proceed until the other gives up).
Creating a hidden gap in synchronization protection. This is caused when a code segment protected by a synchronization mechanism contains a call to a function that frees and then reacquires the synchronization mechanism before it returns to the caller. The result is that it appears to the caller that the global data has been protected when it actually has not.
Mixing UNIX signals with threads--it is better to use the sigwait(2) model for handling asynchronous signals.
Failing to reevaluate the conditions after returning from a call to *_cond_wait(3T) or *_cond_timedwait(3T).
Forgetting that default threads are created PTHREAD_CREATE_JOINABLE and must be reclaimed with pthread_join(3T); note, pthread_exit(3T) does not free up its storage space.
Making deeply nested, recursive calls and using large automatic arrays can cause problems because multithreaded programs have a more limited stack size than single-threaded programs.
And, note that multithreaded programs (especially those containing bugs) often behave differently in two successive runs, given identical inputs, because of differences in the thread scheduling order.
In general, multithreading bugs are statistical instead of deterministic. Tracing is usually a more effective method of finding order of execution problems than is breakpoint-based debugging.
Use the TNF utilities (included as part of the Solaris system) to trace, debug, and gather performance analysis information from your applications and libraries. The TNF utilities integrate trace information from the kernel and from multiple user processes and threads, and so are especially useful for multithreaded code.
With the TNF utilities, you can easily trace and debug multithreaded programs. See the TNF utilities chapter in the Programming Utilities Guide for detailed information on using prex(1), tnfdump(1), and other TNF utilities.
See truss(1) for information on tracing system calls and signals.
When you bind all threads in a multithreaded program, a thread and an LWP are synonymous. Then you can access each thread with the following adb commands that support multithreaded programming.
Table 7-3 MT adb Commands
pid:A |
Attaches to process # pid. This stops the process and all its LWPs. |
:R |
Detaches from process. This resumes the process and all its LWPs. |
$L |
Lists all active LWPs in the (stopped) process. |
n:l |
Switches focus to LWP # n. |
$l |
Shows the LWP currently focused. |
num:i |
Ignores signal number num. |
These commands to set conditional breakpoints are often useful.
Table 7-4 Setting adb Breakpoints
[label],[count]:b [expression] |
Breakpoint is detected when expression equals zero |
foo,ffff:b <g7-0xabcdef |
Stop at foo when g7 = the hex value 0xABCDEF |
With the dbx utility you can debug and execute source programs written in C++, ANSI C, and FORTRAN. dbx accepts the same commands as the Debugger, but uses a standard terminal (TTY) interface. Both dbx and the Debugger support debugging multithreaded programs. For a full overview of dbx and Debugger features see the dbx(1) reference manual page and the Using Sun Workshop user's guide.
All the dbx options listed in Table 7-5 can support multithreaded applications.
Table 7-5 dbx Options for MT Programs