This chapter describes how to compile and debug multithreaded programs.
Many options are available for header files, define flags, and linking.
The following items are required to compile and link a multithreaded program. Except for the C compiler, all should come with your Solaris operating environment.
A standard C compiler
Include files:
The Solaris threads library (libthread), the POSIX threads library (libpthread), and possibly the POSIX realtime library (librt) for semaphores
MT-safe libraries (libc, libm, libw, libintl, libnsl, libsocket, libmalloc, libmapmalloc, and so on)
Certain functions, including the ones listed in Table 7–1, have different semantics in the POSIX 1003.1c standard than in the Solaris 2.4 Operating Environment release, which was based on an earlier POSIX draft. Function definitions are chosen at compile time. See the man Pages(3): Library Routines for a description of the differences in expected parameters and return values.
Table 7–1 Functions With POSIX/Solaris Semantic Differences
sigwait(2) |
|
ctime_r(3C) |
asctime_r(3C) |
ftrylockfile(3S) - new |
getlogin_r(3C) |
getgrnam_r(3C) |
getgrgid_r(3C) |
getpwnam_r(3C) |
getpwuid_r(3C) |
readdir_r(3C) |
ttyname_r(3C) |
The Solaris fork(2) function duplicates all threads (fork-all behavior), while the POSIX fork(2) function duplicates only the calling thread (fork-one behavior), as does the Solaris fork1() function.
The include file <thread.h>, used with the -lthread library, compiles code that is upward compatible with earlier releases of the Solaris Operating Environment. This library contains both interfaces—those with Solaris semantics and those with POSIX semantics. To call thr_setconcurrency(3THR) with POSIX threads, your program needs to include <thread.h>.
The include file <pthread.h>, used with the
-lpthread library, compiles code that is conformant with
the multithreading interfaces defined by the POSIX 1003.1c standard. For complete
POSIX compliance, the define flag _POSIX_C_SOURCE should
be set to a (long
) value ≥ 199506:
cc [flags] file... -D_POSIX_C_SOURCE=N (where N 199506L)
You can mix Solaris threads and POSIX threads in the same application, by including both <thread.h> and <pthread.h>, and linking with either the -lthread or -lpthread library.
In mixed use, Solaris semantics prevail when compiling with -D_REENTRANT and linking with -lthread, whereas POSIX semantics prevail when compiling with -D_POSIX_C_SOURCE and linking with -lpthread.
For POSIX behavior, compile applications with the -D_POSIX_C_SOURCE flag set ≥ 199506L. For Solaris behavior, compile multithreaded programs with the -D_REENTRANT flag. This applies to every module of an application. .
For mixed applications (for example, Solaris threads with POSIX semantics), compile with the -D_REENTRANT and -D_POSIX_PTHREAD_SEMANTICS flags.
To compile a single-threaded application, define neither the -D_REENTRANT nor the -D_POSIX_C_SOURCE flag. When these flags are not present, all the old definitions for errno, stdio, and so on, remain in effect.
Compile single-threaded applications, not linked with either of the thread libraries (libthread.so.1 or libpthread.so.1), without the -D_REENTRANT flag. This eliminates performance degradation incurred when macros, such as putc(3s), are converted into reentrant function calls.
To summarize, POSIX applications that define -D_POSIX_C_SOURCE get the POSIX 1003.1c semantics for the routines listed in Table 7–1. Applications that define only -D_REENTRANT get the Solaris semantics for these routines. Solaris applications that define -D_POSIX_PTHREAD_SEMANTICS get the POSIX semantics for these routines, but can still use the Solaris threads interface.
Applications that define both -D_POSIX_C_SOURCE and -D_REENTRANT get the POSIX semantics.
For POSIX threads behavior, load the libpthread library. For Solaris threads behavior, load the libthread library. Some POSIX programmers might want to link with -lthread to preserve the Solaris distinction between fork() and fork1(). All that -lpthread really does is to make fork() behave the same way as the Solaris fork1() call.
To use libthread, specify -lthread before -lc on the ld command line, or last on the cc command line.
To use libpthread, specify -lpthread before -lc on the ld command line, or last on the cc command line.
Prior to Solaris 9, you should not link a nonthreaded program with -lthread or -lpthread. Doing so establishes multithreading mechanisms at link time that are initiated at runtime. These slow down a single-threaded application, waste system resources, and produce misleading results when you debug your code.
In Solaris 9 and subsequent releases, linking a nonthreaded program with -lthread or -lpthread makes no semantic difference to the program. No extra threads or LWPs are created and the main (and only) thread executes as a traditional single-threaded process. The only effect on the program is to make system library locks become real locks (as opposed to dummy function calls) and you pay the price of acquiring uncontended locks.
Figure 7–1 summarizes the compile options.
In mixed usage, you need to include both thread.h and pthread.h.
All calls to libthread and libpthread are no-ops if the application does not link -lthread or -lpthread. The runtime library libc has many predefined libthread and libpthread stubs that are null procedures. True procedures are interposed by libthread or libpthread when the application links both libc and the thread library.
For C++ programs that use threads, use the -mt option, rather than -lthread, to compile and link your application. The -mt option links with libthread and ensures proper library linking order. Using -lthread might cause your program to core dump.
The Solaris semaphore routines, sema_*(3THR), are contained in the libthread library. By contrast, you link with the -lrt library to get the standard sem_*(3R) POSIX 1003.1c semaphore routines described in Semaphores.
Table 7–2 shows that multithreaded object modules should be linked with old object modules only with great caution.
Table 7–2 Compiling With and Without the _REENTRANT Flag
The Solaris 8 Operating Environment introduced an alternate threads library implementation, located in the directories /usr/lib/lwp (32-bit) and /usr/lib/lwp/64 (64-bit). In the Solaris 9 Operating Environment, this implementation is the standard threads implementation found in /usr/lib and /usr/lib/64.
The following list points out some of the more frequent oversights that can cause bugs in multithreaded programs.
Accessing global memory (shared changeable state) without the protection of a synchronization mechanism.
Creating deadlocks caused by two threads trying to acquire rights to the same pair of global resources in alternate order (so that one thread controls the first resource and the other controls the second resource and neither can proceed until the other gives up).
Creating a hidden gap in synchronization protection. This is caused when a code segment protected by a synchronization mechanism contains a call to a function that frees and then reacquires the synchronization mechanism before it returns to the caller. The result is that it appears to the caller that the global data has been protected when it actually has not.
Mixing UNIX signals with threads—it is better to use the sigwait(2) model for handling asynchronous signals.
Failing to reevaluate the conditions after returning from a call to *_cond_wait() or *_cond_timedwait().
Forgetting that default threads are created PTHREAD_CREATE_JOINABLE and must be reclaimed with pthread_join(3THR); note, pthread_exit(3THR) does not free up its storage space.
Making deeply nested, recursive calls and using large automatic arrays can cause problems because multithreaded programs have a more limited stack size than single-threaded programs.
And, note that multithreaded programs (especially those containing bugs) often behave differently in two successive runs, given identical inputs, because of differences in the thread scheduling order.
In general, multithreading bugs are statistical instead of deterministic. Tracing is usually a more effective method of finding order of execution problems than is breakpoint-based debugging.
Use the TNF utilities (included as part of the Solaris system) to trace, debug, and gather performance analysis information from your applications and libraries. The TNF utilities integrate trace information from the kernel and from multiple user processes and threads, and so are especially useful for multithreaded code.
With the TNF utilities, you can easily trace and debug multithreaded programs. See the TNF manual pages for detailed information on using prex(1) and tnfdump(1).
See truss(1) for information on tracing system calls, signals and user-level function calls.
The following mdb commands can be used to access the LWPs of a multithreaded program.
Table 7–3 MT mdb Commands
pid:A |
Attaches to process # pid. This stops the process and all its LWPs. |
:R |
Detaches from process. This resumes the process and all its LWPs. |
$L |
Lists all active LWPs in the (stopped) process. |
n:l |
Switches focus to LWP # n. |
$l |
Shows the LWP currently focused. |
num:i |
Ignores signal number num. |
These commands to set conditional breakpoints are often useful.
Table 7–4 Setting mdb Breakpoints
[label],[count]:b [expression] |
Breakpoint is detected when expression equals zero |
foo,ffff:b <g7-0xabcdef |
Stop at foo when g7 = the hex value 0xABCDEF |
With the dbx utility you can debug and execute source programs written in C++, ANSI C, and FORTRAN. dbx accepts the same commands as the Debugger, but uses a standard terminal (TTY) interface. Both dbx and the Debugger support debugging multithreaded programs. For a full overview of dbx and Debugger features see the dbx(1) reference manual page and the Using Sun Workshop user's guide.
All the dbx options listed in Table 7–5 can support multithreaded applications.
Table 7–5 dbx Options for MT Programs