Multithreaded Programming Guide

Process Creation—Forking Issues

The default handling of the fork() function in the Solaris operating environment is somewhat different from the way fork() is handled in POSIX threads, although the Solaris operating environment does support both mechanisms.

Table 5–1 compares the differences and similarities of Solaris and pthreads fork() handling. When the comparable interface is not available either in POSIX threads or in Solaris threads, the `—' character appears in the table column.

Table 5–1 Comparing POSIX and Solaris fork() Handling


	Solaris Operating Environment Interface	POSIX Threads Interface
Fork-one model	`fork1(2)`	`fork(2)`
Fork-all model	`fork(2)`	—
Fork safety	—	`pthread_atfork(3THR)`

The Fork-One Model

As shown in Table 5–1, the behavior of the pthreads fork(2) function is the same as that of the Solaris fork1(2) function. Both the pthreads fork(2) function and the Solaris fork1(2) create a new process, duplicating the complete address space in the child, but duplicating only the calling thread in the child process.

This is useful when the child process immediately calls exec(), which is what happens after most calls to fork(). In this case, the child process does not need a duplicate of any thread other than the one that called fork().

In the child, do not call any library functions after calling fork() and before calling exec() because one of the library functions might use a lock that was held in the parent at the time of the fork(). The child process might execute only Async-Signal-Safe operations until one of the exec() handlers is called.

The Fork-One Safety Problem and Solution

In addition to all of the usual concerns such as locking shared data, a library should be well behaved with respect to forking a child process when only one thread is running (the one that called fork()). The problem is that the sole thread in the child process might try to grab a lock that is held by a thread that wasn't duplicated in the child.

This is not a problem most programs are likely to encounter. Most programs call exec() in the child right after the return from fork(). However, if the program wishes to carry out some actions in the child before the call to exec(), or never calls exec(), then the child could encounter deadlock scenarios.

Each library writer should provide a safe solution, although not providing a fork-safe library is not a large concern because this condition is rare.

For example, assume that T1 is in the middle of printing something (and so is holding a lock for printf()), when T2 forks a new process. In the child process, if the sole thread (T2) calls printf(), it promptly deadlocks.

The POSIX fork() or Solaris fork1() duplicates only the thread that calls it. (Calling the Solaris fork() duplicates all threads, so this issue does not come up.)

To prevent deadlock, ensure that no such locks are being held at the time of forking. The most obvious way to do this is to have the forking thread acquire all the locks that could possibly be used by the child. Because you cannot do this for locks like those in printf() (because printf() is owned by libc), you must ensure that printf() is not being used at fork() time.

To manage the locks in your library:

Identify all the locks used by the library.

Identify the locking order for the locks used by the library. (If a strict locking order is not used, then lock acquisition must be managed carefully.)

Arrange to acquire those locks at fork time. In Solaris threads this must be done manually, obtaining the locks just before calling fork1(), and releasing them right after.

In the following example, the list of locks used by the library is {L1,...Ln}, and the locking order for these locks is also L1...Ln.

mutex_lock(L1);
mutex_lock(L2);
fork1(...);
mutex_unlock(L1);
mutex_unlock(L2);

In pthreads, you can add a call to pthread_atfork(f1, f2, f3) in your library's .init() section, where f1(), f2(), f3() are defined as follows:

f1() /* This is executed just before the process forks. */
{
 mutex_lock(L1); |
 mutex_lock(...); | -- ordered in lock order
 mutex_lock(Ln); |
 } V

f2() /* This is executed in the child after the process forks. */
 {
 mutex_unlock(L1);
 mutex_unlock(...);
 mutex_unlock(Ln);
 }

f3() /* This is executed in the parent after the process forks. */
 {
 mutex_unlock(L1);
 mutex_unlock(...);
 mutex_unlock(Ln);
 }

Another example of deadlock would be a thread in the parent process—other than the one that called Solaris fork1(2)—that has locked a mutex. This mutex is copied into the child process in its locked state, but no thread is copied over to unlock the mutex. So, any thread in the child that tries to lock the mutex waits forever.

Virtual Forks–vfork(2)

The standard vfork(2) function is unsafe in multithreaded programs. vfork(2) is like fork1(2) in that only the calling thread is copied in the child process. As in nonthreaded implementations, vfork() does not copy the address space for the child process.

Be careful that the thread in the child process does not change memory before it calls exec(2). Remember that vfork() gives the parent address space to the child. The parent gets its address space back after the child calls exec() or exits. It is important that the child not change the state of the parent.

For example, it is disastrous to create new threads between the call to vfork() and the call to exec().

The Solution—pthread_atfork(3THR)

Use pthread_atfork() to prevent deadlocks whenever you use the fork-one model.

#include <pthread.h>

int pthread_atfork(void (*prepare) (void), void (*parent) (void),
    void (*child) (void) );

The pthread_atfork() function declares fork() handlers that are called before and after fork() in the context of the thread that called fork().

The prepare handler is called before fork() starts.
The parent handler is called after fork() returns in the parent.
The child handler is called after fork() returns in the child.

Any one of these can be set to NULL. The order in which successive calls to pthread_atfork() are made is significant.

For example, a prepare handler could acquire all the mutexes needed, and then the parent and child handlers could release them. This ensures that all the relevant locks are held by the thread that calls the fork function before the process is forked, preventing the deadlock in the child.

Using the fork all model avoids the deadlock problem described in The Fork-One Safety Problem and Solution.

Return Values

pthread_atfork() returns a zero when it completes successfully. Any other return value indicates that an error occurred. If the following condition is detected, pthread_atfork(3THR) fails and returns the corresponding value.

ENOMEM: Insufficient table space exists to record the fork handler addresses.

The Fork-All Model

The Solaris fork(2) function duplicates the address space and all the threads (and LWPs) in the child. This is useful, for example, when the child process never calls exec(2) but does use its copy of the parent address space. The fork-all functionality is not available in POSIX threads.

Note that when one thread in a process calls Solaris fork(2), threads that are blocked in an interruptible system call return EINTR.

Also, be careful not to create locks that are held by both the parent and child processes. This can happen when locks are allocated in memory that is sharable (that is use mmap() with the MAP_SHARED flag). Note that this is not a problem if the fork-one model is used.

Choosing the Right Fork

You determine whether fork() has a “fork-all” or a “fork-one” semantic in your application by linking with the appropriate library. Linking with -lthread gives you the “fork-all” semantic for fork(), and linking with -lpthread gives the “fork-one” semantic for fork() (see Figure 7–1for an explanation of compiling options).

Cautions for Any Fork

Be careful when using global state after a call to any fork() function.

For example, when one thread reads a file serially and another thread in the process successfully calls one of the forks, each process then contains a thread that is reading the file. Because the seek pointer for a file descriptor is shared after a fork(), the thread in the parent gets some data while the thread in the child gets the other. This introduces gaps in the sequential read accesses.