Using dbx with Multithreaded Programs

Multithreaded features are an inherent part of the standard dbx.

The major multithreaded features offered by dbx are:

• The threads command will give a list of all known threads, with their current state, base functions, and current functions.

• You can examine stack traces of each thread.

• You can resume (cont) a specific thread or all threads.

• You can step or next a specific thread.

• The thread command helps navigating between threads.

• The syncs command lists all synchronization objects known to libthread.

• The sync command provides information on a given synchronization object, such as which thread is blocked by it or which thread owns the locks.

You can limit your scope to a specific thread. dbx maintains a cursor to the "current" or "active" thread. It is manipulable by the thread command. The only commands that use the current thread as the default thread are where and thread -info.

Thread Numbering

dbx knows the id of each thread (the type thread_t) as returned by thr_create(). The syntax is t@number.

LWP Numbering

dbx knows the id of each LWP (the type lwpid_t) as presented by the /proc (man procfs(4)) interface. The syntax is l@number.

Breakpoints on a Specific Thread

You can have a breakpoint on a specific thread by filtering a regular breakpoint:

stop in foo -thread t@4

Where the t@4 refers to the thread with id 4.

When a thread hits a breakpoint, all threads stop. This is knows as "sympathetic stop," or "stop the world".

From the point of view of /proc and LWPs this is synchronous debugging.

To ease thread navigation, put this in your .dbxrc.

_cb_prompt() {
        if [ $mtfeatures = "true"]
        then
                PS1='[$thread $lwp]: '
        else
                PS1="(dbx-$proc) "
        fi
}

dbx Identification of Multithreaded Applications

If an application is linked with -lthread, dbx assumes it is multithreaded.

The Collector, RTC, fix and continue, and Watchpoints

The collector and fix and continue work with multithreaded applications. RTC works with multithreaded applications, however a libthread patch is needed.

In the Solaris 2.5.1 operating environment, the implementation of watchpoints does not depend on the operating system, and has the potential of too easily getting the multithreaded application into a deadlock or other obscure problems.

Multithreaded Pitfalls

It is very easy to get your program to deadlock by resuming only a specific thread while other threads are still and hold a resource that the resumed thread might need.

libthread data structures are in user space and might get corrupted by bugs involving rogue pointers. In such cases one suggestion is to work at the LWP level with commands like lwps and lwp, which are analogous to their thread equivalents.

Sleeping Threads

You cannot "force" a sleeping thread to run. In general, when debugging multithreaded applications it is recommended that you take a "stand back and watch" approach rather than trying to alter the program's natural execution flow.

thr_join, thr_create(), and thr_exit

Starting from the threads list, you can determine which thread id came from which start function. The "base function" as it is known, is printed in the thread listing.

When you attach to an existing multithreaded process, it is non-deterministic which thread becomes the active thread.

When the active thread does a thr_create, the current threads stays with the "creating thread". In the follow_fork analogy, it would be parent.

The Sun multithreaded model doesn't have true fork semantics for threads. There is no thread tree, and no parent-child relationships as there is with processes. thr_join() is only a simplified veneer.

When the active thread does a thr_exit, dbx makes a dummy "dead" thread as the active thread. This thread is represented as t@X.