Sun Studio 12: Debugging a Program With dbx

Appendix B Event Management

Event management refers to the capability of dbx to perform actions when events take place in the program being debugged. When an event occurs, dbx allows you to stop a process, execute arbitrary commands, or print information. The simplest example of an event is a breakpoint (see Using Ctrl+C to Stop a Process). Examples of other events are faults, signals, system calls, calls to dlopen(), and data changes (see Setting Data Change Breakpoints).

This appendix is organized into the following sections:

Event Handlers

Event management is based on the concept of a handler. The name comes from an analogy with hardware interrupt handlers. Each event management command typically creates a handler, which consists of an event specification and a series of side-effect actions. (See Setting Event Specifications.) The event specification specifies the event that will trigger the handler.

When the event occurs and the handler is triggered, the handler evaluates the event according to any modifiers included in the event specification. (See Event Specification Modifiers.) If the event meets the conditions imposed by the modifiers, the handler’s side-effect actions are performed (that is, the handler “fires”).

An example of the association of a program event with a dbx action is setting a breakpoint on a particular line.

The most generic form of creating a handler is by using the when command.


when event-specification {action; ... }

Examples in this chapter show how you can write a command (like stop, step, or ignore) in terms of when. These examples are meant to illustrate the flexibility of the when command and the underlying handler mechanism, but they are not always exact replacements.

Event Safety

While dbx provides you with a rich set of breakpoint types through the event mechanism, it also uses many events internally. By stopping on some of these internal events you can easily disrupt the internal workings of dbx. If you modify the process state in these cases the chance of disruption is even higher. See Appendix A, Modifying a Program State and Call Safety.

dbx can protect itself from disruption in some cases but not all cases. Some events are implemented in terms of lower level events. For example, all stepping is based on the fault FLTTRACE event. So, issuing the command stop fault FLTTRACE disrupts stepping.

During the following phases of debugging, dbx is unable to handle user events because they interfere with some careful internal orchestration. These phases include:

In many cases you can use the when command instead of the stop command, and echo the information you would have otherwise acquired interactively.

dbx protects itself by:

For example:...stopped in munmap at 0xff3d503c 0xff3d503c: munmap+0x0004: ta %icc,0x00000008 dbx76: warning: 'stop' ignored -- while doing rtld handshake

Only the stoppage effect, including recording in the $firedhandlers variable, is ignored. Counts or filters are still active. To stop in such a case, set the event_safety environment variable to off.

Creating Event Handlers

Use the when command, stop command, and trace command to create event handlers. (For detailed information, see when Command, stop Command, and trace Command.)

stop is shorthand for a common when idiom.


when event-specification { stop -update; whereami; }

An event-specification is used by the event management commands stop, when, and trace to specify an event of interest. (see Setting Event Specifications).

Most of the trace commands can be handcrafted using the when command, ksh functionality, and event variables. This is especially useful if you want stylized tracing output.

Every command returns a number known as a handler id (hid). You can access this number using the predefined variable $newhandlerid.

Manipulating Event Handlers

You can use the following commands to manipulate event handlers. For more information on each command, see the cited section.

Using Event Counters

An event handler has a trip counter, which has a count limit. Whenever the specified event occurs, the counter is incremented. The action associated with the handler is performed only if the count reaches the limit, at which point the counter is automatically reset to 0. The default limit is 1. Whenever a process is rerun, all event counters are reset.

You can set the count limit using the -count modifier with a stop command, when command, or trace command (see -count n-count infinity). Otherwise, use the handler command to individually manipulate event handlers.


handler [ -count | -reset ] hid new-count new-count-limit

Setting Event Specifications

Event specifications are used by the stop command, stopi command, when command, wheni command, trace command, and tracei command to denote event types and parameters. The format consists of a keyword representing the event type and optional parameters. The meaning of an event specification is generally identical for all three commands; exceptions are documented in the command descriptions (see stop Command, trace Command, and when Command).

Breakpoint Event Specifications

A breakpoint is a location where an action occurs, at which point the program stops executing. The following are event specifications for breakpoint events.

in function

The function has been entered, and the first line is about to be executed. The first executable code after the prolog is used as the actual breakpoint location. This may be a line where a local variable is being initialized. In the case of C++ constructors, execution stops after all base class constructors have executed. If the -instr modifier is used (see -instr), it is the first instruction of the function about to be executed. The function specification can take a formal parameter signature to help with overloaded function names or template instance specification. For example:


 stop in mumble(int, float, struct Node *)

Note –

Do not confuse in function with the-in function modifier.


at [filename:]line_number

The designated line is about to be executed. If you specify filename, then the designated line in the specified file is about to be executed. The file name can be the name of a source file or an object file. Although quotation marks are not required, they may be necessary if the file name contains special characters. If the designated line is in template code, a breakpoint is placed on all instances of that template.

at address_expression

The instruction at the given address is about to be executed. This event is available only with thestopi command (see stopi Command) or with the -instr event modifier (see -instr).

infunction function

Equivalent to in function for all overloaded functions named function or all template instantiations thereof.

inmember function inmethod function

Equivalent to in function or the member function named function for every class.

inclass classname [-recurse | -norecurse]

Equivalent to in function for all member functions that are members of classname, but not any of the bases of classname. -norecurse is the default. If -recurse is specified, the base classes are included.

inobject object-expression [-recurse | -norecurse]

A member function called on the specific object at the address denoted by object-expression has been called.stop inobject ox is roughly equivalent to the following, but unlike inclass, bases of the dynamic type of ox are included. -recurse is the default. If -norecurse is specified, the base classes are not included.


stop inclass dynamic_type(ox) -if this==ox

Data Change Event Specifications

The following are event specifications for events that involve access or change to the contents of a memory address.

access mode address-expression [, byte-size-expression]

The memory specified by address-expression has been accessed.

mode specifies how the memory was accessed. It can be composed of one or all of the letters:

r

The memory at the specified address has been read.

w

The memory has been written to.

x

The memory has been executed.

mode can also contain either of the following:

a

Stops the process after the access (default).

b

Stops the process before the access.

In both cases the program counter will point at the offending instruction. The “before” and “after” refer to the side effect.

address-expression is any expression that can be evaluated to produce an address. If you give a symbolic expression, the size of the region to be watched is automatically deduced; you can override it by specifying byte-size-expression. You can also use nonsymbolic, typeless address expressions; in which case, the size is mandatory. For example:


stop access w 0x5678, sizeof(Complex)

The access command has the limitation that no two matched regions may overlap.


Note –

The access event specification is a replacement for the modify event specification.


change variable

The value of variable has changed. The change event is roughly equivalent to:


when step { if [ $last_value !=$[variable]] 
            then
                 stop
            else
                 last_value=$[variable]
            fi
          }

This event is implemented using single-stepping. For faster performance, use the access event (see access mode address-expression [, byte-size-expression]).

The first time variable is checked causes one event, even though no change is detected. This first event provides access to the initial value of variable. Subsequent detected changes in the value of variable trigger additional events.

cond condition-expression

The condition denoted by condition-expression evaluates to true. You can specify any expression for condition-expression, but it must evaluate to an integral type. The cond event is roughly equivalent to:

stop step -if conditional_expression

System Event Specifications

The following are event specifications for system events.

dlopen [ lib-path ] | dlclose [ lib-path ]

These events occur after a dlopen()() call or a dlclose()() call succeeds. A dlopen()() call or dlclose()() call can cause more than one library to be loaded. The list of these libraries is always available in the predefined variable $dllist. The first shell word in $dllist is a “+” or a “-”, indicating whether the list of libraries is being added or deleted.

lib-path is the name of a shared library. If it is specified, the event occurs only if the given library was loaded or unloaded. In that case, $dlobj contains the name of the library. $dllist is still available.

If lib-path begins with a /, a full string match is performed. Otherwise, only the tails of the paths are compared.

If lib-path is not specified, then the events always occur whenever there is any dl-activity. In this case, $dlobj is empty but $dllist is valid.

fault fault

The fault event occurs when the specified fault is encountered. The faults are architecture-dependent. The following set of faults known to dbx is defined in the proc(4) man page.

Fault  

Description  

FLTILL

Illegal instruction 

FLTPRIV

Privileged instruction 

FLTBPT*

Breakpoint trap 

FLTTRACE*

Trace trap (single step) 

FLTACCESS

Memory access (such as alignment) 

FLTBOUNDS

Memory bounds (invalid address) 

FLTIOVF

Integer overflow 

FLTIZDIV

Integer zero divide 

FLTPE

Floating-point exception 

FLTSTACK

Irrecoverable stack fault 

FLTPAGE

Recoverable page fault 

FLTWATCH*

Watchpoint trap 

FLTCPCOVF

CPU performance counter overflow 


Note –

BPT, TRACE, and BOUNDS are used by dbx to implement breakpoints and single-stepping. Handling them might interfere with how dbx works.



Note –

FLTBPT and FLTTRACE are ignored because they would interfere with basic dbx functionality such as breakpoints and single-stepping (see Event Safety).


These faults are taken from /sys/fault.h. fault can be any of those listed above, in uppercase or lowercase, with or without the FLT- prefix, or the actual numerical code.


Note –

The fault event is not available on Linux platforms.


lwp_exit

The lwp_exit event occurs when lwp has been exited. $lwp contains the id of the exited LWP (lightweight process) for the duration of the event handler.


Note –

The lwpexit event is not available on Linux platforms.


sig signal

The sig signal event occurs when the signal is first delivered to the program being debugged. signal can be either a decimal number or the signal name in uppercase or lowercase; the prefix is optional. This is completely independent of the catch command and ignore command, although the catch command can be implemented as follows:


function simple_catch {
    when sig $1 {
            stop;
            echo Stopped due to $sigstr $sig
            whereami
    }
}

Note –

When the sig event is received, the process has not seen it yet. Only if you continue the process with the specified signal is the signal forwarded to it.


sig signal sub-code

When the specified signal with the specified sub-code is first delivered to the child, the sig signal sub-code event occurs. As with signals, you can type the sub-code as a decimal number, in uppercase or lowercase; the prefix is optional.

sysin code | name

The specified system call has just been initiated, and the process has entered kernel mode.

The concept of system call supported by dbx is that provided by traps into the kernel as enumerated in /usr/include/sys/syscall.h.

This is not the same as the ABI notion of system calls. Some ABI system calls are partially implemented in user mode and use non-ABI kernel traps. However, most of the generic system calls (the main exception being signal handling) are the same between syscall.h and the ABI.


Note –

The sysin event is not available on Linux platforms.



Note –

The list of kernel system call traps in /usr/include/sys/syscall.h is part of a private interface in the Solaris OS that changes from release to release. The list of trap names (codes) and trap numbers that dbx accepts includes all of those supported by any of the versions of the Solaris OS that dbx supports. It is unlikely that the names supported by dbx exactly match those of any particular release of the Solaris OS, and some of the names in syscall.h might not be available. Any trap number (code) is accepted by dbx and works as expected, but a warning is issued if it does not correspond to a known system call trap.


sysout code | name

The specified system call is finished, and the process is about to return to user mode.


Note –

The sysout event is not available on Linux platforms.


sysin | sysout

Without arguments, all system calls are traced. Certain dbx features, for example, the modify event and runtime checking, cause the child to execute system calls for its own purposes and show up if traced.

Execution Progress Event Specifications

The following are event specifications for events pertaining to execution progress.

exit exitcode

The exit event occurs when the process has exited.

next

The next event is similar to the step event except that functions are not stepped into.

returns

The returns event is a breakpoint at the return point of the current visited function. The visited function is used so that you can use the returns event specification after giving a number of step up commands. The returns event is always -temp and can only be created in the presence of a live process.

returns function

The returns function event executes each time the given function returns to its call site. This is not a temporary event. The return value is not provided, but you can find integral return values by accessing the following registers:

SPARC based systems

$o0

x86 based systems

$eax

x64 based systems

$rax, $rdx

The event is roughly equivalent to:


when in func { stop returns; }

step

The step event occurs when the first instruction of a source line is executed. For example, you can get simple tracing with:


when step { echo $lineno: $line; }; cont

When enabling a step event, you instruct dbx to single step automatically next time the contcommand is used.


Note –

The step (and next) events do not occur upon the termination of the step command. The step command is implemented in terms of the step event roughly as follows: alias step="when step -temp { whereami; stop; }; cont"


Other Event Specifications

The following are event specifications for other types of events.

attach

dbx has successfully attached to a process.

detach

dbx has successfully detached from the program being debugged.

lastrites

The process being debugged is about to expire, which can happen for the following reasons:

The final state of the process is usually, but not always, available when this event is triggered, giving you your last opportunity to examine the state of the process. Resuming execution after this event terminates the process.


Note –

The lastrites event is not available on Linux platforms.


proc_gone

The proc_gone event occurs when dbx is no longer associated with a debugged process. The predefined variable $reason may be signal, exit, kill, or detach.

prog_new

The prog_new event occurs when a new program has been loaded as a result of follow exec.


Note –

Handlers for this event are always permanent.


stop

The process has stopped. The stop event occurs whenever the process stops such that the user receives a prompt, particularly in response to a stop handler. For example, the following commands are equivalent:


display x
when stop {print x;}

sync

The process being debugged has just been executed with exec(). All memory specified in a.out is valid and present, but preloaded shared libraries have not been loaded. For example, printf, although available to dbx, has not been mapped into memory.

A stop on this event is ineffective; however, you can use the sync event with the when command.


Note –

The sync event is not available on Linux platforms.


syncrtld

The syncrtld event occurs after a sync (or attach if the process being debugged has not yet processed shared libraries). It executes after the dynamic linker startup code has executed and the symbol tables of all preloaded shared libraries have been loaded, but before any code in the .init section has run.

A stop on this event is ineffective; however, you can use the syncrtld event with the when command.

thr_create [thread_id]

The thr_create event occurs when a thread, or a thread with the specified thread_id, has been created. For example, in the following stop command, the thread ID t@1 refers to creating thread, while the thread ID t@5 refers to the created thread.


stop thr_create t@5 -thread t@1

thr_exit

The thr_exit event occurs when a thread has exited. To capture the exit of a specific thread, use the -thread option of the stop command as follows:


stop thr_exit -thread t@5

throw

The throw event occurs whenever any exception that is not unhandled or unexpected is thrown by the application.


Note –

The throw event is not available on Linux platforms.


throw type

If an exception type is specified with the throw event, only exceptions of that type cause the throw event to occur.

throw -unhandled

-unhandled is a special exception type signifying an exception that is thrown but for which there is no handler.

throw -unexpected

-unexpected is a special exception type signifying an exception that does not satisfy the exception specification of the function that threw it.

timer seconds

The timer event occurs when the program being debugged has been running for seconds. The timer used with this event is shared with collector command. The resolution is in milliseconds, so a floating point value for seconds, for example 0.001, is acceptable.

Event Specification Modifiers

An event specification modifier sets additional attributes of a handler, the most common kind being event filters. Modifiers must appear after the keyword portion of an event specification. A modifier begins with a dash (-). The following are the valid event specification modifiers.

-if condition

The condition is evaluated when the event specified by the event specification occurs. The side effect of the handler is allowed only if the condition evaluates to nonzero.

If the -if modifier is used with an event that has an associated singular source location, such as in or at, condition is evaluated in the scope corresponding to that location. Otherwise, qualify it with the desired scope.

-resumeone

The -resumeone modifier can be used with the -if modifier in an event specification for a multithreaded program, and causes only one thread to be resumed if the condition contains function calls. For more information, see Using a Filter With a Conditional Event.

-in function

The event triggers only if it occurs between the time the first instruction of the given function is reached and the time the function returns. Recursion on the function are ignored.

-disable

The-disable modifier creates the handler in the disabled state.

-count n-count infinity

The -count n and -count infinity modifiers have the handler count from 0 (see Using Event Counters). Each time the event occurs, the count is incremented until it reaches n. Once that happens, the handler fires and the counter is reset to zero.

Counts of all enabled handlers are reset when a program is run or rerun. More specifically, they are reset when the sync event occurs.

The count is reset when you begin debugging a new program with the debug -r command (see debug Command) or the attach -r command (see attach Command).

-temp

Creates a temporary handler. Once the event has occurred it is automatically deleted. By default, handlers are not temporary. If the handler is a counting handler, it is automatically deleted only when the count reaches 0 (zero).

Use the delete -temp command to delete all temporary handlers.

-instr

Makes the handler act at an instruction level. This event replaces the traditional ’i’ suffix of most commands. It usually modifies two aspects of the event handler:

-thread thread_id

The action is executed only if the thread that caused the event matches thread_id. The specific thread you have in mind might be assigned a different thread_id from one execution of the program to the next.

-lwp lwp_id

The action is executed only if the thread that caused the event matches lwp_id. The action is executed only if the thread that caused the event matches lwp_id. The specific thread you have in mind might be assigned a different lwp_id from one execution of the program to the next.

-hidden

Hides the handler in a regular status command. Use status -h to see hidden handlers.

-perm

Normally all handlers are thrown away when a new program is loaded. Using the -perm modifier retains the handler across debugging sessions. A plain delete command does not delete a permanent handler. Use delete -p to delete a permanent handler.

Parsing and Ambiguity

The syntax for event specifications and modifiers is:

Expressions can have spaces embedded in them, causing ambiguous situations. For example, consider the following two commands:


when a -temp
when a-temp

In the first example, even though the application might have a variable named temp, the dbx parser resolves the event specification in favor of-temp being a modifier. In the second example, a-temp is collectively passed to a language-specific expression parser. There must be variables named a and temp or an error occurs. Use parentheses to force parsing.

Using Predefined Variables

Certain read-only ksh predefined variables are provided. The following variables are always valid:

Variable  

Definition  

$ins

Disassembly of the current instruction. 

$lineno

Current line number in decimal. 

$vlineno

Current “visiting” line number in decimal. 

$line

Contents of the current line. 

$func

Name of the current function. 

$vfunc

Name of the current “visiting” function. 

$class

Name of the class to which $func belongs.

$vclass

Name of the class to which $vfunc belongs.

$file

Name of the current file. 

$vfile

Name of the current file being visited. 

$loadobj

Name of the current loadable object. 

$vloadobj

Name of the current loadable object being visited. 

$scope

Scope of the current PC in back-quote notation. 

$vscope

Scope of the visited PC in back-quote notation. 

$funcaddr

Address of $func in hex.

$caller

Name of the function calling $func.

$dllist

After a dlopen or dlclose event, contains the list of load objects just loaded or unloaded. The first word of dllist is a “+” or a “-” depending on whether a dlopen or a dlclose has occurred.

$newhandlerid

ID of the most recently created handler. This variable has an undefined value after any command that deletes handlers. Use the variable immediately after creating a handler. dbx cannot capture all of the handler IDs for a command that creates multiple handlers.

$firedhandlers

List of handler ids that caused the most recent stoppage. The handlers on the list are marked with “*” in the output of the status command.

$proc

Process ID of the current process being debugged. 

$lwp

Lwp ID of the current LWP. 

$thread

Thread ID of the current thread. 

$newlwp

lwp ID of a newly created LWP. 

$newthread

Thread ID of a newly created thread. 

$prog

Full path name of the program being debugged. 

$oprog

Previous value of $prog, which is used to get back to what you were debugging following an exec(), when $prog reverts to “-”. While $prog is expanded to a full path name, $oprog contains the program path as specified on the command line or to the debug command. If exec() is called more than once, there is no way to return to the original program.

$exec32

True if the dbx binary is 32–bit.

$exitcode

Exit status from the last run of the program. The value is an empty string if the process has not exited. 

As an example, consider that whereami can be implemented as:


function whereami {
  echo Stopped in $func at line $lineno in file $(basename $file)
  echo "$lineno\t$line"
}

Variables Valid for when Command

The following variables are valid only within the body of a when command.

$handlerid

During the execution of the body, $handlerid is the id of the when command to which the body belongs. These commands are equivalent:


when X -temp { do_stuff; }
when X  { do_stuff; delete $handlerid; }

$booting

$booting is set to true if the event occurs during the boot process. Whenever a new program is debugged, it is first run without the user’s knowledge so that the list and location of shared libraries can be ascertained. The process is then killed. This sequence is termed booting.

While booting is occurring, all events are still available. Use this variable to distinguish the sync and the syncrtld events occurring during a debug and the ones occurring during a normal run.

Variables Valid for Specific Events

Certain variables are valid only for specific events as shown in the following tables.

Table B–1 Variables Valid for sig Event

Variable  

Description  

$sig

Signal number that caused the event 

$sigstr

Name of $sig

$sigcode

Subcode of $sig if applicable

$sigcodestr

Name of $sigcode

$sigsender

Process ID of sender of the signal, if appropriate 

Table B–2 Variable Valid for exit Event

Variable  

Description  

$exitcode

Value of the argument passed to _exit(2) or exit(3) or the return value of main

Table B–3 Variable Valid for dlopen and dlclose Events

Variable 

Description 

$dlobj

Pathname of the load object dlopened or dlclosed 

Table B–4 Variables Valid for sysin and sysout Events

Variable 

Description 

$syscode

System call number 

$sysname

System call name 

Table B–5 Variable Valid for proc_gone Events

Variable 

Description 

$reason

One of signal, exit, kill, or detach 

Table B–6 Variables Valid for thr_create Event

Variable 

Description 

$newthread

Id of the newly created thread (for example, t@5)

$newlwp

Id of the newly created LWP (for example, l@4)

Setting Event Handler Examples

The following are some examples of setting event handlers.

Setting a Breakpoint for Store to an Array Member

To set a breakpoint on array[99], type:


(dbx) stop access w &array[99]
(2) stop access w &array[99], 4
(dbx) run
Running: watch.x2
watchpoint array[99] (0x2ca88[4]) at line 22 in file "watch.c"    
   22    array[i] = i;

Implementing a Simple Trace

To implement a simple trace, type:


(dbx) when step { echo at line $lineno; }

Enabling a Handler While Within a Function (in function)

To enable a handler while within a function, type:


<dbx> trace step -in foo

This is equivalent to:


    # create handler in disabled state
    when step -disable { echo Stepped to $line; }
    t=$newhandlerid    # remember handler id
    when in foo {
     # when entered foo enable the trace
     handler -enable "$t"
     # arrange so that upon returning from foo,
     # the trace is disabled.
     when returns { handler -disable "$t"; };
    }

Determining the Number of Lines Executed

To see how many lines have been executed in a small program, type:


(dbx) stop step -count infinity     # step and stop when count=inf
(2) stop step -count 0/infinity
(dbx) run
...
(dbx) status
(2) stop step -count 133/infinity

The program never stops—the program terminates. The number of lines executed is 133. This process is very slow. It is most useful with breakpoints on functions that are called many times.

Determining the Number of Instructions Executed by a Source Line

To count how many instructions a line of code executes, type:


(dbx) ...                        # get to the line in question
(dbx) stop step -instr -count infinity
(dbx) step ...
(dbx) status
(3) stop step -count 48/infinity # 48 instructions were executed

If the line you are stepping over makes a function call, the lines in the function are counted as well. You can use the next event instead of step to count instructions, excluding called functions.

Enabling a Breakpoint After an Event Occurs

Enable a breakpoint only after another event has occurred. For example, if your program begins to execute incorrectly in function hash, but only after the 1300’th symbol lookup, you would type:


(dbx) when in lookup -count 1300 {
    stop in hash
    hash_bpt=$newhandlerid
    when proc_gone -temp { delete $hash_bpt; }
}

Note –

$newhandlerid is referring to the just executed stop incommand.


Resetting Application Files for replay

If your application processes files that need to be reset during a replay, you can write a handler to do that each time you run the program:


(dbx) when sync { sh regen ./database; }
(dbx) run < ./database...    # during which database gets clobbered
(dbx) save
...              # implies a RUN, which implies the SYNC event which
(dbx) restore       # causes regen to run

Checking Program Status

To see quickly where the program is while it is running, type:


(dbx) ignore sigint
(dbx) when sig sigint { where; cancel; }

Then type ^C to see a stack trace of the program without stopping it.

This is basically what the collector hand sample mode does (and more). Use SIGQUIT (^\) to interrupt the program because ^C is now used up.

Catch Floating Point Exceptions

To catch only specific floating point exceptions, for example, IEEE underflow, type:


(dbx) ignore FPE               # turn off default handler
(dbx) help signals | grep FPE  # can’t remember the subcode name
...
(dbx) stop sig fpe FPE_FLTUND
...