| Skip Navigation Links | |
| Exit Print View | |
|
Oracle Solaris Studio 12.3: dbxtool Tutorial Oracle Solaris Studio 12.3 Information Library |
| Skip Navigation Links | |
| Exit Print View | |
|
|
Oracle Solaris Studio 12.3: dbxtool Tutorial Oracle Solaris Studio 12.3 Information Library |
December 2011
This tutorial uses a buggy example program to demonstrate how to use dbxtool, the stand-alone graphical user interface (GUI) for the dbx debugger, effectively. It starts with the basics and then moves on to more advanced features.
This tutorial uses a “buggy” example program to demonstrate how to use dbxtool, the stand-alone graphical user interface (GUI) for the dbx debugger, effectively. It starts with the basics and then moves on to more advanced features.
This tutorial uses a simplified and somewhat artificial simulation of the dbx debugger. The source code for this C++ program is available in the sample applications zip file on the Oracle Solaris Studio 12.3 Sample Applications web page at http://www.oracle.com/technetwork/server-storage/solarisstudio/downloads/solaris-studio-samples-1408618.html.
If you have not already done so, download the sample applications zip file, and unpack the file in a location of your choice. The debug_tutorial application is located in the Debugger subdirectory of the SolarisStudioSampleApplications directory.
Build the program:
make CC -g -c main.cc CC -g -c interp.cc CC -g -c cmd.cc CC -g -c debugger.cc CC -g -c cmds.cc CC -g main.o interp.o cmd.o debugger.o cmds.o -o a.out
The program is made up of the following modules:
|
Run the program and try a few dbx commands:
$ a.out
> display var
will display 'var'
> stop in X
> run running ...
stopped in X
var = {
a = '100'
b = '101'
c = '<error>'
d = '102'
e = '103'
f = '104'
}
> quit
Goodby
$
Start dbxtool by typing:
installation_directory/bin/dbxtool
The first time you start dbxtool, the window looks like the following:
If you are reading this tutorial in your web browser, it is probably taking up half your screen, so you will find it beneficial to customize dbxtool to make it a half-screen application as well.
The following are examples of the various ways you can customize dbxtool.
Make the toolbar icons smaller:
Right-click anywhere in the toolbar and choose Small Toolbar Icons.
Move the Call Stack window out of the way:
Click the header of the Call Stack window and drag the window downward and to the right. Let it go when the red outline is in this position:
Now click on the right pointing arrow in the header of the Call Stack window:
The Call Stack window is minimized in the right margin:
If you hold the cursor over the minimized Call Stack icon, the Call Stack window is maximized until you transfer focus to another window. If you click the minimized Call Stack icon, the Call Stack window is maximized until you click the icon again.
Now you should be able to narrow the main window to half-screen:
Minimize the Breakpoints window:
Click the Breakpoints tab.
Click the down arrow on the tab to minimize the Breakpoints window.
Undock the Process I/O window:
Click and hold on the header of the Process I/O window, drag the window outside of the dbxtool window, and drop it onto your desktop. Now you can easily interact with the input and output of programs you are debugging while having easy access to the other tabs in the dbxtool window.
To re-dock the Process I/O window in the dbxtool window, right-click in the Process I/O window and choose Dock window.
Set the font size in the editor. After you have some source code displayed in the Editor window:
Choose Tools > Options.
In the Options window, select the Fonts & Colors category.
On the Syntax tab, make sure All Languages is selected from the Languages drop-down list.
Click the browse button next to the Font text box.
In the Font Chooser dialog box, set the font, style, and size, and click OK.
Click OK in the Options window.
Set the font size in the terminal windows. The Debugger Console and Process I/O windows are ANSI terminal emulators.
Choose Tools > Options.
In the Options window, select the Miscellaneous category.
Click the Terminal tab.
Select settings like Font Size and Click To Type.
Click OK.
Now that you have configured dbxtool to suit your preferences, let's find some bugs.
Run the example program again, except this time press Return without entering a command:
$ a.out > display var will display 'var' > Segmentation Fault (core dumped) $
Now start dbxtool with the executable and the core file:
$ dbxtool a.out core
Tip - Notice that the dbxtool command accepts the same arguments as the dbx command.
dbxtool displays something like the following:
Here are some things to notice:
In the Debugger Console window, you see a message like the following:
program terminated by signal SEGV (no mapping at fault address) 0xff0318f0: strcmp+0x0170: ld [%ol], %gl Current function is Interp::find
Even though the SEGV happened in the strcmp() function,dbx automatically shows the first frame with a function that has debugging information. See how the stack trace in the Call Stack window has a border around the icon for the current frame:
The Call Stack window shows the parameter names and values. You can see that the second parameter passed to strcmp() is 0x0 and that the value of name is NULL.
In the Editor window, the lavender stripe and a triangle instead of a green stripe and arrow, signify the location of the call to strcmp() rather than the actual location of the error:
Tip - If you do not see parameter values, check that the dbx environment variable stack_verbose is set to on in your .dbxrc file. You can also turn on verbose mode in the Call Stack window by right-clicking in the window and selecting the Verbose checkbox to add a check mark.
Functions usually fail when they are passed bad values as parameters. Here are some ways to check the values passed to strcmp():
Check the values of the parameters in the Variables window.
Click the Variables tab.
Note that the value of name is NULL. That value is quite likely to be the cause of the SEGV, but let's check the value of the other parameter, (*cp)->name().
In the Variables window, expand the cp node and then expand the (cp*) node. The name in question is “quit”, which is fine:
Tip - If expanding the *cp node does not show additional variables, check that the dbx environment variable output_inherited_members in your .dbxrc file is set to on. You can also turn on the display of inherited members by right-clicking in the window and selecting the Inherited Members checkbox to add a check mark.
Use Balloon Evaluation to confirm the value of a parameter. In the Editor window, put the cursor over the name variable being passed to strcmp(). A tip is displayed showing the value of name as NULL.
Using balloon evaluation, you can also place the cursor over an expression like (*cp)->name(). In this case, you cannot do so because that expression contains function calls.
Tip -
Balloon evaluation of expressions with function calls is disabled because:
You are debugging a core file.
Function calls might have side effects and you don't want them to occur as a result of casual hovering of the cursor in the Editor window.
Now it is abundantly clear that the value of name should not be NULL. But which code passed this bad value to Interp::find()? To find out you can:
Move up the call stack by choosing Debug > Stack > Make Caller Current or clicking the Make Caller Current button 
on the toolbar:
In the Call Stack window, double-click on the frame corresponding to Interp::dispatch(). The Editor window now highlights the corresponding code:
This code is unfamiliar. And a quick glance doesn't really point to any clues other than that the value of argv[0] is NULL.
It might easier to debug this problem dynamically using breakpoints and stepping.
Breakpoints let you stop a program a bit before the manifestation of a bug and step through the code in the hope of discovering what went wrong.
If you haven't already done so, now might be a good time to undock the Process I/O window.
You ran the program from the command line earlier. Now reproduce the bug by running the program in dbxtool:
Click the Run button 
on the toolbar or type run in the Debugger Console window.
Press Return in the Process I/O window. This time, an alert box tells you about the SEGV:
In the alert box, click Discard and Pause. The Editor window once again highlights the call to strcmp() in Interp::find().
Click the Make Caller Current button in the toolbar to go to the unfamiliar code you saw earlier in Interp::dispatch(). Now you can set a breakpoint a bit before the call to find(). Later you can step through the code in the hope of learning why things went wrong.
There are several ways to set a breakpoint. First, if the line numbers are not showing, enable line numbers in the editor by right-clicking in the left margin and selecting the Show Line Numbers checkbox.
Toggle a line breakpoint by clicking in the left margin next to line 127.
Set a function breakpoint by doing the following:
Select Interp::dispatch in the Editor window.
Choose Debug > New Breakpoint or right-click and choose New Breakpoint. The New Breakpoint dialog box opens.
Notice that the Function field is seeded with the selected function name.
Click OK.
It is easiest to set a function breakpoint from the dbx command line. To do so, type the stop in command in the Debugger Console window:
(dbx) stop in dispatch (4) stop in Interp::dispatch(char*) (dbx)
Notice that you didn't have to type Interp::dispatch. Just the function name sufficed.
By now the Editor is getting cluttered:
You can clean up this clutter using the Breakpoints window.
Click the Breakpoints tab (or maximize it if you minimized it earlier).
Select the line breakpoint and one of the function breakpoints, right-click, and choose Delete.
Setting a line breakpoint by toggling in the editor might be intuitive. However, many dbx users prefer function breakpoints for the following reasons:
It is often easiest just to type si dispatch in the Debugger Console window. It saves you having to open a file in the editor and scroll to a line just to place a breakpoint.
You can create function breakpoints by selecting any text in the editor. So you can set a breakpoint on a function from its call site instead of opening a file.
Tip - si is an alias for stop in. Most dbx users define many aliases and put them in the dbx configuration file ~/.dbxrc. Some common examples are:
alias si stop in alias sa stop at alias s step alias n next alias r run
The name of a function breakpoint is descriptive in the Breakpoints window. The name of a line breakpoint is not - who knows what is at interp.cc:127? (Actually, you can find what's at line 127 by right-clicking the line breakpoint in the Breakpoints window and choosing GoTo Source, or by double-clicking on the breakpoint.)
Function breakpoints persist better. Because dbxtool persists breakpoints, line number breakpoints might easily become skewed if you edit code or do a source code control merge. Function names are less sensitive to edits.
So, now you have a single breakpoint at Interp::dispatch(). If you click Run
again and press Return in the Process I/O window, the program stops at
the first line of the dispatch()function that contains executable code.
You already know that the culprit is the argv[0] being passed to find(), so keep an eye on argv using watches:
Select an instance of argv in the Editor window.
Right-click and choose New Watch. The New Watch dialog box opens seeded with the selected text:
Click OK.
Open the Watches window by choosing Window > Watches.
Expand argv.
What is all of this garbage? Notice that argv is uninitialized and because it's a local variable, it might “inherit” random values left on the stack from previous calls. Could this be the cause of problems? Let's not get ahead of ourselves and proceed methodically.
Click Step Over 
two times until the green PC arrow points to int argc = 0;.
It's clear that argc is going to be an index into argv so keep an eye on it as well and create a watch for it also. Notice that it is also currently uninitialized and might contain garbage values.
Because you created the watch for argc second it appears under argv in the Watches window. It would be nice if it showed up on the first row of the window. You could delete the watches and re-enter them in the desired order. However, in this case, there's a quick trick you can use. Clicking the Name column header sorts the column. Click it until you get something like the following (notice the sort triangle):
Click Step Over . Notice how argc now shows its initialized value of 0. It is displayed in bold to signify that the value just changed.
Our application is going to call strtok(). Click Step Over to step over the function, and observe, for example, by using balloon evaluation, that token is NULL.
Tip - What does strok() do? You can read the strok(3) man page, but in short, it helps break up a string, for example, line, into tokens delimited by one of DELIMETERS/
Clicking Step Over again assigns the token to argv and then there is a call to strtok() in a loop. As you step over, you don't enter the loop (there are no more tokens) and instead a NULL is assigned. Step over that assignment too and you are at the threshold of the call to find. If you recall, this is where our program crashed.
Double check that the program crashes here by stepping over the call to find(). Sure enough, the Signal Caught alert box is displayed again.
Click Discard and Pause as before.
So the first call to find() after stopping in Interp::dispatch is indeed where things go wrong.
This may have been obvious but the point is to illustrate that you can quickly get back to where you were. Here's how:
Click Make Caller Current .
Toggle a line breakpoint at the call site of find().
Open the Breakpoints window and disable the Interp::dispatch() function breakpoint.
dbxtool should look like this:
The downward arrow indicates that two breakpoints are set on line 141 and that one of them is disabled.
Click Run and press Return in the Process I/O window, and the program will end up right back in front of the call to find(). (Notice how the Run button evokes re-starting. When debugging you re-start much more often that just start.)
Tip - If you rebuild your program, for example, after discovering and fixing bugs, you need not exit dbxtool and restart it. When you click the Run button, dbx detects that the program (or any of its constituents) has been recompiled, and reloads it.
So it is more efficient to simply keep dbxtool on your desktop, perhaps minimized, ready to use on your debugging problems.
So where's the bug? Look at the watches again:
Here you can make the great intuitive leap that argv[0] is NULL because the first call to strtok() returns NULL and that is because the line was empty and had no tokens.
Tip - Shall you fix this bug before proceeding with the remainder of this tutorial?
You can. You can also choose to remember not to press Return and create empty lines.
Or, if you will mostly be running the program under the debugger, you can “patch” it in the debugger, as described in Using Breakpoint Scripts to Patch Your Code.
The developer of the example code should probably have tested for this condition and bypassed the rest of Interp::dispatch().
The above example llustrates the most common debugging pattern, where one stops the misbehaving program at some point before things have gone wrong and then steps through the code comparing the intent of the code with the way the code actually behaves.
Could you have found the bug more directly, without all the stepping and watching? In fact, yes, but you will first have to learn some more techniques for using breakpoints.
This section demonstrates some advanced techniques for using breakpoints:
Using breakpoint counts
Using bounded breakpoints
Picking a useful breakpoint count
Watchpoints
Using breakpoint conditions
Micro replay using pop
Using fix and continue
This section, and the example program, are inspired by an actual bug discovered in dbx using much the same sequence described in this section..
The source code includes a sample input file named in, which triggers a bug in our example program. in contains the following:
display nonexistent_var # should yield an error display var stop in X # will cause one "stopped" message and display stop in Y # will cause second "stopped" message and display run cont cont run cont cont
Notice that there are no empty lines so as not to trigger the bug you discovered in the previous section.
When you run the program with the input file, the output is as follows:
$ a.out < in
> display nonexistent_var
error: Don't know about 'nonexistent_var'
> display var
will display 'var'
> stop in X
> stop in Y
> run
running ...
stopped in X
var = {
a = '100'
b = '101
c = '<error>'
d = '102
e = '103'
f = '104'
}
> cont
stopped in Y
var = {
a = '105'
b = '106'
c = '<error>'
d = '107'
e = '108'
f = '109'
}
> cont
exited
> run
running ...
stopped in X
var = {
a = '110'
b = '111'
error: cannot get value of 'var.c'
c = '<error>'
d = '112'
e = '113'
f = '114'
}
> cont
stopped in Y
var = {
a = '115'
b = '116'
error: cannot get value of 'var.c'
c = '<error>'
d = '117'
e = '118'
f = '119'
}
> cont
exited
> quit
Goodby
This output might seem voluminous but the point of this example is to illustrate techniques to be used with long running, complex programs where stepping through code or tracing just aren't practical.
Notice that when showing the value of field c, you get a value of <error>. Such a situation might occur if the field contains a bad address.
Notice that when you ran the program a second time, you received additional error messages that you didn't get on the first run:
error: cannot get value of 'var.c'
The error() function uses a variable, err_silent, to silence error messages in certain circumstances. For example, in the case of the display command, instead of displaying an error message, problems are displayed as c = '<error>'.
The first step is to set up a debug target and configure
it so the bug can easily be repeated by clicking Run .
Start debugging the program as follows:
If you haven't yet compiled the example program, do so by following the instructions in The Example Program.
Choose Debug > Debug Executable.
In the Debug Executable dialog box, browse for or type in the path to the executable.
In the Arguments field, type:
< in
The directory portion of the executable path is displayed in the Run Directory field.
Click Debug.
In a real world situation, you might want to populate the Environment field as well.
You can change any properties of the configuration by choosing Debug > Configure Current Session.
When debugging a program, dbxtool creates a debug target. You can always use the same debugging configuration by choosing Debug > Debug Recent and then choosing the desired executable.
It is sometimes easier to set many of these properties from the dbx command line. They will be stored in the debug target configuration.
The purpose of much of what follows is to sustain easy repeatability as you add breakpoints so that you can always go to a location of interest by clicking Run without having to click Continue on various intermediate breakpoints.
Let's put a breakpoint inside the error() function in the case where it prints an error message. This breakpoint will be a line breakpoint on line 33.
In a larger program, one can easily change the current function in the Editor window by typing the following, for example, in the Debugger Console window:
(dbx) func error
The lavender stripe indicates the match found by the func command.
Create the line breakpoint by clicking in the left margin of the Editor window on top of the number 33.
Click Run to run the program and upon hitting the breakpoint, look at the stack trace. It shows the error message being emitted due to the simulated command in the in file:
> display var # should yield an error
The call to error() is expected behaviour.
Click Continue 
to continue the process and hit the breakpoint again. This time you receive the unexpected error message.
It would be better to arrive at this location repeatedly on each run without having to click Continue after the first hit of the breakpoint due to the command:
> display var # should yield an error
You can edit the program or input script and eliminate the first troublesome display command. However, the specific input sequence you are working with might be a key to reproducing this bug so let's not perturb the situation.
Because you are interested in the second time you reach this breakpoint let's set its count to 2:
In the Breakpoints window, right-click the breakpoint and choose Customize.
In the Customize Breakpoint dialog box, type 2 in the Count Limit field.
Click OK.
Now you can repeatedly arrive at the location you are interested in.
In this case it was trivial to choose a count of 2. But sometimes the place at which you are interested in stopping is called many times. Later you'll see how you can easily choose a good count value. But for now let's explore another way of stopping in error() only in the invocation you are interested in.
Open the Customize Breakpoint dialog box as before for the breakpoint inside error() and disable breakpoint counts by selecting Always stop from the drop-down list for the Count Limit.
Now click Run and pay attention to the stack trace the two times you stop in error(). The first time it looks like this:
The second time it looks like this:
You'd like to arrange to stop at this breakpoint when it's called all the way from runProgram (frame [7]). To do so, open the Customize Breakpoint dialog box again and set the While In field to runProgram.
Now, again, you can repeatedly and trivially arrive at the point you're interested in.
Why is the unwanted error message being emitted? Obviously it is because err_silent is not > 0. Let's look at the value of err_silent with balloon evaluation. Put your cursor over err_silent in line 31 and wait for its value to be displayed.
Let's follow the stack to see where err_silent was set. Two clicks of
Make Caller Current get you to evaluateField(), which has already called evaluateFieldPrepare()
simulating a complex function that might be manipulating err_silent.
Another click of Make Caller Current gets you to printField(). Here err_silent is being incremented. printField() has also already called printFieldPrepare(), also simulating a complex function that might be manipulating err_silent.
Notice how an err_silent++ and an err_silent-- bracket some code.
So it could be that err_silent goes wrong in either printFieldPrepare() or evaluateFieldPrepare(), or that it is already wrong when control gets to printField().
Let's find out whether it was wrong before or after the call to printField()by putting a breakpoint in printField().
Set a breakpoint in printField().
Select printField(), right-click, and choose New Breakpoint.
The New breakpoint type pre-selected and the Function field pre-populated with printfield.
Click OK.
Click Run . The first time you hit the breakpoint is during the first run, on the first stop, and on the first field, var.a. err_silent is 0, which is OK.
Click Continue. err_silent is still OK.
Click Continue again. err_silent is still OK.
It is going to take a while until you reach the particular call to printField() that resulted in the unwanted error message. You need to use a breakpoint count on the printField breakpoint. But what shall the count be set to? In this simple example, one could attempt to count the runs and the stops and the fields being displayed but in practice things might not be so predictable. However there is a way to figure what the count should be semi-automatically.
Open the Customize Breakpoint dialog box for the breakpoint on printField() and set the Count Limit field to infinity.
This setting means that you will never stop at this breakpoint. However, it will still be counting.
At this point, it will be helpful to have the Breakpoints window show more properties, such as counts.
Click the Change Visible Columns button 
at the top right corner of the Breakpoints window.
Select Count Limit, Count, and While In.
Click OK.
Re-run the program. You will hit the breakpoint inside error(); the one bounded by runProgram().
Now look at the count for the breakpoint on printField().
It's 15 and that is the count you want.
Click the drop-down list in the Count Limit column and select Use current Count value to transfer the current count to the count limit, and press Return.
Now if you run the program you will stop in printField() the last time it's called before the unwanted error message!
Use balloon evaluation to inspect err_silent again. Now it is -1. Most likely one err_silent-- too many, or one err_silent++ too few, was executed before you got to printField().
How can you locate this mismatched pair of err_silents? In a small program like this example it can be done by careful code inspection. But in a large program there might be a prohibitive number of pairings of
err_silent++; err_silent--;
A quicker way to locate the mismatched pair is by using watchpoints.
Tip - It might also be the case that it is not a mismatched set of err_silent++; and err_silent--; at all, but a rogue pointer overwriting the contents of err_silent. Watchpoints would be more effective in catching such a problem.
To create a watchpoint on err_silent:
Select the err_silent variable, right-click, and choose New Breakpoint.
Set Breakpoint Type to Access. Notice how the Settings section changes and how the Address field is & err_silent.
Select After in the When field.
Select Write in the Operation field.
Click OK.
Now run the program. You stop in init(). Things look OK here, that is, err_silent was incremented to 1 and execution stopped after that.
Click Continue. You stop in init() again.
Click Continue again. You stop in init() again.
Click Continue again. You stop in init() again.
Click Continue again. Now you stop in stopIn(). Things look OK here too, that is, no -1s.
It might take a while before err_silent is set to -1 and you don't want to be slavishly clicking Continue lest your eyes glaze over and you miss the time it actually changed to -1. But there is a better way.
To add a condition to your watchpoint:
In the Breakpoints window, right-click the After write breakpoint and choose Customize.
Be sure that After is selected in the When field.
Tip - Selecting After is important because you want to know what the value of err_silent was changed to.
Set the Condition field to err_silent == -1.
Click OK.
Now re-run the program. You stop in checkThings(). This is the first time err_silent is set to -1. As you look for the matching err_silent++ you see what looks like a bug: err_silent is incremented only in the else portion of the function.
Could this be the bug you've been looking for?
Let's double check that you indeed went through the else block of the function.
One way to do so would be to set a breakpoint on checkThings() and run the program. But checkThings() might be called many times. You can use breakpoint counts or bounded breakpoints to get to the right invocation of checkThings() but there's a quicker way to replay what was recently executed.
Choose Debug > Stack > Pop Topmost Call.
Tip - Notice the Pop Topmost Call does not undo everything. In particular, the value of err_silent is already wrong. But this should be okay since you are switching from data debugging to control flow debugging
You find yourself at the call to checkThings(). In fact, the process state has reverted to the beginning of the line containing the call to checkThings().
Now you can click Step Into 
and observe as checkThings() is called
again. As you step through checkThings()(), you can verify that indeed the
process executes the if block where err_silent is not incremented and then is decremented
to -1.
It looks like you have found the programming error. But let's triple check it.
Let's fix the code in place and verify that the bug has indeed gone away.
Fix the code by putting the err_silent++ above the if statement so it looks like this:
Choose Debug > Apply Code Changes.
Disable the printField breakpoint and the watchpoint but leave the breakpoint in error() enabled.
Re-run the program.
You'll note that the program completes without hitting the breakpoint in error() and its output is as expected.
The above still illustrates the same pattern as discussed at the end of Using Breakpoints and Stepping, that is, where one stops the misbehaving program at some point before things have gone wrong and then steps through the code comparing the intent of the code with the way the code actually behaves. The main difference is that finding the point before things have gone wrong is a bit more involved.
In Using Breakpoints and Stepping, you discovered a bug where an empty line yields a NULL first token and causes a SEGV. Here's one way to quickly hack around it:
Delete all of the breakpoints you created in previous sections.
Delete the <in argument in the Debug Executable dialog box.
Toggle a line breakpoint at line 130 in interp.cc:
In the Breakpoints window, right-click the breakpoint you just created (newer breakpoints are added at the bottom) and choose Customize.
In the Customize Breakpoint dialog box, type token == 0 in the Condition field.
Select Run Script from the Action drop-down list.
In the Script field, type assign token = line.
Tip - Why not assign token = “dummy”? Because dbx cannot allocate the dummy string in the debugged process. On the other hand, line is known to be equal to "".
The dialog box should look like this:
Click OK.
Now, if you run the program and enter an empty line, instead of crashing, it will behave like this:
How this works might be clearer if you look at the command that dbxtool sent to dbx:
when at "interp.cc":130 -if token == 0 { assign token = line; }