Debugging a Program With dbx

Troubleshooting Tips

After error checking has been enabled for a program and the program is run, one of the following errors may be detected:

librtc.so and dbx version mismatch; Error checking disabled

This may occur if you are using RTC on an attached process and have set LD_PRELOAD to a version of librtc.so other than the one shipped with your Sun WorkShop dbx image. To fix this, change the setting of LD_PRELOAD.

patch area too far (8mb limitation); Access checking disabled

RTC was unable to find patch space close enough to a load object for access checking to be enabled. See "rtc_patch_area".

RTC's Eight Megabyte Limit

When access checking, dbx replaces each load and store instruction with a branch instruction that branches to a patch area. This branch instruction has an eight megabyte range. This means that if the debugged program has used up all the address space within eight megabytes of the particular load/store instruction being replaced, there is no place to put the patch area.

If RTC can't intercept all loads and stores to memory, it cannot provide accurate information and so disables access checking completely. Leak checking is unaffected.

dbx internally applies some strategies when it runs into this limitation and continues if it can rectify this problem. In some cases dbx cannot proceed; when this happens it turns off access checking after printing an error message.

Working Around the Eight Megabyte Limit

Note -

On V9, you can work around the eight megabyte limit by using the setenv command to set the USE_FASTTRAPS environment variable to 1. This workaround makes dbx run more slowly and use more memory.

dbx provides some possible workarounds to users who have run into this limit. These workarounds require the use of a utility called rtc_patch_area which is included with Sun WorkShop.

This utility creates object files or shared object files that can be linked into your program to create patch areas for RTC to use.

There are two situations that can prevent dbx from finding patch areas within 8 megabytes of all the loads and stores in an executable image:

Case 1: The statically linked a.out file is too large.

Case 2: One or more dynamically linked shared libraries is too large.

When dbx runs into this limitation, it prints a message telling how much patch space it needs and directs you to the appropriate case (Case 1 or Case 2).

Case 1

In Case 1, you can use rtc_patch_area to make one or more object files to serve as patch areas and link them into the a.out.After you have seen a message like the following:

Enabling Error Checking... dbx: warning: rtc: cannot find patch space within 8Mb (need 6490432 bytes for ./a.out)
dbx: patch area too far (8Mb limitation); Access checking disabled
         (See `help rtc8M', case 1)

Create an object file patch.o for a patch area with a size less than or equal to 8 megabytes:
```
rtc_patch_area -o patch.o -size 6490432
```
The -size flag is optional; the default value is 8000000.

If the size request from the error message is satisfied, continue to the next step. Otherwise, repeat step 1 and create more .o files as needed.

Relink the a.out, adding the patch.o files to the link line.

Try RTC again with the new binary.

If RTC still fails, you may try to reposition the patch.o files on the link line.

An alternate workaround is to divide the a.out into a smaller a.out and shared libraries.

Case 2

If you are running RTC on an attached process, see case 2a. Otherwise, the only workaround is to rebuild the shared library with extra patch space:

After you have seen a message like the following::

Enabling Error Checking... dbx: warning: rtc: cannot find patch space within 8Mb (need 563332 bytes for ./sh1.so)
dbx: patch area too far (8Mb limitation); Access checking disabled
         (See `help rtc8M', case 2)

Create an object file patch.o for a patch area with a size less than or equal to 8 megabytes:
```
rtc_patch_area -o patch.o -size 563332
```
The -size flag is optional; the default value is 8000000.

If the size request from the error message is satisfied, continue to the next step. Otherwise, repeat step 1 and create more .o files as needed.

Relink sh1.so, adding the patch.o files to the link line.

Try RTC again with the new binary; if dbx requests patch space for another shared library, repeat steps 1-3 for that library.

Case 2a

In Case 2a, if the shared library patch space requested by dbx is eight megabytes or less, you can use rtc_patch_area to make a shared library to serve as a patch area and link it into the a.out.

After you have seen a message like the following:

Enabling Error Checking... dbx: warning: rtc: cannot find patch space within 8Mb (need 563332 bytes for ./sh1.so)
dbx: patch area too far (8Mb limitation); Access checking disabled
         (See `help rtc8M', case 2)

Create a shared object patch1.so for a patch area:
```
rtc_patch_area -so patch1.so -size 563332
```
The -size flag is optional; the default value is 8000000.

2) Relink the program with patch1.so placed adjacent to sh1.so in the link line. If dbx still requests patch space for the same sh1.so, then try placing patch1.so immediately before sh1.so.

It may be necessary to use full pathnames instead of the ld -l option to get the desired shared library ordering.

Try RTC again with the new binary; if dbx requests patch space for another shared library, repeat steps 1-2 for that library.

If the patch space requested by dbx is more than eight megabytes for a given shared library, follow the steps in case 2 above.

`rtc_patch_area`

rtc_patch_area is a shell script that creates object files or shared library files that can be linked into the user's program to add patch area space to programs with large text, data, or bss images.

The object file (or shared library) created contains one RTC patch area of the specified size or 8000000 if size is not supplied.

The name of the resulting object file (or shared library) is written to the standard output. Either the -o or -so option must be used.

Specify the name of the shared library to be created. This name is then written to the standard output:

-so sharedlibname

Specify the name of the object file to be created. This name is then written to the standard output:

-o objectname

Create a patch area of size bytes (default and reasonable maximum is 8000000):

-size size

Use compiler instead of cc to build the object file:

-cc compiler

Examples

Generate a standard eight megabyte patch area object file:

rtc_patch_area -o patch.o

Generate an object file containing a 100,000 byte patch:

rtc_patch_area -size 100000 -o patch.o

Generate a one megabyte patch area shared library:

rtc_patch_area -so rtc1M.so -size 1000000