This section describes the limitations of runtime checking.
Access checking requires some symbol information in the load objects. When a load object is fully stripped, runtime checking might not catch all of the errors. Read from uninitialized (rui) memory errors might be incorrect and therefore are suppressed. You can override the suppression with the unsuppress rui command. To retain the symbol table in the load object, use the –x option when stripping a load object.
Runtime checking cannot catch all array out-of-bounds errors. Bounds checking for static and stack memory is not available without debug information.
Runtime checking instruments memory access instructions for access checking. These instructions are handled by a SIGSEGV handler at runtime. Because runtime checking requires its own SIGSEGV handler and signal alternate stack, an attempt to install a SIGSEGV handler or SIGALTSTACK handler results in an EINVAL error or ignoring the attempt.
SIGSEGV handler calls cannot be nested. Doing so results in the error terminating signal 11 SEGSEGV. If you receive this error, use the rtc skippatch command to skip instrumentation of the affected function.
Two problems might arise if a sufficient patch area is not available within 8 megabytes of all existing code.
Slowness
When access checking is enabled, dbx replaces each load and store instruction with a branch instruction that branches to a patch area. This branch instruction has an 8-megabyte range. If the debugged program has used all the of address space within 8 megabytes of the particular load or store instruction being replaced, no place exists to put the patch area. In this case, dbx invokes a trap handler instead of using a branch. The transfer of control to a trap handler is significantly slower (up to 10 times), but does not suffer from the 8 megabyte limit.
Out register override problem in V8+ mode
The trap handler limitation affects access checking if both of the following conditions apply:
The process being debugged is instrumented using traps.
The process uses the V8+ instruction set.
The problem occurs because the sizes of out registers and in registers on V8+ architecture are different. Out registers are 64 bits long, while in registers are only 32 bits long. When a trap handler is invoked, out registers are copied into in registers and the higher 32 bits are lost. Therefore, if the process being debugged uses the higher 32 bits of out registers, the process might run incorrectly when access checking is enabled.
The compilers use the V8+ architecture by default when creating 32-bit SPARC based binaries, but you can tell the compilers to use the V8 architecture with the –xarch option. Unfortunately, system runtime libraries are unaffected by recompiling your application.
dbx automatically skips instrumentation of the following functions and libraries that are known not to work correctly when instrumented with traps:
server/libjvm.so
client/libjvm.so
`libfsu_isa.so`__f_cvt_real
`libfsu_isa.so`__f90_slw_c4
However, skipping instrumentation might result in incorrect RTC error reports.
If either of the above conditions applies to your program and the program starts to behave differently when you enable access checking, the trap handler limitation probably affects your program. To work around the limitation, you can do the following:
Use the rtc skippatch command to skip instrumentation of the code in your program that uses the functions and libraries listed above. Generally, tracking the problem to a specific function is difficult, so you might want to skip instrumentation of an entire load object. The rtc showmap command displays a map of instrument types sorted by address.
Try using 64-bit SPARC-V9 instead of 32-bit SPARC-V8.
If possible, recompile your program for V9 architecture, in which all of the registers are 64 bits long.
Try adding patch area object files.
You can use the rtc_patch_area shell script to create special .o files that can be linked into the middle of a large executable or shared library to provide more patch space. For more information, see the rtc_patch_area(1) man page.
When dbx reaches the 8-megabyte limit, it tells you which load object was too large (the main program or a shared library) and displays the total patch space needed for that load object.
For the best results, the special patch object files should be evenly spaced throughout the executable or shared library, and the default size (8 megabytes) or smaller should be used. Also, do not add more than 10-20% more patch space than dbx says it requires. For example, if dbx says that it needs 31 megabytes for a.out, then add four object files created with the rtc_patch_area script, each one 8 megabytes in size, and space them approximately evenly throughout the executable.
When dbx finds explicit patch areas in an executable, it prints the address ranges spanned by the patch areas, which can help you to place them correctly on the link line.
Try dividing the large load object into smaller load objects.
Split up the object files in your executable or your large library into smaller groups of object files, then link them into smaller parts. If the large file is the executable, then divide it into a smaller executable and a series of shared libraries. If the large file is a shared library, then rearrange it into a set of smaller libraries.
This technique enables dbx to find space for patch code in between the different shared objects.
Try adding a “pad” .so file.
This solution should be necessary only if you are attaching to a process after it has started up.
The runtime linker might place libraries so close together that patch space cannot be created in the gaps between the libraries. When dbx starts the executable with runtime checking enabled, it asks the runtime linker to place an extra gap between the shared libraries. However, when attaching to a process that was not started by dbx with runtime checking enabled, the libraries might be too close together.
If the runtime libraries are too close together and if you cannot start the program using dbx, then you can try creating a shared library using the rtc_patch_area script and linking it into your program between the other shared libraries. See the rtc_patch_area(1) man page for more details.