Linker and Libraries Guide

Debugging Aids

A debugging library and mdb(1) module are provided with the Solaris linkers. The debugging library enables you to trace the runtime linking process in more detail. The mdb(1) module enables interactive process debugging.

Debugging Library

This debugging library helps you understand, or debug, the execution of applications and dependencies. Although the type of information displayed using this library is expected to remain constant, the exact format of the information might change slightly from release to release.

Some of the debugging output might be unfamiliar to those who do not have an intimate knowledge of the runtime linker. However, many aspects may be of general interest to you.

Debugging is enabled by using the environment variable LD_DEBUG. All debugging output is prefixed with the process identifier and by default is directed to the standard error. This environment variable must be augmented with one or more tokens to indicate the type of debugging required.

The tokens available with this debugging option can be displayed by using LD_DEBUG=help. Any dynamic executable can be used to solicit this information, as the process itself terminates following the display of the information. For example:

$ LD_DEBUG=help prog
11693:           For debugging the runtime linking of an application:
11693:                  LD_DEBUG=token1,token2  prog
11693:           enables diagnostics to the stderr.  The additional
11693:           option:
11693:                  LD_DEBUG_OUTPUT=file
11693:           redirects the diagnostics to an output file created
11593:           using the specified name and the process id as a
11693:           suffix.  All diagnostics are prepended with the
11693:           process id.
11693: audit     display runtime link-audit processing
11693: basic     provide basic trace information/warnings
11693: bindings  display symbol binding; detail flag shows
11693:             absolute:relative addresses
11693: detail    provide more information in conjunction with other
11693:             options
11693: files     display input file processing (files and libraries)
11693: help      display this help message
11693: libs      display library search paths
11693: move      display move section processing
11693: reloc     display relocation processing
11693: symbols   display symbol table processing
11693: tls       display TLS processing info
11693: unused    display unused/unreferenced files
11693: versions  display version processing

This example shows the options meaningful to the runtime linker. The exact options might differ from release to release.

The environment variable LD_DEBUG_OUTPUT can be used to specify an output file for use instead of the standard error. The process identifier is added as a suffix to the output file.

Debugging of secure applications is not allowed.

One of the most useful debugging options is to display the symbol bindings that occur at runtime. The following example uses a very trivial dynamic executable that has a dependency on two local shared objects.

$ cat bar.c
int bar = 10;
$ cc -o -K pic -G bar.c
$ cat foo.c
foo(int data)
        return (data);
$ cc -o -K pic -G foo.c
$ cat main.c
extern  int     foo();
extern  int     bar;
        return (foo(bar));
$ cc -o prog main.c -R/tmp:.

The runtime symbol bindings can be displayed by setting LD_DEBUG=bindings:

$ LD_DEBUG=bindings prog
11753: .......
11753: binding file=prog to file=./ symbol bar
11753: .......
11753: transferring control: prog
11753: .......
11753: binding file=prog to file=./ symbol foo
11753: .......

The symbol bar, which is required by an immediate relocation, is bound before the application gains control. Whereas the symbol foo, which is required by a lazy relocation, is bound after the application gains control when the function is first called. This demonstrates the default mode of lazy binding. If the environment variable LD_BIND_NOW is set, all symbol bindings occur before the application gains control.

Setting LD_DEBUG=bindings,detail, provides additional information regarding the real and relative addresses of the actual binding locations.

When the runtime linker performs a function relocation, it rewrites data associated with the functions .plt so that any subsequent calls will go directly to the function. The environment variable LD_BIND_NOT can be set to any value to prevent this data update. By using this variable together with the debugging request for detailed bindings, you can get a complete runtime account of all function binding. The output from this combination can be excessive, in which case the performance of the application is degraded.

You can use LD_DEBUG to display the various search paths used. For example, the search path mechanism used to locate any dependencies can be displayed by setting LD_DEBUG=libs.

$ LD_DEBUG=libs prog
11775: find; searching
11775:  search path=/tmp:.  (RPATH from file prog)
11775:  trying path=/tmp/
11775:  trying path=./
11775: find; searching
11775:  search path=/tmp:.  (RPATH from file prog)
11775:  trying path=/tmp/
11775:  trying path=./
11775: .......

The runpath recorded in the application prog affects the search for the two dependencies and

In a similar manner, the search paths of each symbol lookup can be displayed by setting LD_DEBUG=symbols. If this is combined with a bindings request, you can obtain a complete picture of the symbol relocation process.

$ LD_DEBUG=bindings,symbols
11782: .......
11782: symbol=bar;  lookup in file=./  [ ELF ]
11782: symbol=bar;  lookup in file=./  [ ELF ]
11782: binding file=prog to file=./ symbol bar
11782: .......
11782: transferring control: prog
11782: .......
11782: symbol=foo;  lookup in file=prog  [ ELF ]
11782: symbol=foo;  lookup in file=./  [ ELF ]
11782: binding file=prog to file=./ symbol foo
11782: .......

In the previous example, the symbol bar is not searched for in the application prog. This is due to an optimization used when processing copy relocations. See Copy Relocations for more details of this relocation type.

Debugger Module

The debugger module provides a set of dcmds and walkers that can be loaded under mdb(1). This module can be used to inspect various internal data structures of the runtime linker. Much of this information requires familiarity with the internals of the runtime linker, and may change from release to release. However, some elements of these data structures reveal the basic components of a dynamically linked process and may aid general debugging.

The following examples show some simple scenarios of using mdb(1) with the runtime linker debugger module.

$ cat main.c
#include  <dlfnc.h>

int main()
        void *  handle;
        void (* fptr)();

        if ((handle = dlopen("", RTLD_LAZY)) == NULL)
                return (1);

        if ((fptr = (void (*)())dlsym(handle, "foo")) == NULL)
                return (1);

        return (0);
$ cc -o main main.c -R. -ldl

If mdb(1) has not automatically loaded the debugger module,, explicitly do so. The capabilities of the debugger module can then be inspected.

$ mdb main
> ::load
> ::dmods -l
  dcmd Bind                 - Display a Binding descriptor
  dcmd Callers              - Display Rt_map CALLERS binding descriptors
  dcmd Depends              - Display Rt_map DEPENDS binding descriptors
  dcmd ElfDyn               - Display Elf_Dyn entry
  dcmd ElfEhdr              - Display Elf_Ehdr entry
  dcmd ElfPhdr              - Display Elf_Phdr entry
  dcmd Groups               - Display Rt_map GROUPS group handles
  dcmd GrpDesc              - Display a Group Descriptor
  dcmd GrpHdl               - Display a Group Handle
  dcmd Handles              - Display Rt_map HANDLES group descriptors
  dcmd List                 - Display entries in a List
  dcmd ListRtmap            - Display a List of Rt_Map's
  dcmd Lm_list              - Display Lm_list structure
  dcmd Rt_map               - Display Rt_map structure
  dcmd Rt_maps              - Display list of Rt_map structures
  walk List                 - Walk a List
  walk Rt_maps              - Walk a List of Rt_maps
> ::bp main
> :r

Each dynamic object within a process is expressed as a link-map, Rt_map, which is maintained on a link-map list. All link-maps for the process can be displayed with Rt_maps.

> ::Rt_maps
Link-map lists (dynlm_list): 0xffbfe0d0
  Lm_list: 0xff3f6f60  (LM_ID_BASE)
    Link-map*  ADDR()     NAME()
    0xff3f9040 0x00010000 main
    0xff3f9460 0xff3b0000 /lib/
    0xff3f977c 0xff280000 /lib/
  Lm_list: 0xff3f6f88  (LM_ID_LDSO)
    0xff3f8cc0 0xff3c0000 /lib/

An individual link-map can be displayed with Rt_map.

> 0xff3f9040::Rt_map
Rt_map located at: 0xff3f9040
     NAME: main
     ADDR: 0x00010000         DYN: 0x000207bc
     NEXT: 0xff3f9460        PREV: 0x00000000
      FCT: 0xff3f6f18    TLSMODID:          0
     INIT: 0x00010710        FINI: 0x0001071c
   GROUPS: 0x00000000     HANDLES: 0x00000000
  DEPENDS: 0xff3f96e8     CALLERS: 0x00000000

The object's .dynamic section can be displayed with the ElfDyn dcmd. The following example shows the first 4 entries.

> 0x000207bc,4::ElfDyn
Elf_Dyn located at: 0x207bc
    0x207bc  NEEDED       0x0000010f
Elf_Dyn located at: 0x207c4
    0x207c4  NEEDED       0x00000124
Elf_Dyn located at: 0x207cc
    0x207cc  INIT         0x00010710
Elf_Dyn located at: 0x207d4
    0x207d4  FINI         0x0001071c

mdb(1) is also very useful for setting deferred break points. In this example, it might be useful to put a break point on the function foo(). However, until the dlopen(3DL) of occurs, this symbol isn't known to the debugger. A deferred break point instructs the debugger to set a real breakpoint when the dynamic object is loaded.

> ::bp`foo
> :c
> mdb: You've got symbols!
> mdb: stop at`foo
mdb: target stopped at:`foo:   save      %sp, -0x68, %sp

At this point, new objects have been loaded:

> *`lml_main::Rt_maps
Link-map*  ADDR()     NAME()
0xff3f9040 0x00010000 main
0xff3f9460 0xff3b0000 /lib/
0xff3f977c 0xff280000 /lib/
0xff3f9ca4 0xff380000 ./
0xff37006c 0xff260000 ./

The link-map for shows the handle returned by dlopen(3DL). You can expand this structure using Handles.

> 0xff3f9ca4::Handles -v
HANDLES for ./
  HANDLE: 0xff3f9f60 Alist[used 1: total 1]
    Group Handle located at: 0xff3f9f28
        owner:               ./
        flags: 0x00000000    [ 0 ]
       refcnt:          1    depends: 0xff3f9fa0 Alist[used 2: total 4]
        Group Descriptor located at: 0xff3f9fac
           depend: 0xff3f9ca4    ./
            flags: 0x00000003    [ AVAIL-TO-DLSYM,ADD-DEPENDENCIES ]
        Group Descriptor located at: 0xff3f9fd8
           depend: 0xff37006c    ./
            flags: 0x00000003    [ AVAIL-TO-DLSYM,ADD-DEPENDENCIES ]

The dependencies of a handle are a list of link-maps that represent the objects of the handle that can satisfy a dlsym(3DL) request. In this case, the dependencies are and

Note –

The above examples provide a basic guide to the debugger module capabilities, but the exact commands, usage, and output may change from release to release. Refer to usage and help information for the exact capabilities available on your system.