JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Programming Interfaces Guide     Oracle Solaris 11 Information Library
search filter icon
search icon

Document Information


1.  Memory and CPU Management

2.  Remote Shared Memory API for Oracle Solaris Clusters

3.  Session Description Protocol API

4.  Process Scheduler

5.  Locality Group APIs

6.  Input/Output Interfaces

7.  Interprocess Communication

8.  Socket Interfaces

9.  Programming With XTI and TLI

10.  Packet Filtering Hooks

11.  Transport Selection and Name-to-Address Mapping

12.  Real-time Programming and Administration

13.  The Oracle Solaris ABI and ABI Tools

What is the Oracle Solaris ABI?

Defining the Oracle Solaris ABI

Symbol Versioning in Oracle Solaris Libraries

Using Symbol Versioning to Label the Oracle Solaris ABI

Oracle Solaris ABI Tools

appcert Utility

What appcert Checks

Private Symbol Usage

Static Linking

Unbound Symbols

What appcert Does Not Check

Working with appcert

appcert Options

Using appcert for Application Triage

appcert Results

Correcting Problems Reported by appcert

Using apptrace for Application Verification

Application Verification

Running apptrace

Interpreting apptrace Output

A.  UNIX Domain Sockets


Oracle Solaris ABI Tools

The Oracle Solaris operating system provides two tools to verify that an application's use of Oracle Solaris interfaces conforms to the Oracle Solaris ABI. The appcert utility statically examines the Oracle Solaris library interfaces used by ELF binaries for instances of private interface usage. The appcert utility produces summary and detailed reports of any potential binary stability problems it finds. The apptrace tool uses the link-auditing capability of the run-time linker to dynamically trace Oracle Solaris library routine calls as the application runs. This capability enables developers to examine an application's use of the Oracle Solaris system interfaces.

The ABI tools enable easy, rapid identification of binaries that might have binary compatibility problems with a given Oracle Solaris release. To check binary stability, perform the following steps:

appcert Utility

The appcert utility is a Perl script that statically examines ELF binaries and compares the library symbols used against a model of public interfaces and private interfaces in a given Oracle Solaris release. The utility runs on either SPARC or x86 platforms. The utility can check interface usage for both SPARC and x86 3F2-bit interfaces as well as the 64-bit interfaces on SPARC. Note that appcert only examines C language interfaces.

As new Oracle Solaris releases become available, some library interfaces might change their behavior or disappear entirely. These changes can affect the performance of applications that rely on those interfaces. The Oracle Solaris ABI defines runtime library interfaces that are safe and stable for application use. The appcert utility is designed to help developers verify an application's compliance with the Oracle Solaris ABI.

What appcert Checks

The appcert utility examines your applications for:

Private Symbol Usage

Private symbols are functions or data that is used by Oracle Solaris libraries to call each other. The semantic behavior of private symbols might change, and symbols might sometimes be removed. Such symbols are called demoted symbols. The mutable nature of private symbols introduces the potential for instability in applications that depend on private symbols.

Static Linking

The semantics of private symbol calls between Oracle Solaris libraries might change between releases. Therefore, the creation of static links to archives degrades an application's binary stability. Dynamic links to the archive's corresponding shared object file avoid this problem.

Unbound Symbols

The appcert utility uses the dynamic linker to resolve the library symbols that are used by the application being examined. Symbols that the dynamic linker cannot resolve are called unbound symbols. Unbound symbols might be caused by environment problems, such as an incorrectly set LD_LIBRARY_PATH variable. Unbound symbols might also be caused by build problems, such as omitting the definitions of the -llib or -z switches at compile time. While these examples are minor, unbound symbols that are reported by appcert might indicate a more serious problem, such as a dependency on a private symbol that no longer exists.

What appcert Does Not Check

If the object file appcert is examining depends on libraries, those dependencies must be recorded in the object. To do so, be sure to use the compiler's -l switch when compiling the code. If the object file depends on other shared libraries, those libraries must be accessible through LD_LIBRARY_PATH or RPATH at the time you run appcert.

The appcert application cannot check 64–bit applications unless the machine is running the 64–bit Solaris kernel. Since Solaris provides no 64–bit static libraries, appcert does not perform static-linking checks on 64–bit applications.

The appcert utility cannot examine:

Working with appcert

To check your application with appcert, type:

appcert object|directory

replacing object|directory with either:

Note - You might run appcert in an environment that is different from the environment in which the application runs. If these environments are different, appcert might not be able to correctly resolve references to Oracle Solaris library interfaces.

The appcert utility uses the Oracle Solaris runtime linker to construct a profile of interface dependencies for each executable or shared object file. This profile is used to determine the Oracle Solaris system interfaces upon which the application depends. The dependencies that are outlined in the profile are compared to the Oracle Solaris ABI to verify conformance. No private interfaces should be found.

The appcert utility recursively searches directories for object files, ignoring non-ELF object files. After appcert has finished checking the application, appcert prints a rollup report to the standard output, usually the screen. A copy of this report is written in the working directory, which is usually /tmp/, in a file that is named Report. In the subdirectory name, pid represents the 1–to–6 digit process ID of that particular instantiation of appcert. See appcert Results for more on the directory structure to which appcert writes output files.

appcert Options

The following options modify the behavior of the appcert utility. You can type any of these options at the command line, after the appcert command but before the object|directory operand.


Runs appcert in batch mode.

In batch mode, the report produced by appcert contains one line for each binary checked.

A line that begins with PASS indicates the binary that is named in that line did not trigger any appcert warnings.

A line that begins with FAIL indicates problems were found in that binary.

A line that begins with INC indicates the binary that is named in that line could not be completely checked.

-f infile

The file infile should contain a list of files to check, with one file name per line. These files are added to any files already specified at the command line. If you use this switch, you do not need to specify an object or directory at the command line.


Prints usage information for appcert.


By default, appcert notes any shared objects in an application, and appends the directories in which the shared objects reside to LD_LIBRARY_PATH. The -L switch disables this behavior.


By default, appcert follows symbolic links when appcert searches directories for binaries to check. The -n switch disables this behavior.


Appends the Oracle Solaris library directories /usr/openwin/lib and /usr/dt/lib to LD_LIBRARY_PATH.

-w working_dir

Specifies a directory in which to run the library components. Temporary files are also created in the directory specified by this switch. If this switch is not specified, appcert uses the /tmp directory.

Using appcert for Application Triage

The appcert utility can be used to quickly and easily discern which applications in a given set have potential stability problems. If appcert does not report any stability problems, the application is not likely to encounter binary stability problems in subsequent Solaris releases. The following table lists some common binary stability problems.

Table 13-1 Common Binary Stability Problems

Course of Action
Use of a private symbol that is known to change
Eliminate use of symbol immediately.
Use of a private symbol that has not changed yet
Application can still be run for now, but eliminate use of symbol as soon as practical.
Static linking of a library with a shared object counterpart
Use shared object counterpart instead.
Static linking of a library with no shared object counterpart
Convert .a file to .so file by using the command ld -z allextract if possible. Otherwise, continue to use static library until shared object is available.
Use of a private symbol for which no public equivalent is available
Contact Sun and request a public interface.
Use of a symbol that is deprecated, or use of a symbol that is planned for removal
Application can still be run for now, but eliminate use of symbol as soon as practical.
Use of a public symbol that has changed

Potential stability problems caused by the use of private interfaces might not occur on a given release. The behavior of private interfaces does not always change between releases. To verify that a private interface's behavior has changed in the target release, use the apptrace tool. Usage of apptrace is discussed in Using apptrace for Application Verification.

appcert Results

The results of the appcert utility's analysis of an application's object files are written to several files that are located in the appcert utility's working directory, typically /tmp. The main subdirectory under the working directory is, where pid is the process ID for that instantiation of appcert. The appcert utility's results are written to the following files:


Contains the mapping between checked binaries and the subdirectory in which appcert output specific to that binary is located.


Contains a copy of the rollup report that is displayed on stdout when appcert is run.


Contains a list of binaries that appcert was asked to check but was forced to skip, along with the reason each binary was skipped. These reasons are in the following list:

  • File is not a binary object

  • File cannot be read by the user

  • File name contains metacharacters

  • File does not have the execute bit set


A separate subdirectory is under the objects subdirectory for each object examined by appcert. Each of these subdirectories contains the following files:


Contains a list of symbols that appcert suspects are demoted Oracle Solaris symbols.


Contains a list of private Oracle Solaris symbols to which the object is directly bound.


Contains a list of public Oracle Solaris symbols to which the object is directly bound.


Contains a list of symbols not bound by the dynamic linker when running ldd -r. Lines returned by ldd containing “file not found” are also included.


Contains a printer-formatted summary of dynamic bindings in the objects appcert examined, including tables of public and private symbols used from each Oracle Solaris library.

Returns one of four exit values.


No potential sources of binary instability were found by appcert.


The appcert utility did not run successfully.


Some of the objects checked by appcert have potential binary stability problems.


The appcert utility did not find any binary objects to check.

Correcting Problems Reported by appcert

Using apptrace for Application Verification

The apptrace utility is a C program which dynamically traces calls to Oracle Solaris library routines as an application runs. apptrace works on either SPARC or x86 platforms. apptrace can trace interface calls for both SPARC and x86 32-bit interfaces, as well as the 64-bit interfaces on SPARC. As with appcert, apptrace only examines C language interfaces.

Application Verification

After using appcert to determine an application is at risk of binary instability, apptrace helps assess the degree of risk in each case. To determine an application's binary compatibility with a given release, verify the successful use of each interface used by the application with apptrace.

The apptrace utility can verify that an application is using public interfaces correctly. For example, an application that is using the open() to open the administrative file /etc/passwd directly should instead use the appropriate programmatic interfaces. This ability to inspect the usage of the Oracle Solaris ABI enables easy and rapid identification of potential interface problems.

Running apptrace

The apptrace utility does not require any modification of the application being traced. To use apptrace, type apptrace, followed by any desired options along with the command line used to run the application of interest. The apptrace utility works by using the link-auditing capability of the runtime linker to intercept the application's calls to Oracle Solaris library interfaces. The apptrace utility then traces the calls by printing the names and values of the call's arguments and return value. The tracing output can be on a single line or arranged across multiple lines for readability. Public interfaces are printed in human-readable form. Private interfaces are printed in hexadecimal.

The apptrace utility enables selective tracing of calls, both at the level of individual interfaces and the level of libraries. For example, apptrace can trace calls to printf() coming from libnsl, or a range of calls within a specific library. The apptrace utility can also verbosely trace user-specified calls. The specifications that dictate apptrace behavior are governed by a syntax that is consistent with the usage of truss(1). The -f option directs apptrace to follow forked child processes. The -o option specifies an output file for apptrace results.

The apptrace utility traces only library-level calls and is loaded into the running application process, gaining a performance increase over truss. With the exception of printf, apptrace cannot trace calls to functions that accept variable argument lists or examine the stack or other caller information, for example, setcontext, getcontext, setjmp, longjmp, and vfork.

Interpreting apptrace Output

The following examples contain sample apptrace output from tracing a simple one-binary application, ls.

Example 13-1 Default Tracing Behavior

% apptrace ls /etc/passwd
ls       -> = 0xff3cb8f0) = 0x0
ls       -> = 0x129a4) = 0x0
ls       -> = 0x32c3
ls       -> = 0x23918) = 0x3b2fe4ef
ls       -> = 0x1) = 0x1
ls       ->, 0x540d, 0xffbff7ac)
ls       ->, 0x5468, 0x23908)
ls       -> = 0x6, locale = "") = "C"
ls       -> = 0x1, elsize = 0x40) = 0x23cd0
ls       -> = "/etc/passwd", buf = 0xffbff6b0) = 0x0
ls       -> = "/etc/passwd", cmd = 0x3, nentries = 0x0,
             aclbufp = 0x0) = 0x4
ls       -> = 0x23cd0, nel = 0x1, width = 0x40,
             compar = 0x12038)
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
ls       -> = "") = 0x0
ls       -> = "/etc/passwd") = 0xb
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
ls       -> = "") = 0x0
ls       -> = 0x12ab8, ...) = 11
ls       ->
format = 0x12abc, ...) = 1
ls       -> = 0)

The previous example shows the default tracing behavior, tracing every library call on the command ls /etc/passwd. The apptrace utility prints a line of output for every system call, indicating:

The output from ls is mixed in with the apptrace output.

Example 13-2 Selective Tracing

% apptrace -t \*printf ls /etc/passwd
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
ls       -> = 0x12ab8, ...) = 11
ls       ->
format = 0x12abc, ...) = 1

The previous example shows how apptrace can selectively trace calls with regular-expression syntax. In the example, calls to interfaces ending in printf, which include sprintf, are traced in the same ls command as before. Consequently, apptrace only traces the printf and sprintf calls.

Example 13-3 Verbose Tracing

% apptrace -v sprintf ls /etc/passwd
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
  buf =    (char *) 0x233d0 ""
  format = (char *) 0x12af8 "%s%s%s"
ls       -> = 0x233d0, format = 0x12af8, ...) = 0
  buf =    (char *) 0x233d0 ""
  format = (char *) 0x12af8 "%s%s%s"

The previous example shows the verbose tracing mode, where the arguments to sprintf are printed on multiple output lines for readability. At the end, apptrace displays the output of the ls command.