2 Introduction to Signature Test Tool

You can use the Signature Test tool to easily compare the signatures of two different implementations of the same API. It verifies that all of the members are present, reports when new members are added, and checks the specified behavior of each API member.

2.1 Signature Test Basics

A signature test compares two implementations of an API and reports the differences. The Signature Test tool compares the signatures of two implementations or versions of the same API and can do the following:

Create and run a test that verifies that all of the members are present
Report when new members are added
Check the specified behavior of each API member
Determine whether the old version of the API can be replaced with the newer one without adversely affecting existing clients of the API

The signature test created by the Signature Test tool can be run independently at the command line, or under the control of the JavaTest™ harness.

Note:

For simplicity, this user's guide refers to the test harness as the JavaTest harness. Note that the open source version of the harness, called JT harness, can be used in its place. The JT harness software can be downloaded from this URL: http://jtharness.dev.java.net/

The Signature Test tool was originally created to assist in the creation of Java technology compatibility test suites (TCKs). It simplified the process of verifying that the API signature of a new implementation of a Java technology matched the signature of a reference implementation of that API.

When used in a software development environment, the Signature Test tool can be used to track and control changes to an API throughout the development process.

2.2 What is Tested

The signature test algorithm compares the API implementation under test with a signature file created from the API you are comparing it to — often referred to as a reference implementation. The signature test checks for compatibility by verifying the equality of API member sets. By checking for mutual compatibility of API member sets, the test verifies that the following conditions are true:

If an API item is defined in the reference implementation of the API, then that item is implemented in the API under test, and vice versa.
Attributes chosen for comparison are identical in both implementations of the API. The tool chooses attributes for comparison according to the type of check being processed. This is described more in Section 2.2.1, "Mutual Binary Compatibility Check" and Section 2.2.2, "Mutual Source Compatibility Check".

By checking for migration compatibility of API member sets, the test verifies that the following conditions are true:

If an API item is defined in the previous version of the API, then that item is implemented in the API version under test
If an API item is not defined in the previous version of the API, but added to the API version under test, the added item does not break backward compatibility. This is described more in Section 2.2.3, "Migration Binary and Source Code Check".
Attributes chosen for comparison are identical in both versions of the API or their change does not break backward compatibility. The tool chooses attributes for comparison according to the type of check being processed. This is described more in Section 2.2.3, "Migration Binary and Source Code Check".

2.2.1 Mutual Binary Compatibility Check

The signature test binary compatibility check mode verifies that a Java technology implementation undergoing compatibility testing and its referenced APIs are mutually binary compatible as defined in Chapter 13, “Binary Compatibility,” of The Java Language Specification. This assures that any application runs with any compatible API without any linkage errors.

This check is less strict than the default source compatibility check, described next. It is for use primarily in the special case of when a technology is developed for Java technology environments that are purely runtime. Such an environment does not provide a Java technology-based compiler (Java compiler), nor does it include class files that could be used to compile applications for that environment. Because of the limited use of such an environment, the API requirements are slightly relaxed compared to environments that support application development.

Java application environments can contain several Java technologies. Not all Java technologies can be combined with each other, and in particular, their sets of API signatures might be incompatible with each other. Relaxing signature checks to the level of mutual binary compatibility allows the developer to combine technologies in a purely runtime environment that cannot be combined otherwise.

2.2.2 Mutual Source Compatibility Check

While binary compatibility is important, it cannot guarantee that an application in binary form as a set of class files can be recompiled without error.

The signature test source compatibility check mode verifies that any application that compiles without error with a compatible API, compiles without error with all other source compatible APIs.

Mutual source compatibility is a stricter check than the mutual binary compatibility and the Signature Test tool performs it by default.

2.2.3 Migration Binary and Source Code Check

Mutual compatibility is generally used in certification processes where the goal is to ensure that an alternative or third-party implementation of an API conforms to a reference implementation. Application developers have a different concern, they must ensure that evolving library APIs that their applications link to continue to work with customers' applications. The Signature Test tool can be used to check APIs as they evolve and ensure both binary and source code migration compatibility.

Migration binary compatibility checking ensures that there will be no linkage errors between pre-existing client binaries and the new version. This determination is based on the Chapter 13, “Binary Compatibility,” of The Java Language Specification.

Migration source code compatibility means that pre-existing client source code can be recompiled with the new version without compilation errors.

2.2.4 Class and Class Member Attributes Checked

A Java platform API consists of classes, and interfaces, and their member fields, methods, and constructors, and documented annotations. In turn, all of these API items can have various attributes such as names, modifiers, a list of parameters, a list of interfaces, exceptions, nested classes, and so forth. A signature test checks that certain members and attributes belonging to the API under test are the same as those defined by the API to which it is being compared. Signature test only checks public and protected API items and in the most cases ignores private and package access items. Members with private and package visibility are taken into account when the tool calculates hiding for more visible members.

The tool checks the following attributes when comparing items in the API implementation under test:

Classes and interfaces, including nested classes and interfaces:
- Set of modifiers except strictfp
- Name of the superclass
- Names of all superinterfaces, direct plus indirect, where order is insignificant
Constructors:
- Set of modifiers
- List of argument types
- In source compatibility mode only, the normalized list of thrown exceptions where order is insignificant
  
  Normalizing the throw lists involves removing all superfluous exception classes. An exception class is superfluous if it is a subclass of either the java.lang.RuntimeException class, the java.lang.Error class, or another class from the same list.
Methods:
- The set of modifiers, except strictfp, synchronized, and native
- The return type
- The list of argument types
- In source mode only, the normalized list of thrown exceptions, described earlier, where order is insignificant
Fields:
- Set of modifiers, except transient
- Field type
Documented annotations with SOURCE and RUNTIME retention of the following types:
- Classes and interfaces
- Fields, methods and constructors
- Parameters and annotation types

The tool performs the check in the following order:

For all top-level public and protected classes and interfaces, it compares the attributes of any classes and interfaces with the same fully qualified name.
Taking into account all declared and inherited members, it compares all public and protected members of the same kind and same simple name, treating constructors as class members for convenience sake.

2.3 Source and Binary Compatibility Modes

Earlier Signature Test tool versions performed a comparison of all exceptions declared in throws clauses for methods and constructors. Certain variations in this area caused an error message during the signature test. Despite these error messages, the source files compiled successfully together. Successful compilation is the basic criteria for source compatibility with the current Signature Test tool, while successful linking as the basic criteria for binary compatibility.

Changes to the throws clause of methods or constructors do not break compatibility with existing binaries because these clauses are checked only at compile time, causing no linkage error. For the purpose of signature testing, this relates directly to binary compatibility as described earlier in Section 2.2.1, "Mutual Binary Compatibility Check".

The adaptation of JSR 68, The Java ME Platform Specification, formalized the use of building blocks in API development. A building block is a subset of an existing API that is approved for reuse in the construction of profiles or optional packages. The building block concept enables a developer to duplicate the functionality provided by another API without having to redefine an entirely new API. For further details see JSR 68 at http://www.jcp.org/en/jsr/detail?id=68.

The use of building blocks created a need for more lenient checking of exception throw lists compared to earlier Signature Test tool versions. Consequently, Signature Test tool 2.2 provides both a source and a binary compatibility mode of operation. This retains compatibility with earlier signature files while adding support for building blocks and eliminating the unnecessary error messages.

The SignatureTest command recognizes the -mode option that takes the values “src” or “bin” as arguments for choosing source mode or binary mode. The choice of which mode to use depends on the type of signature file being used in the test. This is described in more detail later in these sections:

Section 3.3, "Setup Command" describes how to generate a signature file
Section 3.4, "SignatureTest Command" describes how to specify the mode when running a signature test
Section 3.6, "Merge Command" describes how to generate a combined signature file from set of signature files

The difference between the binary and source compatibility modes is how the tool handles the throws list for constructors and methods (as described in Section 2.2.4, "Class and Class Member Attributes Checked"). Constant checking behavior is also different in binary and source compatibility modes. Although constant checking can be applied to binary compatibility, it is a necessary prerequisite for source code compatibility. Section 3.1.2, "Constant Checking in Differing Run Modes" describes these differences in more detail.

2.4 Using Custom Signature Loaders

The signature test has a requirement for the Java Platform, Standard Edition (Java SE platform) runtime environment version 1.4 or later. This requirement might prevent use of the tool on limited or nonstandard environments such as some Java Platform, Micro Edition (Java ME platform) or Java Platform, Enterprise Edition (Java EE platform) configurations.

To overcome this, the tool provides support for custom signature loaders that can be implemented as plug-ins. These plug-ins gather signatures from a runtime environment when the SignatureTest command cannot be run directly. For example, you might create a light-weight remote JavaTest harness agent and run the signature loader on a remote Connected Device Configuration (CDC) compatible device. Another example is using a wrapped J2EE platform bean as a signature loader inside a J2EE platform container where any direct file I/O operations are prohibited.

As an aid in developing such an extension, the Signature Test tool distribution includes a class library that contains a signature serializer and some related utility classes in the SigTest-Directory/lib/remote.jar file. This file contains a subset of the Signature Test tool classes that are necessary to develop a custom plug-in. All of these library classes are CDC 1.0 compatible and have minimal memory requirements. The source code for these classes is distributed in the SigTest-Directory/redistributables/sigtest_src.zip file. The code is designed for running a plug-in with the JavaTest harness using the Java ME Framework. The server and client source code and the HTML test descriptions for an actual plug-in example are located in the SigTest-Directory/examples/remote directory.

Note:

The open source version of the ME Framework is available at: http://cqme.dev.java.net/framework.html.