Skip Headers
JavaTest Harness Architect's Guide,
JavaTest Harness 4.4.1 for the Java Platform
  Go To Table Of Contents
Go To Index


3 Overview

JavaTest test suites are comprised of a number of components, many of which you, as the test suite architect, provide. This chapter introduces you to these components and some underlying concepts that are discussed in much greater detail later in this manual.

Test Suite Components

The most fundamental components of a test suite are the tests themselves. Tests are typically Java programs that exercise aspects of an API or compiler. To work well with the JavaTest harness, these files are organized in the file system hierarchically. The JavaTest harness finds the tests and displays them in the JavaTest GUI test tree based on this hierarchy.

Before the JavaTest harness can execute a test, it must know some fundamental things about the test — for example, where to find the class file that implements the test and what arguments the test takes. This information is contained in a test description. The test description is a group of name/value pairs that can be embodied in different ways — for example, as tag test descriptions and HTML test descriptions. Tag test descriptions are inserted directly into the test source files using Javadoc style tags. HTML test descriptions are HTML tables contained in HTML files separate from the test source and class files. The examples included with the JavaTest Architect's release demonstrate both types of test descriptions.

The JavaTest harness uses a specialized class called a test finder to locate tests, read test descriptions, and pass test description values to the JavaTest harness. As the architect, you specify a test finder that knows how to read the test descriptions you have designed for your test suite. The JavaTest Architect's release includes test finders that know how to read tag and HTML test descriptions; you can use the included test finders as-is, modify them, or create your own.

Once the test finder locates the test and reads the test description, it is up to the test script to actually run the test. The test script is a Java class whose job is to interpret the test description values, run the tests, and report the results back to the JavaTest harness. As the test suite architect, you are responsible for providing the test script that JavaTest uses. Test scripts can be very simple or complex, depending on the requirements of your test suite. A number of test script examples are included with the JavaTest Architect's release that you can use as is, extend, or use as a template for your test script.

In addition to the fundamental information about each test, the test script might also require platform-specific information about each test to execute it. This information is provided by the person running the tests, usually by completing a wizard-like configuration interview designed by you. Platform-specific information includes information such as the location of the JVM to be used when running the tests, the names of remote computers, and other resources required to run the tests.

When test execution is complete, a test normally creates a Status object and passes it back to the test script; the test script then stores the test results in the TestResult object. Test status includes an integer that represents the status (pass, fail, error, not run) and a short message that describes what happened — for example, an error message. Test results include more detailed information about the results of the test's execution — for example, any additional messages produced by the test.

When the JavaTest harness loads a test suite, the first thing it does is read a file named testsuite.jtt located in the top-level directory of the test suite. The testsuite.jtt file is a registry of information about the test suite that includes the paths to the components described above and other static information about the test suite. The JavaTest harness internalizes this information in a TestSuite object that acts as a portal to all information about the test suite. Whenever the JavaTest harness requires information about the test suite, it queries the TestSuite object. As test suite architect, you create and maintain the testsuite.jtt file.

Figure 3-1 shows a graphical representation of the different test suite components:

Figure 3-1 Test Suite Components

Description of Figure 3-1 follows
Description of "Figure 3-1 Test Suite Components"

The following table summarizes the sequence of steps the JavaTest harness uses to run test suites and your responsibility for each step. These steps are described in more detail in the following chapters.

Table 3-1 Summary of JavaTest Harness Operation 

Events Architect's Role Notes


The user starts the JavaTest harness

Optionally, create a wrapper command to start the JavaTest harness in the right location and with the correct arguments.


The JavaTest harness reads testsuite.jtt to gather information about the test suite including the names and class paths for classes such as the finder, test script, and configuration interview. The JavaTest harness calls the TestSuite class, which in turn creates the TestSuite object. The JavaTest harness passes information from testsuite.jtt to the TestSuite class when it is created.

Determine what information is included in testsuite.jtt and what information (if any) is built directly into your test suite class. The components you create, including your test suite class are contained in a Java archive (JAR) file installed in the test suite. The path to the JAR file is specified in the testsuite.jtt file.

Provide your test suite class


The JavaTest harness queries the TestSuite object to determine which test finder to use.

Provide your test finder class


The JavaTest harness starts the test finder. The test finder reads test descriptions and creates TestDescription objects. The JavaTest GUI displays the test tree.



The user starts the test run. If the configuration information is incomplete, the JavaTest harness activates the configuration editor.

Provide the configuration interview


The JavaTest harness asks the TestSuite object to create a fresh copy of the test script for each test. The test script runs tests according to the information in the test description and environment. When the test is complete, the test script reports the test's exit status to the JavaTest harness.

Design the test execution model and create the test script


The test suite updates the TestResult object with the results of the test execution and writes the test results to a file in the work directory. Test results are described in Test Result.



The JavaTest harness updates the GUI and/or displays information at the command line.


Remote Execution

It is often convenient or necessary to run tests on a system other than the one running the JavaTest harness. In this case, an agent must be used to run the tests on the test platform and communicate with JavaTest harness. The JavaTest harness provides a general purpose agent (JavaTest Agent), but test architects can also create custom agents.

The JavaTest Agent is a lightweight program compatible with JDK 1.1 (does not require the Java SE platform, or Swing). The JavaTest Agent uses a bidirectional connection to communicate between the test platform and JavaTest—it supports both the TCP/IP and RS-232 protocols. Other types of connections can be added through the JavaTest API, for example, infrared, parallel, USB, firewire connections can be added and modelled on the existing system. If a test platform meets the following requirements the JavaTest Agent will probably work well:

If the test platform does not meet these requirements, the JavaTest API can be used to create a custom agent. Agents have been created to run tests on devices such as cell phones, PDAs, and pagers.