Skip Headers
JavaTest Harness Architect's Guide,
JavaTest Harness 4.4.1 for the Java Platform
  Go To Table Of Contents
Go To Index


4 Creating a Test Suite

This chapter leads you through the process of creating a very small working test suite quickly and easily by following step-by-step instructions. To simplify the process, conceptual information is generally not provided but is available in later chapters.

The test suite you create here can serve as the basis for your entire test suite. If your tests have no special requirements that the Standard Test Finder and Standard Test Script cannot accommodate, you may be able to create your product test suite by simply adding additional tests and creating a configuration interview to gather the information required to execute your tests.


This chapter describes how to:

  1. Create a test suite directory

  2. Create a testsuite.jtt file

  3. Copy javatest.jar to the test suite lib directory

  4. Add appropriate classes to the classes directory

  5. Create a test

  6. Run the test suite

Other issues of interest regarding test suite creation are discussed at the end of the chapter.

Create a Test Suite

To create a test suite, follow the steps in these simple tasks.

  1. Create a Test Suite Directory

  2. Create the testsuite.jtt File

  3. Copy javatest.jar

  4. Set Up the classes Directory

  5. Use a Simple Test Template

  6. Create and Compile a Simple Test Example

Create a Test Suite Directory

Create the directory and sub-directories for your test suite.

  1. Create the top-level test suite directory.

    Create the directory somewhere convenient in your file system. This directory is referred to as ts_dir for the remainder of this chapter.

  2. Under ts_dir, create sub-directories named tests, lib, and classes.

Create the testsuite.jtt File

As described in Chapter 3, the JavaTest harness reads the testsuite.jtt file to find out information about your test suite. The following steps describe how to create the testsuite.jtt file for this test suite.

  1. Make ts_dir the current directory.

  2. Create the testsuite.jtt file.

    Enter the following information into a text editor:

    # Test Suite properties file for DemoTCK test suite 
    # with tag-style tests
    name=My Test Suite

    You can substitute your own string values for the name and id properties.


    The classpath entry is not used here because the Standard Test Finder, Standard Test Script, and Simple Interview classes are all contained within javatest.jar which is automatically on the class path. If you create your own components, you must include the classpath entry to point to the JAR file that contains these classes. See Chapter 8 for more information about the testsuite.jtt file.

    Save the file as ts_dir\testsuite.jtt.

Copy javatest.jar

The test suite works best if there is a copy of the javatest.jar file in the lib directory of the test suite; this enables the JavaTest harness to automatically locate the test suite.

  1. Copy javatest.jar from jt_install\examples\javatest\simpleTags\demotck\lib to ts_dir\lib.


    The javatest.jar file contains the SimpleInterview class that is used with this test suite (com.sun.javatest.SimpleInterview.Parameters). As your test suite becomes more complicated and customized, you may need to create a custom interview. See Chapter 6 for more information.

Set Up the classes Directory

In order to execute, tests must have access to some of the classes contained in javatest.jar. Extracting these classes eliminates the need for each test to have javatest.jar on its class path. The the most convenient location to place these classes is the ts_dir\classes directory.

  1. Make ts_dir\classes the current directory.

  2. Verify that the Java SE platform (version 1.6 or later) is in your path.

    At a command prompt, enter:

    C:\> java -version
  3. From javatest.jar, extract the classes required to run the tests.

    Use the following command line:

    jar -xvf ..\lib\javatest.jar com\sun\javatest\Test.class


    As your test suite become more complex, you may have to add additional libraries to the classes directory.

Use a Simple Test Template

The following instructions describe how to create a very simple test to add to your test suite. For more detailed instructions about writing TCK tests, see the Test Suite Developers Guide.

  1. Make ts_dir\tests the current directory.

  2. Enter the test code into your favorite text editor.

    The following template can be used as the basis for writing simple tests:

    import com.sun.javatest.Status;
    import com.sun.javatest.Test;
    /** @test
      * @executeClass MyTest
      * @sources
    public class MyTest implements Test{
        public static void main(String[] args) {
            PrintWriter err = new PrintWriter(System.err, true);
            Test t = new MyTest();
            Status s =, null, err);
        public Status run(String[] args, PrintWriter log1, PrintWriterlog2) {
            Status result;
            // your test code here ...
            return result;

    Note that the section delimited with the /** **/ characters is the test description portion of the test. It must be present for the JavaTest harness to locate and recognize the test. You will change all instances of MyTest, and replace the line // your test code here... with your own code. The following table describes the test description entries recognized by the Standard Test Script:

    Test Description Entry Description
    test Identifies the comment block as a test description and the containing file as a test
    executeClass Specifies the name of the test's executable class file (assumed to be located in the classes directory)
    executeArgs Specifies arguments (if any) that the test accepts
    sources Names the source files required to compile the test. This entry is required if you use the JavaTest harness to compile your tests. See Chapter 7 for more information. This tag is also used by the JavaTest harness to display a test's sources in the Files tab of the Test pane.
    keywords Specifies user-defined keywords that direct the JavaTest harness to include or exclude tests from a test run.

    You can create simple tests by replacing the comment:

    // your test code here ...

    with code that tests your API. Note that the test must return a Status object as a result.


    You can find examples of simple tests at: jt_install\examples\javatest\simpleTags\demotck\tests

Create and Compile a Simple Test Example

The following sample is a very simple test you can use to get started.

  1. Save the following file as

    Be sure to copy the entire file, including the test description delimited with the /** **/ characters

    import com.sun.javatest.Status;
    import com.sun.javatest.Test;
    /** @test    
      * @executeClass MyTest
      * @sources
    public class MyTest implements Test {
        public static void main(String[] args) {
            PrintWriter err = new PrintWriter(System.err, true);
            Test t = new MyTest();
            Status s =, null, err);
        public Status run(String[] args, PrintWriter log1, PrintWriter log2) {
            Status result;
            if (1 + 1 == 2)
                result = Status.passed("OK");
                result = Status.failed("Oops");
            return result;
  2. Compile

    Use the following command on WIN32 systems:

    C:\> javac -d ..\classes -classpath ..\classes

    Use the following command on Solaris or Linux systems:

    % javac -d ../classes -classpath ../classes

    MyTest.class is created in the ts_dir\classes directory. As you add more and more tests you should organize them hierarchically in subdirectories.


    As you add more and more tests, you may want to use the JavaTest harness to compile the tests. For more information, see Chapter 7.

Run a Test Suite

You are now ready to run the test suite.

  1. Make ts_dir the current directory.

  2. Start the JavaTest harness.

    At a command prompt enter:

    c:\> java -jar lib\javatest.jar -newdesktop


    The -newdesktop option is used here to ensure that the JavaTest harness starts up like it did in the tutorial — under normal circumstances you should not use this option. For information about JavaTest options, see the online help.

  3. Run the tests the same way you ran the tests in Chapter 2.

    The configuration interview for this test suite contains a question not included in the tutorial configuration interview. Use the following information to answer the question:

    Question Title Answer Description
    Class Path ts_dir\classes The test uses library classes located in the classes directory.

    Click the Add button to activate a file chooser. Select the classes directory and click the Add File button.

Odds and Ends

This section takes a closer look at the components that make up a typical test suite and how they are organized. In addition, the various class paths required to run the JavaTest harness, the agent, and tests classes are discussed.

Note that much of the organization described here is optional; however, experience has shown that it works well for most test suites.

Top-Level Test Suite Directory

The top-level test suite directory generally contains the following files and directories:

Table 4-1 Top-Level Test Suite Files and Directories  

File/Directory Description


A text file that serves as a registry of information about the test suite. This files includes the paths to plug-in components (for example, the test finder, test script, or configuration interview) as well as other static information about the test suite. The presence of this file defines the top-level directory of the test suite; therefore it must be located there. This file is described in detail in Chapter 8.


Contains all of the classes required to execute the JavaTest harness and library classes. The library classes can be used to simplify the creation of tests. If javatest.jar is located in the same directory as the testsuite.jtt file, or in the ts_dir\lib directory, the JavaTest harness automatically locates the test suite and does not prompt the user for the path to test suite directory.

Note that it is very important not to put javatest.jar on the test suite class path. It is very large and scanning it for library classes at every test invocation impacts the performance of your test suite. The best option is to extract any referenced classes into the classes directory as shown in Step 3 in (UNKNOWN STEP NUMBER) . Use of these library classes is described in Chapter 5.


Contains test source files and test descriptions. Tests should be organized hierarchically the way you want them to be displayed in the test tree.

If you use the HTML test finder rather than the tag test finder, include the HTML test description files along with the test sources in the tests directory. For a discussion of test finders, see Chapter 9.


The directory that contains all of the compiled test classes and library classes required to run your tests. This directory is automatically placed on the JavaTest harness class path.


An optional directory that contains any other files required by your the test suite. These files might include:

jttestsuite.jar — If you create a custom interview, or customize any of the JavaTest plug-in classes, you package the classes and interview files in a custom JAR file. See "The Test Suite JAR File" below for details.

test_suite_x.x.jtx — The exclude list is a mechanism used by test suites to identify tests that should not be run.


An optional directory that contains documentation that describes how to run the test suite and specifies the rules for certifying a product.

The Test Suite JAR File

All of the components you create for the test suite should be delivered to the user in a single JAR file installed in the lib directory of the test suite. The JAR file is added to the class path in the testsuite.jtt file as described in Chapter 8. Experience has shown that it is best to organized the JAR file with the following directory structure:

     Interview class files and resource files, More Info help

For example, the JAR file for the demo TCK test suite:


is organized like this:

     Interview class files and resource files, More Info help

If you provide a large number of components, you can further organize them into sub-packages:

     Interview class files and resource files, More Info help
         Everything else (TestSuite, Script, Finder, etc.)

Class Paths

When you create a test suite, it is important to keep in mind the three potential class paths that are involved:

  • JavaTest class path

  • Agent class path

  • Test class path

Two of the ways described in the following sections in which you can set a class path are through a CLASSPATH environment setting or through a -classpath flag in the command line. The CLASSPATH environment setting is generally a safe way to set the classpath, although setting it as an environment variable is less explicit than using the -classpath flag.

The -classpath flag to the particular software development kit tool (such as java and javac) is generally the best way to set the class path if you know it explicitly. The main disadvantage of using the -classpath flag is that it overrides the CLASSPATH environment setting.


If you have required classes that are set in the environment variable and you also use the -classpath flag, you must make special arrangements for the additional class paths to be set through the -classpath parameter.

Shell example:

% CLASSPATH=otherclasses javac -classpath classes -d out foo/

In this example, only classes set by -classpath are on the classpath. The otherclasses set by the CLASSPATH environment setting are dropped.

Shell example:

% CLASSPATH=otherclasses javac -classpath $CLASSPATH:classes d out foo/

In this example, the environment setting is added to the -classpath argument. The resulting classpath is otherclasses:classes.

JavaTest Class Path

This is the class path that the JavaTest harness uses to access its classes, libraries, and your plug-in classes. The JavaTest class path can be set by means of:

  • The CLASSPATH environment variable

  • The -classpath option to the Java runtime

  • The -jar option to the Java runtime (this is the standard)

In addition, each test suite can use the classpath entry in the testsuite.jtt file to extend the class path. The classpath entry is used to add custom plug-in components and interviews that you create.

Agent Class Path

Often you must run tests on a system other than one on which the JavaTest harness runs. In this case you use an agent (such as the JavaTest Agent) to run the tests on that system. The agent class path is used by the agent to access its classes, libraries, and any plug-in classes. The class path can be set by means of:

  • The CLASSPATH environment variable

  • The -classpath option to the Java runtime

  • The -jar option to the Java runtime

  • Some other platform-specific mechanism

Test Class Path

This is the class path used by the tests during execution. It is normally the responsibility of the configuration interview and/or test script to set the class path for each test in the test environment command entry (see Command Strings). Test classes are normally located in the ts_dir\classes directory, you normally include this on the test class path. You can also put any classes that your tests require in ts_dir\classes and they will be found.


If your platform requires that tests run in the same JVM as the agent, you must include the classes required by the tests on the agent class path. In this case your interview need not put a test class path in the test environment command entry.

ing HTML relocated from