|JavaTest Harness Architect's Guide,
JavaTest Harness 4.5 for the Java Platform
This chapter explains how to retrofit existing JUnit 3.x or 4.x test suites to enable them to run with JavaTest Harness. This information can also help you author new JUnit tests that run under the harness.
JUnit is a simple framework for writing and running automated tests. Written by Erich Gamma and Kent Beck in 1997, JUnit exposed test driven development practices from the Smalltalk world into the Java programming community. JUnit is now an open-source project at SourceForge.net (
The JUnit framework provides a fast, simple interface for creating a set of tests and executing them by a common method (for example, using Ant or a shell script). The framework places very few restrictions on what the tester must do to write a test class. The core JUnit distribution has few facilities for GUI interaction or reporting, and it has no robust interface for configuration.
The procedure described here enables JUnit tests to be run under the name="ProductName" content="JavaTest Harness"?> harness. The name="ProductName" content="JavaTest Harness"?> harness provides a robust GUI interface, many reporting options, and an opportunity to build a robust configuration system for the tests. The harness can be configured to allow customization of the GUI, report types, result presentation, and more. These services might be useful for users who want to wrap more powerful facilities around their existing test infrastructure.
How the JUnit tests can be distinguished from other tests.
The version of JUnit that works with the test suite (3.x or 4.x).
Where the tests are stored. For example, are they in a single directory tree?
The libraries or supporting processes required to run the tests.
The configuration settings or files necessary to run the tests.
Tests written to work with JUnit 3.x are typically identified as being a subclass of
junit.framework.TestCase. To find JUnit 3.x tests, use the c
om.sun.javatest.junit.JUnitSuperTestFinder class (located in the
jt-junit.jar archive) to scan the test classes. Each class that is a subclass of
junit.framework.TestCase is designated as a recognized test.
JUnit 4.x style tests do not use a specific superclass, rather, they tag classes with the
org.junit.Test annotation. The name="ProductName" content="JavaTest Harness"?> harness library
jt-junit.jar provides the class
com.sun.javatest.junit.JUnitAnnotationTestFinder to find 4.x style tests. It operates much like the
JUnitSuperTestFinder class, but looks for different criteria.
This procedure describes how to set up files, property settings, and configuration settings before running a JUnit test.
testsuite.jtt file in root of the product directory.
For example, if the product unpacks into directory
testsuite.jtt file should be in that directory. It does not necessarily need to be co-located with the tests.
.jtt file is a properties formatted file, with
key=value pairs on each line. Setting the
id keys is mandatory. The
name is a short descriptive name for your test suite, the
id is an internal key used identify this test suite.
Select your method for scanning for tests by specifying a
The line for specifying the
TestFinder class looks like this:
finder = com.sun.javatest.junit.JUnitSuperTestFinder
TestSuite class, using
com.sun.javatest.junit.JUnitTestSuite if you do not subclass it.
Use a fully qualified class name. This class must be available on the system class path, preferably on the class path defined in your
.jtt file. For example:
testsuite = com.sun.javatest.junit.JUnitTestSuite
Specify the interview.
interview = com.sun.javatest.junit.JUnitBaseInterview
The tests location is important because it is forwarded to the
TestFinder class you specified in Step 2. This location is often relative to the location of the
testsuite.jtt file as described in Step 2. Use forward slashes to make the path platform independent. Do not use absolute paths or relative paths to places above
testsuite.jtt. One of the following lines might serve as an example:
If you are scanning
.java files, they might be located below the
tests = tests
If you are scanning class files, they might be located below the
tests = classes
Make sure that the paths to any classes you specify in the
testsuite.jtt file are assigned to the
classpath key in the
This how the harness locates the classes. For example, if you specify:
interview = com.sun.javatest.junit.JUnitBaseInterview
be sure to add the path to the JAR file that contains that class to the
classpath key as shown here:
classpath = lib/jt-junit.jar lib/jt-myts.jar
Try running the harness to see if it finds your tests.
You have to decide how to arrange your (JAR) files and resolve paths. The general form is:
java -jar lib/javatest.jar -ts .
This starts the harness and forces it to load the test suite located in the current directory (represented by "."). The
testsuite.jtt file must be located in the "." directory.
When the main window comes up, you should see a tree populated with the tests you intended. Check the counters on the main panel to view a summary of the tests that were found. You can check the View > Properties menu item to verify that the plug-in classes are loaded as you expect.
This section describes the two main sets of classes that provide JUnit support. The first is the
JUnitTestFinder (a subclass of
com.sun.javatest.TestFinder). Variations of the
JUnitAnnotationTestFinder classes roughly correspond to JUnit 3.x and 4.x support. The difference is explained below.
JUnitTestSuite class is a very simple class that instructs the harness to use the
JUnitTestRunner to execute tests. If this method is not present, the
DefaultTestRunner is used. This is the traditional way to execute tests requiring a
Script class. Because the
TestRunner class is present, there is full control over how the tests are executed. For example, the harness can determine how many tests are run simultaneously and how they are launched (for example, using
exec). By extending this class, you have access to override other aspects of the harness. See the
TestRunner API for more information. Note that many of the settings that this document describes in the
testsuite.jtt file can be hard coded into the
TestSuite subclass. The
TestSuite base class provides the functionality to instantiate the settings in the
JUnitBaseInterview class is a skeleton interview class that does not require any input from the user. If your JUnit tests do not require a setting from the user, do not modify it. Try one of the following methods to get values from the user:
Read a configuration file from a pre-determined location, perhaps a location relative to the test suite root (
Ask the user for the values directly using the
com.sun.interview API. This is the primary means by which the harness is designed to get values from the user. In either case, the value(s) must end up in the
Map provided in
Map is the set of values that the other classes must have access to, namely the
JUnitTestRunner and classes it creates (
JUnitMultiTest). Read Chapter 6 for more information.
JUnitTestRunner class is responsible for dispatching tests. It has access, via an
Iterator, to the entire list of tests to be executed during a test run. Because a test is represented by a
TestDescription, you must customize your test finder to add any settings that you will want later (in this class). The default implementation executes the test using
JUnitBareMultiTest if the
junit.finderscantype is set to
superclass. If it is not set to
superclass, it uses the
JUnitAnnotationMultiTestclass. You may want to change this behavior, use your own
JUnitMultiTest class, or a subclass of one of these.
This class looks for a superclass that identifies the class as a JUnit test. By default it searches the ancestors of each class for
junit.framework.TestCase. Because a test suite might require further derivations of
junit.framework.TestCase to support its particular needs, you can use the
-superclass option to specify a more specific class.
For example, consider the following class structure:
java.lang.Object junit.framework.TestCase foo.BetterTestCase product.Test0002a
Test0002a is a subclass of
BetterTestCase, and so forth.
JUnitSuperFinder ascends the inheritance chain until it reaches either a matching superclass or
java.lang.Object. It searches for the
TestCase class by default, so when given
Test0002a, it ascends two levels, finds
java.lang.Object, and returns
Test0002a to the harness as a test.
If this test finder is given
java.util.ArrayList, it ascends until it reaches
java.lang.Object, at which point it decides that the class is not a test and moves on.
To change the superclass for which you are scanning, supply the
-superclass argument and specify a class name. You can supply this argument multiple times to scan for multiple superclasses. For example, in the
testsuite.jtt file you might specify the following:
finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass
foo.BetterTestCase -superclass foo.CustomTestCase
Although it does not execute tests, the test finder attempts to pick out test methods by looking for public methods that begin with the string "
test". It then lists these in a space-separated list, without the parameters (just the method name). The list might contain duplicates because the full signature is not evaluated. Semantics for this
TestDescription value are loosely defined at this point. Public comment is welcome (submit your comments to the JT harness interest forum at
This superclass finder generates the
com.sun.javatest.TestDescription) values shown in Table 12-1.
This annotation test finder scans classes for the
org.junit.Test annotation. It uses the same scanning strategy as
This annotation finder generates the
com.sun.javatest.TestDescription) values shown in Table 12-2.
This is the execution class for JUnit 3.x style tests. Execution is accomplished using the class name supplied by the test finder (through the
TestDescription) which is used to execute that class's
TestCase.runBare() method. This might not be sufficient for all test suites. Output from
stderr are captured. The test passes if no exceptions are thrown and fails if there are any
This is the execution class for JUnit 4.x style tests. It takes the class that was identified by the test finder and executes it using the JUnit library
TestResult.Section parts. Also, because execution is turned over to JUnit, it does not report any of its own debugging output during execution. (In the future, it would be useful to take more advantage of the Result API and any observer APIs that are available.)
The use of the
junit4 keywords might be a generalization, since it really represents how the class was found. A test suite might mix use of version 3 and 4 features, meaning it is not necessarily completely 4.x compliant. Nonetheless, perhaps being able to run 3.x style tests out of a mixed set (see
com.sun.javatest.finder.ChameleonTestFinder) can be useful. Do not forget that the
junit keyword is also added so that JUnit tests can be selected from among non-JUnit tests.
Two of the most likely changes you should make is to modify the test finder or modify how to execute the test. To change the test finder, simply subclass
JUnitTestFinder, provide it on the class path in
testsuite.jtt and change the finder setting in
To change the method for executing a test, you must change how it is dispatched in
JUnitTestRunner. To change that, you must subclass
JUnitTestRunner and provide it on the
testsuite.jtt class path. You must also subclass
JUnitTestSuite and change its setting in
testsuite.jtt (see The testsuite.jtt File).
This section lists implementation features that might benefit from user feedback and further development. You can provide this on the JT harness web site
The use of class path is currently not convenient. The general design of the harness is that the setting in
testsuite.jtt affects the tests, rather than the system class path that the harness uses. This area can be more refined.
Some additional base implementations of the interview class would be useful. In particular, providing one that reads a properties file and dumps it directly into the
Interview.export(Map) would provide a "quick and dirty" way for people to configure their test suites. Perhaps the location of the file can be written as a setting in
Users should generally not be instructed to alter
It might be useful to hard code the
Interview class and accept an override in the
testsuite.jtt file, rather than forcing the developer to specify it in the file as documented above. This also applies to the
JUnitTestRunner (or just the
TestRunner class) in the implementation of
JT Harness project at
JUnit 3.X home page
JUnit 4.X home page