JavaTest Harness Architect's Guide, JavaTest Harness 4.4.1 for the Java Platform E20663-02 |
|
Previous |
Next |
This chapter explains how to retrofit existing JUnit 3.x or 4.x test suites to enable them to run with JavaTest Harness. This information can also help you author new JUnit tests that run under the harness.
JUnit is a simple framework for writing and running automated tests. Written by Erich Gamma and Kent Beck in 1997, JUnit exposed test driven development practices from the Smalltalk world into the Java programming community. JUnit is now an open-source project at SourceForge.net (http://sourceforge.net/projects/junit
).
The JUnit framework provides a fast, simple interface for creating a set of tests and executing them by a common method (for example, using Ant or a shell script). The framework places very few restrictions on what the tester must do to write a test class. The core JUnit distribution has few facilities for GUI interaction or reporting, and it has no robust interface for configuration.
The procedure described here enables JUnit tests to be run under the name="ProductName" content="JavaTest Harness"?> harness. The name="ProductName" content="JavaTest Harness"?> harness provides a robust GUI interface, many reporting options, and an opportunity to build a robust configuration system for the tests. The harness can be configured to allow customization of the GUI, report types, result presentation, and more. These services might be useful for users who want to wrap more powerful facilities around their existing test infrastructure.
This section describes the process of retrofitting JUnit tests so that they run on the name="ProductName" content="JavaTest Harness"?> harness.
To undertake a conversion process, you must be familiar with some of the inner workings of the JUnit test suite you are converting. Specifically, you need to know:
How the JUnit tests can be distinguished from other tests.
The version of JUnit that works with the test suite (3.x or 4.x).
Where the tests are stored. For example, are they in a single directory tree?
The libraries or supporting processes required to run the tests.
The configuration settings or files necessary to run the tests.
Tests written to work with JUnit 3.x are typically identified as being a subclass of junit.framework.TestCase
. To find JUnit 3.x tests, use the com.sun.javatest.junit.JUnitSuperTestFinder
class (located in the jt-junit.jar
archive) to scan the test classes. Each class that is a subclass of junit.framework.TestCase
is designated as a recognized test.
JUnit 4.x style tests do not use a specific superclass, rather, they tag classes with the org.junit.Test
annotation. The name="ProductName" content="JavaTest Harness"?> harness library jt-junit.jar
provides the class com.sun.javatest.junit.JUnitAnnotationTestFinder
to find 4.x style tests. It operates much like the JUnitSuperTestFinder
class, but looks for different criteria.
See JUnitSuperTestFinder and JUnitAnnotationTestFinder for more details.
This procedure describes how to set up files, property settings, and configuration settings before running a JUnit test.
Create a testsuite.jtt
file in root of the product directory.
For example, if the product unpacks into directory foo/
, the testsuite.jtt
file should be in that directory. It does not necessarily need to be co-located with the tests.
The .jtt
file is a properties formatted file, with key=value
pairs on each line. Setting the name
and id
keys is mandatory. The name
is a short descriptive name for your test suite, the id
is an internal key used identify this test suite.
Select your method for scanning for tests by specifying a TestFinder
class.
The line for specifying the TestFinder
class looks like this:
finder = com.sun.javatest.junit.JUnitSuperTestFinder
-superclass junit.framework.TestCase
See JUnitAnnotationTestFinder and JUnitAnnotationTestFinder for further information.
Select your TestSuite
class, using com.sun.javatest.junit.JUnitTestSuite
if you do not subclass it.
Use a fully qualified class name. This class must be available on the system class path, preferably on the class path defined in your .jtt
file. For example:
testsuite = com.sun.javatest.junit.JUnitTestSuite
Specify the interview.
If you don't have your own interview, use the line below as the default. This class must be available on the system class path, preferably on the class path setting in your .jtt
file. For example:
interview = com.sun.javatest.junit.JUnitBaseInterview
Provide a tests
setting.
The tests location is important because it is forwarded to the TestFinder
class you specified in Step 2. This location is often relative to the location of the testsuite.jtt
file as described in Step 2. Use forward slashes to make the path platform independent. Do not use absolute paths or relative paths to places above testsuite.jtt
. One of the following lines might serve as an example:
If you are scanning .java
files, they might be located below the tests/
directory.
tests = tests
If you are scanning class files, they might be located below the classes/
directory:
tests = classes
See JUnitSuperTestFinder and JUnitAnnotationTestFinder for further information.
Make sure that the paths to any classes you specify in the testsuite.jtt
file are assigned to the classpath
key in the testsuite.jtt
file.
This how the harness locates the classes. For example, if you specify:
interview = com.sun.javatest.junit.JUnitBaseInterview
be sure to add the path to the JAR file that contains that class to the classpath
key as shown here:
classpath = lib/jt-junit.jar lib/jt-myts.jar
Try running the harness to see if it finds your tests.
You have to decide how to arrange your (JAR) files and resolve paths. The general form is:
> cd mytestsuite/
> java -jar lib/javatest.jar -ts .
This starts the harness and forces it to load the test suite located in the current directory (represented by "."). The testsuite.jtt
file must be located in the "." directory.
When the main window comes up, you should see a tree populated with the tests you intended. Check the counters on the main panel to view a summary of the tests that were found. You can check the View > Properties menu item to verify that the plug-in classes are loaded as you expect.
This section describes the two main sets of classes that provide JUnit support. The first is the JUnitTestFinder
(a subclass of com.sun.javatest.TestFinder
). Variations of the JUnitTestFinder
, JUnitSuperTestFinder
and JUnitAnnotationTestFinder
classes roughly correspond to JUnit 3.x and 4.x support. The difference is explained below.
The second supporting component is the JUnitMultiTest
class that is responsible for executing the tests.
The following additional "glue" classes are provided to connect everything: JUnitTestSuite
, JUnitBaseInterview
, and JUnitTestRunner
. Each supporting class is explained below.
The JUnitTestSuite
class is a very simple class that instructs the harness to use the JUnitTestRunner
to execute tests. If this method is not present, the DefaultTestRunner
is used. This is the traditional way to execute tests requiring a Script
class. Because the TestRunner
class is present, there is full control over how the tests are executed. For example, the harness can determine how many tests are run simultaneously and how they are launched (for example, using exec
). By extending this class, you have access to override other aspects of the harness. See the TestRunner
API for more information. Note that many of the settings that this document describes in the testsuite.jtt
file can be hard coded into the TestSuite
subclass. The TestSuite
base class provides the functionality to instantiate the settings in the testsuite.jtt
.
The JUnitBaseInterview
class is a skeleton interview class that does not require any input from the user. If your JUnit tests do not require a setting from the user, do not modify it. Try one of the following methods to get values from the user:
Read a configuration file from a pre-determined location, perhaps a location relative to the test suite root (TestSuite.getRootDir()
).
Ask the user for the values directly using the com.sun.interview
API. This is the primary means by which the harness is designed to get values from the user. In either case, the value(s) must end up in the Map
provided in Interview.export(Map)
. The Map
is the set of values that the other classes must have access to, namely the JUnitTestRunner
and classes it creates (JUnitMultiTest
). Read Chapter 6 for more information.
The JUnitTestRunner
class is responsible for dispatching tests. It has access, via an Iterator
, to the entire list of tests to be executed during a test run. Because a test is represented by a TestDescription
, you must customize your test finder to add any settings that you will want later (in this class). The default implementation executes the test using JUnitBareMultiTest
if the TestDescription
property junit.finderscantype
is set to superclass
. If it is not set to superclass
, it uses the JUnitAnnotationMultiTest
class. You may want to change this behavior, use your own JUnitMultiTest
class, or a subclass of one of these.
This class looks for a superclass that identifies the class as a JUnit test. By default it searches the ancestors of each class for junit.framework.TestCase
. Because a test suite might require further derivations of junit.framework.TestCase
to support its particular needs, you can use the -superclass
option to specify a more specific class.
For example, consider the following class structure:
java.lang.Object junit.framework.TestCase foo.BetterTestCase product.Test0002a
Test0002a
is a subclass of BetterTestCase
, and so forth.
If given Test0002a
, JUnitSuperFinder
ascends the inheritance chain until it reaches either a matching superclass or java.lang.Object
. It searches for the TestCase
class by default, so when given Test0002a
, it ascends two levels, finds java.lang.Object
, and returns Test0002a
to the harness as a test.
If this test finder is given java.util.ArrayList
, it ascends until it reaches java.lang.Object
, at which point it decides that the class is not a test and moves on.
To change the superclass for which you are scanning, supply the -superclass
argument and specify a class name. You can supply this argument multiple times to scan for multiple superclasses. For example, in the testsuite.jtt
file you might specify the following:
finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass
foo.BetterTestCase -superclass foo.CustomTestCase
Although it does not execute tests, the test finder attempts to pick out test methods by looking for public methods that begin with the string "test
". It then lists these in a space-separated list, without the parameters (just the method name). The list might contain duplicates because the full signature is not evaluated. Semantics for this TestDescription
value are loosely defined at this point. Public comment is welcome (submit your comments to the JT harness interest forum at http://java.net/projects/jtharness
).
This superclass finder generates the TestDescription
(com.sun.javatest.TestDescription
) values shown in Table 12-1.
This annotation test finder scans classes for the org.junit.Test
annotation. It uses the same scanning strategy as JUnitSuperTestFinder
.
This annotation finder generates the TestDescription
(com.sun.javatest.TestDescription
) values shown in Table 12-2.
This is the execution class for JUnit 3.x style tests. Execution is accomplished using the class name supplied by the test finder (through the TestDescription
) which is used to execute that class's TestCase.runBare()
method. This might not be sufficient for all test suites. Output from stdout
and stderr
are captured. The test passes if no exceptions are thrown and fails if there are any Throwable
results.
This is the execution class for JUnit 4.x style tests. It takes the class that was identified by the test finder and executes it using the JUnit library TestResult.Section
parts. Also, because execution is turned over to JUnit, it does not report any of its own debugging output during execution. (In the future, it would be useful to take more advantage of the Result API and any observer APIs that are available.)
The use of the junit3
and junit4
keywords might be a generalization, since it really represents how the class was found. A test suite might mix use of version 3 and 4 features, meaning it is not necessarily completely 4.x compliant. Nonetheless, perhaps being able to run 3.x style tests out of a mixed set (see com.sun.javatest.finder.ChameleonTestFinder
) can be useful. Do not forget that the junit
keyword is also added so that JUnit tests can be selected from among non-JUnit tests.
Two of the most likely changes you should make is to modify the test finder or modify how to execute the test. To change the test finder, simply subclass JUnitTestFinder
, provide it on the class path in testsuite.jtt
and change the finder setting in testsuite.jtt
.
To change the method for executing a test, you must change how it is dispatched in JUnitTestRunner
. To change that, you must subclass JUnitTestRunner
and provide it on the testsuite.jtt
class path. You must also subclass JUnitTestSuite
and change its setting in testsuite.jtt
(see The testsuite.jtt File).
This section lists implementation features that might benefit from user feedback and further development. You can provide this on the JT harness web site http://java.net/projects/jtharness
.
The use of class path is currently not convenient. The general design of the harness is that the setting in testsuite.jtt
affects the tests, rather than the system class path that the harness uses. This area can be more refined.
Some additional base implementations of the interview class would be useful. In particular, providing one that reads a properties file and dumps it directly into the Map
of Interview.export(Map)
would provide a "quick and dirty" way for people to configure their test suites. Perhaps the location of the file can be written as a setting in testsuite.jtt
.
Note: Users should generally not be instructed to altertestsuite.jtt . These settings are designed to be static. Information the user provides should be gathered through the interview system. As an architect, you should configure the testsuite.jtt file for general use during the retrofit process. Once the conversion is completed, the file should remain relatively untouched. |
It might be useful to hard code the Interview
class and accept an override in the testsuite.jtt
file, rather than forcing the developer to specify it in the file as documented above. This also applies to the JUnitTestRunner
(or just the TestRunner
class) in the implementation of JUnitTestSuite
.
JT Harness project at http://java.net/projects/jtharness
JUnit project http://SourceForge.net/projects/junit
JUnit 3.X home page http://junit.sourceforge.net/junit3.8.1/index.html
JUnit 4.X home page http://junit.sourceforge.net
API documentation http://junit.sourceforge.net/javadoc_40/index.html
JUnit Cookbook http://junit.sourceforge.net/doc/cookbook/cookbook/htm