10 Test Scripts

Design Decisions

One of the most significant design decisions that you make is how the test script executes tests. The mechanism that you design can be very simple but inflexible, or it can be more complex and much more flexible.

Runtime r = Runtime.getRuntime();
String[] cmd = {"java", "MyTest"};
String[] env = {"-classpath", testsDir + "/classes"};
Process p = r.exec(cmd, env);
// read output from test using
// p.getInputStream() and p.getErrorStream()
// (best done with a separate thread for each stream)
int rc = p.waitFor();
Status s = (rc == 0 ? Status.passed("OK") : 
Status.failed("process exited with return code " + rc);
// s contains result status from executing command

String[] args = {"-classpath" + testsDir + "/classes", "java", "MyTest"};
PrintWriter out1 = ... // create error message stream
PrintWriter out2 = ... // create output message stream
Command cmd = new ProcessCommand();
Status s = cmd.run(args, out1, out2);
// output from command will be written automatically to
// the out1 and out2 streams
// s contains result status from executing command

command.testExecute=com.sun.javatest.lib.ProcessCommand
\bin\java.exe -classpath $testSuiteRootDir\classesJDKC:\
$testExecuteClass $testExecuteArgs

import com.sun.javatest.*;
 
class MyScript extends Script {
    public Status run(String[] args, TestDescription td, TestEnvironment env) {
    
    ...
        
    // Extract values from the test description
    String executeClass = td.getParameter("executeClass");
    String executeArgs  = td.getParameter("executeArgs");
 
    ...
 
    // Set variables in the template
    env.put("testExecuteClass", executeClass);
    env.put("testExecuteArgs", executeArgs);
    // Invoke the command
    Status s = invokeCommand("testExecute");
    
    ...
    
    return s;
    }
}

Writing Custom Commands

Commands are the means by which the JavaTest harness invokes platform or test components to perform a step of the test execution model embodied in a test script. The JavaTest harness provides standard commands that are suitable for most uses, including test systems that can execute programs in a separate address space, and test systems that provide a single Java virtual machine.

If none of the standard commands are suitable, you can write a new one tailored to the test suite's specific requirements. One scenario that requires a custom command is when the test suite uses a single JVM, and the test invokes a program that does not have a standard interface that can be used by one of the standard commands. In this case, you can write a very simple converter command that connects the interface expected by the JavaTest harness with the interface provided by the program.

The class for a command is similar (apart from the name) to the standard Test interface. The full class name is com.sun.javatest.Command.

abstract class Command {
   Status run(String[] args, PrintWriter out1, PrintWriter out2)
   ...
}

The args argument is constructed in and passed down from the script that invokes the command. Output written to the out1 stream and out2 stream is recorded in the appropriate test result file.

Example 10-1 is an example of a command that invokes a compiler in the same JVM as the JavaTest harness, using an API for the compiler. The example uses the JDK compiler which is usually invoked directly from the command line; however, in this case an undocumented API is used. The details of how to create the PrintStream outStream from the PrintWriter out are omitted here for simplicity; the main point is to illustrate how easy it can be to write a wrapper class that passes arguments through to a non-standard API, and converts the results into the format that the JavaTest harness requires.

See the source code for JavaCompileCommand in the jt_install\examples\javatest\sampleFiles directory for a complete, commented example.

public class JavaCompileCommand implements Command
{
   public Status run (String[] args, PrintWriter out1,PrintWriter out2) 
   {
      PrintStream outStream = ... // create stream from out
      sun.tools.javac.Main compiler = ;
            new sun.tools.javac.Main(outStream, "javac")
      boolean ok = compiler.compile(args);
      return (ok ? Status.passed("Compilation OK") :
            Status.failed("Compilation failed"));
   }
}

For information on the standard commands provided with JavaTest. see Appendix A.

Test Result

To store test results, the JavaTest harness creates and maintains a TestResult object for each test. The test script stores information in a TestResult object while it executes a test. This information is presented to users in the JavaTest GUI and is useful when troubleshooting test runs. The more information the test script provides, the easier it is for the user to understand what occurred when the test was run.

The TestResult object contains elements:

When a test completes execution, the JavaTest harness writes the results to the file testname.jtr in the work directory. Test result files are created in directory hierarchies analogous to the hierarchies in which the tests are organized.

See the API documentation for the TestResult class.