6 Creating a Configuration Interview

Test Script Information

A test script is responsible for running your tests. The test script knows the series of steps required to execute each test. It typically relies on test commands to perform each step and you design your configuration to provide the test commands (and their arguments) that the test script uses to execute each test. Test commands are Java classes that the test script instantiates to run tests.

As an example, the Standard Test Script uses a single step to execute tests; that step is defined in the configuration entry called command.execute. The configuration interview is responsible for setting the value of command.execute so that the Standard Test Script uses the appropriate command and arguments. For example, you can tell the Standard Test Script to use the ExecStdTestOtherJVMCmd command which executes tests in a process on the same computer that runs the JavaTest harness:

command.execute=com.sun.javatest.lib.ExecStdTestOtherJVMCmd args

If you intend to execute the tests differently; for example, on a different computer, you would define command.execute differently in your configuration. For a list of test commands included with the JavaTest release, see Appendix A. For information about creating custom test commands, see Chapter 10.

Test Description Entries

In the previous chapters of this manual, you have seen that most test descriptions are static; these entries consist of values that are known ahead of time and can be specified directly. In some cases these arguments cannot be determined ahead of time, especially test arguments (executeArgs). For example, tests that test network APIs may require the names of hosts on the network to exercise the API. If the test suite runs in different locations and on different networks, these values cannot be known ahead of time by the test developer. The configuration interview is expected to collect this information and make it available to the test.

A script may allow the test developer to specify variables in some test description entries that are defined in the configuration; these variables are prefixed with the "$" character. For example the Standard Test Script allows variables in the executeArg entry; in the case of a network test, here is what the test description might look like:

/** @test
  * @executeClass MyNetworkTest
  * @sources MyNetworkTest.java
  *@executeArgs -host $testHost -port $testPort
 **/

The arguments to the executeClass and sources entries are static — they are known ahead of time and do not change based on the context in which the test runs. The host names or IP addresses cannot be known ahead of time and are specified as variables to which the JavaTest harness assigns values when the test is run. The test suite's configuration interview asks the user to specify the values of the hosts and port numbers required to run the test; the values of $testHost and $testPort are defined from those answers. The configuration interview creates entries in the test environment as name/value pairs. For example:

testHost=129.42.1.50
testPort=8080

Which Tests to Run

The JavaTest harness provides a number of ways that the user can specify which tests in the test suite to run. These standard values can be specified by the user in the configuration editor window question mode or quick set mode. You can easily include interview questions that gather this information at the end of the interview for you and require no extra work on your part.

Example 1

The Standard Test Script uses the value of the command entry command.execute to execute tests. If the tests are executed on the same computer running the JavaTest harness, a typical command entry for the Standard Test Script looks something like the following:

command.execute=com.sun.javatest.lib.ExecStdTestOtherJVMCmd 
   C:\JDK\bin\java.exe -classpath $testSuiteRootDir\classes
   $testExecuteClass $testExecuteArgs

The portion of the entry to the left of the "=" is the name of the test environment entry, the portion to the right is the command string.

Let's examine the command string in detail:

com.sun.javatest.lib.ExecStdTestOtherJVMCmd

The first part of the command string is the name of the test command class used to execute the test classes. In this example the command executes tests in a process on the same computer that runs the JavaTest harness.

Interview implications:

Your configuration interview specifies the command to use to execute the tests. If the API you are testing always runs in a known computing environment, your interview might create this part of the entry without input from the user. However, if the API being tested can be run in different ways, you must ask the user which way they choose to run it and provide the appropriate test command based on the user's input.

Imagine an API that can be tested on the same computer running the JavaTest harness, or on a different computer on the same network. In this case the interview must determine which way the user intends to run the tests and provide the appropriate command — ActiveAgentCommand or ExecStdTestOtherJVMCmd.

-classpath ts_dir\classes

The class path required by the tests. Replace ts_dir with the path to your test suite. To enhance performance, you should place all library classes required to run the test classes in the classes directory.

See Test Environment Variables for a list of available variables.

Interview implications:

You can determine the path to your test suite inside your interview. See Exporting the Test Environment for details. If the test classes require no additional classes be on the class path other than the ones you provide in the test suite's classes directory, your interview can insert the class path value directly into the entry without asking the user for input. If additional class path entries may be required, your interview may include questions that ask the user to provide additional entries that your interview appends to the class path.

This environment entry that can get more complicated if the test suite may be run using different versions of the Java runtime. Some Java runtime systems do not use the -classpath option; for example, they might use a syntax such as -cp or /cp. Additionally, some systems use the ":" character as the class path separator and others use the ";" character. If this is the case, your interview must include additional questions that determine the platform on which the tests are run so that you can create the appropriate command entry.

C:\JDK\bin\java.exe

The path to the Java runtime command used to execute the test classes.

Interview implications:

This path almost certainly differs from user to user, so almost any interview must obtain this path from the user. The interview libraries include a question type named "file" that is very useful for obtaining path names.

Although no additional options or arguments are shown in this example, many Java runtimes or test suites require additional options and arguments. If your tests require any additional options, you include them in additional portions of the entry.

$testExecuteClass

A variable that represents the name of the test class. The test script obtains the class name from the executeClass entry in the test description and provides it at runtime.

Interview implications:

The interview adds the variable to the environment entry.

$testExecuteArgs

A variable that represents the arguments specified in the test description. The test script obtains this value from the test description and provides it at runtime.

Interview implications:

The interview adds the variable to the environment entry.

Example 2

For this example, imagine a test suite that runs in a limited environment — it always runs on a remote system using the JavaTest Agent in passive mode. The command entry looks like this:

command.execute=com.sun.javatest.lib.PassiveAgentCommand
   -host myHost -port 501 
   com.sun.javatest.lib.ExecStdTestSameJVMCmd
   $testExecuteClass $testExecuteArgs

Although this command is quite long, because of its limitations most of it is boilerplate; the only values that your interview has to gather from the user are the arguments to the -host and -port options.

Start the Demo Interview

1. In a command window make the following your current directory:

   jt_install\examples\javatest\interviewDemo\demotck

2. Start the Demo Interview test suite

   At the command prompt enter:

   C:\>java -jar lib\javatest.jar -newDesktop

   The -newdesktop option is used here to ensure that the JavaTest harness loads the correct test suite. For information about JavaTest options, see the online help.

3. Choose Configure > New Configuration to start the interview

   Follow the directions in the interview. You can also browse the source for the interview at:

   jt_install\examples\javatest\interviewDemo\src

Designing Your Questions

Be sure to break down complex environment entries into simple values that can be answered by a single question, then build up the entry from those values. For example, if you are creating an environment entry that requires the name of a remote host and its port address, it's best not to ask for both pieces of information in a single question, but to ask for each piece of information in a separate question.

For example, the following entry (previously seen in Example 1) could be built up from a number of interview answers:

command.execute=com.sun.javatest.lib.ExecStdTestOtherJVMCmd 
   C:\JDK\bin\java.exe -classpath $testSuiteRootDir\classes
   $testExecuteClass $testExecuteArgs

• Questions to determine whether the user plans to run the test locally or on a remote computer, and whether they plan to run the tests in the same JVM as the JavaTest Agent

• A question to determine the path of the Java runtime command

• One or more questions to determine the class path

• Questions that determine the path separator on the test platform

Landing Point Questions

You might find it convenient and useful to include questions that do not gather any information, but rather provide space between sections of the interview or provide a frame of reference to the user about where they are in the interview. You can use the Null question type for this type of interview question. In some cases you can use landing points as bridges between the main interview and sub-interviews.

Sub-Interviews

If your interview contains a large number of questions, you can break it up into sub-interviews. To create a sub interview, create a subtype of an Interview class. For example:

class MySubInterview extends Interview {
      ....
   }

The constructor should take a reference to the parent interview as an argument, and this reference should be passed to the superclass constructor. This identifies this interview as a sub-interview of the parent interview. For example:

MySubInterview(MyParentInterview parent) {
       super(parent, "myTag");
       ...
}

In the constructor, use the setFirstQuestion method to specify the first question in the sub-interview. Subsequent questions are found in the normal way using the getNext method. For example:

MySubInterview(Interview parent) {
   super(parent, "myTag");
   setFirstQuestion(qIntro);
}

By default, a sub-interview shares a resource file and More Info help files (see Creating Question Text and More Info) with its parent interview (another reason to pass in that parent pointer). You can choose to use a different resource file and HelpSet if you want, although that is not typical for simple or moderately complex interviews. See the API specifications for setResourceBundle and setHelpSet for details.

At the end of the interview, have the last question return an instance of a FinalQuestion. This FinalQuestion is only a marker and does not have any question text, More Info, or a getNext method. For example:

Question qXXX = ......... {
   Question getNext() {
      return qEnd;
      }
   };
Question qEnd = new FinalQuestion(this);

For the parent interview to use a sub-interview, it must first create an instance of the sub-interview. This should be created once and stored in a field of the interview. For example:

Interview iMySubInterview = new SubInterview(this);

To call the sub-interview, use callInterview in a getNext method. The callInterview method takes two parameters — a reference to the interview to be called, and a follow-on question to be called when all the questions in the sub-interview have been asked. When the JavaTest harness sees the FinalQuestion at the end of a sub-interview, it goes back to where the interview was called and automatically uses the follow-on question that was specified there. For example:

Question getNext() {
   return callInterview(iMySubInterview, qFollowOnQuestion)
}

Providing the Prolog

The standard prolog always contains a standard welcome question; it also contains optional environment name and description questions. By default, the name and description questions are not displayed. You can enable the name and description questions by calling the setNameAndDescriptionInPrologEnabled method in your interview.

If the standard prolog questions do not meet your needs, you can override the prolog with one of your own. Specify your prolog by means of the standard setFirstQuestion() method of the interview. At the end of your prolog you must call the getPrologSuccessorQuestion() method to determine the first question of the next group.

Providing the Environment Group

This section describes the basic tasks necessary to write the environment portion of the interview. Unless your test suite requires you to make changes to the standard questions (prolog, standard values, epilog), the steps in this section describe what is required for you to produce your interview.

Put the group of questions that gather information for your test environment in your interview class. Remember to implement the getEnvFirstQuestion method to identify the first question of the group.

You must link the last question in the environment group to the rest of the interview (the standard values and epilog). In the getNext() method of the last question of your environment group, use getEnvSuccessorQuestion() to determine the next question in the interview — BasicInterviewParameters provides the rest of the interview.

Finally, you must implement the getEnv() method. The getEnv() method returns a TestEnvironment created from the responses to the questions. The easiest way is to call the interview's export method. The interview's export method calls the export methods for the questions on the current interview path. These questions export their values into a Map object from which you can construct a test environment. For detailed examples see the source code examples in the jt_install\examples directory.

Providing the Resource File for the Interview

In the constructor for your interview class, call:

setResourceBundle(bundle_name);

For example:

setResourceBundle("i18n");

This uses a file called i18n.properties (or a localized variant) in the same directory as the interview class. See Creating Question Text and More Info below for more information.

Providing the More Info Help for the Interview

In the constructor for your interview class, call:

setHelpSet(moreInfo_helpset_name);

For example:

setHelpSet("moreInfo\demotck.hs");

This uses a HelpSet called demotck.hs (or a localized variant) in the moreInfo directory located in the directory that contains the interview class. See Creating Question Text and More Info for more information.

Set Up the More Info System

Create the directories and files used by the More Info system:

1. Create a top-level directory called moreInfo

   The moreInfo directory should be located in the same directory as your interview class file(s).

2. Create directories named default and images in the moreInfo directory

   The default directory contains the default localization. If your test suite is ever localized, the other locales can be added beside the default directory. The images directory contains any images you may use in the More Info help.

3. Copy the Demo TCK HelpSet file to your moreInfo directory and rename it appropriately (retaining the .hs extension)

   The HelpSet file is the XML file that the JavaHelp libraries look for to find all of the help information that defines the HelpSet. Rename it to reflect the name of your test suite. When you write your interview you specify the path to your HelpSet file.

   The path to the Demo TCK HelpSet file is:

   jt_install\examples\javatest\simpleTags\src\moreInfo\demotck.hs
   

4. Edit the HelpSet file

   The Demo TCK HelpSet file looks like:

   <?xml version='1.0' encoding='ISO-8859-1' ?>
   <!DOCTYPE helpset
    
     "http://java.sun.com/products/javahelp/helpset_1_0.dtd">
   <helpset version="1.0">
   <!-- title -->
     <title>DemoTCK Configuration Interview - Help</title>
   <!-- maps -->
     <maps>
        <mapref location="default/map.xml"/>
     </maps>
   </helpset>
   

   Edit the contents of the <title> tag to reflect the name of your test suite.

5. Copy the Demo TCK map file to the default directory

   The JavaHelp map file is an XML file that contains a <mapID> entry for every More Info topic. The JavaHelp system uses it to assign an ID to every HTML file.

6. Copy the Demo TCK style sheet to the default directory

   Use the CSS, level 1 style sheet from the Demo TCK example for your More Info topics. Feel free to change it to suite your needs.

   The path to the Demo TCK style sheet file is:

   jt_install\examples\javatest\simpleTags\src\moreInfo\default\moreInfo.css
   

Create HTML Topics for All Interview Questions

For every question in your interview, you should create an HTML topic file and add an entry for that topic in the map file. The following steps describe how to do both:

1. Create a map entry for the More Info topic

   Every More Info topic file must have a corresponding <mapID> entry in the map.xml file. The JavaHelp system uses the IDs created in these files. The target attribute defines the ID, and the url attribute defines the path to HTML topic file (relative to the map file). The following example shows the map file for the Demo TCK test suite that you copied to your interview in a previous step.

   <?xml version='1.0' encoding='ISO-8859-1' ?>
   <!DOCTYPE map
     PUBLIC "-//Sun Microsystems Inc.//DTD JavaHelp Map Version 1.0//EN"
       "http://java.sun.com/products/javahelp/map_1_0.dtd">
   <map version="1.0">
   <!-- More Info IDs -->
       <mapID target="DemoTCKParameters.cmd Type"            url="cmdType.html" />
       <mapID target="DemoTCKParameters.testVerboseLevel"    url="testVerboseLevel.html" /> 
       <mapID target="DemoTCKParameters.desc"                url="desc.html" />
       <mapID target="DemoTCKParameters.envEnd"              url="envEnd.html" />
       <mapID target="DemoTCKParameters.epilog"              url="epilog.html" />
       <mapID target="DemoTCKParameters.jvm"                 url="jvm.html" />
       <mapID target="DemoTCKParameters.name"                url="name.html" />
       <mapID target="DemoTCKParameters.prolog"              url="prolog.html" />
   </map>
   

   Replace the target and url attributes to match your More Info topics. Remove any extra entries and add new entries as required.

2. Create an HTML More Info topic file in the default directory

   Copy one of the Demo TCK More Info files from:

   jt_install\examples\javatest\simpleTags\src\moreInfo\default
   

   and use it as a template. Be sure to include the <link> tag that references the style sheet.

   Experience has shown that it is helpful for the architect to create "stub" files for every question in the interview. These files are later completed by a technical writer and can provide information that the writer can use.

Customizing Standard Question More Info

The JavaTest package includes default versions of the More Info HTML topics that describe the standard interview questions in both Question mode and Quick Set mode. However, should you wish to customize the content for some or all of these questions, you can override the defaults with files of your own. The following steps describe how to substitute your More Info topics for one of the standard interview questions:

1. Determine the More Info ID for the question

   You will override the More Info ID in your interview HelpSet. To do so, you have to determine the ID name of the standard question.

   1. Open the configuration editor window to the question you wish to override

      Make sure that you establish cursor focus in the question pane.

   2. Press Alt-Shift-D

      This opens the Configuration Editor Details Browser. The More Info ID is listed in the "id" field.

2. Create a map entry in your map file as described in the previous section with the same name as the More Info ID you found in step 1.

   For example:

   <mapID target="TestsInterview.needTests"   url="my_needTests.html" />
   

   Note that the URL must contain the path to a file you create and must be included in your interview HelpSet.

3. Create your custom version of the HTML More Info topic

   Be sure that you create it at the location you specified in the map file URL field.