2 JavaTest Tutorial

Start the JavaTest Harness

1. Verify that the Java SE platform (version 1.6 or later) is in your path.

   At a command prompt, enter:

   C:\> java -version

2. Make jt_install\examples\javatest\simpleTags\demotck the current directory.

   The directory jt_install is the directory into which you installed the JavaTest harness software.

3. Start the JavaTest harness.

   At a command prompt enter:

   C:\> java -jar lib\javatest.jar -newDesktop

   
   

   Note: The -newDesktop option is used here to ensure that the JavaTest harness starts up exactly as described in these instructions — under normal circumstances you should not use this option because you will lose information that the harness saved about your previous session. For information about JavaTest options, see the JavaTest online help.

   
   

   The JavaTest harness should start and display the Quick Start wizard window:

   Figure 2-2 The JavaTest Harness with Quick Start Wizard

   
   Description of "Figure 2-2 The JavaTest Harness with Quick Start Wizard"
   
   

Run the Quick Start Wizard

The Quick Start wizard leads you through the basic steps required to start running the test suite.

1. Panel 1: Welcome to the JavaTest Harness

   Choose "Start a new test run", and click Next

2. Panel 2: Test Suite

   Click the Next button (accept the default).

3. Panel 3: Configuration

   Choose "Create a new configuration", and click Next

4. Panel 4: Work Directory

   The JavaTest harness uses the work directory to store information and to write test results. Click the Browse button to activate a file chooser. Use the file chooser to create a work directory — be sure to create the work directory in a convenient location outside of the test suite directory (demotck). Click Next.

5. Panel 5: Almost Done ...

   Click the Finish button to complete the Quick Start process. For these options, the configuration editor window is started automatically.

Configure Test Information

Because the "Start the configuration editor" checkbox was checked in the last panel of the Quick Start wizard, the configuration editor starts automatically.

You use the configuration editor to configure the information required to run the test suite. As shown below, the configuration editor consists of three panes and a menu bar:

The left pane lists the titles of the questions you have answered, are currently answering, or that the editor deduces must be answered. The current question is highlighted.

The center pane displays the interview questions. You answer the questions by using controls such as text boxes, radio buttons, or combo boxes located below each question. Whenever possible, the editor deduces answers from your system configuration and includes them in text boxes, combo boxes, and radio buttons. You can accept these answers or provide other answers.

The right pane displays important information about each question, such as:

• Background information

• Examples of answers

• Additional information about choosing an answer

1. Answer the questions in the configuration editor.

   The following table presents the titles, answers, and information about each of the thirteen questions in the Demo TCK interview.

   Question Title	Answer	Description
   Welcome!	N/A	Briefly describes the purpose and function of the Demo TCK Configuration Editor.
   Configuration Name	Demo_TCK	Names the interview file.
   Description	tutorial	Describes the configuration.
   How to Run Tests	On this computer	Runs both the JavaTest harness and the tests on the same computer.
   Java Virtual Machine	The absolute path to the java command on a WIN32 system. For example: jdk_inst_dir\bin\java.exe or jre_inst_dir\jre\java.exe	Click the Browse button to activate a file chooser, or type the path directly in the text box.
   Test Verboseness	Medium	Causes all executing tests to emit standard information messages.
   Parameters...	N/A	Introduces the section of questions that collect information about which tests to run and how to run them.
   Specify Tests to Run?	No	Runs all of the tests.
   Specify an Exclude List?	No	Specifies that an exclude list is not used for this test run.
   Specify Status?	No	Specifies that prior run status is not used to filter the test run. Feel free to try it on subsequent runs.
   Concurrency	1	Specifies the default concurrency setting (1).
   Time Factor	1	Specifies the default standard time out value for each test (1).
   Congratulations!	N/A	The configuration editor has collected all of the information it needs to run the tests. Click the Done button to save the interview.

   
   

2. After you select Done, the Save Configuration File window opens. JavaTest interviews are saved to files that end with the .jti suffix. Use the file chooser to specify a file in a convenient location.

Run Tests

1. Set the view filter to Last Test Run.

   Choose "Last Test Run" in the View Filter combo box located in the tool bar. This changes your "view" of the test tree so that you only see the results of the current test run. This is generally the view that most users prefer to begin with.

   
   

   Note: Note that when you change to the Last Run filter before you do a test run, the folders and tests in the tree turn to gray, indicating that they are filtered out. This occurs because there are currently no results from a "last test run".

   
   

2. Choose Run Tests > Start to start the test run.

   The test suite should begin to run. You will see activity in the test tree panel that indicates which tests are running. You can also watch the progress of the test run in the progress monitor on the bottom-right portion of the JavaTest harness window and the pie chart in the Summary tab.

3. Expand the test tree folders to reveal the tests.

   Click on different test folders to expand the test tree.

   Figure 2-3 Expanded Test Tree

   
   Description of "Figure 2-3 Expanded Test Tree"
   
   

   As tests complete, the tests and their folders change color to represent their state. The following table briefly describes the colors and their meaning:

   Color	Description
   green	Passed
   red	Failed
   blue	Error — The test could not run properly. Usually indicates a configuration problem.
   gray	Filtered out — Due to a parameter setting (for example, it is on an exclude list), the test is not selected to be run.
   white	Not run

   
   

   Folders reflect the state of the tests hierarchically beneath them. You know that the entire test suite passed if the test suite root folder is green. See the JavaTest online help for more information.

   
   

   Note: The test lists\DoublyLinkedList\InsertTest.java intentionally contains errors and is supposed to fail as part of the tutorial. If any other tests fail, check your answers to the configuration interview.

   
   

Browse the Results

Now that the test run is complete, you will use the Folder tabbed pane and Test tabbed pane portion of the JavaTest harness to examine the results. You will also examine the output of the test that failed.

The Folder Pane

The Folder tabbed pane displays information about the tests in the selected folder.

Figure 2-4 The Folder Pane

Description of "Figure 2-4 The Folder Pane"

Browse the Results

1. Click on the top folder in the test tree (the test tree root).

2. Click on the Summary tab (shown by default).

   Notice the statistics displayed in the Summary panel. It describes how many tests in the test suite passed, failed, had errors, and were filtered out.

3. Click on any of the other folder icons in the test tree.

   Notice that the Summary panel changes to reflect the statistics for tests hierarchically beneath it.

4. Click on the test tree root folder again.

5. Click on the Passed tab.

   This pane contains a list of the tests that passed during the test run.

6. Click on the Failed tab.

   This pane contains a list of the tests that failed during the test run (only one test in this case).

7. Double-click the lists\DoublyLinkedList\InsertTest.java test in the Failed tab.

   This automatically selects the test in the test tree and changes the display from the Folder pane to the Test pane.

   
   

   Note: To read more information about any of the panes, click on a tab to establish focus, and press F1 to activate online help about that pane.

   
   

The Test Pane

The Test tabbed pane displays information about the selected test. The five tabs provide information about the test and information about the results of its execution.

Figure 2-5 The Test Pane

Description of "Figure 2-5 The Test Pane"

Browse the Results

Click on the different tabs and examine the information the panes contain.

The following table briefly describes each tabbed pane:

Tab	Description
Test Run Messages	Displays messages generated during the selected test's execution
Test Run Details	A table of values generated during the selected test's execution
Configuration	A table of the configuration values used during the selected test's execution
Files	Displays the Java language source code and any other files related to the selected test
Test Description	A table of the test description values specified for the test

Note: To read more information about any of the panes, click on a tab to establish focus, and press F1 to activate the online help about that pane.

1. Click on the Test Run Messages tab.

   This pane provides access to any messages generated by the JavaTest harness or the test during execution. Notice that the various red icons indicate that the test failed.

2. Click on the Execute/Messages entry in the left hand column.

   The display on the right shows the command line used to run the test. Problems can often be debugged by examining how the test was invoked. In this case it was invoked correctly.

   Figure 2-6 Test Messages

   
   Description of "Figure 2-6 Test Messages"
   
   

3. Click on the out1 entry in the left-hand column.

   The display on the right shows errors reported by the test. The messages indicate that either the test or the API contain errors — in this case the test contains errors.

   Figure 2-7 Logged Error Messages

   
   Description of "Figure 2-7 Logged Error Messages"
   
   

Exclude the Failed Test

The JavaTest harness allows you to "exclude" tests from a test suite by specifying an exclude list file. This section shows you how to use the quick set mode of the configuration editor window to specify an exclude list that includes lists\DoublyLinkedList\InsertTest.java. Tests that are excluded are not executed during test runs, and though they are still displayed in the test tree, their status is not reflected in the pass/fail status of the test suite.

1. Choose Configure > Edit Quick Set from the test manager menu bar.

   The configuration editor window opens directly to a panel that allows you to specify an exclude list. This quick set mode allows you to quickly change values that change frequently between test runs. These values are also referred to as standard values. Note that standard values can also be changed using the configuration editor window in question mode.

2. In the Exclude List pane, click Other.

   This activates a tool with which you can specify a set of exclude lists.

3. Click the Add button on the upper right portion of the tool.

   This invokes a file chooser with which you can specify an exclude list. The current directory of the file chooser should be the directory in which you started the JavaTest harness. If it is not, please navigate to that directory.

4. Double-click on the lib directory entry in the file chooser.

5. Choose the demo.jtx entry in the file chooser and click Select.

   Notice that the exclude list (demo.jtx) is added to the Exclude Lists text box.

6. Click Done in the configuration editor.

7. Change the view filter to "Current Configuration".

   The Current Configuration filter shows which tests are selected and filtered out in the configuration, in effect a filter that shows which tests will be run next, as opposed to the Last Test Run filter which shows the tests that were run.

   Notice that the icon for the for InsertTest.java entry in the Test tree changes from red to gray. This indicates that the test has been filtered out and will not be executed. Also notice that the Test Suite Root folder has changed from red to green, indicating that all the currently selected tests have passed.

Generate a Report

You can use the JavaTest harness to generate an HTML report that describes the results of the test run. All of the information contained in the report is available from the GUI; however, the following steps describe how to generate and browse a report that describes the test run done in the previous sections of this tutorial.

1. Choose Report > Create Report.

   The Create a New Report dialog box opens.

2. Specify the directory in which you want the report files to be written.

   If you wish to use a file chooser to specify the directory, click on the Browse button.

3. Click the Create Report(s) button.

   The reports are generated and you are asked whether you want to view the report.

4. Click Yes.

   The reports are displayed in the JavaTest report browser window. Scroll through the report and follow the various links to view data about the test run.

   
   

   Note: If you wish to print the report, you can open the report in your favorite web browser and print it from there.