Java Device Test Suite Developer’s Guide Table Of ContentsPrevious ChapterNext ChapterBook Index

C H A P T E R  27
Writing Evaluation Files

For each test case in an interactive or OTA test class, an evaluation file is required, which instructs the user on manipulating and observing the device and deciding if the test results indicate success. (For an OTA test class, the evaluation file also instructs the user to download a MIDlet to the test device.)

Starting with version 2.1 of Java Device Test Suite, evaluation files are, by default, generated automatically from markup you embed in comment blocks in the source code. Refer to Chapter 4 for details.


Note - If you choose to create evaluation files manually, add this line to your test pack’s build.properties file:
generate.evaluation.files=false

This chapter is intended for legacy situations in which the evaluation files should be directly written.


procedure icon  Procedure for Editing an Evaluation File

1. Locate the file devKitHome/tests/runtime/src/client/com/sun/samples/interactive/SampleInteractiveTest.example1.html.

2. Change the file name to TestClassName.testCaseName.html.

Refer to Test Case Definitions for the meaning of TestClassName and testCaseName.

3. Edit the file as follows:

a. Replace the Sun copyright lines with text that conforms to your organization’s copyright policy.

b. Enter the test name that you want to display in the window title bar between the title tags (such as <title>My Interactive Test</title>)

c. Change SampleInteractiveTest.TestCase1 to TestClassName.testCaseName.

Refer to Test Case Definitions for the meaning of TestClassName and testCaseName.

d. Change Specify the objectives of the testcase to your test case objectives.

e. Change Instruct the user on any interaction that is required to your interaction instructions.

f. Change Specify the expected behaviour to the behavior the device will exhibit.

g. Change Any additional comments to your comments or a space.

4. Save and close the file.

Example Evaluation File Text


CODE EXAMPLE 27-1 Sample Interactive Test Evaluation File Text 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<!--
 @(#)SampleInteractiveTest.TestCase1.html
 
 Copyright 2007 Sun Microsystems, Inc. All rights reserved. 
 Use is subject to license terms.
-->
 
<html>
<head><title></title></head>
<body>
 
<table summary="Test Description" border=2>
   <tr>
      <td><b>Test Name</b></td>
      <td>SampleInteractiveTest.TestCase1</td>
   </tr>
 
   <tr>
      <td><b>Test Objectives</b></td>
      <td>Specify the objectives of the testcase.</td>
   </tr>
 
   <tr>
      <td><b>User Interaction</b></td>
      <td>Instruct the user on any interaction that is required.</td>
   </tr>
 
   <tr>
      <td><b>Test Expected Result</b></td>
      <td>Specify the expected behavior.</td>
   </tr>
 
   <tr>
      <td><b>Comments</b></td>
      <td>Any additional comments if needed.</td>
   </tr>
</table>
 
</body>
</html>


procedure icon  Including Reference Images in an Evaluation File

FIGURE 27-2 shows an example of an evaluation window with references images.

1. Copy the image files to the package directory or a subdirectory, such as images/ or jpg/.

2. Add a Reference Images row to the bottom of the table in TestClassName.testCaseName.html.

CODE EXAMPLE 27-2 shows an example.


CODE EXAMPLE 27-2 Example Reference Image Row 
  <tr>
    <td><b>Reference Images</b></td>
    <td>
    <img src=”./jpg/image1.jpg”><font size=”3” >Ref1</font> 
    <img src=”./jpg/image2.jpg”><font size=”3” >Ref2</font> 
    </td>
  </tr>

3. Add instructions to the Comments row to help users compare the reference images to the images displayed by test devices.

FIGURE 27-2 gives an example.

4. (Optional) Refer to the images in the User Interaction section of the test evaluation file.

5. Ensure that the reference image’s file extension appears in the test pack’s common/build/build.properties file.

The default extensions, such as .html and .gif, are listed after client.res.pattern.

If necessary, override this section in your test pack’s build/build.properties file.

An Evaluation File Rendered by the Harness

FIGURE 27-1 shows an evaluation window rendered by the harness using the HTML source file from CODE EXAMPLE 27-1.

FIGURE 27-1 Sample Interactive Test Evaluation Window


Sample Interactive Test Evaluation Window

FIGURE 27-2 Example Test Evaluation Window With Reference Images


Example Test Evaluation Window with Reference Images