C H A P T E R 4 |
Test Class and Case Comment Blocks |
You must precede each test class and each test case with a descriptive comment block. These blocks identify the classes and methods that represent test classes and cases. They also give the build system and the harness information required to build and bundle tests and to display configurable properties to testers. The comment blocks are similar to Javadoc tool “doc comments” described at http://java.sun.com/j2se/javadoc/writingdoccomments/
This chapter has the following sections:
When you build a test pack, the build system generates the files shown in FIGURE 4-1 from comment blocks you code for each test class and test case.
FIGURE 4-1 Files Generated from Comment Blocks
The default build generates the following files:
You can alternatively create a card file manually, as described in Chapter 25. You can turn off the autogeneration of card files with a build option.
You can verify the contents of a card file using the tool described in Chapter 12.
CODE EXAMPLE 4-1 shows a typical test case comment block.
FIGURE 4-2 and FIGURE 4-3 show the (rendered) test case documentation and evaluation file generated from the block comment shown in CODE EXAMPLE 4-1.
FIGURE 4-2 Typical Generated Test Case Documentation
FIGURE 4-3 Typical Generated Interactive Test Evaluation Instructions
Although there is some flexibility in formatting a comment block, observe the following rules:
* @card.property x=one \ *two
TABLE 4-1 describes the Java Device Test Suite tags that can or must appear in a test class comment block. The Instances column specifies how many instances of a tag are permitted.
htmlLines is the class description in the generated test documentation. In Javadoc tool terms, this is the “main description”. |
||||
A condition that is assumed to be true, but cannot be changed, and, in some cases, even checked by the tester. A typical assumption is an interpretation of a related specification statement. If an assumption does not appear to be true, the test should be considered not applicable to the device, which typically means that the test fails if it is run. htmlLines is the content of the corresponding section of the generated test documentation and applies to all cases in the class. |
||||
A condition that must be true just prior to the execution of the test. If the precondition is not met, the test may give a wrong result. The precondition statement is checkable and achievable: There should be a way to change the environment and guarantee the precondition to be fulfilled. This is typically the tester's responsibility to ensure that the test precondition is fulfilled. Typical precondition is a reminder about some necessary configuration settings. htmlLines is the content of the corresponding section of the generated test documentation and applies to all cases in the class. |
||||
A condition that must be true immediately after successful execution of a test case. “Successful” means no error, unexpected exception, or VM_EXIT. htmlLines is the content of the corresponding section of generated test documentation and applies to all cases in class. |
||||
Keyword(s) that testers can use to filter (subset for execution) this test case or test class. keywordList is a space-separated list of keywords. Interactive tests must have this entry: @keyword interactive. To see the current list of keywords and their definitions, launch the harness and create or open a work directory. Choose Configure > Edit Configuration or Configure > New Configuration. In the Test Selection section of the interview, answer Yes to Specify Keywords? The More Info pane displays the current list of keywords. |
||||
Notes that provide test class information to the tester. Quoting from a specification is one use of this tag. If a note is not necessary, do not use the tag. |
||||
Defines a property for all test cases in the class. See @card.property for details. |
||||
Adds a line to the JAD file in a test bundle that contains this test. See @card.specialproperty for details. |
||||
Ensures that a test bundle that contains this test also contains all files the test needs to run. See @card.requires for details. |
||||
Defines the factors that determine the severity of a test case failure. See @card.attribute for details. Two of these comments are required (at the case or class level) if you want to give testers the ability to select tests and report results by failure severity, as they can with Sun tests. The Java Device Test Suite Tester’s Guide describes the severity concept. |
TABLE 4-2 describes the Java Device Test Suite tags that can or must appear in a test case comment block. The Instances column specifies how many instances of a tag are permitted.
htmlLines is the class description in the generated test documentation. For interactive tests, value is the content of Test Objectives in the evaluation file. In Javadoc tool terms, this is the “main description”. |
||||
A condition that is assumed to be true, but cannot be changed and, in some cases, even checked by the tester. A typical assumption is an interpretation of a related specification statement. If an assumption does not appear to be true, the test should be considered not applicable to the device, which typically means that the test fails if it is run. htmlLines is the content of the corresponding section of the generated test documentation. |
||||
A condition that must be true just prior to the execution of the test. If the precondition is not met, the test may give a wrong result. The precondition statement is checkable and achievable: there should be a way to change the environment and guarantee the precondition to be fulfilled. It is typically the tester's responsibility to ensure that the test precondition is fulfilled. A typical precondition is a reminder about some necessary configuration settings. htmlLines is the content of the corresponding section of the generated test case documentation. |
||||
A condition that must be true immediately after successful execution of a test case. “Successful” means no error, no unexpected exception, and no VM_EXIT. htmlLines is the content of the corresponding section of generated test documentation. |
||||
Condition or conditions that must be met for the test to pass. For interactive tests, htmlLines is the content of Test Expected Result section of the test evaluation instructions. |
||||
Instructions for tester to execute an interactive test. htmlLines is the content of the User Interaction section of the test evaluation instructions. |
||||
URLs is URL(s) of image(s) displayed in evaluation instructions. |
||||
Use instead of @passCriteria. value must be one of: UnitRate, SystemLoad. |
||||
Keywords testers can use to filter (subset for execution) this test case or test class. keywordList is a space-separated list of keywords. Interactive tests must have @keyword interactive. To see the current list of keywords and their definitions, launch the harness and create or open a work directory. Choose Configure > Edit Configuration or Configure > New Configuration. In the Test Selection section of the interview, answer Yes to Specify Keywords? The More Info pane displays the current list of keywords. |
||||
Notes that provide test case information to the tester. For interactive tests, htmlLines is the content of the Comments section of the evaluation file. Quoting a specification or describing unusual test behavior (“On some devices, the screen blinks”) are typical uses of notes. If a note is not necessary, do not use the tag. |
||||
Defines a test case property. See @card.property for details. |
||||
Ensures that a test bundle that contains this test also contains all files the test needs to run. See @card.requires for details. |
||||
Defines the factors that determine the severity of a test case failure. See @card.attribute for details. Two of these comments are required (at the case or class level) if you want to give testers the ability to select tests and report results by failure severity, as they can with Sun tests. The Java Device Test Suite Tester’s Guide describes the severity concept. |
For tags that can be used with both test classes and test cases, use the class tag to specify values that apply to all cases in a class. Use the case tag to specify values that apply to one case only. For example, suppose you specify the following:
* @testclass * @keyword interactive * ... * @testcase * @keyword onePartnerMIDlet
This means that all test cases in the class can be filtered with the interactive keyword but only the specified test case can also be filtered with onePartnerMIDlet.
Tags with more complex syntax or semantics are described in this section.
If a test class or case needs a user-visible property, name the property and specify its default value with the following syntax:
* @card.property PropertyName=DefaultValue
* @card.property MaxDistance=12
Properties that apply to multiple classes can be defined globally at the test pack level. Such properties are defined in the testsuite.info file at the top of the test pack directory in the file system. A property defined at the test pack level can be overridden by defining a property of the same name within the test class or case comment blocks. See Properties and Parameter Expansion for details.
The following test pack property attributes are also supported for test class and case properties.
* @card.property PixelHeightNoLessThan.doc=minimum number of pixels allowed for object height
FIGURE 4-4 shows an example of doc values that appear when a tester right-clicks a test case in the harness test tree and chooses Configure Test. Read-only properties are displayed in gray (for example, JADPath1 in FIGURE 4-4).
FIGURE 4-4 doc Properties in Configure Test Window
For the exact syntax of property attributes, refer to Scope, Read-only Properties, and Online Documentation for a Property
Use a special property definition to direct the harness to add a line to the JAD file that it creates for a test bundle containing this test. In effect, a special property definition is a way to pass a parameter to an application management system (AMS), sometimes called a Java application manager. The static parameter applies only to the test bundle associated with the JAD file.
Use the following syntax to specify a line to be added to the JAD file:
* @card.specialproperty <jad>.n=LineToAdd
n is a number that distinguishes multiple @card.specialproperty entries in the same source file. The harness interprets LineToAdd in this line as a name:value pair taking the first colon symbol as the separator between the name and the value. The harness copies the name:value pair to the JAD file. The test device AMS interprets the name:value pair when it downloads the JAD file. It is the test developer’s responsibility to ensure that name:value pair is meaningful to the AMS and follows the JAD file syntax. The JAD file is defined in the MIDP specification.
For example, to add a line to a JAD file associated with a test bundle containing MIDlet2:AlarmMidlet, include a line like this in the test class block comment:
* @card.specialproperty <jad>.1=MIDlet2:AlarmMidlet
Parameter expansion, described in Properties and Parameter Expansion, also applies to special property definitions. The samples in
devKitRoot/tests/runtime/src/client/com/sun/samples/network/client/push/ illustrate its use.
This tag specifies files that must be in the test bundle for one or more test cases in a class to execute. Examples of required files include media files to play on the test device and helper classes. CODE EXAMPLE 4-2 gives examples for a class and a case:
Observe the following when writing path names for this tag:
* @card.requires Pic1.png *
The path in an @card.requires entry can specify parameter expansion notationas described in Properties and Parameter Expansion. CODE EXAMPLE 4-3 shows an example.
* @card.property req=a * ... * @card.requires com/some/samples/automated/${req}.html |
In this example, changing the value of req to b will include the file com/some/samples/automated/b.html in test bundles.
Note - Properties are not inherited. In particular, required file properties declared in class X do not apply to X’s superclass or subclasses. Be sure to declare the files that each class requires. |
Each test class and case can have a failure severity indicator, which is computed from two factors, functionality and impact. TABLE 4-3 shows the three functionality and impact codes and the failure severity values that the Java Device Test Suite computes from them. The Java Device Test Suite Tester’s Guide gives more information on failure severity.
Use the following guidelines to select a functionality value:
Use the following guidelines to select an impact value:
You express functionality and impact in @card.attribute tags of the following form:
* @card.attribute functionality=value * @card.attribute impact=value
In both cases, value must be 1, 2, or 3. CODE EXAMPLE 4-4 shows an example. TABLE 4-3 shows that this case’s computed failure severity is 4 - Very Low.