JavaTest Harness Architect's Guide, JavaTest Harness 4.4.1 for the Java Platform E20663-02 |
|
Previous |
Next |
This appendix describes file formats and specifications that test architects should know about.
This specification describes how test files must be named to work properly with the JavaTest harness.
The JavaTest harness converts the native path names of tests to an internal format called a test URL. When the JavaTest harness converts a path name to a test URL, it:
Makes the path relative to the root of the test suite
Converts the path separator to a forward slash
A test URL consists of three components:
The folder/directory path name (relative to the test suite tests
directory)
The name of the file that contains the test description
An optional test identifier can be appended to designate a test description table within a test description file
For example:
api/javatest/TestSuite.html#getName
The JavaTest harness evaluates test URLs without regard to case; however, it does preserve case when possible to increase readability for the user.
The path names you create can contain only the following characters:
ISO Latin 1 characters A-Z and a-z
Digits 0-9
Underscore character "_"
Dash character "-"
Period character "." (deprecated)
Open and close parentheses (deprecated)
Test URLs must never contain whitespace characters.
Deprecated characters are included for backward compatibility; please do not use them as they might become unsupported in future releases. Whenever possible, use short names to make best use of screen real estate.
Note: When the result file (.jtr ) is created, the text after the last period is omitted. |
The test identifier may only contain the following characters:
ISO Latin 1 characters A-Z and a-z
Digits 0-9
Underscore character "_"
An test path can be used by a user to specify a particular subset of tests in the test suite hierarchy; for example, in the Tests tab of the JavaTest configuration editor (Standard Values view).
Initial URLs specify a set of tests in the following ways:
A folder that contains one or more tests
A file that contains one or more tests
A single, complete test URL
The test path conforms to the rules specified in the previous section, but is not required to resolve the complete URL. If the test path is an incomplete test URL (for example, a folder), the JavaTest harness generates a list of the tests' URLs contained hierarchically beneath it.
Test suites use the exclude list mechanism to identify tests that should not be run. The JavaTest harness consults the exclude list when it selects tests to run, and does not run the tests on the list. Excluded tests normally appear as filtered out in the JavaTest test tree.
When the JavaTest harness is used with TCK test suites, the exclude list mechanism is used to determine the correct set of tests that must be executed for certification. The exclude list mechanism is a mechanism for "removing" broken or invalid tests in the field without having to ship a new test suite.
The exclude list is a four-column table that uses ISO Latin 1 (ISO 8859-1) character encoding. Lines that are completely empty or contain only whitespace (space, tab, newline) are allowed. Comment lines begin with the "#" character. Each line has the following format:
Test_URL[
Test_Cases]
BugID_List Keywords Synopsis
For example:
api/index.html#attributes[Char2067] 4758373 reference,test Bug is intermittent
Table B-1 Exclude List Field Descriptions
Field | Description |
---|---|
Test_URL[Test_Cases] |
The URL of the test to be excluded. This field can specify an entire test or can use the Test_Cases parameter to specify a subset of its test cases. The test cases field is optional and the brackets must not be present if the field is empty. |
BugID_List |
A comma-separated (no spaces) list of bug identifiers associated with the excluded test. |
Keywords |
A comma-separated (no spaces) list of keywords that can be used to classify exclusions. The particular values are project specific. |
Synopsis |
A short comment that describes the reason the test is excluded. This optional field is recommended because it helps track entries on the exclude list. |
Each field is separated by spaces and/or tabs. A line break terminates the entry. There is no way to indicate that the entry continues on the next line. Comments can appear on any line of the file, see the rules below.
Although it is not recommended, you can omit the synopsis, keywords, or bugID_List field; however, the entry is only valid if everything to the right of the omitted field is also omitted. For example, you cannot omit a bugID and include a keyword; but you can include a bugID and omit the keywords and synopsis.
Entries must not specify overlapping test cases. For example, you cannot exclude an entire test and then exclude a test case inside that test. These two entries can never appear in the same file:
api/java_lang/Character/index.html#attributesFullRange
api/java_lang/Character/index.html#attributesFullRange[Character2067]
The URL must specify a specific test or test case. Entire subsections of the test suite cannot be excluded with a single entry. Continuing with the API example, if a test suite is rooted at the ts_dir\tests
directory and the index.html
file contains many test descriptions, all of the following test URLs are invalid:
api api/java_lang/ api/java_lang/Character api/java_lang/Character/index.html tests/api/java_lang/xyz java_lang/xyz
You can exclude individual test cases within a test description by appending a list of those tests cases at the end of the test URL. The list of test cases must be enclosed within square brackets. The list of test cases is separated by commas with no internal whitespace. There is no whitespace between the end of the test URL and the opening square brackets. The following figure shows valid test URL entries:
vm/instr/ifnull/ifnull003/ifnull00303m1/ifnull00303m1.html api/java_beans/beancontext/BeanContextMembershipEvent/index.html#Constructor api/java_lang/Character/index.html#attributesFullRange[Character2067] api/SystemFlavorMap/index.html#method[SystemFlavorMap0001,SystemFlavorMap0004]
For information about constructing valid test URLs, see Test URL Specification.
The list of bug IDs is separated by commas, and contains no whitespace characters. Items in the BugID_List are entries that correspond to a bug in a bug tracking system. Letters, integers, dashes and underscore characters are valid.
It is recommended that keyword entries be no longer than 20 characters long. Keyword entries must start with a letter and can contain letters, numbers, and the underscore character. The list of keywords is separated by commas, without any whitespace characters.
Any description or notes about a particular entry. There are no special restrictions on the content of this field. It is recommended that the field contain no more than 100 characters.
Comments always extend from column zero of a line to end of the line. To be a comment line, the character in column zero must be "#
"; two consecutive "#
" characters at the beginning of a line are allowed, but the use of three or more is reserved.
Optional (but recommended) header lines can be added to your exclude list file to improve readability. Header lines always begin with "###
" and can be used to designate titles, and revision histories. The format is:
### header_type headingcontent...
The case-sensitive header type specification is separated from the "###
" prefix by white space characters, the heading content is separated from the header type specification by more whitespace characters. These values should appear only once in any exclude list file, and it is recommended that they be placed at the top of the file. Currently, the only supported header type is "title". The title describes the exclude list and must be terminated with a newline.
The following is an example of a valid exclude list:
### title My example exclude list ### revised Mon Jul 23 18:15:04 PDT 2001 api/java_lang/runtimetest.java # this is a comment line api/index.html#attributes[Char2067] 1234567 reference,test Invalid assumption ## this is another comment line api/mytest.java#1 1234568,987654321 spec