Filtering Tests with Exclude Lists

A P P E N D I X D

If you have one or more tests that should not be run on a device, you can filter them with an exclude list that you create with a text editor and associate with a template (if you are an administrator) or a configuration. An exclude list filter is like a keyword filter. It does not remove or hide tests but prevents them from being added to test bundles. If you set View Filter to Current Configuration, excluded tests are colored gray.

Note - This feature might be modified in the future.

This appendix covers these subjects:

Creating an Exclude List

Associating an Exclude List with a Configuration or Template

Creating an Exclude List

An exclude list is a text file whose extension is .jtx. It must be located on a file system that is accessible to harnesses whose templates and configurations refer to it. Multiple templates or configurations can share an exclude list.

Note - Create exclude lists with care. Errors might not be reported or might cause unpredictable behavior.

An exclude list can contain blank lines, comment lines, and test lines.

Comment Lines

Use comment lines to identify an exclude list, and to group and document test lines. A comment line begins with a single “#” character.

Note - Do not write a comment beginning with “###”. This sequence is reserved for possible future use.

Test Lines

Each test line in an exclude list file contains the fully qualified name of a package, class, or test case. Excluding a package or class is shorthand for excluding its contents (analogous to deleting a file system directory). Forward slashes are required in test names.

Here are three examples with comment annotations (ignore line breaks):

# Exclude a test case
CLDC_(JSR_30_and_JSR_139)/com/sun/cldc/cldc1_0/functional/cpack/c04_1/c0410201/test0410201#testcase1
 
# Exclude all cases in a class
Scalable_Vector_Graphics_(JSR_226)/com/sun/jsr226/functional/animation/ControlAnimation
 
# Exclude all packages, classes, and cases in a package Benchmark/com/sun/benchmark/scenarios

To obtain the fully qualified names of tests, do one of the following:

Select the test names in the graphical interface’s Passed, Failed, Error, or Not Run tabs, right-click, and choose Copy > Names as multiple lines. Then paste into a text editor.

Alternatively, after running the tests, create a plain text report and copy the fully qualified names from it.

You can add up to three optional fields after the test name. Each field is separated by spaces and/or tabs. A line break terminates the entry. There is no way to indicate that an entry continues on the next line. The three optional fields are:

BugIDs: A comma-separated (no spaces) list of bug identifiers associated with the excluded test. Your organization chooses the bug identifiers, typically from your bug tracking system. Letters, integers, dashes and underscore characters are valid in BugIDs.

Keywords: A comma-separated (no spaces) list of keywords that can be used to classify exclusions. The keywords are organization-specific and are unrelated to the configuration and template keywords used to select tests. Keyword entries must start with a letter and can contain letters, numbers, and the underscore character. It is good practice to limit keyword names to 20 characters.

Synopsis: The reason the test is excluded. It is good practice to limit synopses to 100 characters.

If you omit an optional field, you must also omit the fields to its right. For example, if you do not specify keywords, you must not specify a synopsis. If you do specify keywords, synopsis is optional, but BugIDs is required.

Example

CODE EXAMPLE D-1 shows a simple exclude list. Part of each test name (...) has been omitted to fit the lines into the available space.

CODE EXAMPLE D-1 Simple Exclude List
# Exclude list for handset Alpha, software version 2.0.2 # Revised Mon Jul 23 18:15:04 PDT 2007 # Test name only AMMS.../camera/CameraControlTest#checkExposureModes # Two BugIDs, Keyword CLDC.../TableLookupSwitch/CaseWithReturn 22720,22333 notSupported # BugID, Keyword, Synopsis MIDP.../write 33655 spec Interpretation dispute

CODE EXAMPLE D-1 Simple Exclude List

# Exclude list for handset Alpha, software version 2.0.2 
# Revised Mon Jul 23 18:15:04 PDT 2007
 
 
# Test name only
AMMS.../camera/CameraControlTest#checkExposureModes
 
# Two BugIDs, Keyword
CLDC.../TableLookupSwitch/CaseWithReturn 22720,22333 notSupported
 
# BugID, Keyword, Synopsis
MIDP.../write 33655 spec Interpretation dispute

Associating an Exclude List with a Configuration or Template

Direct the harness to use an exclude list by associating it with a configuration (or if you are an administrator, with a template). A template or configuration can name multiple exclude lists, in which case their effect is additive.

Specifying an Exclude List with the Configuration or Template Editor

1. Set Specify an Exclude List = Yes

2. Specify Exclude List Files, click Add.

3. Select the exclude list file.

You can remove an exclude list from a configuration or template by selecting it and clicking Remove. Remove does not delete the file. Ignore the Up and Down buttons.

Specifying an Exclude List with the Quick Set Editor

1. Choose Configure Edit Quick Set > Exclude List.

2. Select Other, click Add.

3. Select the exclude list file.

You can remove an exclude list from a configuration or template by selecting it and clicking Remove. Remove does not delete the file. Ignore the Up and Down buttons.