Enabling Test Selection
Test selection in a test run is of value to a user when a test suite includes tests for optional features that were not implemented. Because the assumption of tests is that the target implementation must support these features, the implementation fails tests that are not be applicable to it. This section describes how a developer can enable the filtering of the tests (test selection) in a test suite.
Factors and Mechanisms for Test Selection
The following factors affect how tests are selected for a test run and describe several mechanisms that exist for users to enable test selection.
■Keywords - A standard test selection mechanism provided by the harness that enables a user to filter tests based on specific descriptive keywords assigned by a developer to a test.
This mechanism is convenient for logical test grouping. The following are examples of test filtering using keywords in the test description file.
■Type (distributed, interactive, or OTA)
See “Required Distributed Test Keyword” on page 44, “Required Interactive Test Keywords” on page 50, and “Required OTA Test Keyword” on page 54.
■Security mode (trusted or untrusted)
See “Using Keywords to Mark Security Tests” on page 61.
■Custom keywords specified by the user
See Appendix C for a list of the Framework keywords.
■Prior test status - A
The test developer is not required to perform any action to enable this filtering mechanism. Filtering is normally set by the user through the Prior Status question in the standard configuration interview.
■Exclude list - Excludes certain test cases from the certification test run.
The exclude list (testsuite.jtx)file identifies the tests in a test suite that should not be run. The exclude list is located in the test suite lib directory. Test developers use the following format to add tests to the exclude list:
Test-URL[Test-Cases] BugID Keyword
See Chapter 3 for additional information about the exclude list.
