Tests can be grouped together into branches for easy management. Once entered, they can be associated with the requirements they address or any issues that result from running the test.
When you run a test, you can specify the workstation to run it on. Workstations are configured using the Systems Manager. To run Oracle OpenScript tests on remote systems, you must also install the Oracle Application Testing Suite Remote Agent on the remote system and configure Remote Agent Service login credentials. See the "Installing the Oracle Load Testing Remote Agent" section in the Oracle Application Testing Suite Installation Guide for additional information about installing the agent and configuring the login credentials.
In addition, tests can be scheduled to run once at a specified day and time, daily at a specified time, or weekly at a specified day and time.
Test Set Folders can be used to group test sets. Test Set Folders are automatically numbered when you enter them. You can reorder them using the Move buttons. To add a test folder:
Select the Test Execution tab.
Click the Add button.
Enter a name for the test folder.
Select Test Folder in the Type field.
This dialog box has the following options:
Name - displays the folder or test set name.
Type - displays the type, as follows:
Test Folder - high level descriptions for grouping test sets together.
Test Set - a list of tests that are sequentially run together that can also be scheduled. In addition, the overall results are tracked and displayed.
Owner - select the user that owns the folder. Only users with read permission for this project are listed.
Functionality - enter a description of the functionality being tested.
Priority - displays the priority of the folder. These options can be changed by your system administrator.
Description - displays the folder description.
Attachment - lets you add an attachment or link to this folder.
File - lets you add an attachment to the Attachments section.
Browse - displays the Choose File dialog box for selecting the file to attach.
Link - lets you add a link to the Links section.
Title - enter a descriptive title for the link. This will be displayed in the Links section.
Link - enter the URL. For example, http://www.company.com. If a title is not entered, the URL is displayed in the links section.
Reset - clears the fields and returns the dialog box options to the starting state.
Enter a description of the functionality of the folder.
Select the owner and priority and enter any descriptive information in the Functionality and Description fields.
Add any attachments.
Click Save.
Test Sets specify the tests to execute. Test Sets are automatically numbered when you enter them. You can reorder them using the Move buttons. To add a test set:
Select the Test Execution tab.
Click the Add button.
Enter a name for the test set.
Select Test Set in the Type field.
This dialog box has the following options:
Name - displays the test name.
Type - displays the type, as follows:
Test Folder - high level descriptions for grouping test sets together.
Test Set - a list of tests that are sequentially run together that can also be scheduled. In addition, the overall results are tracked and displayed.
Owner - select the user that owns the test set. Only users with read permission for this project are listed.
Functionality - enter a description of the functionality being tested
Priority - displays the priority of the test set. These options can be changed by your system administrator.
Description - displays the test set description.
Attachment - lets you add an attachment or link to this test set.
File - lets you add an attachment to the Attachments section.
Browse - displays the Choose File dialog box for selecting the file to attach.
Link - lets you add a link to the Links section.
Title - enter a descriptive title for the link. This will be displayed in the Links section.
Link - enter the URL. For example, http://www.company.com. If a title is not entered, the URL is displayed in the links section.
Reset - clears the fields and returns the dialog box options to the starting state.
Enter a description of the functionality of the folder.
Select the owner and priority and enter any descriptive information in the Functionality and Description fields.
Add any attachments.
Click Save.
Once you have created a Test Set, you can add tests to it. Oracle OpenScript tests and manual test can be added to a Test Set. The same test can be added more than once. The tests in the group are run sequentially, and the group can also be scheduled. To add tests:
Click the Test Execution tab.
Select the Test Set to which you want to add tests.
In the right pane, click Add/Edit in the Test List area to display the Tests dialog box. There are two ways you can select tests. You can search for them or select them from the tree view.
To select tests by searching for them:
Click the Find tab.
Saved filters - lists the saved filter configurations. Select a saved filter to edit or delete it.
Save - displays the Save Criteria dialog box for saving the settings. Filters do not have to be saved.
Delete - deletes the selected saved filter.
Search for - displays whether you are searching requirements, tests, or issues.
that match - select how to apply the filtering criteria.
all - match all of the configured criteria. This setting places an AND between the criteria. For example, if you wanted to view all issues with a high priority that are assigned to you, use this option.
any - match any of the configured criteria. This setting places and OR between the criteria. For example, if you wanted to view all issues with either a high priority or a high severity, use this option.
a combination - this setting lets you combine AND and OR. For example, if you wanted to view all issues assigned to Mary or Caren with a high priority, use this option. This option lets you select how to group the criteria using parenthesis as required. For example:
(((Assigned to = Mary OR Assigned to = Caren) AND Priority = High) OR Severity = High)
<left parenthesis> - select the left parenthesis up to three.
Field - select the field that you want to use to select the items to display.
Operator
Equals - causes only items that match the value for the selected field to be displayed.
Not Equal to - causes only items that do not match the value for the selected field to be displayed.
Value - enter or select the value of the field that you want to use to select the items to display.
<right parenthesis> - select the right parenthesis up to three.
<operator> - select the operator to use, AND or OR.
<delete> - deletes the corresponding criteria.
<expression> - shows the filtering expression you are creating as you select criteria.
Results - displays the nodes that match the search criteria.
Select - places the selected nodes in the Selected Tests field.
Select All - places all nodes in the Results field in the Selected Tests field.
Selected Tests - displays the selected nodes.
<up arrow> - moves the selected test up one place.
<down arrow> - moves the selected test down one place.
Remove - removes the selected nodes from the Selected Tests field.
Remove All - removes all nodes from the Selected Tests field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Enter the search criteria or select a saved filter and click Search.
Select the tests you want to add or click Select All to add them all.
Click OK.
To select tests using the tree view:
Click the TreeView tab.
<test list> - double click tests to add and delete them from the list of selected tests or select the tests you want to add and click Select.
Select - places the selected test in the Selected Tests field. The Select button is enabled only when a test node (non-folder node) is selected. The Select All button will be disabled.
Select All - places all tests under the folder selected in the tree view in the Results field in the Selected Tests field. The Select All button is enabled only when a test folder is selected in the tree view. When you click Select All, the tests under the selected folder and any sub-folders will be added to the Test Set. The Select All button will then be disabled until you select any folder again to avoid clicking twice and accidentally adding a test folder twice.
Selected Tests - displays the selected nodes.
Remove - removes the selected nodes from the Selected Tests field.
Remove All - removes all nodes from the Selected Tests field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Select each test you want to add and click Select or select a test folder and click Select All to add all tests under the selected folder. If you select a test folder and click Select All, you must select a test folder in the tree to re-enable the Select All button.
To remove nodes from the Selected Tests field, select the nodes and click Remove or click Remove All to remove all nodes.
Click OK.
There are two ways to associate test sets with requirements. You can search for test sets that contain designated values in one or more fields or you can select tests from a tree view.
To associate requirements with a test using search:
Select the test set with which you want to associate requirements.
Select Add/Edit from the Associated Requirements section of the right pane.
Saved filters - lists the saved filter configurations. Select a saved filter to edit or delete it.
Save - displays the Save Criteria dialog box for saving the settings. Filters do not have to be saved.
Delete - deletes the selected saved filter.
Search for - displays whether you are searching requirements or issues.
that match - select how to apply the filtering criteria.
all - match all of the configured criteria. This setting places an AND between the criteria. For example, if you wanted to view all issues with a high-priority that are assigned to you, use this option.
any - match any of the configured criteria. This setting places and OR between the criteria. For example, if you wanted to view all issues with either a high priority or a high severity, use this option.
a combination - this setting lets you combine AND and OR. For example, if you wanted to view all issues assigned to Mary or Caren with a high priority, use this option. This option lets you select how to group the criteria, using parenthesis as required. For example:
(((Assigned to = Mary OR Assigned to = Caren) AND Priority = High) OR Severity = High)
<left parenthesis> - select the left parenthesis up to three.
Field - select the field that you want to use to select the items to display.
Operator
Equals - causes only items that match the value for the selected field to be displayed.
Not Equal to - causes only items that do not match the value for the selected field to be displayed.
Value - enter or select the value of the field that you want to use to select the items to display.
<right parenthesis> - select the right parenthesis up to three.
<operator> - select the operator to use, AND or OR.
<delete> - deletes the corresponding criteria.
<expression> - shows the filtering expression you are creating as you select criteria.
Results - displays the nodes that match the search criteria.
Select - places the selected nodes in the Associated Nodes field.
Select All - places all nodes in the Results field in the Associated Nodes field.
Associated Nodes - displays the selected nodes.
Remove - removes the selected nodes from the Associated Nodes field.
Remove All - removes all nodes from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Enter the search criteria or select a saved filter and click Search.
Select the requirements you want associated with this test set or click Select All to associate all requirements that matched the search criteria.
Click OK. The requirements will be listed in the right pane under associated requirements.
To associate requirements with a test set using the tree view:
Select the test with which you want to associate requirements.
Select Add/Edit from the Associated Requirements section of the right pane.
<test list> - double click tests to add and delete them from the list of associated tests or select the tests you want to add and click Select.
Select - places the selected test in the Associated Nodes field.
Select All - places all tests in the Results field in the Associated Nodes field.
Associated Nodes - displays the selected nodes.
Remove - removes the selected nodes from the Associated Nodes field.
Remove All - removes all nodes from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Select the requirements you want associated with this test or click Select All to associate all requirements that matched the search criteria.
To remove nodes from the Associated Nodes field, select the nodes and click Remove or click Remove All to remove all nodes.
Click OK. The requirements will be listed in the right pane under associated requirements.
There are two ways to associate issues with test sets. You can search for issues that contain designated values in one or more fields or you can select issues from a tree view.
To associate issues with a test set using search:
Select the test set with which you want to associate issues.
Select Add/Edit from the Associated Issues section of the right pane.
Saved filters - lists the saved filter configurations. Select a saved filter to edit or delete it.
Save - displays the Save Criteria dialog box for saving the settings. Filters do not have to be saved.
Delete - deletes the selected saved filter.
Search for - displays whether you are searching requirements or issues.
that match - select how to apply the filtering criteria.
all - match all of the configured criteria. This setting places an AND between the criteria. For example, if you wanted to view all issues with a high-priority that are assigned to you, use this option.
any - match any of the configured criteria. This setting places and OR between the criteria. For example, if you wanted to view all issues with either a high priority or a high severity, use this option.
a combination - this setting lets you combine AND and OR. For example, if you wanted to view all issues assigned to Mary or Caren with a high priority, use this option. This option lets you select how to group the criteria, using parenthesis as required. For example:
(((Assigned to = Mary OR Assigned to = Caren) AND Priority = High) OR Severity = High)
<left parenthesis> - select the left parenthesis up to three.
Field - select the field that you want to use to select the items to display.
Operator
Equals - causes only items that match the value for the selected field to be displayed.
Not Equal to - causes only items that do not match the value for the selected field to be displayed.
Value - enter or select the value of the field that you want to use to select the items to display.
<right parenthesis> - select the right parenthesis up to three.
<operator> - select the operator to use, AND or OR.
<delete> - deletes the corresponding criteria.
<expression> - shows the filtering expression you are creating as you select criteria.
Results - displays the nodes that match the search criteria.
Select - places the selected nodes in the Associated Nodes field.
Select All - places all nodes in the Results field in the Associated Nodes field.
Associated Nodes - displays the selected nodes.
Remove - removes the selected nodes from the Associated Nodes field.
Remove All - removes all nodes from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Enter the search criteria or select a saved filter and click Search.
Select the issues you want associated with the test and click Select or click Select All to associate all issues that matched the search criteria.
Click OK. The issues will be listed in the right pane under associated issues.
To associate issues with a test using the tree view:
Select the test set with which you want to associate issues.
Select Add/Edit from the Associated Issues section of the right pane.
<test list> - double click tests to add and delete them from the list of associated tests or select the tests you want to add and click Select.
Select - places the selected test in the Associated Nodes field.
Select All - places all tests in the Results field in the Associated Nodes field.
Associated Nodes - displays the selected nodes.
Remove - removes the selected nodes from the Associated Nodes field.
Remove All - removes all nodes from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Select the issues you want associated with the test and click Select or click Select All to associate all issues that matched the search criteria.
To remove nodes from the Associated Nodes field, select the nodes and click Remove or click Remove All to remove all nodes.
Click OK. The issues will be listed in the right pane under associated issues.
There are two ways to associate test plans with test sets. You can search for test plans that contain designated values in one or more fields or you can select test plans from a tree view.
To associate test plans with a test set using search:
Select the test set with which you want to associate test plans.
Select Add/Edit in the Associated Test Plans section of the right pane.
Saved filters - lists the saved filter configurations. Select a saved filter to edit or delete it.
Save - displays the Save Criteria dialog box for saving the settings. Filters do not have to be saved.
Delete - deletes the selected saved filter.
Search for - displays whether you are searching issues, requirements, tests, test plans, or test sets.
that match - select how to apply the filtering criteria.
all - match all of the configured criteria. This setting places an AND between the criteria. For example, if you wanted to view all issues with a high priority that are assigned to you, use this option.
any - match any of the configured criteria. This setting places and OR between the criteria. For example, if you wanted to view all issues with either a high priority or a high severity, use this option.
a combination - this setting lets you combine AND and OR. For example, if you wanted to view all issues assigned to Mary or Caren with a high priority, use this option. This option lets you select how to group the criteria using parenthesis as required. For example:
(((Assigned to = Mary OR Assigned to = Caren) AND Priority = High) OR Severity = High)
<left parenthesis> - select the left parenthesis up to three.
Field - select the field that you want to use to select the items to display.
Operator
Equals - causes only items that match the value for the selected field to be displayed.
Not Equal to - causes only items that do not match the value for the selected field to be displayed.
Value - enter or select the value of the field that you want to use to select the items to display.
<right parenthesis> - select the right parenthesis up to three.
<operator> - select the operator to use, AND or OR.
<delete> - deletes the corresponding criteria.
<expression> - shows the filtering expression you are creating as you select criteria.
Results - displays the nodes that match the search criteria.
Select - places the selected nodes in the Associated Nodes field.
Select All - places all nodes in the Results field in the Associated Nodes field.
Associated Nodes - displays the selected nodes.
Remove - removes the selected nodes from the Associated Nodes field.
Remove All - removes all nodes from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Enter the search criteria or select a saved filter and click Search.
Select the test plans that you want to associate and click Select or click Select All to associate all test plans that matched the search criteria.
Click OK. The test plans are listed in the right pane under associated test plans.
To associate test plans with a test set using the tree view:
Select the test set with which you want to associate test plans.
Select Add/Edit in the Associated Test Plans section of the right pane.
<test list> - double click test plans to add and delete them from the list of associated test plans or select the test plans you want to add and click Select.
Select - places the selected test plan in the Associated Nodes field.
Select All - places all test plans in the Results field in the Associated Nodes field.
Associated Nodes - displays the test plans that are associated with this test set.
Remove - removes the selected test from the Associated Nodes field.
Remove All - removes all tests from the Associated Nodes field.
OK - closes the dialog box and saves changes.
Cancel - closes the dialog box without saving changes.
Select the test plans you want associated with this test set and click Select or click Select All to associate all test plans.
To remove nodes from the Associated Nodes field, select the nodes and click Remove or click Remove All to remove all nodes.
Click OK. The test plans will be listed in the right pane under associated test plans.
You can email a test set to one or more people. For email to work, the SMTP server must be configured using the Oracle Test Manager Administrator. To email a test:
Click the Test Execution tab.
Select the test set that you want to email and click Email.
Send to - enter the email addresses of the people to whom you want to email this test. Separate email addresses by a comma.
To - displays the Select Email recipients dialog box for selecting recipients from a list.
Subject - enter the text that you want to appear in the subject of the email. The default includes the name and id of the test.
Message - enter the text that you want to appear in the message.
Enter the recipient email addresses separated by a comma or click To to display the Select Email Recipients dialog box.
<recipient list> - lists users in you database that have email notification enabled.
Select - adds the selected recipients to the To field.
To - displays the selected recipients separated by a comma.
Select the recipients that you want to receive the test and click Select. Note that only users in your database that have email notification enabled are listed in the dialog box.
Click Done when you are finished.
Change the subject if necessary.
Enter a message if necessary.
Click OK.
The Systems Manager lets you configure the systems on which you will be running tests. When you run or schedule a test, you can specify the workstation on which to run the test.
To add a system:
Select Systems from the Tools menu. The Systems Manager dialog box is displayed.
New - displays the Systems dialog box for adding a new system.
Edit - displays the Systems dialog box for editing the selected system.
Delete - deletes the selected system.
Name - lists the configured systems.
Click New.
Name - enter the system's name.
Host Name or IP - enter the system's host name or IP address.
Username - enter the user name for agent authentication. The Username is the user name specified for the agent Authentication Profile in the Oracle Load Testing Agent Authentication Manager. The username for the default agent Authentication Profile is JMSAdmin. To view other defined agent Authentication Profile Usernames, select Oracle Application Testing Suite from the Programs Start menu, then select Oracle Load Testing Agent Authentication Manager from the Tools submenu. Select an agent Authentication Profile to view the details. On Linux machines, use:
<instdir>/jdk/jre/bin/javaw -jar <instdir>/agentmanager/AMAuthManager.jar
to start the Oracle Load Testing Agent Authentication Manager.
Password - enter the password for agent authentication. The Password is the password specified for the agent Authentication Profile in the Oracle Load Testing Agent Authentication Manager. The password for the default agent Authentication Profile is blank. To change the password for a defined agent Authentication Profile Usernames, start the Oracle Load Testing Agent Authentication Manager and select an agent Authentication Profile to view the details and enter a new password.
Port - enter the port.
Test - attempts to access the system and displays a message stating whether the system is available.
Note:
If a firewall is running on the agent system, you must make sure Port 7 is open. The controller uses Port 7 to ping the agent system to determine if the system can be accessed or not while adding the agent system using the user interface.Enter the system's name.
Enter the system's host name or IP address.
Change the default port, if necessary.
Click Test to see if the system is available.
Click OK to close the test result window.
Click OK.
To edit a system:
Select Systems from the Tools menu.
New - displays the Systems dialog box for adding a new system.
Edit - displays the Systems dialog box for editing the selected system.
Delete - deletes the selected system.
Name - lists the configured systems.
Select the system that you want to change.
Click Edit.
Name - enter the system's name.
Host Name or IP - enter the system's host name or IP address.
Username - enter the user name for agent authentication.
Password - enter the password for agent authentication.
Port - enter the port.
Test - attempts to access the system and displays a message stating whether the system is available.
Make any changes.
Click Test to see if the system is available.
Click OK to exit the test result window.
Click OK.
You can individually run manual tests, Oracle OpenScript functional tests, third party, JUnit, or Test Sets. You can run individual tests in the Tests tab using the Run Test option on the right-click shortcut menu. You can run groups of tests in a Test Set in the Test Execution tab after creating a Test Set and adding one or more tests to the Test Set.
To run an individual Oracle OpenScript test:
Make sure you have added an OpenScript test to the test tree. See Section 6.1, "Adding Oracle OpenScript Scripts" for additional information about adding OpenScript tests.
Select the Test tab.
Select the test you want to run.
Select Run Test from the right-click shortcut menu.
This dialog box has the following options:
System - select the system on which you want to run this test. Systems are configured using the Systems Manager.
Run Name - specify a name to use to identify this test run.
Version - select the version of the test or select Other to enter a version that is not in the list.
Command line run settings - specify any additional arguments or command line settings to use when running an Oracle OpenScript script. See the Oracle OpenScript User's Guide for additional information about using command line settings.
Note:
If your administrator has created Test Run custom fields, they will appear in this dialog box.Select the system on which you want to run the test.
Change or update the Run Name.
Select the version number against which the test will run.
Specify any run settings and click Run. The script is started and plays back in the agent.
Click Refresh to show the results in the Run History section of the right pane. Click the date in the Run History field to display result parameters, summary and detail information. Click View Report in the Result Summary section to view the Results Report.
The following additional steps are required in order to run the following types of OpenScript scripts from Oracle Test Manager:
Siebel Functional
Oracle Forms Functional
Web Functional scripts that rely on system input events, such as key press or mouse click
Web Functional scripts that display modal dialog windows
It is necessary to run these scripts using an interactive desktop of a named Windows user account that is always logged in. See also "Command-Line Script Execution" in the Release Notes for additional information.
For Siebel and Oracle Forms, the named user's account must have visited the Siebel or Oracle Forms site at least once to ensure that all necessary ActiveX controls and plug-ins are installed in the named user's browser.
On the Oracle Test Manager agent machine that will run the scripts, stop the "Oracle Application Testing Suite Agent Service" and configure it to start manually.
On the Oracle Test Manager agent machine, login as the named Windows user account that will run the scripts. From a command prompt, run the following:
Windows:
$installDir\agentmanager\bin\agentmanager.cmd
Linux:
$installDir\agentmanager\bin\agentmanager.sh
This command starts a Java executable running the agentmanager, it will not "run" the service. All output is logged to the agentmanager.log and the logs of individual agents.
Note:
For Windows 7 systems, the command must be run using the elevated command line. From the Start menu, expand the Accessories, right click on the Command Prompt menu option and select Run as administrator from the shortcut menu.The named user account must remain logged into the system at all times that scripts will be run.
When executing OpenScript tests, Oracle Test Manager checks if the requested agent system is available. If the agent system is available, the test will be scheduled to run immediately. If the agent system is not available the test is added to the agent queue until the agent system resource is available. Any test or testset scheduled to run at a specific time will be put into the queue when the scheduled time comes. This means that if a test or testset was scheduled for execution at 1pm, it will be added to the queue at 1pm. If other tests are also in the queue, the test or testset scheduled for 1pm may have to wait until resources become available before execution actually begins. See Section 2.10.2, "The Agent Queue" for additional information.
To check the status of the agent queue, click the Queue icon on the Toolbar of the Test or Test Execution tabs. The agent queue status dialog box shows a list of the testsets or test scripts that have been added to the agent queue. This dialog box has the following options:
Waiting Queue - displays the status of the agent queue and any testsets or test scripts waiting in the queue.
Delete - removes the selected testset or test script from the agent queue. Deleting test sets or test script from the agent queue requires administrator user privileges.
Icon - displays the icon for either a testset or test script to identify the type of test waiting in the queue.
Testset - displays the name of the testset that was submitted for execution and is now in the agent queue awaiting execution. Testsets appear in the queue as one object; however, when the execution time comes, all the tests within the testset will be executed in the order they are specified within the testset.
Test - displays the name of the test script that was submitted for execution and is now in the agent queue awaiting execution.
User/Scheduler - displays the name of the tester who submitted the testset or test script to be executed in Oracle Test Manager or "Scheduler" if the testset or test script was initiated by the scheduler.
Submitted on - displays the date and time the testset or test script was submitted for execution.
Running Queue - display the status of testsets or test scripts currently running on the available agents.
Release - releases the selected agent from the running test. Releasing agents from the agent queue requires administrator user privileges.
Icon - displays the icon for either a testset or test script to identify the type of test running on the agent.
Run Name - displays the testset run name if the running test is a testset. This column is blank if the running test is a single test script.
Testset - displays the name of the testset running on the agent.
Test - displays the name of the test script running on the agent.
User/Scheduler - displays the name of the tester who submitted the testset or test script to be executed in Oracle Test Manager or "Scheduler" if the testset or test script was initiated by the scheduler.
Agent - displays the name of the agent system running the testset or test script.
To run an individual manual test:
Make sure you have added a manual test to the test tree. See Section 6.2, "Adding Manual Tests" for additional information about adding manual tests.
Select the Test tab.
Select the test you want to run.
Click Run Test from the right-click shortcut menu.
This dialog has the following options:
Run Name - specify a name to use to identify this test run.
Version - select the version of the test or select Other to enter a version that is not in the list.
Note:
If your administrator has created Test Run custom fields, they will appear in this dialog box.Change or update the Run Name.
Enter or select the version number and click Run.
This dialog box appears for each step in the test. It has the following options:
First - displays the first step.
Previous - displays the previous step.
Next - displays the next step.
Last - displays the last step.
Minimize/Maximize - toggles the size of the dialog box.
Start Timer - starts the timer. When the timer is on, it keeps track of how long it took to run each step and the complete test. The Duration and Iteration Total Time fields in the Result Detail Section will be populated.
Stop Timer - stops the timer. The Duration field in the Result Detail Section will be populated for the steps that were completed.
Make All - sets the result of all steps to the selected status, either Passed, Failed, Warning, or any user-defined field configured for the Status field in the Administrator.
Action - displays the action to perform.
Expected Result - displays the expected result.
Comment - displays additional information about the step.
Attachments - lists the attachments associated with this step.
Result - select the outcome of the step after performing it.
Add Issue - opens the adds issue dialog box with pre-filled data about the current test step.
[Autosave indicators] - shows the current autosave status of the test step. A blue check mark indicates the last change to the test step status has been saved. A green check mark with a timer icon indicates the autosave for the current test step status is pending.
A backup copy of test steps data is saved when the Run Manual Test window opens to run a manual test case.
Changes are posted in the database when any test step data (either comments or result) is changed.
Changes made using the Make All selection drop down are posted in the database.
When Complete Test Run or Save for Later is clicked.
If Cancel is clicked, the data is reverted to the backup copy created when the Run Manual Test window was opened.
Complete Test Run - opens the Run Manual Test - Summary for setting the completed manual test status. Set the result and click OK.
Save for Later - saves the test run status and shows a result of In Progress in the Run History section for this test run. Click Resume this run to resume testing.
Click the Start Timer button if you want Oracle Test Manager to keep track of how long it takes to complete each step. If you are using the timer the Duration and Iteration Total Time fields in the Result Detail Section will be populated. If you stop the timer or save the run only the Duration field will be populated for the completed steps.
Perform each step and select the result or use Make All to set the same result for each step.
Click Save for Later to save the results for the completed steps without finishing the test run or click Complete Test Run when you are done. You can continue a Save for Later test by clicking the Run Test button in the Run History section to resume the test.
Result - enter the overall outcome of the test.
Summary - enter additional information about the test.
If you click Complete Test Run, select the overall test result, enter any summary comments, and click OK.
When you run a Test Set, Oracle Test Manager runs the tests in the set in sequence. The same test can be added multiple times.
To run a Test Set:
Make sure you have added a Test Set to the test tree and have added the tests you want to run to the Test Set. See Section 7.2, "Adding Test Sets" for additional information about adding Test Sets. See Section 7.3, "Adding Tests to a Test Set" for additional information about adding tests to run to a Test Set.
Select the Test Execution tab.
Select the Test Set and click Create run at the top of the right pane.
Change or update the Run Name.
Enter the version number of the test set.
Specify any additional arguments or command line settings to use when running an Oracle OpenScript script. See the Oracle OpenScript User's Guide for additional information about using command line settings.
Click Save.
The Test Set run is added to the Run History section.
You can run individual tests by clicking the Run Test icon next to the individual test in the Test List. You can also set the result for all tests in the test set using the Set result link on the Actions menu. When finished running tests individually, select Complete this run on the Actions menu, then click OK to confirm. The status of the Test Set is updated in the Run History. Note that once a Test Set is completed, it is no longer editable.
You can run a complete Test Set by clicking the Launch link on the Actions menu in the Test List. Clicking the Launch link executes all non-manual tests, however, manual tests must still be executed one-by-one by clicking the Run Test button for each test.
You can add or remove tests from an existing Test Set run that has a status of incomplete. Only tests that are included in the Test Set can be added to or removed from the Test Set run.
To add or remove tests in a Test Set run:
Make sure you have created a Test Set, created a Test Set run using Create Run, and the Test List for the Test Run appears in the right pane of the Test Execution tab.
Click Add/Remove Tests to this run in the Test List section.
Add tests to or remove tests from the Test Set run, as follows:
Add tests to the Test Set run by selecting one or more tests in the Test Cases available in Test Set list and clicking Select or Select All.
Remove tests from the Test Set run by selecting the one or more tests in the Selected Tests list and clicking Remove or Remove All. Click OK to confirm.
Click OK when finished adding or removing tests in the Test Set run.
To run third-party tests:
Make sure you have added a third-party test to the test tree. See Section 6.4, "Adding Third Party Tests" for additional information about adding third party tests.
Select the Test tab.
Select the test you want to run.
Select Run Test from the right-click shortcut menu.
This dialog box has the following options:
System - select the system on which you want to run this test. Systems are configured using the Systems Manager.
Run Name - specify a name to use to identify this test run.
Version - select the version of the test or select Other to enter a version that is not in the list.
Note:
If your administrator has created Test Run custom fields, they will appear in this dialog box.Select the system on which you want to run the test.
Change or update the Run Name.
Select the version number against which the test will run and click Run.
A message stating that the test is running is displayed. A command line window is displayed showing the batch file execution.
Click Refresh to show the results in the Run History section of the right pane.
To run a JUnit test:
Make sure you have added a JUnit test to the test tree. See Section 6.5, "Adding JUnit Tests" for additional information about adding JUnit tests.
Select the Test tab.
Select the test you want to run.
Select Run Test from the right-click shortcut menu.
This dialog box has the following options:
System - select the system on which you want to run this test. Systems are configured using the Systems Manager.
Run Name - specify a name to use to identify this test run.
Version - select the version of the test select Other to enter a version that is not in the list.
Note:
If your administrator has created Test Run custom fields, they will appear in this dialog box.Select the system on which you want to run the test.
Change or update the Run Name.
Select the version number against which the test will run and click Run.
A message stating that the test is running is displayed. A command line window is displayed showing the file execution.
Click Refresh to show the results in the Run History section of the right pane.
Perform the following steps if the third party test fails to run on the remote machine:
Make sure that the Oracle Load Testing Agent Service is running on the remote machine.
Check the <installdir>\logs\agentmanager.log file on the remote machine to see if the agent service started correctly.
Test the communication port between Agent and Oracle Test Manager.
Check the agent manager log file to verify that the JavaAgent.exe was run when the third party test was started. It is possible to run the same JavaAgent.exe command line specified in the agentmanager.log file from a DOS console. Verify that the command line argument for the third party test package URL is correct. If there is no indication that a Java Agent startup request was issued, verify the Agent version and check the Oracle Test Manager server log.
If it looks like the Java Agent was started with the correct arguments, look in the remote machine's <installdir>\logs\agentmananger.log file for any exceptions. It may help to set the oracle.oats logging category in <installdir>\agent\log4jJavaAgentDebug.xml file to DEBUG. This will show more useful third party test execution details. The third party test Java Agent posts its results over HTTP to a servlet running on the Oracle Test Manager server.
If the agent.log file shows that the third party test ran and that the results were reported successfully to Oracle Test Manager, then the problem is on the Oracle Test Manager server side. Look in the Oracle Test Manager server log for exceptions.
You can set test results for one or more tests. To set test results:
Select the Tests tab.
Select the tests for which you want to set results.
Select Set test result from the right-click menu.
Selected tests - lists the test that you selected. By default, they are selected. Deselect any tests for which you do not want to set the test result.
Result - select the result that you want to apply to the selected tests.
Version - select the version that you are testing against.
Summary - enter any comments.
Deselect any tests for which you do not want to set the result.
Select the result that you want to apply to the selected tests.
Select the version you are testing against.
Enter any comments in the Summary field.
Click Save.
You can set or override the results for one or more test sets. Note that setting or overriding test set results requires "Override result" user privileges. User privileges are set in the Oracle Application Testing Suite Administrator.
To set or override test set results:
Select the Test Execution tab.
Select the test set for which you want to set or override results.
Click the Set result link in the Test List section of the right pane.
Test Set - shows the name of the Test Set.
Run Name - shows the name of the Test Set run.
Version - shows the version of the Test Set run.
Make all selected - sets all tests with selected check boxes in the Test Name list to the status selected from this list.
Test Name - lists the tests in the Test Set. Select the Test Name check box to select all tests or select individual test check boxes to select the tests to change status.
Status - shows the current status of the test. Select a status to change or override the current status.
Owner - shows the user that owns the test.
Exec On - shows the last execution date and time for the test.
Test Type - shows the test type.
Save - saves the changes to the statuses of the tests in the test set.
Reset - resets the statuses of the tests in the test set to the last saved statuses.
Deselect any tests for which you do not want to set or override the result.
Select the result from the Make all selected list that you want to apply to the selected tests.
Click Save.
You can delete test results based on the date and time the test was run. To delete test results:
Select Delete Results from the Run History section in the right pane.
This dialog box displays a list of test results by date and time.
Select the results you want to delete. You can select more than one test result by clicking on each test result that you want to delete.
Click OK.
You can view the result details for each test run of an individual test or the test details for tests run as part of Test Set.
To view test details for an individual test:
Click the Tests tab.
Select the test for which you want to view details.
In the Run History section of the right pane, click the date of the test for which you want to view details. The Test Set Run History is only included if the Include Test Set Runs in Tests Tab option is selected in the Right Hand Pane Display section of the user options (select Options from the Tools menu).
The details of the test run expand and appear under the Run History section.
To view test details for a Test Set:
Click the Test Execution tab.
Select the Test Set for which you want to view details.
If a Test Set run has been created, the Run History section appears below the Test List of the right pane. Click the date of the test run for which you want to view details.
The Test Set Run Details dialog box show the result details for the tests included in the Test Set.
Status of Test Set Run - shows the name and status information for the Test Set run.
Created by/ on - shows the name of the user who created the Test Set run and date the Test Set run was created.
Passed - shows the number and percentage of tests in the Test Set that passed.
Warning - shows the number and percentage of tests in the Test Set that returned a warning.
Failed - shows the number and percentage of tests in the Test Set that failed.
In Progress - shows the number and percentage of tests in the Test Set that are currently in progress.
Skipped - shows the number and percentage of tests in the Test Set that were skipped.
Not Run - shows the number and percentage of tests in the Test Set that were not run.
Result Parameters - shows the run result parameters for the test.
Field - lists the names of the fields specified for the Test Set run.
Value - lists the values specified for each field for the Test Set run.
Test Name - lists the names of the tests in the Test Set. Click on a highlighted test name to view the result details for that specific test in the Result Parameters section below. Test names for tests that have not yet been run will not be highlighted.
Duration - shows the amount of time required to run each test in the Test Set.
Result - shows the run result of the each test in the Test Set.
Summary - shows any summary information entered for each test in the Test Set.
Result Details - shows the selected test name and lists the run result details for the test.
Ran by - shows the name of the user who ran the test.
Exec On - shows the date and time the test was executed.
System - shows the name of the system on which the test was executed. System only appears for automated tests.
For OpenScript tests, the Result Report appears in the Result Details pane.
For manual, third party, and JUnit tests, the following result details appear:
Action - lists the actions performed for the selected test.
Duration - shows the amount of time required to run each action.
Result - shows the run result of the each action in the test.
Summary - shows any summary information entered for each action in the test.
For third party and JUnit tests, a View Report link is also available to open the result report.
The Results report is displayed in a separate browser window and gives detailed information about the test run. To view the Results report:
Select the test for which you want to view the report.
In the Run History section of the right pane, click the date of the test for which you want to view the report.
Click View Report in the Result Summary section of the right pane.