Skip Headers
Oracle® Enterprise Manager Cloud Administration Guide
12c Release 4 (12.1.0.4)

E28814-13
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

26 Using the Testing as a Service Portal

After the TaaS environment has been setup, Test Administrators and Test Designers must perform their up front tasks before the Tester can start testing. In general, the tasks are:

Table 26-1 Tasks in TaaS

Task How To Role

Ensure Prerequisites Are Met

Prerequisites

Test Administrator, Test Designer, Tester

Add application

Setting Up Application Types

Test Administrator

Define Test Drivers

Setting Up Oracle Load Testing Test Drivers

Setting Up Oracle Functional Testing Test Drivers

Setting Up Custom Test Driver Types

Test Administrator

Set Up Quotas

Setting Up Quotas

Test Administrator

Publish Test Environments and Test Assets

Publishing Test Environments, Test Assets, and Test Drivers

Test Administrator

Set Up Chargeback Service

Setting Up Chargeback Service

Test Administrator

Create Test Asset

Creating Test Assets - General Information

Creating Test Asset - Oracle Load Testing

Creating Test Asset - Oracle Functional Testing

Creating Test Asset - Custom Test

Test Designer

Create Test Environments

Creating Test Environments

Test Designer

Verify Available Test Assets and Environments

Verifying Available Test Assets and Environments

Tester

Create Tests

Creating Tests

Creating Tests - Oracle Load Testing Tests

Creating Tests - Oracle Functional Testing Tests

Creating Tests - Custom Tests

Tester

Create and Run Trials

Creating and Running Trials

Creating and Running Oracle Load Testing Trial

Creating and Running Oracle Functional Testing Trial

Tester

Monitor On-Going Trials

Monitoring On-Going Trials

Tester

Compare Trial Results

Comparing Trial Results

Tester

View Activity and Usage Information

Viewing Activity and Usage Information

Tester

View Deployment Information

Viewing Deployment Information

Tester


26.1 Prerequisites

Ensure the following prerequisites are met when using the Testing Cloud Self Service Portal.

Before You Do Anything

  • Ensure Enterprise Manager Cloud Control is set up to manage the cloud environment.

  • Ensure Self Service Application Plug-in is installed because Testing as a Service is part of the SSA plug-in.

Prerequisites Specific to Test Designer

  • Has the privileges to access the Testing as a Service (TaaS) Test Designer options. This user must have the EM_TAAS_DESIGNER role.

  • Must have knowledge about what applications are to be tested using Enterprise Manager and their structure (such as database servers, application servers, and configurations for the servers.)

  • Has created test scripts using the Oracle Application Testing Suite scripting tool OpenScript, the Oracle Load Testing scenario, and the self-contained .scnzip file, and the Oracle Functional Testing scenario with its self-contained .zip file. For information on how to create and record scripts, refer to the Oracle Application Testing Suite documentation available by downloading the documentation set from:

    http://www.oracle.com/technetwork/oem/downloads/index-084446.html

  • Must be familiar with Enterprise Manager Cloud Control features.

  • Has created Test Assets based on scripts and scenarios for Oracle Load Testing, Oracle Functional Testing, or other test drivers and Test Environments based on Oracle assemblies.

  • Has deploy privileges over the Cloud Zones, otherwise he cannot create deployment plans.

  • Has access to the Cloud Infrastructure.

Prerequisites Specific to Test Administrator

  • Has the privileges to access the Testing Cloud Self Service Portal Test Administrator options. This user must have the EM_SSA_ADMINISTRATOR role.

  • Must have knowledge about what applications are being deployed, managed, and tested using Enterprise Manager and their structure (such as database servers, application servers, and configurations for the servers.)

  • Must have basic DB administration knowledge and have a good understanding of virtualization technologies.

  • Must be familiar with Enterprise Manager Cloud Control features.

  • Ensures that the Test Designer has created Test Assets and Test Environments.

  • Ensures that the Testers have the privileges to access the Testing Cloud Self Service Portal user options.

  • Publishes the test assets, test environments, and test drivers to the Tester role and the EM_SSA_USER role.

  • Has deploy privileges over the Cloud Zones, otherwise he cannot create deployment plans.

  • Has access to the Cloud Infrastructure.

Prerequisites Specific to Tester and SSA Users

  • Must have the EM_SSA_USER role.

  • Ensures that the Test Designer has created Test Assets and Test Environments.

  • Ensures that the Test Administrator has assigned Test Assets, Test Environments, and Test Drivers to the Tester role (EM_SSA_USER).

  • Has the privileges to access the TaaS Tester options.

  • Has the privileges to access the Testing Cloud Self Service Portal user options.

  • Must be familiar with Enterprise Manager Cloud Control features.

26.2 Test Administrator Tasks

Enterprise Manager allows TaaS Test Administrators to manage the application infrastructure for testing activities, define user activities for self-service testing, and manage test environments and test assets.

Specific responsibilities include:

  • Manage the test drivers.

  • Manage user roles and access to test drivers (specify which users have access to which plug-in.)

  • Manage access to test environments and test assets based on users and roles.

  • Assign the OVM and Zone privileges to the following roles and users:

    • OVM - View privilege to Test Designer and Test Administrator

    • Zone - Operator privilege to Test Designer and Test Administrator

    Since TaaS cannot know before hand the zones and Oracle Virtual Machines present in user deployments, the Super Administrator, Cloud Administrator, or respective target owner will decide which zones and Oracle Virtual Machines are to be used.

The following tasks are performed by the Test Administrator.

26.2.1 Setting Up Application Types

The Testing Cloud Self Service Portal allows Test Administrators to manage the application infrastructure for testing activities, define user activities for self-service testing, and manage test environments and test assets. Specific responsibilities include:

  • Manage the test drivers.

  • Manage user roles and access to test drivers (specify which users have access to which plug-in.)

  • Manage access to test environments and test assets based on users and roles.

  • Assign the OVM and Zone privileges to the following roles and users:

    • OVM - View privilege to Test Designer and Test Administrator

    • Zone - Operator privilege to Test Designer and Test Administrator

    Since TaaS cannot know before hand the zones and Oracle Virtual Machines present in user deployments, the Super Administrator, Cloud Administrator, or respective target owner will decide which zones and Oracle Virtual Machines are to be used.

Note: Ensure the prerequisites are met before using the Testing Cloud Self Service Portal (see Prerequisites).

Follow this process to set up application types to be tested.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Test Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Application Types located in the left panel to define application types.

  4. Click Add to define the name and description of the application type to make available for testing.

  5. Define the name and description and click Save.

26.2.2 Setting Up Test Drivers

A Test Driver is the package of software that Enterprise Manager uses to generate tests or workload against test systems based on a deployment plan.

Oracle provides two types of drivers: Oracle Load Testing and Oracle Functional Testing. The Oracle Load Testing test driver is used for load, scalability, and stress testing. This allows you to simulate hundreds or thousands of users using the software at the same time. Oracle Functional Testing test driver is used for functional and regression testing.

Since Oracle Load Testing and Oracle Functional Testing are created by Oracle, Oracle knows the specifics of the test drivers and provides seamless integration to these drivers.

You can register your own type of test drivers as long as you can build a deployment plan around your assembly which contains the testing product and software. However, when you create a custom test driver, you need to direct your scripts on how to launch and monitor your tests.

The Custom Test Driver allows Testers to use test tools from third party vendors together with TaaS. Use Custom Test Driver Types to define and register test-driver types which meet your specific testing needs.

The following sections explain how to register specific test drivers.

26.2.3 Setting Up Oracle Load Testing Test Drivers

The Oracle Load Testing test driver is used for load, scalability, and stress testing. This allows you to simulate hundreds or thousands of users using the software at the same time.

Note: Ensure the prerequisites are met before setting up Oracle Load Testing Test Drivers (see Prerequisites).

To register an Oracle Load Testing test driver, follow these steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Test Drivers located in the left panel. The Test Driver page appears.

  4. From the Register menu, select Oracle Load Testing.

  5. Provide the following information:

    • Name: Enter a meaningful name for this Test Driver.

    • Deployment Plan: Click the magnifying glass for selecting a deployment plan on which the test driver is based. In the resulting dialog box, select the Deployment Plan to be used for the test driver and click Select.

      Ensure that Test Designers and Test Administrators create the deployment plan "on a zone". Also, ensure they have correct privileges to this zone. For information regarding deployment plans, see Oracle Enterprise Manager Lifecycle Management Administrator's Guide.

    • Version: Version field is automatically filled in. The information is derived from the deployment plan.

    • Agent Deployment Plan: Click the magnifying glass for selecting an Agent Deployment Plan.

      The Agent deployment plan is a configuration file that defines an independent Agent machine on the cloud. Since TaaS registers the Agent deployment plan with a test driver, the Agent machine instance is used together with the test driver deployment plan to create the load test.

      This allows TaaS to create independent agent machines on the cloud, allowing load generation capacity of a test driver to be scaled up depending upon the trial configuration and demand.

      Ensure that the Agent Deployment Plan is scallable.

      Note: Both the Deployment Plan's version of the Application Testing Suite and the Agent Deployment Plan's version of the Application Testing Suite must be same.

    • Description: Enter a description for the Test Driver.

  6. Click Register.

To view the details of a test driver that has already been registered, highlight the test driver in the table. The deployment plan, agent deployment plan, and assembly structure information are provided.

26.2.4 Setting Up Oracle Functional Testing Test Drivers

The Oracle Functional Testing Test Driver is used for functional and regression testing and validating application functionality.

Note: Ensure the prerequisites are met before setting up Oracle Load Testing Test Drivers and Oracle Functional Testing Test Drivers (see Prerequisites).

To register an Oracle Functional Testing test driver, follow these steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Test Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Testing Cloud Self Service Setup page appears.

  3. Click Test Drivers located in the left pane. The Test Drivers pane appears.

  4. From the Register menu, select Oracle Functional Testing.

  5. Provide the following information:

    • Name: Enter a meaningful name for this Test Driver.

    • Deployment Plan: Click the magnifying glass for selecting a deployment plan on which the test driver is based. In the resulting dialog box, select the Deployment Plan to be used for the test driver and click Select.

      Ensure that Test Designers and Test Administrators create the deployment plan "on a zone". Also, ensure they have correct privileges to this zone. For information regarding deployment plans, see Oracle Enterprise Manager Lifecycle Management Administrator's Guide.

    • Version: Version field is automatically filled in. The information is derived from the deployment plan.

    • Description: Enter a description for the Test Driver.

    • Results Storage: Use either the Result File System or the repository. The Result File System enables you to store, manage, and access the Functional Testing results for a longer duration and without any size limitations. These results can also be accessed from your external web clients and servers using the http and nfs protocols.

      If you choose to use the repository, you are limited to 10 MB of storage. Results stored locally will be purged after two days.

  6. Click Register.

To view the details of a test driver that has already been registered, highlight the test driver in the table. The deployment plan and assembly structure information are provided.

26.2.5 Setting Up Custom Test Driver Types

The Custom Test Driver allows testers to use test tools from third party vendors together with TaaS. Use Custom Test Driver Types to define and register test-driver types which meet your specific testing needs.

Oracle Load Testing and Oracle Functional Testing are types of drivers, and since they are created by Oracle, we (Oracle) know the specifics of the drivers. However, when you create a test driver, you need to direct your scripts on how to launch and monitor your tests. You can register your own type of test drivers as long as you can build a deployment plan around your assembly which contains the testing product and software.

Note:

If you are using third party test automation tools, these tools need to be registered as Custom Test Drivers.

Ensure the prerequisites are met before setting up Oracle Load Testing Test Drivers and Oracle Functional Testing Test Drivers (see Prerequisites).

To register a custom test driver, follow these steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Testing Cloud Self Service Setup page appears.

  3. Click Custom Types located in the left panel. The Custom Test Driver Types page appears

  4. Click Create.

  5. On the Add Custom Test Driver Type dialog, provide the following information:

    1. Enter a meaningful name and description.

    2. The Working Directory is a folder created on your Test Driver Virtual Machine (VM) where the operations will occur.

    3. The Results Directory is a folder created on your Test Driver Virtual Machine (VM) and where the results of your tests will appear, TaaS will zip up the entire folder, send it to Enterprise Manager.

    4. Pre-Execution Script is a file which performs pre-execution steps for your test. Click Browse to locate the file.

    5. Execution Script is a file which launches the test. Click Browse to locate the file.

    6. Progress Monitoring Script is a script which TaaS will run on the test driver, to determine if the test is running or complete (and its outcome). Click Browse to locate the file.

      When the test is complete, TaaS reports the outcome of the test. TaaS looks at the value returned by this script to determine the test outcome (passed, failed, warning).

      Note: For TaaS to know the outcome of the trial, ensure that your script prints one of the following outcomes: passed, failed or warning, as the last line to the standard output.

    7. Post-Execution Script is a file which performs the clean up steps after your test runs. Click Browse to locate the file.

    8. Stop Execution Script is a script which TaaS will run on the test driver to stop the test execution.

    9. Custom Parameters is a property file which contains the run-time parameters that will be passed to the test execution. You can configure these parameters during trial submission.

    10. Click OK.

    Once you provide these inputs, your type is defined and you can now register a new driver of this type.

  6. To register a driver, on the Test Drivers page, select Custom from the Register menu.

  7. Provide a meaningful name for this Test Driver.

  8. Click the magnifying glass for selecting a deployment plan on which the test driver is based. In the resulting dialog box, select the Deployment Plan that should be used for the Custom test driver and click Select.

  9. Enter a description for the Test Driver and click Register.

  10. Select a Test Driver in the list then click on the component to drill down to the Deployment Plan details.

26.2.6 Setting Up Quotas

The Testing as a Service Portal allows you (Test Administrator) to Setup usage quotas for Testers by specific roles. To set up quotas, follow these steps:

Note: Ensure the prerequisites are met before setting up quotas (see Prerequisites).

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Quotas located in the left panel. The Quotas page appears.

  4. Click Create.

  5. For the Role Name, click the magnifying glass for selecting roles. In the resulting dialog box, select the role to set quotas for and click Select.

  6. Enter the Number of Guest VMs (Virtual Machines), Number of CPUs, Memory (GB), Storage Size (GB), and Number of VUs (Virtual Users) to allocate to the selected role and click Create.

26.2.7 Setting Up Result File Systems

The Testing as a Service (TaaS) Portal allows you (Test Administrator) to setup result file systems for Testers.

Use Result File Systems to store, manage, and access the Functional Testing results for a longer duration and without any size limitations. These results can also be accessed from your external web clients and servers using the http and nfs protocols.

Note: Ensure the prerequisites are met before publishing test environments and test assets (see Prerequisites).

To set up result file systems, follow these steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Test Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Result File Systems located in the left panel. The Result File System region appears.

  4. Click Add.

  5. On the Add Result File System dialog box, provide the following:

    • Name

    • Export Path - Full path to file that stores results, for example, nfs-server.mydomain.com:/export/OFTResults

    • HTTP Access URL - HTTP URL corresponding to the NFS path provided.

    • Description

  6. Click Add.

26.2.8 Publishing Test Environments, Test Assets, and Test Drivers

The Testing as a Service (TaaS) Portal allows you (Test Administrator) to publish test asset assets, test environments, and test drivers to selected roles so they can be used when creating tests.

Note: Ensure the prerequisites are met before publishing test environments and test assets (see Prerequisites).

To publish test environments, test assets, and test drivers, follow these steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Publish located in the left panel. The Publish page appears.

  4. Click Add.

  5. From the Add menu in the Select Test Assets, Test Environments, Test Drivers region, select either Test Asset, Test Environment., or Test Driver. In the resulting dialog box, select the appropriate item to be published and click Select.

    The Detail region displays the specific information for each test asset, test environment, and test driver.

  6. Click Add in the Select Roles region. In the resulting dialog box, select the role to be published and click Select.

  7. After all the test assets, test environments, test drivers, and roles have been selected, click Publish.

26.2.9 Setting Up Chargeback Service

Note: Ensure the prerequisites are met before setting up the chargeback service (see Prerequisites).

For information about Chargeback Service, refer to the Chargeback Administration chapter in the Oracle Enterprise Manager Cloud Administration Guide.

Follow this process to set up applications to be tested.

  1. Log in to Enterprise Manager Cloud Control using Test Administrator credentials.

  2. From the Setup menu, select Cloud, then select Testing. The Test Administrator page appears.

  3. Click Chargeback located in the left panel to define the chargeback service.

  4. Click Configure Charge Plan to define Extended Charge Plans. Plans may be defined by configuration or usage. If defining plans by usage, charges may be based on universal metrics (CPU, memory, storage) or by metrics specific to the targets. Use extended charge plans to set individual prices.

  5. Click Configure Targets to assign Charge Plans. Charge plans must be assigned to targets. If targets are grouped into a zone, then a charge plan can be assigned to the zone, and will apply to all targets within that zone. If there are multiple zones, then different charge plans may be assigned to each one. The Self-Service Application (SSA) administrator can determine chargeback pricing by assigning existing charge plans to the different zones.

  6. Click Configure Cost Center to set up Cost Centers (Optional). Cost Centers may be set up to aggregate costs among groups of users, but are not required. If Cost Centers are set up, the chargeback reports by Cost Center are only available to SSA administrators.

26.3 Test Designer Tasks - Creating Test Assets and Test Environments

The following tasks are performed by the Test Designer:

26.3.1 Creating Test Assets - General Information

A Test Asset is a collection of test artifacts (scenarios, scripts, tests) that can be executed on a test environment. A test asset defines the actions that need to be executed to reproduce the workload against the test environment. The Testing Cloud Self Service Portal allows the Test Designer to create Test Assets based upon the available Test Drivers defined by the Test Administrator.

The Test Designer should know the application under test and its relevant Key Performance Indicators (KPIs). The Test Designer should also know the business use-cases for the test scenarios being run. Specific responsibilities include:

  • The Test Designer should be skilled in Oracle OpenScript. If the Oracle Load Testing Test Driver is used, the Test Designer should be skilled in Oracle Load Testing. If a Custom Test Driver is used, the Test Designer should be skilled using the corresponding test scripting tool.

  • The Test Designer should be skilled as DBA.

  • Create Test Assets using the Testing as a Service portal.

  • Create Test Environments using the Testing as a Service portal.

  • If the Test Designer is required to create environments, the Test Designer must know how to create assemblies.

Note: Ensure the prerequisites are met before creating test assets (see Prerequisites).

26.3.2 Creating Test Asset - Oracle Load Testing

Follow this process to create Oracle Load Testing test assets.

  1. Log in to Enterprise Manager Cloud Control using Test Designer credentials.

  2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Infrastructure Cloud Self Service Portal page appears.

  3. Click My Tests. The My Library tab appears.

  4. Click Test Assets to define the test asset to be added to the library.

  5. From the Create menu, select Oracle Load Testing. The Oracle Load Testing Asset Properties page appears.

  6. Provide the following information:

    • Name: Enter a meaningful name for this Test Asset.

    • Description: Enter a meaningful description for this Test Asset.

    • Application Type: Click the Application drop-down list and select the application to be tested from the list.

    • Tags: Enter any tags to use as keywords to further identify this test asset. These tags make test assets easier to identify and to understand the purpose of the asset.

    • Scenario: Click Browse and select the asset file you want to upload. The file type for Oracle Load Testing is SCNZIP. If the Test Asset was going to used to do large scalable testing using a number of agent machines, the scenario configuration file will have the Hardware Estimation Available feature enabled, as well as values for End Condition, Ramp-up Rate, and Number of Virtual Users.

    • Configurable Parameters: Choose none, any, or all of the parameters. The selected parameters will be exposed within TaaS and the parameters can be changed for the TaaS test runs.

  7. Once all the information is provided, click Submit. The Test Asset is added to the library. Click on a test asset name to drill down to the Test Asset Details page.

26.3.3 Creating Test Asset - Oracle Functional Testing

Follow this process to create Oracle Functional Testing test assets.

  1. Log in to Enterprise Manager Cloud Control using Test Designer credentials.

  2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Infrastructure Cloud Self Service Portal page appears.

  3. Click My Tests. The My Library tab appears.

  4. Click Test Assets to define the test asset to be added to the library.

  5. From the Create menu, select Oracle Functional Testing. The Oracle Functional Testing Asset Properties page appears.

  6. Provide the following information:

    • Name: Enter a meaningful name for this Test Asset.

    • Description: Enter a meaningful description for this Test Asset.

    • Application Type: Click the Application drop-down list and select the application to be tested from the list.

    • Tags: Enter any tags to use as keywords to further identify this test asset. These tags make test assets easier to identify and to understand the purpose of the asset.

    • Script: Click Browse and select the script file (or files) you want to upload. The file type for Oracle Functional Testing is ZIP. (Test Script Zip exported from Open Script Tool.)

      Note: When you select multiple scripts, the order is important. The scripts are executed in the order in which they appear in the list.

  7. Once all the information is provided, click Submit.

    The Test Asset is added to the library. If Oracle Load Testing Scenario file contains the Hardware Estimation result, the 'Hardware Estimation Available ' label is set to true.

    Click on a test asset name to drill down to the Test Asset Details page.

26.3.4 Creating Test Asset - Custom Test

Follow this process to create Custom test assets.

  1. Log in to Enterprise Manager Cloud Control using Test Designer credentials.

  2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Infrastructure Cloud Self Service Portal page appears.

  3. Click My Tests. The My Library tab appears.

  4. Click Test Assets to define the test assets to be added to the library.

  5. From the Create menu, select Custom. The Custom Test Asset Properties page appears.

  6. Select the type of Custom test driver.

    Enter a name and description for the test asset. In addition, for Custom asset type, you determine which parameters will be exposed to the user.

  7. Click the Application drop-down list and select the application to be tested from the list.

  8. Enter any tags to use as keywords to further identify this test asset. These tags make test assets easier to identify and to understand the purpose of the asset.

  9. Click Browse and select the asset file you want to upload. The file type for Custom can be whatever file type understood by the test driver involved.

  10. Click OK. The Test Asset is added to the library. Click on a test asset name to drill down to the Test Asset Details page.

26.3.5 Creating Test Environments

A Test Environment is a collection of entities that represents the entire application stack, thus an environment consists of the following elements:

  • Application under test (AuT): Contains the software bits and application data. For example, for a deployed Siebel application, end users can modify settings, configurations, and parameters of the application.

  • Infrastructure under test (IaT): These are entities the Application under test depends upon, for example, the cloud setup/infrastructure in a cloud deployment. Typically an infrastructure is something created for consumption of end users by administrators.

Note: Ensure the prerequisites are met before creating test environments (see Prerequisites).

Follow this process to create test environments.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Test Designer credentials.

  2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Self Service page appears.

  3. Click My Tests. The My Library tab appears.

  4. Click Test Environments to define the test environments to be added to the library.

  5. Click Create. The Test Environment Properties page appears.

  6. Enter a name and description for the test environment.

  7. Click the Application drop-down list and select the application to be tested from the list.

  8. Enter any tags to use as keywords to further identify this test environment. These tags make test environments easier to identify and to understand the purpose of the environment.

  9. Click the magnifying glass icon and select the Deployment Plan to use for this test environment.

    The Deployment Plans must already be created on a Zone where the Test Designer/Administrator has Operator privileges and which (the zone) is part of an Oracle VM Manager with view privileges to the same user.

  10. If the Test Environment contains a Web Application, click Yes.

  11. Click the Select web-application drop-down and select the appliance that will act as the application or web server when the environment is deployed in the cloud.

  12. Click OK. The test environment is added to the library. Click on a test environment name to drill down to the Test Environment Details page. Click on a deployment plan name to drill down to the Deployment Plan Details page.

26.4 Tester Tasks - Using the Testing Environment

Once the Test Administrator and Test Designer have performed their respective tasks, the Tester can now start testing.

The Testing Cloud Self Service Portal allows Testers to create tests from test assets and test environments to perform testing activities. Testers create tests using test assets and environments that have been assigned to them by the Test Administrator.

The Tester should know the application under test and its relevant Key Performance Indicators (KPIs). The Tester should also know the business use-cases for the test scenarios being run. Specific responsibilities include:

  • Running the tests by using the test assets and environments created by the Test Designer and assigned to them by the Test Administrator.

  • Reserving resources from the cloud, cancel upcoming reservations, update configurations of resources that have been reserved (CPU, memory, and so on).

  • Editing test scenarios set up by the Test Designer to specify run time test parameters and schedule the tests for execution.

  • Performing the test run analysis to compare various metrics that are relevant to the application and business use-case.

The following tasks are performed by the Tester.

26.4.1 Verifying Available Test Assets and Environments

The Testing Cloud Self Service Portal allows Testers to verify which test assets and test environments have been assigned to them by the Test Administrator.

Note: Ensure the prerequisites are met before verifying available test assets and environments (see Prerequisites).

Follow this process to verify test assets and test environments assigned to your role.

  1. Log in to Enterprise Manager Cloud Control using Tester credentials. You are logged into Self Service Portal.

  2. Click My Tests. The Testing Cloud Self Service Portal Home tab appears.

  3. Click the My Library tab. The My Library tab shows the test environments and test assets published to you by the Test Administrator.

  4. Click Test Environments.

  5. Click on an environment to view the details about the environment. Click on the component to drill down to the Assembly Structure details.

  6. Click Test Assets.

  7. Click on a test asset to view the details about the asset.

26.4.2 Creating Tests

The Testing Cloud Self Service Portal allows Testers to create tests from test assets and environments to perform testing activities. Testers create tests using test assets and test environments that have been assigned to them by the Test Administrator.

Note: Ensure the prerequisites are met before creating tests (see Prerequisites).

See the following section for the types of tests you can create:

26.4.3 Creating Tests - Oracle Load Testing Tests

Note: Ensure the prerequisites are met before creating Tests (see Prerequisites).

Follow this process to create Oracle Load Testing tests.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into the Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. From the Create menu, select Oracle Load Testing. The Create Test page appears.

  5. Provide the name and description. In addition, click the magnifying glass icon, select the test asset to be used in this test, and click Continue.

  6. Select the Source, click the magnifying glass icon, and select the test environment to be used in this test. Change the Mapping URLs settings if needed.

  7. Click the Test Driver drop-down and select the test driver to use to run the test in the cloud.

  8. The Hardware Estimate region displays the recommended number of agent machines to use for this test based on the Test Asset and Test Driver if the Test driver is configured with Agent Deployment plan.

  9. Click OK. The test is added to the library that specifies an asset, an environment, and a test-driver. Click on a test name to drill down to the details page.

26.4.4 Creating Tests - Oracle Functional Testing Tests

Note: Ensure the prerequisites are met before creating Tests (see Prerequisites).

Follow this process to create Oracle Functional Testing tests.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into the Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. From the Create menu, select Oracle Functional Testing. The Create Test page appears.

  5. Provide the name and description.

  6. Click Add to select one or more Functional Test Assets for the test. The order in which the assets are listed will be the order of their execution in the trial.

    Rearrange the order of the assets as needed by using the Move Up and Move Down arrows. You can also remove assets as needed.

  7. Click Continue.

  8. Select the Source, click the magnifying glass icon, and select the test environment to be used in this test. Change the Mapping URLs as needed.

  9. Click the Test Driver drop-down and select the test driver to use to run the test in the cloud.

  10. Click OK. The test is added to the library that specifies an asset, an environment, and a test-driver. Click on a test name to drill down to the details page.

26.4.5 Creating Tests - Custom Tests

Note: Ensure the prerequisites are met before creating Tests (see Prerequisites).

Follow this process to create Custom tests.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into the Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. From the Create menu, select Custom. The Create Test page appears.

  5. Enter a name and description for the test.

  6. Click the magnifying glass icon, select the test asset to be used in this test, and click Select.

  7. Select the test asset, review the details, and click Continue.

  8. Select the Source and click the magnifying glass icon and select the test environment to be used in this test.

  9. Select the test environment and click Select.

  10. Click the Test Driver drop-down and select the test driver to use to run the test in the cloud.

  11. Click OK. The test is added to the library that specifies an asset, an environment, and a test-driver. Click on a test name to drill down to the details page.

26.4.6 Creating and Running Trials

The Testing Cloud Self Service Portal allows Testers to create and run trials from the test instances that have been created in the portal. The Tester runs trials on the test asset and environment, monitors ongoing trials, and analyzes the results of completed trials.

Note: Ensure the prerequisites are met before creating and running trials (see Prerequisites).

See the following section for the types of trials you can create and run:

26.4.7 Creating and Running Oracle Load Testing Trial

Follow this process to create and run Oracle Load Testing trials:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Infrastructure Cloud Self Service Portal.

  2. Click My Tests in the Self Service Portal.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. Select a test to run for this trial.

  5. Click Create... in the Trials section. The Create Trial page appears.

  6. Enter a name and description for the trial.

  7. In the Application Under Test Region, specify the Application Under Test instance details.

    • Create new instance: You can create new virtual machines on the cloud for the application under test.

    • Use existing instance: If you have created instances for this test or this application before, you can choose to re-use the same set of virtual machines instead of creating new ones.

      Re-using an existing instance is faster because there will be no need to deploy the instance.

    • If you are creating a new instance, enter a password of your choice for the Root Password and reenter to confirm it.

      Important:

      This value is not retrievable. DO NOT FORGET THIS PASSWORD.
    • (Optional) Specify the Stop Instance and Delete Instance settings and click Continue.

  8. In the Test Asset Parameters region, you can:

    • Change the number of virtual users to run for this trial in the Trial Virtual Users column of each script.

      You can set these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

    • Specify the Ramp Up Rate settings for this trial.

      You can change these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

    • Set the End Condition settings for the Trial and click Continue.

      You can set these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

      If you do not set the test end conditions, the test runs until it is stopped manually.

    After you have reviewed the Test Asset Parameters, click Continue.

  9. In the Test Driver region, you can either create a new instance or use an existing instance. If the existing instance is being used by another trial, it will not be available.

    Enter a password of your choice for the Master Password and reenter to confirm it. This is the password the Testing Cloud Self Service Portal will set for the Oracle Load Testing Virtual Machine and all accounts for Oracle Load Testing.

    Specify the Stop Instance and Delete Instance settings and click Continue.

  10. In Hardware Estimate region, you can estimate and add agent machines (hardware estimate feature).

    If this is a Trial, use a Test Driver that has an Agent Deployment plan. The number is calculated based on parameters set in the previous step and the Agent Deployment plan. You can also specify a new number. The Hardware Estimate setting is enabled only when the Test driver has an Agent Deployment plan.

  11. In the Schedule region, specify whether to start the trial immediately or schedule at a later time. A new trial will be created and submitted for execution.

    Click OK. The initial execution of a trial will take some time to deploy the Virtual Machines, copy necessary software, and start the test.

26.4.8 Creating and Running Oracle Functional Testing Trial

Follow this process to create and run Oracle Functional Testing trials:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Infrastructure Cloud Self Service Portal.

  2. Click My Tests in the Self Service Portal.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. Select a test to run for this trial.

  5. Click Create... in the Trials section. The Create Trial page appears.

  6. Enter a name and description for the trial.

  7. In the Application Under Test region, specify the Application Under Test instance details.

    • Creating new instance: Users can create new virtual machines on the cloud for the application under test.

    • Re-using an existing instance: If users have created instances for this test or this application before, they can choose to re-use the same set of virtual machines instead of creating new ones.

      Re-using an existing instance is faster because there will be no need to deploy the instance.

    • If creating a new instance, enter a password of your choice for the Master Password and reenter to confirm it.

      Important:

      This value is not retrievable. DO NOT FORGET THIS PASSWORD.
    • Specify the Stop Instance and Delete Instance settings.

  8. In the Asset Execution Settings region, select the assets that are to be executed. Click Continue.

  9. In the Playback Settings region, change the settings as needed for this trial. To save time, you can copy the settings from another asset.

    Click Reset to go back to the original settings for this trial.

  10. In the Data Banks region, review and modify the data banks used with this asset. Click Continue.

  11. In the Test Driver region, either create a new instance of the test driver or use an existing instance.

    If the existing instance is being used by another trial, it will not be available.

    • Enter a password of your choice for the Master Password and reenter to confirm it.

    • Specify the Stop Instance and Delete Instance settings.

  12. In the Schedule region, specify the schedule for starting the trial run and click OK. A new trial will be created and submitted for execution. The initial execution of a trial will take some time to deploy the Virtual Machines, copy necessary software, and start the test.

26.4.9 Monitoring On-Going Trials

The Testing Cloud Self Service Portal allows Testers to monitor running trials and create charts of the key performance indicators for the application-under-test.

Note: Ensure the prerequisites are met before monitoring on-going trials (see Prerequisites).

Follow this process to monitor on-going tests.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. Select the test to view the trial details. The trial status information appears in the Trials pane.

  5. Click on the trial name link in the Trials pane to navigate to the trial-home page.

    You can monitor the progress of the trial by clicking Status. If the status of the trial is In Progress, click the trial name which will take you to the trial home and the remaining trial information.

  6. Expand the metric nodes and select metrics or counters to plot them in the chart. For Oracle Functional Testing, you can select items for Trial Result to display the result in detail.

  7. View the Agent count associated to the current trial in the Test Driver Instance session.

26.4.10 Comparing Trial Results

The Testing Cloud Self Service Portal allows Testers to compare trial results and create charts of the key performance indicators for the application-under-test.

Note: This feature is only available for Oracle Load Testing Test Driver based trials.

Note: Ensure the prerequisites are met before comparing trial results (see Prerequisites).

Follow this process to compare trial results.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. Select the test to view the trial details. The trial status information appears in the Trials pane.

  5. Select two trials of the same test that have been previously run.

    To select 2 trials of the same test, select one trial, then click Shift/Control to select the other trial. The trials must belong to the same test.

  6. Click Compare Results. You can compare the metrics of the two trials to view the differences.

    When in this mode, selecting any metric will chart two lines - one for each of the selected trials. You can also compare the Oracle Load Testing session summary report side-by-side.

26.4.11 Sharing Tests

To avoid duplication of effort, you can share tests with other users. Note that the person who created and shared the tests will be charged for any resource usage associated with the shared tests.

To share tests, perform the following steps:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

  2. Click My Tests.

  3. Click the My Tests tab in the Testing Cloud Self Service Portal.

  4. Select a test from the list of Tests on the My Tests page.

  5. Click Share.

  6. Select a user to share the Test with.

  7. Click Share.

  8. Click Log Out.

To verify that the test is shared:

  1. Log in as the user whom you shared the test with.

  2. Verify that you have access to the shared test.

26.4.12 Viewing Activity and Usage Information

The Testing Cloud Self Service Portal allows Testers to view summary information for test assets and test environments available to you, your quota as defined by the Test Administrator for your role, your current utilization of hardware resources, and submitted and scheduled trials.

Note: Ensure the prerequisites are met before viewing activity and usage information (see Prerequisites).

Follow this process to view activity and usage information.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

  2. Click My Tests.

  3. Click the Home tab in the Testing Cloud Self Service Portal. This tab shows summary information for test assets and test environments available to you, your quota as defined by the Test Administrator for your role, your current utilization of hardware resources, and submitted and scheduled trials.

26.4.13 Viewing Deployment Information

The Testing Cloud Self Service Portal allows Testers to create tests from test assets and test environments to perform testing activities using the Testing Cloud Self Service Portal. Testers create tests using test asset and environment that have been assigned to them by the Test Administrator. The Tester runs trials on the test asset and environment, monitors ongoing trials, and analyzes the results of completed trials.

Note: Ensure the prerequisites are met before viewing deployment information (see Prerequisites).

Follow this process to view activity and usage information.

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

  2. Click My Tests.

  3. Click the My Deployments tab in the Testing Cloud Self Service Portal. This tab shows a table of all environments and test-drivers deployed for you.

  4. Select a test environment or test driver instance. The details pane shows the appliances within the selected test environment or test driver instance.

  5. Click on an appliance within the selected test environment or test driver instance to navigate to the details view of the appliance.

26.4.14 Browsing the Testing Home Page

The users HOME tab displays a summary of what the user has done, including resources consumed for testing and tests created by the user. From the HOME tab, users can quickly access previously executed test trials in order to compare results or to do more in depth post-run analysis.

The Testing home page provides information regarding the statistics of ongoing testing. General statistics are provided, as well as test assets and environments, ongoing test and trials, Oracle Load Testing, and the Infrastructure. It also provides an overview of the servers and infrastructure resources consumed by the user, like the number of servers provisioned, number of CPUs, as well as RAM and Storage.

When you log in as Test Designer, you only see the My Library tab. When you log in as Test Administrator, you see the Home tab. When you log in as a Super Administrator, you see both the Home and My Library tabs.

The sections on the Home tab are:

  • General - Total counts of various artifacts available in the system, for example, its ”total tests present in system”, ”total trials ran across all trials”, and so on.

  • Test Assets/Environments - Accumulation graph of test assets, environments, and machines in use in the Testing Cloud. Count and trend data are included. This is the Test Administrator's overview of the entire TaaS portal, regardless of Testers.

  • Tests and Trials - Daily totals of the number of tests created or daily totals of the number of trial runs.

  • Infrastructure - Daily consumption of infrastructure resources: CPU, Memory, and Disk space storage.

  • Oracle Load Testing (OLT) - Daily total of the number of OLT Virtual Users (VU) that were run. Oracle Load Testing uses OpenScript scripts to simulate users accessing an application under test.

The vertical axis shows daily usage totals based upon the View selection in each of the regions.

The My Library tab provides details on both Test Assets and Test Environments.

  1. Log in to Enterprise manager using Tester credentials.

  2. Click My Tests radio button.

  3. Click the Home tab.

You will see a summary of test assets and test environments available to you, your quota as defined by the Test Administrator for your role, your current utilization of hardware resources, and the submitted and scheduled trials.

26.4.15 Trial Page

The Trial Page provides the Results, Test Environment, Configuration, and Performance information of the trial.

The top of the page provides the general information of the trial including: status of the trial, results of the trial, start time of the trial with the option to stop the trial, the test being run, and the test environment being used.

Results

Note: The Results tab is only for Oracle Functional Testing.

From the Results tab, you can:

  • Download the results to be viewed at another time.

  • Browse the results location

  • Generate a Report to be published in BI Publisher.

Test Environment

From the Test Environment tab, you can:

  • Start, stop, and restart the Guest Virtual Machines used for the test.

  • Start, stop, and restart the Test Driver Instance used for the test.

Configuration

From the Configuration tab, you can:

  • View the details of the application that was tested.

  • Test Driver used for the test.

  • List of the Test Asset Parameters used.

  • Schedule of the trial.

Performance

Note: The Performance tab is only available for Oracle Load Testing trials.

From the Performance tab, you can:

  • View all the collected Oracle Load Testing counters and Enterprise Manager target metrics.

  • Select interested counter(s)/metric(s) and show them in charts.

Summary

Note: The Summary tab is only available for Oracle Load Testing trials.

From the Summary tab, you can:

  • See the summary report of this trial and AWR report if it is available.

  • Generate Oracle Load Testing trial report and AWR report if it is available.

26.4.16 Viewing TaaS Reports

To facilitate the sharing of information about trials, TaaS provides reports using Business Intelligence Publisher (BI Publisher). These reports include the TaaS Oracle Functional Testing Trial Report and the TaaS Oracle Load Testing Trial Report.

To view these reports:

  1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Infrastructure Cloud Self Service Portal.

  2. Click My Tests.

  3. Click the Home tab in the Testing Cloud Self Service Portal. This tab shows summary information for test assets and test environments available to you, your quota as defined by the Test Administrator for your role, your current utilization of hardware resources, and submitted, scheduled, and completed trials.

  4. Highlight a trial.

  5. On the Results tab, click Generate Report.

26.4.17 Estimating Hardware Needed for Testing

The Test Driver hardware resource requirement depends on factors such as: number of virtual users for your Trial, ramp-up rate, number of script steps, number of variables in the script and script type.

To ensure that TaaS provides the appropriate amount of resources for your test trial, TaaS provides a hardware estimation feature. TaaS calculates and creates agent machines based on estimation results of CPU and memory resources consumed by the test asset in your test trial.

Note:

To use the Hardware Estimation Available feature, you must have selected the Hardware Estimation Data option on the Export File screen when creating the scenario.zip file in the Oracle Load Testing application.

To take advantage of this feature, the Test Administrator and Tester need to perform the following steps:

  1. Define an Agent deployment plan for an Oracle Load Testing Test Driver.

    1. From Setup menu, select Cloud, then select Testing.

    2. On the Testing Cloud Self Service Setup page, select Test Drivers in the navigation tree.

    3. From the Register menu, select Oracle Load Testing.

    4. Provide the information for Name, Deployment Plan, Version, and so on.

      Ensure that the deployment plans you use are scalable.

  2. View hardware estimation result in Test Oracle Load Testing Asset.

    1. From Enterprise menu, select Cloud, then select Self Service Portal.

    2. Select My Tests.

    3. On the Testing Cloud Self Service Portal page, click the My Library tab.

    4. Select Test Assets. Highlight the asset in which you are interested. The Scenario Configuration section states whether Hardware Estimation is available for this asset. An X denotes that the estimate is not available whereas a check mark denotes that it is.

  3. To create a Test Asset, click Create and choose Oracle Load Testing. Provide the requisite information.

    • The scenario zip file is exported from the Oracle Load Testing application.

      Note that it is through the scenario zip file that the Hardware Estimation Data feature is enabled. To learn how to create a scenario, refer to the Oracle Load Testing Load Testing User's Guide available from the 'Oracle Application Testing Suite Downloads' page: (http://www.oracle.com/technetwork/oem/downloads/index-084446.html).

    • By choosing the configurable parameters, you can change these settings when you create a trial using this asset.

  4. Evaluate Test Driver with estimation resource result.

    1. From the My Tests tab, select Create, then select Oracle Load Testing.

    2. Provide the requisite information and ensure that the source (Test Asset) is one that has the Hardware Estimation Data feature.

      Note: It is through the scenario zip file that the Hardware Estimation Data feature is enabled on the Export File page.

      Also note that the Hardware Estimate region contains the recommended number of agent machines to use for testing the selected Test Asset.

  5. Start an Oracle Load Testing Trial Using Estimation Resource Result

    1. Click the My Tests tab. Highlight the test you want to use for the trial.

    2. In the Trials section, select Create.

    3. On the Create Trial screen:

      - Provide the requisite information in the Overview and Application Under Test regions. Click Continue.

      - For the Test Asset Parameters, if needed change the values for the Trial Virtual Users, Ramp-up Rate, and End Condition. Click Continue.

      - Provide the parameters for the Test Driver, Hardware Estimate, and Schedule. Click OK.

      Note that in the Hardware Estimate region, the number of agents is calculated by TaaS to meet the required resource needed for the trial. However, you can change the number of agents if you want.

  6. View Agent Status in Trial Home

    On the My Tests tab, highlight the trial of interest.

  7. View Agent Status in My Deployment Tab.

    On the My Deployments tab, highlight the deployment of interest. The Agent Status will be shown in the Agent tab.

  8. Share/unshare the Agent between different test driver instances.