Identifying Change Impacts

This chapter provides an overview of change impacts and discusses how to:

• Define analysis rules.

• Create test maintenance reports.

• Interpret test maintenance reports.

• Create test coverage reports.

• Interpret test coverage reports.

• Query PTF report tables.

Understanding Change Impacts

In the course of customizations and upgrades, changes are made to, among other elements, application menus, components, pages, records, and fields. Tests that were developed prior to these changes may fail when executed against the new application. For instance, if a field is deleted, moved to another page, or renamed, any step that references that field will fail. Test developers must identify and update every step in each test that is affected by the change.

One way to identify the effects on tests is to run each test against the new application and note where the test fails. This manual process is time-consuming, expensive, and prone to errors. It also fails to identify those areas in the new application that are not covered by existing tests.

Because PeopleSoft Test Framework (PTF) test assets are PeopleTools metadata and because PTF tests incorporate references to PeopleTools metadata—that is, menus, components, pages, records, and fields—PTF is able to automate the process of correlating metadata changes with existing tests.

PTF delivers two tools that enable test developers to help in determining the effect of changes:

• Test maintenance reports

  A test maintenance report correlates PeopleTools compare report data with PTF test metadata to identify certain changes to components, menus, pages, records, and fields that may impact the PTF tests.

• Test coverage reports

  A test coverage report correlates PeopleTools project data with PTF test metadata to identify components, menus, pages, records, and fields that are referenced in PTF tests.

  When used in conjunction with Usage Monitor, test coverage correlations can be extended to include information on all managed objects.

  A test coverage report identifies which objects included in the change project are referenced by which PTF test. Any object included in the change project that is not referenced in the PTF test metadata appears in the report identified as a coverage gap.

Defining Analysis Rules

This section discusses how to define analysis rules.

Access the Define Analysis Rules page (PeopleTools, Lifecycle Tools, Test Framework, Define Analysis Rules).

Use the Define Analysis Rules page to define the priority for each of the attribute checks in a test maintenance report.

A test maintenance report is sorted by test name. Within each test name grouping, the report items are sorted by priority according to the values specified on the Define Analysis Rules page.

If the priority for an analysis category is set to 4 – Ignore, then identified impacts meeting the category criteria will not be printed in the report.

Note. Priorities are used as filters and groupings in a test maintenance report. They do not affect the actual analysis process or change what is analysed.

The following table describes the analysis categories:

Creating Test Maintenance Reports

This section discusses how to create test maintenance reports using the Create Test Maintenance Report wizard.

Access the Create Test Maintenance Report wizard (PeopleTools, Lifecycle Tools, Test Framework, Create Test Maintenance Report).

The wizard consists of three steps:

• Step 1: Manual Tasks

• Step 2: Analyze Compare Data

• Step 3: Generate Report

Step 1 of 3: Manual Tasks

For a given change project the tasks for Step 1 only need to be executed once.

The first step in creating a test maintenance report comprises five manual tasks that you will complete in Application Designer to create a compare report that PTF will use as a basis for the maintenance report.

Perform these tasks to create a compare report from a project file.

Access the Create Test Maintenance Report wizard (PeopleTools, Lifecycle Tools, Test Framework, Create Test Maintenance Report).

As you progress through the Test Maintenance wizard the wizard tracks which tasks you have completed and which page you are on. When exit the wizard and then return to the wizard again, the wizard sends you to the last visited page.

This tracking is done according to user ID. This means that if two users share the same user ID, the second user may not enter the first page when accessing the wizard. The wizard will take the second user to where the previous user left the wizard.

If two users with different user IDs work on the same project, the wizard does not track the state for the second user. For instance, if the first user completed the tasks for Step 1, the second user needs to check the five tasks on Step 1 to bypass that step.

Task 1: Import only the project definition for the change project.

Suppose you are about to apply a change project named SA_PROJ_ UPGD. Before the change project is applied, import the change project, SA_PROJ_ UPGD, as a project definition only.

1. Select Tools, Copy Project, From File.

2. In the selection box, highlight the change project, SA_PROJ_UPGD.

3. Click Select.

   The Copy From File dialog box appears.

4. Click the Deselect All button to deselect all of the definition types so that only the empty project structure is imported.

   At this point, you are importing only the names of the definitions into the project. None of the actual definitions in the upgrade project are copied.

   If a definition exists in the original database with the same name as a definition in the upgrade project, the upgrade project in the database will (correctly) contain the original, unmodified definition of the object.

5. Click Copy.

Task 2: Rename the change project imported in Task 1 to create the snapshot project.

Create a copy of the change project, SA_PROJ_UPGD.

Select File, Save Project As and name the new project SA_PROJ_SNAP.

Creating the Snapshot Project is required to avoid naming conflicts later in the process.

Task 3: Create a snapshot of the original metadata definitions by exporting the snapshot project created in Task 2 to file.

Export the snapshot project, SA_PROJ_SNAP, to file:

1. Select Tools, Copy Project to File.

2. If this project is large, as upgrades often are, you can save time during the compare process by deselecting all definitions in the Copy Options window, except for the five definitions that are referenced by PTF tests:

   • Menus

   • Components

   • Pages

   • Records

   • Fields

3. Accept the defaults and click OK.

Task 4: Import the change project (definition and objects).

Import the original change project, SA_PROJ_UPGD, from file:

1. Select Tools, Copy Project From File.

2. Include all the definition types.

At this point, the database contains the new metadata.

Task 5: Compare the snapshot project (from file) to the database, saving the compare output to tables.

Compare the current (target) database with the pre-change (source) project, SA_PROJ_SNAP:

1. Select Tools, Compare and Report, From File.

2. Highlight the snapshot project, SA_PROJ_SNAP, and click the Select button.

   The Replace existing SA_PROJ_SNAP? dialog box appears.

3. Click Yes.

   The Compare and Report From File dialog box appears.

4. Click Options to access the Upgrade Options dialog box.

5. Select the Report Options tab.

6. Select the Generate Output to Tables check box.

7. Select the Report Filter tab.

8. Click Select All.

9. Click OK.

10. Click Compare.

The wizard will use data from this compare report to create the test maintenance report.

Step 2 of 3: Analyze Compare Data

This example shows Step 2 of 3:

For a given change project the tasks for Step 2 only need to be executed once. This analysis is based only on the compare data, not on the test data, so changes to tests do not affect the result. The relationship between this analysis and test data is calculated in Step 3: Generate Impact Report. You may generate multiple Impact Reports based on a single analysis as tests change.

In this step, PTF analyzes data from the compare you performed in Wizard Step 1, Task 5.

1. For the project compare report you created in Wizard Step 1, Task 5, specify whether the Compare Type is From File or To Database.

2. Click Analyze.

   This will launch the Application Engine program PSPTANALYSIS.

3. When the analysis is complete a pop-up appears. Click Next to continue.

Note. A project name can appear more than once in the list because the Compare Output to Table feature stores data by both project name and compare date/time. The most recent compare will appear first in the list.

Step 3 of 3: Generate Report

This example shows Step 3 of 3:

For a given change project you will use this page to generate multiple reports as changes are made to the PTF test metadata. As you make changes to tests based on the results of this report, rerun the report to verify that the issues were corrected. You can run this report repeatedly as you make changes to tests without having to redo Step 2.

Follow these steps to generate a report:

1. Select the analysis data PTF will use to generate the report.

   Sets of compare data are listed by project name, the date and time the compare was generated, and the date and time the analysis was run.

2. Select the report format.

3. Select the report scope.

4. Click Generate Report to have PTF create the test maintenance report.

Select the Delete check box and click Delete Selected to delete analyses.

Running a Test Report from PTF Client

As you modify a test based on the results of a maintenance report, you can validate the test by running a test report from the PTF client. The report is generated for a single test, using analyses generated in Step 2 of the Test Maintenance Wizard.

1. In PTF Explorer, right-click on a test.

2. From the context menu, select Validate Test.

3. The Validate Test – Analysis Project list dialog box appears.

4. Select an analysis from the list and click Open.

   The test maintenance report opens in a new window in the PTF client.

Interpreting Test Maintenance Reports

A test maintenance report lists the tests that have been impacted by changes to menus, components, records, pages, and fields, and details the changes that impacted those tests. You can choose to output the report to PIA or to XML Publisher (XMLP).

The following example shows a report in XMLP format:

The following example shows a report in PIA format:

From a PIA report, you can click the Download icon to output the report to a spreadsheet. The following example shows a report in spreadsheet format:

The following example shows a Test Maintenance report in the PTF client:

The following columns are on a Test Maintenance report (test data may be positioned differently depending on the report format):

Understanding Test Coverage Reports

In addition to being a record and playback tool, PTF is one element of the broader PeopleTools Lifecycle Management framework, which provides greater visibility into how organizations use their PeopleSoft environments, and how changes to PeopleSoft managed objects might impact their business processes.

Documenting business process tests in PTF enables you to correlate business processes to the underlying managed objects. Using PTF in conjunction with Usage Monitor provides you with even greater levels of information about the objects used in your system.

Test Coverage reports leverage PTF integration with PeopleTools to provide this information to you in a usable form.

Creating Test Coverage Reports

Access the Create Test Coverage Report page by selecting PeopleTools, Lifecycle Tools, Test Framework, Create Test Coverage Report.

By default, a test coverage report correlates PeopleTools project data with PTF test metadata to identify components, menus, pages, records and fields that are referenced in PTF Tests.

When used in conjunction with Usage Monitor, test coverage correlations can be extended to include information on all PeopleTools managed objects.

A test coverage report identifies which objects included in the change project are referenced by which PTF test. Any object included in the Change Project that is not referenced in the PTF test metadata will appear in this report identified as a coverage gap.

All the projects in the database are listed, sorted with most recent first. Click the column headings to change the sort order.

PTF generates a coverage report based on a change project. Select the project you want PTF to use to generate the report.

The following fields are on the Generate Impact Report page:

Using Usage Monitor Data with PTF

PTF used by itself tracks the use of five object types – records, menus, fields, pages, and components. You can use PTF together with Usage Monitor to track every PeopleTools managed object used in the course of a test. This data can then be correlated with change projects (that is, fixes or bundles) to determine which tests need to b executed to test the change project.

For example, suppose you have 100 PTF tests in your test repository. You receive a bundle from PeopleSoft that consists of updates to ten PeopleCode objects. Based solely on the data stored by PTF, you cannot determine which PTF tests you need to run to test the bundle, because PTF does not track references to PeopleCode.

You can use Usage Monitor in conjunction with PTF to address this issue. Usage Monitor tracks references to all PeopleTools managed objects, including PeopleCode. When you associate a test and test case with Usage Monitor all the actions taken while Usage Monitor is active are associated with the test and test case you specified.

A test coverage report correlates project data, PTF data, and Usage Monitor data to identify, in this example, which PTF Tests in the test repository reference one or more of the PeopleCode objects delivered in the bundle.

Configuring Usage Monitor

Your system administrator must configure Performance Monitor and Usage Monitor before you use Usage Monitor with PTF.

See Also

Setting Up the Performance Monitor

Enabling Usage Monitor

Generating Usage Monitor Data

Follow these steps to generate Usage Monitor data along with a PTF test data:

1. Create a new test in PTF.

2. Launch the Recorder.

3. Hook the browser.

4. Start recording.

5. In the PeopleSoft application browser, select the Usage Monitoring link from the main menu.

   Note. Your PTF environment and the application you are testing must be on the same database.

6. The Test Data page appears.

7. Enter Test Name and Test Case.

8. Click Start test.

9. Record the test steps.

10. When you are finished with the test steps, return to the Usage Monitoring page and select Stop Test.

11. In the PTF Recorder click the Stop Recording icon.

This is an example of a Usage Monitoring page:

Administering Usage Monitor for PTF

Before Usage Monitor data can be used in a coverage report it must be aggregated.

To aggregate the Usage Monitor Data:

1. Navigate to PeopleTools, Process Scheduler, System Process Requests.

2. Enter a run control ID or create a new one.

3. Click Run.

4. Select Usage Monitor Aggregator (PTUMAGR) from the list.

5. Click OK to Run the process.

Note. We recommend that you schedule the Usage Monitor Aggregator Process to execute hourly or daily to keep the data in the aggregated data table current. Once data has been aggregated the raw data can be purged to reduce disk space usage.

To verify that Usage Monitor is collecting data, run this query in your query tool:

SELECT * FROM PSPMTRANS35_VW;

The PSPMTRANS35_VW represents the correct logical view of the raw Usage Monitor data.

Note. For any Usage Monitor data to appear in the view, the Usage Monitor buffers need to have been flushed at least once. By default the buffer size is set to 500 unique object references. While you are configuring and testing your Usage Monitor setup you might find it useful to reduce the size of the buffer (PeopleTools, Performance Monitor, Administration, System Defaults).

To verify that Usage Monitor Data has been aggregated correctly, run this query in your query tool:

SELECT * FROM PSPTUMPMTAGR;

Interpreting Test Coverage Reports

Use a Test Coverage report to determine which objects have tests and which do not. Objects without a test represent a potential gap in the test coverage.

This is an example of a test coverage report in PIA format:

The following columns are on a Test Coverage report (test data may be positioned differently depending on the report format):

Querying PTF Report Tables

You can query these tables using PS Query to produce your own maintenance and coverage reports:

• PSPTTSTCOVRGRPT

  This PTF table contains the data that is used to create the Test Coverage report.

• PSPTTSTMAINTRPT

  This PTF table contains the data that is used to create the Test Maintenance report.