Introduction

Overview

Oracle Tuxedo Application Rehosting Test Manager (ART Test Manager) is a part of Tuxedo product family designed to automate and speed up testing in mainframe re-hosting projects. Complementing Tuxedo ART Workbench, which provides tools for automating and speeding up the migration process, the Test Manager focuses on the remaining part of the re-hosting project – testing the results. It’s role is to enable customers to complete the application functional or regression testing more efficiently, faster, and with fewer resources, thus reducing overall project cost and speeding up time to value – when the target environment can be switched into production and expected savings from elimination or reduction of mainframe MIPS begin to accrue.

Identifying the scope of testing and defining a test plan, determining how to test the application and executing test scenarios presents a significant challenge in many rehost projects. To help customers address these challenges, the testing can be driven by captured information – partly from the source analysis and generated migration artifact and partly from capturing baseline test sequences and results from the mainframe execution. The application execution on the mainframe, in test or production environment, becomes the “source of truth” for expected behaviors and outcomes in the re-hosted application.

A key benefit from capturing test scenarios on the mainframe is ability to replay them on the target environment, and then use captured results to compare with those generated on the target. Such automated comparison can help to speed up the evaluation of the test results and identification of potential issues. This moves the test effort from a manual, human-intensive task to an automated, industrialized process. The test plans and automated test cases can also become a part of the ongoing regression test capability as migrated applications continue to change through ongoing maintenance and business evolution. In the next version, ART TM will also provide load testing capabilities to create user-configurable stress tests based on the functional test cases that have passed successfully.

ART Test Manager Outline

ART Test Manager provides customers with end-to-end capabilities from creating test plans to executing the test cases, capturing and comparing results, and providing a tracking dashboard. It provides a modern Web UI for user interaction, with authentication, access control, and auditing built in. Users interact with the Test Manager to perform the following tasks:

•	Configure access to one or more test environments that have been deployed with Tuxedo, relevant CICS, IMS, and Batch runtimes, migrated application components, and any dependencies (e.g., data files, DB clients, MQ client, etc.)

•

Discover online and batch test case units for CICS or IMS transactions and batch jobs, and select relevant subsets to create test plans. Specify the execution order, upload baseline artifacts, configuration dependencies, and custom pre- and post-processing scripts. Test plans can also be filtered to focus on specific tests and extended to add additional custom test scenarios by users.

•

Execute test plans on one or more configured test environments and capture results for automated comparison with the mainframe baseline. Built-in result comparison methods include 3270 screen and data stream compares for online CICS and IMS transactions and return code, file, and database compares for batch jobs. These can also be extended with custom results verification scripts. Review collected logs and traces to diagnose anomalies.

•	Track, Audit, and Report. Track status of test cases and test plans, and generate reports and dashboards. Review audit reports to see all user activity by project.

Online Testing Using ART Test Manager

Online CICS and IMS transactions are defined through CICS CSD configuration file or IMS system configuration macros. These z/OS artifacts are converted into ART CICS or ART IMS resource configuration files by ART Workbench. ART TM uses these resource files to identify relevant transactions and put them in appropriate groups (CICS 3270, CICS DPL, IMS MPP, IMS BMP), where they can be selected and arranged in a specific sequence to form test plans. Online CICS and IMS transactions can be interactive (i.e., using tn3270 terminal emulators driven by users or through macros/screen scraping) or message-driven (i.e., from JEE application servers, MQ, Web Services, etc.) and this guides the approach to capturing baseline test sequence and executing it against ART CICS or IMS.

For interactive tn3270-based transactions, ART TM provides a 3270 socket gateway that is deployed on the socket between tn3270 and target mainframe system. Users can then run test sequences against a mainframe test CICS or IMS region, while the 3270 gateway captures the data stream to and from the mainframe. Once the sequence of transactions and user inputs has been captured as a baseline data stream, ART TM will be able to re-play the inputs against Tuxedo ART CICS or IMS region, while comparing the outputs from ART CICS or IMS servers with those it captured from the mainframe CICS or IMS regions.

Any response discrepancies in the 3270 data stream are flagged for user to analyze and determine if these are acceptable (e.g., based on time difference between mainframe test and Tuxedo ART test) or not in determining the overall success or failure of the test execution.

For non-interactive CICS or IMS transactions, ART TM supports DPL clients for CICS DPL programs and built-in IMS BMP driver for IMS batch programs.

Extensible results verification framework also allows users to add custom comparison tasks to the verification of these tests.

Batch Testing Using ART Test Manager

Batch jobs in most customer environments are executed primarily through schedulers, which determine the sequence of jobs in each scenario based on results from previous jobs and other events. Tuxedo batch runtime provides a command-line tool to submit, control, and query the jobs in ART Batch, and this tool is typically used to integrate with batch schedulers, often through local agents. A version of the same tool in ART TM is used to record the sequence of jobs as submitted from the scheduler and create test plans based on these sequences. Similarly to online test plans, these can be annotated for data and other dependencies, custom verification and before/after scripts, and extended with additional test cases for ad-hoc jobs.

Once test plan is created and ready to be executed, ART TM can run a job in ART batch runtime and then submit it on the mainframe (through an FTP connection to a z/OS test environment). When the jobs complete, results comparison can include the return codes, file and DB comparison, and can be optionally extended to include custom user scripts.

Architecture

The high-level architecture of ART Test Manger is illustrated in the following diagram.

ART Test Manager runs in a web server and is accessed by the end user through a web browser. ART Test Manger can run testing on multiple test machines.

Terms

 

Table 1‑1 Terms

Test Project	Execution environments with all converted source code, configuration files, resource files, data files, etc., deployed by ART Workbench Plug-in.
Test Group	An individual test environment, containing settings of one Tuxedo domain, and test plans and test cases. Currently, there are three types of test group; CICS, BATCH, and IMS). All test cases and test plans within one test group share the same execution environment.
Test Plan	Optional logical container for multiple test cases, which can be used to organize testing based on subsystems or other logical parts of the application.
Test Case	Smallest testing unit. For example, a CICS program, an IMS MPP transaction, an IMS BMP program, or a batch JCL job.

UI Framework

The user interface (UI) of ART Test Manger contains three major panels:

•	Project Tree: Navigation bar that shows project list owned by current user, and the members of a selected project

•	Context Path Bar: Contains “Create project” button, which can be clicked to create a new project, and current context path if a project is opened. The path may contain 4 levels at most: Project -> Group -> [Plan] -> Case

•	Main operation panel: The content of currently open object (project, group, plan, or case) is shown here, and a group of buttons that are appropriate in current context are provided.

The drop down menus in the top right corner provide access to additional user management (drop-down menu under userid) and other general-purpose functions (drop down menu under “menu”: Software Provisioning, System Monitoring, Dashboard, and Auditing).