Data Conversion and Migration Overview
This section provides an overview of data conversion and migration, including:
Conversion Process Overview
The goal of the Conversion Process is to migrate data from a legacy application into a target environment, and to begin running the application in the cloud. Due to cloud-related technical restrictions, legacy data cannot be uploaded directly into the software-as-a-service (SaaS) database.
Legacy data must be extracted into file(s) and compressed. The data files are uploaded to the cloud file storage location and then loaded into the target "staging" tables using Oracle SQL Loader. The data is validated, transformed, and finally inserted into "production" tables. Oracle Utilities cloud services include various tools supporting ad hoc SQL inquiries and reconciliation reports on both staging and production data.
Implementation Effort
Implementers are expected to perform the following tasks for data conversion:
• Analyze the legacy data and decide what portion of it should be converted
• Map the legacy data to target Oracle Utilities Application Framework (OUAF) / Application data
• Develop legacy data extract process and produce input data files
• Adjust default data upload setup in OUAF / application, if needed
• Rehearse data upload and fine-tune configurations and/or legacy data extract, if needed
• Create reconciliation reports in Analytics Publisher
• Use uploaded data to try the subsequent conversion flow(s); bring the end-to-end conversion flow to perfection
• Execute the final conversion data upload run, a.k.a. cut-over
• Execute the application's data conversion processes.
• Disable conversion activities in the environment
What Is in the Newly Provisioned Environment?
The production instance is available for conversion.
Conversion activities do not co-exist well with the rest of the implementation. The massive data uploads, table truncation, and switching schema could disrupt business configurations development and testing. The production environment during project implementation is the best candidate for conversion.
In the newly provisioned instance, the staging area in the database is created according to application specifications. The Analytics Publisher instance and Oracle Database Actions / Oracle REST Data Services are connected to production and staging data.
The environment contains pre-configured conversion data upload setup.
The default configurations are suitable for typical table volumes and common data formats. If your implementation does not include extremely large data volumes, special data formats, or other idiosyncratic requirements, the default setup can be used "as is".
Data Conversion and Migration on Cloud
This section provides an overview of data upload support in Oracle Utilities cloud services, including:
The "SQL Loader Batch Process" portion of the flow shown below is supported by Oracle Utilities Cloud Service Foundation (CSF). The legacy data extract and the input file creation belong to the implementation. The business application provides conversion validation and transformation processes, as well as the definition(s) of the staging area.
Provided by Cloud Service Foundation
Oracle Utilities Cloud Service Foundation (CSF) features metadata-driven configurable and customizable data upload with SQL Loader. It also provides support for basic database operations such as table clean-up (truncate), index enable/disable, and some others.
SQL Loader is an Oracle database utility that allows users to load data from external files into target DB tables. See Oracle DB SQL Loader Documentation for details.
The load of the Input Data File is performed according to the instructions recorded in a Control File. The Control File contains load options and parameters and also a list of data fields with formatting and parsing instructions.
For data upload on cloud, Control Files are pre-generated based on the metadata and conversion configurations and stored in the system.
Cloud Service Foundation allows users to generate control files, and it also provides a batch process that consumes the Input Data File(s), reads the pre-generated Control File and calls SQL Loader.
Cloud Service Foundation delivers the following:
• Batch processes. CSF batch controls are "generic", with no default value for the parameter that specifies target table or maintenance object (MO). These batch controls are used mostly for development and testing purposes. Applications are likely to supply "specific" batch controls for each target table or MO.
• Batch Controls: Load Data into Table or Maintenance Object, Truncate Table or MO's Tables, Disable/Enable Indexes, Disable/Enable Triggers, Update Statistics, Populate Key Table, Cleanup Key Reference and XML resolution Tables, Generate Conversion Artifacts (bulk), and few others.
• Services accessible via online UI.
• Switch Schema - executes the stored procedure that is re-directing the CISUSER synonyms between staging and production.
• Generate Conversion Artifacts - creates input data file specifications and SQL Loader control files for specific converted objects. The artifacts are generated based on the metadata and according to the conversion data upload setup
• The instance of Analytics Publisher that is connected to the database with read-only access to the staging and production schema tables.
• The instance of Oracle Database Actions that is connected to the database with read-only access to the staging and production schema tables.
• Predefined Configurations
• Data delimiters and data format strings for date and date/time fields (Extendable Lookups)
• Default Conversion Instructions (Conversion Task Types) for typical Table, MO and Key Table.
• SQL Loader Control File fragments (Managed Content) for parallel and non-parallel load
• Conversion Data Upload Master Configuration with default setup
• User Groups and Template Users for conversion (suggested setup)
Provided by Applications
Each application comes with its own Conversion Accelerator that includes admin and system data for suggested upload configurations. Applications also provide a set of processes and tools to validate and transform the uploaded "staging" legacy data into real production form.