Apply Step

The apply step is the step where records in the target environment are added or updated. Like the comparison step, the apply step is actually multiple steps to optimally handle high volume and dependencies between records as smoothly as possible. Before explaining the apply steps in detail, the following points highlight the type of data that may be included in a given data set.

  1. Records that have no foreign keys and therefore no dependencies on other records. Examples: Message, Display Profile.

  2. Records that have foreign keys that may already be in the target. Examples: Algorithms for existing algorithm types, To Do Roles for existing To Do Types.

  3. Records that have foreign keys that are new records but also part of the migration. CMA detected the relationship at export time and grouped the objects in the same transaction. Example: Script-based Algorithm Type where the script is also in the migration.

  4. Records that have foreign keys that are new records but also part of the migration. CMA did not detect the relationship. This may occur if the reference the foreign key is in a CLOB or parameter column and the migration plan did not include an instruction to explicitly define the relationship. Example, a Zone that references a visibility script.

  5. Records that have circular references where both records are new and are part of the migration. CMA detected the relationship at export time and grouped the objects in the same transaction. Example: plug-in Script for a BO plug-in spot. The script references the BO and the BO references an algorithm for the script’s algorithm type.

To handle high volume data, the first step in the apply process is to perform the apply logic at the migration object level via a multi-threaded batch job. This should result in all records in categories 1 and 2 above being applied successfully.

For records in categories 3 and 4 above, if a record with a foreign key is added or updated before its related record, the validation will fail. However, if the related record is added first and then the record referring to it is added, validation will pass. Because the migration objects are not ordered, the multi-threaded batch process may not process records in the desired order. To overcome this potential issue, the Apply step has special functionality, described in detail below.

For records in category 5 above, the circular reference will mean that the apply process at the object level will not successfully add or update these records. The apply process at the transaction level will cover these records. This is described in detail below.

Apply Objects

Once the Data Set is in the state of Apply Objects, the Migration Object Monitor — Apply process (F1–MGOAP) runs to attempt to apply the objects. The background process in conjunction with the Apply algorithm have special functionality to ensure records in categories 3 and 4 (above) successfully apply during this step:

Note: When submitting this Apply batch job, be sure to set the number of threads to a number that does not exceed the number of threads supported by the thread pool worker. Doing this will cause the ‘excess’ threads to wait for the supported number of threads to finish, erasing the benefit of the iteration processing.

The following diagram is the portion of the migration object lifecycle that pertains to the Apply step. Note that this diagram is not complete. A subsequent section provides more information about resolving errors.

Migration Object Apply Lifecycle

At the completion of the Apply monitor process, typically the objects will be in the Applied state or the Error Applying state. The records in the Error Applying state are in that state for one of two reasons.

As shown in the diagram, the Apply Objects algorithm may also detect a reason that the object cannot be applied. This may occur if the object in the target environment has been updated since the comparison step, making the SQL captured at that point no longer applicable. If this occurs, after the current migration is fully applied, the original file may imported again, and new comparisons can be generated and applied.

Apply Transactions

Ideally, after the Apply Objects step, all the objects are Applied or are in Error Applying due to the “circular reference” situation. The typical next step is to turn over responsibility to the transactions. The migration transactions can then attempt to apply their objects in bulk.

In order to ensure that multiple background processes are not trying to select migration objects to run the Apply step, the transactions are only eligible to attempt to “apply my objects” if the Data Set is in the Apply Transactions state.

Migration Data Set Apply Lifecycle

A monitor algorithm (executed by the data set monitor batch process) on the Apply Objects state checks to see if all migration objects are no longer Approved and do not have any records in Error Applying. If so, it automatically transitions the record to the Apply Transactions state.

If any objects are in Error Applying, the monitor algorithm does not automatically transition the record. In that case, a user must decide if the record should transition to Apply Transactions. The Resolving Errors section below describes alternative steps that the user may take if there are errors.

Once the Data Set is in the state of Apply Transactions, the Migration Transaction Monitor — Apply process (F1–MGTAP) runs. It attempts to apply the transaction’s objects. If no migration objects are in error, the migration transaction simply transitions to Applied. If any of the migration objects are in Error Applying, the background process and the Apply algorithm have special functionality to try to overcome dependencies in migrated objects:

Note: When submitting this Apply batch job, be sure to set the number of threads to a number that does not exceed the number of threads supported by the thread pool worker. Doing this will cause the ‘excess’ threads to wait for the supported number of threads to finish, erasing the benefit of the iteration processing.

The following diagram is the portion of the migration transaction lifecycle that pertains to the Apply step illustrating the points above. Note that this diagram is not complete. A subsequent section provides more information about resolving errors.

Migration Transaction Apply Lifecycle

If at the end of the transaction level Apply process there are transactions in error (and therefore there are still objects in error), a user must review the errors and determine how to fix them. Refer to the Resolving Errors section below for more information.

Resolving Errors

As mentioned in the previous sections, errors may be received after the Apply Objects process runs and the system will not automatically transition the data set to the Apply Transactions. Rather, a user must make the decision. This allows for the user to review the errors and make a decision:

After the Apply Transactions step, if there are still errors, a user must review the records and determine how to proceed

Because the objects and transactions are in Error Applying, in order to “retry” the Apply step after manually fixing an error, the system needs to move the records back to the state that allows them to be picked up by the appropriate Apply process. For migration objects, records need to be moved back to Approved. For migration transactions, records need to be moved back to Ready to Apply. This is accomplished with a Retry Count. This is different from the Iteration count that is used during the apply steps to handle the iterative selection logic. The following points describe the Retry logic for migration objects.

Analogous logic exists for the migration transactions.

Finalize Apply Step

Once all the migration objects for a migration transaction are in a final state (Applied, Rejected or Cannot Apply), the migration transaction transitions to the Applied state. Once all the migration transactions are in the Applied state, the Migration Data Set record transitions to the Applied and the import is complete.

Note: To review the full lifecycle for each record, refer to the Business Object — Summary tab in the application metadata for the base business objects Migration Data Set Import (F1-MigrObjectImport), Migration Transaction (F1-MigrTransactionImport) and Migration Object (F1-MigrObjectImport).