Batch Validation Process

Batch validation includes six major phases that run sequentially and commit separately. Where noted, a major phase also has intermediate commits. The major phases are:

1. Univariate revalidation. The system checks patient data (question responses) against the definition of their questions. This is called "revalidation" because the system performs the same checks as the data is entered.

2. Discrete Value Group (DVG) resolution. Checks responses against the values defined for their question's discrete value group (list of valid values), if any. Includes subtypes that commit separately, in the following order:

   1. DVG

   2. DVG Subset

   3. Thesaurus DVG

3. Procedures. Checks responses against the values of other responses, as specified in user-defined Validation Procedures. Also uses responses to derive values for derived questions as specified in user-defined Derivation Procedures. Separate commit for each Procedure, with all tracking related to the Procedure, in the following order:

   1. Derivation Procedures. Processed in the order specified in the Sort field in the Maintain Derivation Procedures window. If two Procedures have the same Sort Order Number, they are processed in alphabetical order (by Procedure Name).

   2. Validation Procedures. Processed after all derivation Procedures in alphabetical order by Procedure Name.

4. Indicator Questions. Checks if responses to questions defined as follow-ups to indicator questions have been collected when they should not have been collected or not collected when they should have been collected (see Discrepancy Types). In either case, the system associates the discrepancy with the indicator question value, not the follow-up question value(s).

5. Oracle Thesaurus Management System (TMS) integration. If TMS is fully integrated with Oracle Clinical, Batch Validation checks for verbatim term (Oracle Clinical Response) classifications created in TMS since the last Batch Validation run. For each occurrence of the same parent question with the same response, the system populates each of the associated derived question(s) with the requested TMS dictionary term or attribute. There are separate commits for each verbatim term and each of its derived values.

6. Validation status. The most important tracking milestone for restarting processing after a failure is the successful completion timestamp. "Successful completion" logic assumes that all processing that determines what to process depending upon a timestamp (such as univariate re-execution or modified patient validation) takes the previous successful completion as a starting point until a subsequent Batch Validation succeeds. This means, for instance, that the processing will be repeated, but the derivative work will not need to be performed.

After completing the batch validation job, the system automatically runs the incremental expectedness calculation job for all patients whose data changed. If batch validation detects changes in Enhanced DCI Books, the system runs the full expectedness calculation job instead. See Oracle Clinical Creating a Study for more information.

In general, database changes are a significant part of the work, so a restart should execute the redundant processing significantly faster than the original process. Also, rollback segment use (and, in the TMS case, distributed commits) will be smaller.