5 Run jobs and check history

• View and edit your subscriptions to job status notifications
  
• Run transformations and view run history
  
• Run validation check batches and view run history
  
• Reprocess discrepancies that failed to be sent to InForm
  
• Check if data files loaded
  
• FAQs
  

View and edit your subscriptions to job status notifications

You can subscribe to receive email notifications about the completion status of jobs.

1. Go to the Home page and select the study.
2. In the Studies title bar, click the Notify icon.
3. Select the checkbox for each type of job and completion status that you want to be notified about:

   • Job types: File loads, InForm loads, Transformations, Validation Checks

   • Completion statuses: Success, Warning, Failure

     Note:

     Your subscriptions for the current study appear as selected. An administrator can also subscribe or unsubscribe you from notifications.

   To stop receiving a particular type of notification, deselect its checkbox.

4. Click OK.

To subscribe or unsubscribe to notifications for other studies, select a different study and repeat the process.

Run transformations and view run history

1. Go to the Home page, Transformations tab.
2. To view all jobs, click the View Full Job History icon.

   To view only recent jobs again, click the View Recent Jobs Only icon.

3. See:

• View transformation job history
  
• Run a transformation
  
• Cancel a transformation job
  

View transformation job history

Transformations are displayed by the name of their target clinical data model.

• To view table transformations, click a transformation's node.

• To view run history and pending jobs, select a transformation in the upper pane.

• To view log files, click the icon in the column for the type of job:

  • Log: The most recent manually submitted job.

  • Triggered Job Log: The most recent triggered job.

  • Install Job Log: The most recent installation of the transformation.

Run a transformation

To run or schedule a transformation:

1. Go to the Home page, Transformations tab.
2. Select a transformation and click the Submit Job icon.

   Tip:

   The Submit Job icon is not active if the selected transformation has not been installed. Check the Install Status column.

   If the installation status of the transformation is Warning, you may still be able to run the batch. Check the installation log file.

3. Enter values:

   • Submission Mode: Select one:

     • Full mode includes data deletion. Use Full mode only if you are confident that you are reloading all current data.

     • Incremental is faster and does not include data deletion.

     If you are submitting a transformation for a single table and the table is defined with Unit of Work processing, select:

     • Full UOW includes data deletion. Use Full UOW only if you are confident that you are reloading all current data for each subject or subject visit that has any data included.

     • Incremental UOW is faster and does not include data deletion.

       Tip:

       Set up regular Incremental loads at frequent intervals and do Full loads at longer intervals.

   • Force Execution: Select if you want to run the job even though the source data currency, parameter values, and the version number of the program(s) have not changed since the last run. The system uses Full mode regardless of the Submission Mode setting. Full mode includes data deletion.

     If not selected and all the conditions are the same as the last run, the system does not execute the job and returns a status of Success.

   • Submission Type:

     • Immediate Run the job once, as soon as possible.

     • Scheduled Set up a regular schedule.

     • Deferred Run the job once, at a future time.

   • Trigger Downstream Transformations and Validation Checks: Select to make this job trigger validation checks on the target model and transformations from the target model to all others that come after it, in sequence. This can happen only if the source models are set up to trigger downstream processes.

Click the Refresh icon for an update to the Job Status.

Cancel a transformation job

Select a transformation in the Run History pane and click:

• Cancel Job to cancel the currently running or next pending job.

• Cancel All Jobs to cancel the currently running or next pending job and all future jobs in the schedule for the selected transformation. Available only for scheduled jobs.

  Note:

  If there is an immediate or deferred job for the same transformation, or another schedule for the same transformation, those jobs are not affected.

• Cancel Triggered Job to cancel the currently running or pending job. Available only for jobs that are set up to be triggered by another job.

Run validation check batches and view run history

1. Go to the Home page, Validation Checks tab.
2. From the Model drop-down list, select the clinical data model.
3. Select a validation check batch in the upper pane.

   The system displays information about its validation checks in the middle pane and information about its run history in the lower pane.

4. To view all jobs, click the View Full Job History icon.

   To view only recent jobs again, click the View Recent Jobs Only icon.

   To view log files, click the icon in the column for the type of job:

   • Log (Run History pane) The most recent manually submitted job.

   • Triggered Job Log (Run History pane) The most recent triggered job.

   • Install Job Log (Validation Check Batch pane) The most recent installation of the validation check batch.

   Note:

   A validation check can be disabled so that it is not included in the batch execution. To find out if a check was included in the run, check the log file.

   See:

• Run a validation check batch
  
• Cancel a validation check batch job
  

Run a validation check batch

You must run validation checks as a batch.

1. Go to the Home page, Validation Checks tab.
2. Select a validation check batch and click the Submit Job icon.

   Tip:

   The Submit Job icon does not appear if the validation check batch is not installed. Check the Installed Status.

   If the installation status of a validation check batch is Warning, you may still be able to run the batch. Check the installation log file. If the status is Warning because one of the source tables is not used in the current study, the batch runs without input from that table.

3. Enter values:

   • Submission Mode: Select one:

     • Full mode processes all records.

     • Incremental mode processes only new and changed records.

   • Force Execution: Select to run the job even though the source data currency, parameter values, and the version number of the program(s) have not changed since the last run. The system uses Full mode regardless of the Submission Mode setting. Full mode includes data deletion.

     If not selected and all the conditions are the same as the last run, the system does not execute the job and returns a status of Success.

   • Submission Type: Select:

     • Immediate to run the job once, as soon as possible.

     • Scheduled to set up a regular schedule.

     • Deferred to run the job once, at a future time.

   • Trigger Downstream Transformations and Validation Checks: Select this checkbox if you want the system to detect all transformations and validation checks set up for this data model and all others that come after it, and submit them sequentially.

     Note:

     This option appears only if the validation check batch is set up to allow it.

   • Click the Refresh icon at any time for an update.

   • To check the log file, click the icon in the Log column.

Cancel a validation check batch job

Select a validation check batch in the Run History pane and click:

• Cancel Job to cancel the currently running or next pending job.

• Cancel All Jobs to cancel the currently running or next pending job and all future jobs in the schedule for the selected validation check batch. Available only for scheduled jobs.

  Note:

  If there is an immediate or deferred job for the same batch, or another schedule for the same batch, those jobs are not affected.

• Cancel Triggered Job to cancel the currently running or pending job. Available only for jobs that are set up to be triggered by another job.

Reprocess discrepancies that failed to be sent to InForm

Validation checks with an Initial Action of Send to InForm immediately send discrepancies they create to InForm as queries. To check the status of this operation:

1. Go to the Home page, Validation Checks tab.
2. Click the Failed to Send Discrepancies icon in the upper right corner.

   The system displays counts for the selected study and lifecycle stage:

   • Processing for System shows the number of discrepancies currently being processed.

   • Failed Processing for System shows the number of discrepancies that should have been sent to InForm but weren't.

3. If any discrepancies failed to be sent, click the Reprocess Discrepancies icon to send them again. Click the Refresh icon to see progress.

Check if data files loaded

• View data files not processed
  
• View data load history
  
• Review data load errors (optional)
  

View data files not processed

1. Go to the Home page and select a study.
2. Select the Files Not Processed tab.

   The system displays files that were uploaded but could not be loaded into the selected study. A file may not be loadable because:

   • It is misnamed or in nonmatching case, compared to the file specification defined for the clinical data model.

   • There is a mistake in the File Specification regular expression defined for the model.

   • There is a matching File Specification, but its end date has passed.

   • There is a matching File Specification, but its data loading is suspended.

   • There is a matching File Specification, but the model is not installed.

3. View unprocessed files.

   Columns include:

   • Status: The possible statuses are:

     • DETECTED: The file has been detected in the watched folder but has not yet been submitted.

     • MISSING: The file was detected but deleted before the scheduled deletion or archive date.

     • DELETED: The file was deleted by File Watcher as scheduled.

     • ARCHIVED: The file was archived by File Watcher as scheduled.

   • File Modified: The modification date of the file on the file system.

   • Detection Date: The date and time the file was detected, using the date and time in the DMW database.

   • Archive Date displays the scheduled archive date before the file is archived and the actual archive date afterward.

     Note:

     Archiving is not supported in this release.

   • Deletion Date displays the scheduled deletion date before the file is deleted and the actual deletion date afterward.

   • Date Missing: If the file is overwritten or removed from the file system before it is archived or deleted, then the Date Missing is store here.

   • File Error: Information about the problem.

View data load history

1. Go to the Home page, Data Loads tab.
2. To view all jobs, click the View Full Job History icon.

   To view only recent jobs again, click the View Recent Jobs Only icon.

3. You can filter by entering a value in the blank field above any column. If blank fields are not displayed, click the Query By Example icon.

   Columns include:

   • Data Model: The clinical data model into which data was or will be loaded.

   • Type:

     • FILEWATCHER for data loaded from a file.

     • INFORM for data loaded from InForm.

       Note:

       All pending jobs for InForm data loads are for the Production lifecycle stage. It isn't possible to schedule InForm data loads for the Development or Quality Control lifecycle stage.

   • File Name/InForm Load Process:

     • For files, the full path of the file loaded.

     • For InForm, the type of data load: INFORM_DEFINITION, INFORM_METADATA, INFORM_MANUAL_DATA or INFORM_SCHEDULED_DATA.

   • Last Load Results shows the job status: Success, Warnings, or Failure for completed jobs.

   • Columns with icons: Click the icon to open the error, log, or output file.

4. See the following (if necessary):

• Statuses for uncompleted jobs
  

Statuses for uncompleted jobs

For incomplete jobs, the system displays the job's current status. Click the Refresh icon to update.

• Pending: The job has not yet started running.

• Started: The job has begun pre-processing.

• Executing: The Program has connected to the database and is running.

• Finalizing: The job has begun post-processing.

• Aborted: The job has been manually stopped while underway.

• On Hold: The job is waiting for the quiesce process to complete for the clinical data model work area.

• Expired: The system removed the job from the queue after the timeout interval passed.

• Duplicate: The job is a duplicate of another job; the currency of the source data, parameter values, and executable instance version are the same. The system does not rerun the job unless the person submitting the job chooses to force reexecution.

Review data load errors (optional)

To review data load errors:

1. Go to the Home page and select the study.

2. Make sure the Data Loads tab is selected.

3. Open a file in one of the following columns:

   • Last Load Results column: Contains the results of the most recent data load. This file is produced whether the data load is successful or failed.

   • View Output column: Produced during a data load (the SQL*Loader log). This file is produced whether the data load is successful or failed.

   • View Error File column: Created when a text file for a data load contains more fields than there are columns in the table you're loading the data into. This file is produced for failed data loads only.

     Tip:

     This column is typically empty.

   • View Comprehensive Error Report column: The error log includes a list of the specific data record and field from the file that caused the error. This file is produced for failed data loads only.

FAQs

• Why can't I see a job that ran a while ago?
  
• Why am I seeing a Diagnostics Icon?
  

Why can't I see a job that ran a while ago?

By default you can see only recent jobs. To see a job that happened longer ago, click the View Full Job History icon.

To change back, click the View Recent Jobs Only icon.

This applies to the Home page Data Loads tab and the Run History pane of the Transformations and Validation Checks tab.

Why am I seeing a Diagnostics Icon?

The icon indicates that your administrator is running diagnostics for troubleshooting purposes. Performance may be slower than usual.