Upload and Transforming a File
To upload a file in the ORMB application, you should use the File Transformation and Upload (C1-FTRAN) batch which uploads either an XML file that contains record(s) in compliance with the ORMB service schema or simply transforms and uploads the file in customer format, either in XML or CSV or PSV or JSON format. The C1-FTRAN batch uses the Record Transformation Algorithm to transform the file records into ORMB conform service schema and upload the same in the ORMB system. The uploaded file details are logged with the FILE_ID field in the CI_FILE_REQUEST table with either Complete or Pending status.
-
File Record Transformation is only eligible for XML or CSV or PSV or JSON or Fixed Formats.
-
The file will only be uploaded with Pending status when the all of the records in the file are not uploaded due to failure, else the file will be uploaded with Complete status.
-
Every file will contain records for only a single File Request Type.
The C1-FTRAN batch is required to read the file(s) on the SFTP server at the given file path (passed as the soft parameter). If a File Validation Algorithm is mapped to the corresponding file request type then header, footer, and checksum validation can be performed. This batch reads one file at a time and adds a token on all of the records of the file. Once all the file records are read with a token, you must then create a ThreadWorkUnit for every single token, i.e. Number of records equal to the Number of Work-units. Before logging the file details and creating a ThreadWorkUnit, the file transformation batch validates the following conditions,
Condition | Result |
---|---|
If the File Path is invalid | Batch is terminated with ERROR status without logging any file details in the CI_FILE_REQUEST table. |
If the File Path of Error Log is invalid | Batch is terminated with an ERROR status. |
If the File does not exists in the path | The C1-FTRAN batch is terminated with an ERROR status without logging any file details within the CI_FILE_REQUEST table. |
If the File name Extension does not matches with the one provided in Request Type Configuration | Batch will be terminated with ERROR status without logging any file details in the CI_FILE_REQUEST table. |
If the file size exceeds the limit of 2GB | Batch is terminated with COMPLETE status. The file details are logged with an ERROR status and in the CI_FILE_REQUEST table. |
If the File Request Type is incorrect | Batch is terminated with an Error status. |
If encrypted files are uploaded, File Decryption process gets executed on those files. If File Decryption process fails | Batch is terminated with a COMPLETE status. The file and error details are logged with an ERROR status and in theCI_FILE_REQUEST table. |
If the File Header (first line in the file) does not exist | Batch is terminated with a COMPLETE status. The file and error details are logged with an ERROR status and in theCI_FILE_REQUEST table. |
If the File Footer (last line in the file) does not exist | Batch is terminated with a COMPLETE status. The file and error details are logged with an ERROR status and in theCI_FILE_REQUEST table. |
If the File Header and File Footer are excluded, and the number of records do not match with number of lines in the file | Batch is terminated with a COMPLETE status. The file and error details are logged with an ERROR status and in theCI_FILE_REQUEST table. |
If the uploaded file already exists in the ORMB system | Batch will be terminated with an ERROR status. The file and error details are logged with a REJECT status and in theCI_FILE_REQUEST table. |
If the checksum validation (for example, search for <FILENAME>.checksum parameter to retrieve the checksum value) fails | Batch will be terminated with an ERROR status. The file and error details are logged with a REJECT status and in theCI_FILE_REQUEST table. |
A list of ThreadWorkUnit is created by tokenizing the number of records in the corresponding file. Every single token is a single file record with unique REQUEST_ID identifier in CI_FILE_REQUEST_DETAIL table with PENDING status. All the files on SFTP server having same file extension corresponding to that configured for the given file record type will be read and uploaded in ORMB system. There are two fields to capture its corresponding XML (Payload) record data. Either of this field will be populated.
-
REQUEST with CLOB DataType – This field is populated when the record with File Request Type has the configuration of File Size Greater than 32KBas TRUE.
-
BO_DATA_AREA with VARCHAR2(32000) DataType – This will be populated when, request with File Request Type having configuration of Record Size Greater than 32KB as FALSE.
This handling is for better performance while uploading and processing file request data.
-
File that already exists in ORMB system will be ignored.
-
Java exceptions/Unknown errors will not be handled.
This batch supports both single-thread and multi-threaded execution using File Atomicity flag on File Request Type. Using the errorLogFilePath (Execute batch using File Chunks) soft-parameter this batch supports two ways for creating ThreadWorkUnits that are required for batch execution.
-
Fetch and create ThreadWorkUnits for all these number of requests in PENDING or RETRY status for the given File Request Type from the CI_FILE_REQUEST_DETAIL table.
Field Name Field Description File ID Used to specify a unique identifier for a file. File Name Used to specify the request object that is to be processed. Request Type Used to specify the file request type of the request object to be processed. Request Object Used to specify the payload / dataline in the application. File Retry Used to specify the value for the file retry field to skip the processed record in case the batch has been aborted. Duplicate Used to specify the value for the duplicate field to skip processing the request, if there are multiple records with the same identifier in a same file. ID Used to specify the value for the ID, specific to payment stage upload and transaction business services. XPath Used to specify the value for the XPath, specific to payment stage upload and transaction business services file request transformation. Field Name Used to specify the field name specific to payment stage upload and transaction business services file request transformation.
-
Fetch and create ThreadWorkUnits for all these number of request(s) in PENDING or RETRY status for the given File Request Type from CI_FILE_REQUEST_DETAIL table.
-
Number of files are created in proportion to the number of threads, for example, if the number of threads are 100, then 100 files are created.
-
These files will be created at the same file path on the SFTP server from where the uploaded files are being read.
-
All these newly created files will have approximately equal number of requests i.e. if there are 1000 requests, then, each file will have 10 requests or records.
-
ThreadWorkUnit will have supplementary data only for,
Field Name Field Description ID Used to specify the value for the ID, specific to payment stage upload and transaction business services. XPath Used to specify the value for the XPath, specific to payment stage upload and transaction business services file request transformation. Field Name Used to specify the field name specific to payment stage upload and transaction business services file request transformation. Note:These new temporary files are deleted after all the requests in the file are uploaded in the ORMB system.
-
Attribute Name | Attribute Description |
---|---|
Validate Checksum | Checksum validation is performed before reading and staging the file details in the ORMB system. |
Audit Log Required | Individual file request status (PENDING / PROCESSED / ERROR / SKIPPED / INPROGRESS / RETRY / IGNORE / RETRYLIMITEXCEED) transition will be logged in ORMB system |
Archive File | In-case there is no file validation failure, file is moved to Archive File Location else file is moved to Archive Error File location. |
File Encryption | In-case if the file encryption returns true, then the batch retrieves the encrypted file on SFTP server and decrypts the file using the File Decryption algorithm. |
File Request Type (Attribute Name) | Attribute Description |
---|---|
File Transformation Required |
|
File Transformation Algorithm |
|
File Atomicity |
|
File Upload and Process |
|
Skip Duplicates |
|
Service Logs Required |
|
File Header Required |
|
Header XML Tag |
|
File Footer Required |
|
Footer XML Tag |
|
Root XMl Tag |
|
You can also specify the following soft parameters while executing this batch:
Parameter Name | Description | Mandatory (Yes or No) |
---|---|---|
File Path | Used to specify the relative path where the file gets uploaded.
This path will always begin with either @SHARED_DIRor @INSTALL_DIR
|
Yes |
File Name | Used to specify the name of the file (for example CustomerOnboard).
If this value is left blank then all the files with the file request
type having a corresponding file extension are picked from the given
file path. The File Name parameter also supports
inclusion of the partial value,
|
Yes |
File Request Type | Used to specify the file request type which is uploaded in the system. | Yes |
Error Log File Path | Used to specify the CSV file (along with the error logs)
that are stored at a provided relative path. This path will always
start with either @SHARED_DIR or @INSTALL_DIR.
|
|
Execute Batch using File Chunks | Used to specify the execution behavior of the batch. If the parameter value is Y then the batch is executed using file chunks. | No |
Override Maximum Errors | Used to specify the value to override the maximum number of errors that are allowed before the execution is terminated. | No |
Thread Pool Name | Used to specify the thread pool name on which you want to execute the batch. | No |
Post Execution Check / Clean Up
Once the batch is successfully completed, the logged file details are updated with the COMPLETE status. If the File Upload and Process attribute value for the corresponding File Request Type is set as true, then all of those requests in that file are processed and uploaded with the execution status as either PROCESSED or ERROR or SKIPPED or INPROGRESS or RETRY or IGNORE or RETRYLIMITEXCEED