Following Server side installations must be installed by Oracle Retail Cloud Service Providers: EICS and BDI-SIM Edge App Job Admin.
Before you can load data files use BDI Transmitter CLI tool, you need install BDI Transmitter on Client machine. See following steps for installing BDI Transmitter CLI tools on Client side.
To prepare the tool for use, follow these steps.
The bdi-cli-transmitter home directory (where the tool software package is extracted) contains 'conf' directory where the tool related configuration files will be present, and 'bin' directory where the executable to run the tool will be present.
Configure conf/bdi-file-transmitter.properties. The following describes the properties to be configured. The properties file provides some sample values for reference to start with. The values specified in the properties file can be overridden using the command-line input options if required, when running the tool for file transmission.
Table B-1 bdi-file-transmitter.properties
| Property | Description |
|---|---|
|
source.system.name |
The name of the source system or application that provides the source data to be transmitted. For example, |
|
<receiverAppName>.receiver.url |
The Receiver REST service URL of the BDI Receiver application indicated by <receiverAppName> (should be in lower-case). When use BDI Transmitter CLI tool to integrate with EICS, need to set the sim.receiver.url. For example, if the BDI receiver application is SIM, then specify the property and value as: sim.receiver.url=http://<bdi-sim-app-hostname>:<port>/sim-batch-job-admin/resources/receiver |
|
<receiverAppName>.receiver.url.useralias |
Alias name for the credentials to be used to connect to the corresponding receiver service. The alias name with the credentials are stored in a wallet. <receiverAppName> should be in lowercase. Example: |
|
<InterfaceModuleName>.receiver.appname |
Name of the BDI receiver application for the interface module <InterfaceModuleName>. Specify the name in lowercase. Example:
|
|
<InterfaceModuleName>.dataset.type |
The data set type of the data to be transmitted for the interface module identified by <InterfaceModuleName>. Valid value is FULL or PARTIAL. Example:
|
|
<InterfaceModuleName>.interfaceShortNames |
The interface name(s) for the corresponding interface module <InterfaceModuleName>. Multiple interface names can be specified (each separated by a comma) as multiple interfaces can be part of an interface module. The interface module name and interface names should be the same as expected by the BDI receiver application where the files are transmitted. Example:
|
|
<InterfaceModuleName>.<InterfaceShortName>.input.filepath |
Specify the file location where the corresponding interface data files to be transmitted are present. Each interface in a interface module should have separate file locations. Example:
|
Example B-1 bdi-file-transmitter.properties
source.system.name=ext-source
sim.receiver.url=https://<bdi-sim-app-hostname>:<port>/sim-batch-job-admin/resources/receiver
sim.receiver.url.useralias=simReceiverUrlUserAlias
interfaceModules=ExtPrice_Tx
ExtPrice_Tx.receiver.appname=sim
ExtPrice_Tx.dataset.type=PARTIAL
ExtPrice_Tx.interfaceShortNames=Ext_Price
ExtPrice_Tx.Ext_Price.input.filepath=/home/bdi/ext_price_tx/ext_price/files
Run: bdi-file-transmitter.sh -setup-credentials.
Note:
bdi-file-transmitter.sh will be in the 'bin' directory.Run -setup-credentials to configure the BDI Receiver service user credentials. Running this command will prompt for the username and password for each of the <receiverAppName>.receiver.url.useralias specified in bdi-file-transmitter.properties file.
The credentials entered for each alias will be stored in a secure wallet and used to connect to the corresponding BDI Receiver service for transmission of files.
This is a prerequisite step to use the tool but usually a one-time setup before running bdi-file-transmitter.sh for transmission of files.
Optionally, configure conf/bdi-file-transmitter-runtime.properties that contains parameters (described below) for performance tuning of the tool.
Start with default values as present in the properties file, analyze the performance and choose optimal values for the parameters for better performance if required. The tool will use default values for the parameters (mentioned below) when no values are specified in the properties file.
| Property | Description |
|---|---|
| multiple_files_process_limit | The maximum number of files to process in parallel at any given time. Default value is 5. |
| file_transmission_thread_limit | The number of parallel threads to run to process a single file. Default value is 3. |
| transmission_record_size | The maximum number of records per block or chunk to transmit to the receiver service per service call. Default value is 20000. |
| transmission_timeout | The timeout in minutes for file transmission. The process will timeout and end when the file transmission is still not complete after the specified time. Default value is 300 minutes. |
The BDI CLI Transmitter tool is run using the shell script: bdi-file-transmitter.sh from the 'bin' directory.
The tool can be run in interactive and non-interactive modes.
Interactive Mode: Run bdi-file-transmitter.sh
For user interactive mode where the program prompts for input, just run bdi-file-transmitter.sh with no options.
This will prompt for each input with descriptions which will be self-explanatory. The user can enter value as required or skip optional parameters. When no value is specified for optional parameters, the tool will try to use the default values as specified in the bdi-file-transmitter.properties file or stop executing when no default value is present.
Non-Interactive Mode: Run bdi-file-transmitter.sh [input]
The tool can be run with the following inputs as described below.
Note:
The only required input is interface module name, when the other input values are specified in bdi-file-transmitter.properties file.| Input | Description |
|---|---|
| -m or --interfacemodule <interfaceModuleName> | (Required) The interface module name. Should be the same as the interface module name expected by the BDI receiver application. |
Some examples of running the transmitter tool command-line:
bdi-file-transmitter.sh -m ExtPrice_Tx
The BDI Transmitter tool supports transmission of flat files, for example, .csv files, in UTF-8 format. The BDI Receiver application supports only csv files. Hence the interface data files to be transmitted need to contain records with comma-separated field values.
The order of the fields in the file should be as expected by the BDI Receiver application, so that each value is inserted in the right columns of the destination interface tables. No header line should be present in the file (each line is treated as data record). Each record should be in a newline.
The interface module name and interface names for the files to be transmitted should be same as expected by the BDI Receiver application.
The transmitter tool can process a single file or a directory containing multiple files. But the tool does not process files recursively in subdirectories.
Files are processed and transmitted per interface module. Each run of processing of files of the interface module will be considered a transaction and a Transaction Id will be generated and associated to the transmission of files (at the interface module level). Files of multiple interfaces in an interface module will be part of the same transaction.
Each file transmission within a transaction will have a Transmission Id associated to it. The same transaction Id and transmission Id are sent to the BDI Receiver application, so the corresponding transmission details can be seen in the Job Admin console of the BDI Receiver application.
After successful transmission, the file will be moved to the archive directory:
<inputFileDirectory>/archive/<interfaceModuleName>/<transactionId>
For example, if the input file location is '/home/bdi/interface/files' and the interface module of the files is 'ExtPrice_Tx', and the transaction Id of the file transmission is 'Tx#1475858081837', then after successful transmission the file will be moved to the directory:
/home/bdi/interface/files/archive/ExtPrice_Tx/Tx#1475858081837.
The transmitter tool outputs messages and logs to the terminal console where the command is run.
The tool also creates a log file that contains detailed logs about the processing of files. The log will show the Transaction Id and Transmission Id of each file transmission among other details.
The log file is created in the logs directory under the tool home directory (bdi-cli-transmitter/logs).
The name of the log file will be in the format: bdi-file-transmitter_yyyy-mm-dd_hh:mm:ss, for example bdi-file-transmitter_2016-07-04_10:38:59.
In case of any error in file processing, error in transmission of file to the receiver service, timeout of file transmission, or any other failure, the file will be moved to the 'failed' directory: <inputFileDirectory>/failed/<interfaceModuleName>/<transactionId>
For example, if the input file location is '/home/bdi/interface/files' and the interface module of the files is 'ExtPrice_Tx', and the transaction Id of the file transmission is 'Tx#1475858081837', then if the transmission of file fails, the file will be moved to the directory: /home/bdi/interface/files/failed/ExtPrice_Tx/Tx#1475858081837.
A properties file containing the input details corresponding to the failed file will be created. For example, if the file named 'ExtPrice_1.csv' has failed, then a file named 'ExtPrice_Tx_1.csv.properties' will be created in the 'failed' directory. This acts as the input context that will be used when the file is reprocessed. The user should not delete or modify this properties file, if the data file has to be re-processed with the original input context.
Due to parallel processing of files by the transmitter, there may be a scenario where some records in the file may have been transmitted successfully, but part of the file transmission may have failed. Even in this case, the entire file will be treated as failed and moved to the 'failed' directory.
Reprocessing will be at the file level and not at the block level where the transmission may have failed. In the case of partial transmission of file, the BDI Receiver application also marks the whole transmission as failed and hence the entire file can be retransmitted to be processed again by the receiver application.
To retry failed files (that did not get transmitted successfully in previous transmission) use the below command:
bdi-file-transmitter.sh -retry-failed <inputFileDir or inputFilePath>
For example, bdi-file-transmitter.sh -retry-failed /home/bdi/interface/files/failed/ExtPrice_Tx/Tx#1475858081837
bdi-file-transmitter.sh -retry-failed /home/bdi/interface/files/failed/ExtPrice_Tx/Tx#1475858081837/ExtPrice_1.csv
Once a file is successfully reprocessed, it will be renamed as <filename>-retransmitted. For example, ExtPrice_1.csv-retransmitted. And, the corresponding properties file will be deleted.
Login in EICS and navigate to the Job Scheduler screen.
Select the desired job (for example, External Price TX Import or External RFID TX Import) from the list.
Edit the Job details on right panel.
Set required interval for execution and Enabled to Yes.
Click Apply to exit the edit mode.
For configuring multiple jobs on a go, repeat Step 2 for each job.
Click Save to save the current changes made on the screen.