Deploying BDI Batch Job Administration Applications

4 Deploying BDI Batch Job Administration Applications

Before starting the installation procedure, identify the retail applications that are in-scope for this installation. BDI requires at least 2 applications in-scope, a sender app and a receiver app. Each bdi-app must be deployed on its own WebLogic managed server. Depending on the site specific deployment topology, create separate WebLogic domains OR managed servers to host the BDI apps in-scope.

Note:

Repeat the procedures in this chapter for installing other applications that are in-scope.

Deploying BDI Batch Job Admin Applications for a Sender Application

This section describes the installation process for a sender app with the BDI-RMS app as an example.

Preparing the Database for BDI Batch Job Admin for RMS Installation

Before you begin installing BDI RMS Batch Job Admin, make sure you have the database schema created for BDI RMS Batch Job Admin.

If you are migrating from previous version of BDI RMS Batch Job Admin to a newer version, Follow upgrade instructions for bdi.

Before following upgrade instructions for bdi, you need to provide permissions for database schema. Run the below commands on admin schema where database schema is created, to provide permissions.

Commands:

	CREATE TABLE TO 'user schema';
	CREATE SEQUENCE TO 'user schema';
	CREATE INDEX TO 'user schema';

Ex: User schema name is bdi_rms_app. Run below commands on admin schema.

CREATE TABLE TO 'bdi_rms_app';
CREATE SEQUENCE TO 'bdi_rms_app';
 
CREATE INDEX TO 'bdi_rms_app';

Note:

Database schema auto migration feature should be used from >=16.0.028 version.

Preparing the WebLogic Domain for BDI Batch Job Admin for RMS

Use the instructions provided in BDI Batch Job Admin to install WebLogic 12.2.1.4.0 and create a domain.
Start the Admin and managed servers.

Deploying BDI RMS Batch Job Admin on the WebLogic

To deploy the BDI RMS Batch Job Admin war, take the rpsfollowing steps.

Download BdiEdgeJobAdminPak24.0.000ForRms24.0.000_eng_ga.zip to $BDI_HOME.

Unzip the downloaded archive.The Bdi Job home directory will be created under the current directory.

unzip BdiEdgeJobAdminPak24.0.000ForRms24.0.0007_eng_ga.zip

This command extracts the archive. The relevant directories for the installation are shown (There are more directories than what is shown).

|- - - - - - - bin
|                    ` - - - - - - bdi-job-admin-deployer.sh
|- - - - - - - conf
|                    ` - - - - - - security
|	           |              `- - - - - - - - jazn-data.xml
|	           |              `- - - - - - - - jps-config.xml
|                    ` - - - - - - bdi-job-admin-deployment-env-info.json
|                    ` - - - - - - bdi-job-admin-deployment-env-info.json.template
|                    ` - - - - - - bdi-job-admin-internal-trust-store.jks
|                    ` - - - - - - log4j2.xml
|- - - - - - - dist
|                    ` - - - - - - bdi-batch-job-admin.war
|                    ` - - - - - - rms-jars
|                    ` - - - - - - README.txt
|- - - - - - - scripts
|                    ` - - - - - - DBSchemaMigration.groovy
|                    ` - - - - - - JobAdminDeployer.groovy
|                    ` - - - - - - README.txt
|                    ` - - - - - - WebLogicManager.groovy
|- - - - - - - setup-data
|                    ` - - - - - - dml
|	           |              `- - - - - - - - url_seed_data_1.sql
|	           |              `- - - - - - - - bdi_rms_seed_data.sql
|	           |              `- - - - - - - - BDI_SET_BATCHSTATUS_TO_ABANDON.sql
|	           |              `- - - - - - - - README.txt
|	           |              `- - - - - - - - rms_group_seed_data.sql
|                    ` - - - - - - available-jobs-for-external-app-integration
|                    ` - - - - - - batch-scripts
|	           |              `- - - - - - - - bdi_sftp_push.sh
|                    ` - - - - - - ddl
|	           |              `- - - - - - - - migration
|	           |                                        `- - - - - - - - BDI_CLEANUP_JOB_SQL.sql
|	           |                                        `- - - - - - - - BDI_Database_Util_Spec_Permission.sql
|	           |                                        `- - - - - - - - BDI_Database_Util_Spec.sql
|	           |                                        `- - - - - - - - create_wl_llr_table.sql
|	           |                                        `- - - - - - - - migrate-edge-batch-schema-from-16.0.025-to-16.0.027.sql
|	           |                                        `- - - - - - - - migrate-edge-batch-schema-from-16.0.028-to-16.0.030.sql
|	           |                                        `- - - - - - - - migrate-edge-bdi-infra-schema-from-16.0.025-to-16.0.027.sql
|	           |                                        `- - - - - - - - migrate-edge-bdi-infra-schema-from-16.0.028-to-16.0.030.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.021-to-16.0.023.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.023-to-16.0.025.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.025-to-16.0.027.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.027-to-16.0.028.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.028-to-16.0.030.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.030-to-16.0.031.sql
|	           |                                        `- - - - - - - - migrate-schema-from-16.0.031-to-19.0.000.sql
|	           |                                        `- - - - - - - - migrate-schema-from-19.0.000-to-19.1.000.sql
|	           |              `- - - - - - - - purge
|	           |                                        `- - - - - - - - purge_batch_db_repo.sql
|	           |                                        `- - - - - - - - purge_job_int_repo.sql
|	           |                                        `- - - - - - - - purge_job_rcvr_repo.sql
|	           |              `- - - - - - - - README.txt
|                    ` - - - - - - META-INF
|                    ` - - - - - - batch-jobs
|- - - - - - - target
|                    ` - - - - - - rms-batch-job-admin.war
|                    ` - - - - - - README.txt
|- - - - - - - lib

Note:

To Integrate BDI-RMS with an external application, additional steps are required. Refer to Appendix: Integrating BDI-RMS with External Applications for additional information.

Edit the rms job configuration file to point to deployment environment (conf/bdi-job-admin-deployment-env-info.json) as follows:

To deploy without OAuth, we need to rename or remove the OAuth2 aliases for example:- jobAdminUiOAuth2ApplicationClientAliasRef from bdi-job-admin-deployment-env-info.json, and then deploy. No other extra steps needed for deployment.
```
cd bdi-edge-rms-job-home/conf
vi bdi-job-admin-deployment-env-info.json
```
Note:

BDI does not mandate the use of OAuth2 or IDCS as authorization server. This version of the BDI is backward compatible with basicAuth. New customers can use basicAuth by following the below steps. Existing customers can use upgrade instructions and proceed to use basicAuth as the authentication mechanism.

See How to Install BDI without IDCS.

Provide the following values in the JSON file.

Configuration Property	Description
RmsJobAdminDataSource -> jdbcUrl	The JDBC URL for the Oracle database where the RMS integration schema resides.
RmsReceiverServiceDataSource -> jdbcUrl	The JDBC URL for the Oracle database where the RMS Receiver integration schema resides. BdiRmsJobAdminDataSource and BdiRmsReceiverServiceDataSource can use the same jdbcUrl.
BatchInfraDataSource -> jdbcUrl	The JDBC URL for the schema created using RCU. The schema user is <prefix>_wls (e.g. INT_WLS). The value of prefix that you provided in schema creation using RCU.
JobXmlDataSource -> jdbcUrl	The JDBC URL for the Oracle database where the all job xmls gets created in rms schema.
JobAdminAppServer -> weblogicDomainName	The domain name where BDI RMS Admin app is deployed.
JobAdminAppServer->weblogicDomainHome	The domain path where BDIRMS Admin app is deployed
JobAdminAppServer -> weblogicDomainAdminServerUrl	Admin Server URL of the BDI RMS Admin App - http://<Admin host>:<Admin port>
JobAdminAppServer->weblogicDomainAdminServerProtocol	Admin Server protocol, which is by default t3, For SSL deployment update to t3s.
JobAdminAppServer -> weblogicDomainAdminServerHost	Host Name of the BDI RMS Admin Server
JobAdminAppServer -> weblogicDomainAdminServerPort	BDI RMS Admin Server Port
JobAdminAppServer -> weblogicDomainTargetManagedServerName	Managed Server Name where BDI RMS Admin App is installed (e.g. bdi-rms-server)
JobAdminAppServer -> jobAdminUiUrl	Job Admin URL of BDI RMS http://<host>:<managed server port>/bdi-rms-batch-job-admin
dataSourceProperties	DataSource configurations contributes to the performance of the applications at the runtime. Ex: "connectionPool_MaxCapacity":"100" "general_RowPreferfetchSize":"100"
RfiJobAdminAppServer-> jobAdminUiUrl	Job Admin URL of BDI RFI http://<host>:bdi-rfi-batch-job-admin
ExternalJobAdminAppServer -> jobAdminUiUrl	Job Admin URL of BDI RFI http://<host>:<external managed server port>/external-batch-job-admin
oauth2AuthorizationServerUrl	Provide the IDCS url For example - https://idcs-4ff493196128425c80ce4ecbfc8352e5.identity.c9dev1.oc9qadev.com/oauth2/v1/token
jobAdminUiOAuth2ApplicationClientAliasRef	ICDS Client secret ID and password "name":"rfiJobAdminBaseUrlOAuth2ApplicationClientAlias", "value": "*ricsOauth2ApplicationClientAlias"
SystemOptions	Optional. Allows to provide system options as name value pairs

Note:

The alias names in the configuration files should not be changed.

Below are the changes for bdi-job-admin-deployment-env-info.json.

Below json snippet is to support OAuth2 with IDCS for Job Admin.

"CentralAuthenticationSystem":{
  "IdcsAuthenticationProvider":{
    "oauth2AuthorizationServerUrl":"https://idcs-4ff493196128425c80ce4ecbfc8352e5.identity.c9dev1.oc9qadev.com/oauth2/v1/token",
    "oauth2Application":[
      {
        "oauth2ApplicationName" : "RICS",
        "oauth2ApplicationScopeOfAccess" : {"name":"oauth2.default.scopeOfAccess.*", "val-ue":"urn:opc:idm:__myscopes__"},
        "oauth2ApplicationClientAlias" : "ricsOauth2ApplicationClientAlias",
        "oauth2ApplicationClientId" : "GET_FROM_WALLET",
        "oauth2ApplicationClientSecret" : "GET_FROM_WALLET"
      },
      {
        "oauth2ApplicationName" : "MFCS",
        "oauth2ApplicationScopeOfAccess" : {"name":"oauth2.default.scopeOfAccess.*", "val-ue":"urn:opc:idm:__myscopes__"},
        "oauth2ApplicationClientAlias" : "mfcsOauth2ApplicationClientAlias",
        "oauth2ApplicationClientId" : "GET_FROM_WALLET",
        "oauth2ApplicationClientSecret" : "GET_FROM_WALLET"
      }
    ]
  },
  "OamAuthenticationProvider":{
  }
}

Optional – Users can configure system options in the deployment env info json file. Users have to add the system options as part of the configuration in the json file with name value pairs. After the application starts, the system option tables should have system options provided in the json filea=b and c=d values in the system options table. After deploying Jjob Admin, verify that values are getting added to the system option table properly.

Example:

Job Admin JSON Snippet:

        "JobAdminApplication":{
            "appName":"rms",
            "JobAdminAppUses":[
                "JobAdminDataSource",
                "JobAdminAppServer",
                {
                    "RemoteJobAdminAppServers":[
                       "RfiJobAdminAppServer",
                       "ExternalJobAdminAppServer"
                    ]
                }
            ],
             "SystemOptions":[
                {"name":"MFP_outboundLocation", "value":"/replace/with/outbound/location/for/mfp"},
                {"name":"RDF_outboundLocation", "value":"/replace/with/outbound/location/for/rdf"},
                {"name":"AP_outboundLocation", "value":"/replace/with/outbound/location/for/ap"},
                {"name":"IP_outboundLocation", "value":"/replace/with/outbound/location/for/ip"},
                {"name":"shellCmdWorkingDir", "value":"/replace/with/path/to/working/dir/for/shellCmds"},
                {"name":"shellCmdLocationDir", "value":"/replace/with/path/where/shell_programs/are/present"},
                {"name":"downloadFileLocation", "value":"/replace/with/path/where/to/download_files/before/files/move/to/finalOutboundLocation"},
                {"name":"overwriteOutboundFilesFlag", "value":"replace_with_TRUE_or_FALSE"},
                {"name":"autoPurgeOutboundData.global", "value":"TRUE"},
                {"name":"autoPurgeOutboundDataDelay.global", "value":"30"},
                {"name":"MFP_sftpHost", "value":"replace_with_sftp_host_for_mfp"},
                {"name":"RDF_sftpHost", "value":"replace_with_sftp_host_for_rdf"},
                {"name":"AP_sftpHost", "value":"replace_with_sftp_host_for_ap"},
                {"name":"IP_sftpHost", "value":"replace_with_sftp_host_for_ip"},
                {"name":"MFP_sftpPort", "value":"replace_with_sftp_port_for_mfp"},
                {"name":"RDF_sftpPort", "value":"replace_with_sftp_port_for_rdf"},
                {"name":"AP_sftpPort", "value":"replace_with_sftp_port_for_ap"},
                {"name":"IP_sftpPort", "value":"replace_with_sftp_port_for_ip"},
                {"name":"MFP_sftpUser", "value":"replace_with_sftp_user_for_mfp"},
                {"name":"RDF_sftpUser", "value":"replace_with_sftp_user_for_rdf"},
                {"name":"AP_sftpUser", "value":"replace_with_sftp_user_for_ap"},
                {"name":"IP_sftpUser", "value":"replace_with_sftp_user_for_ip"},
                {"name":"MFP_destinationLocation", "value":"/replace/with/ftp/location/for/mfp"},
                {"name":"RDF_destinationLocation", "value":"/replace/with/ftp/location/for/rdf"},
                {"name":"AP_destinationLocation", "value":"/replace/with/ftp/location/for/ap"},
                {"name":"IP_destinationLocation", "value":"/replace/with/ftp/location/for/ip"},
                {"name":"ftpFilesFlag", "value":"replace_with_TRUE_or_FALSE"}
            ]
        }

Run the deployer script to create the data sources and deploy BDI RMS Batch Job Admin. While deploying, the user needs to provide the IDCS client secret ID & password for ricsOauth2ApplicationClientAlias, mfcsOauth2ApplicationClientAlias
```
cd bdi-edge-rms-job-home/bin
bdi-job-admin-deployer.sh -setup-credentials -deploy-job-admin-app
```

Enter the values prompted by the script for following credentials aliases. JobAdmin user has more privileges (e.g change configuration and run jobs from JobAdmin UI) than JobOperator user. JobOperator can run batch jobs where as JobMonitor has just read privileges.

Alias	Description
bdiAppServerAdminServerUserAlias	WebLogic admin server credentials
rmsJobAdminUiUserAlias	Credentials for Admin Role user for Job Admin app
rmsJobOperatorUiUserAlias	Credentials for Operator Role user for Job Admin app
rmsJobMonitorUiUserAlias	Credentials for Monitor Role user for Job Admin app
rmsJobAdminDataSourceUserAlias	Credentials for the Data Source of the Job Admin Schema
rmsReceiverServiceDataSourceUserAlias	Credentials for the Data Source of the Job Receiver Schema
batchInfraDataSourceUserAlias	Credentials for the Data Source of the Batch Infra Schema
jobXmlDataSourceUserAlias	Credentials for the Data Source of the Job Xml Schema
externalJobAdminBaseUrlUserAlias	BDI EXTERNAL job admin app credentials
rfiJobAdminBaseUrlUserAlias	BDI RFI job admin app credentials
jobAdminUiOAuth2ApplicationClientAliasRef	ICDS Client secret ID and password "name":"rfiJobAdminBaseUrlOAuth2ApplicationClientAlias", "value": "*ricsOauth2ApplicationClientAlias"

Note:

If you have already configured the credentials and can use the same credentials (typically when redeploying the app), you can run the deployer with the -use-existing-credentials option as follows, and you will not be prompted for the credentials again for the deployment.

bdi-job-admin-deployer.sh -use-existing-credentials -deploy-job-admin-app

Deployer script deploys BDI RMS Batch Job Admin to the managed server.
Restrict access to the bdi-edge-rms-job-home folder:
```
cd bdi-edge-rms-job-home
chmod -R 700 . 
```
Bounce the Edge managed server.

Testing the Deployment

After you deploy to the server successfully, BDI RMS Batch Job Admin can be accessed using the following URL:

http://<server>:<port>/rms-batch-job-admin

Log in using credentials provided during the installation and verify that jobs are displayed in the Manage Batch Jobs tab and configuration is displayed in the Manage Configurations tab.

Creating Outbound Interface tables for BDI RMS

Go to the $BDI_HOME/bdi-edge-rms-job-home/setup-data/ddl folder.
Run the DDL script "bdi_rms_ddl.sql" provided in this folder in the BDI RMS database schema.
DDL generates interface tables and PL/SQL packages.

Upgrade Instructions for BDI

BDI Edge JobAdmin App (BdiEdgeJobAdminPak19.1.xFor<app>19.1.x_eng_ga.zip) Upgrade Steps

Download BdiEdgeJobAdminPak24.0.000ForRms24.0.000_eng_ga.zip to $BDI_HOME.
Take the backup of existing bdi-<app>-home.
Login to the BDI Edge Job App (Example: http://host:17011/rms-batch-job-admin)
Go to Manage Configurations -> System Options and make following changes. LOADJOBDEF = TRUE and LOADSEEDDATA = TRUE.
Login to the WLS Console and delete the existing Edge App.
Delete existing BDI datasources from WLS console, for any datasources changes planned during deployment.
Modify bdi job flow configuration file (conf/bdi-job-admin-deployment-env-info.json) to support OAuth2 feature. Below property needs to be updated in json file. Remaining property values should be same as backed up bdi-<app>-home. Click here to see new changes in bdi-job-admin-deployment-env-info.json. The deployment description json format has changed from previous release, to accommodate IDCS client credentials and URL.

"oauth2AuthorizationServerUrl":"https://idcs-4ff493196128425c80ce4ecbfc8352e5.identity.c9dev1.oc9qadev.com/oauth2/v1/token" ? Replace with IDCS server url.
To deploy without OAuth, we need to rename or remove the OAuth2 aliases for example:- jobAdminUiOAuth2ApplicationClientAliasRef from bdi-job-admin-deployment-env-info.json, and then deploy. No other extra steps needed for deployment.

See How to Install BDI without IDCS.
Deploy the BDI Edge job app using -setup-credential option (Follow BDI Installation Guide). -use-existing-credential option will not work the first time, since there is a new entry (IDCS client credentials) now. While deploying user needs to provide the IDCS client secret ID & password for ricsOauth2ApplicationClientAlias, mfcsOauth2ApplicationClientAlias. Cd to <appName>-job-home/bin and run the below command to upgrade and deploy the bdi job app.
```
sh bdi-job-admin-deployer.sh -use-existing-credentials } -run-db-schema-migration -deploy-job-admin-app
```
Note:

Database schema auto migration feature should be used from >=16.0.028 version.
After successful deployment, bounce the managed server.
Login to BDI Edge Job App and make sure no error is displayed in any of the pages.

Note:

BDI does not mandate the use of OAuth2 or IDCS as authorization server. This version of the BDI is backward compatible with basicAuth. New on-prem customers can use basicAuth by following the below steps. Existing customers can use upgrade instructions and proceed to use basicAuth as the authentication mechanism.

How to Install BDI without IDCS

To deploy without OAuth, we need to rename or remove the OAuth2 aliases for example:- jobAdminUiOAuth2ApplicationClientAliasRef from bdi-job-admin-deployment-env-info.json, and then deploy. No other extra steps needed for deployment.

        "RfiJobAdminAppServer": {           
            "comment_1": "For 16.0.028 and older RFI", 
            "comment_1_jobAdminUiUrl":"http://localhost:7001/bdi-rfi-batch-job-admin",
            "comment_2": "For 19.0.000 and later RFI",
            "jobAdminUiUrl":"http://localhost:7001/rfi-batch-job-admin",
            "jobAdminUiUserAlias":"rfiJobAdminBaseUrlUserAlias",
            "jobAdminUiUser":"GET_FROM_WALLET",
            "jobAdminUiPassword":"GET_FROM_WALLET",
        },
        "ExternalJobAdminAppServer": {
            "jobAdminUiUrl":"http://localhost:7001/external-batch-job-admin",
            "jobAdminUiUserAlias":"externalJobAdminBaseUrlUserAlias",
            "jobAdminUiUser":"GET_FROM_WALLET",
            "jobAdminUiPassword":"GET_FROM_WALLET",
        }
        "CentralAuthenticationSystem":{
 
            "IdcsAuthenticationProvider":{
 
                "oauth2AuthorizationServerUrl":"https://idcs-4ff493196128425c80ce4ecbfc8352e5.identity.c9dev1.oc9qadev.com/oauth2/v1/token",
                "oauth2Application":[
                    {
                         "oauth2ApplicationName" : "RICS",
                         "oauth2ApplicationScopeOfAccess" : {"name":"oauth2.default.scopeOfAccess.*", "value":"urn:opc:idm:__myscopes__"},
                         "oauth2ApplicationClientAlias" : "ricsOauth2ApplicationClientAlias",
                         "oauth2ApplicationClientId" : "GET_FROM_WALLET",
                         "oauth2ApplicationClientSecret" : "GET_FROM_WALLET"
 
                    },
                    {
                         "oauth2ApplicationName" : "MFCS",
                         "oauth2ApplicationScopeOfAccess" : {"name":"oauth2.default.scopeOfAccess.*", "value":"urn:opc:idm:__myscopes__"},
                         "oauth2ApplicationClientAlias" : "mfcsOauth2ApplicationClientAlias",
                    }
                ]
            }
        }

Continue with regular installation

Note:

In case any similar errors like ORA-00942: table or view does not exist Verify if the corresponding tables present in the schema or not, as there are cases where Bdi<App>ReceiverServiceDataSource and Bdi<App>JobAdminDataSource might have configured to different schemas.RTG maintains only one migration script for upgrades. Please ignore those errors.

BDI Edge RMS Job Admin Configuration file Changes

The following changes have been introduced to the BDI Edge RMS Job Admin Configuration file in 24.0.000 Release

For BDI RFI the application has been renamed from BdiIntJobAdminPak<VERSION>ForRfi<VERSION>_eng_ga.zip to BdiEdgeAppJobAdminPak<VERSION>ForRfi<VERSION>_eng_ga.zip, due to this change the BDI RMS Configuration for BDI RFI has also changed:

 "RfiJobAdminAppServer": {           
                "comment_1": "For 16.0.028 and older RFI",
                "comment_1_jobAdminUiUrl":"http://localhost:7001/bdi-rfi-batch-job-admin",
                "comment_2": "For 19.0.000 and later RFI",         
                "jobAdminUiUrl":"http://localhost:7001/rfi-batch-job-admin",
                "jobAdminUiUserAlias":"rfiJobAdminBaseUrlUserAlias",
                "jobAdminUiUser":"GET_FROM_WALLET",
                "jobAdminUiPassword":"GET_FROM_WALLET",
            },

New System Options Added to Disable Jobs at the time of installation:

                {"name":"jobSelection.1.pattern", "value":"ToExternalJob"},
                {"name":"jobSelection.1.initialState", "value":"false"},