This document describes the installation, configuration, and operation of the packaging utilities provided with the software development kit. These utilities enable developers to prepare releases of their custom modifications, called CM releases, to the products.. Releases prepared using these utilities may be installed on top of an existing base product environment.
Note. CM releases will correspond to a specific base product version and can only be installed on base product environments of that version. Customers installing a CM release must first verify the corresponding base product version with the Implementation team.
Note. This document describes CM packaging utilities operation for Oracle database only. The application server can be Unix or Windows/DOS operating system. In Unix you must execute the script with .sh suffix, in window the script with .cmd suffix. They both will execute the same Perl script with .plx suffix. For instance:
applyCM.sh: Unix driver script
applyCM.cmd: Windows driver script
applyCM.plx: Perl script
All the examples in this document are related to Unix. If you are in Windows/DOS simply execute the same scripts, but using .cmd extension instead .sh.
Contents
App Server CM Packaging Procedure
CM System Data Packaging Tools
Contents
App Server CM Packaging Overview
The following diagrams describe the app server CM packaging procedure.
The starting point of packaging the app server component is the project repository. The tool extractCMSource is used to get the source from the project repository into the packaging directory.
Note. The packaging directory must not be used for any other purpose except for storing the extracts. Mixing other files into the packaging directory will result in errors in succeeding processes.
applyCM copies the extracted source to the packaging app server. It then does all the necessary steps like generate, compile, etc., to update the packaging app server runtime based on the extracted source.
create_CM_Release is then used to create CM release install package from the packaging app server. The CM release install package contains all CM code that has been applied to the packaging app server.
The install tool applies the CM release install files to either QA or production app servers.
Note. Release install packages are usually applied only to fresh environments, i.e., to apply the first batch of CM code or when upgrading to a new version of the product. To install additional code to an existing environment, patch install packages (described next) are used.
CM patch install packages are used to update an existing installation with the changes since it was last updated. A patch install package is created by create_CM_Patch as the difference between two CM release install packages (a newer one and an older one), e.g., for a monthly update schedule, CM release install packages are created every month and every month, a cm patch install package is created using the release install packages from the previous and the current month.
To create a patch install package, a new release install package must be created first. Note that a release (not patch) install package must be available for the previous period. Executing create_CM_Patch with the two CM release install packages as input creates the patch install package.
When developing off-site, there may be no available environments on the target platform. In this case, development and QA must be done off-site, but packaging and QA must be re-done on-site using environments on the target platform.
Contents
Development and QA (and the packaging) is done using the same procedure as on-site except for the following:
· Packaging and QA are done on environments that may not match the target platform.
· Instead of sending a release or patch install package to the implementation site, only the source from the install package is sent. This package is called the release/patch source package. It is created by executing extractCMSource with the data directory of the install package as the source directory.
Off-site Development Release
Upon receiving the release/patch source package, the on-site team proceeds with the regular packaging procedure starting from the applyCM step using the release/patch source package as the source directory.
On-site Packaging
Applying a CM patch install package to QA or production app servers is the same as for release install package, i.e., it is done simply by executing the install tool from the the package directory. By using these scripts, implementation developers can prepare an installation package containing the contents of their custom modifications. Developers need to build a packaging environment on a server of the same operating system platform as used by the target environment to create CM release packages. A version number must be used to identify each custom modifications (CM) release version. Once developers select a version number format, the version number must be stored on the environment in the file $SPLEBASE/etc/CMVERSION.txt, to achieve this place CMVERSION.txt file in etc subdirectory in your patch directory.
Implementers are strongly recommended to use CM packaging utilities for implementation delivery to customer site. It will ensure the correct installation complying with base product rules and will keep an environment upgradable. Please, note, that web files can be also packaged in archive war format (if $isExpanded environment variable is set to false), in that case it is not possible to just manually copy changes to the directories.
The following utilities are provided in this package for maintaining the packaging environment and creating release versions of customer modifications (CM):
Instructions for using these utilities are described in the following sections.
Using the extractCMSource.plx Utility
Using the create_CM_Release Utility
Using the create_CM_Patch Utility
After the CM Packaging Tools installation, it is required to copy the proper spl-toos-<VERSION>.jar to the actual jar location, e.g.:
cp <CM script dir>/tools/spl-tools-4.1.0.jar <CM script dir>/lib/
This utility written in perl extracts source code from an app server, typically the project repository, or from a release or patch install package.
Note. extractCMSource.plx is a Windows-only utility.
Contents
Extract From Release/Patch Install Package
FW Utility to extract CM sources from Unix environments
To display the usage information, just execute the utility without any parameters from the Windows command prompt.
perl extractCMSource.plx
extractCMSource.plx -s sourceDirectory -d destinationDirectory -v Version [ -n subDirectoryName ] -v Version The Version Number to attach to this release of the Customer Modifications. -s sourceDirectory sourceDirectory is the location to extract from. -d destinationDirectory destinationDirectory is where the extracted files will be placed. This program will then create a subdirectory under that directory for the patch. -n subDirectoryName Subdirectory under "Directory" to put this patch in. If this parameter is not provided an automatic directory name will be generated based on the environment name and date time.
|
To extract the source from a development app server, specify the app server directory as the source directory. For example, the following invocation extracts source from an app server named CM_PROJ1 in the C:\SPL directory into C:\CMExtarct and marks the extract as version CM1.0:
perl extractCMSource.plx -s C:\SPL\CM_PROJ1 -d C:\CMExtract -v CM1.0
|
To extract the source from a release or patch install package, specify the data directory in the install package as the source directory. For example, the following invocation extracts source from a patch install package named CM1.0_1 in the C:\CMInstall directory into C:\CMExtract and marks the extract as version CM1.0_1:
perl extractCMSource.plx -s C:\CMInstall\CM1.0_1\CMCCB\data -d C:\CMExtract -v CM1.0_1
|
The following utility, delivered with the Oracle Utility Application Framework, is to extract CM sources from a Unix environment (note that extractCMSource.plx is the Window only utility):
$SPLEBASE/bin/extractCMSourceUNIX.sh
Usage:
-v <version>
The Version Number to attach to this release of the Customer Modifications. (Example: CM001)
-t <target directory>
Target Direcory is where the extracted files will be placed. This program will then create a subdirectory under that directory for the patch with a timestamp. with a timestamp.
-e
Include CM Application Viewer and Help files (optional).
-l
Language Code when extracting CM Help files (optional).
After an Implementation team has completed CM development on a Windows server or prepared a fix in a development environment, they’ll need to copy and apply the CM modules to a packaging environment on the same platform as the target (e.g. production, testing). In other words, if the target system is a Unix platform, the packaging environment must be on Unix as well.
The applyCM.sh utility (applyCM.cmd for Windows installations) serves this need. It can be used to copy and apply all CM development modules to a packaging environment or any specific extract (patch) of CM development. The script needs to be executed using the full pathname (this is necessary because you need to be located in a different folder, see below). In addition, you need to be set to a target environment (e.g. packaging environment).
Script: <CM scripts>/applyCM.sh
Usage:
(no options)
Apply patch on top of the existing base product and possibly CM integration environment. This mode will add new CM files from patch to the environment and replace the changed ones. But it will not delete the previously existing in the environment CM modules that are not part of the patch.
It needs to be executed from the source root folder.
-d
Remove all previous CM modules from the environment and apply patch on clean base product environment. This option is useful when needed to create the CM integration environment from scratch.
It needs to be executed from the source root folder.
-b
Recompile the existing sources in current environemnt. Usually it is used to execute full recompile a development environment.
It needs to be executed from the application folder root, e.g. $SPLEBASE.
-n
It won’t stop/start automatically the target environment.
The input for applyCM.sh utility is current folder (source root folder), which contains the following subfolders:
· java
· scripts
· etc
· services
· splapp
These subdirectories contain only CM modules created according to the rules of the document “Naming conventions for tailoring application implemetation” (see Installation Guide of the product). This directory structure should be prepared and filled with relevant CM modules on development Windows server, then copied over to the server that hosts a packaging environment by ftp utility. After that you can apply the patch to the packaging environment. Modules that are not created using these conventions will be ignored by applyCM.sh utility. You have to reside in the patch directory to apply the patch. ApplyCM utility will generate and compile java code, will create java jar file (cm.jar) required for customer implementation platform.
The create_CM_Release.sh utility is used to create a CM full release package that will contain only customer modification (CM) files. This is used to install a full set of customizations on top of the base product environment.
In order to build a CM release version that is compatible with the target platform, you need a packaging environment on the same operating system as the one on which the receiving product is installed. The target environment for installing the release version on a customer site can either be a pure base product environment, or an environment that already contains previous CM versions. In the second case, all previous CM modules will be removed by the install utility at the beginning of the installation process.
It is mandatory that every implementation version is identified by its own release version number. This number may be in any free standard and must be recorded in the $SPLEBASE/etc/CMVERSION.txt file on the environment.
Here are the detailed instructions for creating the full release version for CMs:
· Log in to the server with the administrator user id and initialize a packaging environment. You will use this environment to create the CM release version.
· Change the directory to the directory that contains the Developers Tool Suite utilities (CM_packaging).
· Execute the utility using the following command:
./create_CM_Release.sh -e $SPLENVIRON -v $VERNO -d $RELEASEHOME,
where
$SPLENVIRON is the target packaging environment
$VERNO is the CM version number (the content of the file CMVERSION.txt)
$RELEASEHOME is the name of the directory on the server where you want to place the resulting CM release package.
For example:
./create_CM_Release.sh -e M4_Q1_SUNDB2 -v M.4.0.0 –d /versions
Tar and zip the resulting CM release directory for Unix platform or zip it for Windows platform and ship it to your customer.
The customer who wishes to install the delivered package onsite will follow the instructions:
· Decompress and untar the installation media to a temporary directory for the Unix server or unzip it for Windows server.
· Change directory to the target directory.
· Login and initialize the target environment.
· Change to the Installation directory using the following command
cd CMCCB.$VERNO
where
$VERNO is the version number (the content of the file CMVERSION.txt)
· Run the following script
./install.sh - for Unix
install.cmd - for Windows
The utility create_CM_Patch.sh is used to create a patch release of CMs. A patch release version is created as a difference between a previous CM version and a new CM version. This type of release may be useful if the implementation team wants to ship only an update of the previously released version by preparing a smaller package that can be delivered easily by email or ftp to the customer.
Important: Before executing the utility, be sure that both packages are available in the same directory on the server. During the installation process at the customer site, the patch install utility will not remove the previous version of CM modules, and will only install the patch content on top of the previous CM version.
Here is the process for creating a patch release CM version:
· Log in to the server with the administrator user id.
· Change directory to the name of the directory that contains the SDK packaging utilities (CM_packaging).
· Execute the utility by entering the following command:
./create_CM_Patch.sh -d $RELEASEHOME
where:
$RELEASEHOME is the directory that currently holds your CM full release packages - and where you also want to put your new patch package.
Tar and zip the resulting CM patch directory for Unix platform or zip it for Windows platform and ship it to your customer. The customer who wishes to install the delivered package onsite will follow the instructions:
· Decompress and untar the installation media (on Unix) or unzip (on Windows) to a temporary directory.
· Change directory to that directory.
· Login and initialize the target environment.
· Change to the Installation directory by using the following command
cd CMCCB.$VERNO
where
$VERNO is the version number (the content of the file CMVERSION.txt)
· Run the following script
./install.sh - on Unix
install.cmd - on Windows
Contents
CM System Data Packaging Overview
CM System Data Packaging Tools allow the implementers to extract and package ‘CM’ system data from their databases and deliver it to the customers.
Oracle and MS SQL Server are both supported. The example below utilizes the oracle database platform to illustrate the extract and upload process. MS SQL Server requires a different ODBC connection for the extract and upload processes. A description of the ODBC setup is in the extract and upload bat files provided as samples.
As a standard release process, the implementers can add the CM system and Meta data records to the base product database or change base product system and Meta data records according to the specific rules (see “System Table Guide” document in the Installation Guide of the product). Implementers can then choose to migrate the CM data to a customer database as a full extract of CM data, or incremental differences between the current version of the system data on the customer site and the new version of the implementation development database.
Packaging CM system data starts by creating a copy of the project dev database into a project release database. A blueprint file of the system data is then created by executing the OraSDBP tool.
To apply the latest changes to a QA or production database, execute OraSDUp with the blueprint file as input and then specifying the target database.
Extract process involves extracting CM system data based on the rules defined in a parameter file and packaging it in a binary file – blueprint. This file can then be used as an input source by the data upload process.
Oracle and MS SQL Server are both supported. The example below utilizes the oracle database platform to illustrate the extract and upload process. MS SQL Server requires 2 different ODBC connections for the extract process and upload process. A description of the ODBC setup is in the extract and upload bat files provided as samples.
OraSdBp.exe, included in this package, is the extract utility that reads an input parameter file for the list of Oracle database tables, extracts data from these tables and compresses into a binary file (blueprint).
A sample extract parameter file extract_cmsystbls.lst is included in this package to provide the implementers a starting point. This parameter file, as can be seen, defines rules for the tool to extract CM data based on their key definition. However, in some cases, CM data may be stored on ‘CI’ rows. The column user_exit_pgm on CI_MD_PRG_COM table is one such case. For cases like these, the implementers can choose to change the extraction rules in this file to match their requirements.
To extract your data, make a copy of extract_cmsystbls.lst file and edit it to match your requirements. Execute the extract process from a Command-window and provide it with the required parameter when prompted.
The data in input parameter must match the following format:
CI_ALG_TYPE_L;LANGUAGE_CD = 'ENG';VERSION
Where, the first field stores the table name, second field stores the selection criteria (where clause for selecting data) and the third field stores the list of column that should be ignored during extraction. The character semi-colon is used as the field separator. If there are multiple columns that need to be ignored (not included in the data being extracted), comma can be used in the third field as the separator.
OraSdBp accepts the following parameters:
· -d Connect String
Where the Connect String contains:
· Schema owner name (say CISADM)
· Password for schema owner.
· Database name.
This is a mandatory parameter. If not entered, the utility will prompt the user to build the connect string.
Connect String should be entered in the following format:
CISADM,CISADMPSWD,DBNAME
(Comma-separated and no space).
· -i Input Parameter file name.
Name of the input parameter file that the utility reads to get the list of tables and their selection criteria. This parameter is optional. The default input parameter file name is CDXSdBp.Inp.
· -o Output File Name.
This is the name of the binary file that the utility creates. This parameter is optional. The default output file name is “OraSdBp” (without extension).
· -c NLS characterset of the target database
The utility uses this parameter to set the NLS_LANG parameter on the client side. This parameter is then validated against the character set of the source database and is saved in the blueprint. This is mandatory parameter and is prompted for if not set by the user.
· -h Help.
This option will list all the accepted parameters with a brief description.
Data upload process compares the data included in the blueprint file (generated by the extract process) and that extracted from a target database and generates output SQL to synchronize them.
The utility OraSdUpg, included in this package, is used by the process utility to compare and synchronize the data in the target database with that in the input blueprint file.
OraSdUpg reads an input parameter file for the list of the tables to be upgraded along with the selection criteria and upgrade rules for each table.
Each table has a corresponding record in the file with following 6 fields separated by semi-colon:
· Table Name
· The instance of the table. This number should be always set to 1. The cases where more than one instances of a table are processed are extremely rare and are not discussed here.
· Selection Criteria for the table.
· Insert allowed indicator (T/F): Whether records should be inserted into the target database table if they missing in the database but exist in the binary file.
· Update allowed indicator (T/F): Whether records should be updated in the table if they have different values than in the binary file.
· Delete allowed indicator (T/F): Whether the obsolete data in table in the target database. Obsolete records exist in target database but not in the binary file.
· Fresh Install Indicator (T/F): Whether the table should be seeded during the very first install. This indicator is only used when the utility is invoked with “-f” switch.
· List of columns not updated can be specified in the sixth field. Use a comma to separate the column names if multiple columns are to be ignored during updates. These columns will be inserted but will not be updated during the data synchronization process.
Following is the example of how these records should look like in the file:
CI_LOOKUP;1;LANGUAGE_CD = 'ENG';T;T;F;T;DESCR
A sample file upload_cmsystbls.lst has been included in this package. Implementer can make a copy of this file and edit it to match their requirements.
Before making connection to the target database, the utility reads the header from the blueprint and sets NLS_LANG environment variable on the client machine. It then validates this character set setting to the character set of the target db after making a connection and warns user if there is a mismatch.
The utility can be executed in verification and modification modes. In verification modes, the action SQL statements are simply written to the log file but in modification mode they are applied the target database.
It is very important to note that the primary requirement for OraSdUpg is definition (column and primary key) of tables being upgrade in the target database should be same as that in the database from which the binary file was extracted.
Be careful while selecting the table and the selection criteria because to compare the data, the utility, for each table, first loads the data from the binary file and the database in the memory. If a table has huge amount of data and selection criteria set causes the utility to work on large quantity of data, it may run out of memory.
To avert unique key constraint violation error that can be caused by improper sequence of data deletion and insertion on a table and also the foreign key issues, the utility first gathers all the generated action statements for all the tables before executing them. The execution of all the generated statements is done in multiple iterations. After each iteration, all the failed statements during that iteration are collected and executed again in the next iteration. The iterations are repeated till either all the statements in iteration are executed successfully or they fail.
The utility disables and enables all the triggers on the tables being upgraded before and after applying database changes. No triggers get executed during the system data upgrade.
OraSdUpg accepts the following parameters:
· -d Connect String
Where the Connect String contains:
· Schema owner name (say CISADM)
· Password for schema owner.
· Database name.
This is a mandatory parameter. If not entered, the utility will prompt the user to build the connect string.
Connect String should be entered in the following format:
CISADM,CISADMPSWD,DBNAME
(Comma-separated and no space).
· -b Bypass the database character set validation.
Before upgrading data in database, the utility validates character set stored in the blueprint by OraSDBp against that of target database. The user can bypass this validation step by setting this switch.
· -p Input Parameter file name.
Name of the input parameter file that the utility reads to get the list of tables and their selection criteria. This parameter is mandatory.
· -i Input Binary File.
This is the name of the binary file that the utility reads to extract the data that it then uses to upgrade the target database. This is a mandatory parameter.
· -f
Treats the data synchronize process as New install. When set, the flag forces OraSdUpg to use “fresh install indicator” for the tables where INSERT indicator is set to false and compels it to insert missing records in all of them. Optional.
· -u
Makes OraSdUpg run in the modification mode. Optional.
· -l Log File Name.
This is the name of the file that OraSdUpg creates, if it is missing and starts appending the information about the action it is performing.
· -h Help.
This option will list all the accepted parameters with a brief description.
Note. It is recommend that the implementers execute the upload process first in the verification mode and review the SQL before running the tool in the modification mode.