Oracle Object Storage

Introduction

Oracle Object Storage is a highly secure, better performant, durable storage platform. It allows you to store or retrieve unlimited data anytime, safely and securely using its web-based console within the cloud platform.

Oracle Data Integrator (ODI) seamlessly integrates with Oracle Object Storage. With this integration, you can now connect to Oracle Object Storage from ODI for uploading, downloading and deleting files/objects onto/from local directory or HDFS.

Concepts

Oracle Object Storage comprises of the following components:

Objects: All data, regardless of content type, is stored as an object in Oracle Object Storage.
Buckets: A bucket is a container that stores objects. You can store objects in one or multiple buckets under a single tenancy.
Namespace: A namespace is the logical construct that lets you own your personal bucket namespace. Since bucket names are not global you have more freedom and flexibility with how you assign names to buckets. Each Object Storage Cloud tenant is associated with one default system assigned namespace.
Compartments: A compartment is a collection of related resources that can be accessed only by those groups that have been given permission by an administrator to access those resources. Compartment is not necessarily a concept that is exclusively associated with the Oracle Object Storage, but it's pertinent because all buckets exist in a compartment.

Below image helps you to understand how the Oracle Object Storage components fit together. If a user has been granted access to Compartment A, they will only be able to access Bucket1 or Bucket2 and objects that are stored in these buckets.

Figure 30-1 Object Storage Components

Description of "Figure 30-1 Object Storage Components"

Refer to https://docs.us-phoenix-1.oraclecloud.com/Content/Object/Concepts/objectstorageoverview.htm , for more details on Oracle Object Storage.

Installation and Configuration

Make sure you have read the information in this section before you start working with the Oracle Object Storage technology:

System Requirements & Certifications

Before performing any installation, you should read the system requirements and certification documentation to ensure that your environment meets the minimum installation requirements for the products you are installing.

The list of supported platforms and versions is available on Oracle Technical Network (OTN): http://www.oracle.com/technetwork/middleware/data-integrator/documentation/index.html

Technology Specific Requirements

The technology specific requirement for using Oracle Object Storage in ODI are:

A new, dedicated pre-built technology called “Oracle Object Storage”.
Create a new Data Server from this technology and then create corresponding Physical and Logical schema.

Data Type mappings to and from some major platforms are defined for these data types. Supported data types for this technology are:
- Array
- Boolean
- Bytes
- Complex
- Date
- Double
- Enum
- Fixed
- Float
- Integer
- Long
- Map
- Number
- String
- Struct
- Union

Setting up the Topology

Setting up the topology consists of:

Creating an Oracle Object Storage Data Server

Create a data server for the Oracle Object Storage technology using the standard procedure, as described in Creating a Data Server in Administering Oracle Data Integrator. This section details only the fields required or specific for defining Oracle Object Storage data server:

In the Definition tab:

The Data Server page contains the following fields and they are grouped into different categories such as –
- Data Server
  - Name – Name of the data server that will appear in Oracle Data Integrator.
- General
  - Region – Oracle Object Storage region. A region is a localized geographic area, and an availability domain is one or more data centers located within a region. A region is composed of several availability domains. Most Oracle Cloud Infrastructure resources are either region-specific, such as a virtual cloud network, or availability domain-specific, such as a compute instance.
  - Tenant OCID – Tenant’s Oracle Cloud ID. Every Oracle Cloud Infrastructure resource has an Oracle-assigned unique ID called an Oracle Cloud Identifier (OCID). It's included as part of the resource's information in both the Console and API. To find your tenancy's OCID, navigate to Console -> Menu -> Administration -> Tenancy Details -> Tenancy Information -> OCID.
    
    For example, ocid1.tenancy.oc1..aaaaaaaauwjnv47knr7uuuvqar5bs hnspi6xoxsfebh3vy72fi4swgrkvuvq
  - User OCID – Oracle Cloud ID of the user logging into Oracle Object Storage.
    
    In the Console on the page showing the user's details. To get to that page:
    - If you're signed in as the user, click the user icon present in the top-right corner of the Console, and then click User Settings.
    - If you're an administrator doing this for another user, instead click Identity, click Users, and then select the user from the list.
- Security
  - Private Key File – Click the browse button to choose the location of the private key file (in PEM format)
  - Passphrase – Passphrase is the password used while generating the private key
    
    You can use the following OpenSSL commands to generate the key pair in the required PEM format. If you're using Windows, you'll need to install Git Bash for Windows and run the commands with that tool.
    1. If you haven't already, create a .oci directory to store the credentials:
```
mkdir ~/.oci
```
    2. Generate the private key with one of the following commands:
      - To generate the key, encrypted with a passphrase you provide when prompted:
        openssl genrsa -out ~/.oci/oci_api_key.pem -aes128 2048
      - For Windows, you may need to insert -passout stdin to be prompted for a passphrase. The prompt will just be the blinking cursor, with no text.
        openssl genrsa -out ~/.oci/oci_api_key.pem -aes128 2048
      - To generate the key with no pass-phrase:
        openssl genrsa -out ~/.oci/oci_api_key.pem 2048
  - Fingerprint – Fingerprint that is generated for the public key
    You can get the key's fingerprint with the following OpenSSL command. If you're using Windows, you'll need to install Git Bash for Windows and run the command with that tool.
```
openssl rsa -pubout -outform DER -in ~/.oci/oci_api_key.pem | openssl md5 -c
```
    When you upload the public key in the Console, the fingerprint is also automatically displayed there. For example, 12:34:56:78:90:ab:cd:ef:12:34:56:78:90:ab:cd:ef
- Swift Connectivity
  - Tenant Name – Name of the tenant
  - User Name – Name of the user logging into Oracle Object Storage
  - Swift Password – Password generated for swift connectivity.
    
    These Swift Connectivity parameters are used by the Autonomous Data Warehouse Cloud Service to connect to Oracle Object Storage technology.

Creating an Oracle Object Storage Physical Schema

Create an Oracle Object Storage physical schema using the standard procedure, as described in Creating a Physical Schema in Administering Oracle Data Integrator.

Oracle Object Storage specific parameters are:

Name: Name of the physical schema created
Bucket (Schema): It specifies the Oracle Object Storage Bucket name from which upload, download or the delete operation will happen. Select the required bucket from the Bucket Name drop-down list.
Directory (Work Schema): This is the temporary folder on the local system used for getting files from Oracle Object Storage bucket during reverse engineering. If the directory does not exist it will be created. Specify the required location in the local system.

Create a logical schema for this physical schema using the standard procedure, as described in Creating a Logical Schema in Administering Oracle Data Integrator and associate it with a relevant context. We use the created logical schema for getting Oracle Object Storage instance details. These details are used for connecting to Oracle Object Storage technology. It is also used to get the bucket details associated with the physical schema which in turn is associated to this logical schema.

Creating and Reverse-Engineering an Oracle Object Storage Model

This section contains the following topics:

Creating an Oracle Object Storage Model

An Oracle Object Storage model is a set of data stores, corresponding to files stored in an Oracle Object Storage bucket. In a given context, the logical schema corresponds to one physical schema. You can create a model from the logical schema for the Oracle Object Storage technology. The bucket schema of this physical schema is the Oracle Object Storage bucket containing all the files. You can create new ODI Data store that will represent a file in Oracle Object Storage so that it can be used in mappings.

Create an Oracle Object Storage model using the standard procedure, as described in Creating a Model of Developing Integration Projects with Oracle Data Integrator.

Reverse Engineering an Oracle Object Storage Model

Oracle Data Integrator provides specific methods for reverse-engineering Oracle Object Storage files. Oracle Object Storage supports the following types of reverse engineering:

Reverse-Engineering Delimited Files from Oracle Object Storage

To perform a delimited file reverse engineering:

In the Models accordion, right click your Object Storage Model and select New Data store. The Data Store Editor opens.
In the Definition tab, enter the following fields:
- Name: Name of this data store
- Resource Name: Click the Search icon, to select the required file from the list of files present in Oracle Object Storage for the configured bucket.
Go to the Storage tab, to describe the type of file. Set the fields as follows:
- File Format: Delimited
- Heading (Number of Lines): Enter the number of lines of the header. Note that if there is a header, Oracle Data Integrator uses the first line of the header to name the columns in the file.
- Select a Record Separator.
- Select or enter the character used as a Field Separator.
- Enter a Text Delimiter if your file uses one.
- Enter a Decimal Separator, if your file contains decimals.
From the File main menu, select Save.
In the Data Store Editor, go to the Attributes tab.
In the editor toolbar, click Reverse Engineer.
Verify the data type and length for the reverse engineered attributes. Oracle Data Integrator infers the field data types and lengths from the file content, but may set default values (for example 50 for the strings field length) or incorrect data types in this process.
From the File main menu, select Save.

Reverse-engineering Fixed Files from Oracle Object Storage

Oracle Data Integrator provides a graphic wizard to define the columns of a fixed file.

To reverse-engineer a fixed file from Oracle Object Storage using the wizard:

In the Models accordion, right click your Object Storage Model and select New Data store. The Data store Editor opens.
In the Definition Tab, enter the following fields:
- Name: Name of this data store
- Resource Name: Sub-directory (if needed) and name of the file. It lists all the files present in Oracle Object Storage for the configured bucket.
Go to the Storage tab to describe the type of file. Set the fields as follows:
- File Format: Fixed
- Header (Number of Lines): Enter the number of lines of the header.
- Select a Record Separator.
From the File main menu, select Save.
In the Data store Editor, go to the Attributes tab.
In the editor toolbar, click Reverse Engineer. The Attributes Setup Wizard appears. The Attributes Setup Wizard displays the first records of your file.
Click on the ruler (above the file contents) to create markers delimiting the attributes. You can right-click within the ruler to delete a marker.
Attributes are created with pre-generated names (C1, C2, and so on). You can edit the attribute name by clicking in the attribute header line (below the ruler).
In the properties panel (on the right), you can edit all the parameters of the selected attribute. You should set at least the Attribute Name, Data type, and Length for each attribute.
Click OK, when the attributes definition is complete.
From the File main menu, select Save.

Reverse-Engineering JSON, Avro and Parquet Storage Formats

Oracle Object Storage technology supports reverse engineering JSON, Avro and Parquet storage formats , attributes, data types, and data type properties. If one of these storage format types is selected, then reverse engineering is based on the Schema File specified, and not on a sample data file. The schema file should be accessible in local file system.

To reverse-engineer JSON, Avro and Parquet storage formats, perform the following steps:

In the Models accordion, right click your Object Storage Model and select New Data store. The Data Store Editor opens.
In the Definition tab, enter the following fields:
- Name: Name of this data store
- Resource Name: Click the Search icon, to select the required file from the list of files present in Oracle Object Storage for the configured bucket.
From the Storage Tab, select the Storage Format from the Storage Format drop-down list and specify the complete path of the schema file in the Schema File field.

The schema file should be located in the local file system.
From the File main menu, select Save.

Note:

There is no need to import an RKM into the project.

Working with Oracle Object Storage Tools

You can upload, download and delete files to/ from Oracle Object Storage through Oracle Data Integrator. The Object Storage tools that are helpful to perform the following operations are:

Note:

Apart from ODI Studio, you can also work with ODI Object Storage Tools from command line.

Uploading Files/Objects to Oracle Object Storage

ODI Object Storage Upload tool is used to upload single, multiple files, or an entire directory from HDFS or a local file system on to Oracle Object Storage.

Note:

The upload operation fails if the selected bucket does not exist.

To upload file(s) or directories to Object Storage,

Create a new Project.

For more details on how to create a project, see Creating an Integration Project of Developing Integration Projects with Oracle Data Integrator.
Below the created Project folder, create a Package .

For more details on how to create a package, see Creating and Using Packages of Developing Integration Projects with Oracle Data Integrator.
Select OdiObjectStorageUpload tool available in the Toolbox. Move it to the created package.

Note:
All the properties of the tool are displayed under General Tab.

Configure the required properties:

Table 30-1 ODI Object Storage Upload Tool Properties

Parameter	Description
`Target Logical schema`	Target Logical schema name configured for Oracle Object Storage Data Server.
`Source Logical schema`	Name of the source Logical schema configured for File or HDFS Data Server for upload of Local or HDFS Files to Oracle Object Storage.
`File Names filter`	Field to specify one or more files to be uploaded to Oracle Object Storage recursively. It also supports the list of files separated by \| as a delimiter. The pattern followed is: .txt - should upload all the files ending with .txt test - uploads all the files and directories that matches with prefix “test” test - uploads all the files and directories having substring “test” test.xml \| test1.xml \| test2.xml - Uploads all the files specified test* \| test1* - Uploads all the files matching pattern test* and test1* test.xml - Only one file is uploaded
`Overwrite`	This parameter indicates if upload operation should overwrite an existing file or not. Default value for this parameter is No.
`Retry on error`	It represents the number of times the retry attempt should occur when a failure or error happens during upload.
`Retry interval seconds`	Retry interval indicates after how many seconds a retry attempt should happen.

For more details on the usage of the above parameters, refer to ODI Object Storage Upload tool in Oracle Data Integrator Tools Reference.

Save and execute the package.

The required files from the source directory are uploaded to the target location of Oracle Object storage.
Upon successful upload, you can find a detailed log of this upload operation at the Details tab. To get to the details tab, from the Operator tab, expand the associated session for the upload tool and open the Session task window to find the Details tab with the required log information.

The details include:
- Source directory is : <source directory path>
- Target bucket is : < Object storage bucket name>
- Filter used is : <input filter>
- Number of files uploaded:<Total number of files that were uploaded>
- Uploaded files are:<File1, File2>
- Number of files failed:<Total number of files that were not uploaded>
- Failed files are: <File1, File2>

Downloading Files/Objects from Oracle Object Storage

ODI Object Storage Download tool is used to download single, multiple files, or an entire directory to HDFS or a local file system from Oracle Object Storage.

To download file(s) or directories from Object Storage,

Create a new Project

For more details on how to create a project, see Creating an Integration Project of Developing Integration Projects with Oracle Data Integrator.
Below the created Project folder, create a Package

For more details on how to create a package, see Creating and Using Packages of Developing Integration Projects with Oracle Data Integrator.
Select OdiObjectStorageDownload tool available in the Toolbox. Move it to the created package.

Note:
All the properties of the tool are displayed under General Tab.

Configure the required properties:

Table 30-2 ODI Object Storage Download Tool Properties

Parameter	Description
`Source Logical schema`	Name of the Source Logical schema configured for Oracle Object Storage Data Server.
`Target Logical schema`	Generally, the download operation downloads the file from Oracle Object Storage to Local or HDFS file system. The Target logical schema specifies whether the files are downloaded to Local or HDFS file system.
`File Names filter`	Field to specify one or more files to be downloaded from Oracle Object Storage recursively. It also supports delimiter \| for separated files list. The pattern followed is: .txt - should download all files ending with .txt test - Downloads all the files and directories that matches with prefix “test” test - Downloads all the files and directories having substring “test” test.xml \| test1.xml \| test2.xml - Downloads all the files specified test* \| test1* - Downloads all the files matching pattern test* and test1* test.xml - Only one file is downloaded
`Overwrite`	This parameter indicates, if download operation should overwrite an existing file or not. The default value for this parameter is No.
`Retry on error`	It represents the number of times the retry attempt should occur when a failure or error happens during download.
`Retry interval seconds`	Retry interval indicates after how many seconds a retry attempt should happen.

For more details on the usage of the above parameters, refer to ODI Object Storage Download tool of Oracle Data Integrator Tools Reference.

Save and execute the package

The required files from Oracle Object storage are downloaded to the configured target location.
Upon successful Download, you can find a detailed log of this download operation at the Details tab. To get to the details tab, from the Operator tab, expand the associated session for the download tool and open the Session task window to find the Details tab with the required log information.
- Source bucket is : <Object storage bucket name>
- Target directory is : < target directory path>
- Filter used is : <input filter>
- Number of files downloaded:<Total number of files that were downloaded>
- Downloaded files are:<File1, File2>
- Number of files failed:<Total number of files that were not downloaded>
- Failed files are: <File1, File2>

Deleting Files/Objects from Oracle Object Storage

ODI Object Storage Delete tool is used to delete single, multiple files, or an entire directory present in Oracle Object Storage.

To delete file(s) or directories from Object Storage,

Create a new Project

For more details on how to create a project, see Creating an Integration Project of Developing Integration Projects with Oracle Data Integrator.
Below the created Project folder, create a Package

For more details on how to create a package, see Creating and Using Packages of Developing Integration Projects with Oracle Data Integrator.
Select OdiObjectStorageDelete tool available in the Toolbox. Move it to the created package.

Note:
All the properties of the tool are displayed under General Tab.

Configure the required properties:

Table 30-3 ODI Object Storage Delete Tool Properties

Parameter	Description
`Target Logical schema`	Target logical schema has the details of Oracle Object Storage Data Server which contains the files and directories that are to be deleted.
`File Names filter`	Field to specify one or more files or directories to be deleted from Oracle Object Storage recursively. It also supports delimiter \| for separated files list. The pattern followed is: .txt - should delete all files ending with .txt test - Deletes all the files and directories that matches with prefix “test” test - Deletes all the files and directories having substring “test” test.xml \| test1.xml \| test2.xml - Deletes all the files specified test* \| test1* - Deletes all the files and directories matching pattern test* and test1* test.xml - Only one file is deleted.
`Retry on error`	It represents the number of times the retry attempt should occur when a failure or error happens during delete.
`Retry interval seconds`	Retry interval indicates after how many seconds a retry attempt should happen.

For more details on the usage of above parameters, refer to ODI Object Storage Delete Tool of Oracle Data Integrator Tools Reference.

Save and execute the package.

The selected files are deleted from Oracle Object Storage.
Upon successful Deletion, you can find a detailed log of this delete operation at the Details tab. To get to the details tab, from the Operator tab, expand the associated session for the delete tool and open the Session task window to find the Details tab with the required log information.
- Target bucket is : <Object storage bucket name>
- Filter used is : <input filter>
- Number of files deleted:<Total number of files that were deleted>
- Deleted files are:<File1, File2>
- Number of files failed:<Total number of files that were not deleted>
- Failed files are: <File1, File2>

Designing a Mapping

You can use a file/HDFS or SQL technology such as Oracle as a source of a mapping and its target as Oracle Object Storage technology.

Oracle Object Storage physical schema is represented by Object Storage Bucket. Some properties such as user name/password are retrieved from Oracle Object Storage data server. If temporary local files are created (if needed), its location can be defined through the option TEMP_SCHEMA KM. These temporary Object Storage files can be removed through new ODI cleanup tools. If you use transform components, they need to be moved to the source execution unit in case of SQL as a source. File as a source does not support source transformations and no transformations are supported for the target as well.

The KM choice for a mapping or a check determines the abilities and performances of this mapping or check. The recommendations listed here help in the selection of the KM for different situations concerning an Oracle Object Storage data server.

Setting up an Integration Project

Setting up a project using the Oracle Object Storage technology follows the standard procedure. See Creating an Integration Project of the Developing Integration Projects with Oracle Data Integrator.

You can use the following knowledge modules for loading data into Oracle Object Storage:

LKM File to Oracle Object Storage

This LKM helps to upload data from local or HDFS File to Oracle Object Storage.The name of the file as well as its structure remains the same. No transformation on the data can be performed. This LKM is used in a staging area physical node.

This KM invokes the ODI tool OdiObjectStorageUpload, for uploading file(s) to Oracle Object Storage.

For Example — OdiObjectStorageUpload "-TRG_LOGICAL_SCHEMA=Object Storage - SRC1" "-SRC_LOGICAL_SCHEMA=FILE_GENERIC_TMP" "-FILE_NAMES_FILTER=person_upload.csv" "-OVERWRITE=true"

This KM has the following option:

CLEANUP_TEMPORARY_OBJECTS— It cleans-up temporary objects.

Set this property to True, if you wish to automatically clean-up the temporary objects created.

LKM File to Oracle Object Storage Direct

This LKM helps to upload local or HDFS file(s) onto Oracle Object Storage target directly. The name of the file as well as its structure remains the same. No transformation on the data can be performed.

This KM invokes the ODI tool OdiObjectStorageUpload, for uploading file(s) to Oracle Object Storage.

For Example — OdiObjectStorageUpload "-TRG_LOGICAL_SCHEMA=Object Storage - SRC1" "-SRC_LOGICAL_SCHEMA=FILE_GENERIC_TMP" "-FILE_NAMES_FILTER=person_upload.csv" "-OVERWRITE=true"

This KM has the following options:

CLEANUP_TEMPORARY_OBJECTS— It is used to clean-up temporary objects created. Set this property to True, if you wish to automatically clean-up the temporary objects created.
OVERWRITE_TARGET_FILE — It is used to overwrite the target file. If set to True, the Oracle Object Storage target files are overwritten.
ADD_COMPRESSION— It is used to compress data before loading. Set this property to True, if you wish to compress source data before loading it onto Oracle Object Storage. Use the additional options COMPRESSION_TYPE and KEEP_SOURCE_FILES to define compression preferences.
COMPRESSION_TYPE— It is used to define the compression type. Select the type of compression you wish to apply on source data before loading it onto Oracle Object Storage.
KEEP_SOURCE_FILES— It is used to retain the original source files after compression. Set this property to True (by default), if you wish to compress the target files and retain the original files.

Note:
gzip supports KEEP_SOURCE_FILES option, starting from version 1.6 only.

LKM SQL to Oracle Object Storage

This LKM helps to upload the result of a SQL query to Oracle Object Storage. Only transformation on source are supported.SQL data is unloaded to a temporary local file, which is then uploaded to Object Storage. The temporary file location is specified through the TEMP_SCHEMA KMoption. This LKM is used in a staging area physical node.

This KM invokes the ODI tools OdiSqlUnload to unload SQL query data to a file and OdiObjectStorageUpload for uploading the file(s) onto Oracle Object Storage.

Listed below are examples for these tools:

OdiSqlUnload

OdiSqlUnload "-FILE=/tmp/person_upload.csv" "-DRIVER=oracle.jdbc.OracleDriver" "-URL=jdbc:oracle:thin:@//slc03sap:1521/flex" "-USER=system" "-PASS=<@=odiRef.getInfo("SRC_ENCODED_PASS") @>" "-FILE_FORMAT=VARIABLE" "-FIELD_SEP=," "-ROW_SEP=

" "-DATE_FORMAT=yyyy/MM/dd HH:mm:ss" "-CHARSET_ENCODING=ISO8859_1" "-FETCH_SIZE=2"

SELECT
  PER.PID AS PID ,
  PER.PNAME AS PNAME   
FROM
  UT_TD_D_1.PERSON PER  
WHERE
  (PER.PID = 2)

OdiObjectStorageUpload

OdiObjectStorageUpload "-TRG_LOGICAL_SCHEMA=Object Storage - SRC1" "-SRC_LOGICAL_SCHEMA=FILE_GENERIC_TMP" "-FILE_NAMES_FILTER=person_upload.csv" "-OVERWRITE=true"

This KM has the following options:

CLEANUP_TEMPORARY_OBJECTS— It is used to clean-up temporary objects created. Set this property to True, if you wish to automatically clean-up the temporary objects created.
TEMP_SCHEMA — It is used to specify the name of logical schema defining the location the temporary file that will be stored before uploading the data onto Oracle Object Storage. This is a File technology logical schema. The temporary file is stored in a local file system where ODI agent is running.
DATE_FORMAT — It specifies the output format used for date data types.
CHARSET_ENCODING — It specifies the character set encoding.
FETCH_SIZE — It specifies the number of rows (records read) requested by ODI agent on each communication with the data server.

LKM SQL to Oracle Object Storage Direct

This LKM helps to upload the result of a SQL query to Oracle Object Storage. Only transformation on source are supported. SQL data is unloaded to a temporary local file, which is then uploaded to Oracle Object Storage. The temporary file location is specified through the TEMP_SCHEMAKM option. Data can be uploaded in Delimited, Fixed or XML formats. This LKM is a transparent target KM (loads data directly into target). It has to be assigned to target execution unit AP node.

This KM invokes the ODI tools OdiSqlUnload to unload SQL query data to a file and OdiObjectStorageUpload for uploading the file(s).

Listed below are examples for these tools:

OdiSqlUnload

OdiSqlUnload "-FILE=/tmp/person_upload.csv" "-DRIVER=oracle.jdbc.OracleDriver" "-URL=jdbc:oracle:thin:@//slc03sap:1521/flex" "-USER=system" "-PASS=<@=odiRef.getInfo("SRC_ENCODED_PASS") @>" "-FILE_FORMAT=VARIABLE" "-FIELD_SEP=," "-ROW_SEP=

" "-DATE_FORMAT=yyyy/MM/dd HH:mm:ss" "-CHARSET_ENCODING=ISO8859_1" "-FETCH_SIZE=2"

SELECT
  PER.PID AS PID ,
  PER.PNAME AS PNAME   
FROM
  UT_TD_D_1.PERSON PER  
WHERE
  (PER.PID = 2)

OdiObjectStorageUpload

OdiObjectStorageUpload "-TRG_LOGICAL_SCHEMA=Object Storage - SRC1" "-SRC_LOGICAL_SCHEMA=FILE_GENERIC_TMP" "-FILE_NAMES_FILTER=person_upload.csv" "-OVERWRITE=true"

This KM has the following options:

CLEANUP_TEMPORARY_OBJECTS— It is used to clean-up temporary objects created. Set this property to True, if you wish to automatically clean-up the temporary objects created.
OVERWRITE_TARGET_FILE— It is used to overwrite target file. If set to True, the Oracle Object Storage target file will be overwritten.
TEMP_SCHEMA — It is used to specify the name of logical schema defining the location the temporary file that will be stored before uploading the data onto Oracle Object Storage. This is a File technology logical schema. The temporary file is stored in a local file system where ODI agent is running.
DATE_FORMAT — It specifies the output format used for date data types.
STORE_AS_XML — It is used to store data as XML file. Set this property to true, if you wish to store data as XML file in Oracle Object Storage.
CHARSET_ENCODING — It specifies the character set encoding.
XML_CHARSET_ENCODING— It specifies the character encoding indicated in the XML declaration header of the export file.
ADD_COMPRESSION — It is used to compress data before loading. Set this property to True, if you want to compress source data before loading onto Oracle Object Storage. Use the additional options COMPRESSION_TYPEand KEEP_SOURCE_FILES to define compression preferences.
FETCH_SIZE — It specifies the number of rows (records read) requested by ODI agent on each communication with the data server.
COMPRESSION_TYPE — It is used to specify the compression type. Select the type of compression you wish to apply on source data before loading onto Oracle Object Storage.
KEEP_SOURCE_FILES — It is used to retain the original source files after compression. Set this property to True (by default), if you wish to compress the target files and retain the original files.

Note:
gzip supports KEEP_SOURCE_FILES option, starting from version 1.6 only.