6 Managing File Transfers
Note:
File Transfer Services do not replace SFTP in existing customer environments. If your environment is already provisioned then you will continue to use SFTP as described in the product documentation for Retail Insights and AI Foundation, and the section below does not apply to you.File Transfer Services (FTS) for the Retail Analytics and Planning cloud services replaces SFTP for new environments. This allows you to manage the uploading and downloading of files to and from Oracle Cloud Infrastructure Object Storage, which is an internet-scale, high-performance storage platform offering reliable and cost-efficient data durability.
Access to files is through a pre-authenticated request (PAR), which is a URL that requires no further authentication to upload or download files to the cloud. To retrieve a PAR, you must use the appropriate file transfer REST service. These new services will enable you to import files to and export files from Object Storage used by the solutions. The primary role of these services is to ensure that only valid external users can call the service by enforcing authorization policies. Where supported, the files can be zipped and compressed before upload.
Note:
It is important to note that this Object Storage instance is not intended for long-term storage needs. Oracle’s-file retention policy for RAP solutions is between 7 and 30 days depending on the OCI region, after which time any incoming or outgoing files may be purged without notice. It is the customer’s responsibility to maintain backups of all input and output files as needed.The following describes how an external integrating system uses FTS to upload .zip files to Oracle Cloud Services:
-
The 3rd party solution sends a request to the pre-authentication service for uploading files to Object Storage, including the incoming prefix and file name. On receiving the PAR, the file is uploaded using the metadata from the response.
-
Cloud service batch processing sends a request to the pre-authentication service for downloading files from Object Storage, including the incoming prefix. On receiving the PAR, the files are downloaded using the metadata from the response.
-
Batch processing uses the Delete Files Service to delete the file from Object Storage and the incoming prefix to ensure it is not re-processed in the next batch run. The batch process unzips the file and processes the data.
To interact with FTS, use the REST APIs. The table below lists the API end points for different file operations.
| Operation | Method | FTS API Endpoint |
|---|---|---|
|
Ping |
GET |
|
|
List Prefixes |
GET |
|
|
List Files |
GET |
|
|
Move Files |
POST |
|
|
Delete Files |
DELETE |
|
|
Request Upload PAR |
POST |
|
|
Request Download PAR |
POST |
|
Interaction with the File Transfer Services should be developed and automated during implementation, and ongoing maintenance of the process is minimal. As an administrator, your primary task is to upload files to FTS that produced errors during the nightly batch process. Automate this file upload process simplify ongoing maintenance, such as the development of scripts and server commands that are easy to execute after go-live. For reference, the general file upload process is outlined below, but it is not expected that a customer administrator would run these commands as-is in Production environments.
The upload of files is a multi-step process where you first request access to a location to upload the file, and then you push the file to the URL provided in the response. FTS uses OAuth 2.0 and requires authentication tokens to use the services.
Further information on interacting with File Transfer Services can be found in the Retail Analytics and Planning Implementation Guide.