Integrate with Flat File

Prerequisites

Before you install and configure a Flat File orchestrated system, you should consider the following prerequisites and tasks.

Certified Components

The system must be the following:

  • CSV Flat file located in Oracle Cloud Infrastructure (OCI) Object Storage in your tenancy

Supported Modes

Flat File orchestrated system supports the following modes:
  • Authoritative Source
  • Managed System

Supported Operations

The Flat File orchestrated system supports the following operations:
  • Create Account
  • Delete Account
  • Add Entitlement
  • Remove Entitlement

Create a bucket in the OCI Object Storage service for Flat File Orchestrated System Operations

In order to load a flat file into Oracle Access Governance you need to place the data files in a bucket created using the OCI Object Storage service. This bucket can be created in any compartment of your OCI tenancy. For details regarding OCI Object Storage, refer to Managing Buckets.

In order to access the bucket, you need to create a service user that has read, write, and delete access (manage privileges) to the bucket. Follow this process to create this service user:

  • Create a compartment, accessgovernance/
  • Create a local identity user, agcs_user in any domain in your tenancy.
  • Create an identity group, agcs_flatfilegroup in any domain in your tenancy.
  • Assign the identity user agcs_user to the identity group agcs_flatfilegroup.
  • Create a policy, agcs_flatfilepolicy, with the following policy statement: 
    allow group <groupname> to manage objects in compartment <compartmentname>
    where target.bucket.name = 'bucketname'

    For example:

    allow group agcs_flatfilegroup to manage objects in compartment accessgovernance
    where target.bucket.name = 'bucket-20231130-1143'

Configure

You can establish a connection between Flat File and Oracle Access Governance by entering connection details. To achieve this, use the Orchestrated Systems functionality available in the Oracle Access Governance Console.

Navigate to the Orchestrated Systems Page

Navigate to the Orchestrated Systems page of the Oracle Access Governance Console, by following these steps:
  1. From the Oracle Access Governance navigation menu icon Navigation menu, select Service Administration → Orchestrated Systems.
  2. Click the Add an orchestrated system button to start the workflow.

Select system

On the Select system step of the workflow, you can specify which type of system you would like to onboard.

  1. Select Flat File.
  2. Click Next.

Enter details

On the Enter Details step of the workflow, enter the details for the orchestrated system:

  1. Enter a name for the system you want to connect to in the What do you want to call this system? field.
  2. Enter a description for the application in the How do you want to describe this system? field.
  3. Determine if this orchestrated system is an authoritative source, and if Oracle Access Governance can manage permissions for existing users by setting the following checkboxes.

    Table - Authoritative Source/Permission Management

    Checkbox Default Description
    This is the authoritative source for my identities. Selected If selected, this checkbox means that this orchestrated system is a trusted source for user or identity information.
    I want to manage permissions for this system. Selected If selected, this checkbox means that Oracle Access Governance can provision accounts on the orchestrated system, and manage permissions for existing accounts.
  4. Click Next.

Configure

On the Configure step of the workflow, enter the configuration details required to allow Oracle Access Governance to connect to the Flat File.

  1. In the What is the OCI user's OCID? field, add the OCID for the OCI user owning the bucket containing the flat files you want to integrate.
  2. In the What is the fingerprint of the OCI user's API key?. Enter the fingerprint for the OCU user's API key. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
  3. Enter the user's private API key, in PEM format into the What is the OCI user's private API key in PEM format? field. Consult Required Keys and OCIDs in the OCI documentation for details on how to obtain the value for this.
  4. Enter the tenancy into the What is the tenancy of the OCI user? field.
  5. Enter the home region code of the tenancy into the What us the OCI tenancy's home region code? field. Details of region codes can be found in Regions and Availability Domains OCI documentation.
  6. Enter the bucket namespace of the tenancy in the What is the namespace for the bucket? field.
  7. In the What is the name of the bucket? field, enter the name of the bucket where your flat file is stored in OCI object storage.
  8. Enter the encoding into the Encoding field. Default is UTF-8.
  9. In the Field Delimiter field, enter the field delimiter character used in the Flat File. Default is ,.
  10. In the Sub Field Delimiter field, enter the sub field delimiter character used in the Flat File. Default is #.
  11. In the MultiValue Delimiter field, enter the multivalue delimiter character used in the Flat File. Default is ;.
  12. In the Text Qualifier field, enter the character used in the Flat File to act as a text qualifier. Default is ".
  13. If you want to check the connectivity to your Flat File, click the Test Connectivity button.
  14. Click Add to create the orchestrated system.

Finish up

Finally, you are given a choice whether to further configure your orchestrated system before running a data load, or accept the default configuration and initiate a data load. Select one from:
  • Customize before enabling the system for data loads
  • Activate and prepare the data load with the provided defaults

Post Configuration

Check Bucket Folder Structure

After creation of the orchestrated system, the following folder structure should be created in the defined bucket. 

<ServiceInstanceName>/<OrchestratedSystemName>
    failed
    inbox
    outbox
    sample
    schema
These folders fulfill the following purposes:
  • failed: Files with any kind of data issue will be moved to this folder under the respective entity folder, in the event of a dataload operation failure.
  • inbox: Contains the entity subfolders in which CSV files should be placed to be included in the dataload operation.
  • outbox: Used to output the provisioning events for each entity.
  • sample: Contains example CSVs with the expected header. These can be used as a reference for generating data and putting in the inbox for data load. These files should not be altered.
  • schema: contains the JSON representation of each entity's schema.This can be referred to for understanding details like:
    • dataType
    • Mandatory attributes
    • Whether an attribute is multivalued or not
    • If the attribute is complex and has nested attributes (dataType will be CUSTOM)
    • Supported dataTypes are:
      • TEXT
      • NUMBER
      • DECIMAL_NUMBER
      • DATE
      • FLAG
      • CUSTOM

Define Custom Attributes

Custom attributes are supported for the IDENTITY entity. If you want to include custom attributes in your dataload then you need to add them in the <ServiceInstanceName>/<OrchestratedSystemName>/schema/IDENTITY.json file.

Custom attribute names should meet the following requirements:
  • start with a character: A-Z or a-z
  • contain only characters or numbers: A-Z or a-z or 0-9
  • For the DATE type attribute, only long value is supported
  • Custom attributes can only be added, they cannot be deleted
  • A custom attribute of CUSTOM type cannot be added

Once you have added any custom attributes in the IDENTITY.json file, you will need to include them in Oracle Access Governance as described in Fetch Latest Custom Attributes. Once this is completed, the sample CSV will be updated with the newly added custom attribute(s). Update the data files in the inbox to include the custom attribute(s) in your next data load.

Run Dataload

Dataload is on demand. You should run the dataload when you have defined any custom attributes, and have added the relevant CSV data files into the inbox folder. Each time you run a dataload it is a full data load, there is no incremental load. UTF-8-BOM encoding is not supported.

If there is any kind of failure (single record or complete file failure), the data load operation will be marked as failed. The files that have been processed successfully will stay in the inbox while the failed files will be moved to the failed folder. After fixing the data issue, the customer is expected to put the files back in the inbox again and retry the dataload. Data integrity issues, such as a permission being assigned to an account that is missing in the CSV), can cause the dataload operation to fail. However, in such cases the CSV files will not be moved to the failed folder. Files will be moved to the failed folder only when there are issues reading the data itself, such as missing mandatory data.