1 About the Flat File Connector
The Flat File connector integrates Oracle Identity Manager with files of formats such as CSV, LDIF, and XML.
The following topics provide a high-level overview of the connector:
1.1 Introduction to the Connector
Oracle Identity Manager automates access rights management, security, and provisioning of IT resources. Oracle Identity Manager connectors are used to integrate Oracle Identity Manager with external, identity-aware applications. This guide describes the usage of a flat file-based data synchronization approach that you can use to on-board users, entitlements, and entitlement grants from various systems into Oracle Identity Manager by using the Flat File connector.
Enterprise applications generally support the export of users in the form of a file. Some widely used file formats are CSV, LDIF, and XML. The connector will consume the information in a flat file, thereby enabling the import of this data as Oracle Identity Manager user accounts or entitlements. The flat file connector can be used in a number of situations for offline data loading or when a predefined connector is not available.
1.2 Certified Components
These are the software components and their versions required for installing and using the connector.
Table 1-1 Certified Components
| Item | Requirement |
|---|---|
|
You can use one of the following releases of Oracle Identity Governance or Oracle Identity Manager:
|
|
|
Any enterprise target system that can export users, accounts, or entitlements to a flat file. |
|
|
11.1.1.5.0 or later Note: You can download the necessary Java Connector Server from the Oracle Technology Network web page. |
|
|
Connector Server JDK |
JDK 1.6 or later |
|
CSV Note: Formats other than CSV are supported through the use of custom parsers. |
1.3 Certified Languages
The connector will support the languages that are supported by Oracle Identity Manager.
Resource bundles are not part of the connector installation media as the resource bundle entries vary depending on the flat file being used. Field labels in UI forms can be localized, see Localizing Field Labels in UI Forms for more information.
1.4 Connector Architecture
The Flat File connector is a generic solution to retrieve records from flat files that are exported from various enterprise target systems. This connector is implemented using the Identity Connector Framework (ICF) component.
These flat files can be of various formats such as CSV, LDIF, XML, and so on. The connector focuses only on the reconciliation of records from a flat file. The installation media contains scheduled jobs that can be used to load users, accounts, and entitlements from a flat file into an existing resource in Oracle Identity Manager.
Figure 1-1 shows the connector integrating the flat files exported from an enterprise target system with Oracle Identity Manager.
The flat files exported from the enterprise target system are stored in a directory that is accessible from Oracle Identity Manager. The connector will sort the files within the directory in an alphanumeric manner, and will process each file based on this order. Additionally, a schema file that describes the attributes in that flat file must be created in the expected format, see Understanding and Creating the Schema File for more information about the schema file. The location of the directory containing the flat file is specified in the attributes of a scheduled job and the location of the schema file is specified as an IT Resource parameter. When a scheduled job is run, it calls the connector's search implementation, which in turn returns the connector objects to Oracle Identity Manager.
The connector installation media also contains a metadata generator utility to generate Oracle Identity Manager artifacts based on the flat file schema. The metadata generator utility generates a connector package (a zip file) that can be installed in Oracle Identity Manager and used as a resource to load entities by using this connector.
1.5 Use Cases Supported by the Connector
These are the scenarios in which you can use the connector.
1.5.1 Reconciliation of Records
Reconciling records from a flat file exported from an enterprise target system involves loading data from a flat file into Oracle Identity Manager.
You can perform the following operations in this scenario:
-
Reconciliation
-
Certification
Here, the Flat File connector can be used to perform reconciliation runs.
The following example shows how the Flat File connector can be used to load data from a flat file into Oracle Identity Governance to perform certification tasks.
Suppose John works as a Compliance Administrator in ACME Corporation. He uses Oracle Identity Governance to define roles, automate certification processes, and generate business structure reports for auditing. He has a list of users in his enterprise and their entitlements in the form of a CSV file, and he wants to import this data into Oracle Identity Governance, to use this data purely for certification purposes. He needs to create resource objects and forms for all the users, and import the data into these tables.
In the preceding example, by using the flat file connector, John can load accounts from a flat file into a Flat File Resource. He can run the corresponding reconciliation jobs of the flat file to import data from the CSV file into Oracle Identity Governance.
1.5.2 Disconnected Resource
Disconnected resources are targets for which a predefined connector does not exist. Therefore, the provisioning fulfillment for disconnected resources is not automated, but manual.
You can perform the following operations in this scenario:
-
Request
-
Manual fulfillment or provisioning
-
Reconciliation
-
Certification
Here, the Flat File connector can be used to perform reconciliation runs and provisioning operations.
The following example shows how the Flat File connector can be used to load data from a flat file into Oracle Identity Governance for disconnected resources.
Suppose Smith is the chief librarian in the University of Utopia. His responsibilities include providing library access cards to the students of the university. He has a file with the list of students who already have library cards. He wants to transfer this list to Oracle Identity Governance after which he can automate the library transactions for existing members.
In the preceding example, as library cards are modeled as a disconnected resource in Oracle Identity Governance, he can create an application for the disconnected resource, and then load accounts from a flat file into a Library Card Resource using the corresponding reconciliation jobs. By defining a disconnected resource through Oracle Identity Governance, Smith can start reconciling users from the flat file and link them to the desired disconnected resource.
1.5.3 Connected Resource
Connected resources are targets for which a predefined connector is available, for example, Microsoft Active Directory.
You can perform the following operations in this scenario:
-
Request
-
Automatic fulfillment or provisioning
-
Reconciliation
-
Certification
Here, the Flat File connector can be used only to perform reconciliation runs.
The following example shows how the Flat File connector can be used to load data from a flat file into Oracle Identity Manager, although a predefined connector is available.
Suppose Jane works as a Network Administrator at Example Multinational Inc. In Example Multinational Inc., she performs identity and access management tasks on users within the organization. One of Jane's responsibilities is to create and maintain users in Oracle Identity Manager, and to provision these users with resources. At Example Multinational Inc., all the employee details are maintained in the Microsoft Active Directory target system. Jane wants to reconcile about 100,000 user records from the target system to her Oracle Identity Manager instance, as soon as possible. As the AD Server is planned for a maintenance shutdown, she is looking for a means for offline loading of all the user data which has been exported in the form of an LDIF file. Given the time and network constraints, Jane needs a solution for the initial on-boarding of the users into Oracle Identity Manager.
In the preceding example, performing an initial reconciliation or full reconciliation, is a performance and time-intensive operation. Using the Microsoft Active Directory User Management connector to perform the reconciliation operation requires the connection between the target system and Oracle Identity Manager to remain active. In other words, offline loading of users cannot be performed. In this scenario, a native flat file dump from the target system can be used by the Flat File connector to quickly reconcile the users into Oracle Identity Manager.
1.6 Features of the Connector
The features of the connector include support for custom parsers, fault handling, archival, connector server, transformation and validation of account data, full, incremental, limited, and batched reconciliation, and so on.
1.6.1 Support for Both Target Resource and Trusted Source Reconciliation
You can configure the exported flat file as a trusted source or target resource for reconciliation of records into Oracle Identity Manager.
See Attributes of the Scheduled Jobs for more information about the scheduled jobs that are created when you install the connector and their details.
1.6.2 Full and Incremental Reconciliation
After you deploy the connector, you can perform full reconciliation to load all existing user data from the flat file to Oracle Identity Manager.
Any new files that are added after the first full reconciliation run are considered as a source of incremental data. Alternatively, incremental reconciliation can also be performed by explicitly providing the incremental data alone.
You can perform a full reconciliation run at any time. See Performing Full and Incremental Reconciliation for more information.
1.6.3 Limited Reconciliation
You can set a reconciliation filter as the value of the Filter attribute of the scheduled jobs. This filter specifies the subset of newly added and modified enterprise target system records that must be reconciled.
See Limited Reconciliation for more information.
1.6.4 Support for Disconnected Resources
The connector provides support for disconnected resources by generating all artifacts associated with disconnected resources.
In addition, it generates process definitions associated with the default SOA composites that are required for performing manual provisioning. This eliminates the need to manually create disconnected resources and mappings between fields in Oracle Identity Governance and corresponding target system attributes.
To configure your flat file as a disconnected resource, see Understanding Entries in the FlatFileConfiguration.groovy File.
1.6.5 Resource-Specific Generation of Scheduled Jobs
The connector supports generation of resource-specific scheduled jobs. This means that depending on the type of resource (trusted source, target resource, or disconnected resource), the metadata generation utility automatically generates the corresponding scheduled jobs.
For example, if you are using the metadata generation utility and configured your flat file as a trusted resource, then the scheduled jobs such as IT_RES_NAME Flat File User Loaders, IT_RES_NAME Flat File Users Delete Diff Reconciliation, and IT_RES_NAME Flat File Users Delete Reconciliation are automatically generated.
See Reconciliation Scheduled Jobs for more information about the scheduled jobs that are created when you install the ready to use Flat File connector and a connector created by using the metadata generation utility.
1.6.6 Support for Archival
The connector supports archival of the processed flat files.
You can specify the archive directory location in the Archive directory attribute while configuring the scheduled jobs, and the connector will move the files from the source directory to the specified location, once all files are processed.
If you do not specify a value for this attribute, then the connector creates an Archived directory within the directory containing the flat file, and saves the processed files in this location.
See Configuring Archival for more information about archival.
1.6.7 Support for Custom Parsers
By default, the connector supports processing of flat files exported in the CSV format. To support the processing of flat files exported in formats other than CSV, you must create a custom parser and integrate it with the connector.
By default, the connector installation media contains the CSVParser.
See Configuring Custom Parsers for more information about custom parsers.
1.6.8 Support for Reconciling Complex Multivalued Data
The connector supports the reconciliation of complex multivalued data in the form of child forms containing single and multiple fields.
The child form data must be in the same file as the parent form data. The child form values are separated by customizable delimiters.
For example, in CSV files, every line in the flat file represents a single record which includes the parent and the child form data.
See Reconciling Complex Multivalued Data for more information.
1.6.9 Support for Delimiters
The connector supports the use of single character delimiters, which are used to separate values in a record.
You can configure delimiters by specifying the values for the fieldDelimiter, multiValueDelimiter, and subFieldDelimiter entries in the Lookup.FlatFile.Configuration and Lookup.FlatFile.Configuration.Trusted lookup definitions.
See Understanding and Configuring Delimiters for more information about configuring delimiters.
1.6.10 Support for Comment Characters
You can configure the connector to ignore the processing of lines that begin with certain characters like #,$, and so on.
These configurable characters are considered as comment characters, and sentences beginning with such characters are considered as comments. The connector implementation will skip the lines that start with the configured comment character.
This can be configured by specifying the value for the comment character entry of the Lookup.FlatFile.Configuration lookup definition.
See Configuring the Connector to Ignore Comment Characters for more information about comment characters.
1.6.11 Support for Fault Handling
The connector logs record level errors in a separate file while parsing the flat file. This log file will be saved in a directory named "failed" that the connector creates, within the flat file directory.
See Configuring Fault Handling for more information.
1.6.12 Support for Preprocess and Postprocess Handlers
Preprocess and postprocess tasks can be run both before and after the reconciliation of accounts respectively.
You can use these tasks to perform any job on the flat file directory, like zipping and unzipping files, encryption and decryption of the complete file dumps or specific fields in the files, virus scan of the files, or any other tasks limited only by the implementation of these tasks.
See Configuring Preprocess and Postprocess Tasks for more information.
1.6.13 Support for Reconciliation of Deleted Records
You can reconcile data about records that have been deleted on the enterprise target system, by using the exported flat file that has been configured as a trusted source or a target resource.
See Scheduled Jobs for Reconciliation of User Records and Scheduled Jobs for Reconciliation of Accounts for more information about the scheduled jobs used for reconciling data about deleted records.
1.6.14 Support for Transformation and Validation of Account Data
You can configure validation of account data that is brought into or sent from Oracle Identity Manager during reconciliation. In addition, you can configure transformation of account data that is brought into Oracle Identity Manager during reconciliation.
The following sections provide more information:
1.6.15 Support for Connector Server
Connector Server is a component provided by the Identity Connector Framework (ICF).
By using one or more connector servers, the connector architecture permits your application to communicate with externally deployed bundles. In other words, a connector server enables remote execution of an Oracle Identity Manager connector.
A Java connector server is useful when you do not wish to execute a Java connector bundle in the same virtual machine as your application. It can be beneficial to run a Java connector on a different host for performance improvements.
1.7 Lookup Definitions Used During Connector Operations
These lookup definitions are automatically created in Oracle Identity Manager after you deploy the connector/
Following are the Lookup definitions used during connector operations:
1.7.1 Lookup.FlatFile.Configuration
The Lookup.FlatFile.Configuration lookup definition holds connector configuration entries that are used during target resource reconciliation operations.
Table 1-2 lists the default entries in this lookup definition.
Table 1-2 Entries in the Lookup.FlatFile.Configuration Lookup Definition
| Code Key | Decode | Description |
|---|---|---|
|
Bundle Name |
org.identityconnectors.flatfile |
This entry holds the name of the connector bundle package. Do not modify this entry. |
|
Bundle Version |
1.0.1115 |
This entry holds the version of the connector bundle class. Do not modify this entry. |
|
Connector Name |
org.identityconnectors.flatfile.FlatFileConnector |
This entry holds the name of the connector class. Do not modify this entry. |
|
fieldDelimiter |
, |
Delimiter of each field in a row. |
|
multiValueDelimiter |
; |
Delimiter to separate each multivalued data. |
|
subFieldDelimiter |
# |
Delimiter to separate each subfield within a multivalued field. |
|
textQualifier |
" |
Character which determines the start and end of text in a value. Any delimiter within the value qualified by the textQualifier will be ignored. |
|
User Configuration Lookup |
Lookup.FlatFile.UM.Configuration |
This entry holds the name of the lookup definition that contains user-specific configuration properties. Do not modify this entry. |
1.7.2 Lookup.FlatFile.Configuration.Trusted
The Lookup.FlatFile.Configuration.Trusted lookup definition holds connector configuration entries that are used during trusted source reconciliation.
Table 1-3 lists the default entries in this lookup definition.
Table 1-3 Entries in the Lookup.FlatFile.Configuration.Trusted Lookup Definition
| Code Key | Decode | Description |
|---|---|---|
|
Bundle Name |
org.identityconnectors.flatfile |
This entry holds the name of the connector bundle package. Do not modify this entry. |
|
Bundle Version |
1.0.1115 |
This entry holds the version of the connector bundle class. Do not modify this entry. |
|
Connector Name |
org.identityconnectors.flatfile.FlatFileConnector |
This entry holds the name of the connector class. Do not modify this entry. |
|
fieldDelimiter |
, |
Delimiter of each field in a row. |
|
multiValueDelimiter |
; |
Delimiter to separate each multivalued data. |
|
subFieldDelimiter |
# |
Delimiter to separate each sub-field within a multivalued field. |
|
textQualifier |
" |
Character which determines the start and end of text in a value. Any delimiter within the value qualified by the textQualifier will be ignored. |
|
User Configuration Lookup |
Lookup.FlatFile.UM.Configuration.Trusted |
This entry holds the name of the lookup definition that contains user-specific configuration properties. Do not modify this entry. |
1.7.3 Lookup.FlatFile.EntFieldMap
The Lookup.FlatFile.EntFieldMap lookup definition is used for populating the code key and decode values of the lookup definition that holds information about target lookups.
You must manually add entries to this lookup definition. To do so, see Configuring the Connector with the Target Resource.
Table 1-4 lists the default entries in this lookup definition.
Table 1-4 Entries in the Lookup.FlatFile.EntFieldMap Lookup Definition
| Code Key | Decode |
|---|---|
|
CODE |
__NAME__ |
|
DECODE |
__NAME__ |
In the Code Key column of this lookup definition CODE and DECODE are the default values. Do not modify these values. By default, CODE and DECODE are mapped to the NameAttribute(__NAME__) mentioned in the schema file.
If you want to map CODE and DECODE to other attributes, modify the values in the Decode column as follows:
In the Decode column of the CODE entry, specify the name of the attribute that you want to populate into the Code Key column of the target lookup definition. Similarly, in the Decode column of the DECODE entry, specify the name of the attribute that you want to populate into the Decode column of the target lookup definition.
1.7.4 Lookup.FlatFile.UM.Configuration
The Lookup.FlatFile.UM.Configuration lookup definition holds configuration entries that are specific to the user object type.
This lookup definition is used during user management operations when your flat file is configured as a target resource. You must manually modify this lookup definition to map the code key to the resource from which accounts must be reconciled.
To modify this lookup definition, see Configuring the Connector with the Target Resource.
Table 1-5 lists the default entries in this lookup definition.
Table 1-5 Entries in the Lookup.FlatFile.UM.Configuration Lookup Definition
| Code Key | Decode | Description |
|---|---|---|
|
Recon Attribute Map |
Dummy |
This entry holds the name of the lookup definition that maps resource object fields and enterprise target system attributes. |
1.7.5 Lookup.FlatFile.UM.Configuration.Trusted
The Lookup.FlatFile.UM.Configuration.Trusted lookup definition holds configuration entries that are specific to the user object type.
This lookup definition is used during user management operations when your flat file is configured as a trusted source. This lookup definition must be modified manually, to map the code key to the resource from which accounts must be reconciled.
To modify this lookup definition, see Configuring the Connector with the Target Resource.
Table 1-6 lists the default entries in this lookup definition.
Table 1-6 Entries in the Lookup.FlatFile.UM.Configuration.Trusted Lookup Definition
| Code Key | Decode | Description |
|---|---|---|
|
Recon Attribute Map |
Dummy |
This entry holds the name of the lookup definition that maps resource object fields and enterprise target system attributes. |