Extending the Functionality of the Flat File Connector

7 Extending the Functionality of the Flat File Connector

You can extend the functionality of the connector to address your specific business requirements.

This chapter contains the following optional procedures:

7.1 Configuring Custom Parsers

Learn about configuring the Flat File connector to process flat files of format other than CSV.

Note:

If you are using the latest version of this connector in the CI-based mode, then see Oracle Identity Manager Connector Guide for Flat File, Release 11.1.1 for detailed information on configuring custom parsers. In addition, depending on whether you have configured your target system as a target resource or a trusted source, update the Lookup.FlatFile.Configuration or Lookup.FlatFile.Configuration.Trusted lookup definitions to include a new Code Key entry headerRowPresent and set its decode value to True if your flat file contains a header row or False if it does not contain a header row.

This section contains the following topics:

7.1.1 Understanding Custom Parser Configuration

By default, the connector supports processing of flat files exported in the CSV format. To support the processing of flat files exported in formats other than CSV, you must create a custom parser and integrate it with the connector.

Configuring the connector for a custom parser requires you to write a custom parser class that implements the Flat File parser. Then, you must integrate this custom parser class with the connector by creating a JAR to hold the custom parser class and including it in the connector bundle JAR. In addition, in the Advanced Settings section of your application, you must add the parserClassName and customConfigParams attributes and enter relevant values.

Then, if the exported flat file (containing user data) does not contain a header row, you must add the headerRowPresent parameter to the Advanced Settings section and set its value to False. In addition, you must create a CSV file containing only the headers and enter the name and complete path of this file in the flatFileLocation field of the Advanced Settings section. This is to help the connector understand the schema of the file flat that it needs to parse.

7.1.2 Creating the Custom Parser

Learn about creating a custom parser.

7.1.2.1 About Creating the Custom Parser

To configure custom parsers, you must write the code that implements the required custom parser logic in a Java class.

This custom parser class must implement the FlatFileParser interface and the parse method.

Note:

Ensure that the Java version of Oracle Identity Manager and the compiled class is the same.

The following procedure describes the way in which the code must be written to implement the required custom parser logic:

class customParser implements FlatFileParser{
void parse(File flatFile, FlatFileRecordHandler recordHandler, ParserConfig config, boolean isHeaderRowPresent)
{
if isHeaderRowPresent is "true"
{
void parse(File flatFile, FlatFileRecordHandler recordHandler, ParserConfig
config) {

For each record in the flatFile, do the following:

Start loop

1) Perform mandatory attribute validation. If all the mandatory attributes
are present, continue, else skip the record.
See Performing Mandatory Attributes Validation for more
information on mandatory attribute validation.

2) Check for the Filter. Skip this step if the filter is not present. If
the filter is present and the record satisfies the filter, then
continue, else, skip the record.
See Checking for Filters for more information on
filters.

3) Create a new FlatFileRecord object and populate all the
attributes in the record.

a) Get a list of fields to be sent from the parser, by using
attributesToGet() method of ParserConfig object.
The parser should only send these fields back to the requester,
though the flat file may contain many more.

b) Check if the field is a single or multivalued field by using the
isMultiValued(fieldName) method of the FlatFileSchema object that
is returned by the ParserConfig's getSchema() method.

i) If the field is single-valued, then add it to the record by
using the FlatFileRecord's
addSingleValuedAttribute(fieldname,fieldValue).

ii) If the field is a multivalued Attribute, then check if the
attribute is a complex multivalued Attribute. A complex
multivalued attribute is an attribute which contains
subfields. The getSubFields(fieldname)method of the schema
returns the list of subfields if they are present, or returns
null, if they are not.

ii.a) If the multivalued field does not contain subfields,
then add it to the record by using the FlatFileRecord's
method addMultiValueAttribute(fieldname, list of
attribute values).

ii.b) If the multivalued field contains subfields, then the
multivalued field is complex field. This value can be
added to the record by using the FlatFileRecord's method
addComplexAttribute(fieldname, list of map of values).

See Handling Multivalued Attributes
for more information about multivalued attributes.

4) Finally, pass the FlatFileRecord back to the requester by calling the
RecordHandler's handle method.
Endloop

}
}
else
{
For each record in the flatFile, do the following:
Start loop
1) Create a new FlatFileSchema object and fetch all the fieldNames with getFieldNames() method of FlatFileSchema.

2) Create a new FlatFileRecord object and populate all the
attributes in the record.

b) Check if the field is a single or multivalued field by using the
isMultiValued(fieldName) method of the FlatFileSchema object that
is returned by the ParserConfig's getSchema() method.

i) If the field is single-valued, then add it to the record by
using the FlatFileRecord's
addSingleValuedAttribute(fieldname,fieldValue).

ii.a) If the multivalued field does not contain subfields,
then add it to the record by using the FlatFileRecord's
method addMultiValueAttribute(fieldname, list of
attribute values).

See Handling Multivalued Attributes
for more information about multivalued attributes.

3)Finally, pass the FlatFileRecord back to the requester by calling the
RecordHandler's handle method.
Endloop
}
}

Note:

If you are parsing the file without headers, then the sequence of attributes in the file should be same as that of field names in the Flat File Schema Properties table for your flat file application after parsing the file.

7.1.2.2 Performing Mandatory Attributes Validation

If the mandatory attributes have null or empty values, then you can skip processing such records or log these records at the parser level.

The default CSVParser performs the check on mandatory attributes. If any mandatory attribute contains a value null or empty, the CSV parser creates a directory called "failed" under the directory containing the flat file, and copies the failed records to a flat file with the same name.

The getMandatoryAttrs() method of the FlatFileSchema object returns the list of attributes required by the connector. The FlatFileSchema Object is obtained from the parserConfig parameter of the parse method. The getComplexMandatoryAttributes() method returns the list of complex attributes, and the getSimpleMandatoryAttributes() method returns the list of simple attributes.

7.1.2.3 Checking for Filters

Filters can be specified at the parser level. If a record matches the filter, it is processed, otherwise it can be skipped.

Only simple filters, without the 'and' or 'or' expressions, are supported at the parser level. However, you can specify complex filters by specifying a value for the Filter attribute in the scheduled jobs, as they are supported by ICF. In addition, limited reconciliation comes with a performance overhead, as the entire flat file is parsed to check the filter criteria.

The getFilter() method of the ParserConfig parameter of the parse method returns the FlatFileFilter Object. It is a filter object, represented by the attribute name, the attributes value, and the operator.

A filter such as equalsTo('username','johndoe') can be used in the parser as follows:

FlatFileFilter filter=config.getFilter();
String filterFieldname=filter.getFieldName();
String filterValue=filter.getFieldValue();
Operators operator=filter.getOperator();
// since its equalsTO filter, the Operator will be Operators.EQUALS
If(operator.equals(Operators.EQUALS)){
    String userValue=getUserValue(filterFieldname);
    If(userValue.equals(filterValue))
    //process the record
    Else 
    //skip the record
}

7.1.2.4 Handling Multivalued Attributes

Multivalued attributes are of two types, they are complex and simple. Simple multivalued attributes do not contain any subfields. Complex multivalued attributes contain subfields.

Example 1: Roles is a complex multivalued attribute and it contains subfields like Role Name, Start Date, and End Date. The connector requires the complex data to be represented in a list of mappings that contain subfields and their values in a key-value pair. The following is the format in which the data must be represented:

If there are three roles, such as role1, role2, role3 assigned to the user, then the connector requires a list of these maps in the following format:

List of< Role1 Map, Role2 Map, Role3 Map>.

Here, each role value is in itself a map with key as sub-field name and its value as sub-field value.

Example 2: Groups is a simple multivalued attribute, and it contains such as group1, group2, group3. Here, the connector requires a list of all these values in the following format:

List of<Group1, Group2, Group3>.

7.1.3 Integrating the Custom Parser with the Flat File Connector

Perform this procedure to integrate the custom parser with the connector before it can be used.

Create a JAR file to hold the custom parser class.
Run the Oracle Identity Manager Download JARs utility to download the org.identityconnectors.flatfile-12.3.0.jar file from the database. This utility is copied into the following location when you install Oracle Identity Governance:

Note:

Before you use this utility, verify that the WL_HOME environment variable is set to the directory in which Oracle WebLogic Server is installed.

For Microsoft Windows:

OIM_HOME/server/bin/DownloadJars.bat

For UNIX:

OIM_HOME/server/bin/DownloadJars.sh

When you run the utility, you are prompted to enter the login credentials of the Oracle Identity Governance administrator, URL of the Oracle Identity Governance host computer, context factory value, type of JAR file being downloaded, and the location to which the JAR file is to be downloaded. Specify 4 (ICFBundle) as the value of the JAR type.
Update the org.identityconnectors.flatfile-12.3.0.jar file as follows:
1. Extract the contents of the org.identityconnectors.flatfile-12.3.0.jar file into a temporary directory.
2. Create a directory called lib in the temporary directory.
3. Copy the JAR file created in Step 1 into the lib directory.
4. Re-create the org.identityconnectors.flatfile-12.3.0.jar file by running the following command:
```
jar -cvfm org.identityconnectors.flatfile-1.0.1115.jar META-INF/MANIFEST.MF *
```
  Note:
  
  While re-creating the JAR file, ensure that META-INF\MANIFEST.MF file is unchanged.
Run the Oracle Identity Manager Update JARs utility to upload the org.identityconnectors.flatfile-12.3.0.jar file to the database. This utility is copied into the following location when you install Oracle Identity Governance:

For Microsoft Windows:

OIM_HOME/server/bin/UpdateJars.bat

For UNIX:

OIM_HOME/server/bin/UpdateJars.sh

When you run the utility, you are prompted to enter the login credentials of the Oracle Identity Governance administrator, URL of the Oracle Identity Governance host computer, context factory value, type of JAR file being uploaded, and the location from which the JAR file is to be uploaded. Specify 4 (ICFBundle) as the value of the JAR type.
Restart the Oracle Identity Governance server after updating the JAR file.
Log in to Identity Self Service, and update the Advanced Settings section for your application by adding the parserClassName and customConfigParams attributes. To add these new attributes, click Add Attribute and then in the New Attribute window that is displayed, enter values for the following fields and click OK:
- Name: Enter parserClassName as the name of the attribute being added
- Value: Enter the fully qualified name of the class implementing the custom parser. For example, com.extension.parser.XMLParser.
- Category: You need not enter any value for this field. The default value for this field is Custom.
- Display Name: Enter a display name for the attribute being added. For example, Parser Class Name.
Similarly, add the other attribute by entering values for the following fields:
- Name: Enter customConfigParams as the name of the attribute being added
- Value:
  Specify the custom configuration parameters used by the custom parser. The value must be in the name-value format as follows:
  
  NAME1=VALUE1;NAME2=VALUE2
  . For example, Type=DOM;Version=1.0.
- Category: You need not enter any value for this field. The default value for this field is Custom.
- Display Name: Enter a display name for the attribute being added. For example, Custom Config Params.

7.1.4 Creating a CSV File and Updating the Advanced Settings Section

If your flat file containing user data does not have a header row, then you must specify the same in the Advanced Settings section. In addition, create a CSV file with just the header row and update its location in the Advanced Settings section.

The connector needs to understand the schema of your flat file to successfully perform reconciliation and provisioning operations against it. The presence of a header row in your flat file is important for the connector to obtain the schema of the flat file that it needs to parse.

On the computer hosting Oracle Identity Governance, create a CSV file containing only the header row.
In the Advanced Settings section of your flat file application:
1. Click Add Attribute to add a new attribute named headerRowPresent, and set its value to False.
2. In the flatFileLocation field, enter the complete name and path of the CSV file created in Step 1.
3. Click Parse Headers.
  The header row information is extracted from the CSV file and is listed in the Flat File Schema Properties table.

7.2 Configuring Preprocess and Postprocess Tasks

Learn about preprocess and postprocess tasks and the procedure to integrate them with the connector.

This section contains the following topics:

7.2.1 Understanding the Preprocess and Postprocess Tasks

Preprocess and postprocess tasks can be run before and after reconciliation of accounts respectively.

These tasks can be used to perform any job on the flat file directory, like zipping files, unzipping files, encryption and decryption of the complete file dumps or specific fields in the files, virus scan of the files, or any other tasks limited only by the implementation of these tasks.

This section contains the following topics

7.2.1.1 About the Preprocess task

If you are writing code for the preprocess task, then the class must implement the FlatFilePreProcessHandler interface and the preProcess method.

The preProcess method has following parameters:

flatFileDir

This parameter specifies the path to the directory containing the flat files.
configMap

This parameter contains the mapping of all the configuration parameters from the main configuration lookup definition and the scheduled jobs.

The following procedure describes the way in which the code must be written to implement the FlatFilePreProcessHandler interface:

Class preProcessTask implements FlatFilePreProcessHandler{

    void preProcess(java.io.File flatFileDir,
        java.util.Map<java.lang.String,java.lang.Object> configMap) throws java.lang.Exception{

        //perform the pre process task like unzip and decrypting the files etc.

    }

}

7.2.1.2 About the Postprocess Task

If you are writing code for the postprocess task, then the class must implement the FlatFilePostProcessHandler interface and the postProcess method.

The postProcess method has following parameters:

flatFileDir

This parameter specifies the path to the directory containing the flat files.
configMap

This parameter contains the mapping of all the configuration parameters from the main configuration lookup definition and the scheduled jobs.

The following procedure describes the way in which the code must be written to implement the FlatFilePostProcessHandler interface:

Class postProcessTask implements FlatFilePostProcessHandler{

    void postProcess(java.io.File flatFileDir,
        java.util.Map<java.lang.String,java.lang.Object> configMap) throws java.lang.Exception{

        //perform the post process task like encrypting the files,
        //password protecting files etc.

    }

}

7.2.2 Integrating the Preprocess and Postprocess Tasks with the Flat File Connector

This is the procedure to integrate preprocess and postprocess tasks with the connector before they can be used.

To configure preprocess and postprocess tasks:

Create a JAR file to hold the preprocess or postprocess task class.
Run the Oracle Identity Manager Download JARs utility to download the org.identityconnectors.flatfile-12.3.0.jar file from the database. This utility is copied into the following location when you install Oracle Identity Manager:

Note:

Before you use this utility, verify that the WL_HOME environment variable is set to the directory in which Oracle WebLogic Server is installed.
- For Microsoft Windows:
  
  OIM_HOME/server/bin/DownloadJars.bat
- For UNIX:
  
  OIM_HOME/server/bin/DownloadJars.sh
When you run the utility, you are prompted to enter the login credentials of the Oracle Identity Manager administrator, URL of the Oracle Identity Manager host computer, context factory value, type of JAR file being downloaded, and the location to which the JAR file is to be downloaded. Specify 4 (ICFBundle) as the value of the JAR type.
Update the org.identityconnectors.flatfile-12.3.0.jar file as follows:
1. Extract the contents of the org.identityconnectors.flatfile-12.3.0.jar file into a temporary directory.
2. Create a directory called lib in the temporary directory.
3. Copy the JAR file created in Step 1 into the lib directory.
4. Re-create the org.identityconnectors.flatfile-1.0.1115.jar file by running the following command:
```
jar -cvfm org.identityconnectors.flatfile-12.3.0.jar META-INF/MANIFEST.MF *
```
  Note:
  
  While re-creating the JAR file, ensure that META-INF\MANIFEST.MF file is unchanged.
Run the Oracle Identity Manager Update JARs utility to upload the org.identityconnectors.flatfile-12.3.0.jar file to the database. This utility is copied into the following location when you install Oracle Identity Governance:
- For Microsoft Windows:
  
  OIM_HOME/server/bin/UpdateJars.bat
- For UNIX:
  
  OIM_HOME/server/bin/UpdateJars.sh
When you run the utility, you are prompted to enter the login credentials of the Oracle Identity Governance administrator, URL of the Oracle Identity Governance host computer, context factory value, type of JAR file being uploaded, and the location from which the JAR file is to be uploaded. Specify 4 (ICFBundle) as the value of the JAR type.
Restart the Oracle Identity Governance Server after updating the jar.
Repeat Steps 1 through 5 to create a JAR file to hold the postprossess task class and copy the JAR file into Oracle Identity Manager.
Log in to Identity Self Service, and update the Advanced Settings section for your application by adding the preProcessClassName and postProcessClassName attributes. To add these new attributes, click Add Attribute and then in the New Attribute window that is displayed, enter values for the following fields and click OK:
- Name: Enter preProcessClassName as the name of the attribute being added
- Value: Enter the fully qualified name of the class implementing the preprocess task. For example, com.extension.parser.PreProcessHandler.
- Category: You need not enter any value for this field. The default value for this field is Custom.
- Display Name: Enter a display name for the attribute being added. For example, Preprocess Class Name.
Similarly, add the other attribute by entering values for the following fields:
- Name: Enter postProcessClassName as the name of the attribute being added
- Value: Enter the fully qualified name of the class implementing the postproces task. For example, com.extension.parser.PostProcessHandler.
- Category: You need not enter any value for this field. The default value for this field is Custom.
- Display Name: Enter a display name for the attribute being added. For example, Postprocess Class Name.

7.3 Adding New Attributes for Reconciliation

By default, the attributes listed in the Flat File Schema Properties table on the Basic Information page of your application are mapped for reconciliation between the flat file and Oracle Identity Governance. If required, you can add new attributes to the schema for reconciliation.

After creating the application for the flat file, if the exported flat file contains a new attribute, then you can include this new attribute into the schema in one of the following ways:

In the Advanced Settings section of your application, click Parse Headers to fetch all the attributes in the flat file, including the new attribute. Review the Flat File Schema Properties table and make any changes, if required. For example, change the datatype, mark an attribute an multivalued, and so on. Save the changes to your application.
In the Flat File Schema Properties table on the Basic Information page of your application, click Add Field and fill in the required details to add the new attribute manually. Save the changes to your application.

7.4 Configuring Transformation and Validation of Data

Configure transformation and validation of user account data by writing Groovy script logic while creating your application.

You can configure transformation of reconciled single-valued user data according to your requirements. For example, you can use First Name and Last Name values to create a value for the Full Name field in Oracle Identity Governance.

Similarly, you can configure validation of reconciled and provisioned single-valued data according to your requirements. For example, you can validate data fetched from the First Name attribute to ensure that it does not contain the number sign (#). In addition, you can validate data entered in the First Name field on the process form so that the number sign (#) is not sent to the target system during provisioning operations.

To configure transformation or validation of user account data, you must write Groovy scripts while creating your application. For more information about writing Groovy script-based validation and transformation logic, see Validation and Transformation of Provisioning and Reconciliation Attributes of Oracle Fusion Middleware Performing Self Service Tasks with Oracle Identity Governance.

7.5 Configuring the Connector for Multiple Installations of the Target System

You must create copies of configurations of your base application to configure it for multiple installations of the target system.

The following example illustrates this requirement:

The London and New York offices of Example Multinational Inc. have their own installations of the target system, including independent schema for each. The company has recently installed Oracle Identity Governance, and they want to configure it to link all the installations of the target system.

To meet the requirement posed by such a scenario, you must clone your application which copies all configurations of the base application into the cloned application. For more information about cloning applications, see Cloning Applications in Oracle Fusion Middleware Performing Self Service Tasks with Oracle Identity Governance.

7.6 Configuring Action Scripts

For a disconnected resource, you can configure Action Scripts by writing your own Groovy scripts while creating the application. For a connected resource, the connector fetches the action scripts that have been defined for the corresponding target application, for example, Salesforce.

You can configure action scripts to run before or after the create, update, or delete an account provisioning operations. For example, you can configure a script to run before every user creation operation.

For information on adding or editing action scripts, see Updating the Provisioning Configuration in Oracle Fusion Middleware Performing Self Service Tasks with Oracle Identity Governance.