Types of Data Loading

Data loading is the process of importing account information from resources into Identity Manager and assigning these accounts to Identity Manager users. Identity Manager supports the following features that load account data from resources:

• Discovery. Provides basic functions that initially load resource accounts into Identity Manager.

• Reconciliation. Periodically loads resource account information into Identity Manager, taking action on each account according to configured policy.

• Active Sync. Allows information that is stored in an “authoritative” external resource (such as an application or database) to synchronize with Identity Manager user data. An Active Sync-enabled adapter “listens” or polls for changes to the authoritative resource.

Each of these concepts is discussed in detail. A table comparing the types of data loading can be found in Summary of Data Loading Types.

Discovery

The discovery processes are designed to be used when a resource is being deployed for the first time. They provide a means to load account information into Identity Manager quickly. As a result, they do not provide all the features found in reconciliation or Active Sync. For example, the discovery process does not add entries to the Account Index. Nor can you run workflows before or after discovery. However, the discovery processes allow you to determine more quickly whether correlation rules are working as expected.

When you begin a discovery process, Identity Manager determines whether an input account matches (or correlates with) an existing user. If it does, the discovery process merges the account into the user. The process will create a new Identity Manager user from any input account that does not match.

Identity Manager provides the following discovery functions:

• Load From File. Reads accounts listed in a file and loads them into Identity Manager.

• Load From Resource. Extracts accounts from a resource and loads them directly into Identity Manager.

• Create Bulk Action. Executes user creation commands listed in a file.

See the following sections for more information about these discovery processes.

Load from File

The Load from File discovery process reads account information that has been written into an XML or CSV (comma-separated values) file.

Some resources, such as Active Directory, have the ability to export native account information into a comma-separated values (CSV) format. These CSV files can be used to create Identity Manager accounts. See Business Administrator's Guide for more information about CSV formatting.

When you load from a file, you must specify which account correlation and confirmation rules to use. See Correlation and Confirmation Rules for more information.

Load from Resource

The Load from Resource feature scans a target system and returns information on all users. Identity Manager then creates and updates users. An adapter must have been configured for the resource before you can load from the resource.

When you load from a resource, you must specify which account correlation and confirmation rules to use. See Correlation and Confirmation Rules for more information.

Create Bulk Action

Bulk actions allow you to act on multiple accounts at the same time. You can use bulk actions to create, update, and delete Identity Manager and resource accounts, but this discussion will be limited to Identity Manager creating accounts. See Business Administrator's Guide for a full description of bulk actions.

Bulk actions are specified using comma-separated values (CSV). The structure of these values differs from those specified in a Load from File process.

The CSV format consists of two or more input lines. Each line consists of a list of values separated by commas. The first line contains field names. The remaining lines each correspond to an action to be performed on an Identity Manager user, the user’s resource accounts, or both. Each line should contain the same number of values. Empty values will leave the corresponding field value unchanged.

Two fields are required in any bulk action CSV input:

• user. Contains the name of the Identity Manager user.

• command. Contains the action taken on the Identity Manager user. For creating Identity Manager users, this value must be Create.

The third and subsequent fields are from the User view. The field names used are the path expressions for the attributes in the views. See Understanding the User View in Deployment Reference for information on the attributes that are available in the User View. If you are using a customized User Form, then the field names in the form contain some of the path expressions that you can use.

Following is a list of some of the more common path expressions used in bulk actions:

• waveset.roles. A list of one or more role names to assign to the Identity Manager account.

• waveset.resources. A list of one or more resource names to assign to the Identity Manager account.

• waveset.applications. A list of one or more resource groups to assign to the Identity Manager account.

• waveset.organization. The organization name in which to place the Identity Manager account.

• accounts[resource_name].attribute_name. A resource account attribute. The names of the attributes are listed in the schema for the resource.

Some fields can have multiple values. For example, the waveset.resources field can be used to assign multiple resources to a user. You can use the vertical bar (|) character (also known as the “pipe” character), to separate multiple values in a field. The syntax for multiple values can be specified like this:

value0 | value1 [ | value2 ... ]

The following example illustrates Create bulk actions:

command,user,waveset.resources,password.password,password.confirmPassword,accounts[AD].description
,accounts[Solaris].comment
Create,John Doe,AD|Solaris,changeit,changeit,John Doe,John Doe
Create,Jane Smith,AD,changeit,changeit,Jane Smith,

The Create bulk action is more versatile than the from Load from File process. Bulk actions can work with multiple resources, while Load from File loads information from one resource at a time.

Reconciliation

Reconciliation compares the contents of the account index to what each resource currently contains. Reconciliation can perform the following functions:

• Detect new and deleted accounts

• Detect changes in account attribute values

• Correlate accounts with Identity Manager users

• Detect accounts that are not associated with Identity Manager users

• Run a workflow in response to each account situation that it detects

• Detect when a user has been moved from one container on a resource to another container on a resource

An adapter must have been configured for the resource before you can reconcile. See Resource Reference for more information about adapters.

There are two types of reconciliation: full and incremental.

Full Reconciliation

Full reconciliation recalculates the existence, ownership, and situation for each account ID listed by the adapter. It examines each Identity Manager user that claims the resource to recalculate ownership.

An Identity Manager user can claim a resource by:

• Having a role that implies the resource

• Having a direct resource assignment

• Referring to an account on that resource

• Having a resource group

For each account, reconciliation process confirms that any Identity Manager owner recorded in the Account Index still exists and still claims the account. Any account that does not have an owner is correlated with Identity Manager users (as long as reconciliation policy for that resource specifies a correlation rule). If a correlation rule suggests one or more possible owners, then each of them will be double-checked in a confirmation rule (if one is specified). See Correlation and Confirmation Rulesfor more information about rules.

Once a situation has been determined for the account, reconciliation will perform any response that is configured in the reconciliation policy for that resource. If the reconciliation policy specifies a workflow to be performed per-account, full reconciliation will perform this for each account that is reconciled, after the situation action is performed. See Reconciliation Workflows for more information about workflows.

Incremental Reconciliation

Incremental reconciliation is analogous to incremental backup: it is faster than full reconciliation, and does most of what you need, but is not as complete as full reconciliation.

Incremental reconciliation trusts that the information maintained in the account index is correct. Trusting that the list of known account IDs is correct, and that ownership of the account by any Identity Manager owner is correctly recorded, allows incremental reconciliation to skip or shorten several processing phases.

Incremental reconciliation skips the step of examining Identity Manager users that claim the resource. Incremental reconciliation also calculates a situation only for accounts that have been added or deleted since the resource was last reconciled. It does this by comparing the list of account IDs in the account index for that resource to the list of account IDs returned by the resource adapter. New accounts are recorded as existing, deleted accounts are recorded as no longer existing, and only these two sets of accounts are processed further.

Because incremental reconciliation is much faster and uses fewer processing cycles than full reconciliation, you may want to schedule incremental reconciliation more frequently and schedule full reconciliation less often.

Active Sync

Active Sync “listens” or polls for changes to a resource, detecting incremental changes in real time. Because Active Sync is designed to detect changes, it should not be used to load account information into Identity Manager for the first time. Instead, use reconciliation or a discovery process.

In general, you run reconciliation on an Active Sync resource in the following circumstances:

• To perform an initial load on the resource.

• To detect any attributes that have not been updated in Identity Manager because Active Sync has been configured to ignore or filter out the attributes.

Active Sync differs from reconciliation in the following ways:

• Active Sync allows an administrator to specify a user form that ensures attributes across multiple accounts are kept synchronized.

• A process rule can be implemented that fully controls all Active Sync processing. This is typically enabled when extraordinary actions need to be performed when an account on a resource changes, such as editing multiple objects in the repository.

Active Sync requires the use of an Active Sync-enabled adapter that has been properly configured. See Business Administrator's Guide for more information about configuring a resource to implement Active Sync.

Summary of Data Loading Types

The following table compares the capabilities of discovery and reconciliation.