Sun[TM] Identity Manager 8.0 Administration |
Chapter 7
Data Loading and SynchronizationThis chapter provides information and procedures for using Identity Manager data loading and synchronization features. You will learn how to use Identity Manager’s data synchronization tools (discovery, reconciliation, and synchronization) to keep data current.
For an in-depth explanation of how data loading and synchronization works in Identity Manager, see the “Data Loading and Synchronization” chapter in the Identity Manager Deployment Overview book.
Data Synchronization Tools: Which to Use?Identity Manager provides several tools that can be used to import and synchronize account data. For help selecting the correct tool for a given task, refer to Table 7-1.
DiscoveryIdentity Manager account discovery features help facilitate rapid deployment and speed account creation tasks. These features are:
- Extract to File — Extracts the resource accounts returned by a resource adapter to a file (in CSV or XML format). You can manipulate this file before importing the data into Identity Manager.
- Load from File — Reads accounts in a file (in CSV or XML format) and loads them into Identity Manager.
- Load from Resource — Combines the other two discovery features, extracting accounts from a resource and loading them directly into Identity Manager.
Using these tools, you can create new Identity Manager users or correlate accounts on a resource with existing Identity Manager user accounts.
Extract to File
Use this feature to extract resource accounts from a resource to an XML or CSV text file. Doing this allows you to view and make changes to extracted data before importing it into Identity Manager.
To extract accounts, follow these steps:
- From the menu bar, select Accounts, and then select Extract to File.
- Select a resource from which to extract accounts.
- Select a file format for the output account information. You can extract data to an XML file, or to a text file with account attributes arranged in comma-separated value (CSV) format.
- Click Download. Identity Manager displays a File Download dialog, in which you may choose to save or view the extracted file.
If you choose to open the file, you might have to select a program to view it.
Load from File
Use this feature to load resource accounts — either those extracted from a resource through Identity Manager, or from another file source — into Identity Manager. A file created by the Identity Manager Extract to File feature is in XML format. If you are loading a list of new users, the data file typically is in CSV format.
About CSV File Format
Often, accounts to be loaded are listed in a spreadsheet and saved in comma-separated-value (CSV) format for loading into Identity Manager. CSV file contents must follow these format guidelines:
For example, the first three lines of a file might look like the file entries in the following figure:
firstname,middleinitial,lastname,accountId,asciipassword,EmployeeID,Depart ment,Phone
John,Q,Example,E1234,E1234,1234,Operations,555-222-1111
Jane,B,Doe,E1111,E1111,1111,,555-222-4444Figure 7-1 Example of Properly Formatted CSV File for Loading Data
In this example, the second user (Jane Doe) does not have a department. The missing value is represented by adjacent commas (,,).
To load accounts, follow these steps:
- In the Administrator interface, click Accounts in the menu, then click Load from File.
Identity Manager displays the Load Accounts from File page.
- Specify the following load options on the Load Accounts from File page:
- User Form — When load creates an Identity Manager user, the user form assigns an organization as well as roles, resources, and other attributes. Select the user form to apply to each resource account.
- Account Correlation Rule — An account correlation rule selects Identity Manager users that might own each unowned resource account. Given the attributes of an unowned resource account, a correlation rule returns a list of names or a list of attribute conditions that will be used to select potential owners. Select a rule to look for Identity Manager users that may own each unowned resource account.
- Account Confirmation Rule — An account confirmation rule eliminates any non-owner from the list of potential owners that the correlation rule selects. Given the full View of an Identity Manager user and the attributes of an unowned resource account, a confirmation rule returns true if the user owns the account, and false otherwise. Select a rule to test each potential owner of a resource account. If you select No Confirmation Rule, Identity Manager accepts all potential owners without confirmation.
Note
In your environment, if the correlation rule will select at most one owner for each account, then you do not need a confirmation rule.
- Load Only Matching — Select to load into Identity Manager only those accounts that match an existing Identity Manager user. If you select this option, load will discard any unmatched resource account.
- Update Attributes — Select to replace the current Identity Manager user attribute values with the attribute values from the account being loaded.
- Merge Attributes — Enter one or more attribute names, separated by commas, for which values should be combined (eliminating duplicates) rather than overwritten. Use this option only for list-type attributes, such as groups and mailing lists. You must also select the Update Attributes option.
- Result Level — Select a threshold at which the load process will record an individual result for an account:
- Errors only — Record an individual result only when loading an account produces an error message.
- Warnings and errors — Record an individual result when loading an account produces a warning or an error message.
- Informational and above — Record an individual result for every account. This causes the load process to run more slowly.
- In the File to Upload field, specify a file to load, and then click Load Accounts.
Figure 7-2 illustrates the fields and options available in the Load from File screen.
Figure 7-2 Load from File
If an account matches (or correlates with) an existing user, the load process will merge the account into the user. The process will also create a new Identity Manager user from any input account that does not correlate (unless Correlation Required is specified).
The bulkAction.maxParseErrors configuration variable sets a limit on the number of errors that can be found when a file is loaded. By default, the limit is 10 errors. If the maxParseErrors number of errors is found, then parsing stops.
Load from Resource
Use this feature to directly extract and import accounts into Identity Manager according to the load options you specify.
To import accounts, follow these steps:
- In the Administrator interface, click Accounts in the menu, then click Load from Resource.
The “Load Accounts from Resource” page opens.
- Specify the load options on the “Load Accounts from Resource” page.
The load options for this page are the same as those on the “Load from File” page ((more...) ).
ReconciliationUse the reconciliation feature to periodically compare resource accounts in Identity Manager with the accounts actually present on the resources. Reconciliation correlates account data and highlights differences.
Reconciliation in a Nutshell
Because reconciliation is designed for ongoing comparison, it has the following characteristics:
You can also configure reconciliation to launch an arbitrary workflow at each of the following points in processing a resource:
Access Identity Manager reconciliation features from the Resources area. The Resources list shows when each resource was last reconciled and its current reconciliation status.
Note
Reconciliation is carried out by Identity Manager’s reconciler component. For information about reconciler configuration settings, see Reconciler Settings.
About Reconciliation Policies
Reconciliation policies allow you to establish a set of responses, by resource, for each reconciliation task. Within a policy, you select the server to run reconciliation, determine how often and when reconciliation takes place, and set responses to each situation encountered during reconciliation. You can also configure reconciliation to detect changes made natively (not made through Identity Manager) to account attributes.
Editing Reconciliation Policies
To edit a reconciliation policy, follow these steps:
Identity Manager displays the Edit Reconciliation Policy page, where you can make these policy selections:
- Full Reconciliation Schedule — If full mode reconciliation is enabled, it is performed automatically on a fixed schedule. Specify how frequently full reconciliation should be run against resources in the policy.
- Select the Inherit default policy option to inherit the indicated schedule from a higher-level policy.
- Clear the Inherit default policy option to specify a schedule. Use the fields provided to establish a recurring schedule, or, to create a custom adjustment to the reconciliation schedule, use a Task Schedule Repetition rule. For information on creating a Task Schedule Repetition rule, see Using Task Schedule Repetition Rules.
- Incremental Reconciliation Schedule — If incremental mode reconciliation is enabled, it is performed automatically on a fixed schedule.
- Select the Inherit default policy option to inherit the schedule from a higher-level policy.
- Clear the Inherit default policy option to specify a schedule. Use the fields provided to establish a recurring schedule, or, to create a custom adjustment to the reconciliation schedule, use a Task Schedule Repetition rule. For information on creating a Task Schedule Repetition rule, see Using Task Schedule Repetition Rules.
- Attribute-level Reconciliation — Reconciliation can be configured to detect changes made natively (that is, not made through Identity Manager) to account attributes. Specify whether reconciliation should detect native changes to the attributes specified in Reconciled Account Attributes.
- Account Correlation Rule — An account correlation rule selects Identity Manager users that might own each unowned resource account. Given the attributes of an unowned resource account, a correlation rule returns a list of names or a list of attribute conditions that will be used to select potential owners. Select a rule to look for Identity Manager users that may own each unowned resource account.
- Account Confirmation Rule — An account confirmation rule eliminates any non-owner from the list of potential owners that the correlation rule selects. Given the full View of an Identity Manager user and the attributes of an unowned resource account, a confirmation rule returns true if the user owns the account and false otherwise. Select a rule to test each potential owner of a resource account. If you select No Confirmation Rule, Identity Manager accepts all potential owners without confirmation.
- Proxy Administrator — Specify the administrator to use when reconciliation responses are performed. The reconciliation can perform only those actions that the designated proxy administrator is permitted to do. The response will use the user form (if needed) that is associated with this administrator.
Select from one of these response options (available options vary by situation):
- Create new Identity Manager user based on resource account — Runs the user form on the resource account attributes to create a new user. The resource account is not updated as a result of any changes.
- Create resource account for Identity Manager user — Recreates the missing resource account, using the user form to regenerate the resource account attributes.
- Delete resource account and Disable resource account — Deletes/disables the account on the resource.
- Link resource account to Identity Manager user and Unlink resource account from Identity Manager user — Adds or removes the resource account assignment to or from the user. No form processing is performed.
- Do nothing — Select this option if you do not want reconciliation to perform repairs.
You can manually repair any account situation discovered by reconciliation. In the menu click Resources > Examine Account Index. From there you can browse the recorded situation for all accounts which have been reconciled. Right-click on an account and you will see a list of valid repair options. See Examining the Account Index for more information.
- Pre-reconciliation Workflow — Reconciliation can be configured to run a user-specified workflow prior to reconciling a resource. Specify the workflow that reconciliation should run. Select Do not run workflow if no workflow should be run.
- Per-account Workflow — Reconciliation can be configured to run a user-specified workflow after responding to the situation of a resource account. Specify the workflow that reconciliation should run. Select Do not run workflow if no workflow should be run.
- Post-reconciliation Workflow — Reconciliation can be configured to run a user-specified workflow after completing reconciliation for a resource. Specify the workflow that reconciliation should run. Select Do not run workflow if no workflow should be run.
- Explain Situation — If enabled, reconciliation will record additional information explaining how it classified account situations. By default, this option is disabled. Recording explanations will cause the reconciliation process to run longer.
- Error Limit — If enabled, reconciliation will automatically terminate once the specified number of errors have occurred during processing. A value of 0 indicates that there is no limit on errors. De-select the Inherit default policy option to display the Maximum errors allowed field and enter a value.
- Maximum Natively Removed Accounts — This option is a safe-guard that evaluates the number of missing accounts on the resource and, if a threshold is exceeded, prevents the reconciler from unlinking them.
To enable this feature, clear the Inherit default policy checkbox and specify a percentage in the Maximum natively removed accounts allowed field. The threshold must be set to a whole percentage from 0 to 100. (0 turns this feature off.)
If the percentage of removed accounts exceeds the threshold, reconciliation continues all processing not related to the missing accounts and completes with an error.
Click Save to save policy changes.
Starting Reconciliation
Two options are available for starting reconciliation tasks:
To open the Edit Reconciliation Policy page, see Editing Reconciliation Policies and follow the steps.
Reconciliation will run according to the parameters you have set in the policy.
Canceling Reconciliation
To cancel reconciliation, follow these steps:
Viewing Reconciliation Status
There are two main ways to view reconciliation status. To view detailed reconciliation status, open the Reconciliation Summary Results page for a specific resource. Limited reconciliation status is also available directly in the Resource List.
Viewing Detailed Reconciliation Status
View detailed reconciliation status using the Reconciliation Summary Results page.
To view detailed reconciliation status, follow these steps:
Viewing Reconciliation Status in the Resource List
Reconciliation status can also be obtained by viewing the Resource List. (To display the Resource List, open the Administrator interface and click Resources in the menu.)
The Status column reports the following reconciliation status conditions:
- unknown — Status is not known. Results for the latest reconciliation task are not available.
- disabled — Reconciliation is disabled.
- failed — The latest reconciliation failed to complete.
- success — The latest reconciliation completed successfully.
- completed with errors — The latest reconciliation completed, but with errors.
Working with the Account Index
The Account Index records the last known state of each resource account known to Identity Manager. It is primarily maintained by reconciliation, but other Identity Manager functions will also update the Account Index, as needed.
Discovery tools do not update the Account Index.
Searching the Account Index
Search the account index to view the last known state of a given resource account.
To search the account index, follow these steps:
- In the Administrator interface, click Resources in the menu.
- Select the resource in the Resource List for which you want to search the account index.
- Locate the Resource Actions list and select Search Account Index.
The Search Account Index page opens.
- Select a search type, and then enter or select search attributes.
- Resource account name — Select this option, select one of the modifiers (starts with, contains, or is), and then enter part or all of an account name.
- Resource is one of — Select this option, and then select one or more resources from the list to find reconciled accounts that reside on the specified resources.
- Owner — Select this option, select one of the modifiers (starts with, contains, or is), and then enter part or all of an owner name. To search for unowned accounts, search for accounts in the UNMATCHED or DISPUTED situation.
- Situation is one of — Select this option, and then select one or more situations from the list to find reconciled accounts in the specified situations.
- Click Search to search for accounts according to your search parameters. To limit the results of the search, optionally specify a number in the Limit results to first field. The default limit is the first 1000 accounts found.
Click Reset Query to clear the page and make new selections.
Examining the Account Index
It is also possible to view all Identity Manager user accounts and optionally reconcile them on a per-user basis.
To examine the account index, follow these steps:
The table displays all of the resource accounts that Identity Manager knows about (whether or not an Identity Manager user owns the account). This information is grouped by resource or by Identity Manager organization. To change this view, make a selection from the Change index view list.
Working with Accounts
To work with the accounts on a resource, select the Group by resource index view. Identity Manager displays folders for each type of resource. Navigate to a specific resource by expanding a folder. Click + or - next to the resource to display all resource accounts that Identity Manager knows about.
Accounts that have been added directly to the resource since the last reconciliation on that resource are not displayed.
Depending on the current situation of a given account, you may be able to perform several actions. Right-click on an account and you will see a list of valid repair options. You can also view account details or choose to reconcile that one account.
Working with Users
To work with Identity Manager users, select the Group by user index view. In this view, Identity Manager users and organizations are displayed in a hierarchy similar to the Accounts List page. To see accounts currently assigned to a user in Identity Manager, navigate to the user and click the indicator next to the user name. The user’s accounts and the current status of those accounts that Identity Manager knows about are displayed under the user name.
Depending on the current situation of a given account, you may be able to perform several actions. You can also view account details or choose to reconcile that one account.
Using Task Schedule Repetition Rules
Use Task Schedule Repetition Rules to make adjustments to a reconciliation schedule. For example, if you want to push reconciliations scheduled for Saturday to the following Monday, use a Task Schedule Repetition Rule.
Task Schedule Repetition Rules can be used to adjust schedules for both full and incremental reconciliations.
For information on how to select Task Schedule Repetition rules, see Editing Reconciliation Policies.
How Reconciliation Run Times are Scheduled
Upon completing a reconciliation job, the reconciler component checks for its next scheduled run time.
First, the reconciler looks at the default schedule to obtain its next run time. Next, the reconciler runs all applicable Task Schedule Repetition Rules to see if schedule adjustments needs to be made. If an adjustment is needed, the rule schedule overrides the default schedule for that reconciliation.
Note
Task Schedule Repetition Rules cannot overwrite the default schedule. They can only override scheduled start times on a per-job basis.
The “Accept All Dates” Sample Rule
This section describes the built-in sample rule named “Accept All Dates.”
To view the “Accept All Dates” sample rule, follow these steps:
In order for a rule to be listed in the “TaskSchedule Repetition Rule” drop-down menu (on the Edit Reconciliation Policy page), the rule’s subtype attribute must be set to SUBTYPE_TASKSCHEDULE_REPETITION_RULE:
<Rule subtype='SUBTYPE_TASKSCHEDULE_REPETITION_RULE' name='SCHEDULING_RULE_ACCEPT_ALL_DATES'>
As noted previously, Task Schedule Repetition rules can modify the default reconciliation schedule.
The variable calculatedNextDate can either accept the next date, which is calculated in the default manner, or return a different date. As it is written in the sample rule, calculatedNextDate unconditionally accepts the default date:
Code Example 7-1 SCHEDULING_RULE_ACCEPT_ALL_DATES Rule Logic (Excerpt)
<RuleArgument name='calculatedNextDate'/>
<block>
<ref>calculatedNextDate</ref>
</block>
To create a custom schedule, replace the rule logic in between the <block> elements. For example, to change the reconciliation start time to 10:00 AM on Saturdays, include the following JavaScript in between the <block> elements:
In Code Example 7-2, calculatedNextDate is initially set to the default scheduled time. If the next scheduled run date is a Saturday, then the rule schedules reconciliation to start at 10:00. If the next scheduled run date is not a Saturday, Code Example 7-2 returns calculatedNextDate without making any time adjustments, and the default schedule is used.
For more information about creating custom rules for use in Identity Manager, see the “Working with Rules” chapter in Identity Manager Deployment Tools.
Active Sync AdaptersThe Identity Manager Active Sync feature allows information that is stored in an authoritative external resource (such as an application or database) to synchronize with Identity Manager user data. Configuring synchronization for an Identity Manager resource enables it to listen or poll for changes to the authoritative resource.
You can configure how resource attribute changes are flowed into Identity Manager by specifying the Input Form in the resource’s synchronization policy (for the appropriate target object type).
Note
The pages in this chapter focus on how to perform Active Sync tasks using the Administrator interface. To learn about Active Sync in depth, see the “Data Loading and Synchronization” chapter in the Identity Manager Deployment Overview book.
Configuring Synchronization
Identity Manager uses a synchronization policy to enable synchronization for resources.
Editing the Synchronization Policy
Each resource has its own synchronization policy.
To edit or configure synchronization, follow these steps:
Specify the following options in the Edit Synchronization Policy page to configure synchronization:
- Target Object Type — Select the type of users to which the policy applies, either Identity Manager Users or Service Provider Users.
Note
In a Service Provider implementation you must configure a synchronization policy (with Service Provider Users specified as the object type) to enable synchronization of data for those users. For more information about service provider users, see Chapter 17, "Service Provider Administration."
- Scheduling Settings — Use this section to specify the startup method and polling schedule.
Startup Type can be Manual, Automatic, Automatic with Failover, or Disabled:
Use the Start Date and Start Time options to specify when polling begins. Specify the polling cycles by selecting an interval and entering a value for the interval (seconds, minutes, hours, days, weeks, months).
If you set a polling start date and time that is in the future, polling will begin when specified. If you set a polling start date and time that is in the past, Identity Manager determines when to begin polling based on this information and the polling interval. For example:
In this case, the resource will begin polling on July 25, 2005 (the following Monday).
If you do not specify a start date or time, then the resource will poll immediately. If you take this approach, each time the application server is restarted, all resources configured for active synchronization will begin polling immediately. The typical approach, is to set a start date and time.
- Synchronization Servers — In a clustered environment, each server can run synchronization. Select an option to specify which servers will be used to run synchronization for the resource.
- Select Use any available server if it does not matter where synchronization runs. A server will be chosen from the set of possible servers when synchronization starts.
- Select Use the settings in waveset.properties to use servers specified there to run synchronization. (This feature is deprecated.)
- Select Use specified servers, and then select one or more available servers from the Synchronization Servers list, to select specific servers to run synchronization.
- Resource Specific Settings — Use this section to specify how synchronization will determine the data to be processed for the resource.
- Common Settings — Specify the following general settings for data synchronization activities:
- Proxy Administrator — Select the administrator who will process updates. All actions will be authorized through capabilities assigned to this administrator. You should select a proxy administrator with an empty user form.
- Input Form — Select an input form that will process data updates. This optional configuration item allows attributes to be transformed before they are saved on the accounts.
- Rules — You have the option of specifying rules to use during the data synchronization process:
- Process Rule — Select this rule to specify a process rule to run for each incoming account. This selection overrides all other options. If you specify a process rule, the process will be run for every row, regardless of other settings on the resource. It can be either a process name, or a rule evaluating to a process name.
- Correlation Rule — Select a correlation rule to override the correlation rule specified in the resource's reconciliation policy. Correlation rules correlate resource accounts to Identity system accounts.
- Confirmation Rule — Select a confirmation rule to override the confirmation rule specified in the resource's reconciliation policy.
- Resolve Process Rule — Select this rule to specify the name of a Task Definition to run in case of multiple matches to a record in the data feed. This should be a process that prompts an administrator for manual action. It can be a process name or a rule evaluating to a process name.
- Delete Rule — Select a rule, which returns true or false, that will be evaluated for each incoming user update to determine if a delete operation should occur.
- Create Unmatched Accounts — When this option is enabled (true), the adapter will attempt to create accounts that it does not find in the Identity Manager system. If not enabled, the adapter will run the account through the process returned by the Resolve Process Rule.
- Logging Settings — Specify a value for the following logging options:
- Maximum Log Archives — If greater than zero, retain the latest N log files. If zero, then a single log file is re-used. If -1, then log files are never discarded.
- Maximum Active Log Age — After this period of time has elapsed, the active log will be archived. If the time is zero, then no time-based archival will occur. If Maximum Log Archives is zero, then the active log will instead be truncated and re-used after this time period. This age criteria is evaluated independently of the time criteria specified by Maximum Log File Size.
- Log File Path — Enter the path to the directory in which to create the active and archived log files. Log file names begin with the resource name.
- Maximum Log file Size — Enter the maximum size, in bytes, of the active log file. The active log file will be archived when it reaches maximum size. If Maximum Log Archives is zero, then the active log will instead be truncated and re-used after this time period. This size criteria is evaluated independently of the age criteria specified by Maximum Active Log Age.
- Log Level — Enter the level of logging:
Click Save to save the policy settings for the resource.
Editing Active Sync Adapters
Before editing an Active Sync adapter, stop synchronization.
To stop synchronization, follow these steps:
- Open the Edit Synchronization page. (For instructions, see Editing the Synchronization Policy.)
- Under Scheduling Settings, locate Startup Type and select Disabled.
For Service Provider users deselect the Enable Synchronization option.
A warning message will appear to indicate that active synchronization is disabled.
- Click Save.
Disabling synchronization for a resource will result in stopping the synchronization task when the changes are saved.
Tuning Active Sync Adapter Performance
Because synchronization is a background task, Active Sync adapter configuration can affect server performance. Tuning Active Sync adapter performance involves these tasks:
Manage Active Sync adapters through the resources list. Select an Active Sync adapter, and then access start, stop, and status refresh controls actions from the Synchronization section of the Resource Actions list.
Changing Polling Intervals
The polling interval determines when the Active Sync adapter will start processing new information. Polling intervals should be determined based on the type of activity being performed. For example, if the adapter reads in a large list of users from a database and updates all users in Identity Manager each time, consider running this process daily in the early morning hours. Some adapters may have a quick search for new items to process and could be set to run every minute.
Specifying the Host Where the Adapter Will Run
To specify the host where the adapters will run, edit the waveset.properties file. Edit the sources.hosts property to either of the following options:
Active Sync adapters that require more memory and CPU cycles can be configured to run on dedicated servers to help load balance the systems.
Starting and Stopping
Active Sync adapters can be disabled, manually started, or automatically started. You must have the appropriate administrator capability to change Active Sync resources in order to start or stop Active Sync adapters. For information about administrator capabilities, see Capabilities Categories.
When an adapter is set to automatic, the adapter restarts when the application server does. When you start an adapter, it will run immediately and execute at the specified polling interval. When you stop an adapter, the next time the adapter checks for the stop flag, it will stop.
Adapter Logging
Adapter logs capture information about the adapter currently processing. The amount of detail that the log captures depends upon the logging level of the logging you have set. Adapter logs are useful for debugging problems and watching the adapter process progress.
Each adapter has its own log file, path, and log level. You specify these values in the Logging section of the Synchronization Policy for the appropriate user type (Identity Manager or Service Provider).
Deleting Adapter Logs
Adapter logs should be deleted only when the adapter has been stopped. In most cases, make a copy of the log for archive purposes before deleting a log.