Skip Headers
Oracle® Fusion Middleware Deployment Planning Guide for Identity Synchronization for Windows 6.0
11g Release 1 (11.1.1.7.0)

Part Number E28965-01
Go to Documentation Home
Home
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

3 Case Study: Deploying in a High-Availability Environment Over a Wide Area Network Using SSL

The case study provided in this chapter explains how Global Telco implemented Identity Synchronization for Windows at in a four-way Multi-Master Replication (MMR) environment. This chapter also highlights important features of Identity Synchronization for Windows, such as:

3.1 Global Telco Deployment Information

Global Telco, a large company with 500,000 employees world-wide, is using Sun Java System Identity Manager (Identity Manager) to provision users between Active Directory, Directory Server, Oracle RDBMS, Novel NDS, and other systems. They have two main data centers: one in the United States, and the other in Europe.

They have a single Active Directory domain (gt.com) with four domain controllers, and a Sun Java System Directory Server Deployment (dc=gt,dc=com) with four masters and four read-only replicas.

3.1.1 Directory Server Setup

The Sun Java System Directory Server topology includes four masters and four read-only replicas. Directory Server is the corporate directory server used to control access to Web-based applications. The directory server has a single root suffix, dc=gt,dc=com; information about the users are stored in the ou=people, dc=gt,dc=example,dc=com container with uid as the naming attribute.

  • Two masters and two read-only replicas are located in the United States (a separate configuration directory in the United States stores configuration information for these systems).

  • Two masters and two read-only replicas are located in Europe (a separate configuration directory in Europe stores configuration information for these systems).

    All four masters have replication agreements with each other, but the four read-only replicas only have replication agreements with two of the masters as shown in the following figure.

Note:

Identity Synchronization for Windows treats hub replicas the same as read-only replicas. In many scenarios, using a hub replica is preferred to using a read-only replica because a hub can be easily promoted to a master.

Figure 3-1 Data Center Information for Directory Server

Description of Figure 3-1 follows
Description of "Figure 3-1 Data Center Information for Directory Server"

3.1.2 Active Directory Information

The Active Directory deployment has a single domain, gt.com, with two domain controllers located in the United States and two in Europe. The user information is stored in the standard cn=users container in Active Directory (cn=users,dc=gt,dc=com).

The Active Directory samaccountname attribute value matches the Directory Server uid attribute, and the Active Directory domain controller with the PDC FSMO role is located in the United States office.

Figure 3-2 Data Center Information for Active Directory

Description of Figure 3-2 follows
Description of "Figure 3-2 Data Center Information for Active Directory"

Note:

Both ad1-us.gt.com and ad3-eu.gt.com are bridgehead servers which control replication between the two sites.

3.1.3 Requirements

Global Telco wants to achieve the following:

  • The users' passwords for Windows systems must be synchronized with their Directory Server passwords.

  • The users must be able to change passwords using native mechanisms made in either environment, through the Change Password option in the Task Manager dialog on Windows systems, and a web-based portal for Directory Server.

    Identity Synchronization for Windows supports capturing native password changes in Directory Server and Active Directory. Users can continue to change passwords as they always have.

Note:

Passwords can be set in Directory Server by passing a pre-hashed password value. However, Identity Synchronization for Windows cannot synchronize passwords from Directory Server to Windows if the password is pre-hashed. Even in installations without Identity Synchronization for Windows, this is not advisable because it circumvents password policy and password history.

  • Existing Identity Manager functionality must be retained. Identity Manager will continue to provision users to Active Directory and Directory Server.

    Identity Synchronization for Windows requires the users' Directory Server accounts to be explicitly linked to their Windows accounts. This linking is automatically done when Identity Synchronization for Windows is configured to synchronize creations of new users. However, because Identity Manager is provisioning both Active Directory and Directory Server accounts, Identity Synchronization for Windows will not synchronize new users. Global Telco must either run idsync resync command periodically to link newly created users, or Identity Manager must be configured to set the necessary linking attributes when a new Directory Server entry is created.

  • Support for propagating native password changes made in Directory Server to all systems managed by Identity Manager must be added.

    Identity Manager supports synchronizing Active Directory password changes to many other systems because Identity Synchronization for Windows can synchronize password changes from Directory Server to Active Directory. Couple Identity Manager and Identity Synchronization for Windows to synchronize password changes made in Directory Server to any system that Identity Manager supports.

  • High availability for failover redundancy of all services is required in the European office.

    Identity Synchronization for Windows is very robust. Once all components are running, it synchronizes data without losing changes. By default, Identity Synchronization for Windows provides some high availability options such as failover to an alternate Directory Server master, and performing on-demand password synchronization against any Active Directory domain controller. It also includes a watchdog that restarts failed processes.

    However, if the machine with the Identity Synchronization for Windows Core or Connector has a hardware failure, then Identity Synchronization for Windows will not synchronize users until it is re-installed on different hardware.

    This case study addresses Global Telco's HA requirement by installing a completely separate instance of Identity Synchronization for Windows at the European office.

  • All communication must use SSL and trusted certificates where possible.

    Identity Synchronization for Windows supports SSL communication for all over-the-wire communication. By default, it does not require trusted certificates for SSL communication between connectors and directory sources, but it can be configured to require trusted certificates.

3.2 Installation and Configuration Overview

This section gives an overview of the installation and configuration steps for meeting Global Telco's requirements. For details on configuring Identity Manager to coexist with Identity Synchronization for Windows, see Failing Back to the Primary installation.

3.2.1 Primary and Secondary Installations

To provide a high-availability solution, Identity Synchronization for Windows must be installed in two separate environments, one in the United States, and another in Europe. The deployment in the United States is the primary deployment, while the one is Europe is only meant to be used during failover scenarios.

To improve performance, the Identity Synchronization for Windows components are distributed between two machines in each environment. For the deployment in the United States, the Identity Synchronization for Windows Core components are installed on config-us.gt.com, and both connectors are installed on connectors-us.gt.com. For the deployment in Europe, the Identity Synchronization for Windows Core components are installed on config-eu.gt.com and both connectors are installed on connectors-eu.gt.com.

The primary deployment and the various communication paths are shown in the following figure. For simplicity, gt.com is dropped, and only the machine names are shown.

Figure 3-3 Primary Installation of Identity Synchronization for Windows

Description of Figure 3-3 follows
Description of "Figure 3-3 Primary Installation of Identity Synchronization for Windows"

The Directory Server Connector and Active Directory Connector, installed on connectors-us.gt.com, communicate with each other and receive their configuration from the Message Queue that is installed with the Identity Synchronization for Windows Core.

The Active Directory Connector communicates exclusively with the ad1-us.gt.com domain controller, using LDAP. The Directory Server Connector communicates with two Directory Server masters. While it is available, it detects and propagates changes to master1-us.gt.com. If this machine is unavailable, it fails over to master2-us.gt.com to apply changes, but cannot detect further changes made at any master until master1-us.gt.com is available.

Note:

  • If both master1-us.gt.com and master2-us.gt.com are unavailable, then changes synchronized from Active Directory are not applied to the other Directory Server masters, master3-eu.gt.com and master4-eu.gt.com. Identity Synchronization for Windows treats these other masters like read-only replicas, except that external changes to these masters are replicated to the Retrochange Log on master1-us.gt.com and then synchronized to Active Directory.

  • The Identity Synchronization for Windows Directory Server Plugins running on Master1-us and master2-us only communicates with ad2-us, ad3-eu, and ad4-eu if ad1-us is unavailable. Similarly, the other Directory Server masters and replicas only communicate with master2-us if master1-us is unavailable.

Identity Synchronization for Windows's Directory Server Plugin must be enabled on all the eight Directory Server instances, four masters and four read-only replicas.

You can enable Directory Server Plugin using the following:

idsync dspluginconfig -{C/U} -D <bind DN> -w <bind pass word | -> [-h <CD hostname>] [-p <CD port no>] [-s <configuration suffix>] [-Z] [-P <cert db path>] [-m <secmod db path>] [-d <ds plugin hostname>] [-r <ds plug in port>] [-u <ds plugin user>] [-x <ds plugin user password>] [-o <database suf fix>][-q <configuration password | >]

Enter idsync --help for more information.

When a directory server starts up, the Directory Server Plugin establishes a secure connection to the Directory Server Connector. Once the plugin is authenticated, the connector sends the configuration information, and the plugin can send log messages to the central log, through the connector. The configuration includes keys for encrypting modified passwords and Active Directory information for performing on-demand password synchronization.

When a user's Active Directory password changes, Identity Synchronization for Windows sets the dspswvalidate attribute to true in the user's Directory Server entry. This attribute is set because the user can log into any directory server and on-demand password synchronization can originate from any server.

If the user logs into master1-us.gt.com or master2-us.gt.com, then on-demand password synchronization is done directly to the ad1-us.gt.com Active Directory domain controller. Other domain controllers are contacted only if ad1-us.gt.com is unavailable.

If the user logs into one of the other two masters or one of the read-only replicas, then on-demand password synchronization is done against master1-us.gt.com or master2-us.gt.com, and these masters in turn continue the on-demand password synchronization to one of the Active Directory domain controllers.

These two hops are necessary because:

  • A read-only replica cannot update the user's Directory Server entry with the correct password if on-demand password synchronization succeeds

  • With the exception of the preferred and secondary masters, Identity Synchronization for Windows treats all Directory Server instances as read-only replicas.

After the Primary Installation is complete, the second Identity Synchronization for Windows installation is done on the two machines in Europe, config-eu.gt.com and connectors-eu.gt.com as shown in the figure below.

Figure 3-4 Failover Installation while the Primary Installation is Active

Description of Figure 3-4 follows
Description of "Figure 3-4 Failover Installation while the Primary Installation is Active"

The Identity Synchronization for Windows Directory Server Plugins have not been re-installed, so they still receive their configuration from the Directory Server Connector running on connectors-us.gt.com, while the on-demand password synchronization passes through master1-us.gt.com or master2-us.gt.com before reaching the Active Directory domain controllers.

The failover installation remains in this state until Global Telco needs to failover to it. To complete the failover process, the Identity Synchronization for Windows Plugin is enabled on every directory server, which changes its startup configuration to communicate with the Directory Server Connector running on connectors-eu.gt.com.

Figure 3-5 Primary Installation after Reinstalling the Identity Synchronization for Windows Plugins

Description of Figure 3-5 follows
Description of "Figure 3-5 Primary Installation after Reinstalling the Identity Synchronization for Windows Plugins"

Note:

Setting up the secondary installation significantly increases the amount of time required to deploy Identity Synchronization for Windows. However, this up front cost is offset by the ability to quickly failover to the alternate deployment if necessary.

3.2.2 Periodically Linking New Users

Identity Synchronization for Windows is not used to synchronize the creation of new users. Therefore, idsync resync command is executed periodically to establish links between newly provisioned users. An LDAP filter is passed to this command to resynchronize only the subset of users that were created since the command was last executed.

See Periodic idsync resync Operations for more information.

3.2.3 Large Deployment Considerations

Due to the large size of their deployment, Global Telco takes the following steps to increase the performance of Identity Synchronization for Windows.

  • Identity Synchronization for Windows components are distributed across two dedicated machines.

  • The log level is set to INFO except when diagnosing transient problems.

  • The Message Queue Broker's memory limits are increased (see Sun Java System Directory Server Enterprise Edition 11.1.1 5 Installation Guide for more information).

  • The default number of worker threads in the connectors is increased.

  • The idsync resync command is run over batches of users to reduce the peak load on Message Queue.

    Each of these items is discussed in detail in the next section.

3.3 Configuration Walkthrough

This section provides the high-level steps used to configure Identity Synchronization for Windows in an high-availability environment.

Note:

Only important steps are provided and any configuration instructions already discussed in the Example Bank case study have been omitted.

For detailed configuration instructions, see the Sun Java System Directory Server Enterprise Edition 11.1.15 Installation Guide .

3.3.1 Primary Installation

After the Core is installed on config-us.gt.com, the Identity Synchronization for Windows console is started. You configure the Directory Server source first.

Figure 3-6 Configuring the Directory Server Source

Description of Figure 3-6 follows
Description of "Figure 3-6 Configuring the Directory Server Source"

master1-us.gt.com is chosen as the preferred master. The connector communicates with the Directory Server source over SSL.

Figure 3-7 Configuring the Directory Server Source Over SSL

Description of Figure 3-7 follows
Description of "Figure 3-7 Configuring the Directory Server Source Over SSL"

master2-us.gt.com is chosen as the secondary master. The connector communicates with Directory Server over SSL.

Figure 3-8 Configuring Advanced Security Options for the Directory Server Source

Description of Figure 3-8 follows
Description of "Figure 3-8 Configuring Advanced Security Options for the Directory Server Source"

Global Telco requires the strictest security possible, so the Directory Server Connector will require a trusted SSL certificate from the directory server, and the Identity Synchronization for Windows Directory Server Plugins will communicate over SSL to Active Directory. (The Identity Synchronization for Windows Plugins inherit the SSL configuration of the directory server. Therefore, if the Directory Server requires trusted certificates, the plugin can only communicate with Active Directory if it provides a trusted certificate). Enabling these enhanced security options implies additional installation step, outlined below.

Figure 3-9 Configuring the Active Directory Domain

Description of Figure 3-9 follows
Description of "Figure 3-9 Configuring the Active Directory Domain"

ad1-us.gt.com is the PDC FSMO Role Owner, and is selected as the domain with which the controller for the Active Directory Connector will communicate. The connector communicates over SSL.

Figure 3-10 Configuring Failover Active Directory Domain Controllers to Work over SSL

Description of Figure 3-10 follows
Description of "Figure 3-10 Configuring Failover Active Directory Domain Controllers to Work over SSL"

All three remaining domain controllers will be used for failover during on-demand password synchronization.

Figure 3-11 Security Option to Enable for the Active Directory Connector

Description of Figure 3-11 follows
Description of "Figure 3-11 Security Option to Enable for the Active Directory Connector"

Global Telco requires the strictest security possible, so the Active Directory Connector will require a trusted SSL certificate from ad1-us.gt.com. Enabling this advanced security option implies additional installation steps as outlined below.

Figure 3-12 Attribute Modification Flow Setting

Description of Figure 3-12 follows
Description of "Figure 3-12 Attribute Modification Flow Setting"

The only default global setting that is changed is the synchronization of attribute modifications from Active Directory to Directory Server, and from Directory Server to Active Directory.

Figure 3-13 Attribute Setting for Synchronization

Description of Figure 3-13 follows
Description of "Figure 3-13 Attribute Setting for Synchronization"

Only passwords are synchronized. No additional attributes are synchronized.

A single SUL, GT_USERS, is created as shown in Primary Installation.

Figure 3-14 Synchronization User List Creation

Description of Figure 3-14 follows
Description of "Figure 3-14 Synchronization User List Creation"

Active Directory users are stored under the default cn=users,dc=gt,dc=com container. The existing users (Administrator, Guest, TsInternetUser, and iswUser) are excluded from synchronization.

Figure 3-15 Excluding User from the Synchronization Process

Description of Figure 3-15 follows
Description of "Figure 3-15 Excluding User from the Synchronization Process"

The Directory Server users are stored in the default ou=people,dc=gt,dc=com container.

After the configuration is saved, each connector is installed on connectors-us.gt.com, and the Identity Synchronization for Windows Plugin is installed.

bash-2.05# ./idsync printstat -w <password omitted\> -q <password omitted\>
                  Exploring status of connectors, please wait...

                  Connector ID: CNN100
                     Type: Sun Java(TM) System Directory
                     Manages: dc=gt,dc=com (ldaps://master1-us.gt.com:636)
                    (ldaps://master2-us.gt.com:636)
                     State: READY
                     Installed on: connectors-us.gt.com
                     Plugin SUBC100 is installed on ldaps://master1-us.gt.com:636
                     Plugin SUBC101 is installed on ldaps://master2-us.gt.com:636
                     Plugin SUBC102 is installed on ldaps://master3-eu.gt.com:636
                     Plugin SUBC103 is installed on ldaps://master4-eu.gt.com:636
                     Plugin SUBC104 is installed on ldaps://replica1-us.gt.com:636
                     Plugin SUBC105 is installed on ldaps://replica2-us.gt.com:636
                     Plugin SUBC106 is installed on ldaps://replica3-eu.gt.com:636
                     Plugin SUBC107 is installed on ldaps://replica4-eu.gt.com:636

                  Connector ID: CNN101
                     Type: Active Directory
                     Manages: gt.com (ldaps://ad2-us.gt.com:636) (ldaps://ad3-eu.gt.com:636)
                    (ldaps://ad4-eu.gt.com:636) (ldaps://ad1-us.gt.com:636)
                     State: READY
                     Installed on: connectors-us.gt.com

                  Sun Java(TM) System Message Queue Status: Started

                  Checking the System Manager status over the Sun Java(TM) System Message Queue.

                  System Manager Status: Started

                  Remaining Installation and Configuration Steps:

                  1. Install the Sun Directory Server Plugin on every other master 
                  and read-only replica that manage users under dc=gt,dc=com.
                  2. Run 'idsync resync' to establish links between existing Directory Server 
                  and Windows users.
                  3. Start synchronization using the console or the 'idsync startsync' command.
                  SUCCESS

3.3.2 Failover Installation

Once the primary installation is complete, the Identity Synchronization for Windows Core is installed on config-eu.gt.com, and the console is used to configure it.

Figure 3-16 Configuring the Preferred Directory Server

Description of Figure 3-16 follows
Description of "Figure 3-16 Configuring the Preferred Directory Server"

master3-eu.gt.com is the preferred Directory Server master in the failover installation.

Figure 3-17 Configuring the Secondary Directory Server Master

Description of Figure 3-17 follows
Description of "Figure 3-17 Configuring the Secondary Directory Server Master"

master4-eu.gt.com is the secondary directory server master in the failover installation.

Figure 3-18 Configuring the Active Directory Domain Controller

Description of Figure 3-18 follows
Description of "Figure 3-18 Configuring the Active Directory Domain Controller"

ad3-eu.gt.com is chosen as the domain controller with which the Active Directory Connector will communicate.

A warning is displayed stating that the password updates might get slow because ad3-eu.gt.com is not the PDC FSMO role owner. This warning can be ignored because changing the PDC FSMO role to this domain controller is part of the failover procedure. A similar warning is also displayed when the configuration is saved.

Figure 3-19 Configuring Domain Controllers for Failover during On-Demand Synchronization

Description of Figure 3-19 follows
Description of "Figure 3-19 Configuring Domain Controllers for Failover during On-Demand Synchronization"

The remaining domain controllers are selected for failover during on-demand password synchronization.

bash-2.05# /opt/SUNWisw/bin/idsync printstat -q <
omitted password\> -w <omitted password\>
Exploring status of connectors, please wait...
Connector ID: CNN100
Type: Sun Java(TM) System Directory
Manages: dc=gt,dc=com (ldaps://master3-eu.gt.com:636)
(ldaps://master4-eu.gt.com:636)
State: READY
Installed on: connectors-eu.gt.com

Connector ID: CNN101
Type: Active Directory
Manages: gt.com (ldaps://ad1-us.gt.com:636) (ldaps://ad2-us.gt.com:636)
(ldaps://ad4-eu.gt.com:636) (ldaps://ad3-eu.gt.com:636)
State: READY
Installed on: connectors-eu.gt.com

Sun Java(TM) System Message Queue Status: Started

Checking the System Manager status over the Sun Java(TM) System Message Queue.

System Manager Status: Started

Remaining Installation and Configuration Steps:

1. Install the Sun Directory Server Plugin at master ldaps://master3-eu.gt.com:636 
by re-running the installer.
2. Install the Sun Directory Server Plugin at master ldaps://master4-eu.gt.com:636 
by re-running the installer.
3. Install the Sun Directory Server Plugin on every other master and read-only replica 
that manage users under dc=gt,dc=com.
4. Run 'idsync resync' to establish links between existing Directory Server 
and Windows users.
5. Start synchronization using the console or the 'idsync startsync' command.
SUCCESS

3.4 Setting Up SSL

Global Telco requires that all network traffic is encrypted, so SSL is used with trusted certificates for all LDAP connections. This setup includes connections between the following:

Note:

The idsync certinfo command displays the steps for configuring SSL for Identity Synchronization for Windows components, based on the current configuration. It does not have access to each component's certificate database, so it cannot determine if the steps have already been followed.

The output of this command is shown for the primary installation below. The output for the failover installation is identical except that the roles of the US and European machines is reversed.

bash-2.05# /opt/SUNWisw/bin/idsync certinfo -q <omitted password\> -w <omitted password\>
Connector: CNN100
Installation Host: connectors-us
Installation Path: /opt
Certificate Database Location:   /var/opt/SUNWisw/etc/CNN100
**The Directory Server Connector's certificate database must contain the 
CA certificate used to sign Directory Server's SSL certificate. If this 
certificate has not already been added to the connector's certificate 
database, please export the CA certificate and import into Directory Server 
Connector certificate database for server ldaps://master1-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has 
not already been added to the Directory Server's certificate database, please 
export the CA certificate from the Active Directory at ldaps://ad1-us.gt.com:636 
and import into Directory Server certificate database for server 
ldaps://master1-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has 
not already been added to the Directory Server's certificate database, please 
export the CA certificate from the Active Directory at ldaps://ad2-us.gt.com:636 
and import into Directory Server certificate database for server 
ldaps://master1-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has 
not already been added to the Directory Server's certificate database, please 
export the CA certificate from the Active Directory at ldaps://ad3-eu.gt.com:636 
and import into Directory Server certificate database for server 
ldaps://master1-us.gt.com:636.


**The Directory Server Connector's certificate database must contain the 
CA certificate used to sign the Directory Server's SSL certificate. 
If this certificate has not already been added to the connector's certificate 
database, please export the CA certificate and import into Directory Server 
Connector certificate database for server ldaps://master2-us.gt.com:636.

**The Directory Server's certificate database must contain the 
CA certificate used to sign the Active Directory's SSL certificate. 
If this certificate has not already been added to the Directory Server's 
certificate database, please export the CA certificate from the 
Active Directory at ldaps://ad1-us.gt.com:636 and import into 
Directory Server certificate database for server 
ldaps://master2-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has 
not already been added to the  Directory Server's certificate database, please 
export the CA certificate from the Active Directory at 
ldaps://ad2-us.gt.com:636 and import into Directory Server certificate 
database for server ldaps://master2-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has 
not already been added to the Directory Server's certificate database, please 
export the CA certificate from the Active Directory at 
ldaps://ad4-eu.gt.com:636 and import into Directory Server certificate 
database for server ldaps://master1-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has not 
already been added to the Directory Server's certificate database, please 
export the CA certificate from the Active Directory at ldaps://ad3-eu.gt.com:636 
and import into Directory Server certificate database for server 
ldaps://master2-us.gt.com:636.


**The Directory Server's certificate database must contain the CA certificate 
used to sign the Active Directory's SSL certificate. If this certificate has not 
already been added to the Directory Server's certificate database, please export 
the CA certificate from the Active Directory at ldaps://ad4-eu.gt.com:636 and 
import into Directory Server certificate database for server 
ldaps://master2-us.gt.com:636.


Connector: CNN101
Installation Host: connectors-us
Installation Path: /opt
Certificate Database Location: /var/opt/SUNWisw/etc/CNN101


**The Active Directory Connector's certificate database must contain the CA 
certificate used to sign the Active Directory's SSL certificate. If this certificate 
has not already been added to the Active Directory Connector certificate database, 
please export the CA certificate from the Active Directory and import into 
Active Directory Connector's certificate database for server 
ldaps://ad1-us.gt.com:636.

**The Active Directory Connector's certificate database must contain 
the CA certificate used to sign the Active Directory's SSL certificate. If this 
certificate has not already been added to the Active Directory Connector 
certificate database, please export the CA certificate from the Active Directory 
and import into Active Directory Connector's certificate database for server 
ldaps://ad2-us.gt.com:636.


**The Active Directory Connector's certificate database must contain the 
CA certificate used to sign the Active Directory's SSL certificate. 
If this certificate has not already been added to the Active Directory Connector 
certificate database, please export the CA certificate from the Active Directory 
and import into Active Directory Connector's certificate database for server 
ldaps://ad3-eu.gt.com:636.


**The Active Directory Connector's certificate database must contain the 
CA certificate used to sign the Active Directory's SSL certificate. If this 
certificate has not already been added to the Active Directory Connector certificate 
database, please export the CA certificate from the Active Directory and import into 
Active Directory Connector's certificate database for server 
ldaps://ad4-eu.gt.com:636.


SUCCESS

Setting Up SSL summarizes SSL communication between components in this installation, including trust requirements for the primary and failover installations.

Table 3-1 SSL Communication between Components

Component Must Trust Certificates From Required By Comments

Directory Server Connector on connector-us.gt.com

master1-us.gt.com

Primary

Only required only if the Require trusted SSL certificates option is enabled in the console.

 

master2-us.gt.com

Primary

Only required if the Require trusted SSL certificates option is enabled in the console.

Active Directory Connector on connector-us.gt.com

ad1-us.gt.com

Primary

Only required if the Require trusted SSL certificates option is enabled in the console. The output of idsync certinfo erroneously mentions that certificates for the other Active Directory domain controllers are required.

Directory Server Connector on connector-eu.gt.com

master3-eu.gt.com

Failover

Only required if the Require trusted SSL certificates option is enabled in the console.

 

master4-eu.gt.com

Failover

Only required if the Require trusted SSL certificates option is enabled in the console.

Active Directory Connector on connector-eu.gt.com

ad3-eu.gt.com

Primary

Only required if the Require trusted SSL certificates option is enabled in the console. The output of idsync certinfo erroneously mentions that certificates for the other Active Directory domain controllers are required.

master1-us.gt.com

ad1-us.gt.com

Primary

Required for on-demand password synchronization.

 

ad2-us.gt.com

   
 

ad3-us.gt.com

   
 

ad4-us.gt.com

   
 

master3-eu.gt.com

Failover

Required for on-demand password synchronization. idsync certinfo does not mention this requirement.

 

master4-eu.gt.com

   

master2-us.gt.com

ad1-us.gt.com

Primary

Required for on-demand password synchronization.

 

ad2-us.gt.com

   
 

ad3-us.gt.com

   
 

ad4-us.gt.com

   
 

master3-eu.gt.com

Failover

Required for on-demand password synchronization. idsync certinfo does not mention this requirement.

 

master4-eu.gt.com

   

master3-eu.gt.com

ad1-us.gt.com

Failover

Required for on-demand password synchronization.

 

ad2-us.gt.com

   
 

ad3-us.gt.com

   
 

ad4-us.gt.com

   
 

master1-us.gt.com

Primary

Required for on-demand password synchronization. idsync certinfo does not mention this requirement.

 

master2-us.gt.com

   

master4-eu.gt.com

ad1-us.gt.com

Failover

Required for on-demand password synchronization

 

ad2-us.gt.com

   
 

ad3-us.gt.com

   
 

ad4-us.gt.com

   
 

master1-us.gt.com

Primary

Required for on-demand password synchronization. idsync certinfo does not mention this requirement

 

master2-us.gt.com

   

replica1-us.gt.com

replica2-us.gt.com

replica3-eu.gt.com

replica4-eu.gt.com

master1-us.gt.com

Primary

Required for on-demand password synchronization. idsync certinfo does not mention this requirement

 

master2-us.gt.com

   
 

master3-eu.gt.com

Failover

Required for on-demand password synchronization. idsync certinfo does not mention this requirement.

 

master4-eu.gt.com

   

In this installation, Global Telco adds both the CA certificates to the certificate databases of the four connectors and eight directory servers.

Note:

See the Sun Java System Directory Server Enterprise Edition 11.1.1 Installation Guide for detailed instructions on adding certificates to the certificate databases. The Directory Server and connectors must be restarted after the certificates have been added. The Directory Server must be restarted after the Identity Synchronization for Windows Plugin is installed, therefore, it is recommended that you add the CA certificates to the Directory Servers' certificate databases before the Identity Synchronization for Windows Plugin is installed.

3.5 Increasing Connector Worker Threads

By default each Directory Server and Active Directory connector uses four worker threads to apply changes. This value is increased in the Global Telco deployment to improve the connector performance, especially during idsync resync operations. The number of connector threads is stored in the configuration directory in the pswNumOutboundConnectorThreads attribute, present in the pswSunDirectoryGlobals and in the pswActiveDirectoryGlobals entries. Before manually editing the configuration, all Identity Synchronization for Windows consoles must be closed.

To find the pswSunDirectoryGlobals entry, the following command is used:

bash-2.05# ./ldapsearch -b "ou=identitysynchronization,ou=services, dc=gt,dc=com" 
-D "cn=Directory Manager" -w <omitted password\> "(&(objectclass=pswSunDirectoryGlobals)
(pswversion\>=0))"  pswNumOutboundConnectorThreads

dn: cn=136,ou=Sun,ou=Globals,cn=active[13],ou=GlobalConfig,ou=1.1, 
ou=IdentityS ynchronization,ou=Services,dc=gt,dc=com
pswNumOutboundConnectorThreads: 4

The entry that must be modified is:

cn=136,ou=Sun,ou=Globals,cn=active[13],ou=GlobalConfig,ou=1.1,
ou=IdentitySynchronization,ou=Services,dc=gt,dc=com.

To find the pswActiveDirectoryGlobals entry, the following command is used:

bash-2.05# ./ldapsearch -b "ou=identitysynchronization,ou=services, 
dc=gt,dc=com" -D "cn=Directory Manager" -w <omittied password\> 
"(&(objectclass=pswActiveDirectoryGlobals)(pswversion\>=0))"  
pswNumOutboundConnectorThreads dn: cn=110,ou=ActiveDirectory,ou=Globals,cn=active[13],
ou=GlobalConfig, ou=1.1, 
ou=IdentitySynchronization,ou=Services,dc=gt,dc=com
pswNumOutboundConnectorThreads: 4

      

The entry that must be modified is:

cn=110,ou=ActiveDirectory,ou=Globals, cn=active[13],ou=GlobalConfig,ou=1.1,
ou=IdentitySynchronization,ou=Services,dc=gt,dc=com.

These two entries are modified to increase the number of threads to a maximum of 20:

bash-2.05# ./ldapmodify -D "cn=Directory Manager" -w <omitted password\>
dn: cn=136,ou=Sun,ou=Globals,cn=active[13],ou=GlobalConfig,ou=1.1, 
ou=IdentitySynchronization,
ou=Services,dc=gt,dc=com
changetype: modify
replace: pswNumOutboundConnectorThreads
pswNumOutboundConnectorThreads: 20

modifying entry cn=136,ou=Sun,ou=Globals,cn=active[13],ou=GlobalConfig, ou=1.1,
ou=IdentitySynchronization,ou=Services,dc=gt,dc=com
dn: cn=110,ou=ActiveDirectory,ou=Globals,cn=active[13],ou=GlobalConfig, ou=1.1,
ou=IdentitySynchronization,ou=Services,dc=gt,dc=com
changetype: modify
replace: pswNumOutboundConnectorThreads
pswNumOutboundConnectorThreads: 20
modifying entry cn=110,ou=ActiveDirectory,ou=Globals,cn=active[13], 
ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=gt,dc=com

      

After these values are changed, the Identity Synchronization for Windows daemon on the Core machine is restarted to notify the System Manager to pick up the new configuration.

Note:

Increasing the number of connector threads also increases the maximum number of LDAP connections that the connector will keep open to the directory.

3.6 Aligning Primary and Failover Configurations

Even though the primary and failover installations have similar configurations, there are some generated configuration parameters that differ for the two deployments:

3.6.1 Setting Multiple Passwords for uid=PSWConnector

After installing the Directory Server Connector for the primary installation, but before installing the Directory Server Connector for the failover installation, the password for the uid=PSWConnector user is retrieved and saved:

bash-2.05# ./ldapsearch -h master1-us -b "dc=gt,dc=com" -D "cn=Directory Manager" 
-w <omitted password\> "(uid=PSWconnector)" userpassword
version: 1
dn: uid=PSWConnector,dc=gt,dc=com
userpassword: {SSHA}OUYr10Y2mHIyZfyVLM4O0nYi4UZGNSAVlAERRg==

         

{SSHA}OUYr10Y2mHIyZfyVLM4O0nYi4UZGNSAVlAERRg== is the password that the Primary Directory Server Connector uses to connect to the directory server. Installing the Directory Server Connector for the Failover installation overwrites this password. At this point, we retrieve the entry again:

bash-2.05# ./ldapsearch -h master1-us -b "dc=gt,dc=com" -D "cn=Directory Manager" 
-w <omitted password\> "(uid=PSWconnector)" userpassword
version: 1
dn: uid=PSWConnector,dc=gt,dc=com
userpassword: {SSHA}k9AFSUGsY1NK038PvIB4lJzVNb0sQHh4JHJXFQ==

         

{SSHA}k9AFSUGsY1NK038PvIB4lJzVNb0sQHh4JHJXFQ== is the password that the Failover Directory Server Connector users to connect to the directory server. At this point, the Directory Server Connector for the primary installation will no longer be able to log into the directory, so we modify the entry to include both passwords.

bash-2.05# ./ldapmodify -h master1-us -D "cn=Directory Manager" 
-w <omitted password\>
dn: uid=PSWConnector,dc=gt,dc=com
changetype: modify
replace: userpassword
userpassword: {SSHA}OUYr10Y2mHIyZfyVLM4O0nYi4UZGNSAVlAERRg==
userpassword: {SSHA}k9AFSUGsY1NK038PvIB4lJzVNb0sQHh4JHJXFQ==
modifying entry uid=PSWConnector,dc=gt,dc=com

         

Once this is complete, both Directory Server Connectors will be able to log into the directory. To verify this, stop and restart the Identity Synchronization for Windows daemon for the primary installation on connectors-us.gt.com, and for the failover installation on connectors-us.gt.com. Once the connectors start and receive their configuration, they will open a connection to the directory. If there are any problems with the credentials, those will be reported in the central logs.

Note:

Every time the Directory Server Connector is installed, a new password is generated and written to the uid=PSWConnector entry. If either Directory Server Connector is uninstalled and re-installed, this procedure must be followed again. Also, if the Directory Server Connector for the failover installation was installed before the primary uid=PSWConnector password was retrieved, then save the current uid=PSWConnector password (for the failover configuration), uninstall and reinstall the Primary Directory Server Connector, and then retrieve the current uid=PSWConnector password (for the primary configuration).

3.7 Initial idsync resync Operation

The next step in the installation and configuration process is to run idsync resync. In this deployment, idsync resync:

Note:

If you need to increase the log level to debug an issue, then it is recommended that you use the -a option during idsync resync operation to restrict the idsync resync operation to only the affected users.

3.7.1 Initial idsync resync Operation for Primary Installation

When you run idsync resync for the first time, the Active Directory users are divided into five batches to reduce the peak load on Message Queue. Use the -a <ldap-filter\> option with idsync resync to limit the scope of the idsync resync operation to a subset of Active Directory users. When dividing users, the following must be ensured:

  • The union of the LDAP filters includes every user that should be synchronized.

    For example, all characters may not be English or even ASCII, so resynchronization based on the 26 letters of the English alphabet, will skip users with non-English characters.

  • The LDAP filters can overlap, however, a significant overlap will increase the total running time of the operation.

  • Batch sizes are not too large; otherwise, the Message Queue Broker will reach its memory limits.

  • The attributes in the filter are indexed in Active Directory. Consult Microsoft's documentation to determine which Active Directory attributes are indexed. The standard set of indexed attributes includes cn, sn, and samaccountname.

  • Certain types of searches on indexed attributes can be expensive. For example, an absence search, (!(sn=*)), will result in a time-out error even though it is indexed.

  • If the filter includes attributes that are not mandatory, for example, sn, then all entries must have a value for that attribute, to avoid entries being overlooked.

  • If a user attribute used in the filter can change value during the operation, then users whose attributes change might not be synchronized. For example, if an entry's sn value changes from Smith to Anderson after the first batch of entries have been synchronized, then this user will be skipped. The script below exploits Active Directory's uSNChanged attribute to resynchronize all entries that changed during the sequence of idsync resync operations.

Note:

Running idsync resync in batches will not significantly increase the total time required to do a full idsync resync operation, and might even speed the operation as it prevents the Message Queue Broker from thrashing as it reaches its memory limits.

Only a single idsync resync operation can be run at a time, so these operations cannot be run in parallel.

The -a <ldap-filter\> operation always applies to the source of the idsync resync operation. When users are linked with the -f option, Active Directory must be the source of the operation, so we only consider how to divide users based on the available Active Directory attributes.

Due to these requirements, Global Telco is resynchronizing its users based on their cn attribute. cn is a mandatory attribute and is indexed. Users are categorized into the following groups

  • cn <= a

  • a <= cn <= d

  • d <= cn <= h

  • h <= cn <= m

  • m <= cn <= s

  • s <= cn

Note:

If a user's cn exactly matches one of the boundary conditions (for example, cn=d), then the user will be synchronized twice, but this introduces negligible overhead and does not cause any problems.

Global Telco then uses the following Perl script to iterate through the idsync resync operations. The resync-batch.pl script first retrieves the current maximum uSNChanged from Active Directory. The specified Active Directory host must be the same host as the one the Active Directory Connector uses for communication, because uSNChanged values differ between Active Directory domain controllers. The script then iterates through each idsync resync operation, pausing for one minute between the operations to allow the connectors to reset and settle. The last search filter will match all users that changed during the other idsync resync operations.

#! /usr/bin/perl
#
# This script executes a sequence of resync operations.
# Each resync operation occurs on a subset of users
# based on an LDAP filter.
#
# Parameters specific to this deployment
#
my $linkFile = "/var/opt/SUNWisw/samples/linkusers-simple-gt.cfg";
my $idsync = "/opt/SUNWisw/bin/idsync";
my $adHost = "ad1-us.gt.com";
my $secBetweenOps = 60;
my $dsPassword = "<
            omitted password\>";
 my $configPassword = "<
            omitted password";

# Guard against users whose cn changes during this operation
# by retrieving the maximum uSNChanged value before we start
# the operation, and later running resync for only these users.
my $origUsn = "ldapsearch -h $adHost -b '' -s base '(objectclass=*)' \\
highestCommittedUSN | grep highestCommittedUSN | sed 's/[^0-9]//g'";
chomp $origUsn;
# Run each user in a batch
my @BATCH_FILTERS =  ("
            (cn<=a)",             # cn <= a
            "(&(cn\>=a)(cn<=d))", # a <= cn <= d
            "(&(cn\>=d)(cn<=h))", # d <= cn <= h
            "(&(cn\>=h)(cn<=m))", # h <= cn <= m
            "(&(cn\>=m)(cn<=s))", # m <= cn <= s
            "
            (cn\>=s)",            # s <= cn
            "(uSNChanged\>=$origUsn)",# modified users);
# Run each user in a batch
for my $filter (@BATCH_FILTERS) {
       print "Running resync with filter '$filter'.\\n";
       my $command = "$idsync resync -a '$filter' -f $linkFile"."
       -i NEW_LINKED_USERS -q $configPassword -w $dsPassword";
       system($command);
       print "Waiting $secBetweenOps before next iteration.\\n";
       sleep $secBetweenOps;
}

         

Notice that the -i NEW_LINKED_USERS option is passed to idsync resync, which resets all Directory Server passwords to match their Active Directory counterparts after the entries are linked.

Note:

If Global Telco did not want to reset the passwords for the Directory Server accounts, they would not run this command with the -k option, which only links users. The idsync resync command primes the object cache database in the Active Directory Connector, which is used to detect changes to entries. When idsync resync is run with the -k option, the object cache is not primed, and if the connector detects a change to an existing user, it will erroneously assumes it is a new user. If you run idsync resync -k, then you must run the idsync resync command again with the -u option, which updates the object cache only.

3.7.2 Initial idsync resync Operation for Failover Installation

You must run the idsync resync command also for the failover installation. The links between the entries are already established and idsync resync is only required to prime the object cache database for the Active Directory Connector. Therefore, it is run with the -u option. To avoid overloading Message Queue with log messages: the Message Queue Broker's maximum memory is increased, the Identity Synchronization for Windows log level is changed to INFO, and idsync resync is run over the entire user population in batches. The script above is modified slightly and run on the failover installation Core machine, config-eu.gt.com. The parameters at the top of the script are changed, and the -i NEW_LINKED_USERS option is replaced with just -u.

3.8 Periodic idsync resync Operations

3.8.1 Periodic idsync resync Operation for Primary Installation

As Global Telco uses Identity Manager to provision users and has not configured Identity Synchronization for Windows to synchronize the creation of new users, idsync resync must be run periodically to establish links between recently created users. If these users are not linked, then modifications to their passwords will not synchronize. Running a full idsync resync operation for 500 thousand users could take a few hours because synchronization of new changes is delayed during a idsync resync operation. It is unacceptable to have Identity Synchronization for Windows off-line for so long, so Global Telco uses Active Directory's usnCreated attribute to synchronize only those users that have been created recently. They use the resync-recent.pl script given below, which is run every hour, to synchronize users that have been created in the last three days. This script is run over users created in the last three days instead of only the last hour because a new employee might not have a Directory Server account until a few days after the Active Directory account is created.

The script is run periodically with the following arguments, using cron:

resync-recent.pl ad1-us.gt.com usnCreated 
-f /var/opt/SUNWisw/samples/linkusers-simple-gt.cfg 
-i NEW_LINKED_USERS -b -w - -q - < /var/opt/SUNWisw/passwords

The first argument is the name of the Active Directory domain controller that the connector communicates with, the second argument determines whether the idsync resync command should be only run on entries that were created recently, and the remaining arguments are passed directly to resync. In this case, the Directory Server password is reset for all users that are newly linked. The - value used for the two password options directs the idsync resync command to read the values from STDIN; it prevents passwords from appearing in the command line and being available to anyone on the system via commands such as ps. The configuration directory password and the configuration password are written to /var/opt/SUNWisw/passwords, and this file is then protected with the strictest file system permissions.

resync-recent.pl script:
#!/usr/bin/perl

# This sample script shows how to run idsync resync only on users
# that have changed recently and can be used to reduce the
 # impact of running resync frequently.  By default it
 # only resync's users that were modified or created in
 # the last three days.  The script stores a daily history
 # of Active Directory's highestCommittedUSN attribute.
  #
# The arguments of the command are
#
   #  <Active Directory-domain-controller-name\>
 #  (usnChanged|usnCreated)
#  [args-for-resync]
#
# To link users that were created recently run the command
#  ad.example.com usnCreated -k -f link.cfg -q <pw\> -q <pw\>
#
# To synchronize all users that were modified recently
#  ad.example.com usnChanged -q <pw\> -q <pw\>
#
 # To prime the object cache for users that were modified recently
#  ad.example.com usnChanged -u -q <pw\> -q <pw\>
#
 # NOTE: this script is only provided as a guide.
# It must be adapted to the specific Identity
 # Synchronization for Windows environment by
 # changing the paths and options below as appropriate
# and adding additional error handling code.
#
my $USAGE = "USAGE: resync-recent.pl ".
    "<Active Directory-domain-controller-name\> ".
    "(usnChanged|usnCreated) [args-for-resync]\\n";

my $IDSYNC = "/opt/SUNWisw/bin/idsync";
my $USN_FILE = "/opt/SUNWisw/bin/usnHistory";
my $MAX_DAYS_HISTORY = 3;

         

#
# SCRIPT BEGIN
#

#
# Argument parsing
#
my $adDomainController = shift @ARGV or die "$USAGE";
my $usnSearchAttr = shift @ARGV or die "$USAGE";
$usnSearchAttr =~ /usnChanged/i or
     $usnSearchAttr =~ /usnCreated/i or
    die "$USAGE\\n";
my $resyncArgs = getArgsAsString(@ARGV);

#
# Read the highestCommittedUSN history and add today's
# date to it if not it's there and then write the history
# out.
#
my %usnHistory = readUsnHistory();
my $today = getCurrentDate();
if (!$usnHistory{$today}) {
    $usnHistory{$today} = getHighestUsn($adDomainController);
}
writeUsnHistory(%usnHistory);

#
# Run the resync command based on the oldest usnChanged
# value in the history.
#
my $oldestUsn = getOldestUsn(%usnHistory);
my $filter = "($usnSearchAttr\>=$oldestUsn)";
my $resyncCmd = "$IDSYNC resync -a \\"$filter\\" $resyncArgs";
print "Running $resyncCmd\\n";
system($resyncCmd);

#
# SCRIPT END
#

         

#
# Return the current date as a string, e.g. 2004/03/04
#
sub getCurrentDate {
    my ($day, $month, $year) = (localtime(time))[3,4,5];
    $month++;
    $year += 1900;
    return sprintf "%d/%02d/%02d", $year, $month, $day;
}
#
# Searches the root DSE at the specified host and returns
# the highestCommittedUSN value.
#
sub getHighestUsn {
    my $adHost = shift @_;
    my $cmd = "ldapsearch -h $adHost -b \\"\\" -s base ".
        "\\"(objectclass=*)\\" highestCommittedUSN | ".
        "grep highestCommittedUSN | sed \\"s/[^0-9]//g\\"";
    my $highestUsn = '$cmd';
    chomp $highestUsn;
    print "highestCommittedUSN at $adHost is $highestUsn.\\n";
    return $highestUsn;
}

#
# Converts the command line args into a string.
#
sub getArgsAsString {
    my $args = "";
    for my $arg (@_) {
        $args .= " \\"$arg\\"";
    }
    return $args;
}

#
# Returns the oldest usnChanged value from the history.
#
sub getOldestUsn {
    my %usnHistory = @_;

         

    my @dates = getHistoryDates(%usnHistory);

        # Return the last element.
    return $usnHistory{$dates[$#dates]};
}

#
# Return a sorted list of the dates in the history.
#
sub getHistoryDates {
    my %history = @_;
    return reverse sort(keys %usnHistory);
}

#
# Writes the most recent daily history of highestCommittedUSN.
# No more than $MAX_DAYS_HISTORY days history is recorded.
#
sub writeUsnHistory {
    my %usnHistory = @_;
    if (!open(OUT, "\>$USN_FILE")) {
        print STDERR "Could not open $USN_FILE for writing.\\n";
        return;
    }

        # Write the history up to the last $MAX_DAYS_HISTORY days.
    my @dates = getHistoryDates(%usnHistory);
    for (my $i = 0;
         $i <= $#dates and $i < $MAX_DAYS_HISTORY;
         $i++)
     {
        my $date = $dates[$i];
        print OUT "$date:$usnHistory{$date}\\n";
            }

    close(OUT);
}

#
# Reads the daily history of highestCommittedUSN
#

         

sub readUsnHistory {
    my %usnHistory = ();
    if (!open(IN, "<$USN_FILE")) {
        print STDERR "Could not read history from $USN_FILE.\\n";
        return %usnHistory;
    }

    while (my $line = <IN\>) {
        chomp $line;
        my ($date, $usn) = split /:/, $line;
        $usnHistory{$date} = $usn;
    }
    close(IN);

    return %usnHistory
}

         

An alternative to running resync periodically is to modify the provisioning process to set the necessary link information when the Directory Server entry is provisioned. The process is straightforward:

  • Add the dspswuser objectclass to the user entry.

  • Set the dspswuserlink attribute to match the user's objectguid attribute from Active Directory.

  • Set the dspswvalidate attribute to true to force on-demand password synchronization. This step is optional.

3.8.2 Periodic idsync resync Operation for Failover Installation

To speed the failover process, the idsync resync operation is run periodically on the failover installation to keep the object cache database in the Active Directory Connector up-to-date. If the object cache is not kept up-to-date, then the Active Directory Connector will detect and propagate many changes that were already synchronized by the primary installation. Not keeping the object cache database up-to-date will significantly increase the load, and will place a heavier load on Directory Server during the failover scenario.

The same resync-recent.pl script is used in this scenario except that it is run from the failover installation Core machine, config-eu.gt.com. The cron command is used to run the script daily with the following arguments. The -u option is specified to only update the Active Directory Connector's object cache.

resync-recent.pl ad3-eu.gt.com usnCreated -u -b -w - -q - < /var/opt/SUNWisw/passwords

Note:

The more often this script is run in the failover environment, the more likely changes will be lost during the failover process. idsync resync -u should not be run after the primary installation fails. If the command is run often (for example, every hour), then it is likely that it will be run while the primary installation has failed, but the failure has not yet been detected. As this script keeps track of a three-day history of the highestCommittedUSN values, it could be updated to search for entries that were modified in the last three days but not modified in the last day. As long as the primary installation failure is detected within one day and the cron job of this script was stopped, no Active Directory changes are lost.

3.9 Configuring Identity Manager

For details on configuring the Sun Java System Identity Manager to coexist with Identity Synchronization for Windows, see Failing Back to the Primary installation.

3.10 Understanding the Failover Process

The goal of the failover process is to avoid losing any changes that occurred between the time the primary system went down and the failover system was brought up. If changes are lost, then Identity Synchronization for Windows should at least report the changes that are lost.

Once synchronization is started for the first time, an Identity Synchronization for Windows Connector can continue to detect changes that occur:

3.10.1 Directory Server Connector

The Directory Server Connector persistently stores the changenumber of the last retro changelog entry that it has processed. When the connector begins detecting changes again, it can process all changes that occurred when it was not running, by starting with the last changenumber that it processed. This changenumber is stored in the connector's persist directory, for example, /var/opt/SUNWisw/persist/ADP100/accessor.state. If this file does not exist, then the connector will detect only new changes in Directory Server.

This way of processing the changes has an effect on the failover process. If synchronization is never started for the failover installation, then the file will not exist, and the Directory Server Connector will only detect new changes. Changes that occurred during the failover process are lost. However, initializing the accessor.state file presents its own problems. If synchronization is started immediately after installation to produce the accessor.state file, and then stopped, when synchronization is started after failing over, the Directory Server Connector will try to process all Retro changelog entries. To strike a balance between these two options, the Retro changelog Plugin on the failover installation's preferred master (master3-eu.gt.com) is configured to only store changes for one day (this limit is increased during the failover process). Once synchronization is started for the failover installation, the Directory Server Connector only processes one day's worth of changes in the directory server.

The Sun Java System Directory Server Administration Guide contains instructions on how to enable trimming of the retro changelog. Essentially, the nsslapd-changelogmaxage attribute on the cn=Retro Changelog Plugin, cn=plugins,cn=config entry is modified and Directory Server is restarted. Setting this value to 1d trims entries that are older than one day. The value can be modified from the command line using ldapmodify or by directly editing the Directory Server's dse.ldif file while Directory Server is not running.

3.10.2 Active Directory Connector

To allow Active Directory to detect changes that occurred while it was stopped, the Active Directory Connector persistently stores the usnchanged value of the last change that it processed. When it begins processing changes again, it only needs to examine entries with an usnchanged value larger than the previous one that it has processed. Like the Directory Server Connector, the Active Directory Connector first stores this usnchanged high-water mark when synchronization is started for the first time. Therefore, as part of preparing the failover installation, synchronization is started to allow the Active Directory connector to checkpoint.

When failing over and synchronization is restarted for the failover installation, the Active Directory Connector will process all entries that were modified because it was last started. Even though this will be thousands of entries, the processing will be fast because the Active Directory's object cache database is up-to-date with most of the changes. Only the changes made since the last time that idsync resync command was run will be processed.

3.11 Initializing the Connector State

To initialize the connector state for the failover configuration, synchronization must be started. Before synchronization can be started, the Identity Synchronization for Windows Plugin must be enabled on both master3-eu.gt.com and master4-eu.gt.com to point to the failover configuration. Once the plugin has been enabled and the directory servers have been restarted, synchronization can be started. Verify that both connectors have entered the SYNCING state using the console or the idsync printstat command:

bash-2.05# ./idsync printstat -w <password omitted\> -q <password omitted\>
Exploring status of connectors, please wait
Connector ID: CNN100
     Type: Sun Java(TM) System Directory
     Manages:  dc=gt,dc=com (ldaps://master3-eu.gt.com:636)
    (ldaps://master4-eu.gt.com:636)
     State: SYNCING
     Installed on:  connectors-eu.gt.com
     Plugin SUBC100 is installed on ldaps://master3-eu.gt.com:636
     Plugin SUBC101 is installed on ldaps://master4-eu.gt.com:636

Connector ID: CNN101
     Type: Active Directory
     Manages: gt.com (ldaps://ad1-us.gt.com:636) (ldaps://ad2-us.gt.com:636)
    (ldaps://ad4-eu.gt.com:636) (ldaps://ad3-eu.gt.com:636)
     State: SYNCING
     Installed on:  connectors-eu.gt.com

Sun Java(TM) System Message Queue Status:  Started

Checking the System Manager status over the Sun Java(TM) System Message Queue.

System Manager Status:  Started
SUCCESS

      

Once synchronization has started, modify a user password both in Active Directory and in Directory Server and it will force the connectors to persist their state. To verify, do the following:

Directory Server Connector: Check for the presence of the /var/opt/SUNWisw/persist/ADP100/accessor.state file. And check that the highestacknowledgedchangenumber value stored in the file is not -1. (To determine the appropriate ADP subdirectory of persist, find the connector ID using the console or idsync printstat, and then replace CNN with ADP in the connector ID.)

Active Directory Connector: Check that the Active Directory Connector actually propagated the change. There should be an INFO message in the central log that includes the usnchanged value, for example,

[05/Nov/2004:14:07:38.982 -0600] INFO    18  CNN101 connectors-eu  
"The agent is sending the following inbound action to MQ: 
Type: MODIFY SUL: GT_USERS {Data Attrs: } {Other Attrs: cn: Jane Test 
dn: CN=Jane Test,CN=Users,DC=gt,DC=com objectclass: top,person, 
organizationalPerson, user dspswuserlink: Rwyr9YEFk0WYxbFP5Nnrjg== pwdlastset: 
127441696561778218 samaccountname: 3aa00test100001 sn: test100001 usnchanged: 120831 
whenchanged: 20041105230736.0Z passwordchanged: TRUE}." 
(Action ID=CNN102-1000A5846CB-5, SN=2)

Once you have verified that both the connectors have check-pointed their state, stop synchronization for the failover installation, and then reinstall the Directory Server Plugins on master3-eu.gt.com and master4-eu.gt.com to point to the primary configuration.

3.12 Failover Installation Maintenance

Once the connectors' state has been persisted, little maintenance is done on the failover installation. idsync resync -u should be run periodically as described in the sectionPeriodic idsync resync Operation for Failover Installation also be made at the failover installation.

Note:

If any additional attributes are added to the list of synchronized attributes, then a full idsync resync -u operation should be executed and not an incremental one. If this is not done, then when the Active Directory Connector is started after failover, it will incorrectly detect modifications to the added attribute because its object cache database did not store the previous value.

3.13 When to Failover

Identity Synchronization for Windows is a background system that with one exception is not user-facing. Therefore, if it is temporarily unavailable, for example, due to routine hardware maintenance, then most users will be unaffected. Once the system is restored, Identity Synchronization for Windows will synchronize all changes that were made while it was unavailable. The user-facing aspect is the on-demand password synchronization performed from the Directory Server Plugin to Active Directory. If on-demand password synchronization fails, then the user will not be able to log into Directory Server. Therefore, Identity Synchronization for Windows provides more availability options for this area. The Directory Server Plugin can be configured to authenticate to any Active Directory domain controller, so even if all but one Active Directory domain controller is down, on-demand password synchronization will still succeed.

Note:

The Directory Server Plugins receive their configuration from the Directory Server Connector over an encrypted channel. This configuration, which includes the location of the Active Directory domain controllers and credentials, is cached in memory by the plugin, so even if the Directory Server Connector is unavailable, it will still be able to connect to Active Directory. However, if Directory Server is restarted, then the plugin's cached configuration is lost, and on-demand synchronization at that Directory Server will fail until the Directory Server Connector is available.

Depending on the size of the deployment, the failover procedure might take anywhere from minutes to over an hour to perform. Therefore, the failover procedure should not be undertaken if the Identity Synchronization for Windows outage is expected to be short and temporary, for example, during the system restart of the Identity Synchronization for Windows Core machine. Failover is recommended only in situations where Identity Synchronization for Windows must be completely re-installed or a complete idsync resync operation must be run over a large population.

For example:

3.14 Failing Over

The failover process, at a high-level, only involves the following steps:

3.14.1 Stopping Synchronization at the Primary Installation

Before starting synchronization at the failover installation, stop synchronization at the primary installation to prevent unwanted interaction between the two systems. Depending on the reasons for failing over this is accomplished in different ways. If the primary Identity Synchronization for Windows installation is still operating properly, for example, failing over because a domain controller or Directory Server is down, then just stop synchronization using the console or idsync stopsync. Otherwise, stopping the Identity Synchronization for Windows daemon (on Solaris) or service (on Windows) on each Identity Synchronization for Windows system that is still operational is recommended.

3.14.2 Starting Synchronization at the Failover Installation

After synchronization is stopped at the primary installation, it must be started at the failover installation using the console or idsync startsync.

Note:

The Directory Server Connector will not enter the SYNCING state until the Directory Server Plugins are re-installed on the preferred and secondary masters (master3-eu.gt.com and master4-eu.gt.com).

The Active Directory Connector will need to process every entry in Active Directory that was modified since it was last started, it might take several minutes for it to begin propagating changes to Directory Server. Setting the log level to INFO before starting synchronization can reduce the impact of the Active Directory Connector having to catch up.

3.14.3 Re-enabling the Directory Server Plugins

To complete the failover process, the Directory Server Plugin is re-enabled on each Directory Server, which ensures:

  • The plugins running on the masters use the encryption key from the failover installation to encrypt password changes.

  • All directory servers receive updated on-demand password synchronization configuration

  • Logging done by the plugins is sent to the Central Logger of the failover installation.

The plugins must be re-enabled in this order:

  1. Failover installation's preferred master.

  2. Failover installation's secondary master.

  3. All other masters.

  4. All read-only replicas.

    Note:

    The order in which the Directory Server Plugins are enabled is important. If they are enabled in the wrong order, on-demand synchronization requests could loop between two masters, tying up all Directory Server connections.

    When re-enabling the plugins, make sure to specify the configuration directory of the failover installation, for example, config-eu.gt.com

This reinstallation procedure can be automated by doing more work ahead of time:

  1. Install the Directory Server Plugins for the Failover configuration.

  2. Export the plugins' configuration for each master from the cn=pswsync,cn=plugins,cn=config tree and it includes two entries.

  3. Re-enable the Directory Server Plugins for the Primary configuration.

To failover:

  1. Delete the cn=pswsync,cn=plugins,cn=config tree.

  2. Add the failover installation entries using ldapmodify.

  3. Restart the directory server.

3.14.4 Changing the PDC FSMO Role Owner

This step is optional. If the Active Directory Connector in the failover installation is configured to communicate with a domain controller that does not have the PDC FSMO role, then synchronization from Active Directory will be delayed due to the Active Directory replication latency. To avoid this delay, the PDC FSMO role can be transferred to the domain controller that the connector is accessing.

3.14.5 Monitoring the Logs

After the Failover process is complete, monitor the central error log of the failover installation for any unexpected warnings. Warnings similar to the following will likely appear:

[08/Nov/2004:07:58:24.803 -0600] WARNING 25  CNN100 connectors-eu  
"Unable to obtain password of user CN=Jane Test,OU=people,DC=gt,DC=com, 
because the password was encoded by a previous installation of 
Identity Synchronization for Windows Directory Server 
Plugin. The password of this user cannot be synchronized at this time. 
Update the password of this user again in Directory Server."

         

These warnings occur for each password update in the Retro changelog that was made before the Directory Server Plugin was re-installed because the Primary Directory Server Plugin was configured to use a different encryption key from the failover Directory Server Plugin. Many of these password updates were likely synchronized by the primary installation before it went off line, but those that occurred after the primary installation went offline cannot be recovered. These users must change their password either in Active Directory or Directory Server to synchronize their passwords.

3.14.6 Failing Back to the Primary installation

The procedure for failing back to the primary installation is identical.