The data managed by your Directory Server is often imported in bulk. Directory Server Enterprise Edition provides tools for importing and exporting entire suffixes. It also provides tools for making backups of all suffixes at once and for restoring all data from a backup.
Before starting any backup or restore operation, ensure that you design a backup and restore strategy to suit your situation. For more information about the different backup options, considerations to take into account, and guidelines for planning a backup and restore strategy, see Designing Backup and Restore Policies in Sun Java System Directory Server Enterprise Edition 6.1 Deployment Planning Guide.
This chapter covers the following topics:
This section explains how to perform a binary backup of directory data. In addition to the binary backup procedures in this section, you can make a binary copy to use for initializing a suffix in a replication topology. See Initializing a Replicated Suffix by Using Binary Copy.
A binary data backup saves a copy of your directory data that you can use if the database files later become corrupted or deleted. This operation does not back up configuration data. If you want to back up the whole Directory Server for disaster recovery, see Disaster Recovery.
Never stop the server during a backup operation.
Your backup must be performed more frequently than the purge delay. The purge delay, specified by the nsDS5ReplicaPurgeDelay attribute, is the period of time, in seconds, after which internal purge operations are performed on the change log. The default purge delay is 604800 seconds (1 week). The change log maintains a record of updates, which might or might not have been replicated.
If your backup is performed less frequently than the purge delay, the change log might be cleared before it has been backed up. Changes will therefore be lost if you use the backup to restore data.
All backup procedures described in this section store a copy of the server files on the same host by default. You should then copy and store your backups on a different machine or file system for greater security.
Your Directory Server must be stopped to run the dsadm backup command.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Back up your directory data.
$ dsadm backup instance-path archive-dir |
For example:
$ dsadm backup /local/ds /local/tmp/20051205 |
You can back up directory data while the server is running by using the command dsconf backup command. However, if changes are made to the directory data while the backup is running, proper recovery is more difficult. To avoid this problem when using dsconf backup, set replication referrals or make the server read-only.
For more information about the dsadm and dsconf commands, see the dsadm(1M) and dsconf(1M) man pages.
When restoring a server, the dse.ldif configuration file must contain the same configuration information as when the server was backed up.
Back up your dse.ldif configuration file.
$ cp instance-path/config/dse.ldif archive-dir |
When you perform the following actions, Directory Server automatically backs up the dse.ldif configuration file in the directory instance-path/config.
When you start Directory Server, a backup of the dse.ldif file is created in a file named dse.ldif.startOK.
When you make modifications to the cn=config branch, the file is first backed up to a file named dse.ldif.bak in the config directory before the server writes the modifications to the dse.ldif file.
This procedure uses the frozen mode feature. Frozen mode enables you to stop database updates on disk so that a file system snapshot can be taken safely. You can use frozen mode as an additional measure for ensuring a robust backup.
Your server must not write user data on the disk while the file system backup is in progress. If you are sure that no updates will occur during a certain timeframe, make your backup during this time. If you cannot guarantee that there will be no updates, put your server into frozen mode before making a backup.
When frozen mode is on, all configured databases are taken offline. Any internal operations in progress are notified of the database going offline. LDAP operations in progress are completed, and the database environment is flushed. Subsequent incoming operations, including searches to user data, are refused until frozen mode is set to off. You can, however, search configuration parameters while frozen mode is on.
In a single-server topology, operations received when frozen mode is on result in an LDAP error being returned. The error message logged is the standard error for the database being offline. In a replicated topology, a referral is returned. For frozen mode to work correctly, no other tasks should be running on the databases.
For parts of this procedure, you can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help. Other parts of the procedure can only be done using the command line.
(Optional) Put your server into frozen mode.
$ dsconf set-server-prop -h host -p port read-write-mode:frozen |
Back up your file system, using a tool appropriate to your file system type.
If your server is in frozen mode, make the server read-write again.
$ dsconf set-server-prop -h host -p port read-write-mode:read-write |
If your server receives replication updates from another server, replication updates will start as soon as frozen mode is turned off.
Backing up to LDIF allows you to back up directory data to a formatted LDIF file.
You can back up directory data by exporting the contents of a suffix using LDIF. Exporting data can be useful for doing the following:
Backing up the data in your server
Copying your data to another directory server
Exporting your data to another application
Repopulating suffixes after a change to your directory topology
The export operations do not export the configuration information ( cn=config).
Do not stop the server while an export operation is in progress.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Use one of the following commands to export a suffix to an LDIF file:
If your server is local and stopped, type:
$ dsadm export instance-path suffix-DN LDIF-file |
If your server is remote and running, type:
$ dsconf export -h host -p port suffix-DN LDIF-file |
The following example uses dsconf export to export two suffixes to a single LDIF file:
$ dsconf export -h host1 -p 1389 ou=people,dc=example,dc=com \ ou=contractors,dc=example,dc=com /local/ds/ldif/export123.ldif |
The dsadm export and dsconf export commands can also be used with the --no-repl option to specify that no replication information is to be exported. The default is that the replicated suffix is exported to an LDIF file with replication information. The resulting LDIF file will contain attribute subtypes that are used by the replication mechanism. This LDIF file can then be imported on the consumer server to initialize the consumer replica, as described in Initializing Replicas
For more information about these commands, see the dsadm(1M) and dsconf(1M) man pages.
The following procedures describe how to restore suffixes in your directory. Your server must have been backed up using the procedures described in Backing Up Directory Data Only. Before restoring suffixes involved in replication agreements, read Restoring Replicated Suffixes.
Do not stop the server during a restore operation. Because restoring your server overwrites any existing database files, any modifications that were made to the data since the backup are lost.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Use one of the following commands to restore your server:
If your server is local and stopped, type:
$ dsadm restore instance-path archive-dir |
For example, to restore a backup from a backup directory, type:
$ dsadm restore /local/ds/ local/ds/bak/2006_07_01_11_34_00 |
If your server is remote and running, type:
$ dsconf restore -h host -p port archive-dir |
For example, to restore a backup from a backup directory:
$ dsconf restore -h host1 -p 1389 /local/ds/bak/2006_07_01_11_34_00 |
For more information about these commands, see the dsadm(1M) and dsconf(1M) man pages.
Directory Server creates two backup copies of the dse.ldif file in the following directory:
instance-path/config |
The dse.ldif.startOK file records a copy of the dse.ldif file at server start up. The dse.ldif.bak file contains a backup of the most recent changes to the dse.ldif file. Copy the file that contains the most recent changes to your directory.
For parts of this procedure, you can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help. Other parts of the procedure can only be done using the command line.
Stop the server.
$ dsadm stop instance-path |
Change to the directory that contains the configuration files.
$ cd instance-path/config |
Overwrite the dse.ldif file with a backup configuration file that is known to be valid, for example:
$ cp dse.ldif.startOK dse.ldif |
Start the server with the following command:
$ dsadm start instance-path |
You can import data to a Directory Server suffix in the following ways:
Initialize a suffix from an LDIF file. This operation deletes the current data in the suffix and replaces it with the contents of the LDIF file.
Use an LDIF file to perform bulk ldapadd, ldapmodify, or ldapdelete operations. This allows you to add, modify, and delete entries in bulk in any suffix of the directory.
The following table shows the differences between initializing a suffix and adding, modifying, and deleting entries in bulk.
Table 8–1 Comparison of Initializing a Suffix and Importing Data in Bulk
Domain of Comparison |
Initializing Suffixes |
Adding, Modifying, and Deleting Entries in Bulk |
---|---|---|
Overwrites content |
Overwrites content |
Does not overwrite content |
LDAP operations |
Add only |
Add, modify, delete |
Performance |
Fast |
Slower |
Response to server failure |
Atomic (all changes are lost after a failure) |
Best effort (all changes made up to the point of the failure remain) |
LDIF file location |
Local to client or local to server |
On client machine |
Imports configuration information (cn=config) |
Imports configuration information |
Does not import configuration information |
Commands |
If server is local and stopped: dsadm import If server is remote and running: dsconf import |
ldapmodify -B |
Initializing a suffix overwrites the existing data in a suffix with the contents of an LDIF file that contains only entries for addition.
You must be authenticated as the Directory Manager or an Administrator to initialize a suffix.
When the server is running, only the Directory Manager and Administrators can import an LDIF file that contains a root entry. For security reasons, only these users have access to the root entry of a suffix, for example, dc=example,dc=com..
Before restoring suffixes involved in replication agreements, read Restoring Replicated Suffixes.
All LDIF files that you import must use UTF-8 character-set encoding.
When initializing a suffix, the LDIF file must contain the root entry and all directory tree nodes of the corresponding suffix.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Use one of the following commands to initialize the suffix from an LDIF file, that is, import the contents of a database to an LDIF file.
These commands overwrite the data in your suffix.
If your server is local and stopped, type:
$ dsadm import instance-path LDIF-file suffix-DN |
The following example uses the dsadm import command to import two LDIF files into a single suffix:
$ dsadm import /local/ds /local/file/example/demo1.ldif \ /local/file/example/demo2.ldif dc=example,dc=com |
If your server is remote and running, type:
$ dsconf import -h host -p port LDIF-file suffix-DN |
The following example imports an LDIF file using dsconf import. You do not need root privileges to run the command, but you must authenticate as a user with root permissions, such as the Directory Manager.
$ dsconf import -h host1 -p 1389 /local/file/example/demo1.ldif \ ou=People,dc=example,dc=com |
If you run either dsconf import or dsconf reindex or both commands on multiple suffixes in parallel, transaction logs will grow and might negatively affect performance.
For more information on these commands, see the dsadm(1M) and dsconf(1M)man pages.
When you perform an ldapmodify operation, you are able to add, modify, or delete entries in bulk. Entries are specified in an LDIF file that contains update statements to modify or delete existing entries. This operation does not erase entries that already exist.
The changed entries may target any suffix that is managed by your Directory Server. As with any other operation that adds entries, the server will index all new entries as they are imported.
The ldapmodify command will import an LDIF file through LDAP and perform all operations that the file contains. Using this command you can modify data in all directory suffixes at the same time.
Before restoring suffixes involved in replication agreements, see Restoring Replicated Suffixes.
All LDIF files that you import must use UTF-8 character-set encoding.
When importing an LDIF file, parent entries must either exist in the directory or be added first from the file.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
Add, modify, or delete from an LDIF file in bulk.
$ ldapmodify -D cn=admin,cn=Administrators,cn=config -w - -B baseDN -f LDIF-file |
The following example performs an import using the ldapmodify command. You do not need root privileges to run this command, but you must authenticate as a user with root permissions, such as cn=Directory Manager or cn=admin,cn=Administrators,cn=config. The last parameter specifies the name of the LDIF file to import.
$ ldapmodify -D cn=admin,cn=Administrators,cn=config -w - \ -B dc=example,dc=com -f /local/ds/ldif/demo.ldif |
Suffixes that are replicated between supplier servers and consumer servers require special consideration before being restored. If possible, update the suffix through the replication mechanism instead of restoring it from a backup.
When you restore a supplier or hub instance, the server configuration must be the same as it was when the backup was made. To ensure this, restore the dse.ldif file before restoring Directory Server data. See Restoring the dse.ldif Configuration File.
This section explains how and when to restore a replica, and how to ensure that it is synchronized with other replicas after the operation. To initialize a replica, see Initializing Replicas.
If you have a large replicated suffix and you want to add many entries and ensure that replication updates are added correctly, see Incrementally Adding Many Entries to Large Replicated Suffixes.
This section contains information about the following:
A suffix that is a single-master supplier contains the authoritative data for the entire replication topology. Therefore, restoring this suffix is equivalent to reinitializing all data in the entire topology. You should restore a single master only if you want to reinitialize all data from the contents of the backup to be restored.
If the single-master data is not recoverable due to an error, consider using the data on one of the consumers because it might contain updates that are more recent than a backup. In this case, you need to export the data from the consumer replica to an LDIF file, and reinitialize the master from the LDIF file.
Whether you restore a backup or import an LDIF file on a master replica, you must then reinitialize all of the hubs and consumer replicas that receive updates from this replica. A message is logged to the supplier servers’ log files to remind you that reinitialization of the consumers is required.
In multi-master replication, the other masters each contain an authoritative copy of the replicated data. You cannot restore an old backup because it might be out of date with the current replica contents. If possible, allow the replication mechanism to bring the master up to date from the contents of the other masters.
If that is not possible, restore a multi-master replica in one of the following ways:
The simplest way is not to restore a backup, but to reinitialize the intended master from one of the other masters. This ensures that the latest data is sent to the intended master and that the data will be ready for replication. See Replica Initialization From LDIF.
For replicas with millions of entries, it can be faster to make a binary copy to restore a more recent backup of one of the other masters. See Initializing a Replicated Suffix by Using Binary Copy.
If you have a backup of your master that is not older than the maximum age of the change log contents on any of the other masters, the backup may be used to restore this master. See To Modify Change Log Settings on a Master Replica for a description of change log age. When the old backup is restored, the other masters will use their change logs to update this master with all modifications that have been processed since the backup was saved.
Regardless of how you restore or reinitialize, the master replica will remain in read-only mode after the initialization. This behavior allows the replica to synchronize with the other masters, after which time you may allow write operations, as described in Restoring a Master in a Multi-Master Scenario.
The advantage of allowing all replicas to converge before allowing write operations on the restored or reinitialized master is that none of the hub or consumer servers will require reinitialization.
This section applies only in situations where the replication mechanism cannot automatically bring a hub replica up to date. Such situations include if the database files become corrupted or if replication has been interrupted for too long. In these cases, you need to restore or reinitialize the hub replica in one of the following ways:
The simplest way is not to restore a backup, but to reinitialize the hub from one of the master replicas. This ensures that the latest data is sent to the hub and that the data will be ready for replication. See Initializing a Suffix.
For replicas with millions of entries, it can be faster to make a binary copy to restore a more recent backup taken from another hub replicated suffix. See Initializing a Replicated Suffix by Using Binary Copy. If there is no other hub replica to copy, reinitialize the hub as described in the previous item, or restore it as described in the next item, if possible.
If you have a backup of your hub that is not older than the maximum age of the change log contents on any of its suppliers, either hub or master replicas, the backup may be used to restore this hub. When the hub is restored, its suppliers will use their change logs to update this hub with all modifications that have been processed since the backup was saved.
Regardless of how you restore or reinitialize the hub replica, you must then reinitialize all consumers of this hub, including any other levels of hubs.
This section applies only in situations where the replication mechanism cannot automatically bring a dedicated consumer replica up to date. Such situations include if the database files become corrupted or if replication has been interrupted for too long. In these cases, you need to restore or reinitialize the consumer in one of the following ways:
The simplest way is not to restore a backup, but to reinitialize the consumer from one of its suppliers, either a master or a hub replica. This ensures that the latest data is sent to the consumer and that the data will be ready for replication. See Replica Initialization From LDIF.
For replicas with millions of entries, it can be faster to make a binary copy to restore a more recent backup taken from another consumer replicated suffix. See Initializing a Replicated Suffix by Using Binary Copy. If there is no other consumer to copy, reinitialize the replica as described in the previous item or restore it as described in the next item, if possible.
If the backup of your consumer is not older than the maximum age of change log contents on any of its suppliers, either hub or master replicas, the backup may be used to restore this consumer. When the consumer is restored, its suppliers will use their change logs to update the consumer with all modifications that have been processed since the backup was saved.
In the case of multi-master replication, other masters may process change operations while a given master is being restored. Therefore, when restoration is complete, the new master must also receive new updates that were not included in the restore data. Because restoring a master might take a significant amount of time, the number of pending updates might also be significant.
To allow convergence of these pending updates, newly restored masters are automatically set to read-only mode for client operations after restoration. This is true only when restoring a master by importing data from an LDIF file at the command line, or by using a backup to perform a binary copy.
Therefore, after restoration, a master in a multi-master configuration will process replication updates and allow read operations, but it will return referrals for all write operations from clients.
To verify that the new master is fully synchronized with the other masters before allowing updates, manually enable updates on an initialized master.
With master replicas sending referrals because of this new behavior, clients wanting to perform write operations might reach their configured hop limit. You might need to increase the hop limit configuration for clients so they can reach an available master. If all master replicas are initialized or reinitialized, all write operations will fail because no replica will be accepting client updates.
In any case, monitor initialized masters closely, and set the referral attributes appropriately to maximize server response.
You can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help.
The following commands can be used in scripts that automate the process of initializing a multi-master replica.
Use the insync tool to ensure that the replica has converged with all other masters.
Replicas are in sync if the delay between modifications on all servers is zero or if the replica has never had any changes to replicate (delay of -1). For more information, see the insync(1) man page.
Begin accepting updates.
$ dsconf set-suffix-prop -h host -p port suffix-DN repl-accept-client-update-enabled:on |
This command automatically sets the server to read-write mode.
If you want to back up or restore your Directory Server for disaster recovery purposes, use the following procedures.
For parts of this procedure, you can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help. Other parts of the procedure can only be done using the command line.
Make a backup of your database files by using the command dsadn backup or dsconf backup.
Use the procedure in Binary Backup, and store the backup files in a safe place.
Copy the configuration directory instance-path/config to a safe place.
Copy the schema directory instance-path/config/schema to a safe place.
Copy the alias directory instance-path/alias to a safe place.
For parts of this procedure, you can use DSCC to perform this task. For information, see Directory Service Control Center Interface and the DSCC online help. Other parts of the procedure can only be done using the command line.
Install the same version of Directory Server that you had previously on the host.
Create a server instance by using the dsadm create command.
Use the same instance that was used at backup time. See Creating Suffixes.
Restore the configuration directory instance-path /config.
Restore the schema directory instance-path/config/schema .
Restore the alias directory instance-path/alias .
Ensure that the configuration for the restored server is correct.
For example, the directory structure and plug-in configuration must be the same as on the backed up server.
Restore your database files by using the command dsconf restore.
Use the procedure in Binary Restore.