Oracle Internet Directory Administrator's Guide 10g (10.1.4.0.1) Part Number B15991-01 |
|
|
View PDF |
As of 10g (10.1.4.0.1), the bulk tools have been rewritten as C executables. The new versions of the tools provide some new features:
They reside in $ORACLE_HOME/ldap/bin
, the same directory as other Oracle Internet Directory command-line tools.
They have a consistent user interface.
They can be invoked directly from the Windows command-line interface. You no longer need to install Cygwin or MKS Toolkit.
Error and progress reporting have been enhanced.
Each tool has its own log file in $ORACLE_HOME/ldap/log
.
In addition to this chapter, bulk tools are also discussed in the following sections of this book:
"Backing Up and Restoring a Small Directory or Specific Naming Context"
"Installing and Configuring One-Way or Two-Way LDAP-Based Replication"
See Also: The chapter on Oracle Internet Directory server administration tools in Oracle Identity Management User Reference. |
Note: Stop all instances of Oracle Internet Directory before usingbulkload . Before using any of the other bulk tools, either stop all instances of Oracle Internet Directory or disable the entry cache. |
This chapter consists of the following sections:
The bulk loader, bulkload
, is a bulk management tool. It takes input data in LDIF or SQL*Loader format and loads that data directly into Oracle Internet Directory's schema in the metadata repository. It has three main phases: check
, generate
and load
.
In the check
phase, bulkload
parses and verifies the LDIF input data for the schema.
In the generate
phase, bulkload
generates intermediate files in SQL*Loader format.
In the load
phase, bulkload
can use either of two methods: bulk mode loading or incremental mode loading.
When using bulk mode loading, bulkload
loads generated intermediate files into the database. As it does so, it drops old indexes and generates new ones.
When using incremental mode loading, bulkload
loads intermediate files to tables in the database using insert mode. While it loads the data, bulkload updates the indexes.
Bulk mode loading is faster than incremental mode loading.
The bulk loader also supports the following features:
It allows you to specify the number of threads in order to achieve parallelism during the generate and load phases.
It has an encode option that allows you to use data in other languages.
It has a restore
option that allows you to retain the operational attributes specified in the LDIF file.
It has an index
option for index creation.
It has a recover
option that is useful for recovering from bulkload
failures.
When appending the data to existing directories, bulkload
supports both bulk mode and incremental mode loading.
The append
option allows you to load data while the LDAP server is up and running.
At the beginning of the generate
phase, the server's orclServerMode
is changed from read-write
to read-modify
. At the end of the generate
phase, it is left in the read-modify
state so that you cannot add entries to Oracle Internet Directory between the generate
and load
phases. This is necessary to maintain internal sequence numbers. You are expeced to run the load
phase immediately after the generate
phase. At the end of the load
phase, the servers' orclServerMode
is set back to read-write
. Using bulkload
with the recover
option also sets orclServerMode
back to read-write
.
The bulkload
tool generates the following output files in the $ORACLE_HOME/ldap/log
directory:
An output log, bulkload.log
A list of duplicate DNs, duplicateDN.log
Intermediate log files generated by the SQL*Loader, bsl_*.log
It generates the following output files in the $ORACLE_HOME/ldap/load
directory:
A list of bad LDIF entries, badentry.ldif
A list of all dynamic group entries that can be added using ldapadd
, dynGrp.ldif
Intermediate files, *.ctl
and *.dat
Note: If the applicable password policy has thepwdmustchange attribute set to true, then for every new entry loaded by bulkload , the pwdreset attribute is set to 1 by default. See Chapter 19, "Password Policies in Oracle Internet Directory" for more information. |
Note: If you do not use thebulkload utility to populate the directory, then you must run the oidstats.sql tool to avoid significant search performance degradation. |
See Also:
|
The bulkload
tool accepts parameters in key=value
format.
bulkload connect=connect_string {[check="TRUE"|"FALSE" [restore="TRUE"|"FALSE"] [threads=num_of_threads] [file=ldif_file]][generate="TRUE"|"FALSE" [append="TRUE"|"FALSE"] [restore="TRUE"|"FALSE"] [threads=num_of_threads] file=ldif_file] [load="TRUE"|"FALSE" [append="TRUE"|"FALSE"] [threads=num_of_threads]] [index="TRUE"|"FALSE"] [recover="TRUE"|"FALSE"]} [encode=character_set] [debug="TRUE"|"FALSE"] [verbose="TRUE"|"FALSE"]
Some of the parameter combinations are valid while others are invalid.
You must specify at least one of the following actions when you invoke bulkload
: check
, generate
, load
, append
, recover
, or index
.
If check
is TRUE
, bulkload
performs a schema check.
If generate
is TRUE
, bulkload
generates intermediate files.
When using the check
or generate
action, you must specify the pathname to the LDIF data file.
If load
is TRUE
, bulkload
loads intermediate files.
When append
is TRUE
, bulkload
can perform its actions while the server is up and running.
Use the restore
flag only when the LDIF file contains operational attributes, such as orclguid
or creatorsname
.
Do not specify recover
with any other option.
The option combination check
index
verifies the existing indexes.
To import an LDIF file, you use the bulkload
utility. This section discusses the tasks to process an LDIF file through bulkload
.
This section contains these topics:
Before you import the file, back up the Oracle database server as a safety precaution.
See Also: Oracle Database Backup and Recovery Basics in the Oracle Database Documentation Library |
To use bulkload
, you must provide the Oracle Internet Directory password. The default password is ods
, although the system administrator can change it by using the OID Database Password Utility.
On UNIX, the bulkload
tool usually resides in $ORACLE_HOME/ldap/bin
. On Microsoft Windows, this tool usually resides in ORACLE_HOME\ldap\bin
.
Check the input file and generate files for the SQL*Loader by typing:
bulkload connect="connect_string" check="TRUE" generate="TRUE" \ file="path_to_ldif-file_name"
All schema violations are reported in $ORACLE_HOME/ldap/log/bulkload.log
. All bad entries are logged in $ORACLE_HOME/ldap/load/badentry.ldif
. Use a text editor to fix all bad entries, then re-run bulkload with the check
and generate
options.
If there are duplicate entries, their DNs are logged in $ORACLE_HOME/ldap/log/duplicateDN.log
. This is just for information purpose. The bulkload
tool does not generate duplicate data for duplicate entries. It ignores duplicate entries.
When bulkload completes successfully, it generates *.ctl
and *.dat
files in the $ORACLE_HOME/ldap/load
directory to be used by SQL*Loader in load mode. Do not modify these files.
After you have generated the input files, rerun bulkload with the load
option. During this step, the *.dat
files, which are in Oracle SQL*Loader specific format, are loaded into the database and the attribute indexes are created. The syntax is:
bulkload connect="connect_string" load="TRUE"
All loading errors are reported in the $ORACLE_HOME/ldap/log
directory. They reside in bulkload.log
and in the SQL*Loader-generated files *.bad
and bsl_*.log
.If bulk
load fails, the database might be in an inconsistent state. If this occurs, you should restore the database to its state prior to the bulkload
operation. You can do this either by using bulkload
with the recover
option or by restoring Oracle Internet Directory directory from a backup taken before you invoked bulkload
.
The following examples show some uses of bulkload
.
Typically, you load directory data just after installing Oracle Internet Directory. This requires three actions:
Check the LDIF file for schema errors and generate the intermediate files
Load the data into the Oracle Internet Directory metadata repository.
You perform these actions with command lines similar to these:
bulkload connect="conn_str" check="TRUE" generate="TRUE" file="LDIF_file"
bulkload connect="conn_str" load="TRUE"
You can omit the check
phase if the LDIF data is from another Oracle Internet Directory node.
If you must add entries to an Oracle Internet Directory server that already contains data, and the server must be up and running at the same time, then you must use the incremental or append mode. This mode is usually faster than other methods of adding entries to the directory. However, you must ensure that the Oracle Internet Directory LDAP instances are in read-modify mode so that bulkload can append data. You invoke bulkload
in incremental or append mode with command lines similar to these:
bulkload connect="conn_str" check="TRUE" generate="TRUE" append="TRUE" \ file="LDIF_file" bulkload connect="conn_str" load="TRUE" append ="TRUE"
The bulkload
operation can either update indexes or create indexes. Sometimes, however, bulkload
does not update or create the indexes properly. This is typically due to issues like improper sizing. If this happens, you can use bulkload
to verify and re-create all the indexes.
Use the following syntax to invoke bulkload
for verification of indexes:
bulkload connect="conn_str" check="TRUE" index="TRUE"
To re-create indexes, use the following syntax:
bulkload connect="conn_str" index="TRUE"
The load
phase of bulkload can fail because of issues like improper disk sizing. After such a failure, the directory data might be inconsistent. You can use the recover
option to return the directory data to its pre-bulkload state. The syntax is:
bulkload connect="conn_str" recover="TRUE"
The bulkmodify
tool is useful for modifying the attributes of a large number of entries in an existing directory. It can perform add and replace operations on attribute values. It can operate on a naming context. Using filters, it can also operate selectively on a few entries under a specified naming context.
The bulkmodify
tool does not allow add
or replace
operations on the following attributes:
dn
cn
userpassword
orclpassword
orclentrylevelaci
orclaci
orclcertificatehash
orclcertificatematch
any binary attribute
It does not allow replace
operation on the attribute objectclass
.
It does not allow add
for single-valued attributes.Output from bulkmodify
is logged in $ORACLE_HOME/ldap/log/bulkmodify.log
.
Bulkmodify accepts parameters in key=value
pairs.
bulkmodify connect=connect_string basedn=Base_DN {[add="TRUE"|"FALSE"]|[replace="TRUE"|"FALSE"]} attribute=attribute_name value=attribute_value [filter=filter_string] [size=transaction_size] [threads=num_of_threads] [debug="TRUE"|"FALSE"] [encode=character_set] [verbose="TRUE"|"FALSE"]
The number of threads should be from one to six times the number of CPUs.
Select either the add
or the replace
option. By default both are set to FALSE
.
The following examples show some uses of bulkmodify
:
This example adds descriptions for all the entries under "c=us
".
bulkmodify connect="connect_str" basedn="c=us" add="TRUE" \ attribute="description" value="US citizen" filter="objectclass=*"
This example adds telephonenumber
for all the entries under "c=us
" that have the manager Anne Smith.
bulkmodify connect="connect_str" basedn="c=us" add="TRUE" \ attribute="telephoneNumber" value="408-123-4567" filter="manager=Anne Smith"
Bulkdelete is useful for deleting the attributes of a large number of entries in an existing directory. Bulkdelete can delete entries specified under a naming context. By default, it deletes entries completely. It removes all traces of an entry from the database. If you use the option cleandb
=FALSE
, bulkdelete
turns all entries into tombstone entries instead of deleting them completely.
Bulkdelete output is logged in $ORACLE_HOME/ldap/log/bulkdelete.log
.
The bulkdelete
tool accepts all parameters as key=value
pairs.
bulkdelete connect=connect_string {[basedn=Base_DN]|[file=file_name]} [cleandb="TRUE"|"FALSE"] [size=transaction_size] [encode=character_set] [debug="TRUE"|"FALSE"] [threads=num_of_threads] [verbose="TRUE"|"FALSE"]
Select either the basedn
or the file
option.If cleandb
is TRUE
, bulkdelete
removes entries completely from the database. By default cleandb
is set to TRUE
. The number of threads should be from one to six times the number of CPUs.
The following examples demonstrate how to use bulkdelete.
This example deletes all the entries under "c=us
".
bulkdelete connect="connect_str" basedn="c=us" cleandb="TRUE"
The ldifwrite
tool is used to dump of data from an Oracle Internet Directory store to a file. Having the data in a file facilitates loading the data into another node for replication or backup storage. As it writes to the output file, the ldifwrite
tool performs a subtree search, including all entries below the specified DN, as well as the DN itself. It dumps data in LDIF format. It can also dump entries under a specified replication agreement DN.
The ldifwrite
tool can dump entries located by using specified filters.Output from ldifwrite
is logged in $ORACLE_HOME/ldap/log/ldifwrite.log
.
The ldifwrite
tool accepts all parameters in keyword=value
pair format.
ldifwrite connect=connect_string basedn=Base_DN ldiffile=LDIF_Filename [filter=LDAP_Filter] [threads=num_of_threads] [debug="TRUE"|"FALSE"] [encode=character_set] [verbose="TRUE"|"FALSE"]
Use the basedn
option to specify the base DN or replication agreement DN.
The number of threads should be from one to six times the number of CPUs.
The following examples show some uses of ldifwrite
.
This example writes all the entries under "ou=Europe, o=imc, c=us" into the output.ldif
file.
ldifwrite connect="connect_str" basedn="ou=Europe, o=imc, c=us" \ ldiffile=output.ldif
The ldifwrite
tool includes the operational attributes of each entry in the directory, including createtimestamp
, creatorsname
, and orclguid
.
This example uses the following naming context objects defined in partial replication:
dn: cn=includednamingcontext000001, cn=replication namecontext, orclagreementid=000001, orclreplicaid=node replica identifier, cn=replication configuration orclincludednamingcontexts: c=us orclexcludednamingcontexts: ou=Americas, c=us orclexcludedattributes: userpassword objectclass: top objectclass: orclreplnamectxconfig
In this example, all entries under "c=us
" are backed up except "ou=Americas,c=us
". The userpassword
attribute is also excluded. The command is:
ldifwrite connect="conn_str" \ basedn="cn=includednamingcontext000001, cn=replication namecontext, \ orclagreementid=000001,orclreplicaid=node replica identifier,\ cn=replication configuration" ldiffile="ldif_file_name"
This example writes all the entries that satisfy LDAP search filter criteria under "ou=Europe, o=imc, c=us
" into the output.ldif
file.
ldifwrite connect="connect_str" basedn="ou=Europe, o=imc, c=us" filter="uid=abc" \ ldiffile="output.ldif"
The catalog
tool is useful for creating indexes for or dropping indexes from existing attributes. The catalog tool makes an attribute searchable.Output from catalog
is logged in $ORACLE_HOME/ldap/log/catalog.log
.
Catalog accepts all parameters in key=value
pair format.
catalog connect=connect_string {[add="TRUE"|"FALSE"]|[delete="TRUE"|"FALSE"]} {[attribute=attribute_name]|[file=file_name]} [logging="TRUE"|"FALSE"] [threads=num_of_threads] [debug="TRUE"|"FALSE"] [verbose="TRUE"|"FALSE"]
Select either the add
or the delete
option. By default both are set to FALSE
.
The number of threads should be from one to six times the number of CPUs.
If logging is TRUE
, catalog generates a redo log.
You can specify only one attribute
argument on the command line at a time. To add
or delete
more than one attribute in a single command invocation, use the file
option and specify a list of attributes in the file. Use one line per attribute, for example:
description sn title
The following examples show some uses of catalog
:
This example drops an index from the attribute title
.
catalog connect="connect_str" delete="TRUE" attribute="title"