Working with the Bulk Loader


	BEA Home \| Events \| Solutions \| Partners \| Products \| Services \| Download \| Developer Center \| WebSUPPORT
	http://www.oracle.com/technology/documentation/index.html \| Site Map \| Search \| PDF Files \| Contact \| Glossary

Collaborate Documentation \| Administering \| Previous Topic \| Next Topic \| Contents

The following sections describe how to work with the Bulk Loader:

In addition to using the Bulk Loader, you can use the WebLogic Collaborate Administration Console to transfer data to and from the repository, as described in the WebLogic Collaborate Administration Console Online Help. For information about the repository, see Working with the Repository.

Understanding the Terminology

The following table describes some of the terms used in this chapter.

Table 8-1 Terms Used in This Chapter

This term . . .	Refers to . . .
data element	An element in the repository
XML element	An element in an XML file
attribute	An attribute for an XML element
value	The value of an attribute or the value of a data element

For example, consider the following lines of code from a repository data file:

<message-definition name="request.dtd">
    <message-part content-type="text/xml">
        <text-document>request.dtd</text-document>
    </message-part>
</message-definition>

In these lines, the following examples of the definitions are used:

message-definition is an XML element that specifies a message-definition data element.
content-type is an attribute.
text/xml is the value for the content-type attribute.
request.dtd is the value for the text-document data element.

Importing Data into the Repository

The following sections provide information about importing repository data:

How the Bulk Loader Imports Data

As the following figure shows, the Bulk Loader parses a Bulk Loader configuration file to get instructions about transferring data from a repository data file into the repository. Both files are XML files. If the Bulk Loader detects any errors during this procedure, it creates an error log.

Figure 8-1 Using the Bulk Loader to Import Repository Data

The Bulk Loader uses the following logic to import data from a repository data file into the repository:

If the WebLogic Collaborate root element specified in the repository data file does not exist in the repository, the Bulk Loader creates the WebLogic Collaborate root element.
If the WebLogic Collaborate root element specified in the repository data file already exists in the repository, the Bulk Loader uses the following logic to process each data element in the repository data file:

Table 8-2 Logic for Processing an Attribute

Does the Value or Attribute Exist in the Repository Data File?	Attribute Type	Logic
Yes	Does not matter	The Bulk Loader sets the data element value or attribute to the value specified in the repository data file.
No	IMPLIED	The Bulk Loader sets the data element value or attribute to null unless it is one of the attributes that has a special default value, as described in the following table.
No	REQUIRED	The Bulk Loader considers the data to be invalid. For more information, see Checking Data.

The following table lists the default values for special IMPLIED attributes.

Table 8-3 Default Values for Special IMPLIED Attributes

XML Element	Attribute	Default Value
wlc	large-msg-support-on	OFF
wlc	show-hidden	OFF
trading-partner	status	ENABLED

Procedure for Importing Data into the Repository

Note: In addition to using the Bulk Loader, you can use the WebLogic Collaborate Administration Console to transfer data to and from the repository, as described in the WebLogic Collaborate Administration Console Online Help.

Note: You cannot run the Bulk Loader when WebLogic Collaborate is running.

To import data from a repository data file into the repository:

Create a Bulk Loader configuration file.
In it, include load-processing-parameters, which is the XML element that instructs the Bulk Loader to import data from the repository data file into the repository. For information about creating a Bulk Loader configuration file, see Working with the Bulk Loader Configuration File.
Create a repository data file.
For information about creating a repository data file, see Working with the Repository Data File.
To import data from the repository data file into the repository, enter one of the following commands:
- UNIX: bulkloader.sh cfg_file
- Windows: bulkloader cfg_file
  In both commands, cfg_file is the pathname of the Bulk Loader configuration file that you created in step 1. The Bulk Loader configuration file, in turn, specifies the pathname of the repository data file that you created in step 2.
  
  Use the same command for importing, exporting, and deleting data. The Bulk Loader configuration file indicates which action the Bulk Loader should take.
  
  While importing data, the Bulk Loader checks for errors as described in Checking Data.

Exporting Data from the Repository

The following sections provide information about exporting repository data:

How the Bulk Loader Exports Data

As the following figure shows, the Bulk Loader parses a Bulk Loader configuration file to get instructions about transferring data from the repository to a repository data file. Both files are XML files. If the Bulk Loader detects any errors during this procedure, it creates an error log.

Figure 8-2 Using the Bulk Loader to Export Repository Data

Full and Partial Repository Exports

When you export data from the repository, you can specify a full export or a partial export, as described in the following table.

Table 8-4 Full and Partial Repository Exports

Type of Export	Description
Full	The Bulk Loader exports all the data.
Partial	The Bulk Loader exports a subset of the data.

By default, the Bulk Loader performs a full repository export.

To perform a partial repository export, use the entities XML element in the Bulk Loader configuration file. The entities XML element specifies the data elements to be exported. The Bulk Loader uses the entities values to traverse the repository to the specified data elements.

For a complete description of the entities XML element, see WLCConfig.dtd in the dtd subdirectory of your WebLogic Collaborate installation directory.

You can identify a specific data element instance in the Bulk Loader configuration file. For example, to export a specific message definition, specify the name of the WebLogic Collaborate element and the name of the message definition, as shown in the following listing. This strategy applies to all types of data elements.

Listing 8-1 Example of a Bulk Loader Configuration File for Exporting a Specific Message Definition

<?xml version="1.0"?> 
<!DOCTYPE wlc-config SYSTEM "WLCConfig.dtd">
<wlc-config>
    <unload-processing-parameters> 
        <database-url>jdbc:weblogic:oracle:REPO</database-url>
        <database-driver>weblogic.jdbc.oci.Driver</database-driver>
        <database-user-id>scott</database-user-id>
        <database-password>tiger</database-password>
        <xml-file-name>ExportRepoData.xml</xml-file-name>
        <entities>
            <wlc name="office_supplies">
                <message-definition name="request.dtd">
                </message-definition>
            </wlc>
        </entities>
    </unload-processing-parameters>
</wlc-config>

Short and Long Repository Exports

When you export data from the repository, you can specify a short export or a long export, as described in the following table.

Table 8-5 Short and Long Repository Exports

Type of Export	Description
Short (standard)	The Bulk Loader organizes the output data to represent the user's view of the repository.
Long (extensive)	The Bulk Loader organizes the output data as a snapshot of the repository, which can be useful when you migrate repository data from one database to another.

By default, the Bulk Loader exports repository data in the short format.

To perform a long repository export, set the format attribute for the unload-processing-parameters XML element in the Bulk Loader configuration file to long.

Because the long format includes internal repository data, in addition to values for various objects, we recommend that you use the long format only for the following reasons:

To save a backup of the entire repository. For example, before you delete data from the repository, it is a good idea to back up the repository.
To migrate repository data from one environment to another. For example, if you change from one database vendor to another, or if you upgrade your database to a new machine, then you need to use the long format to migrate the entire repository database.

For an example of a Bulk Loader configuration file that specifies a format value, see Listing 8-3. For a complete description of the format attribute, see WLCConfig.dtd in the dtd subdirectory of your WebLogic Collaborate installation directory.

Procedure for Exporting Repository Data

Note: You cannot run the Bulk Loader when WebLogic Collaborate is running.

To export data from the repository to a repository data file:

Create a Bulk Loader configuration file.
In it, include unload-processing-parameters, which is the XML element that instructs the Bulk Loader to export data from the repository to a repository data file. For information about creating a Bulk Loader configuration file, see Working with the Bulk Loader Configuration File.
To export data from the repository to a repository data file, enter one of the following commands:
- UNIX: bulkloader.sh cfg_file
- Windows: bulkloader cfg_file
  In both commands, cfg_file is the pathname of the Bulk Loader configuration file that you created in step 1. The Bulk Loader configuration file, in turn, specifies the pathname of the repository data file into which the Bulk Loader exports the data.
  
  Use the same command for importing, exporting, and deleting data. The Bulk Loader configuration file indicates which action the Bulk Loader should take.
  
  While exporting data, the Bulk Loader checks for errors as described in Checking Data.

Deleting Data from the Repository

The following sections provide information about deleting repository data:

How the Bulk Loader Deletes Data

As the following figure shows, the Bulk Loader parses a Bulk Loader configuration file, which is an XML file, to get instructions about deleting data from the repository. The dotted line in the figure indicates that the Bulk Loader affects the repository without sending any data to it. If the Bulk Loader detects any errors during this procedure, it creates an error log.

Figure 8-3 Using the Bulk Loader to Delete Repository Data

Procedure for Deleting Repository Data

Note: You cannot run the Bulk Loader when WebLogic Collaborate is running.

To delete repository data:

We strongly recommend that you back up the repository before deleting any data from it. To back up the repository, perform a full, long (extensive) export as described in Exporting Data from the Repository.
Create a Bulk Loader configuration file.
In it, include delete-processing-parameters, which is the XML element that instructs the Bulk Loader to delete data from the repository. For information about creating a Bulk Loader configuration file, see Working with the Bulk Loader Configuration File.
To delete data from the repository, enter one of the following commands:
- UNIX: bulkloader.sh cfg_file
- Windows: bulkloader cfg_file
  In both commands, cfg_file is the pathname of the Bulk Loader configuration file that you created in step 2.
  
  Use the same command for importing, exporting, and deleting data. The Bulk Loader configuration file indicates which action the Bulk Loader should take.
  
  While deleting data, the Bulk Loader checks for errors as described in Checking Data.

Working with the Bulk Loader Configuration File

The Bulk Loader configuration file contains the database login information and processing instructions for the Bulk Loader. The Bulk Loader configuration file is an XML file that uses the WLCConfig.dtd file, which is in the dtd subdirectory of your WebLogic Collaborate installation directory.

To create a Bulk Loader configuration file:

Create an XML file that specifies WLCConfig.dtd as the DTD file.
If you are importing or exporting data, set the xml-file-name XML element in the XML file to specify the pathname of the repository data file. If you specify only a filename, the Bulk Loader looks for the repository data file in the current working directory.

The following sections provide example Bulk Loader configuration files for each type of Bulk Loader task:

Bulk Loader Configuration File for Importing Data

The following listing is an example Bulk Loader configuration file for importing data into the repository.

Listing 8-2 Example of a Bulk Loader Configuration File for Importing Data into the Repository

<?xml version="1.0"?> 
<!DOCTYPE wlc-config SYSTEM "WLCConfig.dtd">
<wlc-config>
    <load-processing-parameters database-initialization="no"\
        transaction-level="all"> 
        <database-url>jdbc:weblogic:oracle:REPO</database-url>
        <database-driver>weblogic.jdbc.oci.Driver</database-driver>
        <database-user-id>scott</database-user-id>
        <database-password>tiger</database-password>
        <xml-file-name>ImportRepoData.xml</xml-file-name>
    </load-processing-parameters> 
</wlc-config>

In addition to specifying values that are required in any Bulk Loader configuration file (DTD, database URL, database driver, database user ID, and database password), this example defines the following:

load-processing-parameters—This XML element tells the Bulk Loader to import data from the repository data file into the repository.
database-initialization—The value for this attribute specifies whether the Bulk Loader deletes all data from the repository before performing the repository import. The possible values are yes and no.
transaction-level—The value for this attribute specifies the actions for the Bulk Loader to take if it detects an error. The possible values are all and default. For more information, see Checking Data Integrity.
xml-file-name—The value for this element specifies the pathname of the repository data file from which the Bulk Loader imports the data. If you specify only a filename, the Bulk Loader looks for the repository data file in the current working directory.

Bulk Loader Configuration File for Exporting Data

The following listing is an example Bulk Loader configuration file for exporting data from the repository.

Listing 8-3 Example of a Bulk Loader Configuration File for Performing a Full Export from the Repository

<?xml version="1.0"?> 
<!DOCTYPE wlc-config SYSTEM "WLCConfig.dtd">
<wlc-config>
    <unload-processing-parameters format="long">
        <database-url>jdbc:weblogic:oracle:REPO</database-url>
        <database-driver>weblogic.jdbc.oci.Driver</database-driver>
        <database-user-id>scott</database-user-id>
        <database-password>tiger</database-password>
        <xml-file-name>ExportRepoData.xml</xml-file-name>
    </unload-processing-parameters> 
</wlc-config>

unload-processing-parameters—This XML element tells the Bulk Loader to export data from the repository to the repository data file.
format—The value for this attribute specifies whether the Bulk Loader exports data in the long (extensive) format or in the short (standard) format. The possible values are short and long. For more information, see Short and Long Repository Exports.
xml-file-name—The value for this element specifies the pathname of the repository data file to which the Bulk Loader exports the data. If you specify only a filename, the Bulk Loader looks for the repository data file in the current working directory.

Working with the Repository Data File

The repository data file is an XML file that uses the WLC.dtd file, which is in the dtd subdirectory of your WebLogic Collaborate installation directory. To create a repository data file, create an XML file that specifies WLC.dtd as the DTD file.

The following listing shows a repository data file that creates an extended property set.

Listing 8-4 Example of a Repository Data File for a New Extended Property Set

<?xml version="1.0"?>
<!DOCTYPE wlc SYSTEM "WLC.dtd">
<wlc 
    name="WLC" 
    large-msg-support-on="ON"
    large-msg-min-size="10000"
    large-msg-location="c:\temp"
    proxy-host="andrew"
    proxy-port="7502" 
    description="The WLC Hub" >

    <extended-property-set name="EPS 1">
        <xml-element name="C1">
            <xml-attribute name="C1_A1" value="C1_A1 Value"/>
            <xml-element name="C1G1">
                <xml-element name="C1G1_T1" text="C1G1_T1 Value"></xml-element>
                <xml-element name="C1G1_T2" text="C1G1_T2 Value"></xml-element>
                <xml-element name="C1G1_T3" text="C1G1_T3 Value"></xml-element>
            </xml-element>
            <xml-element name="C1G2" text="C1G2 Value"></xml-element>
            <xml-element name="C1G3" text="C1G3 Value"></xml-element>
        </xml-element>
        <xml-element name="C2">
            <xml-attribute name="C2_A1" value="C2_A1 Value"/>
            <xml-attribute name="C2_A2" value="C2_A2 Value"/>
            <xml-attribute name="C2_A3" value="C2_A3 Value"/>
            <xml-element name="C2G1" text="C2G1 Value"></xml-element>
            <xml-element name="C2G2">
                <xml-element name="C2G2_T1" text="C2G2_T1 Value"></xml-element>
                <xml-element name="C2G2_T2" text="C2G2_T2 Value"></xml-element>
                <xml-element name="C2G2GG1">
                    <xml-attribute name="C2G2GG1_A1" value="C2G2GG1_A1 Value"/>
                    <xml-element name="C2G2GG1_T1" text="C2G2GG1_T1
                        Value"></xml-element>
                </xml-element>
            </xml-element>
            <xml-element name="C2G3" text="C2G3 Value"></xml-element>
        </xml-element>
    </extended-property-set>

</wlc>

Checking Data

The following sections describe how the Bulk Loader checks data:

Creating an Error Log

To keep track of errors, the Bulk Loader creates a file named wlc.log in the current working directory. If a wlc.log file already exists, the Bulk Loader renames the existing file by appending a timestamp to the name in the following format: wlc.log.yyyy.mm.dd.hh.mi.ss. Then the Bulk Loader creates a new wlc.log file. The following table describes the fields in the timestamp.

Table 8-6 Timestamp Appended to Name of Existing Log File

This string . . .	Indicates the . . .
yyyy	Year
mm	Month in numeric format (between 01 - 12)
dd	Day in numeric format
hh	Hour (between 00 - 23)
mi	Minute
ss	Second

Validating XML Files

Before it processes any data, the Bulk Loader validates the XML files. The following table lists the files that the Bulk Loader validates for each type of Bulk Loader task.

Table 8-7 Files Validated by the Bulk Loader

Task	Validated Files
Importing data	Bulk Loader configuration file Repository data file
Exporting data	Bulk Loader configuration file
Deleting data	Bulk Loader configuration file

To validate one of these types of XML files, the Bulk Loader checks it against the corresponding .dtd file. If the Bulk Loader detects an error in the XML file, it stops without processing the data.

Checking Data Integrity

After validating the XML files, the Bulk Loader checks the data integrity while it is processing the data. To check data integrity, the Bulk Loader verifies that the information in the XML files does not conflict with the information in the repository.

For example, if the Bulk Loader is adding a new data element to the repository and if the new data element references another data element, the Bulk Loader makes sure that the referenced data element exists in the repository or in the repository data file.

Checking Data Integrity While Importing or Deleting

To have data integrity checked while data is being imported or deleted, set the transaction-level attribute in the Bulk Loader configuration file as shown in Listing 8-2. If you do not set transaction-level, the Bulk Loader uses the default value. The Bulk Loader performs one of the following sets of actions depending on the value of transaction-level:

all—The Bulk Loader performs a single transaction for all the data. If the Bulk Loader detects invalid data during the transaction, it rolls back the entire transaction and stops. The repository is left in exactly the same condition it was in before the Bulk Loader started importing or deleting data.
default—The Bulk Loader performs a separate transaction for each of the following types of data elements:

Checking Data Integrity During an Export

If the Bulk Loader detects invalid data in the Bulk Loader configuration file, it does not perform the export.