Chapter 7   Importing/Exporting Data


This chapter describes the Import/Export utility, and provides guidelines for using the import/export function from the graphical Admin interface or the command line.

---

Note	The information in this chapter is based on the assumption that you have already read Concepts and Release Notes.

---

This chapter contains the following sections:

• About Importing/Exporting

• Administrator's Importing Tasks

• Creating an Import Data File

• Importing with the Admin Interface

• Importing with the Command Line

• Import/Export Utility and https

Instructions for importing catalog data are contained in Chapter 6 "Administering Catalogs," and in the iPlanet Market Maker Catalog Import Guide.

About Importing/Exporting

---

The Import/Export utility allows you to import batch data from external files into the iPlanet BuyerXpert database, or export business rules. Data for any of the iPlanet BuyerXpert resources can be updated this way, such as:

• Business rules

• Members

• Locations

• Seller catalogs

There are two ways to update your iPlanet BuyerXpert data:

• Individual transactions—Data is entered manually, using the data entry screens of the Admin interface. For the most part, instructions for using the various graphical screens are contained in the Admin interface Help.

  This method is generally used when your transactions are relatively simple, or when you are unfamiliar with the batch transaction process of the Import/Export utility.

• Batch transactions—Data is entered as a batch, using the Import/Export utility. An import session can be initiated either by using the Import screens in the Admin interface, or by invoking the import script from the command line.

  This method is used in some of these common situations:

  • You have a large quantity of updates.

  • The update is a repeating operation, where this updating is done on a regular basis. For example, updating a catalog price list.

  • When the transaction is complex. If you have to go to many screens in the Admin interface to complete the transaction, it may be more efficient to use the batch process, even for a single transaction.

The import process involves two elements:

• Import data file(s)—text files that contain commands and data for updating iPlanet BuyerXpert resources

• Import/Export utility—a program that processes the data in the import data file

Import Data File

An import data file is the text file, in comma-separated values (CSV) format, that contains the data that modifies the iPlanet BuyerXpert database. Import data files must have the file extension .imp, such as people.imp or catalogABC.imp.

An import data file is written to update a particular resource. Each resource has a different set of attributes that are to be updated, some required, some not. A list of attributes for every iPlanet BuyerXpert resource can be found in the templates in the sample file directory:

IAS_Root/buyer/sample_data/impexp_files/

An import data file contains four different types of information lines: comments, commands, headers, body. Table 7-1 describes these information lines.

Table 7-1    Import Data File Contents 

Line Type	Line Starts with	Function	Example
Comment line	# symbol	Describes file contents.	# Import Data File for Location
Command line	< symbol	Identifies a command and resource name, separated by a comma. May contain default values for some resource attributes.	<update, Location
Header line	^ symbol	Describes the format for the body lines. Lists the attributes required for resource mentioned in command line	^country_code, postal_code, from_date
Body line	no preceding character	Lists operands, separated by a comma. Operand is usually one value for each attribute. If more than two values are present, they are separated by special symbols.	France, 1234, 01/01/2001

During processing, blank lines and comments are ignored.

Symbols Used in an Import Data File

The import data file uses special symbols. Table 7-2 lists these symbols and an example of each.

Table 7-2    Import Data File Symbols

Symbol	Data Type	Description	Example
,	List	Separates a list of values.	12345, "this is a string", 11/27/00
|	Old or new value	Separates an old and a new value.	12345 | 54321
&	Multivalues	Separates multiple values.	London & Denver & Paris
" "	Strings	Encloses alphanumeric string values and spaces.	"This is a 5th string"
\	Escape, Continuation	Preserves literal value; indicates line continuation.	This is a string\, wi\|th spe\|\|cial symbols \ inside\]
[ ]	Reference	Encloses a reference to an object within an object.	[ImpTest attrInt=1 attrString= "First Test"], \ 11/27/00

Commands Used in an Import Data File

A command line identifies the command and the resource that will be processed during importing. Any or all of the following commands are recognized by the Import/Export utility and can be included in an import data file:

• insert—Add new data associated with these body lines.

• insert_overwrite—Add data associated with these body lines; if the data already exists in the database, replace it with this data.

• update—Update existing data with the data in these body lines.

• delete—Delete data associated with these body lines.

A command line is followed by a header line; a header line is followed by body lines. Each time a new command in specified, its associated header and body lines follow. For example, for a delete command, there is a set of header-body lines, for an update command, another set of header-body lines, and so on.

Example—Import Data File
When all the lines are combined, they make up an import data file.

# Location Import file

#

# commands supported are:

#    insert: required attributes - common_name, parent_org

#    insert_overwrite: required attributes - common_name, parent_org

#    update: required attributes - reference_name

#    delete: required attributes - reference_name

#

# Command line: insert command

<insert, Location

#

# Header line: Lists fields for body lines;
# Required fields: common_name,parent_org

^common_name, parent_org, country_code, postal_code, \

effective_from_date, effective_to_date,telephone_number, \

fax_number, websites Paristest

#

# Body line

Paristest,[Organization name=BuyerCompany], France, 1234, 01/01/2000, 01/01/2020, \

(408)111-2222&(408)333-4444, (510)111-2222&(510)333-444, a@b.com&a@c.com
#

# Example of multivalued attributes

# Command line

<update, Location

# Header line: Lists fields for body lines
# Required field: reference_name

^ reference_name, add_websites,del_websites, add_telephone_number, \ del_telephone_number, add_fax_number, del_fax_number

#

# Body line

[Location common_name=Paristest organization=BuyerCompany], \

C.b.com&D.b.com&E.b.com,AB.b.com&CD.b.com, 715&716, 716&761, 815&816, 816&861

#

# Command line: insert_overwrite command

<insert_overwrite, Location

#

# Header line: Lists fields for body line

# Required fields: common_name, parent_org

^common_name, parent_org, country_code, postal_code, postal_address, definable_id,telephone_number

#

# Body line

Paristest,[Organization name=BuyerCompany], France, 1234, LocAddress,0000, \

(408)111-2222&(408)333-4444&(408)555-6666

#

# Command line: delete command

# Required field: reference_name

<delete, Location

#

# Header line: Lists fields for body lines

^reference_name

[Location common_name=Paristest organization=BuyerCompany]

Import/Export Utility

Importing can be accomplished either using graphical screens in the Admin interface or using the command line.

Admin Interface Screens

The Admin interface provides screens for defining an import session and managing it through to completion. You can monitor the progress of a session, stop a session, or resume a stopped session. In addition, you can use the Drop option to clean up the Import utility data tables and log files after the session has completed.

---

Note	The main advantage of using the graphical Import screen rather than the Import script, is that you can easily monitor the progress of a session.

---

Instructions are contained in the "Importing with the Admin Interface" section in this chapter or in the Admin interface Help.

Command-Line Interface

iPlanet BuyerXpert provides a UNIX shell script that allows you to run the Import/Export utility as a batch process from the command line. An advantage of the command line is that you can import an unlimited number of files in a single batch.

Instructions for using the import script and its options are contained in the "Importing with the Command Line" section in this chapter.

Administrator's Importing Tasks

---

Table 7-3 lists typical ongoing administrative tasks for importing. The right-hand column provides information about where to find instructions, guidelines, or tools for performing that task.

Table 7-3    Ongoing Importing Tasks 

Task	Location of Instructions/Guidelines/Tools
Create an import data file.	Creating an Import Data File section
Import the data file using the graphical Admin interface Import screens.	Importing with the Admin Interface section
Import the data file using the Import script on the command line.	Importing with the Command Line section

Creating an Import Data File

---

For any importing, you must first create an import data file for the specific resource you are importing, for example, payment data. Each different type of resource usually has its own import data file. Sample import data files are supplied with iPlanet BuyerXpert for the common iPlanet BuyerXpert resources.

---

Tip	It is recommended that you use an appropriate sample file for creating your import data file rather than writing a file from scratch.

---

An import data file has specific contents requirements, which must be followed precisely. The first time you create an import data file for a resource, you will probably encounter a number of import failures before the file is correct. Once you have done a successful import for that type of resource, you can reuse the file for subsequent importing.

Use the following guidelines to create an import data file for a resource:

1. Copy the sample import data file for your resource to your working directory. Sample import data files are located in the following directory:

   IAS_Root/buyer/sample_data/impexp_files/

2. Use a text editor to edit the comment, command, and header lines for your resource.
   

   ---

   Note	Each command you plan to use in the import session needs its own command/header/body section in the import data file.

   ---

   
   

3. Insert the body lines for the import text.

   The sample files contain all the possible attributes that come with a iPlanet BuyerXpert resource. You need to delete or modify the attributes where your data is different.

4. Verify that the attributes listed in the header lines correspond to the values in the body lines.

5. Verify that all symbols are used correctly.

Importing with the Admin Interface

---

After you have created an import data file, you can use the Admin interface Import function to import the data. The following high-level options are available when you click the Import/Export tab in the Admin interface:

• Start New Session—Use to start a new session by specifying the import data file(s).

• Session Manager—Use to manage existing sessions (both active and inactive).

• Session Status—Use to gather status information.

Follow these guidelines to import a data file using the Admin interface (field and procedure details are contained in Help):

1. Select Import/Export.

2. Enter the number of files you want to import, up to ten.

3. Click Change.

4. Enter the file name or URL of the file location. If you are unsure of the name or location, click Browse to find the files you want to import.

5. Click Upload Files.

   The Start Session screen appears.

6. Click Start to import all files that you have selected.

7. The Session Status screen displays the state of the current session.

Follow procedures in Help to perform additional tasks.

Importing with the Command Line

---

The Import/Export utility batch process can be initiated from the command-line by running the BXImport.sh shell script. The script for BuyerXpert is located in the IAS_ROOT /buyer/admin/bin directory along with the default configuration file, cli.conf.

Some values are set during installation (for example, the path to JAVA classes, or the path to the default configuration file). You can enter script options directly on the command-line or indirectly through a configuration file.

---

Note	Options passed from the command line override any options set in a configuration file.

---

Syntax

BXImport.sh [options] import_files

Table 7-4 lists the options that are used with the import script.

Table 7-4    Import Options

Options	Description
-u URL	Web server location for Import/Export utility servlets
-n UserId	User name (login name)
-p password	User password (login password)
-o orgName	User organization (login organization)
-c configFile	Configuration file
-i inDir	Input files directory. Default is current directory.
-d outDir	Output files (report and log) directory. Default is current directory.
-f outFiles	Output files (discard and log). Default is session name + extension.
-s silent	Requires Boolean silent mode (no dialog with user) flag. True/False. Default is False.
-b blind	Requires Boolean blind mode (user only sees fatal errors) flag. True/False. Default is False.
-a true | false	Requires Boolean auth mode (authentication is required) flag. True/False. Default is True.
import_filename	Name of import data file (.imp and.csv are valid extensions)

If you don't specify any options when you invoke the import script on the command line, the default values in the cli.config file are used, as shown in Example 1.

Example 1—Default Configuration File
The following example contains typical values that the import script would use to start the Import/Export utility from the command line.

# cli.conf

URL = http://buyerhost:80

NAME = kevin

PASSWORD = kevin

ORGANIZATION = BuyerCompany

SILENT = false

BLIND = true

INDIR = ./

OUTDIR = ./

If you want to use other values, you can enter specific options when you start the import script. These two methods are illustrated in Example 2 and Example 3.

Example 2—   Using Default Values
The following command imports two files, locations.imp and usergroups.imp, from /import_files directory. All settings are taken from the default configuration file.

# BXImport.sh /import_files/locations.imp /import_files/usergroups.imp

Example 3—Redefining Default Values
This command imports two files, orders.imp and payments.imp, from the /import_files directory. Option settings are:
Login name - admin.login password - admin; Organization - corp.com.

All other settings are taken from the default configuration file.

# BXImport.sh -n admin -p admin -o corp.com /import_files/orders.imp /import_files/payments.imp\

By default, the import script uses the cli.conf file which consists of default settings for the options. If you want to redirect to another configuration file, you can as shown in Example 4.

Example 4—Redirect to a Configuration File
The following command imports one file, /import_files/users.imp, using the option settings from specific configuration file, /myfiles/myconf.conf.

# BXImport.sh -c /myfiles/myconf.conf /import_files/users.imp


Exporting Business Rules

---

Using Export of ASBORules

• Create an export file with read command and specify the various criteria. (The structure of the export file is similar to current BXImport file).

• Login to the BuyerXpert Administrator site as a system administrator.

• Click the System Tab.

• Enter the path to the export file in the text box for file and upload the file.

• Click Start to start the export process.

  If there are rules matching the export criteria, the Get Export File button displays.

• Click the button to get the export results and save the file.

Why Export ASBORules

• Referential Integrity—If a resource is deleted from the BuyerXpert installation, there might be rules that refer to this resource for value or relationship. Using export, it is possible to find all of those rules and store in an import file. Using the Delete option, those rules can be deleted.

• Synchronization of various BuyerXpert systems—If there are multiple BuyerXpert systems, it is difficult to keep them in sync. One way is take database dumps every month or so and import them. This means that all of the data on one system is dumped on other systems, and the data on these systems is lost. This also requires that the systems be shut down. Using export, it is possible to find only rules that were modified during that period, and to generate an import file to insert/update those rules on other systems.

• Rollbacks—Since its possible to find out the modifications that were made in a given period, it is possible to rollback to a previous state in the event of failures.

Creating Input Export File

• The export of ASBORules is accomplished by a read operation on ASBORules. The command line for export is <read, ASBORule.

• The header line appears as ^attribute 1, attribute 2... attribute n.

• The data line appears as value 1,value 2, value3|value31... value n.

• The header and data line form the export criteria.

• A sample export file would appear as follows:

  <read, ASBORule

  ^name, value, reln

  SHIPTO_LOCATION_DEFAULT, [Location name = "San Jose" organization = "BuyerCompany"], USER_CREATED_FOR;[User login_name = "nancy" organization = "BuyerCompany"]

Resultant Output Export File

The output of the export utility can be saved as .csv or .imp file.

If the output of export needs to be reused by the BuyerXpert application, a recommended extension would be .imp. Currently, the export file is generated with an insert operation on ASBORules. Currently, the header line has value, reln enabled, and name attributes of ASBORules.

The sample output export file appears as follows:

# Results for export operation (ID admin:_11-3-01_12:05:15_AM_GMT_750)

# Import file name: [L] /u/bhate/exp/export.imp

############### Export operation for object ASBORule ###############

<insert, ASBORule

^value,reln,enabled,name

POF:Location@guid-sanjose-BuyerCompany,*,true,BILLTO_LOCATION_DEFAULT

Export Criteria

Currently, there are twelve different criteria for the export of ASBORules. Any number (not to exceed twelve) can be specified in the input export file. The resulting output export file contains all of the rules that satisfy the criteria. The and/or of criteria is possible and is explained in later sections.

The criteria and how to use them are described as follows:

Name—This is the name of the ASBORule. If all of the rules of a specific name (for example, SHIPTO_LOCATION_DEFAULT) are to be exported, this criterion is used.

For example:

<read, ASBORule

^name

SHIPTO_LOCATION_DEFAULT

This exports all SHIPTO_LOCATION_DEFAULT rules.

Value—This is the value of the ASBORule. If only those rules that have a specific value are to be exported, this criterion is used. If a rule is multi-valued and one of its values matches the entered value, it is also exported.

For example:

<read, ASBORule

^value

[Location name = "Sunnyvale" organization = "BuyerCompany"]

This exports all of the rules that have the value "Sunnyvale" Location.

Relationship—This is represented by reln in the input export file. This is the relationship attribute of ASBORules. If rules that have specified voters are to be exported, this criterion is used.

For example:

All rules created for User nancy are to be exported.

<read, ASBORule

^reln

USER_CREATED_FOR;[User login_name = "nancy" organization = "BuyerCompany"]

This exports all of the rules created for User nancy.

If multiple relationships are to be provided, `;' should be used as separator.

For example:

USER_CREATED_FOR;[User login_name = "nancy" organization = "BuyerCompany"]; SHIPTO_LOCATION_DEFAULT; [Location name = "Sunnyvale" organization = "BuyerCompany"].

If all values for a relationship are to be considered, `*' can be used as the value of the relationship.

Enabled—This criterion is specified if only enabled or disabled rules are to be exported. If this criterion is not specified, or is specified with `*', export ignores the status of the rule. Valid inputs for enabled are true, false, `*', or a blank space.

Company_owner_id—If only the rules that are owned by a specific company are to be exported, this criterion is used.

For example:

<read, ASBORule

^company_owner_id

[Organization name="BuyerCompany"]

This exports all rules owned by BuyerCompany.

User_owner_id—If only the rules that are owned by a specific user are to be exported, this criterion is used.

For example:

<read, ASBORule

^user_owner_id

[User login_name= "nancy" organization ="BuyerCompany"]

This exports all rules owned by the user nancy.

created_by—If only the rules that are created by a specific user are to be exported, this criterion is used. A user reference or uid is allowed as input.

For example:

<read, ASBORule

^created_by

[User login_name="nancy" organization ="BuyerCompany"]

This exports all rules created by the user nancy.

modif_by—If only the rules that are modified by a specific user are to be exported, this criterion is used. A user reference or uid is allowed as input.

For example:

<read, ASBORule

^modif_by

[User login_name= "nancy" organization ="BuyerCompany"]

This exports all rules modified by the user nancy.

created_on—If rules are to be exported based on the date of their creation, this criterion is used. It is possible to specify an interval for this criterion. In this case, the start and end of the interval are separated by `|'. If only one date is inputted, it creates an exact match to the created_on date. If an interval is specified, all rules that were created during that interval are considered for export.

For example:

<read, ASBORule

^created_on

10\01\2001

This exports all rules created on 10\01\2001.

Or:

<read, ASBORule

^created_on

10\01\2001|10\01\2002

This exports all rules created on or after 10\01\2001, but on or before 10\01\2002.

modif_on—If rules are to be exported based on the date of their update, this criterion is used. It is possible to specify an interval for this criterion. In this case, the start and end of the interval are separated by a `|'. If only one date is inputted, it creates an exact match to the modif_on date. If an interval is specified, all rules that were updated during that interval are considered for export.

For example:

<read, ASBORule

^modif_on

10\01\2001

This exports all rules updated on 10\01\2001.

Or:

<read, ASBORule

^created_on

10\01\2001|10\01\2002

This exports all rules updated on or after 10\01\2001, but on or before 10\01\2002.

eff_from—If rules are to be exported based on their effective-from date, this criterion is used. It is possible to specify an interval for this criterion. In this case, the start and end of the interval are separated by a `|'. If only one date is inputted, all the rules that are effective on or before the inputted date, are considered for export. If an interval is specified, all rules that become effective during that interval are considered for export.

For example:

<read, ASBORule

^eff_from

10\01\2001

This exports all rules in which the eff_from date is on or before 10\01\2001.

Or:

<read, ASBORule

^eff_from

10\01\2001|10\01\2002

This exports all rules in which the eff_from date is on or after 10\01\2001, but on or before 10\01\2002.

eff_to—If rules are to be exported based on their effective-to date, this criterion is used. It is possible to specify an interval for this criterion. In this case, the start and end of the interval are separated by a `|'. If only one date is inputted, all the rules that are effective on or after the inputted date are considered for export. If an interval is specified, all rules that become ineffective during that interval are considered for export.

For example:

<read, ASBORule

^eff_to

10\01\2001

This exports all rules in which the eff_to date is on or after 10\01\2001.

Or:

<read, ASBORule

^eff_ to

10\01\2001|10\01\2002

This exports all rules in which the eff_to date is on or after 10\01\2001, but on or before 10\01\2002.

Specifying Various Criteria in One Input Export File

It is possible to specify various criteria in one input export file. The resultant output export file has a different section for each export criteria.

If there are some attributes in all the criteria that have same value, they can be appended to the command line as name=value pairs separated by a `,'.

If some criteria have the same attributes with different values, a single header line containing the attribute names, followed by separate data lines for each criterion, can be input.

If there are criteria with different attribute names, multiple header line data line sections can represent each criterion.

See the following structures:

<read, ASBORule, company_owner_id=[Organization name= "BuyerCompany"]

# criteria with same attribute names but different values

^reln, enabled

USER_CREATED_FOR;[User login_name= "nancy" organization="BuyerCompany"], true

USER_CREATED_FOR;[User login_name= "nancy" organization="BuyerCompany"], *

# criterion with different attribute names from the previous two

^name, value

SHIPPING_METHOD, [ShipMethod name="FedEx - 2 Day" organization="BuyerCompany"]

In this case, all the three criteria have the same company_owner_id, which BuyerCompany. Therefore, it is appended to the control line.

Two of the criteria have the same attribute names, but different attribute values, which are reln and enabled, and are represented by a middle section with one header line and two datalines.

The third criterion has different attribute names than the first two, and the last section represents that.

Dealing With the Time Intervals

If all rules modified after a certain date (for example, 10/01/2001) are to be exported, then enter 10/01/2001|<todays date> as the input value for modif_on.

If all the rules that are effective from 10/01/2001 to 10/01/2002, for example, are to be exported then enter eff_from as 10/01/2001 and eff_to as

10/01/2002. In this case, all the rules that are effective at least during this period are considered for export.

If the rules which are effective at least for one day during the period, 10/01/2001 to 10/02/2001, enter eff_from as (start of era) 01/01/1990|10/02/2001 and

eff_to as 10/01/2001|11/22/2025 (end of era).

Some examples to create input export files:

I want to export all the rules of PRICE_ADJUSTMENT, created for BuyerCompany, which is enabled and created on 1st December 1999 and which are

effective during 24th December 1999 and 1st January 2000. This I need to know for any promotional offers offered to BuyerCompany during the Christmas of '99.

<read, ASBORule

^name, reln, created_on, eff_from, eff_to

PRICE_ADJUSTMENT, USER_CREATED_FOR;[Organization name= "BuyerCompany"],12/01/1999,12/24/1999,01/01/2000

I want to export all the rules owned by BuyerCompany, which were created during 1st of October 2001 and 31st of October 2001 or modified during the same period. I need to apply these changes on other production machines.

<read, ASBORule, company_owner_id=[Organization name= "BuyerCompany"]

^created_on

10/01/2001|10/31/2001

^modif_on

10/01/2001|10/31/2001

Future work:

1.Currently, the output export file is generated with insert option. The user must be allowed to enter the operation in the input export file.

2.Currently, the output export file has value, reln, enabled, and name as attributes. The user must be allowed to enter the attributes that need to be populated along with their order.

3.Currently, if the input export file results in no rules for output, no export file is generated and there is no feedback to user. Either export file with no rules but some indication that no rules for export, should be generated, or user should be given some feedback on GUI.

Import/Export Utility and https

---

At this time, the command-line functionality of the Import/Export utility supports https only if you perform the additional configuration steps mentioned here.

If you want to use the Import/Export utility with SSL support, you need to make sure that the JSSE support is enabled. For example, make sure the classpath includes the JSSE jar files. For licensing reasons, the JSSE jar files are not bundled with iPlanet BuyerXpert, but you are free to download these files yourself.

To implement command-line functionality with https:

1. Include the following jar files in the classpath:
   • jsse.ja

   • jnet.jar

   • jcert.jar

2. If you use the import script, BXImport.sh, you need to add these files to the classpath parameter of the final Java call.

Example Line
$JAVAEX-classpath /export/cli/jsse.jar./export/cli/jnet.jar:/export/cli/jcert.jar: $JAVA_CLASSES_DIR:$BUYER_ADMIN_CLASS com.iplanet.buyer.admin.impexp.cmdLn.impExpCmd -c "$CONFIG_FILE" "$@"

For a successful SSL session, add the server certificate to cacerts.

Additionally: To change the default SSL protocol (com.sun.net.ssl.internal.www.protocol) and/or the SSL Provider (com.sun.net.ssl.internal.ssl.Provider) values, you can:

• Use additional parameters in command lines (/t (SSL Protocol) and (/v SSL Provider)

  OR

• Set corresponding parameters in the configuration file, SSLPROTOCOL and SSLPROVIDER.