Chapter 8   Importing Data


This chapter describes the Import utility, and provides guidelines for using the import function from the graphical Admin interface or the command line.

---

Note	The information in this chapter is based on the assumption that you have already read Concepts and Release Notes.

---

This chapter contains the following sections:

• About Importing

• Administrator's Importing Tasks

• Creating an Import Data File

• Importing with the Admin Interface

• Importing with the Command Line

Instructions for importing catalog data are contained in Chapter 7 "Administering Catalogs," and in the iPlanet Market Maker Catalog Import Guide.

About Importing

---

The Import utility allows you to import batch data from external files into the BuyerXpert/SellerXpert database. Data for any of the BuyerXpert/SellerXpert resources can be updated this way, such as:

• Business rules

• Members

• Locations

• Seller catalogs

There are two ways to update your BuyerXpert/SellerXpert data:

• Individual transactions—Data is entered manually, using the data entry screens of the Admin interface. For the most part, instructions for using the various graphical screens are contained in the Admin interface Help.

  This method is generally used when your transactions are relatively simple, or when you are unfamiliar with the batch transaction process of the Import utility.

• Batch transactions—Data is entered as a batch, using the Import utility. An import session can be initiated either by using the Import screens in the Admin interface, or by invoking the import script from the command line.

  This method is used in some of these common situations:

  • You have a large quantity of updates.

  • The update is a repeating operation, where this updating is done on a regular basis. For example, updating a catalog price list.

  • When the transaction is complex. If you have to go to many screens in the Admin interface to complete the transaction, it may be more efficient to use the batch process, even for a single transaction.

The import process involves two elements:

• Import data file(s)—text files that contain commands and data for updating BuyerXpert/SellerXpert resources

• Import utility—a program that processes the data in the import data file

Import Data File

An import data file is the text file, in comma-separated values (CSV) format, that contains the data that will modify the BuyerXpert/SellerXpert database. Import data files must have the file extension .imp, such as people.imp or catalogABC.imp.

An import data file is written to update a particular resource. Each resource has a different set of attributes that are to be updated, some required, some not. A list of attributes for every BuyerXpert/SellerXpert resource can be found in the templates in the sample file directory:

IAS_Root/buyer/sample_data/impexp_files/

An import data file contains four different types of information lines: comments, commands, headers, body. Table 8-1 describes these information lines.

Table 8-1    Import Data File Contents 

Line Type	Line Starts with	Function	Example
Comment line	# symbol	Describes file contents.	# Import Data File for Location
Command line	< symbol	Identifies a command and resource name, separated by a comma. May contain default values for some resource attributes.	<update, Location
Header line	^ symbol	Describes the format for the body lines. Lists the attributes required for resource mentioned in command line	^country_code, postal_code, from_date
Body line	no preceding character	Lists operands, separated by a comma. Operand is usually one value for each attribute. If more than two values are present, they are separated by special symbols.	France, 1234, 01/01/2001

During processing, blank lines and comments are ignored.

Symbols Used in an Import Data File

The import data file uses special symbols. Table 8-2 lists these symbols and an example of each.

Table 8-2    Import Data File Symbols

Symbol	Data Type	Description	Example
,	List	Separates a list of values.	12345, "this is a string", 11/27/00
|	Old or new value	Separates an old and a new value.	12345 | 54321
&	Multivalues	Separates multiple values.	London & Denver & Paris
" "	Strings	Encloses alphanumeric string values and spaces.	"This is a 5th string"
\	Escape, Continuation	Preserves literal value; indicates line continuation.	This is a string\, wi\|th spe\|\|cial symbols \ inside\]
[ ]	Reference	Encloses a reference to an object within an object.	[ImpTest attrInt=1 attrString= "First Test"], \ 11/27/00

Commands Used in an Import Data File

A command line identifies the command and the resource that will be processed during importing. Any or all of the following commands are recognized by the Import utility and can be included in an import data file:

• insert—Add new data associated with these body lines.

• insert_overwrite—Add data associated with these body lines; if the data already exists in the database, replace it with this data.

• update—Update existing data with the data in these body lines.

• delete—Delete data associated with these body lines.

A command line is followed by a header line; a header line is followed by body lines. Each time a new command in specified, its associated header and body lines follow. For example, for a delete command, there will be a set of header-body lines, for an update command, another set of header-body lines, and so on.

Example—Import Data File
When all the lines are combined, they make up an import data file.

# Location Import file

#

# commands supported are:

#    insert: required attributes - common_name, parent_org

#    insert_overwrite: required attributes - common_name, parent_org

#    update: required attributes - reference_name

#    delete: required attributes - reference_name

#

# Command line: insert command

<insert, Location

#

# Header line: Lists fields for body lines;
# Required fields: common_name,parent_org

^common_name, parent_org, country_code, postal_code, \

effective_from_date, effective_to_date,telephone_number, \

fax_number, websites Paristest

#

# Body line

Paristest,[Organization name=mercury.com], France, 1234, 01/01/2000, 01/01/2020, \

(408)111-2222&(408)333-4444, (510)111-2222&(510)333-444, a@b.com&a@c.com
#

# Example of multivalued attributes

# Command line

<update, Location

# Header line: Lists fields for body lines
# Required field: reference_name

^ reference_name, add_websites,del_websites, add_telephone_number, \ del_telephone_number, add_fax_number, del_fax_number

#

# Body line

[Location common_name=Paristest organization=mercury.com], \

C.b.com&D.b.com&E.b.com,AB.b.com&CD.b.com, 715&716, 716&761, 815&816, 816&861

#

# Command line: insert_overwrite command

<insert_overwrite, Location

#

# Header line: Lists fields for body line

# Required fields: common_name, parent_org

^common_name, parent_org, country_code, postal_code, postal_address, definable_id,telephone_number

#

# Body line

Paristest,[Organization name=mercury.com], France, 1234, LocAddress,0000, \

(408)111-2222&(408)333-4444&(408)555-6666

#

# Command line: delete command

# Required field: reference_name

<delete, Location

#

# Header line: Lists fields for body lines

^reference_name

[Location common_name=Paristest organization=mercury.com]

Import Utility

Importing can be accomplished either using graphical screens in the Admin interface or using the command line.

Admin Interface Screens

The Admin interface provides screens for defining an import session and managing it through to completion. You can monitor the progress of a session, stop a session, or resume a stopped session. In addition, you can use the Drop option to clean up the Import utility data tables and log files after the session has completed.

---

Note	The main advantage of using the graphical Import screen rather than the Import script, is that you can easily monitor the progress of a session.

---

Instructions are contained in the "Importing with the Admin Interface" section in this chapter or in the Admin interface Help.

Command-Line Interface

BuyerXpert/SellerXpert provides a UNIX shell script that allows you to run the Import utility as a batch process from the command line. An advantage of the command line is that you can import an unlimited number of files in a single batch.

Instructions for using the import script and its options are contained in the "Importing with the Command Line" section in this chapter.

Administrator's Importing Tasks

---

Table 8-3 lists typical ongoing administrative tasks for importing. The right-hand column provides information about where to find instructions, guidelines, or tools for performing that task.

Table 8-3    Ongoing Importing Tasks 

Task	Location of Instructions/Guidelines/Tools
Create an import data file.	Creating an Import Data File section
Import the data file using the graphical Admin interface Import screens.	Importing with the Admin Interface section
Import the data file using the Import script on the command line.	Importing with the Command Line section

Creating an Import Data File

---

For any importing, you must first create an import data file for the specific resource you are importing, for example, payment data. Each different type of resource usually has its own import data file. Sample import data files are supplied with BuyerXpert/SellerXpert for the common BuyerXpert/SellerXpert resources.

---

Tip	It is recommended that you use an appropriate sample file for creating your import data file rather than writing a file from scratch.

---

An import data file has specific contents requirements, which must be followed precisely. The first time you create an import data file for a resource, you will probably encounter a number of import failures before the file is correct. Once you have done a successful import for that type of resource, you can reuse the file for subsequent importing.

Use the following guidelines to create an import data file for a resource:

1. Copy the sample import data file for your resource to your working directory. Sample import data files are located in the following directory:

   IAS_Root/buyer/sample_data/impexp_files/

2. Use a text editor to edit the comment, command, and header lines for your resource.
   

   ---

   Note	Each command you plan to use in the import session will need its own command/header/body section in the import data file.

   ---

   
   

3. Insert the body lines for the import text.

   The sample files contain all the possible attributes that come with a BuyerXpert/SellerXpert resource. You will need to delete or modify the attributes where your data is different.

4. Verify that the attributes listed in the header lines correspond to the values in the body lines.

5. Verify that all symbols are used correctly.

Importing with the Admin Interface

---

After you have created an import data file, you can use the Admin interface Import function to import the data. The following high-level options are available when you click the Import tab in the Admin interface:

• Start New Session—Use to start a new session by specifying the import data file(s).

• Session Manager—Use to manage existing sessions (both active and inactive).

• Session Status—Use to gather status information.

Follow these guidelines to import a data file using the Admin interface (field and procedure details are contained in Help):

1. Select Import.

2. Enter the number of files you want to import, up to ten.

3. Click Change.

4. Enter the file name or URL of the file location. If you are unsure of the name or location, click Browse to find the files you want to import.

5. Click Upload Files.

   The Start Session screen appears.

6. Click Start to import all files that you have selected.

7. The Session Status screen displays the state of the current session.

Follow procedures in Help to perform additional tasks.

Importing with the Command Line

---

The Import utility batch process can be initiated from the command-line by running the BXImport.sh shell script. The script for BuyerXpert is located in the IAS_ROOT /buyer/admin/bin directory along with the default configuration file, cli.conf.The script for SellerXpert is located in the IAS_ROOT /seller/admin/bin directory along with the default configuration file, cli.conf.

Some values are set during installation (for example, the path to JAVA classes, or the path to the default configuration file). You can enter script options directly on the command-line or indirectly through a configuration file.

---

Note	Options passed from the command line override any options set in a configuration file.

---

Syntax

BXImport.sh [options] import_files

Table 8-4 lists the options that are used with the import script.

Table 8-4    Import Options

Options	Description
-u URL	Web server location for Import utility servlets
-n UserId	User name (login name)
-p password	User password (login password)
-o orgName	User organization (login organization)
-c configFile	Configuration file
-i inDir	Input files directory. Default is current directory.
-d outDir	Output files (report and log) directory. Default is current directory.
-f outFiles	Output files (discard and log). Default is session name + extension.
-s silent	Requires Boolean silent mode (no dialog with user) flag. True/False. Default is False.
-b blind	Requires Boolean blind mode (user only sees fatal errors) flag. True/False. Default is False.
-a true | false	Requires Boolean auth mode (authentication is required) flag. True/False. Default is True.
import_filename	Name of import data file (.imp and.csv are valid extensions)

If you don't specify any options when you invoke the import script on the command line, the default values in the cli.config file are used, as shown in Example 1.

Example 1—Default Configuration File
The following example contains typical values that the import script would use to start the Import utility from the command line.

# cli.conf

URL = http://buyerhost:80

NAME = admin

PASSWORD = admin

ORGANIZATION = mercury.com

SILENT = false

BLIND = true

INDIR = ./

OUTDIR = ./

If you want to use other values, you can enter specific options when you start the import script. These two methods are illustrated in Example 2 and Example 3.

Example 2—   Using Default Values
The following command imports two files, locations.imp and usergroups.imp, from /import_files directory. All settings are taken from the default configuration file.

# BXImport.sh /import_files/locations.imp /import_files/usergroups.imp

Example 3—Redefining Default Values
This command imports two files, orders.imp and payments.imp, from the /import_files directory. Option settings are:
Login name - admin.login password - admin; Organization - corp.com.

All other settings are taken from the default configuration file.

# BXImport.sh -n admin -p admin -o corp.com /import_files/orders.imp /import_files/payments.imp\

By default, the import script uses the cli.conf file which consists of default settings for the options. If you want to redirect to another configuration file, you can as shown in Example 4.

Example 4—Redirect to a Configuration File
The following command imports one file, /import_files/users.imp, using the option settings from specific configuration file, /myfiles/myconf.conf.

# BXImport.sh -c /myfiles/myconf.conf /import_files/users.imp