Administrator's Guide | |||||||||||||||||||||||||||||
Introduction Chapter 1 Compass Server Concepts Chapter 2 Configuring Server Preferences Chapter 3 Filling the Database | |||||||||||||||||||||||||||||
Chapter 4 Managing
the Compass Database
| |||||||||||||||||||||||||||||
Chapter 5 Setting Up
Categories Chapter 6 Customizing the User Interface Chapter 7 Monitoring the Server Glossary Index | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
The iPlanet Compass Server stores its descriptions of resources in a database. For the most part, you shouldn't need to work with the database directly, but there are some configuration and maintenance tasks you can perform through the Server Manager. This chapter describes the following database tasks: | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
Normally, the items in your Compass Server database come from the robot: You tell the robot which sites to visit, and it locates and describes all the resources it finds there. But you can also import databases of existing items, either from other Compass Servers, from iPlanet Web Servers or Netscape Enterprise Servers using the AutoCatalog feature, or from databases generated from other sources. There are a number of circumstances in which you might want to import existing databases of resource descriptions instead of sending the robot to create them anew. Most of them deal with either reducing the amount of network traffic or breaking the network into small enough parts that you can complete the indexing in a reasonable amount of time. Here are some examples:
| |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
When a Compass Server needs to import resource descriptions from another server or from a database, it uses an import agent. An import agent is a process that retrieves a number of resource descriptions from an external source and merges that information into the local database. In general, an import agent contains parameters that tell it where to go to import resource descriptions, what to ask for when it gets there, and some other information that fine-tunes the way it goes about the job. For more details, see Editing an Import Agent. The Import Agents form shows a list of all the currently defined import agents. If there are none currently defined, the list includes only a New button, used to create new import agents. If you have defined one or more import agents, there will be several other buttons below the list of import agents. When you run import agents, whether manually or automatically, the Compass Server runs all enabled import agents. An enabled import agent has the On button checked beside it in the list. Newly created import agents are enabled by default. You can enable or disable any of the defined import agents by clicking the On/Off buttons next to the desired items in the import agent list. A disabled import agent does nothing. You can also perform the following import tasks once you have defined at least one import agent: | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
When you want to import resource descriptions into your Compass Server database, you create an import agent to perform the task. Once you have created an import agent, you can reuse it as often as you need to. To create a new import agent, do the following:
| |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
The Import Agent Properties form allows you to change any aspect of an import agent. You use the same form both to create a new import agent and to modify an existing one. To edit an import agent, do the following:
The fields are described below. The following sections describe the fields in the Import Agent Properties form that define the parameters of an import agent.
| |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
There are several times when you might need to delete import agents:
In any case, the procedure for deleting the import agent is the same. To delete one or more import agents, do the following: The Delete Import Agents form shows a list of all existing import agents much like that on the Import Agents form, but instead of the On/Off option, there is a checkbox in front of each import agent.
After you delete import agents, you return to the updated Import Agents form. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
Once you have defined import agents, you can run them in either of two ways, manually (immediately) or automatically as a scheduled, periodic job. When you run import agents, you run them all as a batch. That is, the Compass Server goes through the list of all defined import agents, running all that are marked as "on," or enabled, and ignoring those marked as "off," or disabled. If you temporarily disable a number of import agents to run a manual import with only a specified subset of agents, you must be sure to reenable the disabled agents afterward so they run at the next scheduled time. Your choice of whether to run manually or on schedule probably depends on the nature of the import task. Running Import Agents Manually is most appropriate for either the first time you import, just to get started, or for one-time import jobs. Scheduling Import Agents is best for periodic updates from the same group of sources. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
In general, you need to run import agents manually only when you create a completely new Compass Server that needs an initial set of resource descriptions, when you have added new import agents to an existing Compass Server, or when you know that the source associated with an import agent has a large number of changes you want to incorporate into your database. To run import agents manually, do the following:
The Compass Server opens a new Navigator window and displays the progress
of the import agent process in that window. You can do other work while
the import agents run.
Do not close this import agent status window. Closing the window will cancel the import operation. You can minimize the window to get it out of your way, but do not close it or open another URL in the window until the import agent finishes its job. Note that by default an import agent imports all resource descriptions added to the source or changed since the last time it imported from that source. If you rerun an import agent immediately, it might appear to "fail," because it will not find any new or changed resource descriptions. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
Once you have your Compass Server running, you will probably set up any routine import agents to run automatically on a regular schedule. You should coordinate the import agent schedule with the robot's automated schedule, or at least run the import processes often enough to minimize any lag between fresh information on the remote servers and import into the database. Importing is a very efficient process that does not overburden your server. If anything, you should err on the side of importing too often rather than too infrequently. To run import agents automatically, do the following:
The Schedule task form indicates whether you currently have an automatic schedule activated and shows the day(s) and time of scheduled import agent runs, if any. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
A schema determines what information your Compass Server maintains on each resource, and in what form. The design of your schema determines two factors that affect the usability of your index: The schema is essentially a master data structure for resource descriptions
in the database. Depending on how you define and index the fields in that
data structure, users will have varying degrees of access to the resources.
The schema is intimately tied to the structure of the files used by the Compass Server and its robot. You should only make changes to the data structure by using the schema tools in the iPlanet Server Manager for your Compass Server. You should never edit the schema file (schema.rdm) directly, even though it is a text file. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
A schema in the context of the Compass Server is the definition of the contents of a resource description. That is, the schema determines the names of the fields in the resource description and the type of each field. For example, a schema for a document might have fields for the name of the document, the dates of its creation and last modification, its length, and so on. The schema for a Compass Server appears to the user only when choosing fields to display in search results or when constructing a complex query using the Advanced Search screen. In each case, the schema appears as a hierarchy of boxes, and the user's interaction with them is limited to choosing and arranging individual elements. As the administrator, you have the ability to control what items appear in the schema, how those fields are filled from incoming resource descriptions, and how users can use those items. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
Using the schema editor, you can modify any aspect of the schema for
your Compass Server. In all likelihood, the most common thing you will
do to the schema is add a searchable field, which is shown as an example
below.
To edit the database schema, do the following:
The schema appears in outline form on the left side of the applet. The right side shows the attributes of the selected field. The attributes are described in the table below. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
For each item in the datbase schema, you can change the following attributes:
| |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
When the robot encounters a META tag in a document, it converts the tag into a field in the resource description. If there is already an item in the schema with a corresponding name, it places the META tag contents in that field. If there is no predefined field with that name, it adds one, and assigns the value. There are two key facts here:
Even obscure or unique tags are included in the resource description. For example, suppose a document contains the following tag: <META NAME="Sauce" CONTENT="cranberry"> When the robot indexes the document, it will create a schema attribute called Sauce and assign it the value cranberry. Even though this word is not visible in the document, users will be able to search for the term cranberry and retrieve the document. To continue with the preceding example, the Advanced Search does not provide the ability to search for the word cranberry (or any other particular value) in the field called Sauce. If you want to enable users to search for terms in particular fields, you must define those fields in the schema and make them indexable, as shown in the following example. Suppose Airius Airlines has a corporate standard that every document that deals with a particular type of its planes contains a META tag called PlaneType that contains the names of the planes covered. The Compass Server administrator wants to enable users to search for documents concerning those particular types of planes. These are the steps to do so:
This creates a new schema item named New, with the name selected. When saving, the Compass Server needs to be off, so the Schema Editor can reindex the database. After the reindex is complete, the administrator can restart the server. Users can then go to the Advanced Search and choose PlaneType from the field list, and search for particular types of planes mentioned in that particular field. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
There are several instances where you might encounter discrepancies between the names used for fields in database schemas. One is when you import resource descriptions from one server into another. You cannot always guarantee that the two servers use identical names for items in their schemas. Similarly, when the robot converts HTML META tags from a document into schema fields, the document controls the names. The Compass Server deals with these by allowing you to define schema conversions, which are mappings of external schema names into valid names for fields in your database. You define your schema conversions on the Schema Conversion form. To convert incoming schema field names, do the following:
You can add more conversions by clicking More. If you want to delete the last conversion in the list, click Fewer. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
After you run the robot a number of times, the database files and indexes can become fragmented with empty space, causing them to take up more disk space than necessary and slowing down both user searches and robot operations. The solution to this is to periodically optimize the database. To optimize the database, do the following: | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
iPlanet Compass Server allows you to split the physical files that contain the Compass database across multiple disks, file systems, directories, or partitions. By spreading the database across different physical or logical devices, you can create a larger database than would fit on a single device. The Database Partitions form shows a list of up to 15 partitions defined for the database. By default, the Compass Server sets up the database to use only one directory. You can perform two kinds of manipulations on the database partitions: The Compass Server does not perform any checking to ensure that individual partitions have space remaining. It is your responsibility to maintain adequate free space for the database. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
You can add new database partitions up to a maximum of 15 total
partitions. Keep in mind, however, that once you increase the number of
partitions, you will need to delete the entire database if you later want
to reduce the number again.
To add partitions to your database, do the following:
Note that the Compass Server does not check to ensure that the pathname is valid. If you type an invalid pathname, the operation will fail. This creates the path if needed, redistributes the database records to take advantage of the new space, and reindexes the database. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
You can change the physical location of any of your database partitions by specifying the name of the new location. Similarly, you can rename an existing partition. To move a database partition, do the following: The Compass Server moves the partition to its new location, then reindexes the database. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
At times you will find it necessary to change the contents of one or more resource descriptions. For example, you might need to correct a typographical error copied into a resource description from an original document. You edit resource description contents using the RD Editor. Note: One specialized use of the RD Editor is to assign categories to resource descriptions when the robot fails to do so. This is described in Handling Unassigned Resources. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
There might be times when you want to delete your entire resource database. Such occasions might include extreme corruption of the database files, major redesign of the schema or taxonomy, or a dramatic change in the sites to be indexed. For whatever reason you want to delete the database, the procedure is the same. You should not simply delete database files from the disk. Doing so will likely result in your having to reinstall the Compass Server. To delete the entire resource database, do the following:
Similarly, the robot and all import agents should not be running when you delete the database, as they rely on the database being there. They will crash if you delete the database with them running. After deleting the database, you can run the robot or import resource descriptions to refill the database. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
One attribute of a resource description is its expiration date. Your robots can set the expiration date from HTML META tags or from information provided by the resource's server. By default, resource descriptions expire in three months from creation unless the resource specifies a different expiration date. Periodically your Compass Server should purge expired resource descriptions from its database. You can perform this task manually, or you can schedule it to occur automatically. To purge expired resource descriptions from a server, do the following: To schedule the purging of expired resource descriptions, do the following:
| |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
In certain instances, you might need to reindex the resource description database for the Compass Server. One obvious instance is if you have edited the schema to add or remove an indexed field. You might also need to reindex the database if a disk error corrupts the index file. It's also a good idea to reindex after adding a large number of new resource descriptions. The time required to reindex the database is roughly proportional to the number of records in the database, so if you have a large database, you should probably perform reindexing at a time when the server is not in high demand. To reindex the database, do the following:
The Compass Server rebuilds the search collection and its index files. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
The page-locked main Compass database can become 'hung' if a stale lock is left by an abnormally terminated process. Recovery resets the database state files and will free up hung locks. It also repairs damaged memory caches and transaction logs. Use it if the database appears to be hung and commands that read from or write to the database are freezing, ie, not terminating. Recovery should only be necessary in rare circumstances. | |||||||||||||||||||||||||||||
![]() | |||||||||||||||||||||||||||||
Each Compass Server stores its resource descriptions in a database. You can use the following procedure to get information about the number of sites indexed and the number of resources from each in the database. This report is also available to end users through the About Compass page. The Compass Database Analysis form indicates whether a database analysis page has already been generated, and if so, whether the page is up-to-date. It also includes the site analysis report in the form of a table.
![]()
| |