Chapter 7   Using the RDM API to Access the Search Engine and Database in C

This chapter describes the use of the search engine Resource Description Manager (RDM) API to access the search engine and its database. The RDM API provides routines to parse, modify, and create resource description messages (RDMs) using C.

The robot generates RDs and saves them to the search engine database. You can retrieve the RDs in SOIF format, use the RDM API to modify them, and then have the search engine import them back.

The RDM API is defined in the rdm.h file in the following directory in your search engine installation directory:

BaseDir/SUNWps/sdk/rdm/include

This chapter is restricted to discussing the use of C functions that come with the search engine RDM API. Therefore, it is strongly recommended that you have a basic understanding of the C programming language.

What is an RDM?

A Resource Description Message (RDM) is a messaging format which two processes can use to exchange resource descriptions across a network. In RDM, one process (a client or agent) sends a request RDM message to a remote server which processes the request, then sends a response RDM message, similar to the HTTP/1.0 request/response model.

For more information about RDM, see:

http://www.w3.org/TR/NOTE-rdm.html

For example, you can send an RDM request to a search engine database to request RDs that match certain criteria. The search engine will send back an RDM response that contains the requested RDs (for example, all documents containing the string style_sheets).

RDM Format Syntax

Each RDM message contains a header and a body. The header identifies the nature of the RDM message, while the body contains any data required to carry out the needed request (for example, scoping criteria). Both the header and body of the RDM message is encoded using SOIF.

RDM Header

The RDM header section begins with a SOIF object whose schema name is RDMHEADER which must contain as least the following attributes from Table 7-1:

Table 7-1    RDM Header Attributes

Attributes	Description
rdm-version	A string identifying the version of the message specification. (for example, 1.0).
rdm-type	A string identifying the nature of this message.
rdm-query-language (required only for Request messages)	A string which identifies which query language is used in the given request.
catalog-service-id (optional)	A CSID identifying the catalog to which the request/response applies (for example, x-catalog://siroe.budgie.com:80/finance. If not present, the RDM server will use its default catalog.

The following example shows two RDM headers and their attributes:

@RDMHEADER { - rdm-version{3}: 1.0 rdm-type{14}: status-request } @RDMHEADER { - rdm-version{3}: 1.0 rdm-type{10}: rd-request rdm-query-language{6}: search catalog-service-ID{39}: x-catalog://budgie.siroe.com:80/finance }

RDM Body

The RDM message body contains any data needed to carry out the request. For example, if the header is rd-request, which indicates a query request, the body must contain the query criteria or scope. For example, if you are sending a request to find all documents that contain the string varrius, then the body must be SOIF object whose schema-name is RDMQuery and whose scope attribute is varrius:

@RDMQUERY { - scope{7}:varrius }

If the RDM message is a query request, the body of the message can also indicate which attributes of the resulting RDs are wanted, the number of results, and sort preference. For example, the following search criteria in the RDM body specifies the URL, title, author, and last-modified attributes. The result set contains at most 10 hits and the resulting set is ordered by the title attribute:

@RDMQUERY { - scope{7}:varrius view-attributes{30}: url,title,author,last-modified view-hits{2}: 10 view-order{5}: title }

If the RDM message is another kind of request, such as a Schema-Description-Request, a Server-Description-Request, or a Status-Request, the body of the message should contain appropriate data. See the relevant document for more details on RDM bodies.

About the RDM API

The file rdm.h defines structures and functions for creating RDMs. The intent of these functions is to construct queries and instructions that can be output as RDM and sent via the sendrdm program to a search engine for processing. These queries and instructions are created in the C programming language.

The basic structures are:

• RDMHeader
• RDMQuery

To support each of these main structures, there are additional structures like RDMViewAttributes, RDMViewOrder, and RDMViewHit which are used to represent attributes in an RDMQuery.

The RDM API defined in rdm.h provides functions for creating and modifying these structures. For example, the following statement:

RDMQuery *myquery = RDMQuery_Create("varrius");

creates an RDMQuery that corresponds to the following SOIF:

@RDMQUERY { - scope{7}:varrius }

The following statement:

RDMQuery_SetViewAttr(myquery, "URL,Title");

modifies the RDMQuery so that it corresponds to the following SOIF:

@RDMQUERY { - scope{7}:varrius view-attributes{30}: URL,Title,Author,Last-Modified }

Both the RDMHeader and RDMQuery structures have soif fields, which contain the SOIF data for the header or the query. To extract SOIF data from RDMHeader or RDMQuery, you can access the soif field. Thus, you can also use the RDM SOIF API to create and modify RDMHeader and RDMQuery objects.

Example of Submitting a Query

To send the RDMHeader and RDMQuery objects to the search engine to perform a query request, you can use the pre-built sendrdm program (located in BaseDir/SUNWps/lib directory). This program takes an RDM request (which is a message in SOIF format), sends it to the search engine, and outputs the results as a SOIF stream. You can also use HTTP to post an RDM stream directly to the server through its URL (http://server:port/URI/search). The program must be run in a search-enabled Sun^TM ONE Portal Server software instance directory such as the default /var/opt/SUNWps/serverinstance/portal directory.

You should also change the definitions for MY_CSID, MY_SCOPE and MY_ATTR_VIEW to suit your needs.

Table 7-2    RDMHeader and RDMQuery Object Parameters

Parameter	Description
MY_CSID	ID of the search engine instance. This parameter specifies the specific search engine that you created in the search engine Administration Interface. You can find the exact value in /var/opt/SUNWps/serverinstance/portal/config/search.conf. For SSL enabled server instances, use x-catalogs instead of x-catalog in the CSID.
MY_SCOPE	Query string. The default is to search for documents containing the string varrius. For example, if you want to search for all documents containing the string style_sheets, then set MY_SCOPE to style_sheets.
MY_ATTR_VIEW	Attributes to be printed, such as URL, title and description.

Code Example 7-1 generates an RDM for querying a search engine database. You can pipe the output of the sample program to a temporary file and then use the sendrdm program to post it to the search engine.

---

Code Example 7-1    RDM API to Submit a Query Example 

/******* Example Use of the RDM API to submit a query *******/

#include <stdio.h>

#include "soif.h"

#include "rdm.h"

#define MY_SCOPE "varrius"

#define MY_CSID "x-catalog://budgie.siroe.com:6714/search1"

#define MY_ATTR_VIEW "title,content-type"

int main(int argc, char *argv)

{

RDMQuery *myquery = NULL;

CSID *csid = NULL;

RDMHeader *myheader = NULL;

SOIFStream *out = SOIF_PrintInitFile(stdout);

/* Create the RDMQuery and specify its scope */

if(!(myquery = RDMQuery_Create(MY_SCOPE))) {

perror("RDMQuery_Create\n");

exit(-1);

}

/* Set the view attributes of the RDMQuery */

RDMQuery_SetViewAttr(myquery, MY_ATTR_VIEW);

/* Create the CSID that points to your search engine instance */

if(!(csid = CSID_Parse(MY_CSID))) {

perror("CSID_Parse\n");

exit(-1);

}

/* Create the RDMHeader */

if(!(myheader = RDMHeader_CreateRequest(RDM_RD_REQ, "search",

csid))) {

perror("RDMHeader_CreateRequest\n");

exit(-1);

}

/* print the RDMHeader to the output SOIF stream */

/* print the RDMQuery to the output SOIF stream */

(*out->print)(out, myheader->soif);

(*out->print)(out, myquery->soif);

/* free the structures and exit */

RDMHeader_Free(myheader);

RDMQuery_Free(myquery);

SOIFStream_Finish(out);

exit(0);

}

/*********** EOF ****************/

Running the Example

To run the example described in Code Example 7-1, follow these steps:

1. Edit the definitions for MY_SCOPE, MY_CSID, and MY_ATTR_VIEW as appropriate for your situation. For more information, see Table 7-2.
2. Save the file in BaseDir/SUNWps/sdk/rdm/examples. For example, save the file as example4.c in BaseDir/SUNWps/sdk/rdm/examples.
3. Create a makefile. You can find sample makefiles at BaseDir/SUNWps/sdk/rdm/examples. Edit the makefile to include example4.c. Code Example 7-2 shows the makefile with the changes needed for example4.c in bold:



---

Code Example 7-2    Makefile for SOIF/RDM examples 

# Makefile for SOIF/RDM SDK examples

# Use make and cc.

CC = cc

SDKDIR = ..

SDKLIB = $(SDKDIR)/lib/librdm.a

SDKINC = $(SDKDIR)/include/

CFLAGS = -I$(SDKINC) -DXP_UNIX

CFLAGS += -DSOLARIS

#CFLAGS += -DIRIX

#CFLAGS += -DHPUX

#CFLAGS += -DAIX

EXAMPLES = example1 example2 example3 example4

all: $(EXAMPLES)

example1: example1.o

$(CC) -o $@ $@.o $(SDKLIB)

example2: example2.o

$(CC) -o $@ $@.o $(SDKLIB)

example3: example3.o

$(CC) -o $@ $@.o $(SDKLIB)

example4:       example4.o

$(CC) -o $@ $@.o $(SDKLIB)

4. From the directory BaseDir/SUNWps/sdk/rdm/examples, build the example as follows:

Solaris: gmake

5. From the directory BaseDir/SUNWps/sdk/rdm/examples, run the example4.c program to generate the RDM file.

example4.c > rdm.soif

The file rdm.soif will look like the following example:

@RDMHEADER { - catalog-service-id{40}:x-catalog://budgie.siroe.com:6714/budgie rdm-version{3}: 1.0 rdm-type{10}: rd-request rdm-query-language{6}: search } @RDMQUERY { - view-attributes{18}: title,content-type scope{5}: varrius }

The file rdm.soif created in must be placed into the server instance directory.

6. Send the SOIF contained in rdm.soif to the program sendrdm. For example, type:

./run-cs-cli sendrdm -u /portal/search rdm.soif

The current directory should be the server instance directory. If rdm.soif is not in the server instance directory, reference the file from where is was created. For example:

1. Change directories to server_instace_dir.
2. Type ./run-cs-cli sendrdm -u /portal/search sdk_dir/rdm.soif.

The results of the sendrdm program will be a SOIF stream containing the results of the query, such as the following example:

@RDMHEADER { - catalog-service-id{41}:x-catalog://budgie.siroe.com:6714/budgie rdm-version{3}: 1.0 rdm-type{11}: rd-response rdm-response-interpret{51}:20 results out of 36281 hits across 88985 documents } @DOCUMENT { http://fury.sesta.com:999/it/newsref/pr/newsrelease417.html content-type{9}: text/html score{3}: 100 title{17}: Comunicato stampa } @DOCUMENT { http://fury.sesta.com:999/it/newsref/pr/newsrelease374.html content-type{9}: text/html score{3}: 100 title{17}: Comunicato stampa }

You can pipe the output of sendrdm (which is a SOIFStream) to another program to print the results of the query.

API Reference

The remaining sections in this chapter describe the RDM API, located in directory BaseDir/SUNWps/sdk/rdm/include/rdm.h.

Finding the RDM Version

RDM_Version

NSAPI_PUBLIC const char *RDM_Version(void);

Returns the version of the RDM library.

Creating and Freeing RDM Structures

For most RDMX structures, such as RDMRequest, RDMQuery, and so on, there are two or more creation functions. There is usually an RDMX_Parse() function, which takes SOIF arguments, and an RDMX_Create() function, which takes non-SOIF arguments.

There is also an RDMX_Free() function, which releases the object.

Several RDM structures also have an RDMX_merge() function, which merges data from a SOIF object into an existing RDMX structure.

RDMHeader

An RDMHeader represents the header of an RDM. An RDMHeader structure has one public field, soif, which is a SOIF containing a collection of attribute-value pairs. The allowable attribute names of the attribute-value pairs in the SOIF are:

• rdm-version (required) -- This is set automatically if you use RDMHeader_CreateRequest() or RDMHeader_CreateResponse() to create the RDMHeader.
• rdm-type (required)
• catalog-service-id (recommended, this indicates which search engine instance to send the RDM request to).
• rdm-query-language (required for a request) -- For communicating with the search engine, you can specify "search".

Response RDMs also have several attributes, and you can find them in rdm.h.

The following statements create an RDMHeader whose RDMType is RDM_RD_REQ, and whose query language is search. This RDM will be sent to the search engine instance search1 of the search engine at http://budgie.siroe.com:6714.

CSID *csid = CSID_Parse("x-catalog://budgie.siroe.com:6714/search1");

RDMHeader *myheader = RDMHeader_CreateRequest(RDM_RD_REQ, "search", csid);

Table 7-3 describes the allowable values for RDM-Type:

Table 7-3    Allowable Values in RDM-Type

RDM-Type Value	Description
RDM_MSGTYPE_UNKNOWN	Undefined or unknown message type
RDM_RD_UPD_REQ	Requesting an RD update
RDM_RD_REQ	Requesting an RD update (same as RDM_RD_UPD_REQ)
RDM_RD_DEL_REQ	Requesting an RD delete
RDM_RD_UPD_RES	Responding to an RD update request
RDM_RD_RES	Responding to an RD update request (same as RDM_RD_UPD_RES)
RDM_RD_DEL_RES	Responding to an RD delete request
RDM_SD_REQ	Requesting a server description
RDM_SD_RES	Responding to a server description request
RDM_SCH_REQ	Requesting a schema description
RDM_SCH_RES	Responding to a schema description request
RDM_TAX_REQ	Requesting a taxonomy description
RDM_TAX_RES	Responding to a taxonomy description request
RDM_STAT_REQ	Requesting a status
RDM_STAT_RES	Responding to a status description request

You can use the following macros to get and set the string values of the attributes:

RDMHeader_GetType(RDMHeader) /* returns RDMType */

RDMHeader_GetVersion(RDMHeader)

RDMHeader_GetQueryLanguage(RDMHeader)

RDMHeader_GetCSID(RDMHeader)

RDMHeader_GetResponseInterpret(RDMHeader)

RDMHeader_GetErrorMessage(RDMHeader)

RDMHeader_GetErrorNumber(RDMHeader)

RDMHeader_SetType(RDMHeader,RDMType)

RDMHeader_SetVersion(RDMHeader,stringvalue)

RDMHeader_SetQueryLanguage(RDMHeader,stringvalue)

RDMHeader_SetCSID(RDMHeader,stringvalue)

RDMHeader_SetResponseInterpret(RDMHeader,stringvalue)

RDMHeader_SetErrorMessage(RDMHeader,stringvalue)

RDMHeader_SetErrorNumber(RDMHeader,stringvalue)

RDMHeader_Parse

NSAPI_PUBLIC RDMHeader *RDMHeader_Parse(SOIFStream *ss);

RDMHeader_Create

NSAPI_PUBLIC RDMHeader *RDMHeader_Create(RDMType t);

RDMHeader_CreateRequest

NSAPI_PUBLIC RDMHeader *RDMHeader_CreateRequest(RDMType, char *ql,

CSID *)

RDMHeader_CreateResponse

NSAPI_PUBLIC RDMHeader *RDMHeader_CreateResponse(RDMType, char *ri,

CSID *);

RDMHeader_Free

NSAPI_PUBLIC int RDMHeader_Free(RDMHeader *r);

RDMHeader_Merge

NSAPI_PUBLIC int RDMHeader_Merge(RDMHeader *, SOIF *)

Merges data from a SOIF object into the RDMHeader object.

RDMQuery

An RDMQuery represents the body of an RDM. An RDMQuery structure has one public field, soif, which is a SOIF containing a collection of attribute-value pairs. The allowable attribute names of the attribute-value pairs in the SOIF are:

• scope (required)
• view-attributes (optional)
• view-hit (optional)
• view-order (optional)
• view-template (optional)

You can use the following macros to get and set the string values of these attributes:

RDMQuery_GetScope(query)

RDMQuery_GetViewAttr(query)

RDMQuery_GetViewHits(query)

RDMQuery_GetViewOrder(query)

RDMQuery_GetViewTemplate(query)

RDMQuery_SetScope(query, value)

RDMQuery_SetViewAttr(query,value-list)

RDMQuery_SetViewHits(query,value)

RDMQuery_SetViewOrder(query,value-list)

RDMQuery_SetViewTemplate(query,value)

RDMQuery_Parse

NSAPI_PUBLIC RDMQuery *RDMQuery_Parse(SOIFStream *ss);

RDMQuery_Create

NSAPI_PUBLIC RDMQuery *RDMQuery_Create(const char *scope);

RDMQuery_Free

NSAPI_PUBLIC int RDMQuery_Free(RDMQuery *);

RDMQuery_Merge

NSAPI_PUBLIC int RDMQuery_Merge(RDMQuery *, SOIF *);

Other RDM Structures

In addition to RDMHeader and RDMQuery, the file rdm.h provides definitions and functions for the following auxiliary objects. See the file rdm.h in BaseDir/SUNWps/sdk/rdm/include for details.

• RDMRequest
• RDMResponse
• RDMServer
• RDMView
• RDMViewHits
• RDMViewOrder
• RDMTaxonomy
• RDMClassification
• RDMSchema