soif.h file in the following directory in your Compass Server installation directory:
bin/compass/sdk/rdm/includeThis chapter does not teach you how to use C. This chapter is restricted to discussing the use of C functions that come with the Compass Server SOIF API.
@DOCUMENT { http://www.best.com/~jocelyn/resdogs/index.html
title{20}: Rescuing English Springer Spaniels
author{29}: Jocelyn Becker
}
Each SOIF object has a schema-name (or template type) and an associated URL, and it contains a list of attribute-value pairs. In this case, the schema name is @DOCUMENT, which indicates this resource is a document. Title and author are both attribute names, and you can see that each attribute has a value.
soif.h header file in compassdir/bin/compass/sdk/rdm/include.
The SOIF API defines structures and functions for working with SOIF objects. For example, the following code uses the functions SOIF_Create() and SOIF_InsertStr() to create a SOIF and add some attribute-value pairs to it:
SOIF mysoif=SOIF_Create("DOCUMENT", "http://someurl/doc.htm");
SOIF_InsertStr(mysoif, "title", "All About Style Sheets");
SOIF_InsertStr(mysoif, "author", "Robin Styles");
SOIF_InsertStr(mysoif, "description", "All you need to know about style sheets");
These command create a SOIF that looks like:
@document {http://someurl/doc.htm
title{22}: All About Style Sheets
author{12}: Robin Styles
description{38}: All you need to know about style sheets
}
Each SOIF object contains attribute-value pairs, which are each represented as SoifAVPair objects. Using the SOIF API, you can get and set values of attributes, you can create and delete attribute-value pairs, you can change the values of attributes, and you can add values to existing attributes. (Some attributes can have multiple values.)
Multiple SOIF objects can be grouped together into SOIF streams, which are represented by SOIFStream objects. A SOIFStream object provides functionality for handling a stream of SOIF objects. For example, you can use the stream to filter attributes, and print the desired attributes for every SOIF in the stream.
The relevant data structures are:
compassdir/bin/compass/sdk/rdm/examples.
This section discusses an example that is similar to (but not necessarily identical to) example1.c. It shows how to iterate through a SOIF stream and print the URL and number of attributes of each SOIF in the stream.
This example assumes that you have already created a file containing a SOIF stream which is available on stdin. For example, you could have created a SOIF stream containing one or more RDs from the Compass Server database, which you would do by using the routines in RDM.h (as is discussed in Chapter 12, "Using the RDM API To Accesss the Compass Server Database in C").
This example uses SOIF_ParseInitFile() to create a SOIF stream from the standard input.
/* Example 1 - Simple SOIF Stream Parsing */
#include <stdio.h>
#include <stdlib.h>
#include "soif.h"int main(int argc, char *argv[])
{
/* Define a SOIFStream and SOIF */
SOIFStream *ss;
SOIF *s;
char *titleptr; /* Open a SOIF stream that gets its SOIF from stdin */
ss = SOIF_ParseInitFile(stdin);
/* SOIFStream_IsEOS() checks if this is the end of the stream */
/* ss->parse gets the next SOIF in the stream */ while (!SOIFStream_IsEOS(ss)) {
if (!(s = SOIFStream_Parse(ss)))
/* Exit the loop if the SOIF is invalid */
break; /* Print the URL for each SOIF (will be "-" if there is no URL)*/
printf("URL = %s\n", s->url); /* Print the title if it exists. */
titleptr = SOIF_Findval(s, "title");
printf("Title = %s\n", titleptr ? titleptr : "(none)") /* Print the number of attributes in the SOIF*/
printf("# of Attributes = %d\n", SOIF_GetAttributeCount(s)); /* release the memory used by the SOIF */
SOIF_Free(s);
} /* Close the SOIFStream and exit */
SOIFStream_Finish(ss);
exit(0);
} Getting the Compass Server Database Contents as a SOIFStream
You can retrieve the entire contents of the Compass Server database as a SOIF stream by using the minidba utility.
From the compass-installdir/compass-name/ directory, run the following command:
../bin/compass/bin/minidba dump
This command prints the database contents as a SOIFStream. You can pipe the output to a program that uses SOIFStream routines to parse the SOIFs in the stream.
SOIF API
The functions and objects defined in the soif.h header file are listed here by category. The categories are:
append 149
increase 149
reset 150
SOIF_Apply 139
SOIF_AttributeCompare 140
SOIF_AttributeCompareMV 141
SOIF_Contains 144
SOIF_Create 137
SOIF_DeleteMV 142
SOIF_Find 138
SOIF_Findval 138
SOIF_FindvalMV 143
SOIF_Free 137
SOIF_GetAttributeCount 137
SOIF_GetAttributeSize 137
SOIF_GetTotalSize 137
SOIF_GetValueCount 138
SOIF_GetValueSize 137
SOIF_Insert 139
SOIF_InsertAVP 139
SOIF_InsertMV 142
SOIF_InsertStr 140
SOIF_IsMVAttribute 142
SOIF_Merge 138
SOIF_MVAttributeParse 142
SOIF_ParseInitFile 146
SOIF_ParseInitStr 146
SOIF_PrintInitFile 146
SOIF_PrintInitFn 146
SOIF_PrintInitStr 146
SOIF_Remove 138
SOIF_Rename 140
SOIF_Replace 140
SOIF_ReplaceMV 142
SOIF_ReplaceStr 140
SOIF_SqueezeMV 143
SOIFAVPair_Create 140
SOIFAVPair_Free(SOIFAVPair *avp) 140
SOIFAVPair_IsMV 143
SOIFAVPair_NthValid 143
SOIFAVPair_NthValue 144
SOIFAVPair_NthVsize 144
SOIFBuffer_Create 149
SOIFBuffer_Free 149
SOIFStream_Finish 146
SOIFStream_GetAllowed 148
SOIFStream_GetDenied 149
SOIFStream_IsAllowed 148
SOIFStream_IsEOS 147
SOIFStream_IsParsing 147
SOIFStream_IsPrinting 147
SOIFStream_Parse 147
SOIFStream_Print 147
SOIFStream_SetAllowed 148
SOIFStream_SetDenied 148
SOIFStream_SetFinishFn 147
@DOCUMENT { http://www.netscape.com/
title{20}: Welcome to Netscape!
author{15}: Marc Andreessen
}
A SOIF object has url and schema-name fields to store its url and schema name:
char *url; /* The URL */A SOIF object contains a collection of
char *schema_name; /* The Schema-Name, such as @document or @RDMHeader*/
SoifAVPair objects, which each contain an attribute and one or more values. To access attribute values in a SOIF, use SOIF_find() to retrieve the AVPair for the given attribute, or use SOIF_findval() to retrieve the value string for a given attribute. You must use all lowercase for attribute names for find*(), since only exact attribute name lookups are supported.
You can create SOIF objects by using the SOIF_create() function. You can also read SOIF objects from a SOIF stream.
NSAPI_PUBLIC SOIF *SOIF_Create(char *schema_name, char *url)Creates a SOIF structure with the given schema name and URL.
NSAPI_PUBLIC void SOIF_Free(SOIF *)Frees the given SOIF structure.
NSAPI_PUBLIC int SOIF_GetTotalSize(SOIF *s)Gets the estimated total size of the SOIF in bytes.
NSAPI_PUBLIC int SOIF_GetAttributeCount(SOIF *s)Gets the number of attributes in the SOIF.
NSAPI_PUBLIC int SOIF_GetAttributeSize(SOIF *s)Gets the size of the attributes only.
NSAPI_PUBLIC int SOIF_GetValueSize(SOIF *s)Gets the size of the values only.
NSAPI_PUBLIC int SOIF_GetValueCount(SOIF *s)Gets the number of values only.
NSAPI_PUBLIC int SOIF_Merge(SOIF *dst, SOIF *src);Use this function to merge two SOIF objects (perform a Union of their attribute-values). It returns non-zero on error; otherwise, returns zero and the 'dst' SOIF object contains all the attribute-value pairs from the 'src' SOIF object. If the 'dst' object contains the same attribute as 'src', then the attribute becomes a multi-valued attribute and all of the values are copied over to 'dst'. Only multi-valued attributes are copied over. For single-value attributes, discard the value in 'dst'. Currently only "classification" is a multi-valued attribute.
#define SOIF_Find(soif, attribute-name)retrieves the
AVPair for the given attribute in the given soif. For example, the following statement gets the AVPair for the title attribute in the soif s:
SOIFAVpair avp=SOIF_Find(s, "title");
#define SOIF_Findval(soif, attribute-name)retrieves the value string for the given attribute in the given soif. For example, the following statement prints the value of the title attribute of the soif
s:
printf("Title = %s\n", SOIF_Findval(s, "title"));#define SOIF_Remove(soif, attribute-name)Removes the given attribute from the given soif.
#define SOIF_Insert(soif, attribute-name, value, value-size)Inserts the given attribute and the value of the given size as an
AVPair into the soif.
#define SOIF_InsertAVP(soif, avpair)Inserts the given
AVPair into the given soif.
#define SOIF_Apply(soif, function, user-date)Applies the given function with the given argument (
user-data) to each AVPair in the given soif. For example:
void print_av(SOIF *s, SOIFAVPair *avp, void *unused)
{printf("%s = %s\n", avp->attribute, avp->value);}
/* print every attribute and value in the soif s */
SOIF_Apply(s, print_av, NULL);
SOIFAVPair structures.
The important fields in a SOIFAVPair structure are:
char *attribute; /* Attribute string; '\0' terminated */
char *value; /* primary value; may be '\0' terminated */
size_t vsize; /* # of bytes (8 bits) for primary value */
char **values; /* Multiple values for multivalued attributes */
size_t *vsizes; /* the sizes for the values */
int nvalues; /* number of values associated with attribute */
int last_slot; /* last valid slot - array may contain holes */
NSAPI_PUBLIC SOIFAVPair * SOIFAVPair_Create(char *a, char *v, int vsz);Creates an
AVPair structure with the given attribute a and value v. The value v is a buffer of vsz bytes.
NSAPI_PUBLIC void SOIFAVPair_Free(SOIFAVPair *avp);Frees the memory used by the given
SOIFAVPair structure.
NSAPI_PUBLIC int SOIF_Replace(SOIF *s, char *att, char *val, int valsz);Replaces the value of an existing attribute
att with a new value val of size valsz in the SOIF s.
#define SOIF_InsertStr(soif, attribute, value)Inserts the given attribute with the given value into the soif.
#define SOIF_ReplaceStr(soif, attribute, value)Replaces the existing value of the given attribute in the soif with the given value.
NSAPI_PUBLIC int SOIF_Rename(SOIF *s, char *old_attr, char *new_attr);Renames the given attribute to the given new name.
NSAPI_PUBLIC int SOIF_AttributeCompare(const char *a1, const char *a2);Compares two attribute names. Returns 0 (zero) if they are equal, or non-zero if they are different. Case (upper and lower) and trailing -s are ignored when comparing attribute names. The following table illustrates the results of comparing some attribute names.
| AttributeA | AttributeB | Does SOIF_AttributeCompare() consider them to be the same? |
|---|---|---|
| title | Title | yes |
| title | TITLE | yes |
| title | title- | yes |
| title | title-page | no |
| titl | title | no |
| author | title | no |
Multi-valued Attribute Routines
A SOIF attribute can have multiple values. SOIF supports the convention of using -### to indicate a multivalued attribute. For example, Title-1, Title-2, Title-3, etc. The -### do not need to be sequential positive integers (e.g., 1, 2, 3, ...); the sequence can have "holes" in it.
Compass Server currently supports only "classification" as a multi-valued attribute. In SOIF representation, it is represented using classification-1, classification-2, and so on. For example:
classification-1{5}: robot
classification-2{8}: netscape
classification-3{10}: web crawler
NSAPI_PUBLIC int SOIF_AttributeCompareMV(const char *a1, const char *a2);Compares two attribute names. Returns 0 (zero) if they are equal, or non-zero if they are different. If neither of the attributes is multi-valued then use above routine SOIF_AttributeCompare(). If one or both of the attributes are multi-value, use the base name of the multi-valued attribute for comparison. The "base name" of a multi-valued attribute is the name portion before "-". For example, the base name of "classification-3" is "classification".
NSAPI_PUBLIC int SOIF_MVAttributeParse(char *a)Returns the multivalued number of the given attribute, and strips the attribute string of its -### indicator; otherwise, returns zero in the case of a normal attribute name. For example, "
classification-3" returns the number 3
NSAPI_PUBLIC char *SOIF_IsMVAttribute(const char *a);Returns NULL if the given attribute is not a multivalued attribute; otherwise returns a pointer to where the multivalued number occurs in the attribute string. For example, for the multi-valued attribute "classification-3", it will return the pointer to "3".
NSAPI_PUBLIC int SOIF_InsertMV(SOIF *s, char *a, int slot, char *v,Inserts a new value
int vsz, int useval)
v at index slot for the given attribute a (in non-multivalue form). If set, the useval flag tells the function to use the given value buffer rather than creating its own.
For example:
SOIF_InsertMV(s, "classification", 3, "web crawler", strlen("web crawler"); classification-3{10}: web crawlerNSAPI_PUBLIC int SOIF_ReplaceMV(SOIF *s, char *a, int slot, char *v,
int vsz, int useval);
NSAPI_PUBLIC int SOIF_DeleteMV(SOIF *s, char *a, int slot)Deletes the value at the index
slot in the attribute a. For example:
SOIF_DeleteMV(s, "classification", 3)deletes "classification-3".
NSAPI_PUBLIC const char *SOIF_FindvalMV(SOIF *s, const char *a,Finds the value at the index
int slot)
slot in the attribute a. For example:
SOIF_FindvalMV(s, "classification", 3)returns "
web crawler" (using the previous example).
NSAPI_PUBLIC void SOIF_SqueezeMV(SOIF *s)Forces a re-numbering to ensure that the multivalue indexes are sequentially increasing (e.g., 1, 2, 3,...). This function can be used to fill in any holes that might have occurred during
SOIF_InsertMV() invocations. For example: to insert values explicitly for the multivalue attribute author-*:
SOIF_InsertMV(s, "author", 1, "John", 4, 0);
SOIF_InsertMV(s, "author", 2, "Kevin", 5, 0);
SOIF_InsertMV(s, "author", 6, "Darren", 6, 0);
SOIF_InsertMV(s, "author", 9, "Tommy", 5, 0);
SOIF_FindvalMV(s, "author", 9); /* == "Tommy" */
SOIF_SqueezeMV(s);
SOIF_FindvalMV(s, "author", 9); /* == NULL */
SOIF_FindvalMV(s, "author", 4); /* == "Tommy" */
#define SOIFAVPair_IsMV(avp)Use this to determine if the AVPair has multiple values or not.
#define SOIFAVPair_NthValid(avp,n)Use this to determine if the Nth value is valid or not.
#define SOIFAVPair_NthValue(avp,n) ((avp)->values[n])Use this to access the Nth value. For example:
for (i = 0; i <= avp->last_slot; i++)
if (SOIFAVPair_NthValid(avp, i))
printf("%s = %s\n", avp->attribute,
SOIFAVPair_NthValue(avp, i));
#define SOIFAVPair_NthVsize(avp,n) ((avp)->vsizes[n])Use this to get the size of the Nth value.
NSAPI_PUBLIC boolean_t SOIF_Contains(SOIF *s, char *a, char *v,Indicates if the given attribute contains the given value. It returns
int vsz);
B_TRUE if the value matches one or more of the values of the attribute a in the given SOIF s.
SOIFStream contains one or more SOIF objects.
The general approach is that you use SOIF streams to get streams of SOIF objects. Given a SOIF stream, you can parse it to get the SOIF objects from it. Use the parse() routine to get the next SOIF object in a SOIF stream. You can use SOIFStream_IsEOS() to check whether the last object has been parsed.
You can use filtering functions for a SOIF stream to specify that certain SOIF attributes are allowed or denied. If an attribute is allowed, you can parse and print that attribute for SOIF objects in the stream. If it is denied, you cannot parse or print that attribute of SOIF objects in the stream.
SOIF streams can be disk-based or memory based.
When you create a SOIFStream, you need to specify if you will be printing or parsing the SOIF stream and if you will be using a memory- or disk-based stream. There are several SOIF stream creation functions, and which one you need to use depends on what you will be doing with the SOIF stream.
For creating a SOIF streams that you will be printing SOIFS into, the functions are:
SOIF_PrintInitFile() - creates a disk-based stream ready for printing.
SOIF_PrintInitStr() - creates a memory-based stream ready for printing.
SOIFStream objects have a caller-data field, which you can use as you like:
void *caller_data; /* hook to be used by caller */Use
SOIFStream_Parse() to get the SOIF objects from the SOIF stream, and use SOIFStream_Print() to write SOIF objects to the SOIF stream.
When you've finished with the stream, close it using SOIFStream_Finish(). Use SOIFStream_SetFinishFn() to trigger the given finish_fn function.
The following example code takes a SOIF stream in stdin and prints each SOIF in the stream to stdout. Notice that this code uses SOIF_ParseInitFile() to create the SOIFStream to parse the input file, and uses SOIF_PrintInitFile() to create the stream to print the SOIFs to stdout.
SOIFStream *soifin = SOIF_ParseInitFile(stdin);
SOIFStream *soifout = SOIF_PrintInitFile(stdout);
SOIF *s;
while (!SOIFStream_IsEOS(soifin)) {
if ((s = SOIFStream_Parse(soifin)) {
SOIFStream_print(soifout, s);
SOIF_Free(s);
}
}
NSAPI_PUBLIC SOIFStream *SOIF_PrintInitFile(FILE *file)Creates a disk-based stream ready for printing.
NSAPI_PUBLIC SOIFStream *SOIF_PrintInitStr(SOIFBuffer *memory)Creates a memory-based stream ready for printing.
NSAPI_PUBLIC SOIFStream *SOIF_PrintInitFn(int (*write_fn)(void *data,Creates a generic application-defined stream ready for printing. The given
char *buf, int bufsz), void *data)
write_fn is used to print the stream.
This function allows you to hook up your own routine for printing.
NSAPI_PUBLIC SOIFStream *SOIF_ParseInitFile(FILE *fp)Creates a disk-based stream ready for parsing. The file must contain SOIF-formatted data. The function reads SOIF data from the file
fp.
NSAPI_PUBLIC SOIFStream *SOIF_ParseInitStr(char *buf, int bufsz)Creates a memory-based stream ready for parsing. The character buffer must contain SOIF-formatted data.
NSAPI_PUBLIC int SOIFStream_Finish(SOIFStream *)Call this to close the stream when you have finished with it.
NSAPI_PUBLIC int SOIFStream_SetFinishFn(SOIFStream *,This allows you to hook up a function for cleaning up after the SOIF stream finishes its business. The
int (*finish_fn)(SOIFStream *))
finish_fn will be called when SOIFStream_Finish() has finished executing".
#define SOIFStream_Print(ss, s)Prints another SOIF object to the SOIF stream
ss. Returns 0 on success, or non-zero on error.
#define SOIFStream_Parse(ss)Parses and returns the next SOIF object in the SOIF stream.
#define SOIFStream_IsEOS(s)Returns 1 if the soif is the last one in its stream.
#define SOIFStream_IsPrinting(s)Returns 1 (true) if the soif has been set up in a stream by
SOIF_PrintInitFile() or SOIF_PrintInitStr().
#define SOIFStream_IsParsing(s)Returns 1 (true) if the soif has been setup in a stream by
SOIF_ParseInitFile() or SOIF_ParseInitStr().
SOIFStream_IsAllowed() and SOIFStream_SetAllowed() allow attributes, while SOIFStream_IsDenied() and SOIFStream_SetDenied() deny attributes. You can allow or deny an attribute, but not both.
NSAPI_PUBLIC boolean_t SOIFStream_IsAllowed(SOIFStream *ss,Indicates that the given attribute is allowed (that is, it can be printed or parsed).
char *attribute);
NSAPI_PUBLIC int SOIFStream_SetAllowed(SOIFStream *ss,Sets all the attributes in the
char *allowed_attrs[])
allowed_attrs array to allowed.
NSAPI_PUBLIC int SOIFStream_SetDenied(SOIFStream *ss,Sets all the attributes in the
char *denied_attrs[]);
allowed_attrs array to be denied (that is, they cannot be parsed or printed).
NSAPI_PUBLIC char **SOIFStream_GetAllowed(SOIFStream *ss)Returns an array of all the attributes that are allowed.
NSAPI_PUBLIC char **SOIFStream_GetDenied(SOIFStream *ss);Returns an array of all the attributes that are denied.
SOIFBuffer structure, that is created with the SOIFBuffer_Create() function and freed with the SOIFBuffer-Free() function. The SOIFBuffer structure provides the append(), increase(), and reset() functions for manipulating the data in the buffer.
NSAPI_PUBLIC SOIFBuffer *SOIFBuffer_Create(int default_sz);The SOIFBuffer is used in
SOIF_PrintInitStr(SOIFBuffer *memory). Before you can print SOIF to memory, you need to create a buffer for output.
NSAPI_PUBLIC void SOIFBuffer_Free(SOIFBuffer *sb);This is to release the memory buffer created by
SOIFBuffer_Create().
void (*append)(SOIFBuffer *sb, char *data, int n)copies
n bytes of data into the buffer.
void (*increase)(SOIFBuffer *sb, int add_n)increase the size of the data buffer by
addn bytes.
void (*reset)(SOIFBuffer *sb)Resets the size of the data buffer and invalidates all currently valid data
Last Updated: 02/07/98 20:49:17
Any sample code included above is provided for your use on an "AS IS" basis, under the Netscape License Agreement - Terms of Use