Chapter 2 Creating Custom Filters

This chapter describes how to create custom filters that can be used to intercept and possibly modify the content presented to or generated by another function.

Future Compatibility Issues

The NSAPI interface may change in a future version of Sun Java System Web Proxy Server. To keep your custom plugins upgradeable, do the following:

Make sure plugin users know how to edit the configuration files (such as magnus.conf and obj.conf) manually. The plugin installation software should not be used to edit these configuration files.

Keep the source code so you can recompile the plugin.

The NSAPI Filter Interface

Sun Java System Web Proxy Server 4 extends NSAPI by introducing a new filter interface that complements the existing Server Application Function (SAF) interface. Filters make it possible to intercept and possibly modify data sent to and from the server. The server communicates with a filter by calling the filter's filter methods. Each filter implements one or more filter methods. A filter method is a C function that performs a specific operation, such as processing data sent by the server.

Filter Methods

This section describes the filter methods that a filter can implement. To create a filter, a filter developer implements one or more of these methods. This section describes the following filter methods:

C Prototypes for Filter Methods

int insert(FilterLayer *layer, pblock *pb);
void remove(FilterLayer *layer);
int flush(FilterLayer *layer);
int read(FilterLayer *layer, void *buf, int amount, int timeout);
int write(FilterLayer *layer, const void *buf, int amount);
int writev(FilterLayer *layer, const struct iovec *iov, int iov_size);
int sendfile(FilterLayer *layer, sendfiledata *sfd);

The layer parameter is a pointer to a FilterLayer data structure, which contains variables related to a particular instance of a filter. Following is a list of the most important fields in the FilterLayer data structure:

context->sn: Contains information relating to a single TCP/IP session (the same sn pointer that’s passed to SAFs).

context->rq: Contains information relating to the current request (the same rq pointer that’s passed to SAFs).

context->data: Pointer to filter-specific data.

lower: A platform-independent socket descriptor used to communicate with the next filter in the stack.

The meaning of the context->data field is defined by the filter developer. Filters that must maintain state information across filter method calls can use context->data to store that information.

insert

The insert filter method is called when an SAF such as insert-filter calls the filter_insert function to request that a specific filter be inserted into the filter stack. Each filter must implement the insert filter method.

When insert is called, the filter can determine whether it should be inserted into the filter stack. For example, the filter could inspect the Content-Type header in the rq->srvhdrs pblock to determine whether it is interested in the type of data that will be transmitted. If the filter should not be inserted, the insert filter method should indicate this by returning REQ_NOACTION.

If the filter should be inserted, the insert filter method provides an opportunity to initialize this particular instance of the filter. For example, the insert method could allocate a buffer with MALLOC and store a pointer to that buffer in layer->context->data.

The filter is not part of the filter stack until after insert returns. As a result, the insert method should not attempt to read from, write to, or otherwise interact with the filter stack.

See Also

remove

The remove filter method is called when a filter stack is destroyed (that is, when the corresponding socket descriptor is closed), when the server finishes processing the request the filter was associated with, or when an SAF such as remove-filter calls the filter_remove function. The remove filter method is optional.

The remove method can be used to clean up any data the filter allocated in insert and to pass any buffered data to the next filter by calling net_write(layer->lower, ...).

See Also

flush

The flush filter method is called when a filter or SAF calls the net_flush function. The flush method should pass any buffered data to the next filter by calling net_write(layer->lower, ...). The flush method is optional, but it should be implemented by any filter that buffers outgoing data.

See Also

read

The read filter method is called when a filter or SAF calls the net_read function. Filters that are interested in incoming data (data sent from a client to the server) implement the read filter method.

Typically, the read method will attempt to obtain data from the next filter by calling net_read(layer->lower, ...). The read method may then modify the received data before returning it to its caller.

See Also

write

The write filter method is called when a filter or SAF calls the net_write function. Filters that are interested in outgoing data (data sent from the server to a client) implement the write filter method.

Typically, the write method will pass data to the next filter by calling net_write(layer->lower, ...). The write method may modify the data before calling net_write. For example, the http-compression filter compresses data before passing it on to the next filter.

If a filter implements the write filter method but does not pass the data to the next layer before returning to its caller (that is, if the filter buffers outgoing data), the filter should also implement the flush method.

See Also

writev

The writev filter method performs the same function as the write filter method, but the format of its parameters is different. It is not necessary to implement the writev filter method; if a filter implements the write filter method but not the writev filter method, the server uses the write method instead of the writev method. A filter should not implement the writev method unless it also implements the write method.

Under some circumstances, the server may run slightly faster when filters that implement the write filter method also implement the writev filter method.

See Also

sendfile

The sendfile filter method performs a function similar to the writev filter method, but it sends a file directly instead of first copying the contents of the file into a buffer. It is not necessary to implement the sendfile filter method; if a filter implements the write filter method but not the sendfile filter method, the server will use the write method instead of the sendfile method. A filter should not implement the sendfile method unless it also implements the write method.

Under some circumstances, the server may run slightly faster when filters that implement the write filter method also implement the sendfile filter method.

See Also

Position of Filters in the Filter Stack

All data sent to the server (such as the result of an HTML form) or sent from the server (such as the output of a JSP page) is passed through a set of filters known as a filter stack. The server creates a separate filter stack for each connection. While processing a request, individual filters can be inserted into and removed from the stack.

Different types of filters occupy different positions within a filter stack. Filters that deal with application-level content (such filters that translates a page from XHTML to HTML) occupy a higher position than filters that deal with protocol-level issues (such as filters that format HTTP responses). When two or more filters are defined to occupy the same position in the filter stack, filters that were inserted later will appear higher than filters that were inserted earlier.

Filters positioned higher in the filter stack are given an earlier opportunity to process outgoing data, while filters positioned lower in the stack are given an earlier opportunity to process incoming data. For example, in the following figure, the xml-to-xhtml filter is given an earlier opportunity to process outgoing data than the xhtml-to-html filter.

When you create a filter with the filter_create function, you specify what position your filter should occupy in the stack. You can also use the init-filter-order Init SAF to control the position of specific filters within filter stacks. For example, init-filter-order can be used to ensure that a filter that converts outgoing XML to XHTML is inserted above a filter that converts outgoing XHTML to HTML.

Filters that Alter Content-Length

Filters that can alter the length of an incoming request body or outgoing response body must take special steps to ensure interoperability with other filters and SAFs.

Filters that process incoming data are referred to as input filters. If an input filter can alter the length of the incoming request body (for example, if a filter decompresses incoming data) and there is a Content-Length header in the rq->headers pblock, the filter's insert filter method should remove the Content-Length header and replace it with a Transfer-encoding: identity header as follows:

pb_param *pp;

pp = pblock_remove("content-length", layer->context->rq->headers);
if (pp != NULL) {
param_free(pp);
pblock_nvinsert("transfer-encoding", "identity", layer->context->rq->headers);
}

Because some SAFs expect a Content-Length header when a request body is present, before calling the first Service SAF the server will insert all relevant filters, read the entire request body, and compute the length of the request body after it has been passed through all input filters. However, by default, the server will read at most 8192 bytes of request body data. If the request body exceeds 8192 bytes after being passed through the relevant input filters, the request will be cancelled. For more information, see the description of ChunkedRequestBufferSize in the "Syntax and Use of magnus.conf" chapter in the Sun Java System Web Proxy Server 4 Administrator's Configuration File Reference.

Filters that process outgoing data are referred to as output filters. If an output filter can alter the length of the outgoing response body (for example, if the filter compresses outgoing data), the filter's insert filter method should remove the Content-Length header from rq->srvhdrs as follows:

pb_param *pp;

pp = pblock_remove("content-length", layer->context->rq->srvhdrs);
if (pp != NULL)
param_free(pp);

Creating and Using Custom Filters

Custom filters are defined in shared libraries that are loaded and called by the server. The general steps for creating a custom filter are as follows:

Write the Source Code using the NSAPI functions.

Compile and Link the source code to create a shared library (.so, .sl, or .dll) file.

Load and Initialize the Filter by editing the magnus.conf file.

Instruct the Server to Insert the Filter by editing the obj.conf file to insert your custom filter(s) at the appropriate time.

Restart the Server.

Test the Filter by accessing your server from a browser with a URL that triggers your filter.

Write the Source Code

Write your custom filter methods using NSAPI functions. For a summary of the NSAPI functions specific to filter development, see Overview of NSAPI Functions for Filter Development. For a summary of general purpose NSAPI functions, see NSAPI Function Reference. Each filter method must be implemented as a separate function. See Filter Methods, for the filter method prototypes.

The filter must be created by a call to filter_create. Typically, each plugin defines an nsapi_module_init function that is used to call filter_create and perform any other initialization tasks. See nsapi_module_init and filter_create for more information.

Filter methods are invoked whenever the server or an SAF calls certain NSAPI functions such as net_write or filter_insert. As a result, filter methods can be invoked from any thread and should only block using NSAPI functions (for example, crit_enter and net_read). If a filter method blocks using other functions (for example, the Windows WaitForMultipleObjects and ReadFile functions), the server may hang. Also, shared objects that define filters should be loaded with the NativeThread="no" flag, as described in Load and Initialize the Filter..

If a filter method must block using a non-NSAPI function, KernelThreads 1 should be set in magnus.conf. For more information about KernelThreads, see the description in the chapter "Syntax and Use of magnus.conf" in the Sun Java System Web Proxy Server 4 Administrator’s Configuration File Reference.

Write thread-safe code

IO should only be performed using the NSAPI functions documented in File I/O and Network I/O.

Thread synchronization should only be performed using NSAPI functions documented in Threads.

Blocking may affect performance.

Carefully check and handle all errors.

Compile and Link

Filters are compiled and linked in the same way as SAFs. See Compile and Link, for more information.

Load and Initialize the Filter

For each shared library (plugin) containing custom SAFs to be loaded into the Sun Java System Web Proxy Server, add an Init directive that invokes the load-modules SAF to obj.conf. The syntax for a directive that loads a filter plugin is:

shlib is the local file system path to the shared library (plugin).

NativeThread indicates whether the plugin requires native threads. Filters should be written to run on any type of thread (see Write the Source Code).

When the server encounters such a directive, it calls the plugin's nsapi_module_init function to initialize the filter.

Instruct the Server to Insert the Filter

Add an Input or Output directive to obj.conf to instruct the server to insert your filter into the filter stack. The format of the directive is as follows:

Directive fn=insert-filter filter="filter-name" [name1="value1"]...
[nameN="valueN"]

Directive is Input or Output.

filter-name is the name of the filter, as passed to filter_create, to insert.

nameN="valueN" are the names and values of parameters that are passed to the filter's insert filter method.

Filters that process incoming data should be inserted using an Input directive. Filters that process outgoing data should be inserted using an Output directive.

To ensure that your filter is inserted whenever a client sends a request, add the Input or Output directive to the default object. For example, the following portion of obj.conf instructs the server to insert a filter named example-replace and pass it two parameters, from and to:

<Object name="default">
Output fn=insert-filter
       filter="example-replace"
       from="Old String"
       to="New String"
...
</Object>

Restart the Server

For the server to load your plugin, you must restart the server. A restart is required for all plugins that implement SAFs and/or filters.

Test the Filter

Test your SAF by accessing your server from a browser. You should disable caching in your browser so that the server is sure to be accessed. In Netscape Navigator, you can hold the shift key while clicking the Reload button to ensure that the cache is not used. (Note that the shift-reload trick does not always force the client to fetch images from source if the images are already in the cache.) Examine the access and error logs to help with debugging.

Overview of NSAPI Functions for Filter Development

NSAPI provides a set of C functions that are used to implement SAFs and filters. This section lists the functions that are specific to the development of filters. All of the public routines are described in detail in Chapter 4, "NSAPI Function Reference".

filter_create creates a new filter

filter_insert inserts the specified filter into a filter stack

filter_remove removes the specified filter from a filter stack

filter_name returns the name of the specified filter

filter_find finds an existing filter given a filter name

filter_layer returns the layer in a filter stack that corresponds to the specified filter

Previous Contents Index Next
Sun Java System Web Proxy Server 4.0.1 Developer's Guide