|Sun ONE Web Proxy Server 3.6 SP2 Administrator's Guide - UNIX Version|
Chapter 16 Configuring the Proxy Manually
This chapter describes the configuration files that iPlanet Web Proxy Server uses. These are the files that you're changing when you use iPlanet Web Proxy Server Manager online forms. You can also configure iPlanet Web Proxy Server manually by editing the files directly.
You might need to configure iPlanet Web Proxy Server manually for various reasons. If you accidentally lock your hosts out of the administrative forms or forget your administrative password, you'll have to change information manually in the proxy's configuration files. Perhaps more importantly, you will probably need to develop an understanding of what your configuration files do for you, so that you can write scripts to automate configuration functions that you might want in addition to those available in the online forms. This is especially useful if, for example, you are using many proxy servers or your URL lists require frequent or high-volume updates.
Before you can edit any of the configuration files, you must have permission to read and write to the files. If you are running a Unix proxy server, you might need to log in as root or use the proxy's user account.
The files that you use to configure the proxy are in the server root/proxy-id/config directory. Here's a brief description of each file:
- magnus.conf is the server's main technical configuration file. It controls aspects of the server operation not related to specific resources or documents, such as host name and port.
- obj.conf is the server's object configuration file. It controls access to the proxy server and determines how documents are proxied and cached.
- mime.types is the file the server uses to convert file name extensions such as .GIF into a MIME type like image/gif.
- admpw is the administrative password file. Its format is user:password. The password is DES-encrypted, just like /etc/passwd. This file, as opposed to the other configuration files, is located in the server root/admin-serv directory.
- socks5.conf is a file that contains the SOCKS server configuration. The SOCKS daemon is a generic firewall daemon that controls point-to-point access through the firewall. If you use SOCKS, the SOCKS server configuration file has to be stored in /etc/socks5.conf. This file is described on page 263.
- bu.conf is an optional file that contains batch update directives. You can use these to update many documents at once. You can time batch updates; for example, you can have them occur during off-peak hours to minimize the effect on the efficiency of the server.
- icp.conf is the Internet Cache Protocol (ICP) configuration file. It identifies the information about the parent and sibling servers in a proxy array that uses ICP.
- parray.pat is the Proxy Array Table file. The PAT file is an ASCII file used in proxy to proxy routing. It contains information about a proxy array; including the members' machine names, IP addresses, ports, load factors, cache sizes, etc. For more information on the syntax of the parray.pat file, see The parray.pat File.
- parent.pat is the Proxy Array Table file that contains information about an upstream proxy array. For more information on the syntax of the parent.pat file, see The parent.pat File.
The magnus.conf File
The technical configuration file, magnus.conf, controls all global server operations. All of the items in the magnus.conf file apply to the entire proxy server, as opposed to affecting only one URL or set of URLs. The obj.conf file handles URLs (also called resources).
Every command line in the file has this format:
Directive identifies an aspect of server operation. This string is not case-sensitive.
Value is a number or label you give the directive. Its format depends on the directive. Unlike the Directive string, this string is usually case-sensitive.
Directive lines should not contain white spaces at the beginning of the line or more than one space between the directive and value. Comment lines begin with a # character with no leading white space. If you operate on the configuration files with the Server Manager, when it writes the files out again it does not write comment lines.
The directives in magnus.conf are explained in detail in Appendix C "Proxy Configuration Files."
The obj.conf File
The iPlanet Web Proxy Server object configuration file, obj.conf, uses objects to control how the server performs access control, routes URLs, and initializes server subsystems.
Configuration objects (also called resources) are settings that tell the proxy how to treat URLs. URLs matching a specified wildcard pattern belong to the same configuration object (or resource). This object grouping can then be used to control, in fine detail, the behavior of the proxy server.
Using this object-grouping scheme, you can specify single resources with their complete URL, whole "directories" with the path followed by /.*, and various other groups such as .*\.html. You can then configure the settings you want to use for that object (for example, caching or denying access based on the server's host name or a string in a URL).
The Structure of obj.conf
The obj.conf file must have specific objects in it (the objects are described on page 259). You can add other objects to this file. To specify an object, use this format:
Although <Client> lines are not required, you can have as many as needed.
If you want to control access at the URL level, use regular expression patterns to control which URLs are grouped in the object. You can then specify one or more directives to control what the proxy server does when it encounters any URL matching the regular expression pattern specified with ppath.
You can also set options for specific client hosts. This is a powerful feature. Unlike other proxy servers that simply control whether a host can or cannot access a URL, you can make the proxy act differently depending on which user or host is requesting the URL.
Each directive line (regardless of where it appears in obj.conf) has this format:
Directive fn=function [parameter1=value1]...[parameterN=valueN]
Directive identifies an aspect of server operation. This string is not case-sensitive and must appear at the beginning of a line.
Function is a function and parameters given to the directive. Its format depends on the directive.
Directive lines cannot contain spaces at the beginning of the line or extra spaces between the directive and value. You shouldn't use trailing spaces after the value because they might confuse the server. Long lines can be continued by starting the next line with white space. White space is any keystroke that leaves space on the screen, such as space bar, tab, carriage return, line feed, or vertical tab. Comment lines begin with a # character with no leading white space. If you operate on the configuration files with the Server Manager, when it writes the files out again it does not write comment lines.
If you are using the Administration forms, you shouldn't use continuation lines in the obj.conf file. Instead, put each directive entirely on a single line. If you are absolutely sure you will never use the Administration forms to edit the obj.conf file, you can use the \ character.
A Sample Object
The following sample object applies to all HTTP URLs (the pattern is http://.*). When the proxy receives a request for an HTTP document, it scans the URL for the string play (as specified in PathCheck); if it finds that string in the URL, it doesn't retrieve the document from the remote server, and it denies service to the client.
This object also caches all HTTP documents and refreshes the documents if they are older than four hours or if they need refreshing as determined by the date they were last modified. The Service directive tells the proxy to retrieve the HTTP documents by default. The following code is an example of an obj.conf file on a Unix system:
# Netscape Communications Corporation - obj.conf
# You can edit this file, but comments and formatting changes
# might be lost when the admin server makes changes.
Init fn="flex-init" format.access="%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%]
\"%Req->reqpb.proxy-request%\" %Req->srvhdrs.status% %Req->vars.p2c-cl%"
Init fn="load-types" mime-types="mime.types"
Init fn="init-proxy" timeout2="15" timeout="1200"
Init fn="init-cache" dir="/usr/ns-proxy/cache" status="on" ndirs="1"
Init fn="init-partition" min-avail="5" max-size="10" dir="/usr/ns-proxy/cache" status="on" name="NoName"
Init fn="init-urldb" dir="/usr/ns-proxy/cache/urldb" status="on"
Init conf-file="bu.conf" fn="init-batch-update" dir="/tmp" status="off"
NameTrans from="file:" fn="map" to="ftp:"
NameTrans from="/ns-icons" fn="pfx2dir" dir="/usr/ns-proxy/ns-icons" name="file"
PathCheck fn="check-acl" acl="proxy-TEST_formgen-READ-ACL_allow-1970"
PathCheck fn="check-acl" acl="proxy-TEST_formgen-WRITE-ACL_deny-1970"
AddLog fn="flex-log" name="access"
PathCheck index-names="index.html" fn="find-index"
ObjectType fn="force-type" type="text/plain"
ObjectType fn="cache-setting" max-uncheck="21600"
ObjectType lm-factor="0.100" fn="cache-setting" max-uncheck="3600"
ObjectType fn="cache-setting" max-uncheck="14400"
Service fn="connect" method="CONNECT"
Service fn="connect" method="CONNECT"
Required Objects for obj.conf
Certain objects must be in the obj.conf file to make the Administration forms work for your proxy. If you are familiar with Netscape server software (a regular HTTP server), you might notice that these functions control local file access and CGI execution.
On a proxy server the local access interface is a simplified version, and it cannot be used for any purpose other than the online forms. Special care is taken inside the proxy software to guarantee that it cannot be used, accidentally or otherwise, as a normal HTTP server. If you don't use the online forms, these objects don't necessarily have to be in obj.conf.
The Default Object
The default object contains the required directives. Named objects are objects identified by <Object name=...> in the object configuration file. To control the behavior of the entire server, you would modify the setting for the default object. This object must contain all of the name-translation directives for the server, and it should contain any global configuration changes. Here is an example of a default object for a proxy server running on Windows NT:
NameTrans fn=map from=file: to=ftp:
NameTrans fn=pfx2dir from=ns-icons dir="" name=file
- The first NameTrans directive takes care of URLs that use "file:" by changing them to "ftp:" URLs. If you have any mappings to mirror sites, put them after this mapping.
- The next NameTrans directive maps the ns-icons URL into its directory. These are the only legal uses for the pfx2dir function, which doesn't belong to the actual proxy configuration (you'll get errors if you try to use it anywhere else).
- The deny-service function ensures that by default access isn't granted. (Access isn't granted by default even if you forget this function, but the error message is less descriptive and it is classified as a misconfiguration.)
- The proxy-log function takes care of proxy access logging, whether it is in flexible, common, extended, or extended-2 log format. This function can have the additional iponly=1 parameter, which inhibits reverse DNS lookups and logs only the IP address of the requesting client.
- The urldb-record function logs URLs and has to be called for each object for which you want URL recording.
How the Proxy Server Handles Objects
Netscape servers (the HTTP server and the proxy) respond to an information request by following certain steps. Each step in the process is done once for all objects, then another step is done for all objects, and so on. The process steps that the server performs are:
- Authorization translation. Translate any authorization information given by the client into a user and group. If necessary, decode the message to get the actual request. Also, proxy authorization is available.
- Name translation. Before anything else is done, a URL can be translated into a file-system-dependent name (an administration URL), a redirection URL, or a mirror site URL, or it might be kept intact and retrieved as is (the normal case for proxy).
- Path checks. Perform various tests on the resulting path, largely used to make sure that it's safe for the given client to retrieve the document (only for local access).
- Object type determination. Determine the MIME type information for the given document. MIME types can be registered document types such as text/html and image/gif, or they can be internal document identification types. Internal types always begin with magnus-internal/ and are used to select a server function to use to decode the document (only used for local access; the proxy system calls these routines automatically when necessary).
- Service selection. Select the internal server function that should be used to send the result back to the client. This function can be the normal proxy service routine, or local file blast.
- Logging selection. Select a function to record information about transactions as they finish.
These steps map directly to several configuration directives allowed for each object. Another configuration directive, send-error, controls how the server responds to the client when it encounters an error.
The directives in obj.conf are explained in detail in Appendix C "Proxy Configuration Files."
The mime.types File
The mime.types file tells the server how to convert files with certain extensions (such as .gif) into a MIME type (such as image/gif). MIME files are compact files and transfer quickly. Also, MIME is needed by browsers (like Netscape Navigator); without MIME they can't tell the difference between an HTML page and a graphics file.
The mime.types file contains the global file extensions for all proxy servers. The first line in the file identifies the file format and must read:
#--Netscape Communications Corporation MIME Information
The following code is a sample mime.types file:
#--Netscape Communications Corporation MIME Information
# Don't delete the above line. It identifies this file's type.
# This is a simple MIME types file for Netscape Proxy Server. Most
# of the MIME types are already compiled in the proxy. Types that
# are part of the Administration forms (HTML and GIF) must appear
# here, or they won't be known to the part of the server that
# manages the Administration interface calls.
# Icons (internal-gopher-...) are references to Netscape's
# internal icons. If a client doesn't support these icons, the
# proxy will provide them.
type=application/x-texinfo exts=texinfo,texi icon=internal-gopher-text
type=image/gif exts=gif icon=internal-gopher-image
type=image/jpeg exts=jpeg,jpg,jpe icon=internal-gopher-image
type=image/x-xwindowdump exts=xwd icon=internal-gopher-image
type=text/html exts=htm,html,shtml icon=internal-gopher-text
type=text/plain exts=txt icon=internal-gopher-text
type=text/richtext exts=rtx icon=internal-gopher-text
type=text/tab-separated-values exts=tsv icon=internal-gopher-text
type=text/x-setext exts=etx icon=internal-gopher-text
type=application/x-tar enc=x-gzip exts=tgz
Other non comment lines have this format:
type=type/subtype exts=file extensions icon=icon
where each parameter is as follows.
- type/subtype is the MIME type and subtype.
- exts are the file extensions associated with this type. When the proxy transfers a file with one of these extensions, it uses the MIME type you specify in type.
- icon is the name of the icon the browser displays; the icons are shown in Figure 16-1. Netscape Navigator keeps these images internally. If you use a browser that doesn't have these icons, Netscape Proxy Server delivers them.
Figure 16-1    Internal icons for MIME types
If you set the .pac MIME type to anything other than
application/x-ns-proxy-autoconfig, the proxy autoconguration feature will not work.
The admpw File
The admpw file contains the administration password. If you forget your password, there is no way to find out what it was. You must encrypt a new one and replace the old version with it. The file has the format user:password.
If you forget your administration password, you can edit the admpw file and delete the password section (everything after the semicolon). When you go to the administration server, you don't need to enter a new password, but you should immediately go to Access Control in the iPlanet Web Proxy Server Manager and set a new one.
Because you can replace the Administration password, it is very important to keep secure the proxy's account and to ensure that only that proxy account and, for Unix proxy servers, the root account, have full (read/write) access to the server root directory. This way, only someone running as root or with the proxy's user account can enter the server root/proxy-id/config directory and edit the file.
The socks5.conf File
The SOCKS daemon is a generic firewall daemon that controls point-to-point access through the firewall. By default, the SOCKS daemon features are disabled. The iPlanet Web Proxy Server supports SOCKS versions 4 and 5.
The proxy uses the file socks5.conf to control access to the SOCKS proxy server and its services. Each line defines what the proxy does when it gets a request that matches the line.
When the SOCKS daemon receives a request, it checks the request against the lines in the socks5.conf file. When it finds a line that matches the request, the request is permitted or denied based on the first word in the line (permit or deny). Once it finds a matching line, the daemon ignores the remaining lines in the file. If there are no matching lines, the request is denied. You can also specify actions to take if the client's identd or user ID is incorrect by using #NO_IDENTD: or #BAD_ID as the first word of the line. Each line can be up to 1023 characters long.
Although the SOCKS daemon doesn't know if a host is internal to its network, it does know which host is the requestor and which is the destination (it uses this for access control). This means SOCKS daemon provides access from external hosts into your internal networks in addition to the normal internal-to-external proxy functionality.
Use caution with the external-to-internal functionality. If you don't need external to internal access, you should specifically deny such connections. For example, if 198.95 is your internal network, use the following as the first lines in socks5.conf to protect your internal hosts from external access attempts:
auth 198.95. - -
ban - -
These lines will allow anyone on the 198.95 intranet to authenticate using any type of authentication, and will ban all other hosts from the server.
For information on the syntax of socks5.conf, see "The socks5.conf File".
The bu.conf File
The optional bu.conf file contains batch update directives. You can use these to update many documents at once. You can time these updates to occur during off-peak hours to minimize the effect on the efficiency of the server. The format of this file is described in this section. For more information on batch updating and starting the batch update function, see "init-batch-update (starting batch updates)".
All of the batch update directives must be in Object boundaries.
The pairs of Object boundaries indicate the individual configurations in the bu.conf file. If you give a unique name to each occurrence, you can specify these boundaries any number of times.
Where you see italicized text in the directive syntax examples, substitute your own information in place of the italicized text.
The directives in bu.conf are explained in detail in Appendix C "Proxy Configuration Files."
Examples of bu.conf
Here are some examples of code in a bu.conf file for a proxy server running on Unix:
This example code updates the entire cache every evening:
<Object name=cache update>
Days Sun Mon Tue Wed Thu Fri Sat
Time 20:00 - 3:00
This example code tells the proxy to mirror a company internal site to a local proxy:
<Object name= HumanResourcesMirror>
Days Mon Tue Wed Thu Fri
Time 4:00 - 6:00
The icp.conf File
This file is used to configure the Internet Cache Protocol (ICP) feature of your server. There are three functions in the icp.conf file, and each can be called as many times as necessary. Each function should be on a separate line. The three functions are add_parent, add_sibling, and server.
For more information on this file and the functions within it, see "The icp.conf File".
The parray.pat File
The parray.pat (PAT) file describes each member in the proxy array of which the proxy you are administering is a member. The PAT file is an ASCII file used in proxy to proxy routing. It contains proxy array members' machine names, IP addresses, ports, load factors, cache sizes, etc.
Proxy Array Information/1.0
ConfigID: ID number
name IPaddress proxyport URLforPAT infostring state time status loadfactor cachesize
Proxy Array Information is version information.
ArrayEnabled specifies whether the proxy array is enabled or disabled. Possible values are:
ConfigID is the identification number for the current version of the PAT file. The proxy server uses this number to determine whether the PAT file has changed.
ArrayName is the name of the proxy array.
ListTTL specifies how often the proxy should check the PAT file to see if it has changed. This value is specified in minutes.
name is the name of a specific member of the proxy array.
IPaddress is the IP address of the member.
proxyport is the port at which the master proxy accepts HTTP requests.
URLforPAT is the URL of the PAT file that the member will poll the master proxy for.
infostring is version information.
statetime is the amount of time the member has been in its current state.
status specifies whether the member is enabled or disabled.
loadfactor is an integer that reflects the number of requests that should be routed through the member.
cachesize is the size of the member's cache.
Proxy Array Information/1.0
proxy1 126.96.36.199 8080 http://pat iPlanetWebProxy/3.6 0 on 100 512
proxy2 188.8.131.52 8080 http://pat iPlanetWebProxy/3.6 0 on 100 512
The parent.pat File
The parent.pat file is the Proxy Array Table file that contains information about an upstream proxy array. This file has the same syntax as the parray.pat file.