Chapter 1 Overview

• "Netra Proxy Cache Server Features"

• "Netra Proxy Cache Array Features"

• "Monitoring and Managing"

The Netra Proxy Cache Server is a full-featured proxy cache server that is available with the bonus of array software. The array software enables multiple Netra Proxy Cache Servers to provide a single proxy cache service. We refer to such a group of Netra Proxy Cache Servers as a Netra Proxy Cache Array.

The array software extends the reliability and availability of the proxy cache service. It also makes it easy for you to add and remove servers to and from a Netra Proxy Cache Array.

A significant benefit of the array software is that--in addition to the reliability, availability, and scalability it gives you--members of your Netra Proxy Cache array become "instant proxy cache siblings": Under array control, the Netra Proxy Cache Servers share cached data, effectively extending the cache over the entire set of machines. This feature is discussed in "Hierarchies".

You have the option of configuring a Netra Proxy Cache Server as a single machine, rather than as part of an array.

Netra Proxy Cache Server Features

The Netra Proxy Cache hardware and software implement a proxy cache server with a set of comprehensive features. The Netra Proxy Cache product is shipped with array software, which, where there are multiple Netra Proxy Cache Servers, extends the reliability and availability of the proxy cache service.

Features

Netra Proxy Cache Servers support the following features:

• High-performance CPU with memory and disk amounts chosen for optimum performance in proxying and caching. (Details of the hardware configuration are spelled out in the hardware documentation that accompanies the product.)

• Compatible with the Squid, Harvest, and CERN proxy standards.

• Supports the Inter Cache Protocol (ICP).

• Caches HTTP 1.0, FTP, and Gopher objects. This list includes, among other types, GIF, JPEG, and.exe.

• Supports Secure Sockets Layer (SSL) tunneling.

• Supports persistent HTTP connections, commonly referred to as "keep-alives."

• The cache persists across reboots.

• Configurable cache-object expiration times. The Netra Proxy Cache software ages and deletes a cache object based on attributes specified in its uniform resource locator (URL). The product offers a flexible scheme for cache-object expiration.

• Similar to the preceding point, the software offers a flexible scheme for setting a cache object to non-cacheable, again, based on its URL.

• Supports dynamic parent failover: If Netra Proxy Cache Server has multiple parents and is connected to a parent that fails, the server fails over to the next available parent. Furthermore, the Netra Proxy Cache Server detects when the original parent comes back online.

• Supports conditional retrievals; for example, can retrieve an object if it has been modified in the last day. You can modify the time threshold to suit your needs.

• Caching software imposes no limit on the amount of data cached.

• Enables you to build hierarchies of (or collections of peer) proxy servers. See "Hierarchies".

• Offers a number of auditing features, including hit statistics, detailed user access logs, bandwidth usage statistics, and a number of other proxy- and cache-related statistics.

• Ships with an SNMP MIB and agent, so that you can manage a Netra Proxy Cache Server from an SNMP-conformant management platform, such as Solstice(TM) Domain Manager.

• Offers a variety of filtering features, including blocking and redirecting of HTTP requests based on URL, hostname, or user.

• Shipped with a set of web-based tools for product configuration and monitoring.

Hierarchies

An important feature of Netra Proxy Cache Server is the ability it gives you to create hierarchies of proxy cache servers or, a related feature, create collections of sibling servers. You can create hierarchies simply by pointing proxy cache servers to succeeding proxy cache servers as you proceed toward a firewall. Alternatively, you can take advantage of Netra Proxy Cache software's support for the Inter Cache Protocol (ICP) to build sibling and parent relationships among proxy cache servers.

When you configure a set of Netra Proxy Cache Servers as an array, automatically, those servers become ICP siblings, so that the cache is extended over all of the machines in the Netra Proxy Cache Array.

Figure 1-1 illustrates a simple hierarchy of proxy cache servers.

Figure 1-1 Simple Hierarchy

Referring to Figure 1-1, assume the client browser requests a web object that originated somewhere in the Internet and is, at the moment, not in Netra Proxy Cache Server A's cache. The following sequence ensues:

1. Machine A checks with its parent, machine B.

2. Likewise, B does not have the object in its cache and checks its parent, machine C. If C does not have the object, it goes out through the firewall to the web server to obtain it.

3. Machine C returns the object--obtained from a remote web server or its local cache--to machine B.

4. Machine B returns the object to machine A.

5. Machine A then returns the object to the requesting client.

If the object is cacheable, each proxy stores a copy upon receipt. Note that communication between parent proxies is over TCP connections.

Netra Proxy Cache software also supports a variation of the preceding scenario. This variation is illustrated in Figure 1-2.

Figure 1-2 Multiple Parent Proxies

Referring to Figure 1-2, if a client requests an object of its proxy server, machine A, that is not in A's cache, machine A relays the request to its two parents, machines B and C. If one of the parents has the object, it returns the object to A. If neither has the object, machine A forwards the request to the parent that responds faster, assuming that machine to be less loaded and/or have a better network connection.

If you configure multiple parents, the Netra Proxy Cache software allows you to give greater weight to one or the other, or set up one as the default. When no parent (of multiple parents) has a requested object, the "child" proxy always forwards the request to the default parent.

In addition to supporting hierarchies of parent proxies, the Netra Proxy Cache Server supports sibling proxies. The sibling scenario is illustrated in Figure 1-3.

Figure 1-3 Sibling Proxies

Referring to Figure 1-3, assume a client browser requests an object that is, at the moment, not in Netra Proxy Cache Server A's cache. The following sequence ensues:

1. Machine A checks with its parent, machine B. Machine A has no awareness of machines C and D.

2. Likewise, B does not have the object in its cache. Using ICP over UDP, machine B checks its siblings, machines C and D. If either of those machines has the object, it returns the object to machine B, which returns it to machine A.

If none of B, C, and D have the object, the request is forwarded to B's parent, machine E.

It is important to note that queries among siblings are over the relatively lightweight ICP exchanges, while communication among parents and transfer of web objects occurs over the relatively more resource-intensive TCP connections.

As indicated in Figure 1-3, the siblings (B, C, and D) form a Netra Proxy Cache Array. The establishment of sibling relationships occur automatically when you configure the array.

Note that you can add one or more Netra Proxy Cache Servers to any single proxy server shown in Figure 1-1, Figure 1-2, or Figure 1-3 to form a Netra Proxy Cache Array. For a proxy server that relies on a parent, pointing to single machine is no different from pointing to a Netra Proxy Cache Array. That is, the child proxy requires no additional configuration and needs no awareness that the proxy service is provided by multiple hosts.

Netra Proxy Cache Array Features

A Netra Proxy Cache Array consists of multiple hosts that, together, provide a single instance of a proxy cache service.

Array Features

The array software offers the following features:

• reliability

• scalability

• load balancing

How these features are provided is described in the following subsections.

Reliability

The array software provides reliability by:

1. monitoring individual host hardware and software, and upon failure of some component...

2. moving the service address of a service instance on a host to a different host.

Existing clients of a moved service address continue to be served. The DNS within the Netra Proxy Cache array removes the moved service address from proxy cache service provided by the array, so that new clients are never connected to it.

Scalability

The array software provides scalability by implementing a protocol that responds dynamically to changes in array membership. At a frequent and regular interval, the array daemon multicasts information messages over the control network. These information messages are, at once, a heartbeat and a means of conveying "health" information about each host.

When a host is removed--for example, if a machine is receiving a software upgrade--within milliseconds, the array detects the machine's absence and removes the machine's service address(es) from availability. Similarly, if a machine is added to the array, nearly immediately, the array detects the new host and makes its service address available for incoming client requests.

Load Balancing

The array software provides load balancing through a modified DNS round robin. The Netra Proxy Cache array forms its own DNS zone that consists of the service addresses associated with the proxy cache service. This zone is identified by its own domain name.

When a host fails or becomes overloaded, the array software removes the host from the array's DNS round robin, so that the down host receives no new client requests. This process is discussed in greater detail in "The Role of DNS".

What Array Features Mean to You

Some of the practical benefits of a Netra Proxy Cache Array are as follows:

• With no software configuration, you can halt a machine and remove it from the array, either permanently or temporarily, without interrupting service to your user community.

• With minimal software configuration, or by copying an existing configuration, you can add a machine to an array. Again, this occurs without any disruption of service.

• The collection of proxy cache service instances automatically communicate via the Inter Cache Protocol, which means that the cache of web objects is extended over all the machines in the array.

• User response is enhanced because of high availability and the extended cache provided by the array.

Technical Details

The array software consists of a daemon and software objects that run on each of the machines in a Netra Proxy Cache array. On a periodic basis, the array software monitors the health (reachability, load, presence of server processes) of the hardware resources and proxy cache service on each machine. The array software multicasts these individual-host snapshots over an isolated network, called a control network, to which each member of the Netra Proxy Cache array is connected. Netra Proxy Cache Servers are equipped with a second network interface to enable connection to a control network. This second interface is referred to as the control interface.

---

Note -

The Internet Assigned Numbers Authority (IANA) has designated a multicast address (224.0.1.62) for use with Netra Proxy Cache Array software.

---

The result of the multicasts over the control network is that all hosts in the array are kept up to date on the health of each host. Responding to these regular updates, array software works to provide users with a continuous proxy cache service, in the face of hardware and software failures and in spite of varying loads on individual servers' resources.

The resource managed by array software is a service address. This is a logical IP address that is associated with the proxy cache service on a given machine. If a machine fails or becomes overloaded, array software can remove the availability of the service address or move the address to a different machine in the array.

A service address is associated with the network interface over which proxy cache server-client interactions occur. In terms of a Netra Proxy Cache array, this is the service interface and the network to which the interface is connected is the service network. The service network is, most often, the local area network (LAN) over which clients access a variety of network services.

The concepts of control and service networks are illustrated in Figure 1-4.

Figure 1-4 Control and Service Networks

Figure 1-5 illustrates the concepts of control and service interfaces and addresses.

Figure 1-5 Example Netra Proxy Cache Configuration

The Role of DNS

The array software relies on the DNS to provide load balancing, as described in "Load Balancing".

When all hosts and services are up and under a normal load, the array's DNS works as a conventional DNS round robin. It is when there is a host or service failure, or if a host becomes overloaded, that the array software intervenes to balance the load across the array. The software accomplishes this by removing the service address of the failed or overloaded host from the array's DNS zone. As a result of this intervention, no new client requests will be connected to a service address associated with a failed or overloaded object.

The threshold at which a host becomes overloaded, as well as the threshold at which a formerly-overloaded host is considered available again, are configurable parameters.

As part of its manipulation of the array's DNS zone, in a situation where one or more hosts in the array is moving back and forth between overload and normal load, the array software always keeps the least-loaded hosts in the zone.

In support of the array software's modification of the DNS round robin, you designate one host within the array as the DNS server for the domain formed by the array. (The choice of which host is arbitrary.) The DNS itself operates under control of the array software, so that if the DNS software or the host on which DNS is running fails, the service address of the DNS moves to a different machine in the array. Array software is designed so that one host runs the DNS and the remaining hosts in the array act as hot spares for the DNS host.

In addition to the array software's use of DNS for load balancing, the software also relies on the DNS outside of the array to resolve the name of the subdomain formed by the Netra Proxy Cache Array. This subdomain consists of the DNS zone formed by the service addresses in the array. This means you must configure your existing DNS to point to the array's DNS to resolve the name of the array's proxy cache service. An example of such a configuration is shown in Chapter 3 of the Netra Proxy Cache Array Configuration Guide.

Monitoring and Managing

The Netra Proxy Cache Server offers web-based tools that enable you to monitor:

• An individual server

• An array

• The state of the proxy cache service, for an array and a server

There are also web pages that enable you to monitor proxy-cache-related log files. See Chapter 15, Monitoring a Netra Proxy Cache Array and Proxy Cache Service," for a description of the monitoring web pages. See Chapter 17, Monitoring Proxy Cache Log Files," for a description of the various types of logs available.

The Netra Proxy Cache product is shipped with Management Information Bases (MIBs) that enable you to use any SNMP-conformant management platform (such as Solstice Domain Manager) to monitor and manage a Netra Proxy Cache Server. The software also supports a set of traps that notify you of critical events, ranging from a down server to a failure report on a server component.

The Netra Proxy Cache Server and Array MIBs are described in detail in Chapter 16, Netra Proxy Cache Array MIBs and Traps."