Filtering URLs

You can use a file of URLs to configure what content the proxy server retrieves. You can set up a list of URLs the proxy always supports and a list of URLs the proxy never supports.

For example, if you are an Internet service provider who runs a proxy server with content appropriate for children, you might set up a list of URLs that are approved for viewing by children. You can then have the proxy server retrieve only the approved URLs. If a client tries to go to an unsupported URL, either you can have the proxy return the default “Forbidden” message or you can create a custom message explaining why the client could not access that URL.

To restrict access based on URLs, create a file of URLs through the Server Manager to allow or restrict. You can do this through the Server Manager. Once you have created the file, you can set up the restrictions. These processes are discussed in the following sections.

Creating a Filter File of URLs

A filter file is a file that contains a list of URLs. The filter files that the proxy server uses are plain text files with lines of URLs in the following pattern:

protocol://host:port/path/filename

You can use regular expressions in each of the three sections: protocol, host:port, and path/filename. For example, if you want to create a URL pattern for all protocols going to the netscape.com domain, you’d have the following line in your file:

.*://.*\\.example\\.com/.*

This line works only if you do not specify a port number. For more information about regular expressions, see “Understanding Regular Expressions” in Chapter 16, Managing Templates and Resources.

If you want to create your own file without using the Server Manager, use the Server Manager pages to create an empty file, and then add your text in that file or replace the file with one containing the regular expressions.

To Create a Filter File

1. Access the Server Manager, and click the Filters tab.

2. Click the Restrict URL Filter Access link.

   The Restrict URL Filter Access page is displayed.

3. Choose New Filter from the drop-down list next to the Create/Edit button.

4. Type a name for the filter file in the text box to the right of the drop-down list and then click the Create/Edit button.

   The Filter Editor page is displayed.

5. Use the Filter Content scrollable text box to type URLs and regular expressions of URLs.

   The Reset button clears all the text in this field.

   For more information on regular expressions, see “Understanding Regular Expressions” in Chapter 16, Managing Templates and Resources.

6. Click OK.

   The proxy server creates the file and returns you to the Restrict URL Filter Access page. The filter file is created in the proxy-serverid/conf_bk directory.

Setting Default Access for a Filter File

Once you have a filter file that contains the URLs you want to use, you can set the default access for those URLs.

To Set Default Access for a Filter File

1. Access the Server Manager, and click the Filters tab.

2. Click the Restrict URL Filter Access link.

   The Restrict URL Filter Access page is displayed.

3. Choose the template you want to use with the filters.

   Typically, you will want to create filter files for the entire proxy server, but you might want one set of filter files for HTTP and another for FTP.

4. Use the URL Filter To Allow list to choose a filter file that contains the URLs you want the proxy server to support.

5. Use the URL Filter To Deny list to choose a filter file that contains the URLs to which you want the proxy server to deny access.

6. Choose the text you want the proxy server to return to clients who request a denied URL.

   • Send the default “Forbidden” response that the proxy generates.

     • Send a text or HTML file with customized text. Type the absolute path to this file in the text box.

7. Click OK.

8. Click Restart Required. The Apply Changes page is displayed.

9. Click the Restart Proxy Server button to apply the changes.