| Skip Navigation Links | |
| Exit Print View | |
|
Copying and Creating Oracle Solaris 11.1 Package Repositories Oracle Solaris 11.1 Information Library |
| Skip Navigation Links | |
| Exit Print View | |
|
|
Copying and Creating Oracle Solaris 11.1 Package Repositories Oracle Solaris 11.1 Information Library |
1. Image Packaging System Package Repositories
2. Copying IPS Package Repositories
3. Providing Access To Your Repository
4. Maintaining Your Local IPS Package Repository
Updating Your Local Repository
Checking and Setting Repository Properties
Customizing Your Local Repository
Serving Multiple Repositories Using Multiple Depot Server Instances
Depot Server Apache Configuration
Configuring Caching for the Depot Server
Cache Considerations for the Catalog Attributes File
Cache Considerations for Search
Running the Depot Server Behind a Web Proxy
Recommended Generic Apache Configuration Settings
A Simple Prefixed Proxy Configuration
This section discusses running the depot server behind an Apache web server instance to gain the following benefits:
Improve performance by content caching and load balancing.
Allow hosting multiple repositories under one domain name.
Minimal configuration is required to set up the depot server behind a caching proxy. With the exception of the catalog attributes file and repository search results, which are discussed below, all files served are unique and therefore safe to cache indefinitely if necessary. Also, all depot responses contain the appropriate HTTP headers to ensure files in the cache do not become stale by mistake.
See the Apache Caching Guide for more information about configuring Apache as a caching proxy.
Use the CacheRoot directive to specify the directory to contain the cached files. Make sure the specified directory is writable by the Apache process. No explicit error message is output if Apache cannot write to this directory.
CacheRoot /tank/proxycache
Apache allows you to enable caching for specific directories. You probably want your repository server to cache all the content on the server, as shown in the following directive.
CacheEnable disk /
Use the CacheMaxFileSize directive to set the maximum size of files to be cached. The Apache default of 1 MB might be too small for most repositories. The following directive sets the maximum cached file size to 1 GB.
CacheMaxFileSize 1000000000
Adjust the directory structure of the on-disk cache for the best performance with the underlying file system. In a ZFS dataset, multiple directory levels affect performance more than the number of files in one directory. Therefore, configure one directory level with a large number of files in each directory. Use the CacheDirLevels and CacheDirLength directives to control the directory structure. Set CacheDirLevels to 1. Set CacheDirLength to a value that results in a good balance between the number of directories and the number of files per directory. The value of 2 set below will generate 4096 directories. See the Apache Disk-based Caching documentation for more information.
CacheDirLevels 1 CacheDirLength 2
The repository catalog attributes file (catalog.attrs) contains the current status of the repository catalog. This file can be large enough to warrant caching. However, this file becomes stale if the catalog of the back-end repository has changed. You can use one of the following two methods to address this issue.
Do not cache this file. This solution works best if the repository server runs in a high-bandwidth environment where the additional traffic is not an important consideration. The following partial httpd.conf file shows how to specify not to cache the catalog.attrs file:
<LocationMatch ".*/catalog.attrs">
Header set Cache-Control no-cache
</LocationMatch>Prune this file from the cache whenever the catalog of the back-end repository is updated.
Searching a package repository generates custom responses based on the request. Therefore, search results are not well suited for being cached. The depot server sets the appropriate HTTP headers to make sure search results do not become stale in a cache. However, the expected bandwidth savings from caching are small. The following partial httpd.conf file shows how to specify not to cache search results.
<LocationMatch ".*/search/\d/.*">
Header set Cache-Control no-cache
</LocationMatch>
The pkg(5) depot server enables you to easily provide access to a repository in the local network or on the Internet. However, the depot server does not support serving multiple repositories under one domain name or sophisticated prefixes. To host multiple repositories under one domain name, run the depot server behind a web proxy. Running the depot server behind a web proxy can also improve the performance of the server by enabling load-balancing over multiple depots and enabling content caching.
The examples in this section use the Apache web server as the proxy software. The Oracle Solaris 11.1 OS includes the Apache web server in the web/server/apache-22 package and a basic httpd.conf file in /etc/apache2/2.2. The Apache web server is activated by enabling the svc:/network/http:apache22 service. See Apache HTTP Server Version 2.2 Documentation for additional information.
You should be able to apply the principles shown in these examples to any proxy server software.
The following settings affect performance and security.
HTTP clients can tell the server that they accept compressed data in an HTTP request. Enabling the Apache DEFLATE filter can dramatically reduce the over-the-wire size of metadata such as catalogs and manifests. Metadata such as catalogs and manifests often compress 90%.
AddOutputFilterByType DEFLATE text/html application/javascript text/css text/plain
Packages can contain URL encoded forward slashes. To make sure these forward slashes are not interpreted as directory delimiters, instruct Apache to not decode them.
AllowEncodedSlashes NoDecode
Note - Omitting this setting very negatively impacts search functionality.
Increase the MaxKeepAliveRequests value to allow clients to make a larger number of pipelined requests without closing the connection. The Apache default of 100 is too low.
MaxKeepAliveRequests 10000
The proxy timeout sets how long Apache waits for the back-end depot to respond. For most operations, 30 seconds is satisfactory. Searches with a very large number of results can take significantly longer. You might want a higher timeout value to accommodate such searches.
ProxyTimeout 30
Make sure that forward proxying is disabled.
ProxyRequests Off
This section illustrates multiple repository, non-load-balanced, and load-balanced setups.
This example shows the basic configuration for a non-load-balanced depot server. This example connects http://pkg.example.com/myrepo to internal.example.com:10000.
See Serving Multiple Repositories Using Multiple Depot Server Instances for instructions about setting other properties you need that are not described in this example.
You should configure the depot server with a pkg/proxy_base setting that names the URL at which the depot server can be accessed. Use the following commands to set the pkg/proxy_base:
$ svccfg -s pkg/server add repo $ svccfg -s pkg/server:repo "setprop pkg/proxy_base = astring: http://pkg.example.com/myrepo" $ svcadm refresh pkg/server:repo $ svcadm enable pkg/server:repo
The pkg(5) client opens 20 parallel connections to the depot server when performing network operations. Make sure the number of depot threads matches the expected connections to the server at any given time. Use the following commands to set the number of threads per depot:
$ svccfg -s pkg/server:repo "setprop pkg/threads = 200" $ svcadm refresh pkg/server:repo $ svcadm restart pkg/server:repo
Use nocanon to suppress canonicalization of URLs. This setting is important for properly working search. Also, limit the number of back-end connections to the number of threads the depot server provides. The following partial httpd.conf file shows how to proxy one depot server:
Redirect /myrepo http://pkg.example.com/myrepo/ ProxyPass /myrepo/ http://internal.example.com:10000/ nocanon max=200
The most important reason to run the depot server behind a proxy is to easily run several repositories under one domain name with different prefixes. The example from A Simple Prefixed Proxy Configuration can be easily extended to support multiple repositories.
In this example, three different prefixes of one domain name are connected to three different package repositories:
http://pkg.example.com/repo_one is connected to internal.example.com:10000
http://pkg.example.com/repo_two is connected to internal.example.com:20000
http://pkg.example.com/xyz/repo_three is connected to internal.example.com:30000
The pkg(5) depot server is an SMF managed service. Therefore, to run multiple depot servers on the same host, simply create a new service instance:
$ svccfg -s pkg/server add repo1 $ svccfg -s pkg/server:repo1 setprop pkg/property=value $ ...
Like the previous example, each depot server runs with 200 threads.
Redirect /repo_one http://pkg.example.com/repo_one/ ProxyPass /repo_one/ http://internal.example.com:10000/ nocanon max=200 Redirect /repo_two http://pkg.example.com/repo_two/ ProxyPass /repo_two/ http://internal.example.com:20000/ nocanon max=200 Redirect /xyz/repo_three http://pkg.example.com/xyz/repo_three/ ProxyPass /xyz/repo_three/ http://internal.example.com:30000/ nocanon max=200
You might want to run depot servers behind an Apache load balancer. This example connects http://pkg.example.com/myrepo to internal1.example.com:10000 and internal2.example.com:10000.
Configure the depot server with an appropriate proxy_base setting as shown in A Simple Prefixed Proxy Configuration.
Limit the number of back-end connections to the number of threads each depot is running divided by the number of depots in the load-balancer setup. Otherwise, Apache opens more connections to a depot than are available and they stall, which can decrease performance. Specify the maximum number of parallel connections to each depot with the max= parameter. The example below shows two depots, each running 200 threads. See A Simple Prefixed Proxy Configuration for an example of how to set the number of depot threads.
<Proxy balancer://pkg-example-com-myrepo>
# depot on internal1
BalancerMember http://internal1.example.com:10000 retry=5 max=100
# depot on internal2
BalancerMember http://internal2.example.com:10000 retry=5 max=100
</Proxy>
Redirect /myrepo http://pkg.example.com/myrepo/
ProxyPass /myrepo/ balancer://pkg-example-com-myrepo/ nocanon
The following example includes all the directives you need to add to the httpd.conf file for a repository server hosting a load-balanced and a non-load-balanced depot server setup.
In this example, two different prefixes of one domain name are connected to three different package repositories:
http://pkg.example.com/repo_one is connected to internal1.example.com:10000 and internal2.example.com:10000
http://pkg.example.com/repo_two is connected to internal1.example.com:20000
AddOutputFilterByType DEFLATE text/html application/javascript text/css text/plain
AllowEncodedSlashes NoDecode
MaxKeepAliveRequests 10000
ProxyTimeout 30
ProxyRequests Off
<Proxy balancer://pkg-example-com-repo_one>
# depot on internal1
BalancerMember http://internal1.example.com:10000 retry=5 max=100
# depot on internal2
BalancerMember http://internal2.example.com:10000 retry=5 max=100
</Proxy>
Redirect /repo_one http://pkg.example.com/repo_one/
ProxyPass /repo_one/ balancer://pkg-example-com-repo_one/ nocanon
Redirect /repo_two http://pkg.example.com/repo_two/
ProxyPass /repo_two/ http://internal.example.com:20000/ nocanon max=200
|