Chapter 10 Resource Pools

This chapter discusses resource pools, which are used for partitioning machine resources. Resource pools enable you to separate workloads so that workload consumption of certain resources does not overlap. This resource reservation helps to achieve predictable performance on systems with mixed workloads.

Overview

Resource pools provide a persistent configuration mechanism for processor set configuration and, optionally, scheduling class assignment.

Figure 10-1 Resource Pool Framework

By grouping multiple partitions, pools provide a handle to associate with labeled workloads. Each project entry in the /etc/project database can have a pool associated with it. New work that is started on a project is bound to the appropriate pool.

The pools mechanism is primarily for use on large machines of more than four CPUs. However, small machines can still benefit from this functionality. On small machines, you can create pools that share noncritical resource partitions. The pools are separated only on the basis of critical resources.

When to Use Pools

Resource pools offer a versatile mechanism that can be applied to many administrative scenarios, as described in the following sections.

Batch Compute Server

Use pools functionality to split a server into two pools.

One pool is used for login sessions and interactive work by timesharing users. The other pool is used for jobs that are submitted through the batch system.

Application or Database Server

Partition the resources for interactive applications in accordance with the applications' requirements.

Turning on Applications in Phases

Set user expectations.

You might initially deploy a machine that is running only a fraction of the services that the machine is ultimately expected to deliver. User difficulties can occur if reservation-based resource management mechanisms are not established when the machine comes online.

For example, the fair share scheduler optimizes CPU utilization. The response times for a machine that is running only one application can be misleadingly fast when compared to the response times users see with multiple applications loaded. By using separate pools for each application, you can ensure that a ceiling on the number of CPUs available to each application is in place before all applications are deployed.

Complex Timesharing Server

Partition a server that supports large user populations.

Server partitioning provides an isolation mechanism that leads to a more predictable per-user response.

By dividing users into groups that bind to separate pools, and using the fair share scheduling (FSS) facility, you can tune CPU allocations to favor sets of users that have priority. This assignment can be based on user role, accounting chargeback, and so forth.

Workloads That Change Seasonally

Use resource pools to adjust to changing demand.

Your site might experience predictable shifts in workload demand over long periods of time, such as monthly, quarterly, or annual cycles. If your site experiences these shifts, you can alternate between multiple pools configurations by invoking pooladm from a cron(1M) job.

Real-Time Applications

Create a real-time pool by using the RT scheduler and designated processor resources.

Administering Pools

The commands that are shown in the following table provide the primary administrative interface to the pools facility.

A library API is provided by libpool(3LIB). The library can be used by programs to manipulate pool configurations.

Pools Framework

The resource pools framework stores its view of the machine in a private configuration file. (The location of the file is private to the implementation of the pools framework.) This configuration file represents the pools framework's view of the machine. The file also contains information about configured pools and the organization of partitionable resources. Each pool can contain the following:

• A reference to a processor set or a CPU resource partition

• A property that specifies the pool's default scheduling class

Implementing Pools on a System

Pools can be implemented on a system by using one of these methods.

1. When the Solaris software boots, an init script checks to see if the /etc/pooladm.conf file exists. If this file is found, then pooladm is invoked to make this configuration the active pools configuration. The system creates a private configuration file to reflect the organization that is requested in /etc/pooladm.conf, and the machine's resources are partitioned accordingly.

2. When the Solaris environment is running, a pools configuration can either be activated if it is not already present, or modified by using the pooladm command. By default, pooladm operates on /etc/pooladm.conf. However, you can optionally specify an alternate location and file name, and use this file to update the pools configuration.

Dynamic Reconfiguration Operations and Resource Pools

Dynamic reconfiguration (DR) enables you to reconfigure hardware while the system is running. Because DR affects available resource amounts, the pools facility must be included in these operations. When a DR operation is initiated, the pools framework acts to validate the configuration.

If the DR operation can proceed without causing the current pools configuration to become invalid, then the private configuration file is updated. An invalid configuration is one that cannot be supported by the available resources.

If the DR operation would cause the pools configuration to be invalid, then the operation fails and you are notified by a message to the message log. If you want to force the configuration to completion, you must use the DR force option. The pools configuration is then modified to comply with the new resource configuration.

Creating Pools Configurations

The configuration file contains a description of the pools to be created on the system. The file describes the entities and resource types that can be manipulated.

See poolcfg(1M) for more information on elements that be manipulated.

You can create a structured /etc/pooladm.conf file in two ways.

• You can use poolcfg to discover the resources on the current system and place the results in a configuration file.

  This method simplifies file construction. All active resources and components on the system that are capable of being manipulated by the pools facility are recorded. The resources include existing processor set configurations. You can then modify the configuration to rename the processor sets or to create additional pools if necessary.

• You can use poolcfg to create a new pools configuration.

  Use this method when you develop configurations for other machines or when you create configurations that you want to apply to the current machine at a later time.

Use poolcfg or libpool to modify the /etc/pooladm.conf file. Do not directly edit this file.

How to Create a Configuration by Discovery

Use the discover subcommand argument to the -c option of /usr/sbin/poolcfg to create the pools configuration file. The resulting file, /etc/pooladm.conf, contains any existing processor sets.

1. Become superuser.

2. Type the following:

   # poolcfg -c discover

You can also supply a file name to use instead of the default /etc/pooladm.conf. If the file name is supplied, then the poolcfg commands are applied to the contents of the named file.

For example, to place a discovered configuration in /tmp/foo, do the following:

1. Become superuser.

2. Type the following:

   # poolcfg -c discover /tmp/foo

How to Create a New Configuration

Use the create subcommand argument to the -c option of /usr/sbin/poolcfg to create a simple configuration file for a system named tester. Note that you must quote subcommand arguments that contain white space.

1. Become superuser.

2. Type the following:

   # poolcfg -c 'create system tester'

3. View the contents of the configuration file in readable form.

   # poolcfg -c info system tester int system.version 1 boolean system.bind-default true string system.comment

How to Modify a Configuration

To enhance your simple configuration, create a processor set named batch and a pool named batch. Then join them with an association. Note that you must quote subcommand arguments that contain white space.

1. Become superuser.

2. Create processor set batch.

   # poolcfg -c 'create pset batch (uint pset.min = 2; uint pset.max = 10)'

3. Create pool batch.

   # poolcfg -c 'create pool batch'

4. Join with an association.

   # poolcfg -c 'associate pool batch (pset batch)'

5. Display the edited configuration.

   # poolcfg -c info system tester int system.version 1 boolean system.bind-default true string system.comment pool batch boolean pool.default false boolean pool.active true int pool.importance 1 string pool.comment pset batch pset batch int pset.sys_id -2 string pset.units population boolean pset.default true uint pset.max 10 uint pset.min 2 string pset.comment boolean pset.escapable false uint pset.load 0 uint pset.size 0

How to Associate a Pool With a Scheduling Class

You can associate a pool with a scheduling class so that all processes bound to the pool use this scheduler. To do this, set the pool.scheduler property to the name of the scheduler class. This example shows how to associate the pool batch with the FSS.

1. Become superuser.

2. Modify pool batch to be associated with the FSS.

   # poolcfg -c 'modify pool batch (string pool.scheduler="FSS")'

3. Display the edited configuration.

   # poolcfg -c info system tester int system.version 1 boolean system.bind-default true string system.comment pool batch boolean pool.default false boolean pool.active true int pool.importance 1 string pool.comment string pool.scheduler FSS pset batch pset batch int pset.sys_id -2 string pset.units population boolean pset.default true uint pset.max 10 uint pset.min 2 string pset.comment boolean pset.escapable false uint pset.load 0 uint pset.size 0

How to Use Command Files With poolcfg

poolcfg -f can take input from a text file that contains poolcfg subcommand arguments to the -c option. This technique is appropriate when you want a set of operations to be performed atomically. When processing multiple commands, the configuration is only updated if all of the commands succeed. For large or complex configurations, this technique can be more useful than per-subcommand invocations.

1. Create the input file.

   $ cat > poolcmds.txt create system tester create pset batch (int pset.man = 2; int pset.max = 10) create pool batch associate pool batch (pset batch)

2. Become superuser.

3. Type the following:

   # /usr/sbin/poolcfg -f poolcmds.txt

Activating and Deactivating Pools Configurations

Use pooladm(1M) to make a particular pool configuration active or to remove an active pools configuration.

How to Activate a Pools Configuration

To activate the configuration in the default static configuration file, /etc/pooladm.conf, invoke pooladm with the -c option, "commit configuration."

1. Become superuser.

2. Type the following:

   # /usr/sbin/pooladm -c

How to Deactivate a Pools Configuration

To remove the running configuration and all associated resources, such as processor sets, use the -x option for "remove configuration."

1. Become superuser.

2. Type the following:

   # /usr/sbin/pooladm -x

The -x option to pooladm removes the dynamic private configuration file as well as all resource configurations that are associated with the dynamic configuration. Thus, the -x option provides a mechanism for recovering from a poorly designed pools configuration. All processes share all of the resources on the machine.

Mixing scheduling classes within one processor set can lead to unpredictable results. If you use pooladm -x to recover from a bad configuration, you should then use priocntl(1) to move running processes into a different scheduling class.

Binding to a Pool

You can bind a running process to a pool in two ways.

• You can use the poolbind(1M) command to bind a specific process to a named resource pool.

• You can use the project.pool attribute in the project(4) database to identify the pool binding for a new login session or a task that is launched through newtask(1).

How to Bind Processes to a Pool

The following procedure manually binds a process (for example, the current shell) to a pool named ohare.

1. Become superuser.

2. Type the following:

   # poolbind -p ohare $$

How to Bind Tasks or Projects to a Pool

To bind tasks or projects to a pool, use poolbind with the -i option. The following example binds all processes in the airmiles project to the laguardia pool.

1. Become superuser.

2. Type the following:

   # poolbind -i project -p laguardia airmiles

How to Use project Attributes to Bind New Processes to a Pool

To automatically bind new processes in a project to a pool, add the project.pool attribute to each entry in the project database.

For example, assume you have a configuration with two pools that are named studio and backstage. The /etc/project file has the following contents.

user.paul:1024::::project.pool=studio
user.george:1024::::project.pool=studio
user.ringo:1024::::project.pool=backstage
passes:1027::paul::project.pool=backstage

With this configuration, processes that are started by user paul are bound by default to the studio pool.

How to Use project Attributes to Bind a Process to a Different Pool

Using the previous configuration, user paul can modify the pool binding for processes he starts. He can use newtask to bind work to the backstage pool as well, by launching in the passes project.

1. Launch a process in the passes project.

   $ newtask -l -p passes

2. Verify the pool binding for the process.

   $ poolbind -q $$ process id 6384 : pool 'backstage'