Chapter 1 Overview of Resource Management

This book provides guidelines for creating a resource type for a software application such as Oracle®, Sun Java^TM System Web Server (formerly Sun^TM ONE Web Server), DNS, and so on. As such, this book is intended for developers of resource types.

This chapter provides an overview of the concepts you need to understand in order to develop a data service and contains the following information.

• Sun Cluster Application Environment

• RGM Model

• Resource Group Manager

• Callback Methods

• Programming Interfaces

• Resource Group Manager Administrative Interface

---

Note –

This book uses the terms resource type and data service interchangeably. The term agent, though rarely used in this book, is equivalent to resource type and data service.

---

Sun Cluster Application Environment

The Sun Cluster system enables applications to be run and administered as highly available and scalable resources. The cluster facility known as the Resource Group Manager, or RGM, provides the mechanism for high availability and scalability. The elements that form the programming interface to this facility include the following.

• A set of callback methods you write that enable the RGM to control an application on the cluster

• The Resource Management API (RMAPI), a set of low-level API commands and functions that you can use to write the callback methods. These APIs are implemented in the libscha.so library.

• Process management facilities for monitoring and restarting processes on the cluster

• The Data Service Development Library (DSDL), a set of library functions that encapsulates the low-level API and process-management functionality at a higher level and adds some additional functionality to ease the writing of callback methods. These functions are implemented in the libdsdev.so library.

The following figure shows the interrelationship of these elements.

Figure 1–1 Programming Architecture

Included in the Sun Cluster package is SunPlex^TM Agent Builder, a tool that automates the process of creating a data service (see Chapter 9, SunPlex Agent Builder). Agent Builder generates data service code in either C (using DSDL functions to write the callback methods) or in Korn shell (ksh) (using low-level API commands to write the callback methods).

The RGM runs as a daemon on each cluster node and automatically starts and stops resources on selected nodes according to preconfigured policies. The RGM makes a resource highly available in the event of a node failure or reboot by stopping the resource on the affected node and starting it on another. The RGM also automatically starts and stops resource-specific monitors that can detect resource failures and relocate failing resources onto another node or can monitor other aspects of resource performance.

The RGM supports both failover resources, which can be online on at most one node at a time, and scalable resources, which can be online on multiple nodes simultaneously.

RGM Model

This section introduces some fundamental terminology and explains in more detail the RGM and its associated interfaces.

The RGM handles three major kinds of interrelated objects: resource types, resources, and resource groups. One way to introduce these objects is by means of an example, as described below.

A developer implements a resource type, ha-oracle, that makes an existing Oracle DBMS application highly available. An end user defines separate databases for marketing, engineering, and finance, each of which is a resource of type ha-oracle. The cluster administrator places these resources in separate resource groups so they can run on different nodes and fail over independently. A developer creates a second resource type, ha-calendar, to implement a highly available calendar server that requires an Oracle database. The cluster administrator places the resource for the finance calendar into the same resource group as the finance database resource so that both resources run on the same node and fail over together.

Resource Types

A resource type consists of a software application to be run on the cluster, control programs used as callback methods by the RGM to manage the application as a cluster resource, and a set of properties that form part of the static configuration of a cluster. The RGM uses resource type properties to manage resources of a particular type.

---

Note –

In addition to a software application, a resource type can represent other system resources such as network addresses.

---

The resource type developer specifies the properties for the resource type and sets their values in a resource type registration (RTR) file. The RTR file follows a well-defined format described in Setting Resource and Resource Type Properties and in the rt_reg(4) man page. See also Defining the Resource Type Registration File for a description of a sample resource type registration file.

Table A–1 provides a list of the resource type properties.

The cluster administrator installs and registers the resource type implementation and underlying application on a cluster. The registration procedure enters into the cluster configuration the information from the resource type registration file. The Sun Cluster Data Services Planning and Administration Guide for Solaris OS describes the procedure for registering a data service.

Resources

A resource inherits the properties and values of its resource type. In addition, a developer can declare resource properties in the resource type registration file. See Table A–2 for a list of resource properties.

The cluster administrator can change the values of certain properties depending on how they were specified in the resource type registration (RTR) file. For example, property definitions can specify a range of allowable values and specify when the property is tunable, for example, at creation, any time, or never. Within these specifications, the cluster administrator can make changes to properties using administration commands.

The cluster administrator can create many resources of the same type, each resource having its own name and set of property values, so that more than one instance of the underlying application can run on the cluster. Each instantiation requires a unique name within the cluster.

Resource Groups

Each resource must be configured in a resource group. The RGM brings all resources in a group online and offline together on the same node. When the RGM brings a resource group online or offline, it invokes callback methods on the individual resources in the group.

The nodes on which a resource group is currently online are called its primaries or primary nodes. A resource group is mastered by each of its primaries. Each resource group has an associated Nodelist property, which is set by the cluster administrator, that identifies all potential primaries or masters of the resource group.

A resource group also has a set of properties. These properties include configuration properties that can be set by the cluster administrator and dynamic properties, set by the RGM, that reflect the active state of the resource group.

The RGM defines two types of resource groups, failover and scalable. A failover resource group can be online on one node only at any time while a scalable resource group can be online on multiple nodes simultaneously. The RGM provides a set of properties to support the creation of each type of resource group. See Transferring a Data Service to a Cluster and Implementing Callback Methods for details about these properties.

See Table A–3 for a list of resource group properties.

Resource Group Manager

The Resource Group Manager (RGM) is implemented as a daemon, rgmd, that runs on each member node of the cluster. All of the rgmd processes communicate with each other and act together as a single cluster-wide facility.

The RGM supports the following functions:

• Whenever a node boots or crashes, the RGM attempts to maintain availability of all managed resource groups by automatically bringing them online on appropriate masters.

• If a particular resource fails, its monitor program can request that the resource group be restarted on the same master or switched to a new master.

• The cluster administrator can issue an administrative command to request one of the following actions:

  • Change mastery of a resource group

  • Enable or disable a particular resource within a resource group

  • Create, delete, or modify a resource, a resource group, or a resource type

Whenever the RGM activates configuration changes, it coordinates its actions across all member nodes of the cluster. This kind of activity is known as a reconfiguration. To effect a state change on an individual resource, the RGM invokes a resource-type specific callback method on that resource.

Callback Methods

The Sun Cluster framework uses a callback mechanism to provide communication between a data service and the RGM. The framework defines a set of callback methods, including their arguments and return values, and the circumstances under which the RGM calls each method.

You create a data service by coding a set of individual callback methods and implementing each method as a control program callable by the RGM. That is, the data service does not consist of a single executable but rather consists of a number of executable scripts (ksh) or binaries (C), each of which is directly callable by the RGM.

Callback methods are registered with the RGM through the resource type registration (RTR) file. In the RTR file you identify the program for each method you have implemented for the data service. When a system administrator registers the data service on a cluster, the RGM reads the RTR file, which provides, among other information, the identity of the callback programs.

The only required callback methods for a resource type are a start method (Start or Prenet_start), and a stop method (Stop or Postnet_stop).

The callback methods can be grouped into the following categories:

• Control and initialization methods

  • Start and Stop start and stop resources in a group that is being brought online or offline.

  • Init, Fini, Boot execute initialization and termination code on resources.

• Administrative support methods

  • Validate verifies properties set by administrative action.

  • Update updates the property settings of an online resource.

• Net-relative methods

  • Prenet_start and Postnet_stop do special startup or shutdown actions before network addresses in the same resource group are configured up or after they are configured down.

• Monitor control methods

  • Monitor_start and Monitor_stop start or stop the monitor for a resource.

  • Monitor_check assesses the reliability of a node before a resource group is moved to the node.

See Chapter 4, Resource Management API Reference and the rt_callbacks(1HA) man page for more information on the callback methods. Also see Chapter 5, Sample Data Service and Chapter 8, Sample DSDL Resource Type Implementation for callback methods in sample data services.

Programming Interfaces

For writing data service code, the resource management architecture provides a low-level, or base API, a higher-level library built on top of the base API, and a tool, SunPlex Agent Builder, that automatically generates a data service from basic input that you provide.

RMAPI

The RMAPI (Resource Management API) provides a set of low-level routines that enable a data service to access information about the resources, resource types and resource groups in the system, request a local restart or failover, and set the resource status. You access these functions through the libscha.so library. The RMAPI provides these callback methods both in the form of shell commands and in the form of C functions. See scha_calls(3HA) and Chapter 4, Resource Management API Reference for more information on the RMAPI routines. Also see Chapter 5, Sample Data Service for examples of how to use these routines in sample data service callback methods.

Data Service Development Library (DSDL)

Built on top of the RMAPI is the DSDL, which provides a higher-level integrated framework while retaining the underlying method-callback model of the RGM. The DSDL brings together various facilities for data-service development, including:

• libscha.so—the low-level resource management APIs

• PMF—the process management facility, which provides a means of monitoring processes and their descendants, and restarting them if they die (see pmfadm(1M) and rpc.pmfd(1M)).

• hatimerun—a facility for running programs under a timeout (see hatimerun(1M).

For the majority of applications, the DSDL provides most or all of the functionality you need to build a data service. Note, however, that the DSDL does not replace the low-level API but encapsulates and extends it. In fact, many DSDL functions call the libscha.so functions. Likewise you can directly call libscha.so functions while using the DSDL to code the bulk of your data service. The libdsdev.so library contains the DSDL functions.

See Chapter 6, Data Service Development Library (DSDL) and the scha_calls(3HA) man page for more information about the DSDL.

SunPlex Agent Builder

Agent Builder is a tool that automates the creation of a data service. You input basic information about the target application and the data service to be created.Agent Builder generates a data service, complete with source and executable code (C or Korn shell), customized RTR file, and a Solaris^TM package.

For most applications, you can use Agent Builder to generate a complete data service with only minor manual changes on your part. Applications with more sophisticated requirements, such as adding validation checks for additional properties, might require work that Agent Builder cannot do. However, even in these cases you might be able to use Agent Builder to generate the bulk of the code and manually code the rest. At minimum, you can use Agent Builder to generate the Solaris package for you.

Resource Group Manager Administrative Interface

Sun Cluster provides both a graphical user interface and a set of commands for administering a cluster.

SunPlex Manager

SunPlex Manager is a Web-based tool that enables you to perform the following tasks.

• Install a cluster

• Administer a cluster

• Create and configure resources and resource groups

• Configure data services with the Sun Cluster software

See the Sun Cluster Software Installation Guide for Solaris OS for instructions on how to install SunPlex Manager and how to use SunPlex Manager to install cluster software. SunPlex Manager provides online help for most unique administrative tasks.

Administrative Commands

The Sun Cluster commands for administering RGM objects are scrgadm(1M), scswitch(1M), and scstat(1M) -g.

The scrgadm command allows viewing, creating, configuring and deleting the resource type, resource group, and resource objects used by the RGM. The command is part of the administrative interface for the cluster, and is not to be used in the same programming context as the application interface described in the rest of this chapter. However, scrgadm is the tool for constructing the cluster configuration in which the API operates. Understanding the administrative interface sets the context for understanding the application interface. Refer to the scrgadm(1M) man page for details on the administrative tasks that can be performed by the command.

The scswitch command switches resource groups online and offline on specified nodes and enables or disables a resource or its monitor. See the scswitch(1M) man page for details on the administrative tasks that the command can perform.

The scstat -g command shows the current dynamic state of all resource groups and resources.