Chapter 5 Sample Data Service
This chapter describes a sample Sun Cluster data service, HA-DNS, for the in.named application. The in.named daemon is the Solaris implementation of the Domain Name Service (DNS). The sample data service demonstrates how to make an application highly available, using the Resource Management API.
The Resource Management API supports a shell script interface and a C program interface. The sample application in this chapter is written using the shell script interface.
The information in this chapter includes:
Overview of the Sample Data Service
The sample data service starts, stops, restarts and switches the DNS application among the nodes of the cluster in response to cluster events such as administrative action, application failure, or node failure.
Application restart is managed by the Process Monitor Facility (PMF). If application deaths exceed the failure count within the failure time window, the fault monitor fails the resource group containing the application resource over to another node.
The sample data service provides fault monitoring in the form of a PROBE method. that uses the nslookup command to ensure that the application is healthy. If the probe detects a hung DNS service, it tries to correct the situation by restarting the DNS application locally. If this does not improve the situation and the probe repeatedly detects problems with the service, then the probe attempts to fail over the service to another node in the cluster.
Specifically, the sample data service includes:
-
A resource type registration file that defines the static properties of the data service.
-
A Start callback method invoked by the RGM to start the in.named daemon when the resource group containing the HA-DNS data service is brought online.
-
A Stop callback method invoked by the RGM to stop the in.named daemon when the resource group containing HA-DNS goes offline.
-
A fault monitor to check the availability of the service by verifying that the DNS server is running. The fault monitor is implemented by a user-defined PROBE method and started and stopped by Monitor_start and Monitor_stop callback methods.
-
A Validate callback method invoked by the RGM to validate that the configuration directory for the service is accessible.
-
An Update callback method invoked by the RGM to restart the fault monitor when the system administrator changes the value of a resource property.
Defining the Resource Type Registration File
The resource type registration (RTR) file in this example defines the static configuration of the DNS resource type. Resources of this type inherit the properties defined in the RTR file.
The information in the RTR file is read by the RGM when the cluster administrator registers the HA-DNS data service.
RTR File Overview
The RTR file follows a well-defined format. It begins with resource type properties, followed by system-defined resource properties, and lastly with extension properties. See the rt_reg(4) man page and Setting Resource and Resource Type Properties for more information.
This section describes the specific properties in the sample RTR file. It provides listings of different parts of the file. For a complete listing of the contents of the sample RTR file, see Resource Type Registration File Listing.
Resource Type Properties in the Sample RTR File
The sample RTR file begins with comments followed by resource type properties that define the HA-DNS configuration, as shown in the following listing.
# # Copyright (c) 1998-2001 by Sun Microsystems, Inc. # All rights reserved. # # Registration information for Domain Name Service (DNS) # #pragma ident “@(#)SUNW.sample 1.1 00/05/24 SMI” RESOURCE_TYPE = “sample”; VENDOR_ID = SUNW; RT_DESCRIPTION = “Domain Name Service on Sun Cluster”; RT_VERSION =”1.0”; API_VERSION = 2; FAILOVER = TRUE; RT_BASEDIR=/opt/SUNWsample/bin; PKGLIST = SUNWsample; START = dns_svc_start; STOP = dns_svc_stop; VALIDATE = dns_validate; UPDATE = dns_update; MONITOR_START = dns_monitor_start; MONITOR_STOP = dns_monitor_stop; MONITOR_CHECK = dns_monitor_check;
Tip –
You must declare the Resource_type property as the first entry in the RTR file. Otherwise, registration of the resource type will fail.
Note –
The RGM treats property names as case insensitive. The convention for properties in Sun-supplied RTR files, with the exception of method names, is uppercase for the first letter of the name and lowercase for the rest of the name. Method names—as well as property attributes—contain all uppercase letters.
Some information about these properties follows.
-
The resource type name can be specified by the Resource_type property alone (sample) or using the Vendor_id as a prefix with a “.” separating it from the resource type (SUNW.sample).
If you use Vendor_id, make it the stock symbol for the company defining the resource type. The resource type name must be unique in the cluster.
-
The Rt_version property identifies the version of the sample data service as specified by the vendor.
-
The API_version property identifies the Sun Cluster version. For example, API_version = 2, indicates that the data service runs under Sun Cluster version 3.0.
-
Failover = TRUE indicates that the data service cannot run in a resource group that can be online on multiple nodes at once.
-
RT_basedir points to /opt/SUNWsample/bin as the directory path to complete relative paths, such as callback method paths.
-
Start, Stop, Validate, and so on provide the paths to the respective callback method programs invoked by the RGM. These paths are relative to the directory specified by RT_basedir.
-
Pkglist identifies SUNWsample as the package that contains the sample data service installation.
Resource type properties not specified in this RTR file, such as Single_instance, Init_nodes, and Installed_nodes, get their default value. See Table A–1 for a complete list of the resource type properties, including their default values.
The cluster administrator cannot change the values specified for resource type properties in the RTR file.
Resource Properties in the Sample RTR File
By convention, you declare resource properties following the resource type properties in the RTR file. Resource properties include system-defined properties provided by Sun Cluster and extension properties you define. For either type you can specify a number of property attributes supplied by Sun Cluster, such as minimum, maximum, and default values.
System-Defined Properties in the RTR File
The following listing shows the system-defined properties in the sample RTR file.
# A list of bracketed resource property declarations follows the
# resource-type declarations. The property-name declaration must be
# the first attribute after the open curly bracket of each entry.
# The <method>_timeout properties set the value in seconds after which
# the RGM concludes invocation of the method has failed.
# The MIN value for all method timeouts is set to 60 seconds. This
# prevents administrators from setting shorter timeouts, which do not
# improve switchover/failover performance, and can lead to undesired
# RGM actions (false failovers, node reboot, or moving the resource group
# to ERROR_STOP_FAILED state, requiring operator intervention). Setting
# too-short method timeouts leads to a *decrease* in overall availability
# of the data service.
{
PROPERTY = Start_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Stop_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Validate_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Update_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Monitor_Start_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Monitor_Stop_timeout;
MIN=60;
DEFAULT=300;
}
{
PROPERTY = Thorough_Probe_Interval;
MIN=1;
MAX=3600;
DEFAULT=60;
TUNABLE = ANYTIME;
}
# The number of retries to be done within a certain period before concluding
# that the application cannot be successfully started on this node.
{
PROPERTY = Retry_Count;
MIN=0;
MAX=10;
DEFAULT=2;
TUNABLE = ANYTIME;
}
# Set Retry_Interval as a multiple of 60 since it is converted from seconds
# to minutes, rounding up. For example, a value of 50 (seconds)
# is converted to 1 minute. Use this property to time the number of
# retries (Retry_Count).
{
PROPERTY = Retry_Interval;
MIN=60;
MAX=3600;
DEFAULT=300;
TUNABLE = ANYTIME;
}
{
PROPERTY = Network_resources_used;
TUNABLE = AT_CREATION;
DEFAULT = ““;
}
Although Sun Cluster provides the system-defined properties, you can set different default values using resource property attributes. See Resource Property Attributes for a complete list of attributes available for applying to resource properties.
Note the following about the system-defined resource properties in the sample RTR file:
-
Sun Cluster provides a minimum value (1 second) and a default value (3600 seconds) for all timeouts. The sample RTR file changes the minimum 60 and changes the default to 300 seconds. A cluster administrator can accept this default value or change the value of the timeout to something else, (60 or greater). Sun Cluster has no maximum allowable value.
-
The properties Thorough_Probe_Interval, Retry_count, and Retry_interval, have the TUNABLE attribute set to ANYTIME. This settings means the cluster administrator can change the value of these properties, even when the data service is running. These properties are used by the fault monitor implemented by the sample data service. The sample data service implements an Update method to stop and restart the fault monitor when these or other resource properties are changed by administrative action. See Update Method.
-
Resource properties are classified as
-
required—the cluster administrator must specify a value when creating a resource;
-
optional—if the administrator does not specify a value, the system supplies a default value.
-
conditional—the RGM creates the property only if it is declared in the RTR file.
The fault monitor of the sample data service makes use of the Thorough_probe_interval, Retry_count, Retry_interval, and Network_resources_used conditional properties, so the developer needed to declare them in the RTR file. See the r_properties(5) man page or Resource Properties for information about how properties are classified.
-
Extension Properties in the RTR File
At the end of the sample RTR file are extension properties, as shown in the following listing
# Extension Properties
# The cluster administrator must set the value of this property to point to the
# directory that contains the configuration files used by the application.
# For this application, DNS, specify the path of the DNS configuration file on
# PXFS (typically named.conf).
{
PROPERTY = Confdir;
EXTENSION;
STRING;
TUNABLE = AT_CREATION;
DESCRIPTION = “The Configuration Directory Path”;
}
# Time out value in seconds before declaring the probe as failed.
{
PROPERTY = Probe_timeout;
EXTENSION;
INT;
DEFAULT = 120;
TUNABLE = ANYTIME;
DESCRIPTION = “Time out value for the probe (seconds)”;
}
The sample RTR file defines two extension properties, Confdir and Probe_timeout. Confdir specifies the path to the DNS configuration directory. This directory contains the in.named file, which DNS requires to operate successfully. The sample data service's Start and Validate methods use this property to verify that the configuration directory and the in.named file are accessible before starting DNS.
When the data service is configured, the Validate method verifies that the new directory is accessible.
The sample data services's PROBE method is not a Sun Cluster callback method but a user-defined method. Therefore Sun Cluster doesn't provide a Probe_timeout property for it. The developer has defined an extension property in the RTR file to allow a cluster administrator to configure a Probe_timeout value.
Providing Common Functionality to All Methods
This section describes the following functionality that is used in all callback methods of the sample data service:
Identifying the Command Interpreter and Exporting the Path
The first line of a shell script must identify the command interpreter. Each of the method scripts in the sample data service identifies the command interpreter as follows:
#!/bin/ksh |
All method scripts in the sample application export the path to the Sun Cluster binaries and libraries rather than rely on the user's PATH settings.
####################################################################### # MAIN ####################################################################### export PATH=/bin:/usr/bin:/usr/cluster/bin:/usr/sbin:/usr/proc/bin:$PATH |
Declaring the PMF_TAG and SYSLOG_TAG Variables
All the method scripts (with the exception of Validate) use pmfadm(1M) to launch (or stop) either the data service or the monitor, passing the name of the resource. Each script defines a variable, PMF_TAG that can be passed to pmfadm to identify either the data service or the monitor.
Likewise each method script uses the logger(1) command to log messages with the system log. Each script defines a variable, SYSLOG_TAG that can be passed to logger with the -t option to identify the resource type, resource group, and resource name of the resource for which the message is being logged.
All methods define SYSLOG_TAG in the same way, as shown in the following sample. The dns_probe, dns_svc_start, dns_svc_stop, and dns_monitor_check methods define PMF_TAG as follows (the use of pmfadm and logger is from the dns_svc_stop method):
#########################################################################
# MAIN
#########################################################################
PMF_TAG=$RESOURCE_NAME.named
SYSLOG_TAG=$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME
# Send a SIGTERM signal to the data service and wait for 80% of the
# total timeout value.
pmfadm -s $PMF_TAG.named -w $SMOOTH_TIMEOUT TERM
if [ $? -ne 0 ]; then
logger -p ${SYSLOG_FACILITY}.info \
-t [$SYSLOG_TAG] \
“${ARGV0} Failed to stop HA-DNS with SIGTERM; Retry with \
SIGKILL”
|
The dns_monitor_start, dns_monitor_stop, and dns_update, methods define PMF_TAG as follows (the use of pmfadm is from the dns_monitor_stop method):
######################################################################### # MAIN ######################################################################### PMF_TAG=$RESOURCE_NAME.monitor SYSLOG_TAG=$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME ... # See if the monitor is running, and if so, kill it. if pmfadm -q $PMF_TAG.monitor; then pmfadm -s $PMF_TAG.monitor KILL |
Parsing the Function Arguments
The RGM invokes all of the callback methods—with the exception of Validate—as follows.
method_name -R resource_name -T resource_type_name -G resource_group_name |
The method name is the path name of the program that implements the callback method. A data service specifies the path name for each method in the RTR file. These path names are relative to the directory specified by the Rt_basedir property, also in the RTR file. For example, in the sample data service's RTR file, the base directory and method names are specified as follows.
RT_BASEDIR=/opt/SUNWsample/bin; START = dns_svc_start; STOP = dns_svc_stop; ... |
All callback method arguments are passed as flagged values, with -R indicating the name of the resource instance, -T indicating the type of the resource, and -G indicating the group into which the resource is configured. See the rt_callbacks(1HA) man page for more information on callback methods.
Note –
The Validate method is called with additional arguments (the property values of the resource and resource group on which it is called). See Handling Property Updates for more information.
Each callback method needs a function to parse the arguments it is passed. Because the callbacks are all passed the same arguments, the data service provides a single parse function that is used in all the callbacks in the application.
The following shows the parse_args function used for the callback methods in the sample application.
#########################################################################
# Parse program arguments.
#
function parse_args # [args ...]
{
typeset opt
while getopts 'R:G:T:' opt
do
case "$opt" in
R)
# Name of the DNS resource.
RESOURCE_NAME=$OPTARG
;;
G)
# Name of the resource group in which the resource is
# configured.
RESOURCEGROUP_NAME=$OPTARG
;;
T)
# Name of the resource type.
RESOURCETYPE_NAME=$OPTARG
;;
*)
logger -p ${SYSLOG_FACILITY}.err \
-t [$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME] \
"ERROR: Option $OPTARG unknown"
exit 1
;;
esac
done
}
|
Note –
Although the PROBE method in the sample application is user defined (not a Sun Cluster callback method), it is called with the same arguments as the callback methods. Therefore, this method contains a parse function identical to the one used by the other callback methods.
The parse function is called in MAIN as:
parse_args “$@” |
Generating Error Messages
It is recommended that callback methods use the syslog facility to output error messages to end users. All callback methods in the sample data service use the scha_cluster_get command to retrieve the number of the syslog facility used for the cluster log, as follows:
SYSLOG_FACILITY=`scha_cluster_get -O SYSLOG_FACILITY` |
The value is stored in a shell variable, SYSLOG_FACILITY and can be used as the facility of the logger(1) command to log messages in the cluster log. For example, the Start method in the sample data service retrieves the syslog facility and logs a message that the data service has been started, as follows:
SYSLOG_FACILITY=`scha_cluster_get -O SYSLOG_FACILITY`
...
if [ $? -eq 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} HA-DNS successfully started"
fi
|
See the scha_cluster_get(1HA) man page for more information.
Obtaining Property Information
Most callback methods need to obtain information about resource and resource type properties of the data service. The API provides the scha_resource_get command for this purpose.
Two kinds of resource properties, system-defined properties and extension properties, are available. System-defined properties are predefined whereas you define extension properties in the RTR file.
When you use scha_resource_get to obtain the value of a system-defined property, you specify the name of the property with the -O parameter. The command returns only the value of the property. For example, in the sample data service, the Monitor_start method needs to locate the probe program so it can launch it. The probe program resides in the base directory for the data service, which is pointed to by the RT_BASEDIR property, so the Monitor_start method retrieves the value of RT_BASEDIR, and places it in the RT_BASEDIR variable, as follows.
RT_BASEDIR=`scha_resource_get -O RT_BASEDIR -R $RESOURCE_NAME -G \ $RESOURCEGROUP_NAME` |
For extension properties, you must specify with the -O parameter that it is an extension property and supply the name of the property as the last parameter. For extension properties, the command returns both the type and value of the property. For example, in the sample data service, the probe program retrieves the type and value of the probe_timeout extension property, and then uses awk(1) to put the value only in the PROBE_TIMEOUT shell variable, as follows.
probe_timeout_info=`scha_resource_get -O Extension -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME Probe_timeout`
PROBE_TIMEOUT=`echo $probe_timeout_info | awk '{print $2}'`
|
Controlling the Data Service
A data service must provide a Start or Prenet_start method to activate the application daemon on the cluster, and a Stop or Postnet_stop method to stop the application daemon on the cluster. The sample data service implements a Start and a Stop method. See Deciding on the Start and Stop Methods to Use for information about when you might want to use Prenet_start and Postnet_stop instead.
Start Method
The RGM invokes the Start method on a cluster node when the resource group containing the data service resource is brought online on that node or when the resource group is already online and the resource is enabled. In the sample application, the Start method activates the in.named (DNS) daemon on that node.
This section describes the major pieces of the Start method for the sample application. It does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the Start method, see Start Method Code Listing.
Start Overview
Before attempting to launch DNS, the Start method in the sample data service verifies the configuration directory and configuration file (named.conf) are accessible and available. Information in named.conf is essential to successful operation of DNS.
This callback method uses the process monitor facility (pmfadm) to start the DNS daemon (in.named). If DNS crashes or fails to start, the PMF attempts to start it a prescribed number of times during a specified interval. The number of retries and the interval are specified by properties in the data service's RTR file.
Verifying the Configuration
In order to operate, DNS requires information from the named.conf file in the configuration directory. Therefore, the Start method performs some sanity checks to verify that the directory and file are accessible before attempting to launch DNS.
The Confdir extension property provides the path to the configuration directory. The property itself is defined in the RTR file. However, the cluster administrator specifies the actual location when configuring the data service.
In the sample data service, the Start method retrieves the location of the configuration directory using the scha_resource_get(1HA) command.
Note –
Because Confdir is an extension property, scha_resource_get returns both the type and value. The awk(1) command retrieves just the value and places it in a shell variable, CONFIG_DIR.
# find the value of Confdir set by the cluster administrator at the time of
# adding the resource.
config_info=`scha_resource_get -O Extension -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME Confdir`
# scha_resource_get returns the "type" as well as the "value" for the
# extension properties. Get only the value of the extension property
CONFIG_DIR=`echo $config_info | awk '{print $2}'`
|
The Start method then uses the value of CONFIG_DIR to verify that the directory is accessible. If it is not accessible, Start logs an error message and exits with error status. See Start Exit Status.
# Check if $CONFIG_DIR is accessible.
if [ ! -d $CONFIG_DIR ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Directory $CONFIG_DIR is missing or not mounted"
exit 1
fi
|
Before starting the application daemon, this method performs a final check to verify that the named.conf file is present. If it is not present, Start logs an error message and exits with error status.
# Change to the $CONFIG_DIR directory in case there are relative
# pathnames in the data files.
cd $CONFIG_DIR
# Check that the named.conf file is present in the $CONFIG_DIR directory
if [ ! -s named.conf ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} File $CONFIG_DIR/named.conf is missing or empty"
exit 1
fi
|
Starting the Application
This method uses the process manager facility (pmfadm) to launch the application. The pmfadm command allows you to set the number of times to restart the application during a specified time frame. The RTR file contains two properties, Retry_count, which specifies the number of times to attempt restarting an application, and Retry_interval, which specifies the time period over which to do so.
The Start method retrieves the values of Retry_count and Retry_interval using the scha_resource_get command and stores their values in shell variables. It then passes these values to pmfadm using the -n and -t options.
# Get the value for retry count from the RTR file.
RETRY_CNT=`scha_resource_get -O Retry_Count -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME`
# Get the value for retry interval from the RTR file. This value is in seconds
# and must be converted to minutes for passing to pmfadm. Note that the
# conversion rounds up; for example, 50 seconds rounds up to 1 minute.
((RETRY_INTRVAL=`scha_resource_get -O Retry_Interval -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME` / 60))
# Start the in.named daemon under the control of PMF. Let it crash and restart
# up to $RETRY_COUNT times in a period of $RETRY_INTERVAL; if it crashes
# more often than that, PMF will cease trying to restart it.
# If there is a process already registered under the tag
# <$PMF_TAG>, then PMF sends out an alert message that the
# process is already running.
pmfadm -c $PMF_TAAG -n $RETRY_CNT -t $RETRY_INTRVAL \
/usr/sbin/in.named -c named.conf
# Log a message indicating that HA-DNS has been started.
if [ $? -eq 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} HA-DNS successfully started"
fi
exit 0
|
Start Exit Status
A Start method should not exit with success until the underlying application is actually running and available, particularly if other data services are dependent on it. One way to verify success is to probe the application to verify it is running before exiting the Start method. For a complex application, such as a database, be certain to set the value for the Start_timeout property in the RTR file sufficiently high to allow time for the application to initialize and perform crash recovery.
Note –
Because the application resource, DNS, in the sample data service launches quickly, the sample data service does not poll to verify it is running before exiting with success.
If this method fails to start DNS and exits with failure status, the RGM checks the Failover_mode property, which determines how to react. The sample data service does not explicitly set the Failover_mode property, so this property has the default value NONE (unless the cluster administrator has overridden the default and specified a different value). In this case, the RGM takes no action other than to set the state of the data service. User intervention is required to restart on the same node or fail over to a different node.
Stop Method
The Stop method is invoked on a cluster node when the resource group containing the HA-DNS resource is brought offline on that node or if the resource group is online and the resource is disabled. This method stops the in.named (DNS) daemon on that node.
This section describes the major pieces of the Stop method for the sample application. It does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the Stop method, see Stop Method Code Listing.
Stop Overview
There are two primary considerations when attempting to stop the data service. The first is to provide an orderly shutdown. Sending a SIGTERM signal through pmfadm is the best way to accomplish this.
The second consideration is to ensure that the data service is actually stopped to avoid putting it in Stop_failed state. The best way to accomplish this is to send a SIGKILL signal through pmfadm.
The Stop method in the sample data service takes both these considerations into account. It first sends a SIGTERM signal. If this signal fails to stop the data service, the method sends a SIGKILL signal.
Before attempting to stop DNS, this Stop method verifies that the process is actually running. If the process is running, Stop uses the process monitor facility (pmfadm) to stop it.
This Stop method is guaranteed to be idempotent. Although the RGM should not call a Stop method twice without first starting the data service with a call to its Start method, the could call a Stop method on a resource even though the resource was never started or it died of its own accord. Therefore, this Stop method exits with success even if DNS is not running.
Stopping the Application
The Stop method provides a two-tiered approach to stopping the data service: an orderly or smooth approach using a SIGTERM signal through pmfadm and an abrupt or hard approach using a SIGKILL signal. The Stop method obtains the Stop_timeout value (the amount of time in which the Stop method must return). Stop then allocates 80% of this time to stopping smoothly and 15% to stopping abruptly (5% is reserved), as shown in the following sample.
STOP_TIMEOUT=`scha_resource_get -O STOP_TIMEOUT -R $RESOURCE_NAME \ -G $RESOURCEGROUP_NAMÈ ((SMOOTH_TIMEOUT=$STOP_TIMEOUT * 80/100)) ((HARD_TIMEOUT=$STOP_TIMEOUT * 15/100))
The Stop method uses pmfadm -q to verify that the DNS daemon is running. If it is, Stop first uses pmfadm -s to send a TERM signal to terminate the DNS process. If this signal fails to terminate the process after 80% of the timeout value has expired Stop sends a SIGKILL signal. If this signal also fails to terminate the process within 15% of the timeout value, the method logs an error message and exits with error status.
If pmfadm terminates the process, the method logs a message that the process has stopped and exits with success.
If the DNS process is not running, the method logs a message that it is not running and exits with success anyway. The following code sample shows how Stop uses pmfadm to stop the DNS process.
# See if in.named is running, and if so, kill it.
if pmfadm -q $PMF_TAG; then
# Send a SIGTERM signal to the data service and wait for 80% of
the
# total timeout value.
pmfadm -s $RESOURCE_NAME.named -w $SMOOTH_TIMEOUT TERM
if [ $? -ne 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME] \
“${ARGV0} Failed to stop HA-DNS with SIGTERM; Retry with \
SIGKILL”
# Since the data service did not stop with a SIGTERM signal, use
# SIGKILL now and wait for another 15% of the total timeout value.
pmfadm -s $PMF_TAG -w $HARD_TIMEOUT KILL
if [ $? -ne 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG]
“${ARGV0} Failed to stop HA-DNS; Exiting UNSUCCESFUL”
exit 1
fi
fi
else
# The data service is not running as of now. Log a message and
# exit success.
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
“HA-DNS is not started”
# Even if HA-DNS is not running, exit success to avoid putting
# the data service resource in STOP_FAILED State.
exit 0
fi
# Could successfully stop DNS. Log a message and exit success.
logger -p ${SYSLOG_FACILITY}.err \
-t [$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME]
\
“HA-DNS successfully stopped”
exit 0
|
Stop Exit Status
A Stop method should not exit with success until the underlying application is actually stopped, particularly if other data services have dependencies on it. Failure to do so can result in data corruption.
For a complex application, such as a database, be certain to set the value for the Stop_timeout property in the RTR file sufficiently high to allow time for the application to clean up while stopping.
If this method fails to stop DNS and exits with failure status, the RGM checks the Failover_mode property, which determines how to react. The sample data service does not explicitly set the Failover_mode property, so it has the default value NONE (unless the cluster administrator has overridden the default and specified a different value). In this case, the RGM takes no action other than to set the state of the data service to Stop_failed. User intervention is required to stop the application forcibly and clear the Stop_failed state.
Defining a Fault Monitor
The sample application implements a basic fault monitor to monitor the reliability of the DNS resource (in.named). The fault monitor consists of:
-
dns_probe, a user-defined program that uses nslookup(1M) to verify that the DNS resource controlled by the sample data service is running. If DNS is not running, this method attempts to restart it locally, or depending on the number of restart attempts, requests that the RGM relocate the data service to a different node.
-
dns_monitor_start, a callback method that launches dns_probe. The RGM automatically calls dns_monitor_start after the sample data service is brought online if monitoring is enabled.
-
dns_monitor_stop, a callback method that stops dns_probe. The RGM automatically calls dns_monitor_stop before bringing the sample data service offline.
-
dns_monitor_check, a callback method that calls the Validate method to verify that the configuration directory is available when the PROBE program fails the data service over to a new node.
Probe Program
The dns_probe program implements a continuously running process that verifies the DNS resource controlled by the sample data service is running. The dns_probe is launched by the dns_monitor_start method, which is automatically invoked by the RGM after the sample data service is brought online. The data service is stopped by the dns_monitor_stop method, which the RGM invokes before bringing the sample data service offline.
This section describes the major pieces of the PROBE method for the sample application. It does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the PROBE method, see PROBE Program Code Listing.
Probe Overview
The probe runs in an infinite loop. It uses nslookup(1M) to verify that the proper DNS resource is running. If DNS is running, the probe sleeps for a prescribed interval (set by the Thorough_probe_interval system-defined property) and then checks again. If DNS is not running, this program attempts to restart it locally, or depending on the number of restart attempts, requests that the RGM relocate the data service to a different node.
Obtaining Property Values
This program needs the values of the following properties:
-
Thorough_probe_interval – To set the period during which the probe sleeps
-
Probe_timeout – to enforce the time-out value of the probe on the nslookup command that does the probing
-
Network_resources_used – To obtain the IP address on which DNS is running
-
Retry_count and Retry_interval – To determine the number of restart attempts and the period over which to count them
-
Rt_basedir – To obtain the directory containing the PROBE program and the gettime utility
The scha_resource_get command obtains the values of these properties and stores them in shell variables, as follows.
PROBE_INTERVAL=`scha_resource_get -O THOROUGH_PROBE_INTERVAL \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAME`
probe_timeout_info=`scha_resource_get -O Extension -R $RESOURCE_NAME
\
-G $RESOURCEGROUP_NAME Probe_timeout`
PROBE_TIMEOUT=`echo $probe_timeout_info | awk '{print $2}'`
DNS_HOST=`scha_resource_get -O NETWORK_RESOURCES_USED -R $RESOURCE_NAME
\
-G $RESOURCEGROUP_NAME`
RETRY_COUNT=`scha_resource_get -O RETRY_COUNT -R $RESOURCE_NAME
-G\
$RESOURCEGROUP_NAME`
RETRY_INTERVAL=`scha_resource_get -O RETRY_INTERVAL -R $RESOURCE_NAME
-G\
$RESOURCEGROUP_NAME`
RT_BASEDIR=`scha_resource_get -O RT_BASEDIR -R $RESOURCE_NAME -G\
$RESOURCEGROUP_NAME`
Note –
For system-defined properties, such as Thorough_probe_interval, scha_resource_get returns the value only. For extension properties, such as Probe_timeout, scha_resource_get returns the type and value. Use the awk(1) command to obtain the value only.
Checking the Reliability of the Service
The probe itself is an infinite while loop of nslookup(1M) commands. Before the while loop, a temporary file is set up to hold the nslookup replies. The probefail and retries variables are initialized to 0.
# Set up a temporary file for the nslookup replies. DNSPROBEFILE=/tmp/.$RESOURCE_NAME.probe probefail=0 retries=0 |
-
Sets the sleep interval for the probe
-
Uses hatimerun(1M) to launch nslookup passing the Probe_timeout value and identifying the target host
-
Sets the probefail variable based on the success or failure of the nslookup return code
-
If probefail is set to 1 (failure), verifies that the reply to nslookup came from the sample data service and not some other DNS server
Here is the while loop code.
while :
do
# The interval at which the probe needs to run is specified in the
# property THOROUGH_PROBE_INTERVAL. Therefore, set the probe to sleep
# for a duration of THOROUGH_PROBE_INTERVAL.
sleep $PROBE_INTERVAL
# Run an nslookup command of the IP address on which DNS is serving.
hatimerun -t $PROBE_TIMEOUT /usr/sbin/nslookup $DNS_HOST $DNS_HOST \
> $DNSPROBEFILE 2>&1
retcode=$?
if [ $retcode -ne 0 ]; then
probefail=1
fi
# Make sure that the reply to nslookup comes from the HA-DNS
# server and not from another nameserver mentioned in the
# /etc/resolv.conf file.
if [ $probefail -eq 0 ]; then
# Get the name of the server that replied to the nslookup query.
SERVER=` awk ' $1=="Server:" { print $2 }' \
$DNSPROBEFILE | awk -F. ' { print $1 } ' `
if [ -z "$SERVER" ]; then
probefail=1
else
if [ $SERVER != $DNS_HOST ]; then
probefail=1
fi
fi
fi
|
Evaluating Restart Versus Failover
If the probefail variable is something other than 0 (success), it means the nslookup command timed out or that the reply came from a server other than the sample service's DNS. In either case, the DNS server is not functioning as expected and the fault monitor calls the decide_restart_or_failover function to determine whether to restart the data service locally or request that the RGM relocate the data service to a different node. If the probefail variable is 0, then a message is generated that the probe was successful.
if [ $probefail -ne 0 ]; then
decide_restart_or_failover
else
logger -p ${SYSLOG_FACILITY}.err\
-t [$SYSLOG_TAG]\
"${ARGV0} Probe for resource HA-DNS successful"
fi
|
The decide_restart_or_failover function uses a time window (Retry_interval) and a failure count (Retry_count) to determine whether to restart DNS locally or request that the RGM relocate the data service to a different node. It implements the following conditional code (see the code listing for decide_restart_or_failover in PROBE Program Code Listing).
-
If this is the first failure, restart the data service. Log an error message and bump the counter in the retries variable.
-
If this is not the first failure, but the window has been exceeded, restart the data service. Log an error message, reset the counter, and slide the window.
-
If the time is still within the window and the retry counter has been exceeded, then fail over to another node. If the fail over does not succeed, log an error and exit the probe program with status 1 (failure).
-
If time is still within the window but the retry counter has not been exceeded, restart the data service. Log an error message and bump the counter in the retries variable.
If the number of restarts reaches the limit during the time interval, the function requests that the RGM relocate the data service to a different node. If the number of restarts is under the limit, or the interval has been exceeded so the count begins again, the function attempts to restart DNS on the same node. Note the following about this function:
-
The gettime utility is used to track the time between restarts. This is a C program residing in the (Rt_basedir) directory.
-
The Retry_count and Retry_interval system-defined resource properties determine the number of restart attempts and the interval over which to count. These properties default to 2 attempts in a period of 5 minutes (300 seconds) in the RTR file, though the cluster administrator could change them.
-
The restart_service function is called to attempt to restart the data service on the same node. See the next section, Restarting the Data Service, for information about this function.
-
The scha_control API command, with the GIVEOVER option, brings the resource group containing the sample data service offline and back online on a different node.
Restarting the Data Service
The restart_service function is called by decide_restart_or_failover to attempt to restart the data service on the same node. This function does the following.
-
It determines if the data service is still registered under PMF. If the service is still registered, the function:
-
Obtains the Stop method name and the Stop_timeout value for the data service.
-
Uses hatimerun to launch the Stop method for the data service, passing the Stop_timeout value.
-
(If the data service is successfully stopped) obtains the Start method name and the Start_timeout value for the data service.
-
Uses hatimerun to launch the Start method for the data service, passing the Start_timeout value.
-
-
If the data service is no longer registered under PMF, the implication is that the data service has exceeded the maximum number of allowable retries under PMF, so the scha_control function is called with the GIVEOVER option to fail the data service over to a different node.
function restart_service
{
# To restart the data service, first verify that the
# data service itself is still registered under PMF.
pmfadm -q $PMF_TAG
if [[ $? -eq 0 ]]; then
# Since the TAG for the data service is still registered under
# PMF, first stop the data service and start it back up again.
# Obtain the Stop method name and the STOP_TIMEOUT value for
# this resource.
STOP_TIMEOUT=`scha_resource_get -O STOP_TIMEOUT \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAMÈ
STOP_METHOD=`scha_resource_get -O STOP \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAMÈ
hatimerun -t $STOP_TIMEOUT $RT_BASEDIR/$STOP_METHOD \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAME \
-T $RESOURCETYPE_NAME
if [[ $? -ne 0 ]]; then
logger-p ${SYSLOG_FACILITY}.err -t [$SYSLOG_TAG] \
“${ARGV0} Stop method failed.”
return 1
fi
# Obtain the START method name and the START_TIMEOUT value for
# this resource.
START_TIMEOUT=`scha_resource_get -O START_TIMEOUT \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAMÈ
START_METHOD=`scha_resource_get -O START \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAMÈ
hatimerun -t $START_TIMEOUT $RT_BASEDIR/$START_METHOD \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAME \
-T $RESOURCETYPE_NAME
if [[ $? -ne 0 ]]; then
logger-p ${SYSLOG_FACILITY}.err -t [$SYSLOG_TAG] \
“${ARGV0} Start method failed.”
return 1
fi
else
# The absence of the TAG for the dataservice
# implies that the data service has already
# exceeded the maximum retries allowed under PMF.
# Therefore, do not attempt to restart the
# data service again, but try to failover
# to another node in the cluster.
scha_control -O GIVEOVER -G $RESOURCEGROUP_NAME \
-R $RESOURCE_NAME
fi
return 0
}
|
Probe Exit Status
The sample data service's PROBE program exits with failure if attempts to restart locally have failed and the attempt to fail over to a different node has failed as well. It logs the message, “Failover attempt failed”.
Monitor_start Method
The RGM calls the Monitor_start method to launch the dns_probe method after the sample data service is brought online.
This section describes the major pieces of the Monitor_start method for the sample application. This section does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the Monitor_start method, see Monitor_start Method Code Listing.
Monitor_start Overview
This method uses the process monitor facility (pmfadm) to launch the probe.
Starting the Probe
The Monitor_start method obtains the value of the Rt_basedir property to construct the full path name for the PROBE program. This method launches the probe using the infinite retries option of pmfadm (-n -1, -t -1), which means if the probe fails to start, PMF tries to start it an infinite number of times over an infinite period of time.
# Find where the probe program resides by obtaining the value of the # RT_BASEDIR property of the resource. RT_BASEDIR=`scha_resource_get -O RT_BASEDIR -R $RESOURCE_NAME -G \ $RESOURCEGROUP_NAME` # Start the probe for the data service under PMF. Use the infinite retries # option to start the probe. Pass the resource name, type, and group to the # probe program. pmfadm -c $RESOURCE_NAME.monitor -n -1 -t -1 \ $RT_BASEDIR/dns_probe -R $RESOURCE_NAME -G $RESOURCEGROUP_NAME \ -T $RESOURCETYPE_NAME |
Monitor_stop Method
The RGM calls the Monitor_stop method to stop execution of dns_probe when the sample data service is brought offline.
This section describes the major pieces of the Monitor_stop method for the sample application. This section does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the Monitor_stop method, see Monitor_stop Method Code Listing.
Monitor_stop Overview
This method uses the process monitor facility (pmfadm) to see if the probe is running, and if so, to stop it.
Stopping the Monitor
The Monitor_stop method uses pmfadm -q to see if the probe is running, and if so, uses pmfadm -s to stop it. If the probe is already stopped, the method exits successfully anyway, which guarantees the idempotency of the method.
# See if the monitor is running, and if so, kill it.
if pmfadm -q $PMF_TAG; then
pmfadm -s $PMF_TAG KILL
if [ $? -ne 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Could not stop monitor for resource " \
$RESOURCE_NAME
exit 1
else
# could successfully stop the monitor. Log a message.
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Monitor for resource " $RESOURCE_NAME \
" successfully stopped"
fi
fi
exit 0
|
Caution – Be certain to use the KILL signal with pmfadm to stop the probe and not a maskable signal such as TERM. Otherwise the Monitor_stop method can hang indefinitely and eventually time out. The reason for this problem is that the PROBE method calls scha_control when it is necessary to restart or fail over the data service. When scha_control calls Monitor_stop as part of the process of bringing the data service offline, if Monitor_stop uses a maskable signal, it hangs waiting for scha_control to complete and scha_control hangs waiting for Monitor_stop to complete.
Monitor_stop Exit Status
The Monitor_stop method logs an error message if it cannot stop the PROBE method. The RGM puts the sample data service into MONITOR_FAILED state on the primary node, which can panic the node.
Monitor_stop should not exit before the probe has been stopped.
Monitor_check Method
The RGM calls the Monitor_check method whenever the PROBE method attempts to fail the resource group containing the data service over to a new node.
This section describes the major pieces of the Monitor_check method for the sample application. This section does not describe functionality common to all callback methods, such as the parse_args function and obtaining the syslog facility, which are described in Providing Common Functionality to All Methods.
For the complete listing of the Monitor_check method, see Monitor_check Method Code Listing.
The Monitor_check method must be implemented so that it does not conflict with other methods running concurrently.
The Monitor_check method calls the Validate method to verify that the DNS configuration directory is available on the new node. The Confdir extension property points to the DNS configuration directory. Therefore Monitor_check obtains the path and name for the Validate method and the value of Confdir. It passes this value to Validate, as shown in the following listing.
# Obtain the full path for the Validate method from
# the RT_BASEDIR property of the resource type.
RT_BASEDIR=`scha_resource_get -O RT_BASEDIR -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAMÈ
# Obtain the name of the Validate method for this resource.
VALIDATE_METHOD=`scha_resource_get -O VALIDATE \
-R $RESOURCE_NAME -G $RESOURCEGROUP_NAMÈ
# Obtain the value of the Confdir property in order to start the
# data service. Use the resource name and the resource group entered to
# obtain the Confdir value set at the time of adding the resource.
config_info=`scha_resource_get -O Extension -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME Confdir`
# scha_resource_get returns the type as well as the value for extension
# properties. Use awk to get only the value of the extension property.
CONFIG_DIR=`echo $config_info | awk `{print $2}'`
# Call the validate method so that the dataservice can be failed over
# successfully to the new node.
$RT_BASEDIR/$VALIDATE_METHOD -R $RESOURCE_NAME -G $RESOURCEGROUP_NAME \
-T $RESOURCETYPE_NAME -x Confdir=$CONFIG_DIR
|
See the Validate Method to see how the sample application verifies the suitability of a node for hosting the data service.
Handling Property Updates
The sample data service implements Validate and Update methods to handle updating of properties by a cluster administrator.
Validate Method
The RGM calls the Validate method when a resource is created and when administrative action updates the properties of the resource or its containing group. The RGM calls Validate before the creation or update is applied, and a failure exit code from the method on any node causes the creation or update to be canceled.
The RGM calls Validate only when resource or group properties are changed through administrative action, not when the RGM sets properties, or when a monitor sets the resource properties Status and Status_msg.
Note –
The Monitor_check method also explicitly calls the Validate method whenever the PROBE method attempts to fail the data service over to a new node.
Validate Overview
The RGM calls Validate with additional arguments to those passed to other methods, including the properties and values being updated. Therefore this method in the sample data service must implement a different parse_args function to handle the additional arguments.
The Validate method in the sample data service verifies a single property, the Confdir extension property. This property points to the DNS configuration directory, which is critical to successful operation of DNS.
Note –
Because the configuration directory cannot be changed while DNS is running, the Confdir property is declared in the RTR file as TUNABLE = AT_CREATION. Therefore, the Validate method is never called to verify the Confdir property as the result of an update, but only when the data service resource is being created.
If Confdir is one of the properties the RGM passes to Validate, the parse_args function retrieves and saves its value. Validate then verifies that the directory pointed to by the new value of Confdir is accessible and that the named.conf file exists in that directory and contains some data.
If the parse_args function cannot retrieve the value of Confdir from the command-line arguments passed by the RGM, Validate still attempts to validate the Confdir property. Validate uses scha_resource_get to obtain the value of Confdir from the static configuration. Then it performs the same checks to verify that the configuration directory is accessible and contains a non-empty named.conf file.
If Validate exits with failure, the update or creation of all properties, not just Confdir, fails.
Validate Method Parsing Function
The RGM passes the Validate method a different set of parameters than the other callback methods so Validate requires a different function for parsing arguments than the other methods. See the rt_callbacks(1HA) man page for more information on the parameters passed to Validate and the other callback methods. The following shows the Validate parse_args function.
#########################################################################
# Parse Validate arguments.
#
function parse_args # [args...]
{
typeset opt
while getopts 'cur:x:g:R:T:G:' opt
do
case "$opt" in
R)
# Name of the DNS resource.
RESOURCE_NAME=$OPTARG
;;
G)
# Name of the resource group in which the resource is
# configured.
RESOURCEGROUP_NAME=$OPTARG
;;
T)
# Name of the resource type.
RESOURCETYPE_NAME=$OPTARG
;;
r)
# The method is not accessing any system defined
# properties so this is a no-op
;;
g)
# The method is not accessing any resource group
# properties, so this is a no-op
;;
c)
# Indicates the Validate method is being called while
# creating the resource, so this flag is a no-op.
;;
u)
# Indicates the updating of a property when the
# resource already exists. If the update is to the
# Confdir property then Confdir should appear in the
# command-line arguments. If it does not, the method must
# look for it specifically using scha_resource_get.
UPDATE_PROPERTY=1
;;
x)
# Extension property list. Separate the property and
# value pairs using "=" as the separator.
PROPERTY=`echo $OPTARG | awk -F= '{print $1}'`
VAL=`echo $OPTARG | awk -F= '{print $2}'`
# If the Confdir extension property is found on the
# command line, note its value.
if [ $PROPERTY == "Confdir" ]; then
CONFDIR=$VAL
CONFDIR_FOUND=1
fi
;;
*)
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"ERROR: Option $OPTARG unknown"
exit 1
;;
esac
done
}
|
As with the parse_args function for other methods, this function provides a flag (R) to capture the resource name, (G) to capture the resource group name, and (T) to capture the resource type passed by the RGM.
The r flag (indicating a system-defined property), g flag (indicating a resource group property), and the c flag (indicating that the validation is occurring during creation of the resource) are ignored, because this method is being called to validate an extension property when the resource is being updated.
The u flag sets the value of the UPDATE_PROPERTY shell variable to 1 (TRUE). The x flag captures the names and values of the properties being updated. If Confdir is one of the properties being updated, its value is placed in the CONFDIR shell variable and the variable CONFDIR_FOUND is set to 1 (TRUE).
Validating Confdir
In its MAIN function, Validate first sets the CONFDIR variable to the empty string and UPDATE_PROPERTY and CONFDIR_FOUND to 0.
CONFDIR="" UPDATE_PROPERTY=0 CONFDIR_FOUND=0 |
Validate then calls parse_args to parse the arguments passed by the RGM.
parse_args “$@” |
Validate then checks if Validate is being called as the result of an update of properties and if the Confdir extension property was on the command line. Validate then verifies that the Confdir property has a value, and if not, exits with failure status and an error message.
if ( (( $UPDATE_PROPERTY == 1 )) && (( CONFDIR_FOUND == 0 )) ); then
config_info=`scha_resource_get -O Extension -R $RESOURCE_NAME \
-G $RESOURCEGROUP_NAME Confdir`
CONFDIR=`echo $config_info | awk '{print $2}'`
fi
# Verify that the Confdir property has a value. If not there is a failure
# and exit with status 1
if [[ -z $CONFDIR ]]; then
logger -p ${SYSLOG_FACILITY}.err \
"${ARGV0} Validate method for resource "$RESOURCE_NAME " failed"
exit 1
fi
|
Note –
Specifically, the preceding code checks if Validate is being called as the result of an update ($UPDATE_PROPERTY == 1) and if the property was not found on the command line (CONFDIR_FOUND == 0), in which case it retrieves the existing value of Confdir using scha_resource_get. If Confdir was found on the command line (CONFDIR_FOUND == 1), the value of CONFDIR comes from the parse_args function, not from scha_resource_get.
The Validate method then uses the value of CONFDIR to verify that the directory is accessible. If it is not accessible, Validate logs an error message and exits with error status.
# Check if $CONFDIR is accessible.
if [ ! -d $CONFDIR ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Directory $CONFDIR missing or not mounted"
exit 1
fi
|
Before validating the update of the Confdir property, Validate performs a final check to verify that the named.conf file is present. If it is not, the method logs an error message and exits with error status.
# Check that the named.conf file is present in the Confdir directory
if [ ! -s $CONFDIR/named.conf ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} File $CONFDIR/named.conf is missing or empty"
exit 1
fi
|
If the final check is passed, Validate logs a message indicating success and exits with success status.
# Log a message indicating that the Validate method was successful.
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Validate method for resource "$RESOURCE_NAME \
" completed successfully"
exit 0
|
Validate Exit Status
If Validate exits with success (0) Confdir is created with the new value. If Validate exits with failure (1), Confdir and any other properties are not created and a message indicating why is sent to the cluster administrator.
Update Method
The RGM calls the Update method to notify a running resource that its properties have been changed. The RGM invokes Update after an administrative action succeeds in setting properties of a resource or its group. This method is called on nodes where the resource is online.
Update Overview
The Update method doesn't update properties—that is done by the RGM. Rather, it notifies running processes that an update has occurred. The only process in the sample data service affected by a property update is the fault monitor, so it is this process the Update method stops and restarts.
The Update method must verify the fault monitor is running and then kill it using pmfadm. The method obtains the location of the probe program that implements the fault monitor, then restarts it using pmfadm again.
Stopping the Monitor With Update
The Update method uses pmfadm -q to verify that the monitor is running, and if so kills it with pmfadm -s TERM. If the monitor is successfully terminated, a message to that effect is sent to the administrative user. If the monitor cannot be stopped, Update exits with failure status and sends an error message to the administrative user.
if pmfadm -q $RESOURCE_NAME.monitor; then
# Kill the monitor that is running already
pmfadm -s $PMF_TAG TERM
if [ $? -ne 0 ]; then
logger -p ${SYSLOG_FACILITY}.err \
-t [$SYSLOG_TAG] \
"${ARGV0} Could not stop the monitor"
exit 1
else
# could successfully stop DNS. Log a message.
logger -p ${SYSLOG_FACILITY}.err \
-t [$RESOURCETYPE_NAME,$RESOURCEGROUP_NAME,$RESOURCE_NAME] \
"Monitor for HA-DNS successfully stopped"
fi
|
Restarting the Monitor
To restart the monitor, the Update method must locate the script that implements the probe program. The probe program resides in the base directory for the data service, which is pointed to by the Rt_basedir property. Update retrieves the value of Rt_basedir and stores it in the RT_BASEDIR variable, as follows.
RT_BASEDIR=`scha_resource_get -O RT_BASEDIR -R $RESOURCE_NAME -G \ $RESOURCEGROUP_NAME` |
Update then uses the value of RT_BASEDIR with pmfadm to restart the dns_probe program. If successful, Update exits with success and sends a message to that effect to the administrative user. If pmfadm cannot launch the probe program, Update exits with failure status and logs an error message.
Update Exit Status
Update method failure causes the resource to be put into an “update failed” state. This state has no effect on RGMmanagement of the resource, but indicates the failure of the update action to administration tools through the syslog facility.
- © 2010, Oracle Corporation and/or its affiliates