Cluster Membership Monitor (Sun Cluster Concepts Guide for Solaris OS)

Sun Cluster Concepts Guide for Solaris OS

Cluster Membership Monitor

To ensure that data is kept safe from corruption, all nodes must reach a consistent agreement on the cluster membership. When necessary, the CMM coordinates a cluster reconfiguration of cluster services (applications) in response to a failure.

The CMM receives information about connectivity to other nodes from the cluster transport layer. The CMM uses the cluster interconnect to exchange state information during a reconfiguration.

After detecting a change in cluster membership, the CMM performs a synchronized configuration of the cluster, where cluster resources might be redistributed based on the new membership of the cluster.

Unlike previous Sun Cluster software releases, CMM runs entirely in the kernel.

See Quorum and Quorum Devices for more information on how the cluster protects itself from partitioning into multiple separate clusters.

Failfast Mechanism

If the CMM detects a critical problem with a node, it calls upon the cluster framework to forcibly shut down (panic) the node and to remove it from the cluster membership. The mechanism by which this occurs is called failfast. Failfast will cause a node to shut down in two ways.

If a node leaves the cluster and then attempts to start a new cluster without having quorum, it is “fenced” from accessing the shared disks. See Failure Fencing for details on this use of failfast.
If one or more cluster-specific daemons die (clexecd, rpc.pmfd, rgmd, or rpc.ed) the failure is detected by the CMM and the node panics.

When the death of a cluster daemon causes a node to panic, a message similar to the following will display on the console for that node.

panic[cpu0]/thread=40e60: Failfast: Aborting because "pmfd" died 35 seconds ago.
409b8 cl_runtime:__0FZsc_syslog_msg_log_no_argsPviTCPCcTB+48 (70f900, 30, 70df54, 407acc, 0)
%l0-7: 1006c80 000000a 000000a 10093bc 406d3c80 7110340 0000000 4001 fbf0

After the panic, the node might reboot and attempt to rejoin the cluster or, if the cluster is composed of SPARC based systems, stay at the OpenBoot^TM PROM (OBP) prompt. The action that is taken is determined by the setting of the auto-boot? parameter. You can set auto-boot? with eeprom(1M), at the OpenBoot PROM ok prompt.