Registering to Receive Notifications

To receive notifications, the applications or services can use the cmm_cmc_register function to register a callback with the nhcmmd daemon. When a membership change occurs in the cluster, the nhcmmd daemon notifies the application or service through the callback function by sending a notification. Applications or services receive notifications by polling, using a function such as the poll function. For more information, see the poll(2) man page.

To change a registration, an application or service must first cancel the existing registration by using the cmm_cmc_unregister function, and then register a new notification by using the cmm_cmc_register function.

For an example of how to use these functions, see EXAMPLE 7-2. For further information, see the cmm_cmc_register(3CMM) and cmm_cmc_unregister(3CMM) man pages.

Filtering Notifications

By default, when an application registers to receive notifications, the application receives a notification for every change in the cluster state. These notifications can be filtered by using the cmm_cmc_filter function.

For each notification present in the filter, as selected by using the cmm_cmc_filter function, the registered callback is invoked. The defined filter is applied for further calls to the cmm_notify_dispatch function.

For an example of how to use these functions, see EXAMPLE 7-2. For further information about how to use the cmm_cmc_filter function, see the cmm_cmc_filter(3CMM) man page.

Receiving and Dispatching Notifications

Applications or services that are registered to receive notifications can use the cmm_notify_getfd function. This function returns the file descriptor through which the notifications are delivered.

An application uses a polling function, such as poll, to monitor the file descriptors returned by cmm_notify_getfd. When the polling function indicates activity on this file descriptor, cmm_notify_dispatch must be called. For each pending notification, before invoking the callback, the CMM API checks that this notification is present in the filter (as selected with cmm_cmc_filter).

The callback is invoked with the notification and the client_data argument passed to the cmm_cmc_register function.

For an example of how to use these functions, see EXAMPLE 7-2. For further information see the cmm_cmc_register(3CMM), cmm_notify_getfd(3CMM), cmm_notify_dispatch(3CMM), and poll(2) man pages.

Retrieving Change Notifications

The following example demonstrates how to use the functions that enable you to filter, receive and dispatch notifications:

#include <stdio.h>
#include <poll.h>
#include <cmm/cmm.h>
 
#include "common.h"
 
static void notif_callback(const cmm_cmc_notification_t *notif,
                           void *data)
{
    cmm_member_t member ;
 
        switch (notif->cmchange) {
        case CMM_VICEMASTER_ELECTED:
            printf("Node %u elected vicemaster\n", notif->nodeid);
            if (cmm_vicemaster_getinfo(&member) == CMM_OK)
                print_member(&member) ;
            break ;
        case CMM_VICEMASTER_DEMOTED:
            printf("vicemaster %u demoted\n", notif->nodeid);
            break ;
        default::
            printf("Unexpected notification received\n",
            break ;
        }
}
 
int main(void)
{
    cmm_error_t     result;
    cmm_cmchanges_t wanted_notifs[2] = { CMM_VICEMASTER_ELECTED, 
                                            CMM_VICEMASTER_DEMOTED } ;
    struct pollfd   notif_poll;
    int             poll_res; 
 
    result = cmm_cmc_filter(CMM_CM_NOTIFY_SET, wanted_notifs, 2);
    if (result != CMM_OK) {
        printf("Couldn’t set notification filter: %s\n",
                cmm_strerror(result)) ;
        return 1;
    }
 
    result = cmm_notify_getfd(&notif_poll.fd);
    if (result != CMM_OK) {
        printf("Couldn’t get notification fd: %s\n",
                cmm_strerror(result));
        return 1;
    }
 
    notif_poll.events = POLLIN;
    notif_poll.revents = 0;
 
    fprintf ("Waiting for notifications\n");
 
    while (notif_poll.revents & POLLHUP) == 0) {
 
        poll_res = poll(&notif_poll, 1, -1);
        if (poll_res == 1 && (notif_poll.revents & POLLHUP) == 0) {
            result = cmm_notify_dispatch();
            if (result != CMM_OK) {
                printf("Failed dipatch: %s\n",
                       cmm_strerror(result));
                /* Force leaving the loop */
                notif_poll.revents = POLLHUP
            }
        }
    }
    (void) cmm_cmc_unregister();
    }
    return 0;
}

Removing or Excluding a Node

A node can be removed from the cluster by using the cmm_membership_remove function. The cmm_membership_remove function temporarily takes the current node out of the cluster by giving it the CMM_OUT_OF_CLUSTER role and all other peer nodes learn that it is not an active peer node.

A node with this role is not actually excluded from the cluster. It is still configured to be in the cluster. For an explanation of this role, see Membership Roles.

If you want to exclude a node completely from the cluster, first use the cmm_membership_remove function to remove the node from the cluster. Then remove the entry for this node from the cluster node table. It is better to remove the node completely from the cluster node table instead of attributing the node with an excluded value (X) in the cluster_nodes_table file. For more information, see the cluster_nodes_table(4) man page.

The nhcmmd issues notifications of the node's exclusion. For more information, see Triggering a Failover by Using the cmm_membership_remove Function.

Removing the Master Node

Removing the master node must be done in two stages so as to avoid triggering a failover. First, the node that is master should be released from the role of master by using the cmm_mastership_release function. This triggers a switchover. Only then should the node be removed from the cluster. The node is removed from the cluster by calling the cmm_membership_remove function from the node. This gives the node the CMM_OUT_OF_CLUSTER role, effectively taking the node out of the cluster. For more information about how to trigger a switchover, see Triggering a Switchover. For an explanation of the notifications sent when you trigger a switchover, see Switchover Notifications.

If the cmm_membership_remove function is called directly from the master node, without first triggering a switchover to a qualified vice-master node, then a failover is triggered.

---

Caution - Triggering a failover should be done for test purposes only.

---

For more information about how to trigger a failover, see Triggering a Failover.

For an explanation of the notifications sent when you remove the master role with the cmm_membership_remove function, see TABLE 6-15. For an explanation of the notifications sent when you trigger a failover, see Failover Notifications.

Setting the Qualification of a Node

The cmm_member_setqualif function sets the qualification of a node. Qualification level is only relevant for master-eligible nodes. This function can only be called from the master node. The nodeid and the qualification level must be provided as input parameters to this function. The nodeid of the current node can be retrieved by using the cmm_node_getid function.

The following example demonstrates how to use the cmm_member_setqualif function to disqualify a node with the nodeid 12.

new_qualif = (qualify ? CMM_QUALIFIED_MEMBER : 
              CMM_DISQUALIFIED_MEMBER);
if (cmm_member_setqualif (12, CMM_DISQUALIFIED_MEMBER) != CMM_OK) 
    /* handle the error  */;

Triggering a Switchover

A switchover is usually triggered by the system administrator for maintenance of a node. There are two ways to trigger a switchover:

• By calling the cmm_mastership_release function.

• By using the nhcmmstat tool.

For information about the nhcmmstat tool, see the nhcmmstat(1M) man page.

#include <stdio.h>
#include <cmm/cmm.h>
 
int main(void)
{
    cmm_error_t  result;
 
    result = cmm_mastership_release();
    switch (result) {
    case CMM_OK:
        printf("Switch-over requested\n");
        break;
    case CMM_EPERM:
        printf("Not running on the master node\n";
        break;
    case CMM_ECANCELED:
        printf("No vicemaster in the cluster\n";
        break;
    default:
        printf("Failed to release the mastership: %s\n",
                cmm_strerror(result));
        break;
    }
 
    return (result == CMM_OK ? 0 : 1);
}

Triggering a Failover

A failover can be triggered by removing the master node by using the cmm_membership_release function, or by disqualifying the master node by using the cmm_member_setqualif function.

The sections that follow contain these topics:

• Triggering a Failover by Using the cmm_membership_remove Function.

• Triggering a Failover by Using the cmm_member_setqualif Function.

Caution - Trigger a failover only for test purposes.

#include <stdio.h>
#include <cmm/cmm.h>
 
 
int main(void)
{
    cmm_error_t  result;
    cmm_nodeid_t curr_node;
    cmm_member_t master_info;
 
    result = cmm_node_getid(&curr_node);
    if (result != CMM_OK) {
        printf("Couldn’t get node id: %s\n", cmm_strerror(result));
        return 1;
    }
 
    result = cmm_master_getinfo(&master_info);
    if (result != CMM_OK) {
        printf("Couldn't get node id: %s\n", cmm_strerror(res));
        return 1;
    }
 
    if (master_info.nodeid != curr_node) {
        printf("Not running on the master node\n");
        return 1;
    }
 
    result = cmm_membership_remove(); 
    switch (result) {
    case CMM_OK:
        printf("Failover triggered and node removed from the
                cluster\n”);
        break;
    case CMM_OK:
        printf("Failover triggered and node removed from the
                cluster\n”);
        break;
    default:
        printf("Failed to release the membership: %s\n”),
                cmm_strerror(result));
        break;
    }
 
    return (result == CMM_OK ? 0 : 1);
}

Rejoining the Cluster

When a node has the CMM_OUT_OF_CLUSTER role, two of the possible reasons are:

• It is configured not to join the cluster automatically (CMM.StartUp.Join=False in nhfs.conf(4)).

• It has been removed from the cluster after a call to cmm_membership_remove.

If the node has the CMM_OUT_OF_CLUSTER role for one of these two reasons, then you can rejoin it to the cluster using a call to the function cmm_membership_include. If the node is already in the cluster, the function will succeed.

A successful return of this function does not mean that the node has joined the cluster. It means that the request is being taken into consideration and an attempt will be made to rejoin the node to the cluster. The node could fail to join the cluster if some of the required conditions are not met, such as if the node is excluded or disqualified.

#include <stdio.h>
#include <cmm/cmm.h>
 
 
int main(void)
{
     cmm_member_t member_info;
     cmm_error_t  result;
     cmm_nodeid_t current;
 
     result = cmm_node_getid(&current);
     if (result != CMM_OK) {
          printf("Failed to retrieve the current node’s id\n: %s",
               cmm_strerror(result));
          return 1;
     }
 
     result = cmm_potential_getinfo(current, &member_info);
     switch (result) {
     case CMM_OK:
          break;
     case CMM_ESRCH:
          printf("Current node is NOT configured in the cluster\n");
          break;
     default:
          printf("Couldn’t get member node information: %s\n",
               cmm_strerror(result));
          break;
     }
 
     if (!cmm_member_isoutofcluster(&member_info)) {
          printf("Node is already in the cluster\n");
     } else {
          result = cmm_membership_include();
          switch (result) {
          case CMM_OK:
               printf("Trying to join the cluster\n");
                    cmm_strerror(result));
               break;
          }
     }
 
     return (result == CMM_OK ? 0 : 1);
}