An ordinary, non-cluster-aware application must meet particular requirements to be a candidate for high availability (HA). The section Analyzing the Application for Suitability lists these requirements. This appendix provides additional details about particular items in that list.
An application is made highly available by configuring its resources into resource groups. The application's data is placed on a highly available cluster file system, making the data accessible by a surviving server in the event that one server fails. See information about cluster file systems in the Oracle Solaris Cluster Concepts Guide .
For network access by clients on the network, a logical network IP address is configured in logical host name resources that are contained in the same resource group as the data service resource. The data service resource and the network address resources fail over together, causing network clients of the data service to access the data service resource on its new host.
This appendix covers the following topics:
Binding to INADDR_ANY as Opposed to Binding to Specific IP Addresses
The highly available cluster file systems' devices are multihosted so that when a physical host crashes, one of the surviving hosts can access the device. For an application to be highly available, its data must be highly available. Therefore, the application's data must be located in file systems that can be accessed from multiple cluster nodes. Local file systems that you can make highly available with Oracle Solaris Cluster include UNIX File System (UFS) , Quick File System (QFS), and Oracle Solaris ZFS.
The cluster file system is mounted on device groups that are created as independent entities. You can choose to use some device groups as mounted cluster file systems and others as raw devices for use with a data service, such as HA for Oracle.
An application might have command-line switches or configuration files that point to the location of the data files. If the application uses hard-wired path names, you could change the path names to symbolic links that point to a file in a cluster file system, without changing the application code. See Using Symbolic Links for Multihosted Data Placement for a more detailed discussion about using symbolic links.
In the worst case, the application's source code must be modified to provide a mechanism for pointing to the actual data location. You could implement this mechanism by creating additional command-line arguments.
The Oracle Solaris Cluster software supports the use of UNIX UFS and ZFS file systems and HA raw devices that are configured in a volume manager. When installing and configuring the Oracle Solaris Cluster software, the cluster administrator must specify which disk resources to use for UFS or ZFS file systems and which disk resources to use for raw devices. Typically, raw devices are used only by database servers and multimedia servers.
Occasionally, the path names of an application's data files are hard-wired, with no mechanism for overriding the hard-wired path names. To avoid modifying the application code, you can sometimes use symbolic links.
For example, suppose the application names its data file with the hard-wired path name /etc/mydatafile. You can change that path from a file to a symbolic link that has its value pointing to a file in one of the logical host's file systems. For example, you can make the path a symbolic link to /global/phys-schost-2/mydatafile.
A problem can occur with this use of symbolic links if the application, or one of its administrative procedures, modifies the data file name as well as its contents. For example, suppose that the application performs an update by first creating a new temporary file /etc/mydatafile.new. Then, the application renames the temporary file to have the real file name by using the rename() system call (or the mv command). By creating the temporary file and renaming it to the real file name, the data service is attempting to ensure that its data file contents are always well formed.
Unfortunately, the rename() action destroys the symbolic link. The name /etc/mydatafile is now a regular file and is in the same file system as the /etc directory, not in the cluster's cluster file system. Because the /etc file system is private to each host, the data is not available after a failover or switchover.
The underlying problem is that the existing application is not aware of the symbolic link and was not written to handle symbolic links. To use symbolic links to redirect data access into the logical host's file systems, the application implementation must behave in a way that does not obliterate the symbolic links. So, symbolic links are not a complete remedy for the problem of placing data in the cluster's file systems.
You must determine whether the data service ever needs to know the host name of the server on which it is running. If so, the data service might need to be modified to use a logical host name, rather than the physical host name. In this sense, a logical host name is a host name that is configured into a logical host name resource that is located in the same resource group as the application resource.
Occasionally, in the client-server protocol for a data service, the server returns its own host name to the client as part of the contents of a message to the client. For such protocols, the client could be depending on this returned host name as the host name to use when contacting the server. For the returned host name to be usable after a failover or switchover, the host name should be a logical host name of the resource group, not the name of the physical host. In this case, you must modify the data service code to return the logical host name to the client.
The term multihomed host describes a host that is located on more than one public network. Such a host has multiple host names and IP addresses. It has one host name–IP address pair for each network. Oracle Solaris Cluster is designed to permit a host to appear on any number of networks, including just one (the non-multihomed case). Just as the physical host name has multiple host name–IP address pairs, each resource group can have multiple host name–IP address pairs, one for each public network. When Oracle Solaris Cluster moves a resource group from one physical host to another physical host, the complete set of host name–IP address pairs for that resource group is moved.
The set of host name–IP address pairs for a resource group is configured as logical host name resources contained in the resource group. These network address resources are specified by the cluster administrator when the resource group is created and configured. The Oracle Solaris Cluster Data Service API contains facilities for querying these host name–IP address pairs.
Most off-the-shelf data service daemons that have been written for the Oracle Solaris Operating System already handle multihomed hosts correctly. Many data services do all their network communication by binding to the Oracle Solaris wildcard address INADDR_ANY. This binding automatically causes the data services to handle all the IP addresses for all the network interfaces. INADDR_ANY effectively binds to all IP addresses that are currently configured on the machine. A data service daemon that uses INADDR_ANY generally does not need to be changed to handle the Oracle Solaris Cluster logical network addresses.
Even when non-multihomed hosts are used, the Oracle Solaris Cluster logical network address concept enables the machine to have more than one IP address. The machine has one IP address for its own physical host, and additional IP addresses for each network address (logical host name or shared address) resource that it currently masters. When a machine becomes the master of a network address resource, it dynamically acquires additional IP addresses. When it gives up mastery of a network address resource, it dynamically relinquishes IP addresses.
Some data services cannot work correctly in an Oracle Solaris Cluster environment if they bind to INADDR_ANY. For example, if an application needs to use the logical address as the source for outbound traffic, the deprecated flag set by the network address resource will prevent this if the application calls bind(3SOCKET) with INADDR_ANY as the address. For such an application to work properly in a clustered environment, it must be able to be configured to explicitly bind(3SOCKET) to the logical address, which overrides the deprecated flag when the source address for outbound traffic is selected. The application must implement this feature, because the cluster framework cannot alter application behavior.
When the application resource and the network address resource reside in different resource groups, the Resource_dependencies property lets you ensure a specific set of network address resources are available when the application resource eventually calls bind(3SOCKET). When the application resource declares a dependency upon the network address resource, Oracle Solaris Cluster starts the network address before starting the application, so that the application that is binding to the network address will not fail to find it. The resource dependency also ensures that when the resource group is going offline, the application is stopped before stopping the network address.
Without a resource dependency declared, the order in which the application and network address resources are started or stopped is nondeterministic.
Network address resource types include SUNW.LogicalHostname and SUNW.SharedAddress.
To force the application resource group and the network address resource group to always start on the same node, declare a strong positive affinity with delegated failover (a +++ affinity) of the application resource group upon the network address resource group. For more information, see the RG_affinities section in the rg_properties(5) man page.
If the application resource and the network address resource are configured in the same resource group, then it is usually not necessary to declare an explicit resource dependency. By default there will be an implicit resource dependency of the application upon the network address. For more information, see the Implicit_network_dependencies section in the rg_properties(5) man page.
To a network client, a failover or switchover appears to be a crash of the logical host followed by a fast reboot. Ideally, the client application and the client-server protocol are structured to do some amount of retrying. If the application and protocol already handle the case of a single server crashing and rebooting, they can also handle the case of the resource group being taken over or switched over. Some applications might elect to retry endlessly. More sophisticated applications notify the user that a long retry is in progress and enable the user to choose whether to continue.