test

Sun Cluster Data Service for Network File System (NFS) Guide for Solaris OS

Registering and Configuring Sun Cluster HA for NFS

This procedure describes how to use the scrgadm(1M) command to register and configure Sun Cluster HA for NFS.

Note –

Other options also enable you to register and configure the data service. See “Tools for Data Service Resource Administration” in Sun Cluster Data Services Planning and Administration Guide for Solaris OS for details about these options.

Before you register and configure Sun Cluster HA for NFS, run the following command to verify that the Sun Cluster HA for NFS package, SUNWscnfs, is installed on the cluster.


# pkginfo -l SUNWscnfs

If the package has not been installed, see Installing Sun Cluster HA for NFS Packages for instructions on how to install the package.

How to Register and Configure Sun Cluster HA for NFS

This section describes how to register and configure Sun Cluster HA for NFS.

  1. Become superuser on a cluster member.

  2. Verify that all of the cluster nodes are online.


    # scstat -n

  3. Create the Pathprefix directory.

    The Pathprefix directory exists on the cluster file system that Sun Cluster HA for NFS uses to maintain administrative and status information.

    You can specify any directory for this purpose. However, you must manually create an admin-dir directory for each resource group that you create. For example, create the directory /global/nfs. 


    # mkdir -p /global/admin-dir

  4. Create a failover resource group to contain the NFS resources.


    # scrgadm -a -g resource-group -y Pathprefix=/global/admin-dir [-h nodelist]
    -a

    Specifies that you are adding a new configuration.

    -g resource-group

    Specifies the failover resource group.

    -y Pathprefix=path

    Specifies a directory on a cluster file system that Sun Cluster HA for NFS uses to maintain administrative and status information.

    [-h nodelist]

    Specifies an optional, comma-separated list of physical node names or IDs that identify potential masters. The order here determines the order in which the Resource Group Manager (RGM) considers primary nodes during failover.

  5. Verify that you have added all of your logical hostname resources to the name service database.

    To avoid any failures because of name service lookups, verify that all IP addresses to hostname mappings that are used by Sun Cluster HA for NFS are present in the server's and client's /etc/inet/hosts file. This requirement also applies to any IPMP test IP addresses of logical hostnames that are used by Sun Cluster HA for NFS.

  6. Configure name service mapping in the /etc/nsswitch.conf file on the cluster nodes to first check the local files before trying to access NIS or NIS+ for rpc lookups.

    Doing so prevents timing-related errors for rpc lookups during periods of public network or nameservice unavailability.

  7. Modify the hosts entry in /etc/nsswitch.conf so that upon resolving a name locally the host does not first contact NIS/DNS, but instead immediately returns a successful status.

    Doing this enables HA-NFS to fail over correctly in the presence of public network failures.


    # hosts: cluster files [SUCCESS=return] nis
# rpc: files nis

  8. (Optional) Customize the nfsd or lockd startup options.

    1. To customize nfsd options, on each cluster node open the /etc/init.d/nfs.server file, find the command line starting with /usr/lib/nfs/nfsd, and add any additional arguments desired.

    2. To customize lockd startup options, on each cluster node open the /etc/init.d/nfs.client file, find the command line starting with/usr/lib/nfs/lockd, and add any command line arguments desired.

      Starting with Solaris 9, you can set the lockd grace period with the LOCKD_GRACE_PERIOD parameter in the /etc/default/nfs file. However, if the grace period is set in a command line argument in the /etc/init.d/nfs.client file, this will override the value set in LOCKD_GRACE_PERIOD.

    Note –

    The command lines must remain limited to a single line. Breaking the command line into multiple lines is not supported. The additional arguments must be valid options documented in man pages nfsd(1M) and lockd(1M).

  9. Add the desired logical hostname resources into the failover resource group.

    You must set up a logical hostname resource with this step. The logical hostname that you use with Sun Cluster HA for NFS cannot be a SharedAddress resource type.


    # scrgadm -a -L -g resource-group -l logical-hostname, … [-n netiflist]
    -a

    Specifies that you are adding a new configuration.

    -L -g resource-group

    Specifies the resource group that is to hold the logical hostname resources.

    -l logical-hostname, …

    Specifies the logical hostname resource to be added.

    -n netiflist

    Specifies an optional, comma-separated list that identifies the IP Networking Multipathing groups that are on each node. Each element in netiflist must be in the form of netif@node. netif can be given as an IP Networking Multipathing group name, such as sc_ipmp0. The node can be identified by the node name or node ID, such as sc_ipmp0@1 or sc_ipmp@phys-schost-1.

    Note –

    Sun Cluster does not currently support using the adapter name for netif.

  10. From any cluster node, create a directory structure for the NFS configuration files.

    Create the administrative subdirectory below the directory that the Pathprefix property identifies in Step 4, for example, /global/nfs/SUNW.nfs.


    # mkdir Pathprefix/SUNW.nfs

  11. Create a dfstab.resource file in the SUNW.nfs directory that you created in Step 10, and set up share options.

    1. Create the Pathprefix/SUNW.nfs/dfstab.resource file.

      This file contains a set of share commands with the shared path names. The shared paths should be subdirectories on a cluster file system.

      Note –

      Choose a resource name suffix to identify the NFS resource that you plan to create (in Step 13). A good resource name refers to the task that this resource is expected to perform. For example, a name such as user-nfs-home is a good candidate for an NFS resource that shares user home directories.

    2. Set up the share options for each path that you have created to be shared.

      The format of this file is exactly the same as the format that is used in the /etc/dfs/dfstab file.


      share [-F nfs] [-o] specific_options [-d “description”] pathname
      -F nfs

      Identifies the file system type as nfs.

      -o specific_options

      Grants read-write access to all of the clients. See the share(1M) man page for a list of options. Set the rw option for Sun Cluster.

      -d description

      Describes the file system to add.

      pathname

      Identifies the file system to share.

    When you set up your share options, consider the following points.

    • When constructing share options, do not use the root option, and do not mix the ro and rw options.

    • Do not grant access to the hostnames on the cluster interconnect.

      Grant read and write access to all of the cluster nodes and logical hosts to enable the Sun Cluster HA for NFS monitoring to do a thorough job. However, you can restrict write access to the file system or make the file system entirely read-only. If you do so, Sun Cluster HA for NFS fault monitoring can still perform monitoring without having write access.

    • If you specify a client list in the share command, include all of the physical hostnames and logical hostnames that are associated with the cluster, as well as the hostnames for all of the clients on all of the public networks to which the cluster is connected.

    • If you use net groups in the share command (rather than names of individual hosts), add all of those cluster hostnames to the appropriate net group.

    The share -o rw command grants write access to all of the clients, including the hostnames that the Sun Cluster software uses. This command enables Sun Cluster HA for NFS fault monitoring to operate most efficiently. See the following man pages for details.

    • dfstab(4)

    • share(1M)

    • share_nfs(1M)

  12. Register the NFS resource type.


    # scrgadm -a -t resource-type
    -a -t resource-type

    Adds the specified resource type. For Sun Cluster HA for NFS, the resource type is SUNW.nfs.

  13. Create the NFS resource in the failover resource group.


    # scrgadm -a -j resource -g resource-group -t resource-type
    -a

    Specifies that you are adding a configuration.

    -j resource

    Specifies the name of the resource to add, which you defined in Step 11. This name can be your choice but must be unique within the cluster.

    -g resource-group

    Specifies the name of a previously created resource group to which this resource is to be added.

    -t resource-type

    Specifies the name of the resource type to which this resource belongs. This name must be the name of a registered resource type.

  14. Run the scswitch(1M) command to perform the following tasks.

    • Enable the resource and the resource fault monitor.

    • Manage the resource group.

    • Switch the resource group into the ONLINE state.


    # scswitch -Z -g resource-group

Example – Setting Up and Configuring Sun Cluster HA for NFS

The following example shows how to set up and configure Sun Cluster HA for NFS.


(Create a logical host resource group and specify the path to the administrative 
files used by NFS (Pathprefix).)
# scrgadm -a -g resource-group-1 -y Pathprefix=/global/nfs
 
(Add logical hostname resources into the logical host resource group.)
# scrgadm -a -L -g resource-group-1 -l schost-1
 
(Make the directory structure contain the Sun Cluster HA for NFS configuration 
files.)
# mkdir -p /global/nfs/SUNW.nfs
 
(Create the dfstab.resource file under the nfs/SUNW.nfs directory and set share 
options.)
# share -F nfs -o rw=engineering -d “home dirs” nfs/SUNW.nfs
 
(Register the NFS resource type.)
# scrgadm -a -t SUNW.nfs
 
(Create the NFS resource in the resource group.)
# sc
rgadm -a -j r-nfs -g resource-group-1 -t SUNW.nfs
 
(Enable the resources and their monitors, manage the resource group, and switch 
the resource group into online state.)
# scswitch -Z -g resource-group-1

How to Change Share Options on an NFS File System

If you use the rw, rw=, ro, or ro= options to the share -o command, NFS fault monitoring works best if you grant access to all of the physical hosts or netgroups that are associated with all of the Sun Cluster servers.

If you use netgroups in the share(1M) command, add all of the Sun Cluster hostnames to the appropriate netgroup. Ideally, grant both read access and write access to all of the Sun Cluster hostnames to enable the NFS fault probes to do a complete job.

Note –

Before you change share options, read the share_nfs(1M) man page to understand which combinations of options are legal.

You can also modify shared paths and options dynamically without bringing offline the Sun Cluster HA for NFS resource. See How to Dynamically Update Shared Paths on an NFS File System.

To modify the share options on an NFS file system while the Sun Cluster HA for NFS resource is offline, perform the following steps.

  1. Become superuser on a cluster node.

  2. Turn off fault monitoring on the NFS resource.


    # scswitch -n -M -j resource
    -M

    Disables the resource fault monitor

  3. Test the new share options.

    1. Before you edit the dfstab.resource file with new share options, execute the new share command to verify the validity of your combination of options.


      # share -F nfs [-o] specific_options [-d “description”] pathname
      -F nfs

      Identifies the file system type as NFS.

      -o specific_options

      Specifies an option. You might use rw, which grants read-write access to all of the clients.

      -d description

      Describes the file system to add.

      pathname

      Identifies the file system to share.

    2. If the new share command fails, immediately execute another share command with the old options. When the new command executes successfully, proceed to Step 4.

  4. Edit the dfstab.resource file with the new share options.

    1. To remove a path from the dfstab.resource file, perform the following steps in order.

      1. Execute the unshare(1M) command.

      2. From the dfstab.resource file, delete the share command for the path that you want to remove.


      # unshare [-F nfs] [-o specific_options] pathname
# vi dfstab.resource
      -F nfs

      Identifies the file system type as NFS.

      -o specific_options

      Specifies the options that are specific to NFS file systems.

      pathname

      Identifies the file system that is made unavailable.

    2. To add a path or change an existing path in the dfstab.resource file, verify that the mount point is valid, then edit the dfstab.resource file.

    Note –

    The format of this file is exactly the same as the format that is used in the /etc/dfs/dfstab file. Each line consists of a share command.

  5. Enable fault monitoring on the NFS resource.


    # scswitch -e -M -j resource

How to Dynamically Update Shared Paths on an NFS File System

You can dynamically modify the shared paths on an NFS file system without bringing offline the Sun Cluster HA for NFS resource. The general procedure consists of modifying the dfstab.resource file for Sun Cluster HA for NFS and then manually running the appropriate command, either the share command or the unshare command. The command is immediately effective, and Sun Cluster HA for NFS handles making these paths highly available.

Ensure that the paths that are shared are always available to Sun Cluster HA for NFS during failover so that local paths (on non-HA file systems) are not used.

If paths on a file system that is managed by HAStoragePlus are shared, the HAStoragePlus resource must be in the same resource group as the Sun Cluster HA for NFS resource, and the dependency between them must be set correctly.

  1. Use the scstat -g command to find out the node on which the Sun Cluster HA for NFS resource is online.

  2. On this node run the /usr/sbin/share command to see the list of paths currently shared. Determine the changes you want to make to this list.

  3. To add a new shared path, perform the following steps.

    1. Add the share command to the dfstab.resource file.

      Sun Cluster HA for NFS shares the new path the next time it checks the file. The frequency of these checks is controlled by the Thorough_Probe_Interval property (by default 120 seconds).

    2. Run the share command manually to make the newly added shared path effective immediately. Running the command manually is recommended because the user can be certain that the shared paths are available to potential clients. Sun Cluster HA for NFS detects that the newly added path is already shared and does not complain.

  4. To unshare a path, perform the following steps.

    1. Run the dfmounts(1M) command to ensure that no clients are currently using the path.

      Although a path can be unshared even if clients are using it, these clients would get a stale file error handle and would need special care (forced umount, or even reboot) to recover.

    2. Remove the shared path from the dfstab.resource file.

    3. Run the unshare command manually.

  5. To modify options for an existing shared path, perform the following steps.

    1. Modify the dfstab.resource file as needed.

    2. Run the appropriate command (share or unshare) manually.

How to Tune Sun Cluster HA for NFS Method Timeouts

The time that Sun Cluster HA for NFS methods require to finish depends on the number of paths that the resources share through the dfstab.resource file. The default timeout for these methods is 300 seconds.

As a general guideline, allocate 10 seconds toward the method timeouts for each path that is shared. Default timeouts are designed to handle 30 shared paths.

Update the following method timeouts if the number of shared paths is greater than 30.

Prenet_start_timeoutPostnet_stop_timeoutMonitor_Start_timeout
Start_timeoutValidate_timeoutMonitor_Stop_timeout
Stop_timeoutUpdate_timeoutMonitor_Check_timeout

To change method timeouts, use the scrgadm -c option, as in the following example.


% scrgadm -c -j resource -y Prenet_start_timeout=500

How to Configure SUNW.HAStoragePlus Resource Type

The SUNW.HAStoragePlus resource type was introduced in Sun Cluster 3.0 5/02. This new resource type performs the same functions as SUNW.HAStorage, and synchronizes actions between HA storage and Sun Cluster HA for NFS. SUNW.HAStoragePlus also has an additional feature to make a local file system highly available. Sun Cluster HA for NFS is both failover and disk-intensive, and therefore, you should set up the SUNW.HAStoragePlus resource type.

See the SUNW.HAStoragePlus(5) man page and “Relationship Between Resource Groups and Disk Device Groups” in Sun Cluster Data Services Planning and Administration Guide for Solaris OS for background information. See “Synchronizing the Startups Between Resource Groups and Disk Device Groups” in Sun Cluster Data Services Planning and Administration Guide for Solaris OS for the procedure. (If you are using a Sun Cluster 3.0 version prior to 5/02, you must set up SUNW.HAStorage instead of SUNW.HAStoragePlus. See “Synchronizing the Startups Between Resource Groups and Disk Device Groups” in Sun Cluster Data Services Planning and Administration Guide for Solaris OS for the procedure.)