Sun Cluster 3.0 5/02 Release Notes

Known Problems

The following known problems affect the operation of the Sun Cluster 3.0 5/02 release. For the most current information, see the online Sun Cluster 3.0 5/02 Release Notes Supplement at http://docs.sun.com.

BugId 4490386

Problem Summary: When using Sun Enterprise 10000 servers in a cluster, panics have been observed in these servers when a certain configuration of I/O cards is used.

Workaround: Do not install UDWIS I/O cards in slot 0 of an SBus I/O board in Sun Enterprise 10000 servers in a cluster.

BugId 4501655

Problem Summary: Record locking does not work on another node(s) when the device trying to be locked is a global device, for example, /dev/global/rdsk/d4s0.

Record locking appears to work well when the program is run multiple times in the background on any particular node. The expected behavior is that after the first copy of the program locks a portion of the device, other copies of the program block waiting for the device to be unlocked. However, when the program is run from a different node, the program succeeds in locking the device again when in fact it should block waiting for the device to be unlocked.

Workaround: There is no workaround.

BugId 4504311

Problem Summary: When a Sun Cluster configuration is upgraded to Solaris 8 10/01 software (required for Sun Cluster 3.0 12/01 upgrade), the Apache application start and stop scripts are restored. If an Apache data service (Sun Cluster HA for Apache) is already present on the cluster and configured in its default configuration (the /etc/apache/httpd.conf file exists and the /etc/rc3.d/S50apache file does not exist), the Apache application starts on its own, independent of the Sun Cluster HA for Apache data service. This prevents the data service from starting because the Apache application is already running.

Workaround: Do the following for each node.

Before you shut down a node to upgrade it, determine whether the following links already exist, and if so, whether the file names contain an uppercase K or S.
/etc/rc0.d/K16apache /etc/rc1.d/K16apache /etc/rc2.d/K16apache /etc/rc3.d/S50apache /etc/rcS.d/K16apache
If these links already exist and contain an uppercase K or S in the file name, no further action is necessary. Otherwise, perform the action in the next step after you upgrade the node to Solaris 8 10/01 software.

After the node is upgraded to Solaris 8 10/01 software, but before you reboot the node, move aside the restored Apache links by renaming the files with a lowercase k or s.

# mv /a/etc/rc0.d/K16apache /a/etc/rc0.d/k16apache
# mv /a/etc/rc1.d/K16apache /a/etc/rc1.d/k16apache
# mv /a/etc/rc2.d/K16apache /a/etc/rc2.d/k16apache
# mv /a/etc/rc3.d/S50apache /a/etc/rc3.d/s50apache
# mv /a/etc/rcS.d/K16apache /a/etc/rcS.d/k16apache

BugId 4511699

Problem Summary: Sun Cluster HA for NFS requires files [SUCCESS=return] for the hosts lookup entry in the /etc/nsswitch.conf file, and requires that all cluster private IP addresses be present in the /etc/inet/hosts file on all cluster nodes.

Otherwise, Sun Cluster HA for NFS will not be able to fail over correctly in the presence of public network failures.

Workaround: Perform the following steps on each node of the cluster.

Modify the hosts entry in the /etc/nsswitch.conf file so that, upon success in resolving a name locally, it returns success immediately and does not contact NIS or DNS.
hosts: cluster files [SUCCESS=return] nis dns

Add entries for all cluster private IP addresses to the /etc/inet/hosts file.

You only need to list the IP addresses plumbed on the physical private interfaces in the /etc/nsswitch.conf and /etc/inet/hosts files. The logical IP addresses are already resolvable through the cluster nsswitch library.

To list the physical private IP addresses, run the following command on any cluster node.

% grep ip_address /etc/cluster/ccr/infrastructure

Each IP address in this list must be assigned a unique hostname that does not conflict with any other hostname in the domain.

Note –

Sun Cluster software already requires that any HA IP addresses (LogicalHostname/SharedAddresses) be present in /etc/inet/hosts on all cluster nodes and that files is listed before nis or dns. The additional requirements mandated by this bug are to list [SUCCESS=return] after files and to list all cluster private IP addresses in the /etc/inet/hosts file.

BugId 4526883

Problem Summary: On rare occasions, private interconnect transport paths ending at a qfe adapter fail to come up.

Workaround: Perform the following steps.

Identify the adapter that is at fault.

Scstat -W output should show all transport paths with that adapter as one of the path endpoints in the “faulted” or the “waiting” states.

Use scsetup(1M) to remove all cables connected to that adapter from the cluster configuration.

Use scsetup again to remove that adapter from the cluster configuration.

Add the adapter and the cables back to the cluster configuration.

Verify whether these steps fixed the problem and whether the paths are able to come back up.

If removing the cables and the adapter and then adding them back does not work, repeat the procedure a few times. If that does not help, reboot the node that has the problem adapter. There is a good chance that the problem will be gone when the node boots up. Before you reboot the node, ensure that the remaining cluster has enough quorum votes to survive the node reboot.

BugId 4620185

Problem Summary: If the rpc.pmfd daemon monitors a process that forks a new process as the result of handling a signal, then using pmfadm -k tag signal might result in an infinite loop. This might occur because pmfadm(1M) attempts to kill all processes in the tag's process tree while the newly forked processes are being added to the tree (each one being added as a result of killing a previous one).

Note –

This bug should not occur with pmfadm -s tag signal.

Workaround: Use pmfadm -s tag signal instead of pmfadm -k. The -s option to pmfadm does not suffer from the same race condition as the -k option.

BugId 4629536

Problem Summary: Using the forcedirectio mount option and the mmap(2) function concurrently might cause data corruption and system hangs or panics.

Workaround: Observe the following restrictions.

Do not remount a file system with the directio mount option added at remount time.
Do not set the directio mount option on a single file by using the directio ioctl.

If there is a need to use directio, mount the whole file system with directio options.

BugId 4634409

Problem Summary: If an attempt is made to mount the same device on different mount points, the system will catch this error most of the time and cause the second mount to fail. However, under certain rare conditions, the system might not be able to catch this error and could allow both mounts to succeed. This happens only when all four of the following conditions hold true.

The two mounts are performed concurrently
The same device is mounted
The device is mounted on two different mount points
One mount is global while the other mount is local.

Workaround: System administrator should exercise caution while mounting file systems on the cluster.

BugId 4638586

Problem Summary: The scconf(1M) command may not reminor the VxVM disk groups in some cases and will give out the error saying that the device is already in use in another device group.

Workaround: Perform the following steps to assign a new minor number to the disk group.

Find the minor numbers already in use.

Observe the minor numbers in use along with the major number listed in the following output.

% ls -l /dev/vx/rdsk/*/*
 
crw-------   1 root     root     210,107000 Mar 11 18:18 /dev/vx/rdsk/fix/vol-01
crw-------   1 root     root     210,88000 Mar 15 16:31 /dev/vx/rdsk/iidg/vol-01
crw-------   1 root     root     210,88001 Mar 15 16:32 /dev/vx/rdsk/iidg/vol-02
crw-------   1 root     root     210,88002 Mar 15 16:33 /dev/vx/rdsk/iidg/vol-03
crw-------   1 root     root     210,88003 Mar 15 16:49 /dev/vx/rdsk/iidg/vol-04
crw-------   1 root     root     210,13000 Mar 18 16:09 /dev/vx/rdsk/sndrdg/vol-01
crw-------   1 root     root     210,13001 Mar 18 16:08 /dev/vx/rdsk/sndrdg/vol-02

Choose any other multiple of 1000 that is not in use as the base minor number for the new disk group.

Assign the unused minor number to the disk group in error.

Use the vxdg command's reminor option.

Retry the failed scconf command.

BugId 4644289

Problem Summary: On Solaris 9, the Sun Cluster HA for Oracle data service's stop method can time out, in case of public network failure, if external name services are not available. The Sun Cluster HA for Oracle data service uses the su(1M) user command to start and stop the database.

Workaround: On each node that can be a primary for the oracle_server or oracle_listener resource, modify the /etc/nsswitch.conf file to include the following entries for passwd, group, publickey and project databases.

passwd:       files
group:        files
publickey:    files
project:      files

These modifications ensure that the su(1M) command does not refer to the NIS/NIS+ name services, and ensures that the data service starts and stops correctly in case of network failure.

BugId 4648767

Problem Summary: Use of sendfile(3EXT) will panic the node.

Workaround: There is no workaround for this problem except not to use sendfile.

BugId 4651392

Problem Summary: On Solaris 9, a cluster node that is being shut down might panic with the following message on its way down.

CMM: Shutdown timer expired. Halting

Workaround: There is no workaround for this problem. The node panic has no other side effects and can be treated as relatively harmless.

BugId 4653151

Problem Summary: Creation of an HAStoragePlus resource fails if the order of the file-system mount points specified in the FilesystemMountPoints extension property is not the same as the order specified in the /etc/vfstab file.

Workaround: Ensure that the mount point list specified in the FilesystemMountPoints extension property matches the sequence specified in the /etc/vfstab file. For example, if the /etc/vfstab file specifies file system entries in the sequence /a, /b, and /c, the FilesystemMountPoints sequence can be “/a,/b,/c” or “/a,/b” or “/a,/c” but not “/a,/c,/b.”

BugId 4653788

Problem Summary: If the Failover_enabled extension property is set to FALSE, this is supposed to prevent the resource monitor from initiating a resource group failover.

However, if the monitor is attempting a resource restart, and the START or STOP method fails or times out, then the monitor will attempt a giveover no matter what is the setting of Failover_enabled.

Workaround: There is no workaround for this bug.

BugId 4655194

Problem Summary: Solstice DiskSuite soft partition-based device groups on locally mounted VxFS can trigger errors if device group switchover commands (scswitch -D device-group) are issued.

Solstice DiskSuite internally performs mirror resync operations which may take a significant amount of time. Mirror resyncs degrade redundancy. VxFS reports errors at this juncture causing fault monitor/application IO failures, resulting in application restarts.

Workaround: For any Solstice DiskSuite device group configured with HAStoragePlus, do not switch over the device group manually. Instead, switch over the resource group, which in turn will cause error-free device switchovers.

Alternately, configure locally mounted VxFS file systems on VxVM disk groups.

BugId 4656367

Problem Summary: Some error messages were not included on the Sun Cluster 3.0 5/02 CD‐ROM.

Workaround: These error messages are documented in New Error Messages.

BugId 4656391

Problem Summary: fsck(1M) of a file system resident on a Sun Cluster global Solstice DiskSuite/VxVM device group fails if executed from a non-primary (secondary) node. This has been observed on Solaris 9, although it is possible that earlier Solaris releases could exhibit this behavior.

Workaround: Invoke the fsck command only on the primary node.

BugId 4656531

Problem Summary: A Sun Cluster HA for Oracle listener resource does not behave correctly if multiple listener resources are configured to start listeners with the same listener name.

Workaround: Do not use the same listener name for multiple listeners running on a cluster.

BugId 4657088

Problem Summary: Dissociating/detaching a plex from a VxVM disk group under Sun Cluster 3.0 might panic the cluster node with the following panic string.

  panic[cpu2]/thread=30002901460: BAD TRAP: type=31 rp=2a101b1d200 addr=40  
  mmu_fsr=0 occurred in module "vxfs" due to a NULL pointer dereference

Workaround: Before you dissociate/detach a plex, unmount the corresponding file system.

BugId 4657833

Problem Summary: Failover does not occur when the resource group property auto_start_on_new_cluster is set to false.

Workaround: Each time the whole cluster reboots, for resource groups that have the auto_start_on_new_cluster property set to false, set the auto_start_on_new_cluster property to true, then reset the auto_start_on_new_cluster property to false.

# scrgadm -c -g rgname -y auto_start_on_new_cluster=true
# scrgadm -c -g rgname -y auto_start_on_new_cluster=false

BugId 4659042

Problem Summary: For globally mounted VxFS file systems, the /etc/mnttab file system might not display the global option.

Workaround: If an /etc/mnttab entry is found on all the nodes of the cluster for the given file system, this shows that the file system is globally mounted.

BugId 4659091

Problem Summary: On remounting a globally mounted file system, /etc/mnttab is not updated.

Workaround: There is no workaround.

BugId 4660479

Problem Summary: When using Sun Cluster HA for NFS with HAStoragePlus, blocking locks are not recovered during failovers and switchovers. As a result, lockd cannot be restarted by Sun Cluster HA for NFS, which leads to failure of the nfs_postnet_stop method, causing the cluster node to crash.

Workaround: Do not use Sun Cluster HA for NFS on HAStoragePlus. Cluster file systems do not suffer from this problem, therefore configuring Sun Cluster HA for NFS on a cluster file system can be used as a workaround.

BugId 4660521

Problem Summary: When an HTTP server is killed on a node, it leaves a PID file on that node. Next time the HTTP server is started, it checks if the PID file exists and checks if any process with the PID is already running (kill -0). Since PIDs are recycled, there could be some other process with the same PID as the last HTTP server PID. This will cause the HTTP server startup to fail.

Workaround: If the HTTP server fails to start with an error like the following, manually remove the PID file for the HTTP server to restart correctly.

Mar 27 17:47:58 ppups4 uxwdog[939]: could not log PID to PidLog 
/app/iws/https-schost-5.example.com/logs/pid, server already running (No such file or directory)

BugId 4662264

Problem Summary: To avoid panics when using VERITAS products such as VxFS with Sun Cluster software, the default thread stack size needs to be increased.

Workaround: Increase the stack size by putting the following lines in the /etc/system file.

set lwp_default_stksize=0x6000
set svc_default_stksize 0x8000

The svc_default_stksize entry is needed for NFS operations.

After installing VERITAS packages, verify that VERITAS has not added similar statements to the /etc/system file. if so they should be resolved to one statement using the higher value.

BugId 4663876

Problem Summary: In a greater-than-two-node device group with an ordered node list, if the node being removed is not the last in the ordered list, then the scconf output will show partial information about the node list.

Workaround:

To prevent the user from hitting this bug, remove the nodes one by one, starting from the last node seen in the node list until the selected node is removed. Then add the other nodes back to the device group.
To repair the cluster state once you've hit this bug, do the following:
1. Stop any services (file systems, data services) using the device group.
2. Unregister (remove) the device group from the cluster's memory by using scsetup(1m). Note: Leave the device-group object itself intact.
3. Reregister the device group (like it was a new device group) using scsetup(1m), adding the correct list of nodes.
When re-adding the removed node back into the device group, explicitly give an ordered list with all valid nodes included.

BugId 4664510

Problem Summary: After powering off one of the Sun StorEdge T3 Arrays then running scshutdown, rebooting both nodes puts the cluster in a non-working state.

Workaround: If half the replicas are lost, perform the following steps:

Ensure that the cluster is in cluster mode.

Forcibly import the diskset.
# metaset -s set-name -f -C take

Delete the broken replicas.

# metadb -s set-name -fd /dev/did/dsk/dNsX

Release the diskset.
# metaset -s set-name -C release
Now the file system can be mounted and used. However, the redundancy in the replicas has not been restored. If the other half of replicas is lost, then there will be no way to restore the mirror to a sane state.

Recreate the databases after the above repair procedure is applied.