Sun GlassFish Enterprise Server 2.1 Troubleshooting Guide

HADB Database Creation Fails

The database creation may fail due to the following reasons:

failed to start database : HADB Database creation failed

To determine the cause of the problem, use the Log Viewer and/or inspect the install_dir/hadb/4/log directory. Some possible errors are:

Problems Related to Shared Memory

Description

This problem may occur due to any of the following reasons:

Cause 1

Shared memory is not configured or the configuration is not working.

Solution 1

Follow the instructions described in the Sun GlassFish Enterprise Server Installation Guide. Remember to reboot the system after configuring shared memory settings.

Cause 2

The physical memory is not enough to satisfy the node requirements. You may see the following error message:

HADB-S-05512: Attaching shared memory segment with key <xx\> failed, 
OS status=12 OS message: Not enough space.

Solution 2

Verify that shared memory is configured and the configuration is working, as mentioned above.

For production systems, reduce the number of nodes on the host or increase the physical memory on the host.

For test/development systems, reduce the shared memory usage by setting the LogBufferSize and DataBufferPoolSize to a value lower than the default values of 48 and 200MB, respectively. The allowed minimum for these variables are 32 and 64MB, respectively.

Cause 3

The size of a shared memory segment has exceeded the allowed maximum size.

HADB-S-05510: Getting shared memory segment with key <xx\> failed, 
OS status=22. OS message: Invalid argument.

Solution 3

Verify that shared memory is configured and the configuration is working, as mentioned above.

If shared memory is configured correctly, check whether you have specified any shared memory segment size (LogBufferSize or DataBufferPoolSize) larger than the system-configured maximum value set in the operating system configuration files (shmsys:shminfo_shmmax in /etc/system on Solaris).

Cause 4

There is already a shared memory segment created with the specified identifier:

HADB-S-05515: Shared memory segment with key <segment_key\> exists already.

Solution 4

List the shared memory segments and check. The ipcs can be used to list the segments in UNIX. Windows uses memory mapped files for shared memory. HADB uses the getTempPath system call to get the system-defined temporary directory where these files, named as f_segmentid, are stored.

Check whether there is already another running database or any other program using the shared memory segment with this identifier. If so, create a database with another port base. If there are no running databases or other programs using this segment, free the segment with hadbm delete unused_database.

Check whether the segments are freed. If they are still there, remove them (use ipcrm in UNIX and delete $TMP/f_* in Windows). The file name consists of the f_ prefix followed by the segment_key translated into hexadecimal. For example, if the error message indicates that segment key 15201 still exists, the temp file would be named f_3B61.

Too Few Semaphores

Description

HADB-E-05521: Operation on semaphore with key "46025" failed, OS status=28 :
No space left on device

This can be caused when the number of semaphores is too low. Since the semaphores are provided as a global resource by the operating system, the configuration depends on all processes running on the host, not only the HADB. This can occur either while starting the HADB, or during runtime.

Solution

Configure the semaphore settings by editing the /etc/system file. Instructions and guidelines are contained in the Configuring Shared Memory and Semaphores section of the Preparing for HADB Setup chapter of the Sun GlassFish Enterprise Server Installation Guide.

Database Nodes Cannot Be Reached and the Database Does Not Function

Solution

The IP addresses of the involved hosts should be fixed. HADB uses the fixed IP addresses present at database creation, so you cannot use dynamic IP addresses (DHCP) for production systems.

The Management Agents Could Not Establish a Domain

Description

The HADB management system is dependent on UDP Multicast messages on multicast address 228.8.8.8. If these messages cannot get through, the createdomain command fails with the following message:

The management agents could not establish a domain, please check that the
hosts can communicate with UDP multicast.

Possible causes include:

Solution 1

If the hosts have several network interfaces on different subnets, the management agent must be configured to use one of the subnets. Set the ma.server.mainternal.interfaces attribute.

Solution 2

Configure the needed network infrastructure to support multicast messages.

hadbm create or hadbm addnodes Command Hangs

Description

Some hosts in the host list given to hadbm create or addnodes have multiple network interfaces, while others have only one, and the hadbm create/addnodes command hangs.

Solution

For the hosts having multiple network interfaces, specify the dotted IP address of the network interface (for example, 129.241.111.23) to be used by hadb when issuing hadbm create/addnodes. If the host name is used instead of IP address, the first interface registered on the host will be used, and there is no guarantee that the nodes will be able to communicate.

ma (Management Agent Process) Crashes

Description

The ma (Management Agent process) crashes for various reasons.

Solution

Display diagnostic information by using hadbm listdomain. Typically, the remedy is to restart the failed agent. If that does not help, restart all agents in turn.