The hadbm command and its many subcommands and options are provided for administering the high-availability database (HADB). The hadbm command is located in the install_dir/SUNWhadb/4/bin directory.
Refer to the chapter on Configuring the High Availability Database in the Sun GlassFish Enterprise Server Administrator's Guide for a full explanation of this command. Specifics on the various hadbm subcommands are explained in the hadbm man pages.
The following problems are addressed in this section:
The command fails with the error:
The agents <url\> could not be reached.
The hosts in the URL could be unreachable either because the hosts are down, because the communication pathway has not been established, because the port number in the URL is wrong, or because the management agents are down.
Verify that the URL is correct. If the URL is correct, verify that the hosts are up and running and are ready to accept communications; for example:
ping hostname1ping hostname2...
The hadbm command can be run from the current directory, or you can set the search PATH to access the hadb commands from anywhere, which is much more convenient. The error, “hadbm: Command not found,” indicates that neither of these conditions has been met.
cd to the directory that contains the hadbm command and run it from there:
cd install_dir/SUNWhadb/4/bin/ ./hadbm
Use the full path to invoke the hadbm command:
You can use the hadbm command from anywhere by setting the PATH variable. Instructions for setting the PATH variable are contained in the “Preparing for HADB Setup” chapter of the Sun GlassFish Enterprise Server Installation Guide.
To verify that the PATH settings are correct, run the following commands:
which asadmin which hadbm
These commands should echo the paths to the utilities.
The message “hadbm: <path\>: Invalid Java home location” indicates that the JAVA_HOME environment variable has not been set properly.
Instructions for setting the PATH variable are contained in the “Preparing for HADB Setup” chapter of the Sun GlassFish Enterprise Server Installation Guide.
If running the HADB management agent on a host with multiple network interfaces, the createdomain command may fail if not all network interfaces are on the same subnet:
hadbm:Error 22020: The management agents could not establish a domain, please check that the hosts can communicate with UDP multicast.
By default, the management agents use the “first” interface for UDP multicasts (“first” as returned by java.net.NetworkInterface.getNetworkInterfaces()).
The best solution is to tell the management agent which subnet to use by setting ma.server.mainternal.interfaces in the configuration file; for example:
Alternatively, one may configure the router between the subnets to route multicast packets. By default, the management agent uses multicast address 18.104.22.168.
After issuing the hadbm create command, an error similar to the following appears on the console:
./hadbm create ... ... hadbm: Error 22022: Path path does not exist on host host
This error message can also appear when new nodes are added without the specified paths do not exist on the machines.
Log in to the host and create paths for the HADB devices and HADB history files. Run hadbm create and specify the --devicepath and --historypath options to the paths created. Also make sure that the user running the management agent on the host has read and write access to these directories.
HADB executables cannot be installed on different paths on different hosts.
The create or start command fails with the console error message:
hadbm: Error 22095: Database could not be started...
Consider the following possibilities:
Start may fail if the resources (shared memory, disk space) allocated for that node are taken by some other processes, after the node is stopped.
Refer to Problems Related to Shared Memory for suggestions on resolving this issue.
If the problem still persists, inspect the HADB history files. Some of the more likely error messages to look for are:
Could not verify node address
This message occurs when another process is using the port that an HADB server is trying to process. It can occur in several situations:
The portBase is used by another process running on this host machine.
Set the PortBase attribute to another value using the following command:
hadbm set portbase=value
An attempt to stop the HADB node for maintenance failed.
Try again to stop the node with the hadbm command. If that fails, kill the OS process clu_nsup_srv for this node without the -9 option. The nsup process should then stop its hadb child process. If the parent process nsup does not exist, kill all the child processes using kill -9.
The HADB node was stopped for maintenance and an inetd process restarted the HADB node before you intended to start it.
Make sure that inetd does not start the HADB node before stopping it.
hadbm command fails with internal error:"The database could not be started”
Check the following:
Shared memory is all correct on all machines in the HADB configuration.
No other HADB databases are running on the machines, or any other processes that could be using the same port numbers.
All necessary directories exist and have write permissions.
There is enough space in directory where devices are going to be written.
After verifying that none of the above errors have occurred, try the following remedies, in order:
Delete the database and retry.
Delete the database, reboot, and retry.
Delete database, reinstall the HADB software, and retry.
For more information, refer to the Error Message Reference.
The clear command reinitializes the database device files residing on disks. This may fail due to problems with disk or disk access. Check whether any error message from hadbm indicates this. If not, look into the ma.log files and check whether devinit has generated any error messages.
The asadmin create-session-store command could fail for one of these reasons:
This error occurs when the --dbsystempassword supplied to the create-session-store command is not the same password as the one given at the time of database creation.
Try the command again with the correct password.
If you cannot remember the dbsystem password, you need to clear the database using hadbm clear and provide a new dbsystem system password.
The create-session-store produces the error: SessionStoreException: java.sql.SQLException: No suitable driver.
This error can occur when asadmin is not able to find hadbjdbc4.jar from the AS_HADB path defined in asenv.conf in the Enterprise Server config directory.
The solution is to change AS_HADB to point to the location of the HADB installation.
Here is a sample AS_HADB entry from an asenv.conf file:
This error can also occur if you provide the incorrect value for --storeUrl. To solve this problem, obtain the correct URL using hadbm get jdbcURL.
If the management agent with which the hadbm communicates dies before the operation finishes, then the hadbm process may hang. Check whether the all the agents are running.
HADB restart does not work after a double node failure. Additional recovery actions are needed before HADB can be restarted.
Symptoms of a double node failure include:
hadbm status shows that the HADB status is non-operational.
The node status shows that the nodes are in Starting or Recovering state. Even after stopping and then restarting each of the nodes, they remain in the Starting state. Eventually, the node status changes to Stopped.
This problem occurs when mirror HADB host machines have failed or been rebooted, typically after a power outage, or when a machine is rebooted without first stopping the HADB (in a single-machine installation), or when a pair of mirror machines from both Data Redundancy Units (DRUs) are rebooted.
HADB cannot heal itself automatically in such “double failure” situations because the part of the data that resided on the pair nodes is lost. In such cases, the hadbm start command does not succeed, and the hadbm status command shows that HADB is in a non-operational state.
For more information on the DRUs and HADB confutation, see “Administering the High Availability Database” in the Administration Guide, and the Deployment Guide.
If the HADB exhibits strange behavior (for example consistent timeout problems), and you want to check whether a restart cures the problem, use the hadbm restart command.
When the HADB is restarted in this manner, HADB services remain available. Conversely, if HADB is started and stopped in separate operations using hadbm stop and hadbm start, HADB services are unavailable while HADB is stopped.
Verify that the node states show Starting/Recovering, then reset the database. Follow the instructions under “Recovering from Session Data Corruption” in the “Administering the High Availability Database” chapter of the Administration Guide.