Sun Java System Application Server Enterprise Edition 8.1 2005Q2 High Availability Administration Guide

Chapter 3 Administering High Availability Database

This chapter describes the high availability database (HADB) in the Sun Java System Application Server Enterprise Edition environment. It explains how to configure and administer the HADB. Before you can create and administer the HADB, you must first determine the topology of your systems and install the HADB software on the various machines.

This chapter discusses the following topics:

Using the HADB Management Agent

The management agent, ma, executes management commands on HADB hosts. The management agent also ensures availability of the HADB node supervisor processes by restarting them if they fail.

Management Agent Command Syntax

The syntax of the management agent ma command is:

ma [common-options] 
[ service-options] 
config-file

Where:

Table 3–1 Management Agent Common Options

Option 

Description 

Default 

--define name=value-D

Assign value to property name, where property is one of the properties defined in Configuration File. This option can be repeated multiple times.

None 

--help-? 

Display help information. 

False 

--javahome path-j

Use Java Runtime Environment (1.4 or later) located at path.

None 

--systemroot path-y

Path to the operating system root as normally set in %SystemRoot%. 

None 

--version-V 

Display version information. 

False 

Management Agent Command Syntax describes options for starting the management agent as a Windows service. The -i, -r, and -s options are mutually exclusive; that is, use only one of them at a time.

On Windows, when specifying paths for property values in the configuration file or on the command line, escape file paths containing spaces with double quotes ("). Escape colon (:) drive separators, , and backslash (\) directory separators, with double quotes and a backslash, like this: "\: and "\\.

Table 3–2 Management Agent Service Options (Windows Only)

Option 

Description 

Default 

--install-i 

Install the agent as a Windows service and start the service. Use only one of -i, -r, and -s options. 

False 

--name servicename-n

Use specified name for the service when running multiple agents on a host. 

HADBMgmtAgent 

--remove-r 

Stop the service and delete the agent from the Windows service manger. Use only one of -i, -r, and -s options. 

False 

--service-s 

Run the agent as a Windows service. Use only one of -i, -r, and -s options. 

False 

Customizing Management Agent Configuration

HADB includes a configuration file that you can use to customize the management agent settings. When you start the management agent without specifying a configuration file, it uses default values. If you specify a configuration file, the management agent will use the settings in that file. You can re-use the configuration file on all hosts in a domain.

ProcedureTo customize the management agent configuration on each HADB host

  1. Edit the management agent configuration file and set the values as desired.

  2. Start the management agent, specifying the customized configuration file as the argument.

Configuration File

With Java Enterprise System, all the entries in the configuration file are commented out. No changes are required to use the default configuration. To customize the management agent configuration, remove the comments from the file, change the values as desired, then start the management agent specifying the configuration file as an argument.

The management agent configuration file is installed to:

With the standalone installer, the management agent configuration file is installed to:

The following table describes the settings in the configuration file.

Table 3–3 Configuration File Settings

Setting Name 

Description 

Default 

console.loglevel

Console log level. Valid values are SEVERE, ERROR, WARNING, INFO, FINE, FINER, FINEST 

WARNING 

logfile.loglevel

Log file log level. Valid values are SEVERE, ERROR, WARNING, INFO, FINE, FINER, FINEST 

INFO 

logfile.name

Name and location of log file. Must be a valid path with read/write access. 

Solaris and Linux:/var/opt/SUNWhadb/ma/ma.log

Windows: HADB_install_dir\ma.log

ma.server.type

Client protocol. Only JMXMP is supported. 

jmxmp 

ma.server.
jmxmp.port

Port number for internal (UDP) and external (TCP) communication. Must be a positive integer. Recommended range is 1024-49151. 

1862 

ma.server.
mainternal.interfaces

Interfaces for internal communication for machines with multiple interfaces. Must be a valid IPv4 address mask. All management agents in a domain must use the same subnet 

For example, if a host has two interfaces, 10.10.116.61 and 10.10.124.61, use 10.10.116.0/24 to use the first interface. The number after the slash indicates the number of bits in the subnet mask. 

None 

ma.server.
dbdevicepath

Path to store HADB device information. 

Solaris and Linux: /var/opt/SUNWhadb/4

Windows: HADB_install_dir \device

ma.server.
dbhistorypath

Path to store HADB history files. 

Solaris and Linux: /var/opt/SUNWhadb

Windows: REPLACEDIR (replaced by the actual URL at runtime.) 

ma.server.
dbconfigpath

Path to store node configuration data. 

Solaris and Linux: /var/opt/SUNWhadb/dbdef

Windows: C:\Sun\SUNWhadb\dbdef

repository.dr.path

Path to domain repository files. 

Solaris and Linux: /var/opt/SUNWhadb/repository

Windows: C:\Sun\SUNWhadb\repository

Starting the Management Agent

You can start the management agent two ways:

Starting the Management Agent as a Service

Starting the management agent as a service ensures that it will continue to run until the system shuts down or you explicitly stop it.

Starting the Management Agent as a Service with Java Enterprise System on Solaris or Linux

To start the management agent as a service, use this command:

/etc/init.d/ma-initd start

To stop the service, use this command:

/etc/init.d/ma-initd stop

Starting the Management Agent as a Service with Java Enterprise System on Windows

To start the management agent as a Windows service, use this command: HADB_install_dir\bin\ma -i [config-file ]

The optional argument config-file specifies the management agent configuration file. Use a configuration file only if you want to change the default management agent configuration.

To stop the management agent and remove (deregister) it as a service, use the command: HADB_install_dir\bin\ma -r [config-file ]

To perform administration, choose Administrative Tools | Services, which enables you to start and stop the service, disable automatic startup, and so on.

Starting the Management Agent as a Service with Standalone Application Server on Solaris or Linux

To start the management agent as a service, use this command:

HADB_install_dir/bin/ma-initd start

To stop the service, use this command:

HADB_install_dir/bin/ma-initd stop

Starting the Management Agent as a Service with Standalone Application Server on Windows

To start the management agent as a Windows service, use this command: HADB_install_dir\bin\ma -i [config-file ]

The optional argument config-file specifies the management agent configuration file. Use a configuration file only if you want to change the default management agent configuration.

To stop the management agent and remove (deregister) it as a service, use the command: HADB_install_dir\bin\ma -r [config-file ]

To perform administration, choose Administrative Tools | Services, which enables you to start and stop the service, disable automatic startup, and so on.

Ensuring Automatic Restart of the Management Agent

On Windows platforms, once you have started the management agent as a service, use the Windows administrative tools to set the service Startup type to “Automatic,” and the desired Recovery options.

On Solaris and Linux platforms, use the procedures in this section to ensure the availability of the management agent in case the ma process fails or the operating system reboots. Doing so is appropriate for a production deployment.

The following procedures ensure the management agent starts only when the system enters:

Entering other runlevels stops the management agent.

ProcedureTo Configure automatic restart with Java Enterprise System on Solaris or Linux

Before You Begin

This section assumes you have a basic understanding of operating system initialization and runlevels. For information on these topics, see your operating system documentation.

  1. Ensure that your system has a default runlevel of 3 or 5.

    To check the default runlevel of your system, inspect the file /etc/inittab , and look for a line near the top similar to this:

    id:5:initdefault:

    This example shows a default runlevel of 5.

  2. Create the following softlinks to the file /etc/init.d/ma-initd :


    /etc/rc0.d/K20ma-initd
    /etc/rc1.d/K20ma-initd
    /etc/rc2.d/K20ma-initd
    /etc/rc3.d/S99ma-initd
    /etc/rc5.d/S99ma-initd
    /etc/rcS.d/K20ma-initd
  3. Reboot the machine.

Next Steps

To deactivate automatic start and stop of the agent, remove the links or change the letters K and S in the link names to lowercase.

ProcedureTo Configure automatic restart with Standalone Application Server on Solaris or Linux

  1. In a shell, change your current directory to HADB_install_dir /bin.

  2. Edit the shell script ma-initd.

    Make sure the default values of HADB_ROOT and HADB_MA_CFG in the script to reflect your installation:

  3. Copy ma-initd to the directory /etc/init.d

  4. Create the following soft links to the file /etc/init.d/ma-initd :


    /etc/rc0.d/K20ma-initd
    /etc/rc1.d/K20ma-initd
    /etc/rc2.d/K20ma-initd
    /etc/rc3.d/S99ma-initd
    /etc/rc5.d/S99ma-initd
    /etc/rcS.d/K20ma-initd
Next Steps

To deactivate automatic start and stop of the agent, remove the links or change the letters K and S in the link names to lowercase.

Starting the Management Agent in Console Mode

You may wish to start the management agent manually in console mode for evaluation or testing. Do not start the management agent this way in a production environment, because the ma process will not restart after a system or process failure and will terminate when the command window is closed.

Starting management agent in Console Mode with Java Enterprise System on Solaris or Linux

To start the HADB management agent in console mode, use the command:

opt/SUNWhadb/bin/ma [config-file]

The default management agent configuration file is /etc/opt/SUNWhadb/mgt.cfg

To stop the management agent, kill the process or close the shell window.

Starting management agent in Console Mode with Java Enterprise System on Windows

To start the management agent in console mode, use the command:

HADB_install_dir\bin\ma [config-file]

The optional argument config-file is the name of the management agent configuration file. For more information on the configuration file, see Customizing Management Agent Configuration.

To stop the agent, kill the process.

Starting management agent in Console mode with Standalone Application Server on Windows

To start the management agent in console mode, use the command:

HADB_install_dir\bin\ma [config-file]

The optional argument config-file is the name of the management agent configuration file; for more information, see Customizing Management Agent Configuration

To stop the management agent, kill the process.

Starting management agent in Console Mode with Standalone Application Server on Solaris or Linux

To start the HADB management agent in console mode, use the command:

HADB_install_dir/bin/ma [config-file]

The default management agent configuration file is HADB_install_dir /bin/ma.cfg

To stop the management agent, kill the process or close the shell window.

Using the hadbm Management Command

Use the hadbm command-line utility to manage an HADB domain, its database instances, and nodes. The hadbm utility (also called the management client) sends management requests to the specified management agent, acting as a management server, which has access to the database configuration from the repository.

This section describes the hadbm command-line utility, with the following topics:

Command Syntax

The hadbm utility is located in the HADB_install_dir /bin directory. The general syntax of the hadbm command is:

hadbm subcommand  
[-short-option [option-value]] 
[--long-option [option-value]] 
[operands]

The subcommand identifies the operation or task to perform. Subcommands are case-sensitive. Most commands have one operand (usually dbname), but some have none, and some have two.

Options modify how hadbm performs a subcommand. Options are case-sensitive. Each option has a long form and a short form. Precede the short form with a single dash (-); precede the long forms with two dashes (--). Most options require argument values, except for boolean options, which must be present to switch a feature on. Options are not required for successful execution of the command.

If a subcommand requires a database name, and you do not specify one, hadbm will use the default database, hadb.


Example 3–1 Example of hadbm command

The following illustrates the status subcommand:

hadbm status --nodes

Security Options

For security reasons, all hadbm commands require an administrator password. Use the --adminpassword option to set the password when you create a database or domain. From then on, you must specify that password when you perform operations on the database or domain.

For enhanced security, use the --adminpasswordfile option to specify a file containing the password, instead of entering it on the command line. Define the password in the password file with the following line:

HADBM_ADMINPASSWORD=password

Replace password with the password. Any other content in the file is ignored.

If you specify both the --adminpassword and --adminpasswordfile options, the --adminpassword takes precedence. If a password is required, but is not specified in the command, hadbm prompts you for a password.


Note –

You can change the administrator password, if required, using the command setadminpassword. For more information about the command, see the man page for the command setadminpassword.


In addition to the administrator password, HADB also requires a database password to perform operations that modify the database schema. You must use both passwords when using the following commands: hadbm create, hadbm addnodes, and hadbm refragment.

Specify the database password on the command line with the --dbpassword option. Similar to the administrator password, you can also put the password in a file and use the --dbpasswordfile option, specifying the file location. Set the password in the password file with the following line:

HADBM_DBPASSWORD=password

For testing or evaluation, you can turn off password authentication with the --no-adminauthentication option when you create a database or domain. For more information, see Creating a Database and Creating a Management Domain

The following table summarizes the hadbm security command line options.

Table 3–4 hadbm Security Options

Option (Short Form) 

Description 

--adminpassword=password

-w 

Specifies administrator password for the database or domain. If you use this option when you create a database or domain, then you must provide the password each time you use hadbm to operate on the database or domain. 

Use either this option or --adminpasswordfile, but not both. 

--adminpasswordfile=filepath

-W 

Specifies file that contains the administrator password for the database or domain. If you use this option when you create a database or domain, then you must provide the password each time you use hadbm to operate on the database or domain. 

Use either this option or --adminpassword, but not both. 

--no-adminauthentication 

-U 

Use this option when you create a database or domain to specify that no administrator password is required. For security reasons, do not use this option in a production deployment. 

--dbpassword= password

-p 

Specifies the database password. If you use this option when you create the database, then you must provide the password each time you use an hadbm command to operate on the database. Creates a password for the HADB system user. Must be at least 8 characters. Use either this option or --dbpasswordfile, but not both.

--dbpasswordfile= filepath

-P 

Specifies a file that contains the password for the HADB system user. Use either this option or --dbpassword, but not both.

General Options

General command options can be used with any hadbm subcommand. All are boolean options that are false by default. The following table describes the hadbm general command options.

Table 3–5 hadbm General Options

Option(Short Form) 

Description 

--quiet 

-q 

Execute the subcommand silently without any descriptive messages. 

--help 

-? 

Display a brief description of this command and all the supported subcommands. No subcommand is required. 

--version 

-V 

Display the version details of the hadbm command. No subcommand is required.

--yes 

-y 

Execute the subcommand in non-interactive mode. 

--force 

-f 

Execute the command non-interactively and does not throw an error if the command’s post condition is already achieved. 

--echo 

-e 

Display the subcommand with all the options and their user-defined values or the default values, then executes the subcommand. 

--agent=URL

-m 

URL to the management agents. URL is: hostlist:port, where hostlist is a comma separated list of hostnames or IP-addresses, and port is the port number on which the management agent is operating.

Default is localhost:1862. 

NOTE: This option is not valid with hadbm addnodes.

Environment Variables

For convenience, you can set an environment variable instead of specifying a command option. The following table describes environment variables that correspond to hadbm command options.

Table 3–6 HADB Options and Environment Variables

Long Form 

Short Form 

Default 

Environment Variable 

--adminpassword 

-w 

none 

$HADBM_ADMINPASSWORD 

--agent 

--m 

localhost:1862 

$HADBM_AGENT 

--datadevices 

-a 

1

$HADBM_DATADEVICES 

dbname 

none 

hadb

$HADBM_DB 

--dbpassword 

-p 

none 

$HADBM_DBPASSWORD 

--dbpasswordfile 

-P 

none 

$HADBM_DBPASSWORDFILE 

--devicepath 

-d 

Solaris and Linux: /var/opt/SUNWhadb

Windows: C:\Sun\AppServer\SUNWhadb\vers, where vers is the HADB version number.

$HADBM_DEVICEPATH 

--devicesize 

-z 

none 

$HADBM_DEVICESIZE 

--echo 

-e 

False 

$HADBM_ECHO 

--fast 

-F 

False 

$HADBM_FAST 

--force 

-f 

False 

$HADBM_FORCE 

--help 

-? 

False 

$HADBM_HELP 

--historypath 

-t 

Solaris and Linux: /var/opt/SUNWhadb

Windows: REPLACEDIR, replaced by the actual URL at runtime. 

$HADBM_HISTORYPATH 

--hosts 

-H 

none 

$HADBM_HOSTS 

--interactive 

-i 

True 

$HADBM_INTERACTIVE 

--no-refragment 

-r 

False 

$HADBM_NOREFRAGMENT 

--portbase 

-b 

15200 

$HADBM_PORTBASE 

--quiet 

-q 

False 

$HADBM_QUIET 

--repair 

-R 

True 

$HADBM_REPAIR 

--rolling 

-g 

True 

$HADBM_ROLLING 

--saveto 

-o 

none 

$HADBM_SAVETO 

--set 

-S 

none 

$HADBM_SET 

--spares 

-s 

$HADBM_SPARES 

--startlevel 

-l 

normal 

$HADBM_STARTLEVEL 

--version 

-V 

False 

$HADBM_VERSION 

--yes 

-y 

False 

$HADBM_YES 

Configuring HADB

This section describes the following basic HADB configuration tasks:

Creating a Management Domain

The command hadbm createdomain creates a management domain containing the specified HADB hosts. The command initializes internal communication channels between hosts and the persistence configuration store.

The syntax of the command is:

hadbm createdomain
 [--adminpassword=password |--adminpasswordfile=
file | --no-adminauthentication] [--agent=maurl]
 hostlist

The hostlist operand is a comma-separated list of HADB hosts, each of which is a valid IPv4 network address. Include all the hosts that you want to be in the new domain in the hostlist.

See General Options for a description of the command options.

Before using this command, be sure an HADB management agent is running on every host in the hostlist. Additionally, the management agents must:

After hadbm creates the management domain, it enables all the hosts in the domain. Then the management agents are ready to manage databases. After creating HADB domains, the next step is to create the HADB database. For more information on creating HADB databases, see Creating a Database .


Example 3–2 Creating an HADB Management Domain

The following example creates a management domain on the four specified hosts:

hadbm createdomain --adminpassword= password host1,host2,host3,host4

After hadbm successfully executes the command, you will see the message:

Domain host1,host2,host3, host4 created.

After creating HADB domains, register the path and version of the HADB packages with the management agents.


Creating a Database

Use the hadbm create command to create a database manually.

Before you use this command to create a database, create the management domain and register the HADB package. If you have not performed these two steps when you run hadbm create , it implicitly performs them. Although this might seem like less work, failures in any of the commands can make debugging difficult. Besides, hadbm create is not atomic, that is, if one of the implicit commands fails, the commands that executed successfully will not be rolled back. Therefore, it is best to create the database only after creating the domain and registering the HADB package.

For example, if hadbm createdomain and hadbm registerpackage execute successfully but hadbm create database fails, the changes made by hadbm createdomain and hadbm registerpackage will persist.

ProcedureTo create a database

  1. Create the management domain.

    For more information, see Creating a Management Domain

  2. Register the HADB package.

    For more information, see Registering HADB Packages for more information.

  3. Use the hadbm create command to create the database.

    For information on command syntax, see the following section.

hadbm create Command Syntax

hadbm create [--package=name] [--packagepath=path] [--historypath=path] [--devicepath=path] [--datadevices=number ] [--portbase=number] [--spares=number] [--set=attr-val-list] [--agent=maurl] [--no-cleanup] [ --no-clear ] [ --devicesize =size] [--dbpassword=password | --dbpasswordfile=file ] --hosts=host list [--adminpassword=password | --adminpasswordfile=file | --no-adminauthentication ] [dbname]

The dbname operand specifies the database name, which must be unique. To make sure the database name is unique, use the hadbm list command to list existing database names. Use the default database name unless you need to create multiple databases. For example, to create multiple clusters with independent databases on the same set of HADB machines, use a separate database name for each cluster.

The hadbm create command writes error messages to the console, not log files.

Table 3–7 describes the special hadbm create command options. See General Options for a description of additional command options.

Table 3–7 hadbm create Options

Option(Short Form) 

Description 

Default 

--datadevices= number

-a 

Number of data devices on each node, between one and eight inclusive. Data devices are numbered starting at 0. 

--devicepath= path

-d 

Path to the devices. There are four devices: 

  • DataDevice

  • NiLogDevice (node internal log device)

  • RelalgDevice (relational algebra query device)

  • NoManDevice (node manager device).

    This path must exist and be writable. To set this path differently for each node or each device, see Setting Heterogeneous Device Paths

Solaris and Linux: /var/opt/SUNWhadb

Windows: C:\Sun\AppServer\SUNWhadb\vers, where vers is the HADB version number.

Default is specified by ma.server.dbdevicepath in management agent configuration file. For more details, see Configuration File

--devicesize= size

-z 

Device size for each node. For more information, see Specifying Device Size.

Increase the device size as described in Adding Storage Space to Existing Nodes

1024MB 

Maximum size is lesser of maximum operating system file size or 256 GB. Minimum size is: 

(4 x LogbufferSize + 16MB) / n

Where n is the number of data devices given by the option --datadevices.

--historypath= path

-t 

Path to the history files. This path must already exist and be writable. 

For more information on history files, see Clearing and Archiving History Files

Default is specified by ma.server.dbhistorypath in management agent configuration file. For details, see Configuration File

Solaris and Linux:/var/opt/SUNWhadb

On Windows: REPLACEDIR (replaced by the actual URL at runtime.) 

--hosts= hostlist

-H 

Comma-separated list of host names or IP addresses (IPv4 only) for the nodes in the database. Use IP addresses to avoid dependence on DNS lookups. Host names must be absolute. You cannot use localhost or 127.0.0.1 as a host name.

See Specifying Hosts for more information.

None 

--package=name -k

Name of the HADB package (version). If the package is not found, a default package is registered. 

This option is deprecated. Use the hadbm registerpackage command to register a package in the domain. 

None 

--packagepath=path-L

Path to the HADB software package. Use only if the package is not registered in the domain. 

This option is deprecated. Use the hadbm registerpackage command to register a package in the domain. 

None 

--portbase= number

-b 

Port base number used for node 0. Successive nodes are automatically assigned port base numbers in steps of 20 from this number. Each node uses its port base number and the next five consecutively numbered ports. 

To run several databases on the same machine, have a plan for allocating port numbers explicitly. 

15200 

--spares= number

-s 

Number of spare nodes. This number must be even and must be less than the number of nodes specified in the --hosts option.

--set=attr-val-list

-S 

Comma-separated list of database configuration attributes in name =value format. For explanations of database configuration attributes, see Clearing and Archiving History Files

None 


Example 3–3 Example of creating a database

The following command is an example of creating a database:

hadbm create --spares 2 --devicesize 1024 --dbpassword secret123
 --hosts n0,n1,n2,n3,n4,n5

Specifying Hosts

Use the --hosts option to specify a comma-separated list of host names or IP addresses for the nodes in the database. The hadbm create command creates one node for each host name (or IP address) in the list. The number of nodes must be even. Use duplicate host names to create multiple nodes on the same machine with different port numbers. Make sure that nodes on the same machine are not mirror nodes.

Nodes are numbered starting at zero in the order listed in this option. The first mirrored pair are nodes zero (0) and one (1), the second two (2) and three (3), and so on. Odd numbered nodes are in one DRU, even numbered nodes in the other. With --spares option, spare nodes are those with the highest numbers.

For information about configuring double network interfaces, see Configuring Network Redundancy

Specifying Device Size

Specify the device size using the --devicesize option. The recommended device size is:

(4x / nd + 4l/d) / 0.99 

Where

Setting Heterogeneous Device Paths

To set a different device path for each node or service, use the -- set option of hadbm create. There are four types of devices: the DataDevice, the NiLogDevice (node internal log device), the RelalgDevice (relational algebra query device), and the NoManDevice (node manager device). The syntax for each name =value pair is as follows, where -devno is required only if the device is DataDevice:

node-nodeno.device-devno.Devicepath

For example:

--set Node-0.DataDevice-0.DevicePath=/disk0,
Node-1.DataDevice-0.DevicePath=/disk 1

You can also set a heterogeneous path to history files, as follows:

node-nodeno.historypath=path

For information on history files, see Clearing and Archiving History Files

Any device path that is not set for a particular node or device defaults to the --devicepath value.


Note –

Change device paths and location of history files using hadbm set and hadbm addnodes commands.


Troubleshooting

If you have difficulty creating a database, check the following:

Viewing and Modifying Configuration Attributes

You can view and modify database configuration attributes with the hadbm get and hadbm set commands, respectively.

Getting the Values of Configuration Attributes

To get the values of configuration attributes, use the hadbm get command. For a list of valid attributes, see Configuration Attributes. The command syntax is:

hadbm get attribute-list | --all  
[dbname]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]

The dbname operand specifies the database name. The default is hadb.

The attribute-list operand is a comma-separated or quote-enclosed space-separated list of attributes. The --all option displays values for all attributes. For a list of all attributes for hadbm get, see Configuration Attributes.

See General Options for a description of command options.


Example 3–4 Example of using hadbm get

hadbm get JdbcUrl,NumberOfSessions

Setting the Values of Configuration Attributes

To set the values of configuration attributes, use the hadbm set command. For a list of valid attributes, see Configuration Attributes

hadbm set [dbname] attribute
=value[,attribute=
value...]
 [--adminpassword=password | --adminpasswordfile=file]
 [--agent=maurl]

The dbname operand specifies the database name. The default is hadb.

The attribute=value list is a comma-separated or quote-enclosed space-separated list of attributes.

See General Options for a description of command options.

If this command executes successfully, it restarts the database in the state it was in previously, or in a better state. For information about database states, see Getting the Status of HADB restart the HADB as described in Restarting a Database.

You cannot set the following attributes with hadbm set. Instead, set them when you create a database (see Creating a Database).


Note –

Using hadbm set to set any configuration attribute, except ConnectionTrace or SQLTraceMode, causes a rolling restart of HADB. In a rolling restart, each node is stopped, and started with the new configuration, one at a time; HADB services are not interrupted.

If you set ConnectionTrace or SQLTraceMode, no rolling restart occurs, but the change only takes effect for new HADB connections made from an Application Server instance.


Configuration Attributes

The following table lists the configuration attributes that you can modify with hadbm set and retrieve with hadbm get.

Table 3–8 Configuration Attributes

Attribute 

Description 

Default 

Range 

ConnectionTrace

If true, records a message in the HADB history files when a client connection (JDBC, ODBC) is initiated or terminated. 

False 

True or False 

CoreFile

Do not change the default value. 

False 

True or False 

DatabaseName

Name of the database. 

hadb 

 

DataBufferPoolSize

Size of the data buffer pool allocated in shared memory. 

200MB 

16 - 2047 MB 

DataDeviceSize

Specifies the device size for the node. For information on the recommended DataDeviceSize, see Specifying Device Size

The maximum value is the smaller of 256GB or the maximum operating system file size. The minimum value is: 

(4 x LogbufferSize + 16MB) / n

where n is number of data devices.

1024MB 

32 - 262144 MB 

PackageName 

Name of HADB software package used by the database. 

V4.x.x.x 

None 

DevicePath

Location of the devices. Devices are: 

  • Data device (DataDevice)

  • Node internal log device (NiLogDevice)

  • Relational algebra query device (RelalgDevice)

Solaris and Linux: /var/opt/SUNWhadb

Windows: C:\Sun\AppServer\SUNWhadb\vers, where vers is the HADB version number.

 

EagerSessionThreshold

Determines whether normal or eager idle session expiration is used. 

In normal idle session expiration, sessions that are idle for more than SessionTimeout seconds are expired.

When the number of concurrent sessions exceeds the EagerSessionThreshold percentage of the maximum number of sessions, sessions that are idle for more than EagerSessionTimeout seconds are expired.

Half of NumberOfSessions attribute 

0 - 100 

EagerSessionTimeout

The time in seconds a database connection can be idle before it expires when eager session expiration is used. 

120 seconds 

0-2147483647 seconds 

EventBufferSize

Size of the event buffer, where database events are logged. If set to 0, no event buffer logging is performed.

During failures, the event buffer is dumped. This gives valuable information on the cause of the failures and is useful during trial deployment. 

Writing events to memory has a performance penalty. 

0 MB

0-2097152 MB 

HistoryPath

Location of the HADB history files, which contain information, warnings, and error messages. 

This is a read-only attribute. 

Solaris and Linux: /var/opt/SUNWhadb

Windows: REPLACEDIR (replaced by the actual URL at runtime.) 

 

InternalLogbufferSize

Size of the node internal log device, which keeps track of operations related to storing data. 

12MB 

4 - 128 MB 

JdbcUrl

The JDBC connection URL for the database. 

This is a read-only attribute. 

none 

 

LogbufferSize

Size of the log buffer, which keeps track of operations related to data. 

48MB 

4 - 2048 MB 

MaxTables

Maximum number of tables allowed in an HADB database. 

1100 

100 - 1100 

NumberOfDatadevices

Number of data devices used by an HADB node. 

This is a read-only attribute. 

1 - 8 

NumberOfLocks

Number of locks allocated by an HADB node. 

50000 

20000-1073741824 

NumberOfSessions

Maximum number of sessions (database connections) that can be opened for an HADB node. 

100 

1 - 10000 

PortBase

Base port number used to create different port numbers for different HADB processes. 

This is a read-only attribute. 

15200 

10000 - 63000 

RelalgDeviceSize

Size of the device used in relational algebra queries. 

128 MB 

32 - 262144 MB 

SessionTimeout

Amount of time a database connection can be idle before it expires when normal session expiration is used. 

1800 seconds 

0-2147483647 seconds 

SQLTraceMode

Amount of information about executed SQL queries written to the history files. 

If SHORT, login and logout of SQL sessions are recorded. If FULL, all SQL queries being prepared and executed, including parameter values, are recorded.

NONE 

NONE/SHORT/ FULL 

StartRepairDelay

Maximum time a spare node allows for a failed active node to perform a node recovery. If the failed node cannot recover within this time interval, the spare node starts copying data from the failed node’s mirror and becomes active. Changing the default value is not recommended. 

20 seconds 

0 - 100000 seconds 

StatInterval

Interval at which an HADB node writes throughput and response time statistics to its history file. To disable, set to 0. 

Here is an example of a statistics line: 

Req-reply time: # 123, min= 69 avg= 1160 max= 9311 %=100.0

The number after the has sign (#) is the number of requests serviced over the StatInterval. The next three numbers are the minimum, average, and maximum time in microseconds taken by transactions completed over the StatInterval. The number afer the percent sign ( %) is the number of transactions completed successfully within 15 milliseconds over the StatInterval.

600 seconds 

0 - 600 seconds 

SyslogFacility

Facility used when reporting to syslog. The syslog daemon should be configured (see man syslogd.conf for details).

Use a facility that is not used by other applications running on the same machine. 

Set to none to disable syslog logging.

local0 

local0, local1, local2, local3, local4, local5, local6, local7, kern, user, mail, daemon, auth, syslog, lpr, news, uucp, cron, none 

SysLogging

If true, an HADB node writes information to the operating system’s syslog files.

True 

True or False 

SysLogLevel

Minimum level of HADB message saved to operating system’s syslog files. All messages of that level or higher will be logged. For example, “info” logs all messages.

warning 

nonealert errorwarninginfo 

SyslogPrefix

Text string inserted before all syslog messages written by the HADB.

hadb -dbname 

 

TakeoverTime

Time between when a node fails and when its mirror takes over. Do not change the default value. 

10000 (milliseconds) 

500 - 16000 milliseconds 

Configuring the JDBC Connection Pool

Application Server communicates with HADB using the Java Database Connectivity (JDBC) API. The asadmin configure-ha-cluster command automatically creates a JDBC connection pool for use with HADB (for a cluster cluster-name ). The name of the connection pool is cluster-name-hadb-pool. The JNDI URL of JDBC resource is jdbc/cluster-name-hastore.

The initial configuration of the connection pool is normally sufficient. When you add a node, change the steady pool size so that there are eight connections for each HADB active node. See Adding Nodes.

This chapter covers the following topics:

Getting the JDBC URL

Before you can set up the JDBC connection pool, you need to determine the JDBC URL of HADB using the hadbm get command as follows:

hadbm get JdbcUrl [dbname]

For example:

hadbm get JdbcUrl

This command displays the JDBC URL, which is of he following form:

jdbc:sun:hadb:host:port,
host:port,...

Remove the jdbc:sun:hadb: prefix and use the host:port, host:port... part as the value of the serverList connection pool property, described in Table 3–10.

Creating a Connection Pool

The following table summarizes connection pool settings required for the HADB. Change the Steady Pool Size when adding nodes, but do not change other settings.

Table 3–9 HADB Connection Pool Settings

Setting 

Required Value for HADB 

Name

The HADB JDBC resource’s Pool Name setting must refer to this name 

Database Vendor

HADB 4.4 

Global Transaction Support

Unchecked/false 

DataSource Classname

com.sun.hadb.jdbc.ds.HadbDataSource

Steady Pool Size

Use 8 connections for each active HADB node. For more detailed information, see the System Deployment Guide.

Connection Validation Required

Checked/true 

Validation Method

meta-data

Table Name

Do not specify 

Fail All Connections

Unchecked/false 

Transaction Isolation

repeatable-read

Guarantee Isolation Level

Checked/true 

The following table summarizes connection pool properties required for the HADB. Change serverList when adding nodes, but do not change other properties.

Table 3–10 HADB Connection Pool Properties

Property 

Description 

username

Name of the storeuser to use in the asadmin create-session-store command.

password

Password (storepassword ) to us in the asadmin create-session-store command.

serverList

JDBC URL of the HADB. To determine this value, see Getting the JDBC URL

You must change this value if you add nodes to the database. See Adding Nodes.

cacheDatabaseMetaData

When false , as required, ensures that calls to Connection.getMetaData() make calls to the database, which ensures that the connection is valid.

eliminateRedundantEndTransaction

When true , as required, improves performance by eliminating redundant commit and rollback requests and ignoring these requests if no transaction is open.

maxStatement

Maximum number of statements per open connection that are cached in the driver statement pool. Set this property to 20.


Example 3–5 Creating a Connection Pool

Here is an example asadmin create-jdbc-connection-pool command that creates an HADB JDBC connection pool:

asadmin create-jdbc-connection-pool
--user adminname --password secret 
--datasourceclassname com.sun.hadb.jdbc.ds.HadbDataSource
--steadypoolsize=32
--isolationlevel=repeatable-read
--isconnectvalidatereq=true
--validationmethod=meta-data
--property username=storename:password=secret456:serverList=
host\:port,host\:port,
host\\:port,host\:port,
host\:port,host\:port
:cacheDatabaseMetaData=false:eliminateRedundantEndTransaction=true hadbpool

On Solaris, escape colon characters (:) within property values with double backslashes (\\). On Windows, escape colon characters (:) with single backslashes ( \).


Creating a JDBC Resource

The following table summarizes JDBC resource settings required for HADB.

Table 3–11 HADB JDBC Resource Settings

Setting 

Description 

JNDI Name

The following JNDI name is the default in the session persistence configuration: jdbc/hastore. You can use the default name or a different name.

You must also specify this JNDI name as the value of the store-pool-jndi-name Persistence Store property when you activate the availability service.

Pool Name

Select from the list the name (or ID) of the HADB connection pool used by this JDBC resource. For more information, see Configuring Network Redundancy

Data Source Enabled

Checked/true 

Managing HADB

You generally need to perform management operations when you replace or upgrade your network, hardware, operating system, or HADB software. The following sections explain various management operations:

Managing Domains

You can perform the following operations on an HADB domain:

See Security Options and General Options for a description of command options.

Extending a Domain

Use extenddomain to add hosts to an existing management domain. The command syntax is:

hadbm extenddomain  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
hostlist

IP addresses of HADB hosts must be IPv4 addresses.

For more information, see hadbm-extenddomain(1).

Deleting a Domain

Use deletedomain to remove a management domain. The command syntax is:

hadbm deletedomain  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]

For more information, see hadbm-deletedomain(1).

Removing Hosts from a Domain

Use reducedomain to remove hosts from the management domain. The command syntax is:

hadbm reducedomain  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
host_list

For more information, see hadbm-reducedomain(1).

Listing Hosts in a Domain

Use listdomain to list all hosts defined in the management domain. The command syntax is:

hadbm listdomain  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]

For more information, see hadbm-listdomain(1).

Managing Nodes

You can perform the following operations on individual nodes:

Starting a Node

You might need to manually start an HADB node that was stopped because its host was taken off-line for a hardware or software upgrade or replacement. Also, you might need to manually start a node if it fails to restart for some reason (other than a double failure). For more information on how to recover from double failures, see Clearing a database.

In most cases, you should first attempt to start the node using the normal start level. You must use the repair start level if the normal start level fails or times out.

To start a node in the database, use the hadbm startnode command. The syntax is:

hadbm startnode
 [--adminpassword=password | --adminpasswordfile=file]
 [--agent=maurl]
 [--startlevel=level]
 nodeno
 [dbname]

The dbname operand specifies the database name. The default is hadb.

The nodeno operand specifies the number of the node to start. Use hadbm status to display the numbers of all nodes in a database.

For more information, see hadbm-startnode(1).

Start level option

The hadbm startnode command has one special option, --startlevel (short form -l), that specifies the level at which to start the node.

Node start levels are:

See General Options for a description of other command options.


Example 3–6 Example of starting a node

hadbm startnode 1

Stopping a Node

You might need to stop a node to repair or upgrade the host machine’s hardware or software. To stop a node, use the hadbm stopnode command. The command syntax is:

hadbm stopnode  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[--no-repair]  
nodeno  
[dbname]

The nodeno operand specifies the number of the node to stop. The mirror node of this node number must be running. Use hadbm status to display the numbers of all nodes in a database.

The dbname operand specifies the database name. The default is hadb.

The hadbm stopnode command has one special option, --no-repair (short form -R) that indicates no spare node is to replace the stopped node. Without this option, a spare node starts up and takes over the functioning of the stopped node.

See General Options for a description of other command options. For more information, see hadbm-stopnode(1).


Example 3–7 Example of stopping a node

hadbm stopnode 1

Restarting a Node

You might have to restart a node if you notice unusual behavior such as excessive CPU consumption.

To restart a node in the database, use the hadbm restartnode command. The command syntax is:

hadbm restartnode  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[--startlevel=level]  
nodeno  
[dbname]

The dbname operand specifies the database name. The default is hadb.

The nodeno operand specifies the number of the node to restart. Use hadbm status to display the numbers of all nodes in a database.

The hadbm restartnode command has one special option, --startlevel (short form -l), that specifies the level at which to start the node. See Start level option for more information.

See General Options for a description of other command options. For more information, see hadbm-restartnode(1).


Example 3–8 Example of restarting a node

hadbm restartnode 1

Managing Databases

You can perform the following operations on HADB databases:

Starting a Database

To start a database, use the hadbm start command. This command starts all nodes that were running before the database was stopped. Individually stopped (offline) nodes are not started when the database is started after a stop.

The command syntax is:

hadbm start  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[dbname]

The dbname operand specifies the database name. The default is hadb.

See General Options for a description of command options. For more information, see hadbm-start(1).


Example 3–9 Example of starting a database

hadbm start

Stopping a Database

When you stop and start a database in separate operations, data is unavailable while it is stopped. To keep data available, you can restart a database as described in Restarting a Database.

Stop a database to:

Before stopping a database, either stop dependent Application Server instances that are using the database, or configure them to use a Persistence Type other than ha.

When you stop the database, all the running nodes in the database are stopped and the status of the database becomes Stopped. For more information about database states, see Getting the Status of HADB.

To stop a database, use the hadbm stop command. The command syntax is:

hadbm stop  
[--adminpassword=password | --adminpasswordfile= file]  
[--agent=maurl]  
[dbname]

The dbname operand specifies the database name. The default is hadb.

See General Options for a description of command options. For more information, see hadbm-stop(1).


Example 3–10 Example of stopping a database

hadbm stop

Restarting a Database

You might want to restart a database if you notice strange behavior (for example consistent timeout problems). In some cases, a restart may solve the problem.

When you restart a database, y, the database and its data remain available. When you stop and start HADB in separate operations, data and database services are unavailable while HADB is stopped. This is because by default hadbm restart performs a rolling restart of nodes: it stops and starts the nodes one by one. In contrast, hadbm stop stops all nodes simultaneously.

To restart a database, use the hadbm restart command. The command syntax is:

hadbm restart  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[--no-rolling]  
[dbname]

The dbname operand specifies the database name. The default is hadb.

This command has one special option, --no-rolling (short form -g), that specifies to restart all nodes at once, which causes loss of service. Without this option, this command restarts each of the nodes in the database to the current state or a better state.

See General Options for a description of other command options. For more information, see hadbm-restart(1).

For example:

hadbm restart

Listing Databases

To list all the databases in an HADB instance, use the hadbm list command. The command syntax is:

hadbm list  
[--agent=maurl] 
[--adminpassword=password | --adminpasswordfile=file]

See General Options for a description of command options. For more information, see hadbm-list(1).

Clearing a database

Clear a database when:

The hadbm clear command stops the database nodes, clears the database devices, then starts the nodes. This command erases the Application Server schema data store in HADB, including tables, user names, and passwords. After running hadbm clear, use asadmin configure-ha-cluster to recreate the data schema, reconfigure the JDBC connection pool, and reload the session persistence store.

The command syntax is:

hadbm clear  [--fast]  [--spares=number]  
[--dbpassword=password | --dbpasswordfile= file]  
[--adminpassword=password | --adminpasswordfile= file]  
[--agent=maurl] 
[dbname]

The dbname operand specifies the database name. The default is hadb.

The following table describes the special hadbm clear command options. See General Options for a description of other options.

For more information, see hadbm-clear(1).

Table 3–12 hadbm clear Options

Option 

Description 

Default 

--fast

-F 

Skips device initialization while initializing the database. Do not use if the disk storage device is corrupted. 

Not applicable 

--spares= number

-s 

Number of spare nodes the reinitialized database will have. Must be even and less than the number of nodes in the database. 

Previous number of spares 

For example:

hadbm clear --fast --spares=2 --dbpassword secret123

Removing a Database

To remove an existing database, use the hadbm delete command. This command deletes the database’s configuration files, device files, and history files, and frees shared memory resources. The database you want to remove must exist and must be stopped. See Stopping a Database.

The command syntax is:

hadbm delete  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[dbname]

The dbname operand specifies the database name. The default is hadb.

See General Options for a description of command options. For more information, see hadbm-delete(1).


Example 3–11 Example of removing a database

The command:

hadbm delete

deletes the default database, hadb.


Recovering from Session Data Corruption

The following are indications that session data may be corrupted:

ProcedureTo bring the session store back to a consistent state

If you determine that the session store has been corrupted, bring it back to a consistent state by following this procedure:

  1. Clear the session store.

    Determine if this action corrects the problem. If it does, then stop. If not—for example, if you continue to see errors in the server log—then continue.

  2. Re-initialize the data space on all the nodes and clear the data in the database.

    See Clearing a database .

    Determine if this action corrects the problem. If it does, then stop. If not—for example, if you continue to see errors in the server log—then continue.

  3. Delete and then recreate the database.

    See Removing a Database and Creating a Database

Expanding HADB

There are two reasons to expand your original HADB configuration:

This section describes how you can expand HADB without shutting down your Application Server cluster or database, in particular:

Also see related information in Maintaining HADB Machines .

Adding Storage Space to Existing Nodes

Add HADB storage space:

You may also want to add storage space to existing nodes if there is unused disk space on the nodes or when you add disk capacity. For information on the recommended data device size, see Specifying Device Size

To add storage space to nodes, use the hadbm set command to increase data device size.

The command syntax is:

hadbm set DataDeviceSize=size

where size is the data device size in MBytes.

See General Options for a description of command options.

Changing the data device size for a database in a FaultTolerant or higher state upgrades the system without loss of data or availability. The database remains in operational during the reconfiguration. Changing device size on a system that is not FaultTolerant or better causes loss of data. For more information about database states, see Database States.


Example 3–12 Example of setting data device size

The following command is an example of setting data device size:

hadbm set DataDeviceSize=1024

Adding Machines

You may want to add machines if HADB requires more processing or storage capacity. To add a new machine on which to run HADB, install HADB packages with or without the Application Serveras described in Chapter 2, Installing and Setting Up High Availability Database. For an explanation of node topology alternatives, see Chapter 3, Selecting a Topology, in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Deployment Planning Guide.

ProcedureTo add new machines to an existing HADB instance

  1. Start management agents on the new nodes.

  2. Extend the management domain to the new hosts.

    For details, see hadbm extenddomain command.

  3. Start the new nodes on these hosts.

    For details, see Adding Nodes

Adding Nodes

To increase processing and storage capacity of an HADB system, create new nodes and add them to the database.

After you add nodes, update the following properties of the HADB JDBC connection pool:

To add nodes, use the hadbm addnodes command. The command syntax is:

hadbm addnodes  [--no-refragment]  [--spares=sparecount]  
[--historypath=path]  
[--devicepath=path]  
[--set=attr-name-value-list]  
[--dbpassword=password | --dbpasswordfile=file ]  
[--adminpassword=password | --adminpasswordfile=file]  
--hosts=hostlist  [dbname]

The dbname operand specifies the database name. The default is hadb. The database must be in HAFaultTolerant or FaultTolerant state. For more information about database states, see Getting the Status of HADB.

If you do not specify the --devicepath and --historypath options, the new nodes will have the same device path and use the same history files as the existing database.

Adding nodes performs a refragmentation and redistribution of the existing data to include the new nodes in the system. Online refragmenting requires that the disks for the HADB nodes have enough space to contain the old data and the new data simultaneously until refragmenting is finished, that is, the user data size must not exceed 50% of the space available for user data. For details, see Getting Device Information


Note –

The best time to add nodes is when the system is lightly loaded.



Example 3–13 Example of adding nodes

For example:

hadbm addnodes --dbpassword secret123 -adminpassword=
password --hosts n6,n7,n8,n9

The following table describes the special hadbm addnodes command options. See General Options for a description of other options.

Table 3–13 hadbm addnodes Options

Option 

Description 

Default 

--no-refragment

-r 

Do not refragment the database during node creation; In this case, refragment the database later using the hadbm refragment command to use the new nodes. For details about refragmentation, see Refragmenting the Database

If you do not have sufficient device space for refragmentation, recreate the database with more nodes. See Adding Nodes by Recreating the Database

Not applicable 

--spares= number

-s 

Number of new spare nodes in addition to those that already exist. Must be even and not greater than the number of nodes added. 

--devicepath= path

-d 

Path to the devices. Devices are: 

  • DataDevice

  • NiLogDevice (node internal log device)

  • RelalgDevice (relational algebra query device)

    This path must already exist and be writable. To set this path differently for each node or each device, see Setting Heterogeneous Device Paths

Solaris and Linux: HADB_install_dir/device

Windows: C:\Sun\AppServer\SUNWhadb\vers, where vers is the HADB version number.

--hosts= hostlist

-H 

Comma-separated list of new host names for the new nodes in the database. One node is created for each comma-separated item in the list. The number of nodes must be even. IP addresses of HADB hosts must be IPv4 addresses. 

Using duplicate host names creates multiple nodes on the same machine with different port numbers. Make sure that nodes on the same machine are not mirror nodes. 

Odd numbered nodes are in one DRU, even numbered nodes in the other. If --spares is used, new spare nodes are those with the highest numbers.

If the database was created with double network interfaces, the new nodes must be configured in the same way. See Configuring Network Redundancy .

None 

Refragmenting the Database

Refragment the database to store data in newly-created nodes. Refragmentation distributes data evenly across all active nodes.

To refragment the database, use the hadbm refragment command. The command syntax is:

hadbm refragment  [--dbpassword=password | --dbpasswordfile=file]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[dbname]

The dbname operand specifies the database name. The default is hadb. The database must be in HAFaultTolerant or FaultTolerant state. For more information about database states, see Getting the Status of HADB.

See General Options for a description of command options. For more information, see hadbm-refragment(1).

Online refragmentation requires that the disks for the HADB nodes have enough space to contain the old data and the new data simultaneously until refragmenting is finished, that is, the user data size must not exceed 50% of the space available for user data. For details, see Getting Device Information


Note –

The best time to refragment the database is when the system is lightly loaded.


If this command fails after multiple attempts, see Adding Nodes by Recreating the Database


Example 3–14 Example of refragmenting the database

For example:

hadbm refragment --dbpassword secret123

Adding Nodes by Recreating the Database

If online refragmentation fails persistently when you add new nodes (either due to lack of data device space or other reasons), recreate the database with new nodes. This will lead to the loss of existing user data and schema data.

ProcedureTo add nodes by recreating the database

This procedure enables you to maintain HADB availability throughout the process.

  1. For each Application Server instance:

    1. Disable the Application Server instance in the load balancer.

    2. Disable session persistence.

    3. Restart the Application Server instance.

    4. Re-enable the Application Server instance in the load balancer.

    If you do not need to maintain availability, you can disable and re-enable all the server instances at once in the load balancer. This saves time and prevents failover of outdated session data.

  2. Stop the database as described in Stopping a Database .

  3. Delete the database as described in Removing a Database .

  4. Recreate the database with the additional nodes as described in Creating a Database

  5. Reconfigure the JDBC connection pool as described in Configuring the JDBC Connection Pool

  6. Reload the session persistence store.

  7. For each Application Server instance:

    1. Disable the Application Server instance in the load balancer.

    2. Enable session persistence.

    3. Restart the Application Server instance.

    4. Re-enable the Application Server instance in the load balancer.

    If you do not need to maintain availability, you can disable and re-enable all the server instances at once in the load balancer. This saves time and prevents failover of outdated session data.

Monitoring HADB

You can monitor the activities of HADB by:

These sections briefly describe the hadbm status, hadbm deviceinfo, and hadbm resourceinfo commands. For information on interpreting HADB information, see Performance in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Performance Tuning Guide.

Getting the Status of HADB

Use the hadbm status command to display the status of the database or its nodes. The command syntax is:

hadbm status  
[--nodes]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl] 
[dbname]

The dbname operand specifies the database name. The default is hadb.

The --nodes option (short form -n) displays information on each node in the database. For more information , see Node Status. See General Options for a description of other command options.

For more information, see hadbm-status(1).


Example 3–15 Example of getting HADB status

For example:

hadbm status --nodes

Database States

A database’s state summarizes its current condition. The following table describes the possible database states.

Table 3–14 HADB States

Database State 

Description 

High-Availability Fault Tolerant (HAFaultTolerant) 

Database is fault tolerant and has at least one spare node on each DRU. 

Fault Tolerant 

All the mirrored node pairs are up and running. 

Operational 

At least one node in each mirrored node pair is running. 

Non Operational 

One or more mirrored node pairs is missing both nodes. 

If the database is non-operational, clear the database as described in Clearing a database.

Stopped 

No nodes are running in the database. 

Unknown 

Cannot determine the state of the database. 

Node Status

Use the--nodes option to make the hadbm status command display the following information for each node in the database:

A node’s role and state can change as described in these sections:

Roles of a Node

A node is assigned a role during its creation and can take any one of these roles:

States of a Node

A node can be in any one of the following states:

Getting Device Information

Monitor free space in HADB data (disk storage) devices:

Use the hadbm deviceinfo command to get information about free space in data devices. This command displays the following information for each node of the database:

The command syntax is:

hadbm deviceinfo  [--details]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  [dbname]

The dbname operand specifies the database name. The default is hadb.

The --details option displays the following additional information:

See General Options for a description of other command options.

For more information, see hadbm-deviceinfo(1).

To determine the space available for user data, take the total device size, then subtract the space reserved for HADB: four times the LogBufferSize + 1% of the device size. If you do not know the size of the log buffer, use the command hadbm get logbufferSize. For example, if the total device size is 128 MB and the LogBufferSize is 24 MB, the space available for user data is 128 – (4 x 24) = 32 MB. Of the 32 MB, half is used for replicated data and around one percent is used for the indices, and only 25 percent is available for the real user data.

The space available for user data is the difference between the total size and reserved size. If the data is refragmented in the future, the free size must be approximately equal to 50% of the space available for user data. If refragmentation is not relevant, the data devices can be exploited to their maximum. Resource consumption warnings are written to the history files when the system is running short on device space.

For more information about tuning HADB, see the Sun Java System Application Server Performance Tuning Guide.


Example 3–16 Example of getting device information

The following command:

hadbm deviceinfo --details

Displays the following example results:

NodeNO Totalsize Freesize Usage NReads NWrites DeviceName
0      128       120      6%    10000  5000    C:\Sun\SUNWhadb\hadb.data.0
1      128       124      3%    10000  5000    C:\Sun\SUNWhadb\hadb.data.1
2      128       126      2%     9500  4500    C:\Sun\SUNWhadb\hadb.data.2
3      128       126      2%     9500  4500    C:\Sun\SUNWhadb\hadb.data.3

Getting Runtime Resource Information

The hadbm resourceinfo command displays HADB runtime resource information. You can use this information to help identify resource contention, and reduce performance bottlenecks. For details, see Tuning HADB in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Performance Tuning Guide.

The command syntax is:

hadbm resourceinfo  [--databuf]  [--locks]  [--logbuf]  [--nilogbuf]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]  
[dbname]

The dbname operand specifies the database name. The default is hadb.

The following table describes the hadbm resourceinfo special command options. See General Options for a description of other command options.

For more information, see hadbm-resourceinfo(1).

Table 3–15 hadbm resourceinfo Command Options

Option 

Description 

--databuf

-d 

Display data buffer pool information. 

See Data Buffer Pool Information below for more information.

--locks

-l 

Display lock information. 

See Lock Information below for more information.

--logbuf

-b 

Display log buffer information. 

See Log Buffer Information below for more information.

--nilogbuf

-n 

Display node internal log buffer information. 

See Node Internal Log Buffer Information below for more information.

Data Buffer Pool Information

Data buffer pool information contains the following:

When a user transaction performs an operation on a record, the page containing the record must be in the data buffer pool. If it is not, a miss or a page fault occurs. The transaction then has to wait until the page is retrieved from the data device file on the disk.

If the miss rate is high, increase the data buffer pool. Since the misses are cumulative, run hadbm resourceinfo periodically and use the difference between two runs to see the trend of miss rate. Do not be concerned if free space is very small, since the checkpointing mechanism will make new blocks available.


Example 3–17 Example data buffer pool information

For example:


NodeNO Avail Free Access Misses Copy-on-Write
0 256 128 100000 50000 10001 256 128 110000 45000 950

Lock Information

Lock information is as follows:

One single transaction cannot use more than 25% of the available locks on a node. Therefore, transactions performing operations in large scale should be aware of this limitation. It is best to perform such transactions in batches, where each batch must be treated as a separate transaction, that is, each batch commits. This is needed because read operations running with repeatable read isolation level, and delete, insert, and update operations use locks that are released only after the transaction terminates.

To change the NumberOfLocks, see Clearing and Archiving History Files


Example 3–18 Example lock information

For example:


NodeNO Avail Free Waits
0 50000 20000 101 50000 20000 0

Log Buffer Information

Log buffer information is:

Do not worry if free space is very small, since HADB starts compressing the log buffer. HADB starts compression from the head of the ring buffer and performs it on consecutive log records. Compression cannot proceed when HADB encounters a log record that has not been executed by the node and received by the mirror node


Example 3–19 Example of log buffer information

For example:


NodeNO Avail Free
0 16 21 16 3

Node Internal Log Buffer Information

Node internal log buffer information is:


Example 3–20 Example of internal log buffer information

For example:

NodeNO Avail Free

0 16 21 16 3


Maintaining HADB Machines

HADB achieves fault tolerance by replicating data on mirror nodes. In a production environment, a mirror node is on a separate DRU from the node it mirrors, as described in Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Deployment Planning Guide.

A failure is an unexpected event such as a hardware failure, power failure, or operating system reboot. The HADB tolerates single failures: of one node, one machine (that has no mirror node pairs), one or more machines belonging to the same DRU, or even one entire DRU. However, HADB does not automatically recover from a double failure, which is the simultaneous failure of one or more mirror node pairs. If a double failure occurs, you must clear HADB and recreate its session store, which erases all its data.

There are different maintenance procedures, depending on whether you need to work on a single machine or multiple machines.

ProcedureTo perform maintenance on a single machine

This procedure is applicable to both planned and unplanned maintenance, and does not interrupt HADB availability.

  1. Perform the maintenance procedure and get the machine up and running.

  2. Ensure that ma is running.

    If ma runs as a Windows service or under init.d scripts (recommended for deployment), it should have been started by the operating system. If not start it manually. See Starting the Management Agent.

  3. Start all nodes on the machine.

    For more information, see Starting a Node.

  4. Check whether the nodes are active and running.

    For more information, see Getting the Status of HADB

ProcedureTo perform planned maintenance on all HADB machines

Planned maintenance includes operations such as hardware and software upgrades. This procedure does not interrupt HADB availability.

  1. For each spare machine in the first DRU, repeat the single machine procedure as described in To perform maintenance on a single machine, one by one, for each machine.

  2. For each active machine in the first DRU, repeat the single machine procedure as described in To perform maintenance on a single machine, one by one, for each machine.

  3. Repeat step 1 and step 2 for the second DRU.

ProcedureTo perform planned maintenance on all HADB machines

This procedure is applicable when HADB is on single or multiple machines. It interrupts HADB service during the maintenance procedure.

  1. Stop HADB. See Stopping a Database .

  2. Perform the maintenance procedure and get all the machines up and running.

  3. Ensure ma is running.

  4. Start HADB.

    For more information, see Starting a Database.

    After you complete the last step, HADB data becomes available again.

ProcedureTo perform unplanned maintenance in the event of a failure

  1. Check the database state.

    See Getting the Status of HADB

    • If the database state is Operational or better:

      The machines needing unplanned maintenance do not include mirror nodes. Follow the single machine procedure for each failed machine, one DRU at a time. HADB service is not interrupted.

    • If the database state is Non-Operational:

      The machines needing unplanned maintenance include mirror nodes. One such case is when the entire HADB is on a single failed machine. Get all the machines up and running first. Then clear HADB and recreate the session store. See Clearing a database. This interrupts HADB service.

Clearing and Archiving History Files

HADB history files record all database operations and error messages. HADB appends to the end of existing history files, so the files grow over time. To save disk space and prevent files from getting too large, periodically clear and archive history files.

To clear a database’s history files, use the hadbm clearhistory command.

The command syntax is:

hadbm clearhistory  
[--saveto=path]  
[dbname]  
[--adminpassword=password | --adminpasswordfile=file]  
[--agent=maurl]

The dbname operand specifies the database name. The default is hadb.

Use the --saveto option (short form -o) to specify the directory in which to store the old history files. This directory must have appropriate write permissions. See General Options for a description of other command options.

For more information, see hadbm-clearhistory(1).

The --historypath option of the hadbm create command determines the location of the history files. The names of the history files are of the format dbname.out. nodeno. For information on hadbm create, see Creating a Database

History File Format

Each message in the history file contains the following information:

Messages about resource shortages contain the string “HIGH LOAD.”

You do not need a detailed knowledge of all entries in the history file. If for any reason you need to study a history file in greater detail, contact Sun customer support.