After performing these tasks, see Chapter 3, Administering High Availability Database.
For the latest information on HADB, see Sun Java System Application Server Enterprise Edition 8.1 2005Q2 Release Notes.
Before setting up and configuring HADB, make sure your environment meets the following requirements:
IPv4 is enabled. HADB supports IPv4 only. Disable IPv6 on the interfaces being used for HADB.
The network (routers, switches, and network interfaces on the hosts) must be configured for User Datagram Protocol (UDP) multicast. If HADB hosts span multiple subnets, configure routers between the subnets to forward UDP multicast messages between the subnets.
Configure any firewalls located between HADB hosts, or between HADB and Application Server hosts to allow all UDP traffic, both ordinary and multicast.
Do not use dynamic IP addresses (assigned by Dynamic Host Configuration Protocol, or DHCP) for hosts that are part of hadbm createdomain, hadbm extenddomain, hadbm create, or hadbm addnodes commands.
Configuring a redundant network will enable HADB to remain available, even if there is a single network failure. You can configure a redundant network in two ways:
On Solaris 9, you can set up network multipathing.
Configure a double network, supported on all platforms except Windows Server 2003.
Before setting up network multipathing, refer to the Administering Network Multipathing section in the IP Network Multipathing Administration Guide.
Set network interface failure detection time.
For HADB to properly support multipathing failover, the network interface failure detection time must not exceed one second (1000 milliseconds), as specified by the FAILURE_DETECTION_TIME parameter in /etc/default/mpathd. Edit the file and change the value of this parameter to 1000 if the original value is higher:
To put the change into effect, use this command:
pkill -HUP in.mpathd
Set up IP addresses to use with HADB.
As described in the IP Network Multipathing Administration Guide, multipathing involves grouping physical network interfaces into multipath interface groups. Each physical interface in such a group has two IP addresses associated with it:
a physical interface address used for transmitting data.
a test address for Solaris internal use only.
Specify only one physical interface address from the multipath group when you use hadbm create --hosts.
Suppose there are two host machines named host1 and host2. If they each have two physical network interfaces, then set up the two interfaces as a multipath group. Run ifconfig -a on each host.
The output on host1 is:
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5 inet 220.127.116.11 netmask ffffff00 broadcast 18.104.22.168 groupname mp0 bge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 5 inet 22.214.171.124 netmask ffffff00 broadcast 126.96.36.199 bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6 inet 188.8.131.52 netmask ffffff00 broadcast 184.108.40.206 groupname mp0 bge1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 6 inet 220.127.116.11 netmask ff000000 broadcast 18.104.22.168
The output on host2 is:
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 22.214.171.124 netmask ffffff00 broadcast 126.96.36.199 groupname mp0 bge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3 inet 188.8.131.52 netmask ff000000 broadcast 184.108.40.206 bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4 inet 220.127.116.11 netmask ffffff00 broadcast 18.104.22.168 groupname mp0 bge1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4 inet 22.214.171.124 netmask ff000000 broadcast 126.96.36.199
In this example, the physical network interfaces on both hosts are listed after bge0 and bge1. Those listed after bge0:1 and bge1:1 are multipath test interfaces (marked DEPRECATED in the ifconfig output), as described in the IP Network Multipathing Administration Guide.
To set up HADB in this environment, select one physical interface address from each host. In this example, HADB uses IP address 188.8.131.52 from host1 and 184.108.40.206 from host2. To create a database with one database node per host, use the command hadbm create --host. For example
hadbm create --host 220.127.116.11,18.104.22.168
To create a database with two database nodes on each host, use the command:
hadbm create --host 22.214.171.124,126.96.36.199, 188.8.131.52,184.108.40.206
In both cases, you must configure the agents on host1 and host2 with separate parameters to specify which interface on the machines the agents should use. So, on host1 use:
And on host2 use:
For information on the ma.server.mainternal.interfaces variable, see Configuration File.
To enable HADB to tolerate single network failures, use IP multipathing if the operating system (for example, Solaris) supports it. Do not configure HADB with double networks on Windows Server 2003—the operating system does not work properly with double networks.
If your operating system is not configured for IP multipathing, and HADB hosts are equipped with two NICs, you can configure HADB to use double networks. For every host, the IP addresses of each of the network interface card (NIC) must be on separate IP subnets.
Within a database, all nodes must be connected to a single network, or all nodes must be connected to two networks.
Routers between the subnets must be configured to forward UDP multicast messages between subnets.
When creating an HADB database, use the –hosts option to specify two IP addresses or host names for each node: one for each NIC IP address. For each node, the first IP address is on net-0 and the second on net-1. The syntax is as follows, with host names for the same node separated by a plus sign (+):
--hosts=node0net0name+node0net1name ,node1net0name+node1net1name ,node2net0name+node2net1name , ...
For example, the following argument creates two nodes, each with two network interfaces. The following host option is used to create these nodes:
Thus, the network addresses
For node0 are 10.10.116.61 and 10.10.124.61
For node1 are 10.10.116.62 and 10.10.124.62
Notice that 10.10.116.61 and 10.10.116.62 are on the same subnet, and 10.10.124.61 and 10.10.124.62 are on the same subnet.
In this example, the management agents must use the same subnet. Thus, the configuration variable ma.server.mainternal.interfaces must be set to, for example, 10.10.116.0/24. This setting can be used on both agents in this example.
You must configure shared memory and semaphores before installing HADB. The procedure depends on your operating system.
Log in as root.
Configure shared memeory.
Set the value of shmmax to the size of the physical memory on the HADB host machine. The maximum shared memory segment size must be larger than the size of the HADB database buffer pool. For example, for a machine with a 2 GByte (0x8000000 hexadecimal) main memory, add the following to the /etc/system file:
set shmsys:shminfo_shmmax=0x80000000 set shmsys:shminfo_shmseg=20
On Solaris 9 and later, shmsys:shminfo_shmseg is obsolete.
Set shminfo_shmmax to the total memory in your system (in hexadecimal notation the value 0x80000000 shown is for 2 Gigabytes of memory).
Specify the value of shmsys:shminfo_shmmax using the hexadecimal value for the memory size. To determine your host’s memory, use this command:
prtconf | grep Memory
Check the /etc/system file for semaphore configuration entries. This file might already contain semmni, semmns, and semmnu entries. For example:
set semsys:seminfo_semmni=10 set semsys:seminfo_semmns=60 set semsys:seminfo_semmnu=30
If the entries are present, increment the values by adding 16, 128, and 1000 respectively. So, the entries in the example above would change to:
set semsys:seminfo_semmni=26 set semsys:seminfo_semmns=188 set semsys:seminfo_semmnu=1030
If the /etc/system file does not these entries, add them at the end of the file:
set semsys:seminfo_semmni=16 set semsys:seminfo_semmns=128 set semsys:seminfo_semmnu=1000
This is sufficient to run up to 16 HADB nodes on the computer. For information on setup for more than 16 nodes, see the HADB chapter in the Sun Java System Application Server Enterprise Edition 8.1 2005Q1 Performance Tuning Guide.
Reboot the machine.
Log in as root.
Edit the file /etc/sysctl.conf
Set the kernel.shmax and kernel.shmall parameters.
The kernel.shmax parameter defines the maximum size in bytes for a shared memory segment. The kernel.shmall parameter sets the total amount of shared memory in pages that can be used at one time on the system. Set the value of both of these parameters to the amount physical memory on the machine. Specify the value as a decimal number of bytes. For example, for a machine having 512 Mbytes of physical memory:
Reboot the machine. using this command:
sync; sync; reboot
Windows does not require any special system settings. However, if you want to use an existing J2SE installation, set the JAVA_HOME environment variable to the location where the J2SE is installed.
You must synchronize clocks on HADB hosts, because HADB uses time stamps based on the system clock. HADB uses the system clock to manage timeouts and to time stamp events logged to history files. For troubleshooting, you must analyze all the history files together, since HADB is a distributed system. So, it is important that all the hosts’ clocks be synchronized
Do not adjust system clocks on a running HADB system. Doing so can cause problems in the operating system or other software components that can in turn cause problems such as hangs or restarts of HADB nodes. Adjusting the clock backward can cause some HADB server processes to hang as the clock is adjusted.
To synchronize clocks:
On Solaris, use xntpd (network time protocol daemon).
On Linux, use ntpd.
On Windows, use NTPTime on Windows
If HADB detects a clock adjustment of more than one second, it logs it to the node history file, for example:
NSUP INF 2003-08-26 17:46:47.975 Clock adjusted. Leap is +195.075046 seconds.
This section describes some restrictions of HADB with certain file systems.
HADB supports the ext2 and ext3 file systems on Red Hat Enterprise Linux 3.0. For Red Hat Enterprise Linux 2.1, HADB supports the ext2 file system.
When using the Veritas File System on Solaris, HADB writes the message WRN: Direct disk I/O mapping failed to the history files. This message indicates that HADB cannot turn on direct input/output (I/O) for the data and log devices. Direct I/O reduces the CPU cost of writing disk pages. It also reduces overhead of administering “dirty” data pages in the operating system.
To use direct I/O with Veritas File System, do one of the following:
Create the data and log devices on a file system that is mounted with the option mincache=direct. This option applies to all files created on the file system. Check the mount_vxfs(1M) command for details.
Use the Veritas Quick I/O facility to perform raw I/O to file system files. For details, see the VERITAS File System 4.0 Administrator’s Guide for Solaris.
These configurations have not been tested with the Sun Java System Application Server.