test

Sun Java System Application Server Enterprise Edition 8.1 2005Q2 High Availability Administration Guide

Preparing for HADB Setup

This section discusses the following topics:

Prerequisites

Before setting up and configuring HADB, make sure your environment meets the following requirements:

Configuring Network Redundancy

Configuring a redundant network will enable HADB to remain available, even if there is a single network failure. You can configure a redundant network in two ways:

Setting Up Network Multipathing

Before setting up network multipathing, refer to the Administering Network Multipathing section in the IP Network Multipathing Administration Guide.

ProcedureTo configure HADB host machines that already use IP multipathing:

  1. Set network interface failure detection time.

    For HADB to properly support multipathing failover, the network interface failure detection time must not exceed one second (1000 milliseconds), as specified by the FAILURE_DETECTION_TIME parameter in /etc/default/mpathd. Edit the file and change the value of this parameter to 1000 if the original value is higher:


    FAILURE_DETECTION_TIME=1000

    To put the change into effect, use this command:


    pkill -HUP in.mpathd

  2. Set up IP addresses to use with HADB.

    As described in the IP Network Multipathing Administration Guide, multipathing involves grouping physical network interfaces into multipath interface groups. Each physical interface in such a group has two IP addresses associated with it:

    • a physical interface address used for transmitting data.

    • a test address for Solaris internal use only.

    Specify only one physical interface address from the multipath group when you use hadbm create --hosts.

Example 2–1 Setting up Multipathing

Suppose there are two host machines named host1 and host2. If they each have two physical network interfaces, then set up the two interfaces as a multipath group. Run ifconfig -a on each host.

The output on host1 is:


bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>
mtu 1500 index 5 inet 129.159.115.10 netmask ffffff00 broadcast 129.159.115.255 
groupname mp0

bge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 5 inet 129.159.115.11 netmask ffffff00 broadcast 129.159.115.255

bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> 
mtu 1500 index 6 inet 129.159.115.12 netmask ffffff00 broadcast 129.159.115.255 
groupname mp0

bge1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> 
mtu 1500 index 6 inet 129.159.115.13 netmask ff000000 broadcast 129.159.115.255

The output on host2 is:


bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> 
mtu 1500 index 3 inet 129.159.115.20 netmask ffffff00 broadcast 129.159.115.255 
groupname mp0

bge0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> 
mtu 1500 index 3 inet 129.159.115.21 netmask ff000000 broadcast 129.159.115.255

bge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> 
mtu 1500 index 4 inet 129.159.115.22 netmask ffffff00 broadcast 129.159.115.255 
groupname mp0

bge1:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> 
mtu 1500 index 4 inet 129.159.115.23 netmask ff000000 broadcast 129.159.115.255

In this example, the physical network interfaces on both hosts are listed after bge0 and bge1. Those listed after bge0:1 and bge1:1 are multipath test interfaces (marked DEPRECATED in the ifconfig output), as described in the IP Network Multipathing Administration Guide.

To set up HADB in this environment, select one physical interface address from each host. In this example, HADB uses IP address 129.159.115.10 from host1 and 129.159.115.20 from host2. To create a database with one database node per host, use the command hadbm create --host. For example


hadbm create --host 129.159.115.10,129.159.115.20

To create a database with two database nodes on each host, use the command:


hadbm create --host 129.159.115.10,129.159.115.20,
129.159.115.10,129.159.115.20

In both cases, you must configure the agents on host1 and host2 with separate parameters to specify which interface on the machines the agents should use. So, on host1 use:


ma.server.mainternal.interfaces=129.159.115.10

And on host2 use:


ma.server.mainternal.interfaces=129.159.115.20

For information on the ma.server.mainternal.interfaces variable, see Configuration File.

Configuring Double Networks

To enable HADB to tolerate single network failures, use IP multipathing if the operating system (for example, Solaris) supports it. Do not configure HADB with double networks on Windows Server 2003—the operating system does not work properly with double networks.

If your operating system is not configured for IP multipathing, and HADB hosts are equipped with two NICs, you can configure HADB to use double networks. For every host, the IP addresses of each of the network interface card (NIC) must be on separate IP subnets.

Within a database, all nodes must be connected to a single network, or all nodes must be connected to two networks.

Note –

Routers between the subnets must be configured to forward UDP multicast messages between subnets.

When creating an HADB database, use the –hosts option to specify two IP addresses or host names for each node: one for each NIC IP address. For each node, the first IP address is on net-0 and the second on net-1. The syntax is as follows, with host names for the same node separated by a plus sign (+):

--hosts=node0net0name+node0net1name
,node1net0name+node1net1name
,node2net0name+node2net1name
, ...

For example, the following argument creates two nodes, each with two network interfaces. The following host option is used to create these nodes:

--hosts 10.10.116.61+10.10.124.61,10.10.116.62+10.10.124.62

Thus, the network addresses

Notice that 10.10.116.61 and 10.10.116.62 are on the same subnet, and 10.10.124.61 and 10.10.124.62 are on the same subnet.

In this example, the management agents must use the same subnet. Thus, the configuration variable ma.server.mainternal.interfaces must be set to, for example, 10.10.116.0/24. This setting can be used on both agents in this example.

Configuring Shared Memory and Semaphores

You must configure shared memory and semaphores before installing HADB. The procedure depends on your operating system.

ProcedureTo configure shared memory and semaphores on Solaris

  1. Log in as root.

  2. Configure shared memeory.

    Set the value of shmmax to the size of the physical memory on the HADB host machine. The maximum shared memory segment size must be larger than the size of the HADB database buffer pool. For example, for a machine with a 2 GByte (0x8000000 hexadecimal) main memory, add the following to the /etc/system file:


    set shmsys:shminfo_shmmax=0x80000000
 set shmsys:shminfo_shmseg=20

    On Solaris 9 and later, shmsys:shminfo_shmseg is obsolete.

    Set shminfo_shmmax to the total memory in your system (in hexadecimal notation the value 0x80000000 shown is for 2 Gigabytes of memory).

    Note –

    Specify the value of shmsys:shminfo_shmmax using the hexadecimal value for the memory size. To determine your host’s memory, use this command:


    prtconf | grep Memory

  3. Configure semaphores.

    Check the /etc/system file for semaphore configuration entries. This file might already contain semmni, semmns, and semmnu entries. For example:


    set semsys:seminfo_semmni=10
set semsys:seminfo_semmns=60
set semsys:seminfo_semmnu=30

    If the entries are present, increment the values by adding 16, 128, and 1000 respectively. So, the entries in the example above would change to:


    set semsys:seminfo_semmni=26
set semsys:seminfo_semmns=188
set semsys:seminfo_semmnu=1030

    If the /etc/system file does not these entries, add them at the end of the file:


    set semsys:seminfo_semmni=16
set semsys:seminfo_semmns=128
set semsys:seminfo_semmnu=1000

    This is sufficient to run up to 16 HADB nodes on the computer. For information on setup for more than 16 nodes, see the HADB chapter in the Sun Java System Application Server Enterprise Edition 8.1 2005Q1 Performance Tuning Guide.

  4. Reboot the machine.

ProcedureTo configure shared memory on Linux

  1. Log in as root.

  2. Edit the file /etc/sysctl.conf

  3. Set the kernel.shmax and kernel.shmall parameters.

    The kernel.shmax parameter defines the maximum size in bytes for a shared memory segment. The kernel.shmall parameter sets the total amount of shared memory in pages that can be used at one time on the system. Set the value of both of these parameters to the amount physical memory on the machine. Specify the value as a decimal number of bytes. For example, for a machine having 512 Mbytes of physical memory:


    kernel.shmmax=536870912
 kernel.shmall=536870912

  4. Reboot the machine. using this command:

    sync; sync; reboot

Procedure for Windows

Windows does not require any special system settings. However, if you want to use an existing J2SE installation, set the JAVA_HOME environment variable to the location where the J2SE is installed.

Synchronizing System Clocks

You must synchronize clocks on HADB hosts, because HADB uses time stamps based on the system clock. HADB uses the system clock to manage timeouts and to time stamp events logged to history files. For troubleshooting, you must analyze all the history files together, since HADB is a distributed system. So, it is important that all the hosts’ clocks be synchronized

Do not adjust system clocks on a running HADB system. Doing so can cause problems in the operating system or other software components that can in turn cause problems such as hangs or restarts of HADB nodes. Adjusting the clock backward can cause some HADB server processes to hang as the clock is adjusted.

To synchronize clocks:

If HADB detects a clock adjustment of more than one second, it logs it to the node history file, for example:

NSUP INF 2003-08-26 17:46:47.975 Clock adjusted.
 Leap is +195.075046 seconds.

File System Support

This section describes some restrictions of HADB with certain file systems.

Red Hat Enterprise Linux

HADB supports the ext2 and ext3 file systems on Red Hat Enterprise Linux 3.0. For Red Hat Enterprise Linux 2.1, HADB supports the ext2 file system.

Veritas File System

When using the Veritas File System on Solaris, HADB writes the message WRN: Direct disk I/O mapping failed to the history files. This message indicates that HADB cannot turn on direct input/output (I/O) for the data and log devices. Direct I/O reduces the CPU cost of writing disk pages. It also reduces overhead of administering “dirty” data pages in the operating system.

To use direct I/O with Veritas File System, do one of the following:

Note –

These configurations have not been tested with the Sun Java System Application Server.