Sun Java logo     Previous      Contents      Index      Next     

Sun logo
Sun Java System Application Server Standard and Enterprise Edition 7 2004Q2 Update 2 Installation Guide 

Chapter 2
Preparing for HADB Setup

After the high-availability components are installed on the servers that will be part of a cluster, you are ready to set up high availability.

The following topics are addressed here:

After you have done the tasks here, proceed to the Sun Java System Application Server Administration Guide for comprehensive instructions on configuring and managing the cluster, the load balancer plug-in, and the high-availability database (HADB).

Information on high-availability topologies is available in the Sun Java System Application Server System Deployment Guide.

Configuring Shared Memory and Semaphores

This section contains instructions for configuring shared memory and System V semaphores for the HADB host machines. You must configure the shared memory before working with HADB.

Information on high-availability topology is available in the Sun Java System Application Server System Deployment Guide.

Configuring Shared Memory on Solaris

  1. Log in as root.
  2. Set the shmmax value to the size of the physical memory on the HADB host machine. The maximum shared memory segment size must be larger than the size of the HADB database buffer pool. For a machine with a 2 GBytes (0x8000000 hexadecimal) main memory, add the following to the /etc/system file:

    3. set shmsys:shminfo_shmmax=0x80000000
    set shmsys:shminfo_shmseg=20

    Note

    • shmsys:shminfo_shmseg is obsolete in Solaris 9 and later.
    • The shmsys:shminfo_shmmax value is specified using the hexadecimal value of the memory size. To determine your host’s memory, run this command: prtconf | grep Memory

  3. Check your /etc/system file for semaphore configuration entries. This file may already contain semmni, semmns, and semmnu entries. For example:

    4. set semsys:seminfo_semmni=10
    set semsys:seminfo_semmns=60
    set semsys:seminfo_semmnu=30

    If the entries are present, increment the values by 16, 128, and 1000 respectively. The entries in the example above would change to:

    set semsys:seminfo_semmni=26
    set semsys:seminfo_semmns=188
    set semsys:seminfo_semmnu=1030

    If your /etc/system file does not contain the above mentioned entries, add the following entries at the end of the file:

    set semsys:seminfo_semmni=16
    set semsys:seminfo_semmns=128
    set semsys:seminfo_semmnu=1000

    This is sufficient to run up to 16 HADB nodes on the computer. Consult the HADB chapter in the Sun Java System Performance Tuning Guide if there will be more than 16 nodes.

  4. To make these changes take effect, restart the machine.

Configuring Shared Memory on Linux

  1. Log in as root.
  2. Set the size of the maximum shared memory segment to the same value as the physical memory on the machine by adding the following entry to the /etc/sysctl.conf file:

    3. kernel.shmmax=536870912

    This value is given as number of bytes in decimal numbering. This example is for a machine having 512 Mbytes of physical memory.

  3. Reboot the machine.

Network Configuration Requirements

Configuration requirements for the network include the following:

To ensure HADB availability in spite of single network failures, use one of the following:

Time Synchronization

Synchronize the clocks on the hosts running HADB, because HADB uses time stamps based on the system clock for debugging purposes as well as for controlling internal events. The events are written to history files prefixed by time stamps. Since HADB is a distributed system, history files from all HADB nodes are analyzed together in troubleshooting. HADB also uses the system clock internally to manage time-dependent events such as timeouts.

Adjusting system clocks on running HADB systems is not recommended. HADB has been implemented to handle this in general, but you should note the following points:

To synchronize clocks, use one of these:

If a clock adjustment of more than 1 second is detected, this line is output into the history file of the HADB nodes:

NSUP INF 2003-08-26 17:46:47.975 Clock adjusted.
Leap is +195.075046 seconds.

File System Support

To configure HADB, you need to understand the following, if you use one of these file systems:

Running HADB Node Supervisor Processes with Real-time Priority

Node supervisor processes (NSUP) ensure the high availability of the HADB with the help of exchanging I’m alive messages in a timely manner. Timing is affected when an NSUP process is co-located with other processes and resource starvation results. False network partitioning and node restarts (preceded by a warning Process blocked for .. seconds in HADB history files) occur, resulting in aborted transactions and other exceptions.

To solve this, NSUP executable clu_nsup_srv in the install_dir/lib/server directory should have setuid root. To avoid any security impact by using setuid, the real-time priority is set immediately after the process is started and the process falls back to the effective uid once the priority has been changed. Other HADB processes run with normal priority.

If NSUP cannot set the real-time priority, it issues a warning Could not set realtime priority in Windows. In Solaris and Linux errno is set to EPERM. The error is written to the ma.log file, and the process continues without real-time priority.

Setting real-time priorities is not possible when:

NSUP is not CPU consuming, its footprint is small, and running it with realtime priority does not affect performance.

Starting the HADB Management Agent

HADB requires that a management agent must be running on every HADB host before you issue any hadbm management commands, including the creation of the database instance.

Chapter 3, “Administering the High Availability Database,” of the Sun Java System Application Server Administration Guide describes how the management agents are started.

Setting Up the User Environment

Note

The procedures mentioned in this chapter contain UNIX specific examples. Except where indicated, the same commands and examples, with appropriate modifications, are applicable for Microsoft Windows platforms.

After you have set up host communication, you can run the hadbm command from the install_dir/SUNWhadb/4/bin directory location as follows:

However, it is much more convenient to set up your local environment to use the high-availability management client commands from anywhere. To set this up, perform the following steps.

Note

The examples in this section apply to using csh. If you are using another shell, refer to the man page for your shell for instructions on setting variables.

  1. Set the PATH variable:

    2. setenv PATH ${PATH}:install_dir/bin:install_dir/SUNWhadb/4/bin

  2. Verify that the PATH settings are correct by running the following commands:

    3. which asadmin
    which hadbm

  3. If multiple Java versions are installed, ensure that the JAVA_HOME environment is accessing JDK version 1.4.2_06 for Enterprise Edition.

    4. setenv JAVA_HOME java_install_dir
    setenv PATH ${PATH}:${JAVA_HOME}/bin

  4. If your HADB device files and log files are not in the default location (appserver_install_dir/SUNWhadb/4),use the following hadbm command to locate these important files:

    5. hadbm get configpath
    hadbm get devicepath
    hadbm get historypath
    hadbm get installpath

    Backup the locations listed by these commands.

Setting Up Administration for Non-Root

By default, during the initial installation or setup of the Sun Java System Application Server, write permissions of the files and paths created for Sun Java System Application Server are given to root only. For a user other than root to create or manage the Sun Java System Application Server, write permissions on the associated files must be given to that specific user, or to a group to which the user belongs. The files that are affected are (with their default locations):

You can create a user group for managing the Sun Java System Application Server as described in the following procedure. (An alternate approach is to set permissions and ownership for the specific user.)

To create a Sun Java System Application Server user group and set permissions on the installation root directory, repeat the following process for each affected file:

  1. Log in as root.
  2. From the command prompt, create the Sun Java System Application Server user group. For example:

    3. # groupadd sjsasuser

    You can type groupadd at the command line to see appropriate usage.

  3. Change the group ownership for each affected file to the newly-created group. For example:

    4. chgrp -R sjsasuser install_config_dir/cl*.conf

  4. Set the write permission for the newly-created group:

    5. chmod -R g+rw install_config_dir/cl*.conf

  5. Repeat steps 3 and 4 for each affected file.
  6. Make the clsetup and cladmin commands executable by the newly-created group. For example:

    7. chmod -R g+x install_dir/bin/cl*

  7. Delete and recreate the default domain, domain1, using the --sysuser option. The sysuser must also belong to the newly-created group. For example:

    8. asadmin delete-domain domain1

    asadmin create-domain --sysuser bleonard --adminport 4848 --adminuser admin --adminpassword password domain1

Using clsetup

The purpose of the clsetup utility is to automate the process of setting up a basic cluster in a typical configuration. The utility is located in install_dir/bin, where install_dir is the directory where the Sun Java System Application Server software is installed.

The clsetup utility, as well as the cladmin utility are bundled with Sun Java System Application Server.

Note

The cladmin command is used to streamline the process of configuring and administering the cluster after all installation and configuration tasks are complete. Refer to the Sun Java System Application Server Administration Guide for instructions on creating the HADB and on using the cladmin command.

The following topics are addressed:

How clsetup Works

The clsetup utility is a set of Sun Java System Application Server commands that allow a cluster to be configured automatically, based on prepopulated input files. As part of cluster setup, an HADB is created. However, you must set up your working cluster using the hadbm commands as described in the Sun Java System Application Server Administration Guide.

Note

The clsetup utility interface is unstable. An unstable interface may be experimental or transitional, and may therefore change incompatibly, be removed, or be replaced by a more stable interface in the next release.

The following topics are addressed in this section:

How the Input Files Work

Three input files are used by the clsetup utility to configure the cluster:

Use the clsetup configuration parameters as they are preconfigured to set up a typical cluster configuration. To support a different configuration, make edits to any or all of the configuration files.

What clsetup Accomplishes

Using the pre-populated values in the clsetup input files, the clsetup utility command:

Commands Used by clsetup

The clsetup utility uses a number of hadbm and asadmin commands to set up the cluster. Table 2-1 lists the clsetup tasks and the commands used to accomplish these tasks.

Table 2-1  hadbm and asadmin Commands Used by the clsetup Utility

Task Performed by clsetup

Command

Checks to see if database exists.

hadbm status

Creates and starts the HADB.

hadbm create

Gets the JDBC URL.

hadbm get jdbcURL

Creates the session store.

asadmin create-session-store

Checks the instance status.

asadmin show-instance-status

Creates the instance.

asadmin create-instance

Creates the JDBC connection pool.

asadmin create-jdbc-connection-pool

Registers the data source.

asadmin create-jdbc-resource

Configures the persistence type

asadmin configure-session-persistence

Configures RMI/IIOP failover

asadmin add-iiop-cluster-endpoint

Configures SFSB failover

asadmin set

Reconfigures the instance.

asadmin reconfig -u admin

Requirements and Limitations of clsetup

The following requirements and limitations apply to the clsetup utility:

Editing the clsetup Input Files

The input files that are needed by the clsetup utility are installed under the configuration installation directory, (by default /etc/opt/SUNWappserver7 on UNIX and c:\Sun\AppServer7\config\ on Windows), as part of the installation procedure. These input files are pre-populated with the values to set up a typical configuration, you can edit them as needed for your configuration.

This section addresses:

The clinstance.conf File

For clsetup to work, all application server instances that are part of a cluster must be defined in the clinstance.conf file. During installation, a clinstance.conf file is created with entries for two instances. If you add more instances to the cluster, you must add information about these additional instances as follows:

One set of entries is required for each instance that is part of the cluster. Any line that starts with a hash mark (#) is treated as a comment.

Note

The order in which these entries appear in the clinstance.conf file is important and must not be changed from the order specified here. If you add information about more application server instances, entries for these instances must appear in the same order. Comments can be added anywhere in the file.

Table 2-2 provides information about the entries in the clinstance.conf file. The left column contains the parameter name, the middle column defines the parameter, and the right column contains the default value.

Table 2-2  Entries in the clinstance.conf File 

Parameter

Definition

Default Value

instancename

Application Server instance name

server1, server2

user

Admin Server user name

admin

host

Host name

localhost

port

Admin Server port number

4848

domain

Administrative domain name

domain1

instanceport

Application Server instance port

80, 81

master

Master instance (used for cluster verification)

false

Example clinstance.conf File

This clinstance.conf file contains information about two instances.

#Instance 1

instancename server1
user admin
host localhost
port 4848
domain domain1
instanceport 80

#Instance 2

instancename server2
user admin
host localhost
port 4848
domain domain1
instanceport 81

The clpassword.conf File

When clsetup runs, it launches the asadmin command which needs the Admin Server password specified in the clpassword.conf file during installation.

The format of the clpassword.conf file is as follows:

where password is the Admin Server password.

Permissions 0600 are preset on the clpassword.conf file, which can only be accessed by the root user.

The clresource.conf File

During installation, the clresource.conf file is created to set up a typical configuration. The clresource.conf file contains information about the following resources that are part of the cluster:

On UNIX platforms, permissions 0600 are preset on the clresource.conf file, which can only be accessed by the root user.

Note

Before running clsetup, the values specified in the clresource.conf file can be modified for optimization, or for setting up a different configuration. If you edit the values, make sure that the order and format of the file is not changed.

Any line that begins with a hash mark (#) is treated as a comment.

The parameters of the clresource.conf file are described in the following tables. The left column contains the parameter name, the middle column defines the parameter, and the right column contains the default value.

Table 2-3 describes the HADB parameters in the clresource.conf File.

Table 2-3  HADB Parameters in the clresource.conf File 

Parameter

Definition

Default Value

historypath

Path for the history files.

/var/tmp

On Windows: REPLACEDIR (REPLACEDIR is replaced by the actual URL at runtime.)

devicepath

Path for the data and log devices.

/opt/SUNWappserver7/SUNWhadb/4

On Windows: C:\Sun\AppServer7\SUNWhadb\4.4.0-x

datadevices

Number of data devices on each node.

1

portbase

Port base number used for node 0. Other nodes are then assigned port number bases in increments of 20 from the number specified here (a random number in the range 10000 - 63000).

15200

spares

Number of spare nodes.

0

devicesize

Size of device in MB. This size is applicable to all devices.

512

dbpassword

Password for the HADB user.

password

hosts

All hosts used for all data nodes.

Values are populated automatically based on the hosts specified during installation.

The database name is specified at the end of the [HADBINFO] section in the clresource.conf file.

Table 2-4 lists the HADB Agent parameters in the clresource.conf file.

Table 2-4  

Parameter

Definition

Default Value

agent

Specifies the host and port number where the management agent is listening. This value can be modified to be a remote machine if HADB is located on a different machine than where Application Server is installed.

The value for the remote agent would be of the form:
hostname (or IPaddress):port. For example, nila.example.com:1862

localhost:1862

This value must be changed if the agent is started on another machine. Port settings can be changed in the file mgt.cfg located in install_dir/SUNWhadb/4.4.1-X/lib. The attribute to change in mgt.cfg is ma.server.jmxmp.port.

adminpassword

Password of the HADB Agent host administrator.

This is introduced in HADB 4.4 to enhance security.

password

Users may change this default value.

HADB Agent Information

Table 2-5 describes the session store parameters in the clresource.conf file.

Table 2-5  Session Store Parameters in the clresource.conf File 

Parameter

Definition

Default Value

storeurl

URL of the HADB store

REPLACEURL

NOTE: Value is replaced by actual URL at runtime.

storeuser

User who has access to the session store

appservusr

NOTE: Must match the username property in Table 2-6.

storepassword

Password for the storeuser

password

NOTE: Must match the password property in Table 2-6.

dbsystempassword

Password for the HADB system user

password

Table 2-6 describes the JDBC connection pool parameters in the clresource.conf file.

Table 2-6  JDBC Connection Pool Parameters in the clresource.conf File 

Parameter

Definition

Default Value

steadypoolsize

Minimum and initial number of connections maintained in the pool.

8

maxpoolsize

Maximum number of connections that can be created.

32

datasourceclassname

Name of the vendor-supplied JDBC datasource.

Name of the vendor-supplied JDBC datasources capable datasource class will implement javax.sql.XADatasource interface.

Non-XA or Local transactions only datasources will implement javax.sql.Datasource interface.

com.sun.hadb.jdbc.ds.HadbDataSource

isolationlevel

Specifies the transaction isolation level on the pooled database connections.

repeatable-read

isisolationguaranteed

Transaction isolation level guaranteed

true

validationmethod

Specifies the type of validation method.

meta-data

property

Property used to specify username, password, and resource configuration.

username=appservusr:password=password:cacheDataBaseMetaData=false:eliminateRedundantEndTransaction=true:serverList=REPLACEURL

NOTE: Make sure that the username and password properties use the same values as shown in the Session Store Parameters table. REPLACEURL is replaced by the actual URL at runtime.)

The connection pool name is specified at the end of the [JDBC_CONNECTION_POOL] section in the clresource.conf file.

Table 2-7 describes the JDBC resource parameters in the clresource.conf file.

Table 2-7  JDBC Resource Parameters in the clresource.conf File 

Parameter

Definition

Default Value

connectionpoolid

Name of the connection pool

appservCPL

NOTE: Connection pool name is specified in Table 2-6.

The JDBC resource name is defined at the end of the [JDBC_RESOURCE] section in the clresource.conf file.

Table 2-8 describes the session persistence parameters in the clresource.conf file.

Table 2-8  Session Persistence Parameters in the clresource.conf File 

Parameter

Definition

Default Value

type

Session persistence type

ha

frequency

Session frequency

web-method

scope

Session scope

session

store

Session store

jdbc/hastore

NOTE: Store name is defined at end of the [JDBC_RESOURCE] section.

Table 2-9 describes the stateful session bean parameter in the clresource.conf file.

Table 2-9  Stateful Session Bean Parameters in the clresource.conf File 

Parameter

Definition

Default Value

sfsb

Stateful session bean failover

false

Table 2-10 describes the RMI/IIOP failover parameter in the clresource.conf file.

Table 2-10  RMI/IIOP Failover Parameters in the clresource.conf File 

Parameter

Definition

Default Value

rmi_iiop

RMI/IIOP cluster configuration

false

Table 2-11 describes the cluster identification parameter in the clresource.conf file.

Table 2-11  Cluster Identification Parameters in the clresource.conf File 

Parameter

Definition

Default Value

cluster_id

Cluster ID

cluster1

Example clresource.conf File on UNIX

[HADBINFO]
agent         localhost:1862
historypath   /var/tmp
devicepath    /opt/SUNWappserver7/SUNWhadb/4
datadevices   1
portbase      15200
spares 0
devicesize 512
dbpassword password
hosts machine1,machine1
hadb

[AGENTINFO]
agent localhost       1862
adminpassword       password

[SESSION_STORE]
storeurl                 REPLACEURL
storeuser              appservusr
storepassword             password
dbsystempassword        password

[JDBC_CONNECTION_POOL]
steadypoolsize         8
maxpoolsize         32
datasourceclassname     com.sun.hadb.jdbc.ds.HadbDataSource
isolationlevel          repeatable-read
--isisolationguaranteed=true
validationmethod        meta-data
property username=appservusr:password=password:cacheDataBaseMetaData=false:eliminat eRedundantEndTransaction=true:serverList=REPLACEURL

appservCPL

[JDBC_RESOURCE]
connectionpoolid      appservCPL
jdbc/hastore

[SESSION_PERSISTENCE]
type       ha
frequency  web-method
scope      session
store      jdbc/hastore

[EJB_FAILOVER]

sfsb    true

[RMI_IIOP_FAILOVER]

rmi_iiop    true

[CLUSTER_ID]

cluster_id    cluster1

Example clresource.conf file on Windows

[HADBINFO]

agent localhost:1862

package V4.4.1-x

historypath   REPLACEDIR

devicepath    C:\Sun\AppServer7\SUNWhadb\4.4.1-x

datadevices    1

portbase    15200

spares    0

#set    LogbufferSize=32,DataBufferPoolSize=128

devicesize    208

dbpassword    password

hosts    machine1,machine2

adminpassword    password

hadb

[SESSION_STORE]

storeurl    REPLACEURL

storeuser    appservusr

storepassword    password

dbsystempassword    password

[JDBC_CONNECTION_POOL]

steadypoolsize    8

maxpoolsize    32

datasourceclassname    com.sun.hadb.jdbc.ds.HadbDataSource

isolationlevel     repeatable-read

--isisolationguaranteed=true

validationmethod    meta-data

property    username=appservusr:password=password:cacheDataBaseMetaData=fa lse:eliminateRedundantEndTransaction=true:serverList=REPLACEURL

appservCPL

[JDBC_RESOURCE]

connectionpoolid    appservCPL

jdbc/hastore

[SESSION_PERSISTENCE]

type    ha

frequency    web-method

scope    session

store    jdbc/hastore

[EJB_FAILOVER]

sfsb    true

[RMI_IIOP_FAILOVER]

rmi_iiop    true

[CLUSTER_ID]

cluster_id    cluster1

Running clsetup

The syntax for running clsetup is as follows:

If no arguments are specified, clsetup assumes the following defaults:

You can override these arguments by providing custom input file locations. For example:

When providing custom input files, follow the required format found in the input files. For information on doing this, see "Editing the clsetup Input Files," on page 60.

To run clsetup:

  1. Verify that the requirements have been met as described in Requirements and Limitations of clsetup.

    2. Note

    If you want to run clsetup as a user other than root, see Setting Up Administration for Non-Root to set this up.

  2. Verify that the input files have the required information to set up the cluster. If necessary, edit the input files following the guidelines in Editing the clsetup Input Files.
  3. Go to the Sun Java System Application Server installation /bin directory.
  4. Invoke the clsetup command:

    5. On Unix:./clsetup

    On Windows: clsetup.bat

    The clsetup command runs in verbose mode. The various commands are displayed on the screen as they run, and the output is redirected to the log file, /var/tmp/clsetup.log (on UNIX) or the default Windows temp directory (on Windows).

    If a vital error occurs, the configuration stops and the error is recorded in the log file. If the log file already exists, the output is appended to the existing log file.

    If the entities to be handled (HADB nodes and Application Server instances) already exist, clsetup does not delete or reconfigure them, and the respective configuration steps are skipped. This type of event is recorded in the log file.

  5. When clsetup completes the configuration, scan the log file after each run to review the setup.
  6. Upon completion, clsetup returns the exit codes described in Table 2-12:

    7. Table 2-12  Exit Codes for the clsetup Command 

    Exit Code

    Description

    0

    Successful exit

    2

    Usage error

    3

    Instance file not found

    4

    Instance file cannot be read

    5

    Resource file not found

    6

    Resource file cannot be read

    7

    Password file not found

    8

    Password file cannot be read

    10

    Script cannot find asadmin

    11

    Script cannot find hadbm

    12

    Cannot create temporary file

    13

    Session store configuration failed

    14

    Create HADB failed

    15

    HADB get jdbcURL failed

    16

    User exits in welcome message

Cleanup Procedures for clsetup

After running clsetup, errors that have occurred are logged in the log file /var/tmp/clsetup.log. Examine the log file after every run of the clsetup command and correct any significant errors that are reported (for example, failure to create a non-existing instance).

You can undo all or part of the configuration as follows:

See the Man pages for detailed examples of each of these commands. You are now ready to proceed to the Sun Java System Application Server Administration Guide for instructions on configuring the HADB and managing the cluster, the load balancer plug-in, and the HADB.



Previous      Contents      Index      Next     

Copyright 2005 Sun Microsystems, Inc. All rights reserved.