Sun ONE Application Server 7, Enterprise Edition Installation Guide |
Chapter 3
Preparing for HADB SetupAfter the high-availability components have been installed on the servers that will be part of an cluster, perform the tasks in this chapter to prepare for setting up high availability. Refer to "Installation Roadmap" to see the full sequence of events for implementing the Sun ONE Application Server 7, Enterprise Edition product.
The following topics are addressed here:
After you have done the tasks here, proceed to the Sun ONE Application Server Administrator’s Guide for comprehensive instructions on configuring and managing the cluster, the load balancer plug-in, and the high-availability database (HADB).
Information on high-availability topologies is available in the Sun ONE Application Server System Deployment Guide.
Configuring Shared Memory and SemaphoresYou will need to configure shared memory for the HADB host machines before beginning to work with the HADB.
- Log in as root.
Note
If necessary, make sure permissions are set correctly to administer the HADB as non-root user. See "Setting up RSH for HADB Administration", Step 5.
- Add lines similar to the following to the /etc/system file if they do not already exist:
For shared memory:
set shmsys:shminfo_shmmax=0x80000000
set shmsys:shminfo_shmseg=20This example sets maximum shared memory shmmax to 2GB (hexadecimal 0x80000000) which is sufficient for most configurations.
The shmsys:shminfo_shmmax setting is calculated as 10,000,0000 per 256 MB and should set to be identical to the memory size for the host. To determine your host’s memory, run this command:
prtconf | grep Memory
Then plug the value into the following formula:
((<host> MB / 256 MB) * 10,000,000)
For semaphores:
Your /etc/system file may already contain semmni, semmns, and semmnu entries. For example:
set semsys:seminfo_semmni=10
set semsys:seminfo_semmns=60
set semsys:seminfo_semmnu=30
If the entries are present, increment the values by adding 16, 128, and 1000 respectively, as follows:
set semsys:seminfo_semmni=26
set semsys:seminfo_semmns=188
set semsys:seminfo_semmnu=1030If your /etc/system file does not contain the above mentioned entries, add the following entries at the end of the file:
set semsys:seminfo_semmni=16
set semsys:seminfo_semmns=128
set semsys:seminfo_semmnu=1000This is sufficient to run up to 16 HADB nodes on the computer.
- Reboot the machine for changes to take effect.
For an explanation of HADB nodes, see Configuring the High Availability Databasein the Sun ONE Application Server Administrator’s Guide.
Setting Up Host CommunicationTo implement remote access for HADB administration, all machines that will be used for running HADB servers and the HADB management client must be configured for Remote Shell (RSH) or Secured Shell (OpenSSH/SSH).
RSH is a simple remote shell command and does not have any security features. The SSH communication channel provides a level of security by encrypting the data that passes between the HADB nodes.
Note
For Solaris 9, it is recommended that you use the default installation of SSH. However, you can use RSH if preferred by following the instructions in "Setting up RSH for HADB Administration" and then editing the clresource.conf file to specific RSH as described in "Running the clsetup Command".
On Solaris 8, by default SSH is not installed. Follow the instructions in "Installing SSH for Solaris 8" if SSH is not on your Solaris 8 system.
If you want to use SSH, but it is not configured or not available, you will not be able to use the hadbm command. Refer to "SSH Requirements and Limitations" to verify that SSH is recognized.
This section contains instructions for the following tasks:
Setting up RSH for HADB Administration
If you want to use RSH instead of SSH, you must explicitly specify RSH using the set managementProtocol option. Refer to Table 3-3 for guidelines on setting this parameter in the clresource.conf file.
Note
SSH is the strongly recommended default for the hadbm create command because SSH is more secure than RSH.
To implement RSH, perform these steps:
- Log in as root.
- Edit the /etc/hosts file to contain entries for all the selected HADB hosts, including the host name of the local host. Use localhost format. For example:
computer1.xbay.company.com
computer99.zmtn.company.com- Append this file to the /etc/hosts file of all selected installation hosts.
- Create a .rhosts file in the $HOME directory of the HADB user, if one does not already exist.
vi .rhosts
- Verify that permissions are set to Read Only for group and other. For example:
rw-r--r--
- Add the host name of each HADB host, including the name of your local host, followed by the name of your database user. For example, if the database user is Jon:
computer1.xbay.company.com Jon
computer99.zmtn.company.com Jon
mine456.red.mycompany.com Jon- Append this file to the .rhosts file of each HADB host.
- Check host communication for each host. For example:
rsh computer99.zmtn.company.com uname -a
If all is well, the identity will be returned from the other host.
Setting Up SSH for HADB Administration
SSH is strongly recommended for using the hadbm create command because SSH is more secure than RSH.
This section contains the following sections:
SSH Requirements and Limitations
Note
Although SSH is installed by default on Solaris 9 systems, on Solaris 8, by default SSH is not installed. Instructions for installing SSH for Solaris 8 are contained in "Installing SSH for Solaris 8".
You may need to take action on any or all of the following requirements during your SSH setup:
/usr/bin
- If the binaries are on your system but this location is not correct, you will need to make a symbolic link from /usr/bin to the correct location.
- If you are on a Solaris 8 system, the SSH binaries are not installed by default and so may not be present. If this is the case, follow the instructions in "Installing SSH for Solaris 8" on page 59.
- Support—The only tested support is for SunSSH and OpenSSH. If you are using another version of SSH, it is best to refer to the setup instructions in that product’s documentation to ensure that your SSH communications work correctly.
- OpenSSH clients and daemons—If you are running in an environment with OpenSSH clients and daemons, you should name the key file as follows:
~/.ssh/authorized_keys2 or ~/.ssh/authorized_keys.
- Running as root—If you are running the HADB admin clients as root, make sure that the sshd configuration (/etc/ssh/sshd_config) on all machines has the PermitRootLogin parameter set to yes.
- No SSH protocol version 2 support—If your SSH clients and daemons do not support SSH protocol version 2, you will need to run ssh-keygen without options. The key file will then be named identity.pub instead of id_dsa.pub. This file must be appended to ~/.ssh/authorized_keys.
- Mixed SSH environment—If you are operating in a mixed SSH environment, you will need to create both files ~/.ssh/authorized_keys2 and ~/.ssh/authorized_keys; the latter may contain both version 1 and version 2 keys.
- Co-location—If the Sun ONE Application Server and the HADB are co-located on the same machine, you will need to create a known_hosts file under the .ssh directory by running one of the following commands:
ssh localhost
or
ssh hostname
Installing SSH for Solaris 8
The ssh and scp binaries are not installed by default on Solaris 8 systems. If the binaries are not on your Solaris 8 system, perform these steps:
- Go to the following site:
http://www.sunfreeware.com/openssh8.html
On this site, you may receive a message similar to the following:
===PLEASE NOTE!!!............ make a note of some of the mirror sites so that if the servers are down, you can still download from a mirror site.
If you receive such a message, try one of the many mirror sites listed in the FTP/Mirror Sites link. For example:
http://sunfreeware.secsup.org/
- On this site, follow the instructions in the Installation Steps to download and install all the necessary OpenSSH packages and patches.
- After you have installed OpenSSH, proceed to the next section on Configuring SSH.
Configuring SSH
To set up SSH on a system where the ssh and scp binaries are already installed, perform the steps in one of the following sections:
SSH for Non-Mounted Home Directories
To implement SSH in systems with home directories that are not mounted, perform these steps:
- Verify that SSH requirements have been understood and met as specified in "SSH Requirements and Limitations".
- Log in to the host as the HADB user.
- Generate your keys by running the following:
ssh-keygen -t dsa
For SSH1 and OpenSSH/1, you normally do not need to give any parameters to the ssh-keygen command.
- For the next three prompts, accept the default options by pressing Enter.
- Repeat steps 1, 2, and 3 for all machines in your cluster.
A file called identity.pub or id_dsa.pub (depends on whether you are using SSH version 1 or version 2) located in your ~/.ssh directory holds the public key. To connect to a machine without being asked for a password, the content of this file must be appended to a file called authorized_keys on all the machines.
- To set up login identity, go to your user directory:
~/.ssh.
For SSH1, OpenSSH/1:
- Copy the identity.pub file and name it authorized_keys.
- For each of the other machines in the cluster, copy the content of the identity.pub file and append it to the local authorized_keys file.
OpenSSH/2:
- Copy the id_dsa.pub file and name it authorized_keys2.
- For each of the other machines in the cluster, copy the content of the id_dsa.pub file and append it to the local authorized_keys2 file.
- Copy the authorized_keys file to the ~/.ssh directory on all the HADB machines.
- Verify that the .ssh directory, HADB user’s home directory, and the .ssh/authorized_keys file do not have write permissions for group and other.
If needed, disable these group/other write permissions as follows:
chmod og-w ~/.ssh
chmod og-w ~/.ssh/authorized_keys
chmod og-w $HOMEReplace $HOME with the home directory of the HADB user. For example:
chmod og-w ~/johnsmith
- To enable login without any user input, at initial SSH usage (after the SSH environment is set up) you need to add the node machine name to the known_hosts file under the /.ssh directory as follows:
- To verify that SSH is set up correctly, SSH to each host in the cluster before trying to run the management tool for HADB.
You are automatically logged in without a password requirement.
SSH for Mounted Home Directories
To implement SSH in systems with mounted home directories, perform these steps:
- Verify that SSH requirements have been understood and met as specified in "SSH Requirements and Limitations"
- Log in to host as the HADB user.
- Generate your keys by running the following:
ssh-keygen -t dsa
For SSH1 and OpenSSH/1, you normally do not need to give any parameters to the ssh-keygen command.
- For the next three prompts, accept the default options by pressing Enter.
A file called identity.pub or id_dsa.pub (depends on whether you are using SSH version 1 or version 2) located in your ~/.ssh directory holds the public key. To connect to a machine without being asked for a password, the content of this file must be appended to a file called authorized_keys2 on all the machines. This can be done as follows:
- To set up login identity, go to your user directory:
~/.ssh.
For SSH1, OpenSSH/1—Copy the identity.pub file and name it authorized_keys.
For OpenSSH/2—Copy the id_dsa.pub file and name it authorized_keys.
- Verify that the .ssh directory and the .ssh/authorized_keys file do not have write permissions for group and other.
If necessary, disable these group/other write permissions as follows:
chmod og-w ~/.ssh
chmod og-w ~/.ssh/authorized_keys
chmod og-w /$HOMEReplace HOME with the home directory of the HADB user. For example:
chmod og-w ~/johnsmith.
- To enable login without any user input, at initial SSH usage (after the SSH environment is set up) you need to add the node machine name to the known_hosts file under the /.ssh directory
- To verify that SSH is set up correctly, SSH to each host in the cluster before trying to run the management tool for HADB.
You are automatically logged in without a password requirement.
Setting Up the User EnvironmentAfter you have set up host communication, you can run the hadbm command from the install_dir/SUNWhadb/4/bin directory location as follows:
However, it is much more convenient to set up your local environment to use the high-availability management client commands from anywhere. To set this up, perform the following steps.
Note
The examples in this section apply to using csh. If you are using another shell, refer to the man page for your shell for instructions on setting variables.
- Set the PATH variable as follows.
setenv PATH ${PATH}:install_dir/bin:install_dir/SUNWhadb/4/bin
- Verify that the PATH settings are correct by running the following commands:
which asadmin
which hadbm- If multiple Java versions are installed on your system, you must ensure that the JAVA_HOME environment variable points to the correct Java version (1.4.1_03 for Enterprise Edition).
setenv JAVA_HOME java_install_dir
setenv PATH ${PATH}:${JAVA_HOME}/bin
Setting Up Administration for Non-RootBy default, during the initial installation or setup of the Sun ONE Application Server, write permissions of the files and paths created for Sun ONE Application Server are given to root only. For a user other than root to create or manage the Sun ONE Application Server, write permissions on the associated files must be given to that specific user, or to a group to which the user belongs. The files that are affected are the following (with their default locations):
You can create a user group for managing the Sun ONE Application Server as described in the following procedure. (An alternate approach is to set permissions and ownership for the specific user.)
To create a Sun ONE Application Server user group and set permissions on the installation root directory, repeat the following process for each affected file:
- Log in as root.
- From the command prompt, create the Sun ONE Application Server user group. For example:
# groupadd s1asuser
You can type groupadd at the command line to see appropriate usage.
- Change the group ownership for each affected file to the newly-created group. For example:
chgrp -R s1asuser install_config_dir/cl*.conf
- Set the write permission for the newly-created group:
chmod -R g+rw install_config_dir/cl*.conf
- Repeat steps 3 and 4 for each affected file.
- Make the clsetup and cladmin commands executable by the newly-created group. For example:
chmod -R g+x install_dir/bin/cl*
- Delete and recreate the default domain, domain1, using the --sysuser option. The sysuser must also belong to the newly-created group. For example:
asadmin delete-domain domain1
asadmin create-domain --sysuser bleonard --adminport 4848 --adminuser admin --adminpassword password domain1
Using the clsetup CommandThe purpose of the clsetup command is to automate the process of setting up a basic cluster in a typical configuration. The clsetup command is located in install_dir/bin, where install_dir is the directory where the Sun ONE Application Server software is installed.
The clsetup command is bundled with the Sun ONE Application Server software along with the cladmin command.
The following topics are addressed in this section:
How the clsetup Command Works
The clsetup command is a set of Sun ONE Application Server commands that are gathered together in a script that allows a cluster to be configured automatically, based on prepopulated input files. As part of cluster setup, an HADB is created, but you will still need to set up your working cluster using the hadbm commands as described in the Sun ONE Application Server Administrator’s Guide.
The following topics are addressed in this section:
How the Input Files Work
Three input files are used by the clsetup command to configure the cluster:
- clinstance.conf—This file is pre-populated with information about application server instances server1 and server2. Refer to "The clinstance.conf File" for information on the contents of this file.
- clpassword.conf—This file is pre-populated with the Admin Server password for domain1, which you provided when you installed the Sun ONE Application Server 7, Enterprise Edition software. Refer to "The clpassword.conf File" for information on the contents of this file.
- clresource.conf—This file is pre-populated with information about the cluster resources: HADB, JDBC connection pool, JDBC resource, and session store and persistence. Refer to "The clresource.conf File" for information on the contents of this file.
You can use the clsetup configuration parameters as they are preconfigured to set up a typical cluster configuration. To support a different configuration, you can make edits to any or all of the configuration files.
What the clsetup Command Accomplishes
Using the pre-populated values in the clsetup input files, the clsetup command accomplishes the following:
- Creates a new server instance named server2 in the default domain named domain1. The HTTP port number for server2 is the next sequential number after the HTTP port number specified for server1 during installation (for example, if port number 80 is provided for server1 during installation, the port number for server2 is 81).
- Creates the HADB named hadb with two nodes on the local machine. The port base is 15200, and the database password is password.
- Creates the HADB tables required to store session information in the HADB.
- Creates a connection pool named appservCPL in all the instances listed in the clinstance.conf file (server1, server2).
- Creates a JDBC resource named jdbc/hastore in all the instances listed in the clinstance.conf file (server1, server2).
- Configures the session persistence information in all the instances listed in the clinstance.conf file (server1, server2).
- Enables high availability in all the instances listed in the clinstance.conf file (server1, server2).
Commands Used by the clsetup Command
The clsetup command uses a number of hadbm and asadmin commands to perform the steps for setting up the cluster. In Table 3-1, the clsetup task is described in the left column and the command used to accomplish the task is listed in the right column.
clsetup Requirements and Limitations
The following requirements and limitations apply to the clsetup command:
- The install paths, device paths, configuration paths, and so on must be the same on all machines that are of the cluster.
- Before you can use the clsetup command, the asadmin and hadbm commands must be available on the local machine. Therefore, this command can only be run on a machine where the following are installed:
- Before you can use the clsetup command, you must have configured shared memory as described in "Configuring Shared Memory and Semaphores". The clsetup command does not set any shared memory values.
- Before you can use the clsetup command, you must have set up the HADB cluster host communication for SSH or RSH as described in "Setting Up Host Communication".
- If you are using RSH (which is not the default), you will need to uncomment the following line in the clresource.conf file (remove the # sign):
- If you are co-locating the Application Server and the HADB on the same machine using SSH, a known_hosts file must exist under the .ssh directory. If it does not, run either the ssh localhostor the ssh hostname command before using the clsetup command.
- Before running the clsetup command, you must start the Admin Servers of all the Sun ONE Application Server instances that are part of the cluster.
- The administrator password must be the same for all domains that are part of the cluster.
- If the entities to be handled (HADB nodes and Application Server instances) already exist, the clsetup command does not delete or reconfigure them, and the respective configuration steps are skipped.
- The values specified in the input files will be the same for all the instances in a cluster. The clsetup command is not designed to set up instances with different values. For example, this command cannot create a JDBC connection pool with different settings for each instance.
- The clsetup command does not perform any inetd configuration; the HADB is created with no inetd settings. Instructions for performing inetd configuration are contained in the Sun ONE Application Server Administrator’s Guide.
- Host names in the shell initialization files—If prompts are included with host names in your .cshrc or .login files, the clsetup command may appear to hang. You will need to remove any prompts and excess output in any remote command invocations. For example, running the hostname command on hostB should print hostB without a prompt.
- To run the clsetup command as a user other than root, you'll need to make the changes as described in "Setting Up Administration for Non-Root".
Editing the clsetup Input Files
The input files that are needed for the clsetup command are installed under the configuration installation directory, default /etc/opt/SUNWappserver7, as part of the installation procedure. The installation program pre-populates these files with the values to set up a typical configuration, but you can edit any or all of them as needed using a text editor.
This section addresses the following topics:
The clinstance.conf File
For the clsetup command to work properly, all application server instances that are part of a cluster must be defined in the clinstance.conf file. During installation, the installation program creates a clinstance.conf file with entries for two instances. If you add more instances to the cluster, you must add information about these additional instances.
The format of the clinstance.conf file is as follows:
One set of entries is required for each instance that is part of the cluster. Any line that starts with a hash mark (#) is treated as a comment.
Table 3-2 provides information about the entries in the clinstance.conf file. The left column contains the parameter name, the middle column defines the parameter, and the right column contains the default value specified by the installation program.
Example clinstance.conf File
This clinstance.conf file contains information about two instances.
#Instance 1
instancename server1
user admin
host localhost
port 4848
domain domain1
instanceport 80#Instance 2
instancename server2
user admin
host localhost
port 4848
domain domain1
instanceport 81The clpassword.conf File
When the clsetup command is run, the asadmin command needs the Admin Server password, which is specified in the clpassword.conf file during installation.
The format of the clpassword.conf file is as follows:
where password is the Admin Server password.
Permissions 0600 are preset on the clpassword.conf file, which can only be accessed by the root user.
The clresource.conf File
During installation, the installation program creates the clresource.conf file to set up a typical configuration. The clresource.conf file contains information about the following resources that are part of the cluster:
Permissions 0600 are preset on the clresource.conf file, which can only be accessed by the root user.
The parameters of the clresource.conf file are described in the following tables. The left column contains the parameter name, the middle column defines the parameter, and the right column contains the default value specified by the installation program.
Table 3-3 describes the HADB parameters in the clresource.conf file.
Note
The database name is specified at the end of the [HADBINFO] section in the clresource.conf file.
Table 3-4 describes the session store parameters in the clresource.conf file.
Table 3-4 Session Store Parameters in the clresource.conf File
Parameter
Definition
Default Value
storeurl
URL of the HADB store
REPLACEURL
NOTE: Value is replaced by actual URL at runtime.
storeuser
User who has access to the session store
appservusr
NOTE: Must match the username property in Table 3-5.
storepassword
Password for the storeuser
password
NOTE: Must match the password property in Table 3-5.
dbsystempassword
Password for the HADB system user
password
Table 3-5 describes the JDBC connection pool parameters in the clresource.conf file.
Note
The connection pool name is specified at the end of the [JDBC_CONNECTION_POOL] section in the clresource.conf file.
Table 3-6 describes the JDBC resource parameters in the clresource.conf file.
Table 3-6 JDBC Resource Parameters in the clresource.conf File
Parameter
Definition
Default Value
connectionpoolid
Name of the connection pool
appservCPL
NOTE: Connection pool name is specified in Table 3-5.
Note
The JDBC resource name is defined at the end of the [JDBC_RESOURCE] section in the clresource.conf file.
Table 3-7 describes the session persistence parameters in the clresource.conf file.
Example clresource.conf File
[HADBINFO]
historypath /var/tmp
devicepath /opt/SUNWappserver7/SUNWhadb/4
datadevices 1
portbase 15200
spares 0
#set managementProtocol=rsh
inetd false
inetdsetupdir /tmp
devicesize 512
dbpassword password
hosts machine1,machine1
hadb[SESSION_STORE]
storeurl REPLACEURL
storeuser appservusr
storepassword password
dbsystempassword password[JDBC_CONNECTION_POOL]
steadypoolsize 8
maxpoolsize 32
datasourceclassname com.sun.hadb.jdbc.ds.HadbDataSource
isolationlevel repeatable-read
validationmethod meta-data
property username=appservusr:password=password:cacheDataBaseMetaData=false:e liminateRedundantEndTransaction=true:serverList=REPLACEURLappservCPL
[JDBC_RESOURCE]
connectionpoolid appservCPL
jdbc/hastore[SESSION_PERSISTENCE]
type ha
frequency web-method
scope session
store jdbc/hastoreRunning the clsetup Command
The syntax for running the clsetup command is as follows:
If no arguments are specified, the clsetup command assumes the following defaults:
You can override these arguments by providing custom input file locations. For example:
./clsetup --resourcefile /tmp/myappservresource.conf
Note
When providing custom input files, follow the required format found in the input files. For information on doing this, see "Editing the clsetup Input Files" on page 70.
To run the clsetup command, perform the following steps:
- Verify that the requirements have been met as described in "clsetup Requirements and Limitations".
Note
If you want to run the clsetup command as a user other than root, follow the instructions in "Setting Up Administration for Non-Root" to set this up.
- Verify that the input files have the information that is required to set up the cluster. If necessary, edit the input files following the guidelines in "Editing the clsetup Input Files".
- If you are using RSH, edit the clresource.conf file to uncomment the following line (remove the # sign):
#set managementProtocol
- Go to the Sun ONE Application Server installation /bin directory:
cd install_dir/bin
- Invoke the clsetup command using the appropriate syntax. For example, to run the command using the defaults:
./clsetup
The clsetup command displays the welcome message, the prerequisites for configuring the cluster, and the following message:
Do you want to start configuring your cluster? [Yes/No]
- To start configuring, type Yes and press Enter.
The clsetup command runs in verbose mode. The various commands are displayed on the screen as they run, and the output is redirected to the log file, /var/tmp/clsetup.log.
If a vital error occurs (for example, failure to create a non-existing HADB), the configuration stops and the error is recorded in the log file. If the log file already exists, the output is appended to the existing log file.
- When the clsetup command completes the configuration, you are advised about the location of the log file. It’s a good idea to scan the log file after each run.
- Upon completion, the clsetup command returns the exit codes as described in Table 3-8:
You can obtain a list of the exit codes by running the following command from the command line immediately after running the clsetup command:
Cleanup Procedures for the clsetup Command
After running the clsetup command, errors that have occurred are logged in the log file /var/tmp/clsetup.log. Examine the log file after every run of the clsetup command and correct any significant errors that are reported (for example, failure to create a non-existing instance).
You can undo all or part of the configuration as follows:
After you have completed the tasks in this chapter (and the post-installation tasks in the following chapter, if needed), proceed to the Sun ONE Application Server Administrator’s Guide for instructions on configuring the HADB and managing the cluster, the load balancer plug-in, and the HADB.