How to Install and Configure Oracle Application Server (Sun Cluster Data Service for Oracle Application Server Guide for Solaris OS)

Sun Cluster Data Service for Oracle Application Server Guide for Solaris OS

Previous: Planning the Sun Cluster HA for Oracle Application Server Installation and Configuration
Next: Verifying the Installation and Configuration of Oracle Application Server

How to Install and Configure Oracle Application Server

Caution –

Oracle 9iAS (version 9.0.2 and 9.0.3) and Oracle 10g AS (version 9.0.4) does not support installation on a hardware cluster. So for the duration of the install and post install configuration, Sun Cluster must be stopped. You should have access to the console before the next step.Your data service configuration might not be supported if you do not adhere to these requirements.

Note –

This sections assumes you have setup a Cluster File System to support either the Oracle 9iAS Infrastructure or Oracle 10g AS Infrastructure installation.

Edit /etc/vfstab to temporarily remove the global mount option

As the Infrastructure mount point will be mounted while the node is in non-cluster mode you must edit /etc/vfstab to temporarily remove the global mount option, if the Infrastructure mount point is a Global File System.

It is recommended that you repeat the line and comment out the original entry and amend the copied entry to omit the global option.

Stop the Cluster
# scshutdown -g0 -y

Boot each Cluster Node in Non-Cluster Mode
ok boot -x

Take ownership of the Infrastructure File System diskset/diskgroup

For Solaris Volume Manager

#  metaset -s <diskset> -f -C take 
#  mount  <Infrastructure mount point>

For Veritas Volume Manager

#  vxdg -C import <diskgroup> 
#  vxdg -g <diskgroup> startall
#  mount  <Infrastructure mount point>

Plumb the Infrastructure logical IP address

#  ifconfig <interface> addif <logical host> up

Install the Oracle 9iAS Infrastructure

Note –

Refer to Step 7 if you are installing the Oracle 10g AS Infrastructure

For this section, follow the Oracle Application Server, Installation Guide manual to install Oracle Application Server .

Enable and test the interposed logical host

This step requires that the Sun Cluster HA for Oracle Application Server data service has been installed on all nodes that will be part of the Sun Cluster.

If you did not install the Sun Cluster HA for Oracle Application Server package during your initial Sun Cluster installation, proceed to Installing the Sun Cluster HA for Oracle Application Server Packages before you continue with this step.

Note –

Oracle 9iAS Infrastructure requires that the logical host is interposed for any gethostname() calls. You must carry out this step on all nodes that will be part of the Sun Cluster. The logical hostname being set in LHOSTNAME is the short name not the fully qualified name.

# cd /opt/SUNWsc9ias/lib
# cp libloghost_32.so.1 /usr/lib/libloghost_32.so.1 
# cp libloghost_64.so.1 /usr/lib/sparcv9/libloghost_64.so.1
#
# cd /usr/lib/secure
# ln -s /usr/lib/libloghost_32.so.1 libloghost_32.so.1
#
# cd /usr/lib/secure/sparcv9
# ln -s /usr/lib/sparcv9/libloghost_64.so.1  libloghost_64.so.1  
#
# su - <Oracle Application Server Userid> 
$ LD_PRELOAD_32=libloghost_32.so.1
$ LHOSTNAME=<logical hostname>
$ export LHOSTNAME  LD_PRELOAD_32
$
$ uname -n
<logical host>     <- The logical host should be returned

Setting ORACLE_HOME during the install

When prompted for the ORACLE_HOME variable during the install, point to a directory on the Infrastructure File System you mounted in Step 4.

The hostname picked up by runInstaller will be the logical hostname you plumbed in Step 5

Pre-task before running the configuration assistants

After installing the software, the Oracle 9iAS installer runs the configuration assistants for the various Oracle 9iAS Infrastructure components. Prior to starting the configuration assistants, the installer will prompt to execute a script — root.sh. Before running the root.sh script, the following needs to be done in another window. This is because OHS uses configuration information defined in apachectl and does not inherit the variable values LD_PRELOAD_32 and LHOSTNAME.

Add the following three lines to the CONFIGURATION section in apachectl (you can add these three lines just before the PIDFILE= setting in the file).

$ cd $ORACLE_HOME/Apache/Apache/bin 
$
$ vi apachectl

and add the following three lines to the CONFIGURATION section in apachectl 
(you can add these three lines just before the PIDFILE= setting)

LD_PRELOAD_32=libloghost_32.so.1
LHOSTNAME=<logical hostname>
export LHOSTNAME LD_PRELOAD_32

Install Patches

Note –
It is important that you follow the patch installations very carefully within the patch README files. Failure to do so will mean that certain functions within the Sun Cluster HA for Oracle Application Server will not work correctly.

Issues have been traced back to patches not being installed correctly. Oracle 9iAS version 9.0.2 requires that it is patched correctly before being used within Sun Cluster, including any post patch installation steps. Time spent here will be worthwhile in the end.

Tip –
When installing patches it is worthwhile checking $ORACLE_HOME/opmn/conf/opmn.xml to see if any more sections have been added by the patch. You should perform this check only if you are requested to run a root.sh script.

If you are requested to run a root.sh script, then before you hit OK for that script, you should check $ORACLE_HOME/opmn/conf/opmn.xml. If any new sections have been added please include LD_LIBRARY_PATH, LD_PRELOAD_32, LHOSTNAME and DISPLAY as listed within the next step, before hitting enter to the OK message.

Check the latest release notes for any post install patches.

Edit opmn.xml

Stop opmn by issuing the following command

$ $ORACLE_HOME/opmn/bin/opmnctl stopall

Edit the $ORACLE_HOME/opmn/conf/opmn.xml file and add entries for the LD_PRELOAD_32 and LHOSTNAME environment variables in the environment section for OC4J_DAS, home, OC4J_Demos, CUSTOM, etc. For example

Note –

In the example below, the LD_LIBRARY_PATH value should be replaced by your $ORACLE_HOME/lib, the DISPLAY value replaced with your logical hostname and the LHOSTNAME replaced by the fully qualified logical hostname.

 $ cat /global/ora9ias/infra/opmn/config/opmn.xml
<ias-instance xmlns="http://www.oracle.com/ias-instance">
  <notification-server>
    <port local="6100" remote="6200" request="6003"/>
    <log-file path="/global/ora9ias/infra/opmn/logs/ons.log" level="3"/>
  </notification-server>
  <process-manager>
    <ohs gid="HTTP Server" maxRetry="3">
      <start-mode mode="ssl"/>
    </ohs>
    <oc4j maxRetry="3" instanceName="home" numProcs="1">
      <config-file path="/global/ora9ias/infra/j2ee/home/config/server.xml"/>
      <oc4j-option value="-properties"/>
      <port ajp="3000-3100" jms="3201-3300" rmi="3101-3200"/>
      <environment>
        <prop name="DISPLAY" value="ora9ias:0.0"/>
        <prop name="LD_LIBRARY_PATH" value="/global/ora9ias/infra/lib"/>
        <prop name="LHOSTNAME" value="ora9ias.com"/>
        <prop name="LD_PRELOAD_32" value="libloghost_32.so.1"/>
      </environment>
    </oc4j>
    <oc4j maxRetry="3" instanceName="OC4J_DAS" gid="OC4J_DAS" numProcs="1">
      <config-file path="/global/ora9ias/infra/j2ee/OC4J_DAS/config/server.xml"/>
      <java-option value="-server -Xincgc -Xnoclassgc -Xmx256m "/>
      <oc4j-option value="-properties"/>
      <port ajp="3001-3100" jms="3201-3300" rmi="3101-3200"/>
      <environment>
<prop name="DISPLAY" value="ora9ias:0.0"/>
        <prop name="LD_LIBRARY_PATH" value="/global/ora9ias/infra/lib"/>
        <prop name="LHOSTNAME" value="ora9ias.com"/>
        <prop name="LD_PRELOAD_32" value="libloghost_32.so.1"/>
       </environment>
    </oc4j>
    <oc4j maxRetry="3" instanceName="OC4J_Demos" gid="OC4J_Demos" numProcs="1">
      <config-file path="/global/ora9ias/infra/j2ee/OC4J_Demos/config/server.xml"/>
      <java-option value="-Xmx512M "/>
      <oc4j-option value="-userThreads  -properties"/>
      <port ajp="3001-3100" jms="3201-3300" rmi="3101-3200"/>
      <environment>
        <prop name="%LIB_PATH_ENV%" value="%LIB_PATH_VALUE%"/>
<prop name="DISPLAY" value="ora9ias:0.0"/>
        <prop name="LD_LIBRARY_PATH" value="/global/ora9ias/infra/lib"/>
        <prop name="LHOSTNAME" value="ora9ias.com"/>
        <prop name="LD_PRELOAD_32" value="libloghost_32.so.1"/> 
       </environment> 
    </oc4j>
    <custom gid="dcm-daemon" numProcs="1" noGidWildcard="true">
      <start path="/global/ora9ias/infra/dcm/bin/dcmctl daemon -logdir 
	/global/ora9ias/infra/dcm/logs/daemon_logs"/>
      <stop path="/global/ora9ias/infra/dcm/bin/dcmctl shutdowndaemon"/>
      <environment>
<prop name="DISPLAY" value="ora9ias:0.0"/>
        <prop name="LD_LIBRARY_PATH" value="/global/ora9ias/infra/lib"/>
        <prop name="LHOSTNAME" value="ora9ias.com"/>
        <prop name="LD_PRELOAD_32" value="libloghost_32.so.1"/>              
</environment>
    </custom>
    <log-file path="/global/ora9ias/infra/opmn/logs/ipm.log" level="3"/>
  </process-manager>
</ias-instance>

Restart OPMN

$  $ORACLE_HOME/opmn/bin/opmnctl startall

Update the OPMN configuration

$  $ORACLE_HOME/dcm/bin/dcmctl updateConfig -ct opmn

Stop all the Infrastructure processes

After the install, all the Infrastructure processes will be started. These need to be stopped so that they can be started under the control of Sun Cluster.

Install the Oracle 10g AS Infrastructure

Note –
Refer to Step 6 if you are installing the Oracle 9iAS Infrastructure

For this section, follow the Oracle Application Server, Installation Guide manual to install Oracle Application Server.
1. Setting ORACLE_HOME during the install
  
  When prompted for the ORACLE_HOME during the install, point to a directory on the Infrastructure File System you mounted in Step 4.
2. Install Patches
  
  Check the latest release notes for any post install patches.
3. Stop all the Infrastructure processes
  
  After the install, all the Infrastructure processes will be started. These need to be stopped so that they can be started under the control of Sun Cluster.

Prepare the other Cluster Nodes

Since the ORACLE_HOME is on shared storage, there is no need to install the software again on the other nodes within the cluster. Once the Oracle 9iAS or Oracle 10g AS Infrastructure file system is under the control of Sun Cluster, either as a Global File System or Failover File System, the Infrastructure files can be made available to other nodes within Sun Cluster.

It is however necessary to update the oratab and emtab files in /var/opt/oracle on the other nodes within the cluster. If there are no other Oracle database instances on the other nodes, simply copy the two files from the first node to the second node etc. If there are other Oracle database instances, then add the iasdb entries from the oratab and emtab files on node 1 to the same files on node 2 etc. Create the files on node 2 if necessary.

Preparation if installing the Middle Tier on the same nodes as the Infrastructure.

Unlike the Oracle 9iAS or Oracle 10g AS Infrastructure, the middle tier may be installed on multiple active nodes to achieve high availability. Typically the middle tier and infrastructure are installed on separate nodes. However, you may wish to install the middle tier on the node(s) where the clustered infrastructure has been installed. This can be done by installing the middle tier on local disks of the nodes that could host the infrastructure.

However, whenever the middle tier and infrastructure share a cluster node, two "/var/opt/oracle" areas must be maintained. One for the infrastructure where the "oraInst.loc" points to the "oraInventory" directory on the shared disk and another for the middle tier instance installed on local disk on each cluster node where it's "oraInst.loc" points to another "oraInventory" directory on local disk of that node. These directories are needed for applying patches and performing other upgrades or maintenance tasks and therefore must be saved.

Note –
When applying patches or upgrades on the infrastructure or middle tiers, the corresponding original copy of "/var/opt/oracle" must be reinstated and made available before the Oracle installer is invoked.

Save /var/opt/oracle on each cluster node where the middle tier and infrastructure are installed together.
# cp -rp /var/opt/oracle /var/opt/oracle_infra
Note –
You will need to supply the directory for the Oracle 9iAS Infrastructure "/var/opt/oracle" location when you register the Sun Cluster HA for Oracle Application Server data service later on. In this case we have copied this to /var/opt/oracle_infra.

Stop the Oracle Application Server Infrastructure

For Oracle 9iAS Infrastructure

#  su -  <Oracle Application Server Userid>
$ <Oracle Home>/bin/emctl stop
$ <Oracle Home>/opmn/bin/opmnctl stopall
$ <Oracle Home>/bin/oidctl server=oidldapd configset=0 instance=1 stop
$ <Oracle Home>/bin/oidmon stop

For Oracle 10g AS Infrastructure

#  su -  <Oracle Application Server Userid>
$ <Oracle Home>/bin/emctl stop iasconsole
$ <Oracle Home>/opmn/bin/opmnctl stopall

Stop the Oracle Database and Listener

#  su -  <Oracle Database  Userid>
$ <Oracle Home>/bin/lsnrctl  stop
$ <Oracle Home>/bin/sqlplus "/ as sysdba"
$ shutdown normal
$ quit

Edit /etc/vfstab to reinstate the global mount option

If the Infrastructure mount point is a Global File System and you temporarily removed the global option in Step 1 you must now reinstate the global mount option.

You must edit /etc/vfstab to reinstate the global mount option. If you commented out the original entry just remove just that comment and comment out the copied entry you made in Step 1.

Boot each Node in Cluster Mode
# reboot