Sun Cluster Data Service for Oracle E-Business Suite Guide for Solaris OS

Previous: Preface

Installing and Configuring Sun Cluster HA for Oracle E-Business Suite

This chapter explains how to install and configure Sun Cluster HA for Oracle E-Business Suite.

This chapter contains the following sections.

Sun Cluster HA for Oracle E-Business Suite Overview

Note –

Throughout this document a non-global zone may be referred to as a zone. A global zone will always be referred to as a global zone.

The Sun Cluster HA for Oracle E-Business Suite data service provides a mechanism for the orderly startup and shutdown, fault monitoring, and automatic failover of the Oracle E-Business Suite.

Oracle E-Business Suite is a complete set of business applications that enables you to efficiently manage business processes by using a unified open architecture. This architecture is a framework for multi tiered, distributed computing that supports Oracle products. The tiers that compose Oracle E-Business Suite are the database tier, applications tier, and desktop tier. These tiers can be distributed as a logical grouping and can be grouped on one or more nodes.

Table 1 Oracle E-Business Suite Architecture


Desktop Tier	Application Tier	Database Tier
	Web Server
	Forms Server
Web Browser	Concurrent Server	Database Server
	Reports Server
	Admin Server
	Discoverer Server

The distributed nature of Oracle E-Business Suite requires more than one cluster data service if all application and database tiers are to be managed by the cluster.

The following tables list the Oracle E-Business Suite components and the corresponding cluster data service that provides high availability to that component.

Table 2 Protection of Components


Component	Protected by
Database Server	Sun Cluster HA for Oracle (Database and Listener)
Web Server	Sun Cluster HA for Apache
Forms Server	Sun Cluster HA for Oracle E-Business Suite
Concurrent Manager Server	Sun Cluster HA for Oracle E-Business Suite
Concurrent Manager Listener	Sun Cluster HA for Oracle (Listener)
Reports Server	Sun Cluster HA for Oracle E-Business Suite

The Admin Server and Discoverer Server are not normally run within Sun Cluster and therefore are not protected by Sun Cluster HA for Oracle E-Business Suite.

Overview of Installing and Configuring Sun Cluster HA for Oracle E-Business Suite

The following table summarizes the tasks for installing and configuring Sun Cluster HA for Oracle E-Business Suite and provides cross-references to detailed instructions for performing these tasks. Perform the tasks in the order that they are listed in the table.

Table 3 Tasks for Installing and Configuring Sun Cluster HA for Oracle E-Business Suite


Task	Instructions
Plan the installation	Planning the Sun Cluster HA for Oracle E-Business Suite Installation and Configuration
Install and configure the Oracle E-Business Suite software	How to Install and Configure Oracle E-Business Suite
Verify the installation and configuration	How to Verify the Installation and Configuration of Oracle E-Business Suite
Install Sun Cluster HA for Oracle E-Business Suite packages	How to Install the Sun Cluster HA for Oracle E-Business Suite Packages
Register and configure Sun Cluster HA for Oracle E-Business Suite resources	How to Register and Configure Sun Cluster HA for Oracle E-Business Suite
Verify the Sun Cluster HA for Oracle E-Business Suite installation and configuration	How to Verify the Sun Cluster HA for Oracle E-Business Suite Installation and Configuration
Upgrade the Sun Cluster HA for Oracle E-Business Suite data service	How to Upgrade to the New Version of Sun Cluster HA for Oracle E-Business Suite
Tune the Sun Cluster HA for Oracle E-Business Suite fault monitor	Understanding the Sun Cluster HA for Oracle E-Business Suite Fault Monitor
Debug Sun Cluster HA for Oracle E-Business Suite	How to Turn on Debugging for Sun Cluster HA for Oracle E-Business Suite

Planning the Sun Cluster HA for Oracle E-Business Suite Installation and Configuration

This section contains the information you need to plan your Sun Cluster HA for Oracle E-Business Suite installation and configuration.

Configuration Restrictions

The configuration restrictions in the subsections that follow apply only to Sun Cluster HA for Oracle E-Business Suite.

Caution –

Your data service configuration might not be supported if you do not observe these restrictions.

Restriction for the supported configurations of Sun Cluster HA for Oracle E-Business Suite

The Sun Cluster HA for Oracle E-Business Suite data service can only be configured as a failover service.

Oracle E-Business Suite can be deployed in the global zone or non-global zone.

Oracle E-Business Suite is installed using rapidwiz onto a single-node, two-node or multi-node installation.

In a single-node installation, you install the Database, Web, Forms, Concurrent Manager, and Reports Server onto a single node.

In a two-node installation, one node contains the Database, Concurrent Manager and Reports server; and the other node contains the Forms and Web Server.

In a multi-node installation, you can specify any combination of up to five nodes to install the Database, Web, Forms, Concurrent Manager, and Reports Server.

Database Server, using rapidwiz, install the Database Server as a single database instance. The Database Server must be managed by Sun Cluster HA for Oracle as a failover service in the cluster.
Web Server, using rapidwiz, install the Web Server (Apache) onto a node. If this will run in the cluster, then the Web Server is managed by Sun Cluster HA for Apache and can be deployed as either a failover or scalable service in the cluster.
Forms, Concurrent Manager and Reports Server, depending on how you install using rapidwiz, you can install the Forms, Concurrent Manager and Reports Server onto the same node or onto different nodes. All of these components are managed by Sun Cluster HA for Oracle E-Business Suite as a failover service in the cluster.

Restriction for the location of Oracle E-Business Suite files

The Oracle E-Business Suite files are the data files that are created when you install Oracle E-Business Suite using the rapidwiz installer.

The Oracle E-Business Suite files must be placed on shared storage as either a cluster file system or a highly available local file system. The following tables show the mount points and acceptable file system types.

Mount Point	Filesystem Type
<dbname>DATA	Cluster file system or highly available file system.
<dbname>DB	Local, cluster file system or highly available file system.
<dbname>ORA	Local, cluster file system or highly available file system.

Mount Point	Filesystem type
<dbname>COMN_TOP	Cluster file system or highly available file system.
<dbname>APPL_TOP	Cluster file system or highly available file system.
<dbname>APPLCSF	Cluster file system or highly available file system.

Configuration Requirements

The configuration requirements in this section apply only to Sun Cluster HA for Oracle E-Business Suite.

Caution –

If your data service configuration does not conform to these requirements, the data service configuration might not be supported.

Determine which Solaris zone Oracle E-Business Suite will use

Solaris zones provides a means of creating virtualized operating system environments within an instance of the Solaris 10 OS. Solaris zones allow one or more applications to run in isolation from other activity on your system. For complete information about installing and configuring a Solaris Container, refer to System Administration Guide: Solaris Containers-Resource Management and Solaris Zones.

You must determine which Solaris zone Oracle E-Business Suite will use. Oracle E-Business Suite can run within a global zone or non-global zone configuration.

Installing and Configuring Oracle E-Business Suite

This section contains the procedures you need to install and configure Oracle E-Business Suite.

How to Install and Configure Oracle E-Business Suite

This section contains the procedures you need to install and configure Oracle E-Business Suite.

On a cluster member, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.

Determine which Solaris zone to use.

Refer to Determine which Solaris zone Oracle E-Business Suite will use for more information.

If a zone will be used, create the zone.

Refer to System Administration Guide: Solaris Containers-Resource Management and Solaris Zones for complete information about installing and configuring a zone.

If a zone is being used, ensure the zone is booted.

Repeat this step on all nodes of the cluster if a zone is being used.

Boot the zone if it is not running.
# zoneadm list -v # zoneadm -z zonename boot

Create a cluster file system or highly available local file system for the Oracle E-Business Suite files.

Refer to Sun Cluster Software Installation Guide for Solaris OS for information about creating a cluster file system and to Sun Cluster Data Services Planning and Administration Guide for Solaris OS for information about creating a highly available local file system.

Mount the highly available local file system.

Perform this step from the global zone on one node of the cluster.
- If a non-ZFS highly available local file system is being used for Oracle E-Business Suite, perform one of the following steps.
  
  Note –
  Ensure that the node has ownership of the disk set or disk group.
  
  For Solaris Volume Manager, type:
  # metaset -s disk-set -t
  For Veritas Volume Manager, type:
  # vxdg -C import disk-group # vxdg -g disk-group startall
  - If the global zone is being used for Oracle E-Business Suite, type:
    # mount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, create the mount point on all zones of the cluster that are being used for Oracle E-Business Suite.
    # zlogin zonename mkdir highly-available-local-filesystem
    Mount the highly available local file system on one of the zones being used.
    # mount -F lofs highly-available-local-filesystem \ > /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available local file system is being used for Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # zpool import -R / HAZpool
  - If a zone is being used for Oracle E-Business Suite, type:
    # zpool export -f HAZpool # zpool import -R /zonepath/root HAZpool

Plumb the Oracle E-Business Suite logical hostname.

Perform this step in the global zone on one node of the cluster for each logical hostname being used by Oracle E-Business Suite.
- If the global zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up
- If a zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up zone zonename

Enable logical host interpositioning.

Perform this step on all cluster nodes where Oracle E-Business Suite will run.

To provide logical host interpositioning for Oracle E-Business Suite you must create a symbolic link from

/usr/lib/secure/libschost.so.1 to /usr/cluster/lib/libschost.so.1

On all cluster nodes where Oracle E-Business Suite will run.
# cd /usr/lib/secure # ln -s /usr/cluster/lib/libschost.so.1 libschost.so.1

Install the Oracle E-Business Suite software.

Perform this step in the global zone or zone on one node of the cluster.
1. As root user, execute rapidwiz.
  
  When running rapidwiz you must enter the logical hostname as the node name for the Database, Administration, Concurrent Manager, Forms and Web Server. If you are performing a two node or multi node installation you must specify the appropriate logical hostname as the appropriate node for the Database, Administration, Concurrent Manager, Forms and Web Server.
  
  When executing rapidwiz save the config.txt file in a permanent location, for example, /var/tmp/config.txt. If you are installing Oracle E-Business Suite version 12 or later, save the /var/tmpconf_SID.txt file before clicking OK on rapidwiz message No install actions found.
  # cd oracle-ebusiness-suite-install-directory # ./rapidwiz
2. As root user, execute rapidwiz for each logical hostname.
  
  Execute rapidwiz for each logical hostname that you entered when generating the /var/tmp/config.txt file or /var/tmp/conf_SID file.
  # cd oracle-ebusiness-suite-install-directory # ./rapidwiz -servername logical-hostname

(Optional) Resolving validation error JSP is not responding, waiting 15 seconds and retesting.

If after installing the Web Server, you encounter the validation error JSP is not responding, waiting 15 seconds and retesting, leave the rapidwiz GUI and follow Step a through Step e to resolve the validation error. Then you must return to the rapidwiz GUI to retry the validation.

Edit the httpd_pls.conf file.

Edit the httpd_pls.conf file and add the following entries for each physical cluster node within the VirtualHost _default_:* section.

Allow from cluster-node
Allow from cluster-node.fqdn

# su - oraapp-user
$ cd base-directory/sidora/iAS/Apache/Apache/conf
$ vi httpd_pls.conf
$ exit

The following example shows the logical hostname lhost1 and the cluster nodes clnode1 and clnode2 that have been added.

<VirtualHost _default_:*>
  <Location />
    Order deny,allow
    Deny from all
    Allow from localhost
    Allow from lhost1
    Allow from lhost1.example.com
    Allow from clnode1
    Allow from clnode1.example.com
    Allow from clnode2
    Allow from clnode2.example.com
  </Location>
</VirtualHost>

Edit the oprocmgr.conf file.

Edit the oprocmgr.conf file and add the following entries for each physical cluster node within the VirtualHost _default_:8100 section.

Allow from cluster-node
Allow from cluster-node.fqdn

# su - oraapp-user
$ cd base-directory/sidora/iAS/Apache/Apache/conf
$ vi oprocmgr.conf
$ exit

The following example shows logical hostname lhost1 and the cluster nodes clnode1 and clnode2 that have been added.

<IfModule mod_oprocmgr.c>
   Listen 8000
   ProcNode lhost1.example.com 8100

  <VirtualHost _default_:8100>
<IfDefine SSL>
     SSLEngine off
</IfDefine>
     Port 8100
     <Location />
       Order Deny,Allow
       Deny from all
       Allow from localhost
       Allow from lhost1
       Allow from lhost1.example.com
       Allow from clnode1
       Allow from clnode1.example.com
       Allow from clnode2
       Allow from clnode2.example.com
     </Location>/base-directory/sidappl/admin/SID_lhost1.xml
     <Location /oprocmgr-service>
       SetHandler oprocmgr-service
     </Location>
     <Location /oprocmgr-status>
       SetHandler oprocmgr-status
     </Location>
   </VirtualHost>
</IfModule>

Edit the jserv.properties file.

Edit the jserv.properties file and add the following entries for all physical cluster node within the security.allowedAddresses entry.

security.allowedAddresses=127.0.0.1,lhost.fqdn,clnode1.fqdn,clnode2.fqdn

# su - oraapp-user
$ cd base-directory/sidora/iAS/Apache/Jserv/etc
$ vi jserv.properties
$ exit

The following example shows logical hostname lhost1 and the cluster nodes clnode1 and clnode2 that have been added.

security.allowedAddresses=127.0.0.1,lhost1.example.com,clnode1.example.com,clnode2.example.com

Restart Oracle E-Business Suite.

# su - oraapp-user
$ cd base-directory/sidcomn/admin/scripts/SID_logical-hostname
$ ./adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd base-directory/siddb/9.2.0/appsutil/scripts/SID_logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate
$ ./addbctl.sh start
$ ./addlnctl.sh start SID
$ exit
# su - oraapp-user
$ cd base-directory/sidcomn/admin/scripts/SID_logical-hostname
$ ./adstrtal.sh apps/apps
$ exit

Return to the rapidwiz GUI.

Return to the rapidwiz GUI and click Retry. The JSP check should now work.

Post Oracle E-Business Suite Installation, edit listener.ora.

Perform this step from the global zone or zone where you installed Oracle E-Business Suite.

If you are installing Oracle E-Business Suite version 12 or later, create a customSID_app-logical-host.env file.

# su - oraapp-user
$ vi app-base_directory/apps/apps_st/appl/customSID_app-logical-host.env

Add the following lines to the customSID_app-logical-host.env file:

LD_PRELOAD_32=/usr/lib/secure/libschost.so.1
SC_LHOSTNAME=app-logical-host

export LD_PRELOAD_32 SC_LHOSTNAME

If you are installing Oracle E-Business Suite version 11.5.10 or earlier, edit the listener.ora file and add the following entries to the envs= parameter for the SID_NAME=FNDSM entry.

envs='LD_LIBRARY_PATH=/usr/dt/lib:/user/openwin/lib:basedir/sidora/8.0.6/lib, \
LD_PRELOAD_32=/usr/lib/secure/libschost.so.1,SC_LHOSTNAME=lhost,MYAPPSORA=...

# su - oraapp-user
$ cd base-directory/sidora/8.0.6/network/admin/SID_logical-hostname
$ vi listener.ora

The following example shows the LD_PRELOAD_32 and SC_LHOSTNAME entries that have been added.

SID_LIST_APPS_PROD =
  (SID_LIST =
   ( SID_DESC = ( SID_NAME = FNDSM )
      ( ORACLE_HOME = /base-directory/sidora/8.0.6 )
      ( PROGRAM = /base-directory/sidappl/fnd/11.5.0/bin/FNDSM )
      ( envs='LD_LIBRARY_PATH=/usr/dt/lib:/user/openwin/lib:basedir/sidora/8.0.6 \
/lib,LD_PRELOAD_32=/usr/lib/secure/libschost.so.1,SC_LHOSTNAME=lhost1,MYAPPS=

Stop Oracle E-Business Suite.

Perform this step from the global zone or zone where you installed Oracle E-Business Suite.

If you are installing Oracle E-Business Suite version 12 or later, type:

# su - oraapp-user
$ cd app-base-directory/inst/apps/SID_app-logical-hostname/admin/scripts
$ ./adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SID/db/tech_st/*/appsutil/scripts/SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

If you are installing Oracle E-Business Suite version 11.5.10 or earlier, type:

# su - oraapp-user
$ cd app-base-directory/SIDcomn/admin/scripts/SID_app-logical-hostname
$ ./adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SID/db/9.2.0/appsutil/scripts/SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

Unmount the highly available local file system.

Perform this step from the global zone on the node where you installed Oracle E-Business Suite.
- To unmount a non-ZFS highly available local file system that is being used for the Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # umount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, type:
    # umount /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available local file system is being used for Oracle E-Business Suite, type:
  # zpool export -f HAZpool

Unplumb the Oracle E-Business Suite logical hostname.

Perform this step from the global zone on one node of the cluster for each logical hostname being used by Oracle E-Business Suite.
# ifconfig interface removeif logical-hostname

Verifying the Installation and Configuration of Oracle E-Business Suite

This section contains the procedure you need to verify the installation and configuration.

How to Verify the Installation and Configuration of Oracle E-Business Suite

This procedure does not verify that your application is highly available because you have not yet installed your data service.

Perform this procedure on one node or zone of the cluster unless a specific step indicates otherwise.

On a cluster member, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.

If a zone is being used for Oracle E-Business Suite, ensure that the zone is booted.

Repeat this step on all nodes on the cluster if a zone is being used.

Boot the zone if it is not running.
# zoneadm list -v # zoneadm -z zonename boot

Mount the highly available local file system.

Perform this step from the global zone on one node of the cluster.
- If a non-ZFS highly available local file system is being used for the Oracle E-Business Suite files, perform one of the following steps.
  
  Note –
  Ensure that the node has ownership of the disk set or disk group.
  
  For Solaris Volume Manager, type:
  # metaset -s disk-set -t
  For Veritas Volume Manager, type:
  # vxdg -C import disk-group # vxdg -g disk-group startall
  - If the global zone is being used for Oracle E-Business Suite, type:
    # mount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, mount the highly available local file system on one of the zones being used.
    # mount -F lofs highly-available-local-filesystem \ > /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available file system is being used for Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # zpool import -R / HAZpool
  - If a zone is being used for Oracle E-Business Suite, type:
    # zpool import -R /zonepath/root HAZpool

Plumb the Oracle E-Business Suite logical hostname.

Perform this step for each logical hostname being used by Oracle E-Business Suite.
1. If the global zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up
2. If a zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up zone zonename

If a zone is being used, log in to the zone.
# zlogin zonename

Start Oracle E-Business Suite.

If you are using Oracle E-Business Suite version 12 or later, perform the following:

# su - oradb-user
$ cd db-base-directory/SID/db/tech_st/*/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addbctl.sh start
$ ./addlnctl.sh start SID
$ exit
# su - oraapp-user
$ cd app-base-directory/inst/apps/SID_app-logical-hostname/admin/scripts
$ ./adstrtal.sh apps/apps
$ exit

If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:

# su - oradb-user
$ cd db-base-directory/SID/db/9.2.0/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addbctl.sh start
$ ./addlnctl.sh start SID
$ exit
# su - oraapp-user
$ cd app-base-directory/SIDcomn/admin/scripts/SID_app-logical-hostname
$ ./adstrtal.sh apps/apps
$ exit

Test that a client can access Oracle E-Business Suite by using a Windows Client.
- If you are using Oracle E-Business Suite version 12 or later, perform the following:
  1. Log in to Standalone Diagnostics at
    
    http://ebs-logical-host.domainname:8000/OA_HTML/DiagLogin.jsp.
  2. Log in with Userid sysadmin and Password sysadmin. Then click Login.
  3. Click the Basic tab and click Run All.
    
    Note –
    Some diagnostics tests might fail as Oracle E-Business Suite is not fully configured.
  4. From the Application drop-down menu, select HTML Platform. Then click Run All Groups.
  5. From the Application drop-down menu, select Application Object Library. Then click Run All Groups.
  6. From the Application drop-down menu, select CRM Foundation. Then click Run All Groups.
- If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:
  1. Log in to Standalone Diagnostics at
    
    http://ebs-logical-host.domainname:8000/OA_HTML/US/ICXINDEX.htm.
  2. Log in with Userid sysadmin and Password sysadmin. Then click System Admin.
  3. Double-click Requests and double-click Run.
  4. Click OK on Single requests.
  5. Type Active Users in the Name field.
  6. Click OK to submit the request.
  7. Click Refresh Data until you see the message Active Users is Completed.
  8. Click View Output.

Stop Oracle E-Business Suite.

If you are using Oracle E-Business Suite version 12 or later, perform the following:

# su - oraapp-user
$ cd app-base-directroy/inst/apps/SID_app-logical-hostname/admin/scripts
$ .adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SID/db/tech_st/*/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:

# su - oraapp-user
$ cd app-base-directory/SIDcomn/admin/scripts/SID_app-logical-hostname
$ .adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SID/db/9.2.0/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

If a zone is being used, log out from the zone.
# exit

Unmount the highly available local file system.

Perform this step only in the global zone.
- To unmount a non-ZFS highly available local file system that is being used for Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # umount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, unmount the highly available local file system from the zone.
    # umount /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available file system is being used for Oracle E-Business Suite, type:
  # zpool export -f HAZpool

Unplumb the Infrastructure logical IP address.

Perform this step for each logical hostname being used by Oracle E-Business Suite.
# ifconfig interface removeif logical-hostname

Relocate the shared storage to another node and mount the highly available local file system.

Perform this step on another node of the cluster.
- If a non-ZFS highly available local file system is being used for the Oracle E-Business Suite files, perform one of the following steps.
  
  Note –
  Ensure that the node has ownership of the disk set or disk group.
  
  For Solaris Volume Manager, type:
  # metaset -s disk-set -t
  For Veritas Volume Manager, type:
  # vxdg -C import disk-group # vxdg -g disk-group startall
  - If the global zone is being used for Oracle E-Business Suite, type:
    # mount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, mount the highly available local file system on one of the zones being used.
    # mount -F lofs highly-available-local-filesystem \ > /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available file system is being used for Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # zpool import -R / HAZpool
  - If a zone is being used for Oracle E-Business Suite, type:
    # zpool import -R /zonepath/root HAZpool

Plumb the Oracle E-Business Suite logical hostname.

Perform this step on another node of the cluster for each logical hostname being used by Oracle E-Business Suite.
- If the global zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up
- If a zone is being used for Oracle E-Business Suite, type:
  # ifconfig interface addif logical-hostname up zone zonename

If a zone is being used, log in to the zone.
# zlogin zonename

Start Oracle E-Business Suite.

If you are using Oracle E-Business Suite version 12 or later, perform the following:

# su - oradb-user
$ cd db-base-directory/SID/db/tech_st/*/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addbctl.sh start
$ ./addlnctl.sh start SID
$ exit
# su - oraapp-user
$ cd app-base-directory/inst/apps/SID_app-logical-hostname/admin/scripts
$ ./adstrtal.sh apps/apps
$ exit

If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:

# su - oradb-user
$ cd db-base-directory/SID/db/9.2.0/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addbctl.sh start
$ ./addlnctl.sh start SID
$ exit
# su - oraapp-user
$ cd app-base-directory/SIDcomn/admin/scripts/SID_app-logical-hostname
$ ./adstrtal.sh apps/apps
$ exit

Test that a client can access Oracle E-Business Suite by using a Windows Client.
- If you are using Oracle E-Business Suite version 12 or later, perform the following:
  1. Log in to Standalone Diagnostics at
    
    http://ebs-logical-host.domainname:8000/OA_HTML/DiagLogin.jsp.
  2. Log in with Userid sysadmin and Password sysadmin. Then click Login.
  3. Click the Basic tab and click Run All.
    
    Note –
    Some diagnostics tests might fail as Oracle E-Business Suite is not fully configured.
  4. From the Application drop-down menu, select HTML Platform. Then click Run All Groups.
  5. From the Application drop-down menu, select Application Object Library. Then click Run All Groups.
  6. From the Application drop-down menu, select CRM Foundation. Then click Run All Groups.
- If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:
  1. Log in to Standalone Diagnostics at
    
    http://ebs-logical-host.domainname:8000/OA_HTML/US/ICXINDEX.htm.
  2. Log in with Userid sysadmin and Password sysadmin. Then click System Admin.
  3. Double-click Requests and double-click Run.
  4. Click OK on Single requests.
  5. Type Active Users in the Name field.
  6. Click OK to submit the request.
  7. Click Refresh Data until you get the message Active Users is Completed.
  8. Click View Output.

Stop Oracle E-Business Suite.

If you are using Oracle E-Business Suite version 12 or later, perform the following:

# su - oraapp-user
$ cd app-base-directroy/inst/apps/SID_app-logical-hostname/admin/scripts
$ .adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SIDdb/tech_st/*/appsutil/scripts/SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:

# su - oraapp-user
$ cd app-base-directory/SIDcomn/admin/scripts/SID_app-logical-hostname
$ .adstpall.sh apps/apps
$ exit
# su - oradb-user
$ cd db-base-directory/SIDdb/9.2.0/appsutil/scripts/ \
SID_db-logical-hostname
$ ./addlnctl.sh stop SID
$ ./addbctl.sh stop immediate

If a zone is being used, log out from the zone.
# exit

Unmount the highly available local file system.

Perform this step in the global zone only.
- To unmount a non-ZFS highly available local file system that is being used for Oracle E-Business Suite, perform one of the following steps.
  - If the global zone is being used for Oracle E-Business Suite, type:
    # umount highly-available-local-filesystem
  - If a zone is being used for Oracle E-Business Suite, unmount the highly available local file system from the zone.
    # umount /zonepath/root/highly-available-local-filesystem
- If a ZFS highly available file system is being used for Oracle E-Business Suite, type:
  # zpool export -f HAZpool

Unplumb the Oracle E-Business Suite logical hostname.

Perform this step for each logical hostname being used by Oracle E-Business Suite.
# ifconfig interface removeif logical hostname

Installing the Sun Cluster HA for Oracle E-Business Suite Packages

If you did not install the Sun Cluster HA for Oracle E-Business Suite packages during your initial Sun Cluster installation, perform this procedure to install the packages. To install the packages, use the Sun Java^TM Enterprise System Installation Wizard.

How to Install the Sun Cluster HA for Oracle E-Business Suite Packages

Perform this procedure on each cluster node where you are installing the Sun Cluster HA for Oracle E-Business Suite packages.

You can run the Sun Java Enterprise System Installation Wizard with a command-line interface (CLI) or with a graphical user interface (GUI). The content and sequence of instructions in the CLI and the GUI are similar.

Note –

Even if you plan to configure this data service to run in non-global zones, install the packages for this data service in the global zone. The packages are propagated to any existing non-global zones and to any non-global zones that are created after you install the packages.

Before You Begin

Ensure that you have the Sun Java Availability Suite DVD-ROM.

If you intend to run the Sun Java Enterprise System Installation Wizard with a GUI, ensure that your DISPLAY environment variable is set.

On the cluster node where you are installing the data service packages, become superuser.

Load the Sun Java Availability Suite DVD-ROM into the DVD-ROM drive.

If the Volume Management daemon vold(1M) is running and configured to manage DVD-ROM devices, the daemon automatically mounts the DVD-ROM on the /cdrom directory.

Change to the Sun Java Enterprise System Installation Wizard directory of the DVD-ROM.
- If you are installing the data service packages on the SPARC® platform, type the following command:
  # cd /cdrom/cdrom0/Solaris_sparc

Start the Sun Java Enterprise System Installation Wizard.
# ./installer

When you are prompted, accept the license agreement.

If any Sun Java Enterprise System components are installed, you are prompted to select whether to upgrade the components or install new software.

From the list of Sun Cluster agents in Availability Services, select the data service for Oracle E-Business Suite.

If you require support for languages other than English, select the option to install multilingual packages.

English language support is always installed.

When prompted whether to configure the data service now or later, choose Configure Later.

Choose Configure Later to perform the configuration after the installation.

Follow the instructions on the screen to install the data service packages on the node.

The Sun Java Enterprise System Installation Wizard displays the status of the installation. When the installation is complete, the wizard displays an installation summary and the installation logs.

(GUI only) If you do not want to register the product and receive product updates, deselect the Product Registration option.

The Product Registration option is not available with the CLI. If you are running the Sun Java Enterprise System Installation Wizard with the CLI, omit this step.

Exit the Sun Java Enterprise System Installation Wizard.

Unload the Sun Java Availability Suite DVD-ROM from the DVD-ROM drive.
1. To ensure that the DVD-ROM is not being used, change to a directory that does not reside on the DVD-ROM.
2. Eject the DVD-ROM.
  # eject cdrom

Next Steps

See Registering and Configuring Sun Cluster HA for Oracle E-Business Suite to register Sun Cluster HA for Oracle E-Business Suite and to configure the cluster for the data service.

Registering and Configuring Sun Cluster HA for Oracle E-Business Suite

This section contains the procedures you need to configure Sun Cluster HA for Oracle E-Business Suite.

Some procedures within this section require you to use certain Sun Cluster commands. Refer to the relevant Sun Cluster command man page for more information about these commands and their parameters.

How to Register and Configure Sun Cluster HA for Oracle E-Business Suite

Perform this procedure on one node of the cluster only.

This procedure assumes that you installed the data service packages during your initial Sun Cluster installation.

If you did not install the Sun Cluster HA for Oracle E-Business Suite packages as part of your initial Sun Cluster installation, go to How to Install the Sun Cluster HA for Oracle E-Business Suite Packages.

On a cluster member, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.

# clresourcetype register SUNW.HAStoragePlus
# clresourcetype register SUNW.gds

Create a failover resource group for Oracle E-Business Suite.
# clresourcegroup create -n nodelist ebs-rg

Create a resource for the Oracle E-Business Suite Logical Hostname.

# clreslogicalhostname create -g ebs-rg \
> -h logical-hostname \
> logical-hostname-resource

Create a resource for the Oracle E-Business Suite Disk Storage.

If a ZFS highly available local file system is being used, perform the following:

# clresource create -g ebs-rg \
> -t SUNW.HAStoragePlus \
> -p Zpools=oracle-ebusiness-suite-zspool \
> oracle-ebusiness-suite-hastorage-resource

If a cluster file system or a non-ZFS highly available local file system is being used, perform the following:

# clresource create -g ebs-rg \
> -t SUNW.HAStoragePlus \
> -p FilesystemMountPoints=oracle-ebusiness-suite-filesystem-mountpoint \
> oracle-ebusiness-suite-hastorage-resource

Enable the Resource Group.
# clresourcegroup online -M ebs-rg

For complete information about creating and registering a cluster resource for the Oracle Database, refer to Sun Cluster Data Service for Oracle Guide for Solaris OS.

Note –

Before creating corresponding resources, you need to register the SUNW.oracle_server resource type.

# clresource create -g ebs-rg \
> -t SUNW.oracle_Server \
> -p Connect_string=apps/apps \
> -p ORACLE_SID=SID \
> -p ORACLE_HOME=oracle-home \
> -p Alert_log_file=oracle-home/admin/SID \
> _db-logical-hostname/bdump/alert_SID.log \
> -p Restart_type=RESOURCE_GROUP_RESTART \
> -p Resource_dependencies=oracle-ebusiness-suite-hastorage-resource \
> oracle-resource
# clresource enable oracle-resource

For complete information about creating and registering a cluster resource for the Oracle Listener, refer to Sun Cluster Data Service for Oracle Guide for Solaris OS.

Note –

The copy_env script is used to copy and format the sid.env to sid_ha.env, which is used by the User_env= parameter in the following example.

Note –

Before creating corresponding resources, you need to register the SUNW.oracle_listener resource type .

# cd /opt/SUNWscebs/cmg/util
# ./copy_env oracle-home SID_db-logical-host 
# clresource create -g ebs-rg \
> -t SUNW.oracle_listener \
> -p Listener_name=SID \
> -p ORACLE_HOME=oracle-home \
> -p User_env=oracle-home/SID_db-logical-hostname_ha.env \
> -p Resource_dependencies=oracle-ebusiness-suite-hastorage-resource \
> listener-resource
# clresource enable listener-resource

Create and register a resource for the Concurrent Manager Listener.

On all cluster nodes that will run the Concurrent Manager Listener, type:

# touch /opt/SUNWscor/.use_su

Note –

The copy_env script is used to copy and format the SID.env to sid_ha.env, which is used by the User_env= parameter in the following example.

If you are using Oracle E-Business Suite version 12 or later, perform the following:

# cd /opt/SUNWscebs/cmg/util
# ./copy_env app-base-directory/inst/apps/ \
# SID_app-logical-hostname/ora/10.1.2/ \
# SID_app-logical-hostname
# cd oracle-home/apps/tech_st/10.1.2/bin
# echo "Required for SC cmg Listener resource" > oracle

# clresource create -g ebs-rg \
> -t SUNW.oracle_listener \
> -p Listener_name=APPS_SID \
> -p ORACLE_HOME=app-base-directory//apps/tech_st/10.1.2 \
> -p User_env=app-base-directory/SID_app-logical-hostname/ora/10.1.2/ \
> SID_app-logical-hostname_ha.env \
> -p Resource_dependencies=ebs-hastorage-rs \
> concurrent-manager-listener-resource

# clresource enable concurrent-manager-listener-resource

If you are using Oracle E-Business Suite version 11.5.10 or earlier, perform the following:

# cd /opt/SUNWscebs/cmg/util
# ./copy_env app-base-directory/SIDora/8.0.6 SID_app-logical-hostname
# cd app-base-directory/SIDora/8.0.6/bin
# echo "Required for SC cmg Listener resource" > oracle

# clresource create -g ebs-rg
> -t SUNW.oracle_listener \
> -p Listener_name=APPS_SID
> -p ORACLE_HOME=app-base-directory/SIDora/8.0.6 \
> -p User_env=app-base-directory/SIDora/8.0.6/ \
> SID_app-logical-hostname_ha.env \
> -p Resource_dependencies=ebs-hastorage-rs \
> concurrent-manager-listener-resource

# clresource enable concurrent-manager-listener-resource

Create and register a resource for the Concurrent Manager.

Edit the /opt/SUNWscebs/cmg/util/cmg_config file and follow the comments within that file. After you have edited the cmg_config file, you must register the resource.

# cd /opt/SUNWscebs/cmg/util
# vi cmg_config
# ./cmg_register

If you are using Oracle E-Business Suite version 12 or later, use:
COMNTOP=app-base-directory/inst/apps/SID_app-logical-hostname

If you are using Oracle E-Business Suite version 11.5.10 or earlier, use:

COMNTOP=app-base-directory/SIDcomn

The following example shows edits of the cmg_config file.

RS=ebs-cmg
RG=ebs-rg
LH=ebs-lh
HAS_RS=ebs-has
LSR_RS=ebs-cmglsr
VERSION=11.5.10
COMNTOP=use appropriate COMNTOP
APPSUSER=oraapp-user
APP_SID=SID
APPS_PASSWD=apps
ORACLE_HOME=oracle_home
CON_LIMIT=70
#
# Required for Oracle E-Business Suite version 11.5.10 CU2 or later
#
ORASVR_RS=ebs1-orasvr
ORALSR_RS=ebs1-oralsr

Create and register a resource for the Forms Server.

Note –
This step is required only if you are using Oracle E-Business Suite version 11.5.10 or earlier. If you are using Oracle E-Business Suite version 12 or later, go to Step 14.

Edit the /opt/SUNWscebs/frm/util/frm_config file and follow the comments within that file. After you have edited the frm_config file, you must register the resource.
# cd /opt/SUNWscebs/frm/util # vi frm_config # ./frm_register
The following example shows edits of the frm_config file.
RS=ebs-frm RG=ebs-rg LH=ebs-lh HAS_RS=ebs-has COMNTOP=base-directory/sidcomn APPSUSER=oraapp-user APP_SID=SID VERSION=11.5.10

Create and register a resource for the Reports Server.

Note –
This step is required only if you are using Oracle E-Business Suite version 11.5.10 or earlier. If you are using Oracle E-Business Suite version 12 or later, go to Step 14.

Edit the /opt/SUNWscebs/rep/util/rep_config file and follow the comments within that file. After you have edited the rep_config file, you must register the resource.
# cd /opt/SUNWscebs/rep/util # vi rep_config # ./rep_register
The following example shows edits of the rep_config file.
RS=ebs-rep RG=ebs-rg LH=ebs-lh HAS_RS=ebs-has COMNTOP=base-directory/sidcomn APPSUSER=oraapp-user APP_SID=SID VERSION=11.5.10

(Optional) Create and register a resource for the Web Server.

Note –

This step is required only if you are using Oracle E-Business Suite version 11.5.10 or earlier. If you are using Oracle E-Business Suite version 12 or later, go to Step 14.

# cd base-directory/sidcomn/admin/scripts/SID_logical-hostname
# ln -s adapcctl.sh apachectl

Edit adapcctl.sh and modify the "$control_code" test to add -a "$control_code" != "configtest".

# vi adapcctl.sh

The following output shows an amended adapcctl.sh.

	Note: The output has been realigned to fit the page

 	Find control_code (/control_code)

	Modify
	------ 

control_code="$1"

if test "$control_code" != "start" -a "$control_code" != "stop" \
   -a "$control_code" != "status" ; then
   printf "\n$program: You must either specify \
			   'start', 'stop', 'status'\n\n"
   printf "\n$program: You must either specify \
		      'start', 'stop', 'status'\n\n" >> $LOGFILE
   exit 1;
fi

	To (Note: We've simply added a test to allow "configtest"
	---------------------------------------------------------

control_code="$1"

if test "$control_code" != "start" -a "$control_code" != "stop" \
   -a "$control_code" != "status" \
				-a "$control_code" != "configtest" ; then      
   printf "\n$program: You must either specify \
				'start', 'stop', 'status'\n\n"
   printf "\n$program: You must either specify \
				'start', 'stop', 'status'\n\n" >> $LOGFILE
   exit 1;
fi

# clresource create -g ebs-rg \
> -t SUNW.apache \
> -p Port_list=8000/tmpSID \
> -p Bin_dir=base-directory/sidcomn/admin/scripts/ \
> SID_logical-hostname \
> -p Resource_dependencies=oracle-ebusiness-suite-hastorage-resource \
> apache-resource

Create and register a resource for the OPMN Server.

If you are installing Oracle E-Business Suite version 12 or later, perform the following:

Edit the /opt/SUNWscebs/opmn/util/opmn_config file and follow the comments within that file. After you have edited the opmn_config file, you must register the resource.
# cd /opt/SUNWscebs/opmn/util # vi opmn_config # ./opmn_register
The following example shows the edits of the opmn_config file.
RS=ebs-opmn RG=ebs-rg LH=ebs-lh HAS_RS=ebs-has VERSION=12.0 COMNTOP=app-base-directory/inst/apps/SID_app-logical-host APPSUSER=oraapp-user APP_SID=PROD APPS_PASSWD=apps OPMN_COMPONENTS=all

Enable the Oracle E-Business Suite resources.
# clresource enable -g ebs-rg +

Verifying the Sun Cluster HA for Oracle E-Business Suite Installation and Configuration

This section contains the procedure you need to verify that you installed and configured your data service correctly.

How to Verify the Sun Cluster HA for Oracle E-Business Suite Installation and Configuration

On a cluster member, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.

Ensure that all the Oracle E-Business Suite resources are online.
# cluster status
Enable any Oracle E-Business Suite resources that are not online.
# clresource enable oracle-ebusiness-suite-resource

Switch the Oracle E-Business Suite resource group to another cluster node or node:zone.
# clresourcegroup switch -n node[:zone] ebs-rg

Upgrading Sun Cluster HA for Oracle E-Business Suite

Upgrade the Sun Cluster HA for Oracle E-Business Suite data service if the following conditions apply:

You are upgrading from an earlier version of the Sun Cluster HA for Oracle E-Business Suite data service.
You need to use the new features of this data service.

How to Upgrade to the New Version of Sun Cluster HA for Oracle E-Business Suite

You must perform all the steps within this procedure.

Note –

Before performing this procedure you should consider if your current Oracle E-Business Suite resources have been modified to have specific timeout values that suit your deployment. If timeout values were previously adjusted you should reapply those timeout values to your new Oracle E-Business Suite resources.

On a cluster member, become superuser or assume a role that provides solaris.cluster.modify RBAC authorization.

Disable the Oracle E-Business Suite resources.

# clresource disable oracle-ebusiness-suite-resource

Install the new version of Sun Cluster HA for Oracle E-Business Suite on each cluster.

Refer to How to Install the Sun Cluster HA for Oracle E-Business Suite Packages for more information.

Delete the Oracle E-Business Suite resources.

# clresource delete oracle-ebusiness-suite-resource

Upgrade the logical hostname interpositioning.

Note –
You must repeat the following steps, as the logical host interpositioning file name and variable names have changed. Change the following:
- LHOSTNAME to SC_LHOSTNAME
- libloghost_32.so.1 to libschost.so.1

Repeat Step 8 and Step 11 from How to Install and Configure Oracle E-Business Suite.

Reregister the Oracle E-Business Suite resources.

Refer to How to Register and Configure Sun Cluster HA for Oracle E-Business Suite for more information.

Enable the Oracle E-Business Suite resources

# clresource enable oracle-ebusiness-suite-resource

Understanding the Sun Cluster HA for Oracle E-Business Suite Fault Monitor

This section describes the Sun Cluster HA for Oracle E-Business Suite fault monitor probing algorithm or functionality, states the conditions, and recovery actions associated with unsuccessful probing.

For conceptual information about fault monitors, see the Sun Cluster Concepts Guide.

Resource Properties

The Sun Cluster HA for Oracle E-Business Suite fault monitor uses the same resource properties as resource type SUNW.gds. Refer to the SUNW.gds(5) man page for a complete list of resource properties used.

Probing Algorithm and Functionality

The Sun Cluster HA for Oracle E-Business Suite fault monitor is controlled by the extension properties that control the probing frequency. The default values of these properties determine the preset behavior of the fault monitor. The preset behavior should be suitable for most Sun Cluster installations. Therefore, you should tune the Sun Cluster HA for Oracle E-Business Suite fault monitor only if you need to modify this preset behavior.

Setting the interval between fault monitor probes (Thorough_probe_interval)
Setting the timeout for fault monitor probes (Probe_timeout)
Setting the number of times the fault monitor attempts to restart the resource (Retry_count)

The Sun Cluster HA for Oracle E-Business Suite fault monitor performs a check within an infinite loop. During each cycle, the fault monitor checks the relevant component and reports either a failure or success.

If the fault monitor is successful, it returns to its infinite loop and continues the next cycle of probing and sleeping.

If the fault monitor reports a failure, a request is made to the cluster to restart the resource. If the fault monitor reports another failure, another request is made to the cluster to restart the resource. This behavior continues whenever the fault monitor reports a failure.

If successive restarts exceed the Retry_count within the Thorough_probe_interval, a request is made to fail over the resource group onto a different node or zone.

Concurrent Manager Probe

Test whether at least one FND (Concurrent Manager) process is running. If this test fails, the probe restarts the Concurrent Manager Server resource.
Test whether the probe can still connect to the Oracle Database. If this test fails, the probe restarts the Concurrent Manager Server resource.
Calculate the number of concurrent processes running as a percentage of the maximum number of concurrent processes allowed. Then test whether that percentage is less than CON_LIMIT, when the Concurrent Manager Server resource was defined. If the percentage is less than CON_LIMIT, the probe restarts the Concurrent Manager Server resource.

Forms Server Probe

Test whether the f60srvm process is running. If f60srvm is found, then test whether f60webmx process is running. If f60webmx is not found, the probe retests after another iteration of the probe to determine whether f60webmx is still missing, because f60srvm usually restarts f60webmx. If after two successive probes, f60webmx is still missing or f60srvm is not found on any probe, the probe restarts the Forms Server resource.

Reports Server Probe

Test whether rwmts60 process is running for REP_APPSUSER. If this test fails, the probe restarts the Forms Server resource.

Debugging Sun Cluster HA for Oracle E-Business Suite

How to Turn on Debugging for Sun Cluster HA for Oracle E-Business Suite

Sun Cluster HA for Oracle E-Business Suite can be used by multiple Oracle E-Business Suite instances. It is possible to turn debugging on for all Oracle E-Business Suite instances or a particular Oracle E-Business Suite instance.

/opt/SUNWscebs/xxx/etc/config allows you to turn on debugging for all Oracle E-Business Suite instances or for a specific Oracle E-Business Suite instance on a particular node or zone within the cluster. If you require debugging to be turned on for Sun Cluster HA for Oracle E-Business Suite across the whole cluster, repeat this step on all nodes within the cluster.

Edit the /etc/syslog.conf file and change daemon.notice to daemon.debug.

# grep daemon /etc/syslog.conf
*.err;kern.debug;daemon.notice;mail.crit        /var/adm/messages
*.alert;kern.err;daemon.err                     operator
#

Change the daemon.notice file to daemon.debug and restart the syslogd command. Note that the following output, from grep daemon /etc/syslog.conf, shows that daemon.debug has been set.

# grep daemon /etc/syslog.conf
*.err;kern.debug;daemon.debug;mail.crit        /var/adm/messages
*.alert;kern.err;daemon.err                    operator

Restart the syslog daemon.

If you are running Solaris 9, type:
# pkill -1 syslogd

If you are running Solaris 10, type:

# svcadm disable system-log
# svcadm enable system-log

Edit the /opt/SUNWscebs/cmg/etc/config file.

Perform this step for each component that requires debug output, on each node of Sun Cluster as required.

Edit the /opt/SUNWscebs/cmg/etc/config file and change DEBUG= to DEBUG=ALL or DEBUG=sun-cluster-resource.

# cat /opt/SUNWscebs/cmg/etc/config
#
# Copyright 2006 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)config     1.1     06/03/06 SMI"
#
# Usage:
#       DEBUG=<RESOURCE_NAME> or ALL
#
DEBUG=ALL

Note –

To turn off debug, reverse the previous steps.

Previous: Preface