16 Configuring Administration Server High Availability

This chapter describes how to configure and validate the Oracle WebLogic Server Administration Server for high availability.

This chapter includes the following topics:

• Section 16.1, "Enabling Administration Server High Availability"

• Section 16.2, "Oracle HTTP Server Configuration"

• Section 16.3, "Validating the Administration Server"

• Section 16.4, "Manually Failing Over the Administration Server to CRMHOST2"

• Section 16.5, "Failing the Administration Server Back to CRMHOST1"

16.1 Enabling Administration Server High Availability

The Administration Server is a singleton application, so it cannot be deployed in an active-active configuration. By default, the Administration Server is only available on the first installed node. If this node becomes unavailable, then the Administration Console and Fusion Middleware Control also become unavailable. To avoid this scenario, the Administration Server and the applications deployed to it must be enabled for failover. The enterprise deployment architecture in this guide calls for the deploying the Administration Server on a disk shared between the primary node and the secondary node.

The following domains are deployed as part of the Oracle Fusion Customer Relationship Management enterprise deployment implementation:

• Oracle Fusion Customer Relationship Management Domain

• Oracle Fusion Common Domain

• Oracle Fusion Financials Domain

• Oracle Fusion Human Capital Management Domain

• Oracle Fusion Supply Chain Management Domain

• Oracle Fusion Incentive Compensation Domain

• Oracle Business Intelligence Domain

The process described in this guide initially deploys each domain-specific Administration Server in shared storage (/u01/oracle) mounted on CRMHOST1, and Managed Servers in the local disk (/u02/local/oracle).

This section contains the following topics:

• Section 16.1.1, "Enabling Administrative Virtual Host on CRMHOST1"

• Section 16.1.2, "Adding a New Machine in the Oracle WebLogic Server Console"

• Section 16.1.3, "Enabling the Administration Server to Listen on the Virtual IP Address"

16.1.1 Enabling Administrative Virtual Host on CRMHOST1

Note:

CRMADMINVH is used as a generic name in this chapter. For domain-specific administrative virtual host names, see Table 3-1 in Section 3.6, "IPs and Virtual IPs."

The Administration Server must be configured to listen on a virtual IP Address to enable it to seamlessly failover from one host to another. In case of a failure, the Administration Server, along with the virtual IP Address, can be migrated from one host to another.

However, before the Administration Server can be configured to listen on a virtual IP Address, one of the network interface cards on the host running the Administration Server must be configured to listen on this virtual IP Address. The steps to enable a virtual IP Address are completely dependent on the operating system.

To enable a virtual IP Address on CRMHOST1:

Note:

In a UNIX environment, the command must be run as the root user.

1. On CRMHOST1, run the ifconfig command to get the value of the netmask. In a UNIX environment, run this command as the root user. For example:

   [root@CRMHOST1 ~] # /sbin/ifconfig
   eth0     Link encap:Ethernet  HWaddr 00:11:43:D7:5B:06
        inet addr:139.185.140.51  Bcast:139.185.140.255  Mask:255.255.255.0
        inet6 addr: fe80::211:43ff:fed7:5b06/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:10626133 errors:0 dropped:0 overruns:0 frame:0
        TX packets:10951629 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:4036851474 (3.7 GiB)  TX bytes:2770209798 (2.5 GiB)
        Base address:0xecc0 Memory:dfae0000-dfb00000
   

2. On CRMHOST1, bind the virtual IP Address to the network interface card using ifconfig. In a UNIX environment, run this command as the root user. Use a netmask value that was obtained in Step 1.

   The syntax and usage for the ifconfig command is as follows:

   /sbin/ifconfig networkCardInterface Virtual_IP_Address netmask netMask
   

   For example:

   /sbin/ifconfig eth0:1 100.200.140.206 netmask 255.255.255.0
   

3. Update the routing table using arping. In a UNIX environment, run this command as the root user.

   /sbin/arping -q -U -c 3 -I networkCardInterface Virtual_IP_Address
   

   For example:

   /sbin/arping -q -U -c 3 -I eth0 100.200.140.206
   

16.1.2 Adding a New Machine in the Oracle WebLogic Server Console

Create a new machine and assign the Administration Server to the new machine using the Administration Console:

1. Log in to the Administration Console.

2. In the Change Center, click Lock & Edit.

3. In the Environment section of the Home page, click Machines.

4. On the Summary of Machines page, select the machine that is associated with the Administration Server from under the Machines table and click Clone. For example: CRMHOST1.MYCOMPANY.COM.

5. On the Clone a Machine page, enter the name of the machine under the Machine Identity section and click OK. For example, enter ADMINHOST as the machine name.

6. On the Summary of Machines page, click the newly created machine link.

7. On the Settings page for the ADMINHOST machine, select the Servers tab.

8. Click Add under the Servers table.

9. On the Add a Server to Machine page, select Select an existing server, and associate it with this machine option.

10. Choose the AdminServer from the dropdown list.

11. Click Finish to associate the Administration Server with the machine.

12. In the Change Center, click Activate Changes.

16.1.3 Enabling the Administration Server to Listen on the Virtual IP Address

Ensure that you have performed the steps described in Section 16.1.1, "Enabling Administrative Virtual Host on CRMHOST1" before setting the Administration Server listen address.

To set the Administration Server listen address:

1. Log in to the Administration Console.

2. In the Change Center, click Lock & Edit.

3. Expand the Environment node in the Domain Structure window.

4. Click Servers. The Summary of Servers page is displayed.

5. Select AdminServer(admin) in the table. The Settings page for AdminServer(admin) is displayed.

6. Set the Listen Address to CRMADMINVH (domain-specific administrative virtual host).

7. Click Save.

8. Click Activate Changes.

9. The changes will not take effect until the Administration Server is restarted. Follow these steps to restart the Administration Server:

   1. In the Summary of Servers page, select the Control tab.

   2. Select AdminServer(admin) in the table and then click Shutdown.

10. Set the following environment variable:

    WLST_PROPERTIES="-Dweblogic.security.SSL.trustedCAKeyStore=ORACLE_
    BASE/products/fusionapps/wlserver_10.3/server/lib/fusion_trust.jks"
    

11. Start the Administration Server again from the command line. Use the nmconnect username and password you specified in the Installation Location Screen in Chapter 5.

    CRMHOST1> ORACLE_BASE/products/fusionapps/wlserver_10.3/common/bin/wlst.sh
    
    CRMHOST1> nmConnect(username='username', password='password',
    domainName='domain_name', host='CRMHOST1',port='5556', nmType='ssl', domainDir='ORACLE_BASE/config/domains/CRMHOST1/domain_name')
    
    CRMHOST1> nmStart('AdminServer')
    

16.2 Oracle HTTP Server Configuration

To configure Oracle HTTP Server:

1. On WEBHOST1:

   1. cd ORACLE_BASE/config/CommonDomain_webtier/config/OHS/ohs1/moduleconf.

   2. Edit the domain-specific virtual host config file. For example:

      cp FusionVirtualHost_crm.conf FusionVirtualHost_crm.conf.org
      

2. Edit the FusionVirtualHost_crm.conffile, adding the Administrative virtual host and port. Example 16-1 shows sample code.

   Note:

   Replace CRMADMINVH and port with domain-specific Administrative virtual host and port number.

   Example 16-1 Add AdministrativeVirtual Host and Port

   ## Context roots for application em
       <Location /em>
           SetHandler weblogic-handler
           WebLogicCluster CRMADMINVH:port 
       </Location>
   
   ## Context roots for application console
       <Location /console >
           SetHandler weblogic-handler
           WebLogicCluster CRMADMINVH:port
       </Location>
   

3. Restart Oracle HTTP Server: cd to ORACLE_BASE/config/CommonDomain_webtier/bin and enter the following:

   WEBHOST1> ./opmnctl stopall
   WEBHOST1> ./opmnctl startall
   

4. Repeat Steps 1 through 3 on WEBHOST2.

16.3 Validating the Administration Server

Perform these steps to ensure that the Administration Server and Oracle Enterprise Manager Fusion Middleware Control are properly configured:

1. Ensure that you can access the domain-specific Oracle WebLogic Server Administration Console and Oracle Enterprise Manager Fusion Middleware Control. For example, for the Oracle Fusion Customer Relationship Management domain:

   http://crminternal.mycompany.com:7777/console

   http://crminternal.mycompany.com:7777/em

2. After completing the steps in Section 16.1 and Section 16.2 for other domains, repeat Step 1 for other domains by replacing the domain-specific URL.

16.4 Manually Failing Over the Administration Server to CRMHOST2

In case a node fails, you can fail over the Administration Server to another node. This section describes how to fail over the Administration Server from CRMHOST1 to CRMHOST2.

16.4.1 Prerequisites

Ensure the following:

• The Administration Server is configured to listen on a domain-specific administrative virtual host, and not on any address

• When failover happens, the Administration Server is failed over from CRMHOST1 to CRMHOST2 and the two nodes have the following IPs:

  • CRMHOST1 to : 100.200.140.165

  • CRMHOST2: 100.200.140.205

  • CRMADMINVH: 100.200.140.206. This is the VIP where the domain-specific Administration Server is running, assigned to ethX:Y, available in CRMHOST1 to and CRMHOST2.

  • The domain directory where the Administration Server is running on CRMHOST1 to is on shared storage and is mounted from CRMHOST2

16.4.2 Performing the Failover

The following procedure explains how to fail over the Administration Server to a different node (CRMHOST2) with the Administration Server still using the same Oracle WebLogic Server machine. (This machine is a logical machine, not a physical one.)

To fail over the Administration Server:

1. Stop the Administration Server.

2. Migrate the IP to the second node:

   1. Run the following command as root on CRMHOST1 to (where X:Y is the current interface used by CRMADMINVH):

      CRMHOST1> /sbin/ifconfig ethX:Y down
      

   2. Run the following command as root on CRMHOST2:

      CRMHOST2> /sbin/ifconfig interface:index IP_Address netmask netmask
      

      For example:

      /sbin/ifconfig eth0:1 100.200.140.206 netmask 255.255.255.0
      

      Note:

      Ensure that the netmask and interface to be used to match the available network configuration in CRMHOST2.

3. Update the routing tables with arping. For example, run the following command as root:

   CRMHOST2> /sbin/arping -b -A -c 3 -I eth0 100.200.140.206
   

4. Start the Administration Server on CRMHOST2 using the procedure in Section 16.1.3.

5. Test access to the Administration Server on CRMHOST2:

   1. Ensure that you can access the domain-specific Oracle WebLogic Server Administration Console and Oracle Enterprise Manager Fusion Middleware Control. For example, for the Oracle Fusion Customer Relationship Management domain, use these URLs:

      • http://crminternal.mycompany.com:7777/console

      • http://crminternal.mycompany.com:7777/em

   2. Repeat Step a for other domain by replacing the domain-specific URL.

Note:

The Administration Server does not use Node Manager for failing over. After a manual failover, the machine name that appears in the Current Machine field in the Administration Console for the server is CRMHOST1, and not the failover machine, CRMHOST2. Since Node Manager does not monitor the Administration Server, the machine name that appears in the Current Machine field, is not relevant and you can ignore it.

16.5 Failing the Administration Server Back to CRMHOST1

You also must ensure that you can fail back the Oracle WebLogic Server Administration Server, that is, stop it on CRMHOST2 and run it on CRMHOST1. To do this, migrate CRMADMINVH back to CRMHOST1 node.

To migrate CRMADMINVH :

1. Run the following command as root from CRMHOST2 to shut down the network stack virtual interface:

   CRMHOST2> /sbin/ifconfig ethX:Y down
   

2. Run the following command as root from CRMHOST1 to restart the virtual interface::

   CRMHOST1> /sbin/ifconfig ethX:Y 100.200.140.206 netmask 255.255.255.0
   

   Note:

   Ensure that the netmask and interface to be used match the available network configuration in CRMHOST1.

3. Run the following command from CRMHOST1 to update the routing tables through arping:

   CRMHOST1> /sbin/arping -b -A -c 3 -I ethZ 100.200.140.206
   

4. Start the Administration Server again on CRMHOST1 using the procedure in Step 3 in Section 16.1.3.

5. Test access to the Administration Server on CRMHOST1:

   1. Ensure that you can access the domain-specific Oracle WebLogic Server Administration Console and Oracle Enterprise Manager Fusion Middleware Control. For example, for the Oracle Fusion Customer Relationship Management domain, use these URLs:

      • http://crminternal.mycompany.com:7777/console

      • http://crminternal.mycompany.com:7777/em

   2. Repeat Step a for other domain by replacing the domain-specific URL.