16 Additional Configuration Procedures for Scaling Out Oracle SOA Suite Server

This chapter describes the additional scaleout steps required for the soa_server1 and soa_server2 servers on FINHOST1 and FINHOST2.

Notes:

• The Oracle SOA Suite server uses the Java Message Service (JMS) server. JMS requires a shared file system for its file store and transactional log. Each Oracle SOA Suite Managed Server in a cluster uses a separate file on the shared folder. During a node failure, the Oracle SOA Suite server must be moved to a targeted node to run the same server using the exact JMS file store and transaction log. To enable this server migration, each Oracle SOA Suite server must be configured with its own virtual IP, which can be floated on any server where the Oracle SOA Suite server is migrated.

• For Java Database Connectivity (JDBC), each Oracle SOA Suite Managed Server in a cluster uses a dedicated table per server on a database using the shared data source connection. During a node failure, the Oracle SOA Suite server must be moved to a targeted node to run the same server using the exact JDBC store and transaction log.

The procedures in this chapter use the Oracle SOA Suite server in the Oracle Fusion Financials domain as an example. You must perform the same procedures for the Oracle SOA Suite servers in all other domains except the BIDomain domain, which has no Oracle SOA Suite servers.

Note:

For Oracle Fusion Financials, the Oracle SOA Suite virtual IPs for FINHOST1 and FINHOST2 are called FINSOAVH1 and FINSOAVH2. For all other domains, replace the first three characters with domain-specific syntax. For HCMSOAVH1, SCMSOAVH1, and so on.

This chapter includes the following topics:

• Section 16.1, "Enabling Virtual IPs on FINHOST1 and FINHOST2"

• Section 16.2, "Setting the Listen Address for soa_server1"

• Section 16.3, "Setting the Listen Address for soa_server2"

• Section 16.4, "Updating the FusionVirtualHost_fin.conf Configuration File"

• Section 16.5, "Configuring UMSJMSServer_auto_1 and UMSJMSServer_2"

• Section 16.6, "Configuring JMS Servers with JDBC Store Persistence"

• Section 16.7, "Configuring Oracle Coherence for Deploying Composites"

• Section 16.8, "Configuring a JDBC Transaction Log Store for Transaction Recovery"

• Section 16.9, "Disabling Host Name Verification for the soa_servern Managed Servers"

• Section 16.10, "Restarting Node Manager on FINHOST1"

• Section 16.11, "Starting and Validating soa_server1 on FINHOST1"

• Section 16.12, "Restarting Node Manager on FINHOST2"

• Section 16.13, "Starting and Validating soa_server2 on FINHOST2"

Note:

Before performing any of the procedures in this chapter, ensure that soa_server1 is running on FINHOST1 and soa_server2 is running on FINHOST2.

16.1 Enabling Virtual IPs on FINHOST1 and FINHOST2

To enable the virtual IP on Linux:

Note:

In this example, ethX is the ethernet interface (eth0 or eth1) and Y is the index (0, 1, 2, and so on). In addition, the FINSOAVH1 and FINSOAVH2 VIPs will be used.

1. On FINHOST1:

   1. Run the ifconfig command as root:

      /sbin/ifconfig interface:index IPAddress netmask netmask
      

      For example:

      /sbin/ifconfig ethX:Y 100.200.140.206 netmask 255.255.255.0
      

   2. Enable your network to register the new location of the virtual IP:

      /sbin/arping -q -U -c 3 -I interface IPAddress
      

      For example:

      /sbin/arping -q -U -c 3 -I ethX 100.200.140.206
      

   3. Validate that the address is available by pinging it from another node.

      For example:

      /bin/ping 100.200.140.206
      

2. Repeat Steps a through c on FINHOST2.

16.2 Setting the Listen Address for soa_server1

Ensure that you have performed the steps described in Section 16.1, and the scale-out steps described in Section 8.3, Section 8.4, and Section 8.5 before setting the soa_server1 listen address.

To set the listen address for the Managed Server:

1. Log in to the Administration Console.

2. In the Change Center, click Lock & Edit.

3. Expand the Environment node in the Domain Structure window.

4. Click Servers. The Summary of Servers page is displayed.

5. Select soa_server1 in the table. The Setting page for soa_server1 is displayed.

6. Set the Listen Address to FINSOAVH1.

7. Click Save.

8. Click Activate Changes.

9. The changes will not take effect until the soa_server1 Managed Server is restarted (ensure that Node Manager is up and running):

   1. On the Summary of Servers page, select the Control tab.

   2. Select soa_server1 in the table and then click Shutdown.

   3. After the server has shut down, select soa_server1 in the table and then click Start.

16.3 Setting the Listen Address for soa_server2

Ensure that you have performed the steps described in Section 16.1 before setting the soa_server2 listen address.

Perform these steps to set the listen address for the Managed Server:

1. Log in to the Administration Console.

2. In the Change Center, click Lock & Edit.

3. Expand the Environment node in the Domain Structure window.

4. Click Servers. The Summary of Servers page is displayed.

5. Select soa_server2 in the column of the table. The Settings page for soa_server2 is displayed.

6. Set the Listen Address to FINSOAVH2.

7. Click Save.

8. Click Activate Changes.

9. The changes will not take effect until the soa_server2 Managed Server is restarted (ensure that Node Manager is up and running):

   1. On the Summary of Servers page, select the Control tab.

   2. Select soa_server2 in the table and then click Shutdown.

   3. After the server has shut down, select soa_server2 in the table and then click Start.

16.4 Updating the FusionVirtualHost_fin.conf Configuration File

To enable Oracle HTTP Server to route to soa_cluster, which contains the soa_servern Managed Servers, you must set the WebLogicCluster parameter to the list of nodes in the cluster.

To set the parameter:

1. Update the WebLogicCluster parameter in the FusionVirtualHost_fin.conf file to contain a cluster list of virtual host:port entries. (On WEBHOST1: ORACLE_BASE/config/CommonDomain_webtier/config/OHS/ohs1/moduleconf/FusionVirtualHost_fin.conf. On WEBHOST2: ORACLE_BASE/config/CommonDomain_webtier1/config/OHS/ohs2/moduleconf/FusionVirtualHost_fin.conf.) For example:

   <Location /soa-infra>
    SetHandler weblogic-handler
    WebLogicCluster FINSOAVH1:7416,FINSOAVH2:7416
   </Location>
   

2. Restart Oracle HTTP Server on both WEBHOST1 and WEBHOST2:

   WEBHOST1> ORACLE_BASE/config/CommonDomain_webtier/bin/opmnctl restartproc 
   ias-component=ohs1
   
   WEBHOST2> ORACLE_BASE/config/CommonDomain_webtier1/bin/opmnctl restartproc 
   ias-component=ohs2
   

The servers specified in the WebLogicCluster parameters are only important at startup time for the plug-in. The list must provide at least one running cluster member for the plug-in to discover other members in the cluster. The listed cluster member must be running when Oracle HTTP Server is started. Oracle WebLogic Server and the plug-in work together to update the server list automatically with new, failed, and recovered cluster members.

Sample scenarios include the following:

• Example 1: If you have a two-node cluster and then add a third member, you do not need to update the configuration to add the third member. The third member will be discovered dynamically at runtime.

• Example 2: You have a three-node cluster, but only two nodes are listed in the configuration. However, if both listed nodes are down when you start Oracle HTTP Server, then the plug-in would fail to route to the cluster. You must ensure that at least one of the listed nodes is running when you start Oracle HTTP Server.

If you list all the members of the cluster, then you guarantee you can route to the cluster, assuming at least one member is running when Oracle HTTP Server is started. For more information on configuring the Oracle WebLogic Server plug-in, see Oracle Fusion Middleware Using Web Server 1.1 Plug-Ins with Oracle WebLogic Server.

16.5 Configuring UMSJMSServer_auto_1 and UMSJMSServer_2

After FINHOST1 has been provisioned, the UMSJMSServer_auto_1 server is set up and configured for FINHOST1, and the file store is created for FINHOST1. You now must configure the file store for FINHOST1, and create and configure the UMSJMSServer_2 server and file store for FINHOST2. Configure the location for the persistence store file to a directory visible from both nodes.

To configure the file store for FINHOST1:

1. Log in to the Oracle WebLogic Server Administration Console.

2. Click Lock & Edit.

3. In the Domain Structure window, expand the Services node and then click the Persistence Stores node.

   The Summary of Persistence Stores page appears.

4. Click a persistence-store name. For example, UMSJMSFileStore_auto_1 .

5. In the Directory field, enter ORACLE_BASE/config/domains/FINHOST1/FinancialDomain/UMSJMSFileStore_auto_1.

6. Click Save and then click Activate Changes.

To create and configure the JMS servers and file store for FINHOST2:

1. Log in to the Oracle WebLogic Server Administration Console.

2. Click Lock & Edit.

3. In the Domain Structure window, expand the Services node and then click the Persistence Stores node.

   The Summary of Persistence Stores page appears.

4. Click New, and then Create File Store.

5. In the file store fields, enter the following:

   • Name: for example, UMSJMSFileStore_2

   • Target: soa_server2

   • Directory:

     ORACLE_BASE/config/domains/FINHOST1/FinancialDomain/UMSJMSFileStore_auto_1

   Note:

   The same shared-directory path is used for file stores 1 and 2.

6. Click OK and then click Activate Changes.

7. Click Lock & Edit.

8. In the Domain Structure window, expand the Services node and then click the Messaging > JMS Servers node.

   The Summary of Summary of JMS Servers page appears.

9. Click New.

10. Enter a name (for example, UMSJMSServer_2), then select UMSJMSFileStore_2 in the Persistence Store dropdown list.

11. Click Next.

12. Select soa_server2 as the target.

13. Click Finish and then click Activate Changes.

14. Click Lock & Edit.

15. In the Domain Structure window, expand the Services node and then click the Messaging > JMS Modules node.

    The Summary of Summary of JMS Modules page appears.

16. Click UMSJMSSystemResource and then click the Subdeployments tab.

17. Click UMSJMSServerxxxxx under Subdeployments.

18. Add the new UMSJMSServer_2 server as an additional target for the subdeployment.

19. Click Save and then click Activate Changes.

16.6 Configuring JMS Servers with JDBC Store Persistence

After FINHOST1 has been provisioned, the Java Message Service (JMS) servers are set up and configured and Java Database Connectivity (JDBC) stores are created and fully configured for FINHOST1. You now must create and configure the JMS servers and JDBC stores for FINHOST2.

To create and configure the JMS servers and JDBC stores for FINHOST2:

1. Log in to the Oracle WebLogic Server Administration Console.

2. Click Lock & Edit.

3. In the Domain Structure window, expand the Services node and then click the Persistence Stores node.

   The Summary of Persistence Stores page appears.

4. Click New, and then Create JDBC Store.

5. In the file store fields, enter the following:

   • Name: for example, SOAJMSJDBCStore_2

   • Target: soa_server2

   • Data Source: SOALocalTxDataSource

   • Prefix Name: FIN_FUSION_SOAINFRA.SOAJMS_2_. (Do not forget to include the ending "_".)

6. Click OK.

7. Repeat Steps 3 through 6 for the remaining the three remaining JDBC stores, using the same target and data-source field values:

   • AGJMSJDBCStore_2

   • BPMJMSJDBCStore_2

   • PS6SOAJMSJDBCStore_2

8. Click Activate Changes.

9. Click Lock & Edit.

10. In the Domain Structure window, expand the Services node and then click the Messaging > JMS Servers node.

    The Summary of Summary of JMS Servers page appears.

11. Click New.

12. Enter a name (for example, SOAJMSServer_2), then select SSOAJMSJDBCStore_2in the Persistence Store dropdown list.

13. Click Next.

14. Select soa_server2 as the target.

15. Click Finish.

16. Repeat Steps 10 through 15 for the remaining JMS servers:

    • AGJMSServer_2

    • BPMJMSServer_2

    • PS6SOAJMSServer_2

17. Click Activate Changes.

18. Click Lock & Edit.

19. In the Domain Structure window, expand the Services node and then click the Messaging > JMS Modules node.

    The JMS Modules page appears.

20. Click SOAJMSModule and then click the Subdeployments tab.

21. Click SOAJMSServerxxxxx under Subdeployments.

22. Add the new SOAJMSServer_2 server as an additional target for the subdeployment.

23. Click Save.

24. Repeat Steps 19 through 23 for the BPMJMSModule JMS module.

    Note:

    Do not make any changes to these JMS modules:

    • ADSJMSAQModule

    • JRFWSAsyncJmsModuleAQ

    • AGJMSModule

    • PS6SOAJMSModule

25. Click Activate Changes.

16.7 Configuring Oracle Coherence for Deploying Composites

Although deploying composites uses multicast communication by default, Oracle recommends using unicast communication instead in SOA enterprise deployments. Use unicast if you disable multicast communication for security reasons.

Note:

An incorrect configuration of the Oracle Coherence framework that is used for deployment may prevent the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs. Oracle recommends the configuration described in this section.

Multicast communication enables Oracle Fusion Middleware SOA to discover all of the members of a cluster to which it deploys composites dynamically. However, unicast communication does not enable nodes to discover other cluster members in this way. Consequently, you must specify the nodes that belong to the cluster. You do not need to specify all of the nodes of a cluster, however. You need only specify enough nodes so that a new node added to the cluster can discover one of the existing nodes. Consequently, when a new node has joined the cluster, it is able to discover all of the other nodes in the cluster. Additionally, in configurations such as SOA enterprise deployments, where multiple IPs are available in the same box, you must configure Oracle Coherence to use a specific host name to create the Oracle Coherence cluster.

Tip:

To guarantee high availability during deployments of SOA composites, specify enough nodes so that at least one of them is running at any given time.

Specify the nodes using the tangosol.coherence.wka n system property, where n is the number for each Oracle HTTP Server. The numbering starts at 1. This numbering must be sequential and must not contain gaps. In addition, specify the host name used by Oracle Coherence to create a cluster through the tangosol.coherence.localhost system property. This local host name should be the virtual host name used by the SOA server as the listener addresses (FINSOAVH1 and FINSOAVH2). Set this property by adding the -Dtangosol.coherence.localhost parameters to the Arguments field of the Oracle WebLogic Server Administration Console's Server Start tab.

Note:

FINSOAVH1 is the virtual host name that maps to the virtual IP where soa_server1 is listening (in FINHOST1). FINSOAVH2 is the virtual host name that maps to the virtual IP where soa_server2 is listening (in FINHOST2).

To add the host name used by Oracle Coherence:

1. Log in to the Oracle WebLogic Server Administration Console.

2. In the Domain Structure window, expand the Environment node.

3. Click Servers.

   The Summary of Servers page appears.

4. Select soa_server1 (represented as a hyperlink) from the column of the table.

   The Settings page appears.

5. Click Lock & Edit.

6. Click the Server Start tab.

7. Append the following for soa_server1 and soa_server2 to the Arguments field.

   For soa_server1:

   -Dtangosol.coherence.wka1=FINSOAVH1
   -Dtangosol.coherence.wka2=FINSOAVH2
   -Dtangosol.coherence.localhost=FINSOAVH1
   -Dtangosol.coherence.localport=8089
   -Dtangosol.coherence.wka1.port=8089
   -Dtangosol.coherence.wka2.port=8089
   

   For soa_server2:

   -Dtangosol.coherence.wka1=FINSOAVH2 
   -Dtangosol.coherence.wka2=FINSOAVH1
   -Dtangosol.coherence.localhost=FINSOAVH2
   -Dtangosol.coherence.localport=8089
   -Dtangosol.coherence.wka1.port=8089
   -Dtangosol.coherence.wka2.port=8089
   

   Note:

   There should be no breaks in lines between the different -D parameters. Do not copy or paste the code from above to your Administration Console's Arguments text field. This may result in HTML tags being inserted in the Java arguments. The code should not contain other characters than those included in the example above.

8. Click Save and Activate Changes.

Note:

You must ensure that these variables are passed to the Managed Server correctly. (They should be reflected in the server's output log.) Failure of the Oracle Coherence framework can prevent the soa-infra application from starting.

Note:

The multicast and unicast addresses are different from the ones used by the Oracle WebLogic Server cluster for cluster communication. Oracle SOA Suite guarantees that composites are deployed to members of a single Oracle WebLogic Server cluster even though the communication protocol for the two entities (the Oracle WebLogic Server cluster and the groups to which composites are deployed) are different.

Note:

By default, the Oracle Coherence cluster uses port 8088 for deployment. This port can be changed by specifying the -Dtangosol.coherence.wkaX.port startup parameter. To avoid a port number conflict when configuring coherence parameters in different domains, ensure that you increment port 8089 by 1, or choose the next free port in this sequence.

16.8 Configuring a JDBC Transaction Log Store for Transaction Recovery

Each server has a transaction log that stores information about committed transactions that are coordinated by the server that may not have been completed. Oracle WebLogic Server uses this transaction log for recovery from system crashes or network failures. To leverage the migration capability of the Transaction Recovery Service for the servers within a cluster, store the transaction log in a location accessible to a server and its backup servers.

To set the location for the transaction log store:

1. Log in to the Oracle WebLogic Server Administration Console (http://fininternal.mycompany.com:7777/console).

2. In the Change Center, click Lock & Edit.

3. In the Domain Structure window, expand the Environment node and then click the Servers node.

   The Summary of Servers page appears.

4. Click the server name soa_server2 (represented as a hyperlink) in the table. The Settings page for the selected server appears, and defaults to the Configuration tab.

5. In the Configuration tab, click the Services tab.

6. In the Transaction Log Store section of the page, do the following:

   1. For the type, change the dropdown list selection from Default Store to JDBC.

   2. For the data store, select SOALocalTxDataSource from dropdown list.

   3. For prefix name, enter FIN_FUSION_SOAINFRA.TLOG_soa_server2_. (Do not forget to include the ending "_".)

      Note:

      The prefix name is specific for each domain.

      • Oracle Fusion Financials domain: FIN_FUSION_SOAINFRA.TLOG_soa_server2_

      • Oracle Fusion Common domain: SETUP_FUSION_SOAINFRA.TLOG_soa_server2_

      • Oracle Fusion Human Capital Management domain: HCM_FUSION_SOAINFRA.TLOG_soa_server2_

      • Oracle Fusion Supply Chain Management domain: SCM_FUSION_SOAINFRA.TLOG_soa_server2_

      • Oracle Fusion Projects domain: PRJ_FUSION_SOAINFRA.TLOG_soa_server2_

      • Oracle Fusion Procurement domain: PRC_FUSION_SOAINFRA.TLOG_soa_server2_

      Note:

      When the JDBC transaction log store configuration is complete, and after bouncing the soa_server2 server, do the following:

      • Append the table name in the prefix name with WLStore. For example, TLOG_soa_server2_WLStore.

      • Verify that the new table, TLOG_soa_server2_WLStore, has been created in FIN_FUSION_SOAINFRA.

7. Click Save and then click Activate Changes.

8. Restart the Managed Server to activate the changes (ensure that Node Manager is up and running):

   1. Log in to the Oracle WebLogic Server Administration Console (http://fininternal.mycompany.com:7777/console).

   2. In the Summary of Servers screen, select the Control tab.

   3. Select soa_server2 in the table and then click Shutdown.

   4. Start the soa_server2 server.

16.9 Disabling Host Name Verification for the soa_servern Managed Servers

This step is required if you have not set up the appropriate certificates to authenticate the different nodes with the Administration Server. By default, Host Name Verification should be set to None. If it is not, follow the steps below.

If you have not configured the server certificates, you will receive errors when managing the different WebLogic servers. To avoid these errors, disable host name verification while setting up and validating the topology, and enable it again after the enterprise deployment topology configuration is complete.

To disable Host Name Verification:

1. Log in to Oracle WebLogic Server Administration Console. For example, http://fininternal.mycompany.com:7777/console.

2. Click Lock & Edit.

3. Expand the Environment node in the Domain Structure window.

4. Click Servers.

   The Summary of Servers page appears.

5. Select soa_server1 (represented as a hyperlink) in the table.

   The Settings page appears.

6. Select the SSL tab.

7. Expand the Advanced section of the page.

8. Set Hostname Verification to None.

9. Click Save.

10. Repeat Steps 1 through 9 for the soa_server2 Managed Server.

11. Save and activate the changes.

16.10 Restarting Node Manager on FINHOST1

To restart Node Manager on FINHOST1:

1. Stop Node Manager by stopping the process associated with it:

   1. If it is running in the foreground in a shell, simply use CTRL+C.

   2. If it is running in the background in the shell, find the associated process and use the kill command to stop it. For example:

      FINHOST1> ps -ef | grep NodeManager
      orcl 9139 9120 0 Mar03 pts/6 00:00:00/bin/sh ./startNodeManager.sh
      

   3. Run the following command:

      FINHOST1>kill -9 9139 
      

2. Start Node Manager:

   FINHOST1> ORACLE_BASE/config/nodemanager/FINHOST1/startNodeManagerWrapper.sh
   

16.11 Starting and Validating soa_server1 on FINHOST1

To start the soa_server1 Managed Server on FINHOST1:

1. Access the Administration Console. For example, http://fininternal.mycompany.com:7777/console.

2. Click Servers.

3. Open the Control tab.

4. Select soa_server1.

5. Click Start.

To validate the soa_server1 Managed Server on FINHOST1:

1. Verify that the server status is reported as "Running" in the Administration Console. If the server is shown as "Starting" or "Resuming," wait for the server status to change to "Started." If another status is reported (such as "Admin" or "Failed"), check the server output log files for errors.

2. Access http://FINSOAVH1:7416/soa-infra and http://fininternal.mycompany.com:7777/soa-infra to verify status of soa_server1.

   Note:

   Although the soa_server1 server may be up, some applications may be in a failed state. Therefore, Oracle recommends verifying the above URLs and watching for errors pertaining each individual application in the server's output file.

16.12 Restarting Node Manager on FINHOST2

To restart Node Manager on FINHOST2, follow the steps in Section 16.10, "Restarting Node Manager on FINHOST1."

16.13 Starting and Validating soa_server2 on FINHOST2

To start the soa_server2 Managed Server on FINHOST2 and ensure that it is configured correctly:

1. From the Administration Console, start the soa_server2 Managed Server.

2. Verify that the server status is reported as "Running" in the Administration Console. If the server is shown as "Starting" or "Resuming," wait for the server status to change to "Started." If another status is reported (such as "Admin" or "Failed"), check the server output log files for errors.

3. Access http://FINSOAVH2:7416/soa-infra and http://fininternal.mycompany.com:7777/soa-infra.

   Note:

   Although soa_server2 server may be up, some applications may be in a failed state. Therefore, Oracle recommends verifying the above URLs and watching for errors pertaining each individual application in the server's output file.