9 Scaling Out the Oracle Fusion Common Domain

This chapter describes how to scale out the Oracle Fusion Common domain.

This chapter includes the following topics:

• Section 9.1, "Overview of the Oracle Fusion Common Domain"

• Section 9.2, "Prerequisites for Scaling Out the Oracle Fusion Common Domain"

• Section 9.3, "Adding a New Machine in the Oracle WebLogic Server Console"

• Section 9.4, "Creating a Common Location for the Oracle WebCenter Content Managed Servers"

• Section 9.5, "Packing and Unpacking the Managed Server Domain Home"

• Section 9.6, "Cloning Managed Servers and Assigning Them to CRMHOST2"

• Section 9.7, "Scaling Out the WebCenter Content Inbound Refinery Server"

• Section 9.8, "Oracle HTTP Server Configuration"

• Section 9.9, "Configuring Server Migration for the Managed Servers"

• Section 9.10, "Validating the System"

9.1 Overview of the Oracle Fusion Common Domain

The Oracle Fusion Common domain is shared by all Oracle Fusion Applications product families. Oracle Fusion Customer Relationship Management implementation is dependent on the Oracle Fusion Common domain for the following components:

• Oracle Fusion Functional Setup Manager, for all setup task flows

• Help Portal, for centralized help

• Home page application, which has the Oracle Fusion Customer Relationship Management home page and launch pad

• Oracle WebCenter Content, to store all marketing collateral as well as all attachments. Import flow also uses WebCenter Content to stage the CSV files that the user uploads

• Oracle Secure Enterprise Search

• Oracle WebCenter Community Space and forums

• Oracle WebLogic Communication Services (OWLCS) and Oracle WebLogic SIP Server

Figure 2-4 in Chapter 2, "Introduction to the Enterprise Deployment Reference Topologies," shows the topology for the Oracle Fusion Common domain within the overall reference enterprise deployment topology.

9.2 Prerequisites for Scaling Out the Oracle Fusion Common Domain

Before you begin, ensure the following:

• Node Manager has been started in the Secure Sockets Layer (SSL) mode by following the instructions in Chapter 7, "Setting Up Node Manager for an Enterprise Deployment"

• You are starting with a clean machine if it is the first time it is being used for a scale out

• The /etc/hosts file has proper entries. To verify, ping this machine with the fully qualified name of the machine

• The user created on CRMHOST2 should be the same as the user on CRMHOST1

• The directory structure /u01/oracle is mounted to same shared file system as CRMHOST1

• The directory structure /u02/local/oracle/config on CRMHOST2 has been created

• The initial Oracle Fusion Customer Relationship Management deployment on CRMHOST1 has already been done and verified by provisioning

9.3 Adding a New Machine in the Oracle WebLogic Server Console

To add a new machine:

1. Log in to the Administration Server: http://commoninternal.mycompany.com:7777/console.

2. Navigate to CommonDomain > Environment > Machines.

   LocalMachine is located in the right-hand pane.

3. In the left-hand pane, click Lock & Edit.

4. In the right-hand pane, first click New to add the remote machine, and then specify the following:

   • Name - enter CRMHOST2

   • Machine operating system - Unix

5. Click Next.

6. In the window that opens, set the following attributes:

   • Type - SSL

   • Listen Address - <CRMHOST2>

     Note:

     The "localhost" default value here is wrong.

   • Listen port - 5556

7. Click Finish and activate the changes.

   Note:

   If you get an error when activating the changes, see Section 19.8.18, "Administration Console Redirects from Internal URL to Container URL after Activation" for the temporary solution.

9.4 Creating a Common Location for the Oracle WebCenter Content Managed Servers

Do the following:

1. Ensure that the UCM_server1 Managed Server is functioning properly. You should be able to access and log in to the following:

   • http://CRMHOST1:7012/cs

   • http://CRMHOST1:7012/ibr

2. Shut down the UCM_server1 Managed Server:

   1. Log in to the Oracle WebLogic Server Administration Console (http://commoninternal.mycompany.com:7777/console).

   2. In the Domain Structure window, first navigate to Environment > Servers, and then select the Control tab.

   3. Select the UCM_server1 checkbox.

   4. Click Shutdown and then Force Shutdown Now.

3. Copy CRMHOST1 /u02/local/oracle/config/domains/CRMHOST1/CommonDomain/ucm/cs directory from the local to the shared location: ORACLE_BASE/config/domains/CRMHOST1/CommonDomain/ucm/cs.

4. In the ORACLE_BASE/config/domains/CRMHOST1/CommonDomain/ucm/cs/bin/intradoc.cfg file and the /u02/local/oracle/config/domains/CRMHOST1/CommonDomain/ucm/cs/bin/intradoc.cfg file on CRMHOST1, create the variables IntradocDir, vaultDir, and weblayoutDir if they do not already exist, and point them to the shared location, ORACLE_BASE/config/domains/CRMHOST1/CommonDomain/ucm/cs.

5. Start the UCM_server1 Managed Server.

9.5 Packing and Unpacking the Managed Server Domain Home

Since the CRMHOST1 domain directory file system is also available from CRMHOST2, both the pack and unpack commands can be executed from CRMHOST2.

To pack and unpack the Managed Server domain home:

1. Change directory to ORACLE_BASE/products/fusionapps/oracle_common/common/bin.

2. Run the pack command:

   CRMHOST2> ./pack.sh -managed=true -domain=ORACLE_BASE/config/domains/
   CRMHOST1/CommonDomain -template=ORACLE_BASE/user_templates/
   CommonDomain_managed.jar -template_name="Common_Managed_Server_Domain" 
   

3. Ensure that /u02/local/oracle/config/domains/CRMHOST2/CommonDomain is empty, and then run the unpack command:

   CRMHOST2> ./unpack.sh -domain=/u02/local/oracle/config/domains/CRMHOST2/
   CommonDomain -template=ORACLE_BASE/user_templates/CommonDomain_managed.jar
   

   Here, ORACLE_BASE is shared, and /u02/local is local to CRMHOST2.

9.6 Cloning Managed Servers and Assigning Them to CRMHOST2

To add a Managed Server and assign it to CRMHOST2:

1. Log in to the Administration Server: http://commoninternal.mycompany.com:7777/console.

2. Navigate to CommonDomain > Environment > Servers.

3. Switch to Lock & Edit mode.

4. Select the Managed_Server checkbox (for example, HomePageServer_1) and then click Clone.

5. Specify the following Server Identity attributes:

   • Server Name - HomePageServer_2

     Note:

     To ensure consistency in naming, copy the name of the server shown in Server Identity and paste it into the Server Name field. Then change the number to "_2".

   • Server Listen Address - <CRMHOST2>

   • Server Listen Port - leave "as is"

     Note:

     For Oracle SOA Suite server, add a port value that is different than the soa_server1 server value. This will help in server migration.

6. Click OK.

   You now should see the newly cloned sales server, HomePageServer_2.

7. Click HomePageServer_2 and change the following attributes:

   • Machine - <CRMHOST2>

   • Cluster Name - accept the default, HomePageCluster

     Note:

     Ensure that this cluster name is the same as the cluster name of the original Managed Server.

8. Click Save and then Activate Changes.

9. Navigate to CommonDomain > Environment > Servers.

10. From the Name column, click the HomepageServer_2 scaled-out server link.

11. Click Lock & Edit, and then select the Configuration tab.

12. Select the Keystores tab, and then ensure that the keystores value is Custom Identity and Custom Trust.

13. Do the following:

    1. Change the Custom Identity Keystore path to point to the ORACLE_BASE/config/keystores/CRMHOST2_fusion_identity.jks file.

    2. Leave the Custom Identity Keystore type blank.

    3. Change the Custom Identity Keystore Passphrase entry. This should be the same as the keystorepassword field described in the first bullet in Step 4 in Section 7.2, "Creating the Identity Keystore on CRMHOST2."

    4. Re-enter the Confirm Custom Identity Keystore Passphrase.

    5. Ensure that the Confirm Custom Trust Keystore path is pointing to the ORACLE_BASE/config/keystores/fusion_trust.jks file.

    6. Leave the Custom Trust Keystore type blank.

    7. Change the Custom Trust Keystore Passphrase entry. This should be the same as the keystorepassword field described in the first bullet in Step 4 in Section 7.2, "Creating the Identity Keystore on CRMHOST2."

    8. Re-enter the Custom Trust Keystore Passphrase.

    9. Click Save.

14. Select the SSL tab.

    1. Make sure that Identity and Trust Locations is set to Keystores.

    2. Change the Private Key Alias to CRMHOST2_fusion.

    3. Change the Private Key Passphrase to the keypassword, as described in the second bullet in Step 4 in Section 7.2, "Creating the Identity Keystore on CRMHOST2."

    4. Re-enter the keypassword from Step c for the Confirm Private Key Passphrase.

    5. Click Save.

15. Select the Server Start tab.

    Change the Arguments to reflect the name of your cloned Managed Server and make sure the server group is the correct cluster name. For example, you should see the following:

    -DJDBCProgramName=DS/CommonDmain/HomePageServer_2
    -Dserver.group=HomePageCluster
    

    Click Save.

16. Select the Logging tab, and then select the HTTP tab.

17. Do the following:

    1. Change the Log file name to logs/access.log.%yyyyMMdd%.

    2. Change the rotation type to By Time.

    3. Leave the Limit number of retained files option unchecked.

    4. Leave the Rotate log file on startup option unchecked.

    5. Click Save.

    6. Expand Advanced.

    7. Change the format to Extended.

    8. Change the extended logging format fields to the following:

       date time time-taken cs-method cs-uri 
       sc-status sc(X-ORACLE-DMS-ECID) 
       cs(ECID-Context) cs(Proxy-Remote-User) 
       cs(Proxy-Client-IP)
       

    9. Click Save.

18. Click Activate Changes.

19. Repeat Steps 2 to 18 for all the Managed Servers on this domain.

20. Set the following environment variable on CRMHOST2:

    WLST_PROPERTIES="-Dweblogic.security.SSL.trustedCAKeyStore=ORACLE_BASE/
    config/keystores/fusion_trust.jks"
    

21. Stop the domain's Administration Server:

    CRMHOST1> ORACLE_BASE/config/domains/CRMHOST1/CommonDomain/bin/stopWebLogic.sh
    

22. Restart the domain's Administration Server:

    CRMHOST2> ORACLE_BASE/products/fusionapps/wlserver_10.3/common/bin/wlst.sh
    
    CRMHOST2> nmConnect(username='<username>', password='<password>',
    domainName='CommonDomain', host='CRMHOST1',port='5556', 
    nmType='ssl', domainDir='ORACLE_BASE/config/domains/CRMHOST1/CommonDomain')
    
    CRMHOST2> nmStart('AdminServer')
    

    Note:

    The username and password used in the nmConnect are the Node Manager credentials (username and password) specified when creating the provisioning response file. This is shown in Figure 5-3 in "Using the Provisioning Process to Install Components for an Enterprise Deployment".

23. For the wlcs_sipstate2 scaled-out server only, do the following:

    1. Change directory to:

       ORACLE_BASE/config/domains/CRMHOST1/CommonDomain/config/custom
       

    2. In the datatier.xml file, after the following line:

       <server-name>wlcs_sipstate1</server-name>
       

       add

       <server-name>wlcs_sipstate2</server-name>
       

24. Ensure that the edits you made in Step 4 in Section 9.4 you also make to the intradoc.cfg file in /u02/local/oracle/config/domains/CRMHOST2/CommonDomain/ucm/cs/bin.

25. Run the newly created Managed Servers:

    1. Log in to the Administration Server: http://commoninternal.mycompany.com:7777/console.

    2. Navigate to CommonDomain > Environment > Servers > Control.

    3. Select the newly created Managed Server and click Start.

    4. Navigate to CommonDomain > Environment > Servers and check the State to verify that the newly created Managed Servers are running.

9.7 Scaling Out the WebCenter Content Inbound Refinery Server

You also must scale out the WebCenter Content Inbound Refinery server.

Note:

/u02/local/oracle/config/domains/CRMHOST1/CommonDomain/ucm/ibr on CRMHOST1 must already be on the local disk.

1. Edit the /u02/local/oracle/config/domains/CRMHOST1/CommonDomain/ucm/ibr/config/config.cfg file on CRMHOST1 with the following:

   • IDC_Name=CRMHOST1usoraclecom7012 (In this instance, CRMHOST1 is not the fully qualified name.)

   • InstanceMenuLabel=CRMHOST1usoraclecom7012

   • SocketHostAddressSecurityFilter=127.0.0.1|cluster node's addresses|CRMHOST1 IP ADDRESS|CRMHOST2 IP ADDRESS|0.0.0.0.0.0.0.1 (The cluster node's address is optional.)

   • HttpServerAddress=CRMHOST1:7012 (CRMHOST1's server address instead of load balancer, as it is a singleton node.)

2. Restart the ucm_server2 server.

3. Navigate to http://CRMHOST2:7012/ibr (CRMHOST2's ibr) and log in when prompted.

   The post-installation configuration page displays, as it is not a clustered node.

4. On the post-installation configuration page:

   1. Set the server socket port to 7035.

   2. Ensure the port has a unique server instance name and unique local paths on CRMHOST2.

   3. Set the Incoming Socket Connection Address Security Filter:

      SocketHostAddressSecurityFilter=127.0.0.1|
      cluster node's addresses|CRMHOST1 
      IP ADDRESS|CRMHOST2
      IP ADDRESS|0.0.0.0.0.0.0.1 
      

      Note:

      The cluster node's address is optional.

   4. Click Submit.

   5. Restart the ucm_server2 server.

5. Navigate to http://CRMHOST1:7012/cs and select Administration > providers.

   ibrprovider is already defined in this list.

6. Add a new outgoing provider:

   1. Click Add.

   2. Set the values to match the values of ibrprovider.

      Note:

      The value for the new outgoing ibrprovider must differ from the value of the default ibrprovider.

   3. Assign the following unique values: to Provider Name, Provider Description, Server Host Name, Server Port, Instance Name, and Relative Web Root.

      – Provider Name: ibr_provider2

      – Provider Description: ibrprovider for second server

      – Server Host Name: the second host address's CRMHOST2 IP address

      – Server Port: 7035

      – Instance Name: CRMHOST2usoraclecom7012

      – Relative Web Root: /cs

7. Navigate to Inbound Refinery on second node: http://CRMHOST2:7012/ibr.

8. Select Conversion Settings and update the primary web rendition.

9. Navigate to Third-Party Applications Settings > General OutsideIn Filter Options > Options and set the following path to the fonts: /usr/share/X11/fonts/TTF.

   Note:

   The font location can be specific to the operating system.

10. Restart the UCM_server1 and UCM_server2 Managed Servers.

11. Create a COMMONUCMVH1 TCP VIP with an unused port on the load-balancing router (LBR) with CRMHOST1:7012 and CRMHOST2:7012 as its members.

After scaling out the Oracle WebCenter Portal Managed Server, do the following:

1. Log in to: http://commoninternal.mycompany.com:7777/em.

   Note:

   If you don't see WebCenter Content after logging in to Oracle Enterprise Manager, bounce the WebCenter Content Managed Servers.

2. Navigate to Farm_CommonDomain > WebCenter >Portal > Spaces > webcenter ( 1.1.*.*) (wc_spaces) > WebCenter Portal (top left or right pane) > Settings > Service configuration.

   Click Content Repository.

3. Highlight WebCenter-UCM and click Edit.

4. Select Socket and enter COMMONUCMVH1 as the host and LBR port for the server.

5. If updating the first instance of wc_spaces did not automatically update the second instance, repeat Steps 2 through 4 for wc_spaces2.

6. Restart the Oracle WebCenter Portal Managed Servers.

9.8 Oracle HTTP Server Configuration

To configure Oracle HTTP Server:

1. On WEBHOST1:

   1. Change directory to ORACLE_BASE/config/CommonDomain_webtier/config/OHS/ohs1/moduleconf.

   2. Copy FusionVirtualHost_fs.conf to FusionVirtualHost_fs.conf.org.

2. Edit the FusionVirtualHost_fs.conf file, adding the scaled-out host and port to all the WebLogic Application Clusters. Add this to both the Internal and External part of the FusionVirtualHost_fs.conf file. Example 9-1 shows sample code for HomePageServer.

   Notes:

   • Do not add these values for Oracle Enterprise Manager, Oracle WebLogic Server Administration Console, or Oracle Authorization Policy Manager.

   • If the Managed Servers are running on VIPs, replace CRMHOST1 and CRMHOST2 with the VIP addresses in Example 9-1.

   Example 9-1 Sample "HomePageServer" Code

   <Location /HomePage>
           SetHandler weblogic-handler
           WebLogicCluster <CRMHOST1FINHOST1:port>,<CRMHOST2FINHOST2:port>
       </Location>
   

3. Repeat Step 2 for all applications.

4. Restart Oracle HTTP Server: cd to ORACLE_BASE/config/CommonDomain_webtier/bin and enter the following:

   WEBHOST1> ./opmnctl stopall
   WEBHOST1> ./opmnctl startall
   

5. Repeat Steps 1 through 4 on WEBHOST2.

9.9 Configuring Server Migration for the Managed Servers

Server migration is required for proper failover of Oracle Fusion Customer Relationship Management components in the event of failure in any of the CRMHOST1 and CRMHOST2 nodes. For more information, see Chapter 17, "Setting Up Server Migration for an Enterprise Deployment."

9.10 Validating the System

You should verify URLs to ensure that the appropriate routing and failover are working.

To verify the URLs:

1. Log in to the CommonDomain Oracle WebLogic Server Administration Console and stop all the Managed Servers on CRMHOST1 while the Managed Servers on CRMHOST2 are running.

2. Access the following URLs to verify that routing and failover are functioning properly. (Ensure the log in prompt is visible.)

   • https://commonexternal.mycompany.com/helpPortal/faces/AtkHelpPortalMain

   • https://commonexternal.mycompany.com/homePage/faces/AtkHomePageWelcome

3. Log in to the CommonDomain Oracle WebLogic Server Administration Console and stop all the Managed Servers on CRMHOST2.

4. Start the Managed Servers on CRMHOST1.

5. Repeat Step 2. (Ensure the log in prompt is visible.)

6. Start all the Managed Servers on CRMHOST2 and verify that they are running on CRMHOST1 and CRMHOST2.