Installing and Configuring the WebLogic Server Cluster

5 Installing and Configuring the WebLogic Server Cluster

This chapter describes how you can install and configure a WebLogic Server cluster in preparation for an Oracle Communications Order and Service Management (OSM) installation.

Preparing WebLogic Server for an OSM Cluster Installation

You prepare a WebLogic Server clustered environment for an OSM installation by doing the following:

Prepare the operating system that will host WebLogic Server. See "Preparing the Operating System."
Install WebLogic Server. See "Installing WebLogic Server Software."
Create required database schemas using the Repository Creation Utility (RCU). See "Creating Database Schemas Using RCU."
Create the WebLogic Server domain and configure the required server instances and cluster. See "Creating the WebLogic Server Domain."
Replicate the WebLogic Server domain on all the machines within the domain. See "Replicating the Domain on Other Machines."
Configure the WebLogic Server domain and managed servers. See "Configuring the Domain and Managed Servers."

Preparing the Operating System

Use the following system and user limits:

Core file size: Limit core file size to zero. If a core dump occurs or the JVM crashes, very large memory and data heaps might be written to the disk. Oracle recommends setting a positive value for core file size only if a crash occurs and must be debugged on the next occurrence.
Number of open files: OSM typically references and loads large numbers of internal and third-party JAR files. Also, each application opens and maintains several configuration files, log files, and numerous network socket and JDBC connections. All these activities use a large number of open files, during startup and during application deployment and redeployment. Oracle recommends increasing the number of open file limits for OSM.
Number of user processes: For the same reasons as increasing the number of open files, Oracle recommends increasing the limit for user processes.
Socket buffers: To help minimize packet loss for the Coherence cluster, Oracle recommends setting the socket buffers of the operating system to at least 2 MB. For deployments with a high order processing rate, Oracle recommends 16 MB.
Number of clients: Set somaxconn should be set to at least 1024 to allow for a large number of client server connections.
Number of queued packets: Set netdev_max_backlog to at least 32768 to minimize packet loss.

Table 5-1 summarizes of the recommendations for configuring the operating system.

Table 5-1 Configuration Recommendations by Operating System

Configuration Item	Solaris	Linux
core file size (soft)	0	0
core file size (hard)	0	0
open files (soft)	8192	65536
open files (hard)	65536	65536
max_user_processes (soft)	29995	774889
max_user_processes (hard)	29995	774889

Note:

With engineered systems, such as Sparc SuperCluster or Exadata, most of the operating system tuning is preconfigured with appropriate values by default.

Installing WebLogic Server Software

The software for WebLogic Server and Application Development Framework (ADF) is included in the OSM software media pack. You download the OSM software media pack from the Oracle software delivery website:

https://edelivery.oracle.com/

You install WebLogic Server on all machines that will participate in your domain. The installation directories must be the same on all machines. For complete installation instructions and general information about installing and configuring WebLogic Server, see the WebLogic Server documentation.

Note:

See OSM Compatibility Matrix for WebLogic Server version and patch information. Ensure that you use the WebLogic Server documentation specific to the required WebLogic Server version.

To install WebLogic Server:

Ensure that you have installed the 64-bit version of Java and the Java Development Kit (JDK) that is supported by OSM.
Set environment variables for the version of Java that is supported by OSM, not the version included with WebLogic Server. Do the following:
- Set JAVA_HOME to the location of the supported Java version.
- On a UNIX system, add $JAVA_HOME/bin to the PATH variable.
- On a Windows system, add %JAVA_HOME%/bin to the PATH variable.
Install the WebLogic Server software as described in Oracle Fusion Middleware Installing and Configuring Oracle WebLogic Server and Coherence. When prompted for the installation type, select Complete.
Download and install any necessary patches from Oracle support. Follow the instructions in the README.txt file that is included with the patch.

Creating Database Schemas Using RCU

After you install the WebLogic Server software, create schemas in the database. You create the schemas using the Repository Creation Utility (RCU), which is included in the WebLogic Server installation. For complete information about using RCU, see Oracle Fusion Middleware Creating Schemas with the Repository Creation Utility.

The schemas are required for creating the WebLogic Server domain. Each schema can be used by only one domain. If you create a new domain, you must also create new schemas.

Before creating the schemas, ensure that you have your database connection string, port, administrator credentials, and service name ready.

When you create database schemas using RCU, keep the following considerations in mind:

When entering database details for an Oracle RAC instance, use the host name of one of the Oracle RAC instances. Do not use the SCAN IP address.
When selecting components, ensure that you create a new prefix and select all of the following components:
- Common Infrastructure Services (prefix_STB)
- Oracle Platform Security Services (prefix_OPSS)
- Audit Services (prefix_IAU)
- Audit Services Append (prefix_IAU_APPEND)
- Audit Services Viewer (prefix_IAU_VIEWER)
When prompted for passwords, select Use the same password for all schemas and enter the password of your choice.
Use the default tablespace configuration provided by the installer.

Note:

RCU may take several minutes to create the schemas. If creating the schemas takes an unusual amount of time, Oracle recommends that you purge the database recycle bin to ensure that RCU schemas are created more quickly the next time. For more information, see "OSM and RCU Installers Are Slow to Run Database Tablespace Query."

Creating the WebLogic Server Domain

This section describes how to create the WebLogic server domain and configure the required server instances and cluster using the Fusion Middleware Configuration Wizard.

For more information about WebLogic clustering, refer to the WebLogic Server documentation. For more information about the Configuration Wizard, including detailed descriptions of the Configuration Wizard screens, see Oracle Fusion Middleware Creating WebLogic Domains Using the Configuration Wizard.

To configure the WebLogic server domain for an OSM cluster:

Ensure the following:
- If you are configuring the managed servers for whole server migration, verify that you have available interfaces on each machine hosting a managed server. For example, eth0. Ensure that the IP address you want to use for each managed server is configured with a host name on a DNS server.
  
  Note:
  
  After you have completed all procedure relating to whole server migration, node manager automatically creates floating IP addresses for each managed server when you start managed servers from the Administration console. For example, assuming you have configured a machine to run two managed servers, node manager would create eth0:1 for managed server 1 and eth0:2 for managed server 2.
- If you are configuring managed servers without whole server migration, ensure that you have static IP addresses and port numbers for each managed server.
Tip:

Oracle recommends that you use DNS host names to facilitate any IP address changes during the lifetime of the domain.
Ensure that the WebLogic Server software has been installed on each machine that will be part of the domain.
Log on to the machine that will be hosting the administration server in your domain.
Do one of the following:
- On a Windows platform, start the configuration wizard from the Start menu.
- On UNIX platforms, run the following command:
```
WebLogic_home/common/bin/config.sh
```
The Configuration Wizard launches and the Configuration Type screen is displayed.
Ensure that Create a new domain is selected, and enter the path to the domain in the Domain Location field, and click Next.

The Templates screen is displayed.
Ensure that Create Domain Using Product Templates is selected, choose the following templates from the Available Templates list:

On the Templates screen:
- Select the Oracle JRF and WebLogic Coherence Cluster Extension templates.
- Select the Oracle Enterprise Manager template if you want to use Oracle Enterprise Manager Fusion Middleware Control to view and manage OSM logs.
- The Basic WebLogic Server Domain template is selected by default and you cannot deselect it.
Click Next.
Do one of the following:
- If you selected Oracle Enterprise Manager, the Application Location screen is displayed. Go to step 9.
- If you did not select Oracle Enterprise Manager, The Administrator Account screen is displayed. Go to step 11.
In the Application location field, enter the path to the application directory.
Click Next.

The Administrator Account screen is displayed.
In the Name field, enter the WebLogic Administrator user name.
In the Password field and again in the Confirm password field, enter the WebLogic Administrator password.

Note:

Alphanumeric characters are mandatory in the Password field.
Click Next.

The Domain Mode and JDK screen is displayed.
Select Production Mode. Selecting this ensures that the default optimization settings are applied.
Select the default JDK, or browse to the location of a 64-bit JDK, and click Next.

The Database Configuration Type screen is displayed.
Connect to the database by doing the following:
1. Select the RCU Data option.
2. In the Vendor and Driver drop-down lists, use the default selection.
3. In the DBMS/Service field, enter the database DBMS name, or service name if you selected a service type driver.
4. In the Host Name field, enter the name of the server hosting the database. The host name is the Oracle RAC instance you used to create the RCU repository.
5. In the Port field, enter the port number on which the Oracle RAC instance listens.
6. In the Schema Owner field, enter the user name for connecting to the service table schema. For example:
```
prefix_STB
```
  where prefix is the new prefix you created on the Select Components screen of RCU.
7. In the Schema Password field, enter the password that you created for the schemas on the Schema Passwords screen of RCU.
8. Click Get RCU Configuration. The Configuration Wizard tests the connection to the database and displays the results in the Connection Result Log area.
9. Click Next.
The JDBC Component Schema screen is displayed with pre-loaded configuration data for the RCU schemas.
Verify the pre-loaded schema configuration data for each of the following schemas in the component schema table:
- OPSS Audit Schema
- OPSS Audit Viewer Schema
- OPSS Schema
- LocalSvcTbl Schema
Select all schemas in the component schema table.
Select Convert to RAC multi data source.
Click Next.

The Oracle RAC Multi Data Source Component Schema screen is displayed.
In the first row of the Host Name column, enter the host name of the second Oracle RAC instance.
In the first row of the Instance Name column, enter the SID of the second Oracle RAC instance.
In the first row of the Port column, enter the port number of the Oracle RAC listener.
If you have additional Oracle RAC instances, click Add Hosts to add new Oracle RAC instance rows and repeat steps 21 to 23.
In the Service Name field, enter the service name.
Click Next.

The JDBC Component Schema Test screen is displayed.
Select all of the component schemas in the table and click Test Selected Connections.

If any of the tests fail, go back to the previous screens and check that you entered the correct values in the fields.

When all of the tests are successful, click Next.

The Advanced Configuration screen is displayed.
Select Administration Server, Node Manager, Managed Servers, Clusters and Coherence, and Deployments and Services, and click Next.

The Administration Server screen is displayed.
In the Server Name field, enter a name for the administration server.
In the Listen Address field, enter a value for the listen address. For example, an IP address or DNS name. Oracle recommends that you use a specific host name in the Listen Address field to facilitate any future IP address changes that may occur. In addition, avoid selecting All Local Addresses because this causes WebLogic Server to bind to every available IP address on the machine, potentially reducing performance.
In the Listen Port field, enter a value for the listen port.
(Optional) Select Enable SSL and enter a value for the SSL listen port in the SSL listen Port field. Oracle recommends enabling SSL and configuring an SSL listen port to ensure secure communication over the Internet.
Click Next.

The Node Manager screen is displayed.
Select Per Domain Default Location for the Node Manager type.
In the Username field, enter a user name for Node Manager.
In the Password field and again in the Confirm Password field, enter a password for Node Manager.

The Node Manager uses the user name and password that you provide to authenticate connections between Node Manager and clients. They are independent from the server administration credentials.
Click Next.

The Managed Servers screen is displayed.
Click Add, which creates a managed server. Repeat this step for any additional managed servers.

Note:

If you are setting up a development system and you do not have a dedicated load balancer for HTTP and HTTPS messages, you must add a managed server that you can later designate as a proxy HTTP server.
Enter managed server names, IP addresses or host names (Oracle recommends DNS host names for high availability), and port numbers. If required, enable SSL and enter an SSL listen port. Click Next.

Note:

If you are using a proxy HTTP server, you can also designate a managed server as the proxy HTTP server during this step.

The Clusters screen is displayed.
Click Add, which adds a cluster row.
In the Name field, enter a name.
In the Cluster address field, add the address for the cluster. The cluster address should be one of the following:
- If you are using an external load balancer, this should be the address of the DNS name that maps to the IP addresses or DNS names of each WebLogic managed server in the cluster. The managed servers should all have addresses with different IP addresses or DNS names but the same port numbers. For information on how to set this up, please consult the documentation for your load balancer.
- If you are using a proxy HTTP server, you can define a list that contains the DNS name (or IP address) and port of each managed server in the cluster, delimited by a comma. For example:
```
host1.example.com:9910,host1.example.com:9920,host2.example.com:9100
```
  For performance reasons, this option is primarily used for development and test systems.
Click Next.

The Assign Servers to Clusters screen is displayed.
In the Server area, select all managed servers. Do not select the proxy HTTP server (if applicable).
Click the right arrow, which moves all managed servers under the cluster.
Click Next.

The HTTP Proxy Applications screen is displayed.
If you designated a managed server to be a proxy HTTP server, select Create HTTP Proxy, select the proxy HTTP server from the list, and click Next.

The Coherence Clusters window is displayed.
In the Name field, enter a name for the Coherence cluster.
In the Unicast Listen Port field, leave the default value of 0.
Click Next.

The Machines screen is displayed.
On the Unix Machine tab:
1. Click Add to add any UNIX machines in your domain that will run Node Manager.
2. Enter machine names, Node Manager listen addresses, and Node Manager listen ports.
3. (Optional) Select Post bind GID enabled and enter the UNIX group ID under which a server on this machine will run after it finishes all privileged startup actions.
4. (Optional) Select Post bind UID enabled and enter the UNIX user ID under which a server on this machine will run after it finishes all privileged startup actions.
Click Next.

The Assign Servers to Machines window is displayed.
Add the available servers to the appropriate machines as decided upon in your configuration details. Click Next.

The Deployments Targeting screen is displayed.
Verify that all of the deployments under the Library and Application folders are targeted to your new cluster and the administration server. If they are not, assign deployments to your cluster and administration server as follows:
1. In the Targets list, select the cluster where you want to install OSM.
2. In the Deployments list, select the Library and Application folders.
  
  Note:
  
  Do not select any deployments to be assigned to the proxy server (if used).
3. Click the right arrow to include the OSM components you want to install on the cluster.
Click Next.

The Services Targeting screen is displayed.
Target services to your cluster and proxy server as follows:
1. In the Targets list, select the cluster where you want to install OSM.
2. In the Services list, select all of the services that you want install on the cluster.
3. Click the right arrow to assign the services to the cluster.
4. If you are using a proxy server, select it in the Targets list.
5. In the Services list, select the JDBC System Resources folder.
6. Click the right arrow to assign the JDBC data sources to the proxy server.
7. Click Next.
The Configuration Summary screen is displayed, which provides a summary of the applications, services, and libraries to be deployed in the domain.
Review the information in the Configuration Summary screen and confirm that the cluster organization matches your requirements. If you find any discrepancies, click the Previous button to return to the appropriate screen and make the necessary changes. When you are done, click Create.

The Configuration Progress screen shows the progress of your domain creation.
When the domain creation process completes and the Configuration Success screen is displayed, click Finish.

Replicating the Domain on Other Machines

The newly created domain is now installed on a single machine. This section describes the steps necessary to replicate the domain installation on other machines. WebLogic provides two utilities to do this: pack and unpack.

Note:

If the unpack.jar command fails with a write error, see "Command for unpack.jar Fails with a Write Error" for troubleshooting information.

Starting and Configuring Credentials on the First Machine

Before you can replicate the domain to another machine, you must first start nodemanager and the administration server on the first machine. You can then create a boot.properties file for the administration server to enable each server to start without prompting you for the administrator username and password.

To create the boot.propeties file:

Open a terminal.
Go to domain_home/bin for your base domain.
Run the following command which starts nodemanager:
```
startNodeManager.sh
```
Open a second terminal.
Go to domain_home/bin for your base domain.
Run the following command which starts the administration server:
```
startWebLogic.sh
```
When the following text appears, enter the administration server username:
```
Enter username to boot WebLogic server:
```
When the following test appears, enter the administration server password:
```
Enter password to boot WebLogic server:
```
After the administration server is in the running state, open a third terminal.
Go to the domain_home/servers/AdminServer directory (where AdminServer is the name of the administration server.)
Create the following directory:
```
mkdir -p security
```
In the directory you just created, create the boot.properties file.
```
touch boot.properties
```
Using a text editor, add the following lines to the file:
```
username=username
password=pwd
```
where
- username is the administrator user name for the WebLogic administration server.
- pwd is the password for the WebLogic administration server.
These values are entered in clear text but will be encrypted when you start the server for the first time.
Save and close the boot.properties file.
Close the third terminal.
In the second terminal, stop the administration server using Ctrl-C.
Close the second terminal.
In the first terminal, stop node manager using Ctrl-C.
Close the first terminal.

Creating a Domain Template for Use on Other Machines

To create a domain directory (template) that can be used on other machines within the domain:

On the machine that contains the administration server and the definition of managed servers, go to the WebLogic_home/common/bin directory.
Run the following command:
```
pack.sh -domain=domain_home -template=template.jar -template_name="template_name" 
```
where:
- domain_home is the full or relative path of the WebLogic domain from which the template is to be created
- template is the full or relative path of the template, and the filename of the template to be created
- template_name is a descriptive name for the template
For example:
```
pack.sh -domain=/opt/oracle/Middleware/user_projects/domains/cluster_demo -template=/opt/oracle/Middleware/user_projects/domains/cluster_demo.jar -template_name="cluster_demo"
```

Replicating the Domain Template on Other Machines

Use the following steps to replicate the created template file to all other machines in the domain.

Establish a session with the remote machine and copy the template to it.
Navigate to the WebLogic_home/common/bin directory.

Run the following command:

unpack.sh -template=template.jar -domain=domain

where:

template is the full or relative path of the template that you copied to the remote machine
domain is the full or relative path of the domain to be created

For example:

unpack.sh -template=/opt/oracle/Middleware/user_projects/domains/cluster_demo.jar -domain=/opt/oracle/Middleware/user_projects/domains/cluster_demo

Starting the Administration Server

To start the WebLogic Server administration server:

Navigate to the domain_home directory of the machine that runs the administration server, and start the server by running the following command:
```
nohup ./startWebLogic.sh 2>&1 &
```
Verify that the administration server starts properly by running the following command:
```
tail -f nohup.out
```
A log entry should indicate that the server is running with the following line:
```
Server started in RUNNING mode
```

Configuring the Domain and Managed Servers

The following sections describe how you should configure the domain and managed servers.

Configuring Oracle Coherence for an OSM Cluster

This section provides configuration suggestions and best practices to avoid conflicts with Oracle Coherence in a clustered OSM environment.

For information about configuring and troubleshooting Oracle Coherence, refer to the Coherence documentation. For performance tuning details, see the Coherence Knowledge Base website:

https://docs.oracle.com/cloud/latest/fmw121300/COHAG/tune_perftune.htm#COHAG217

Increasing Buffer Sizes to Support Coherence

Oracle recommends that you configure your system for larger buffers:

On Oracle Linux and Red Hat Enterprise Linux, run the following commands as root:

sysctl -w net.core.rmem_max=2097152
sysctl -w net.core.wmem_max=2097152

Note:

This change can be made permanent by adding, or changing the values of, the following parameters in /etc/sysctl.conf:

net.core.rmem_max=2097152
net.core.wmem_max=2097152

On Oracle Solaris run the following command as root:

ndd -set /dev/udp udp_max_buf 2097152

On IBM AIX run the following command as root:

no -o rfc1323=1
no -o sb_max=4194304

Note:

Windows does not impose a buffer size restriction by default, so no changes must be made to increase buffer sizes on Windows.

Preventing Unnecessary Use of Swap Space

Do not use swap space in an OSM production environment for the WebLogic Servers if at all possible because it can significantly increase the duration of full garbage collection. Setting the swappiness to 0 ensures that the operating system only uses swap space when absolutely necessary.

To set swappiness to 0, do the following:

On a machine running a managed server, log into a terminal as root user.

Run the following command:

echo vm.swappiness=0 >> /etc/sysctl.conf

Repeat this procedure for all other machines running managed servers.

Securing Coherence

When you create a WebLogic Server with a cluster, the OSM installer automatically configures the Oracle Coherence connections and well known addresses (WKA) for the cluster, but does not automatically configure security settings for the cluster. For example, the installer does not configure authorized hosts.

Oracle recommends securing the coherence connections. For more information, see Oracle Fusion Middleware Securing Oracle Coherence.

Configuring Coherence for Load Balancing

After upgrading to OSM release 7.4.1, or on a new installation of OSM 7.4.1, when running WebLogic 12.2.1, the Coherence cache is loaded on only one node in a two-node cluster.

Coherence does not load balance the cache when there are only two nodes in the cluster. Because all the cache entries are in one node, all the requests are processed by that node only. This is an expected behavior in Coherence 12.2.1.2.x, 12.2.1.3.x, and 12.2.1.4.

To avoid this behavior, if you use a two-node cluster, add the following command line option to the startup parameters:

-Dcoherence.distribution.2server=false

This option configures Coherence to load balance and distribute cache data on both the nodes in a two-node cluster setup.

Configuring Maximum Message Size

To configure the maximum number of bytes allowed in messages that are received over all supported protocols, add the following command line option to the startup parameters:

-Dweblogic.MaxMessageSize=300000000

The default is 10000000 bytes and this can cause errors (for example, BEA-000403 error message) when this limit is reached. For details about the error, see Diagnosing BEA-000403 Error Message (Doc ID: 1493101.1) on My Oracle Support.

Configuring Node Manager on All Machines in the Domain

This section describes how to configure the machines in your domain that will host Node Manager. OSM recommends using Node Manager to automatically restart managed servers after unexpected failure. Oracle also recommends that you configure whole server migration

Configuring Node Manager for Starting and Stopping Managed Servers

For each machine that will host Node Manager, do the following:

Open the domain_home/nodemanager/nodemanager.properties file.

Set the following values:

StartScriptEnabled=true
StopScriptEnabled=true

Configuring Node Manager for Whole Server Migration

You can optionally use node manager to create floating IP addresses for the managed servers in the cluster. This enables a managed server to migrate from one machine to another in the event of repeated server recover failure.

For each machine that will host Node Manager with whole server migration enabled, do the following:

Open the domain_home/nodemanager/nodemanager.properties file.
Set the following values:
```
CrashRecoveryEnabled=true
ethx=ip_address_range,NetMask=networkmask
UseMACBroadCast=true
```
where:
- x is the interface number that the floating IPs are created on. For example, eth0.
  
  When node manager creates the floating IP address, it adds ethx:n where n is the number it assigns to a floating IP. For example, if a machine were to host two managed servers using the eth0 interface, node manager would create the eth0:1 and eth0:2 floating IP addresses. If a third managed server were to migrate to the machine, it would receive eth0:3. Node manager automatically creates and removes these interfaces as required.
- ip_address_range is the range of IP addresses associated to managed servers on the cluster that node manager can use as floating IP addresses. For example, you can specify a single IP address for a managed server, or a range of IP addresses associated to managed servers, such as 192.168.56.201-192.168.56.204, or * to specify all IP addresses of every managed server in the cluster.
- networkmask is the net mask for the interface for the floating IP. networkmask should be the same as the net mask on the interface. This parameter is optional.

Configured Whole Server Migration Floating IP Controls

For each machine with a node manager configured with whole server migration, do the following:

Set the UNIX PATH environment variable to the directories that contain the WebLogic Server files indicated in Table 5-2.

Table 5-2 Files Required for the PATH Environment Variable

File or Directory	Located in This Directory	Example (bash)
wlsifconfig.sh	domain_home/bin/server_migration	export WLSIFCONFIG_HOME=$DOMAIN_HOME/bin/server_migration export PATH=$PATH:$WLSIFCONFIG_HOME
wlscontrol.sh	domain_home/bin/nodemanager	export WLSCONTROL_HOME=$DOMAIN_HOME/bin/nodemanager export PATH=$PATH:$WLSCONTROL_HOME
node manager	domain_home/nodemanager	export NODEMANAGER_DOMAINS_HOME=$DOMAIN_HOME/nodemanager export PATH=$PATH:$NODEMANAGER_DOMAINS_HOME

File or Directory

Located in This Directory

Example (bash)

wlsifconfig.sh

domain_home/bin/server_migration

export WLSIFCONFIG_HOME=$DOMAIN_HOME/bin/server_migration

export PATH=$PATH:$WLSIFCONFIG_HOME

wlscontrol.sh

domain_home/bin/nodemanager

export WLSCONTROL_HOME=$DOMAIN_HOME/bin/nodemanager

export PATH=$PATH:$WLSCONTROL_HOME

node manager

domain_home/nodemanager

export NODEMANAGER_DOMAINS_HOME=$DOMAIN_HOME/nodemanager

export PATH=$PATH:$NODEMANAGER_DOMAINS_HOME

Grant sudo privileges to the WebLogic Server user account so that the wlsifconfig.sh script can:
- Work without a password prompt.
- Have execute privilege on the /sbin/ifconfig and /sbin/arping binaries that are involved in creating floating IP address.
To grant sudo privileges, do the following:
1. Log in as the root user.
2. Enter the following command:
```
/usr/sbin/visudo
```
  The /etc/sudoers file opens in a text editor.
3. Add the following line:
```
Defaults:weblogic_user !requiretty
weblogic_user ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
```
  where weblogic_user is the user account you created for running WebLogic Server.

Run the following commands to test whether the wlsifconfig.sh script is functioning:

List all IP addresses configured on the interface.

export ServerDir=/tmp
wlsifconfig.sh -listif ethx

where x is the interface number. For example

export ServerDir=/tmp
wlsifconfig.sh -listif eth0
eth0 192.168.56.26

Assign a floating IP address of one of the managed servers you want to node manager to manage.

wlsifconfig.sh -addif ethx ip_address netmask

where ip_address and netmask are the floating IP address and network mask you want to configure. For example:

wlsifconfig.sh -addif eth0 192.168.56.124 255.255.255.0
Generated command - sudo /sbin/ifconfig eth0:1 192.168.56.124 netmask 255.255.255.0
Successfully brought 192.168.56.124 netmask 255.255.255.0 online on eth0:1

Confirm that the floating IP address was added.

/sbin/ifconfig
...
eth0:1  Link encap:Ethernet  HWaddr 08:00:27:5A:C0:09  
        inet addr:192.168.56.124  Bcast:192.168.56.255  Mask:255.255.255.0
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
...

Remove the floating IP address.

wlsifconfig.sh -removeif ethx ip_address

For example:

wlsifconfig.sh -removeif eth0 192.168.56.124
Successfully removed 192.168.56.124 netmask  from eth0:1.

If any command fails, verify that the /etc/sudoers file is properly configured.

Enrolling Each Machine with the Domain

Each machine that will host Node Manager to start and stop the managed servers and proxy server must be enrolled with the domain. This procedure is applicable to all node manager configurations.

To enroll each machine that will host Node Manager:

Start the administration server.
Run WebLogic_home/common/bin/wlst.sh tool.
At the command prompts, run the following commands:
```
connect('username', 'pwd', 't3://ip_address:port')

nmEnroll('domain_home', 'domain_home/nodemanager')

exit()
```
where
- username is the administrator user name for the WebLogic administration server.
- pwd is the password for the WebLogic administration server.
- ip_address is the IP address for the WebLogic administration server.
- port is the port number for the WebLogic administration server.

Starting Node Manager on Each Machine

After Node Manager has been prepared on each machine, start them. For each machine, follow these steps:

Navigate to WebLogic_home/server/bin/.
Start Node Manager by running the following command:
```
nohup ./startNodeManager.sh 2>&1 &
```
Verify that the Node Manager starts properly by running the following command:
```
tail -f nohup.out
```
A log entry should indicate that Node Manager is running with the following line:
```
INFO: Secure socket listener started on port port
```
where port is the port number used by Node Manager.

Configuring a Multicast IP Address for the Cluster Messaging Mode

By default, the WebLogic Configuration Wizard enables unicast messaging mode. For greater reliability and scalability, switch to multicast cluster messaging mode. See "About the WebLogic Messaging Mode and OSM Cluster Size" for more information about which messaging mode option to choose.

To configure a multicast IP address for the cluster messaging mode:

Configure the multicast IP address on your machine. For example, on an Oracle Linux machine, you can do the following:
1. From the desktop, select System, then Preferences, then Network Connections.
  
  The Network Connections screen appears.
2. Select the interface you want to configure a multicast IP address on, and click Edit.
  
  The Editing System screen appears.
3. Click the IPv4 Settings tab.
4. Click Routes…
  
  The Editing IPv4 routes for System screen appears.
5. In the Address field, enter the multicast IP address you want to use for cluster communications.
6. In the Netmask field, enter a network mask.
7. Click OK.
8. Click Apply.
9. Click Close.
10. In the top right hand corner, click the Network icon.
11. Below the interface you have configured an multicast IP address for, click Disconnect.
12. When the interface has disconnected, click Connect.
13. Open a terminal.
14. Go to WebLogic_home/server/bin.
15. Source the setWLSEnv.sh script.
```
source ./setWLSEnv.sh
```
16. Run the following command from the machine:
```
java utils.MulticastTest -N machine_name -A multicast_ip -P multicast_port
```
  where
  - machine_name is any name that identifies the sender of the message. Use a different name for every test you start.
  - multicast_ip is the multicast IP address you want to use for the cluster. Use any IP address between 224.0.0.0 and 239.255.255.255. If there are any running WebLogic clusters using multicast, ensure that you do not use the same IP address during this test.
  - multicast_port is the multicast port number you want to use for the cluster. If there are any other applications running on the machine that use multicast, ensure that you do not use the same port.
  For example:
```
java utils.MulticastTest -N osm_test_from_machine1 -A 239.192.0.0 -P 7521
***** WARNING ***** WARNING ***** WARNING *****
Do NOT use the same multicast address as a running WLS cluster.
 
 
Starting test.  Hit any key to abort
 
 
Using multicast address 239.192.0.0:7521
Will send messages under the name osm_test_from_machine1 every 2 seconds
Will print warning every 600 seconds if no messages are received
 
		I (osm_test_from_machine1) sent message num 1
New Neighbor sm_test_from_machine1 found on message number 1
Received message 2 from sm_test_from_machine1
		I (osm_test_from_machine1) sent message num 2
Received message 3 from sm_test_from_machine1
		I (osm_test_from_machine1) sent message num 3
etc...
```
Start the administration server as described in "Starting the Administration Server."
Log on to the WebLogic Server Administration Console.
Click Lock & Edit.
From the Domain Structure tree, expand Environment, and select Clusters.
From the table of clusters, select the OSM cluster.
From the Configuration tab, select the Messaging sub tab.
From the Messaging Mode drop-down list, select Multicast.
Expand the Advanced button.
In the Multicast address field, enter a multicast address. The multicast address range can be between 224.0.0.1 to 239.255.255.255.
In the Multicast port field, enter a multicast port number. The multicast port can be between 1 to 65535.
In the Idle Periods Until Timeout field, enter 5.
Click Save.
From the Configuration tab, select the Health Monitoring sub tab.
In the Heath Check Interval field, enter 20000.
Click Save.
Click Activate Changes.

Preventing Connection Timeout when Using a Remote Database

To prevent a connection timeout when your database and server are in separate locations, do the following:

Increase the value of the Stuck Thread Max time parameter as follows:
1. Log in to the WebLogic Server Administration Console.
2. In the Domain Structure tree, expand Environment and then click Servers.
  
  The Summary of Servers page is displayed.
3. Click the name of the WebLogic server where you want to deploy the cartridges.
  
  The configuration parameters for the server are displayed on a tabbed page.
4. Click the Tuning tab, and modify the value of the Stuck Thread Max Time parameter to an appropriate value above 1200 seconds. WebLogic considers a thread to be stuck when the thread takes more than a specified amount of time to process a single request. Oracle recommends that you set this value to an optimal level based on performance and stress testing.
5. Click Save.
Increase the value of the Timeout Seconds Java Transaction API parameter (JTA timeout) as follows:
1. In the Domain Structure tree of the WebLogic Server Administration Console, click the name the WebLogic server domain.
  
  The configuration parameters for the domain are displayed on a tabbed page.
2. Click the JTA tab, and modify the value of the Timeout Seconds parameter to an appropriate value. In most cases, a value of 600 seconds is enough.
  
  Note:
  
  If the value is less than 600 when you run the Installer, you are prompted to increase it during installation.
3. Click the Security tab and select Anonymous Admin Lookup Enabled.
4. Click Save.
Note:

The Java Transaction API parameter applies to the domain globally. After you have installed OSM, you can configure an oms-config.xml parameter that controls this value dynamically when OSM deploys or un-deploys a cartridge. For more information, see "Preventing Connection Timeout Issues During Cartridge Deployment".

See the WebLogic Server documentation for more information about these parameters.

Recommended Configuration for WebLogic Servers for Production Systems

Recommendations for creating and configuring managed servers for OSM include the following:

64-bit flags: Ensure to use the -d64 option for each server so that available native libraries or performance packs are used when you start WebLogic servers.
Logging:
- Consider limiting the number of log files, the log file maximum size, and rotation policy. Limit the number of log files to 10 and ensure log files are rotated at start up so that a fresh log file is available.
- Consider turning off logging messages to the domain for each managed server. WebLogic server instances send messages to a number of destinations, including the local server log file and the centralized domain log, which can affect performance. For more information, see the chapter about understanding WebLogic logging services in the document Oracle Fusion Middleware Configuring Log Files and Filtering Log Messages for Oracle WebLogic Server.
JDBC logging: Disable JDBC logging in production systems because it has a substantial impact on performance.
WebLogic networking: Enable native input/output (IO). WebLogic uses software modules called muxers (multiplexers) to read incoming requests on the server and incoming responses on the client. To maximize network socket performance, ensure to use native, platform-optimized muxers.
Transaction timeouts: Set timeouts for correct rollback handling, keeping in mind that global transactions span multiple transaction sources, such as JMS and JDBC.
Longest transaction time: Set the longest time a transaction can be active. The global transaction timeout, which is the longest time that a transaction can be active, is determined by the Java Transaction API (JTA) timeout. All transactions typically complete within only a few seconds (excluding the first transaction after server startup). You can determine the optimal level for your system based on performance and stress testing. Change the JTA timeout value by using the WebLogic Server Administration Console.

Note:

If you are using Oracle Exalogic Elastic Cloud, enable optimizations in order to improve thread count management and request processing, and to reduce lock contention.

Oracle recommends that you do not leave the WebLogic Server listen address undefined on a computer that uses multiple IP addresses. In this case, the server binds to all available IP addresses, which slows down server startup time. Bind a WebLogic server to a fully qualified host name, rather than an IP address. This ensures that SSL server-to-server communication works correctly without requiring host name verification. It also allows administrators to change IP addresses without reconfiguring WebLogic.

Table 5-3 includes a summary of the recommended WebLogic server configurations.

Table 5-3 Configuration Recommendations for the Server

Configuration Item	Value	Notes
JTA Timeout	600	Set this value for better performance.
Enable Exalogic Optimizations	true	When you are using an Exalogic server.
-d64	enabled	Ensure that you use 64-bit native libraries in the startup paths.
Log: Rotate log on startup	true	Not Applicable
Native IO enabled	true	Not Applicable
Log: Number of files limited	true	Not Applicable
Log: File count	10	Not Applicable
All file-based persistent stores use synchronous write policy	Direct-Write-With-Cache	Not Applicable
All JMS Servers > Persistent Stores	enabled	File-based persistence has better performance that a JDBC store, but JDBC offers consistent backup snapshots of OSM data and JMS messages.

Configuring Managed Server Startup Parameters

Oracle recommends that you use the startup parameters specified in this section for each managed server in your cluster. These ensure properly configured managed servers and settings such as the Java heap, garbage collection logging, and so on. These startup parameters only work when using node manager.

To configure the managed server startup parameters:

Log on to the WebLogic Administration Console.
Click Lock & Edit.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
For each managed server, select the server name.

The Configuration tab screen is displayed.
Click the Server Start subtab.

In the Arguments field, enter the following settings:

-server 
-Djava.net.preferIPv4Stack=true
-Dtangosol.coherence.ipmonitor.pingtimeout=9s
-Dtangosol.coherence.log=domain_home/servers/managed_server/logs/managed_server_coherence.log
-Dweblogic.jdbc.remoteEnabled=false
-Xms31g -Xmx31g 
-Xmn14g 
-XX:+UseCompressedOops 
-XX:-UseAdaptiveSizePolicy 
-XX:SurvivorRatio=3 
-XX:TargetSurvivorRatio=70 
-XX:CodeCacheMinimumFreeSpace=16m 
-XX:ReservedCodeCacheSize=256m 
-XX:+UseParallelOldGC 
-XX:ParallelGCThreads=processors
-verbose:gc 
-XX:+PrintGCDetails 
-XX:+PrintGCDateStamps 
-XX:+PrintGCTimeStamps
-XX:+PrintTenuringDistribution 
-XX:+PrintAdaptiveSizePolicy
-Xloggc:domain_home/servers/managed_server/logs/managed_server_gc%t.log
-XX:+DisableExplicitGC 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=domain_home/servers/managed_server/logs/
-XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder
-XX:+ParallelRefProcEnabled
-XX:+AlwaysPreTouch

where:

processors is the number of processors you want to allocate to the managed server. If you have more than one managed server running on the system, ensure that you have enough processors available for each.
managed_server is the name of the managed server.

Click Save.
Click Activate Changes.

Configuring Cluster Settings

Before you install OSM, you must prepare the domain by enabling the WebLogic plug-in, and configuring the hardware load balancer.

To prepare the domain for OSM installation:

Log on to the WebLogic Administration Console.
If you are using HTTPS for communicating with the OSM web clients, do the following:
1. From the Domain Structure tree, expand Environment and select Servers.
2. Select either the administration server or one of the managed servers.
3. From the Configuration tab, select the General subtab.
4. Expand Advanced.
5. Select yes from the WebLogic Plug-In Enabled drop-down list.
6. Click Save.
7. Repeat steps 2.b to 2.f for all managed servers and the administration server.
If you are running a hardware load balancer, do the following:
1. From the Domain Structure tree, expand Environment and select Clusters.
2. Select the cluster to which you are installing OSM.
3. From the Configuration tab, select the HTTP subtab.
4. In the Frontend Host field, enter the host name or IP address of the load balancer.
5. In the Frontend HTTP Port field, enter the HTTP port of the load balancer.
6. In the Frontend HTTPS Port field, enter the HTTPS port of the load balancer.
7. Click Save.

Configuring Server Settings

To configure server settings, do the following:

Log on to the WebLogic Administration Console.
Click Lock & Edit.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Select a server (for example, the administration server, a managed server, or a proxy server).
Click the Logging tab.
In the General sub tab, click Advanced.
In the Domain log broadcaster area, from the Severity Level list, select OFF.
Click Save.
Repeat steps 3 to 8 for all other servers in the cluster.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Select a server.
In the Configuration tab, click the Services sub tab.
In the Default Store area, from the Synchronous Write Policy list, select Direct-Write-With-Cache.
Click Save.
Repeat steps 10 to 14 for all other servers in the cluster.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Select a server.
In the Logging tab, click the HTTP sub tab.
Deselect HTTP access log file enabled.
Click Save.
Repeat steps 16 to 20 for all other servers in the cluster.
Click Activate Changes.

Starting and Verifying all Machines in the Cluster

This section contains information about starting the WebLogic Server cluster and verifying the setup of the cluster.

To start and verify the cluster:

Go to domain_home/bin for your base domain.
Run the following command which starts the administration server:
```
startWebLogic.sh
```
Go to domain_home/bin for your base domain on each machine running a managed server.
Run the following command:
```
startupscript.sh server_name http://IP_address:port
```
where:
- startupscript is the name of the startup script for the managed server you are starting.
- server_name is the name of the managed server or proxy server you are starting.
- IP_address is the IP address of the administration server.
- port is the port number of the administration server.
Log in to the WebLogic Server Administration Console.

The Administration Console is displayed.
Click Servers.

The summary page shows which servers belong to the cluster along with their configured listen address, port, state, and health status.

Configuring Whole Server Migration

The procedures in the section are applicable if you want to use whole server migration and have done the following:

Configured floating IP addresses for the managed server. See "Creating the WebLogic Server Domain" for more information.
Configured node manager properties for whole server migration. See "Configuring Node Manager for Whole Server Migration" for more information.
Configured wlsifconfig.sh and tested floating IP address creation. See "Configured Whole Server Migration Floating IP Controls."

Creating the Leasing Tablespace and Active Table in the Database

To create a leasing tablespace in the Oracle database, do the following:

sqlplus sys/Password123@'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=database-cluster-scan.localdomain)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=pdb1)))' as sysdba

Confirm which ASM disk group you want to use to store the leasing tablespace. For example, the following command shows that there is one ASM disk group available:
```
select group_number, name from v$asm_diskgroup;
 
GROUP_NUMBER NAME
------------ ------------------------------
	           1 DATA
```

Verify the current tablespaces. For example:

select tablespace_name from dba_data_files;
 
TABLESPACE_NAME
------------------------------
SYSTEM
SYSAUX
USERS
DEV_IAS_OPSS
DEV_IAS_IAU
DEV_STB
LARGE_DATA
LARGE_INDEX
 
8 rows selected.

Check the available space on the disk group. For example:

select name, state, total_mb, free_mb from v$asm_diskgroup;
 
NAME			       STATE	       TOTAL_MB	   FREE_MB
---------- ----------- ---------- ----------
DATA			       CONNECTED   	20456	      6908

Create a tablespace for the leasing table. For example:

create tablespace leasing logging datafile '+DATA' size 100M extent management local uniform size 64K;

Verify the leasing table space has been created. For example:

select tablespace_name from dba_data_files;
 
TABLESPACE_NAME
------------------------------
SYSTEM
SYSAUX
USERS
DEV_IAS_OPSS
DEV_IAS_IAU
DEV_STB
LEASING
LARGE_DATA
LARGE_INDEX
 
9 rows selected.

Create a user account with permissions for the leasing tablespace. For example:

create user leasing identified by Password123;
grant create table to leasing;
grant create session to leasing;
alter user leasing default tablespace leasing;
alter user leasing quota unlimited on leasing;

Exit the SQL*Plus session. For example:
```
quit
```
Copy the leasing.ddl file from WebLogic_home/server/db/oracle/920 on one of the WebLogic Server machines. For example, from the current database directory, you can run the following command to copy over the file:
```
scp weblogic_user@ip_address:WebLogic_home/server/db/oracle/920/leasing.ddl .
weblogic_user@ip_address's password: weblogic_password
leasing.ddl   
```
where
- weblogic_user is the WebLogic Server user account of one of the machines running a WebLogic Server.
- ip_address is the IP address of the machine running a WebLogic Server.
- weblogic_password is the password of the WebLogic Server user account.

From the current directory where the leasing.ddl file is located, log in to the newly created leasing tablespace using the SCAN address. For example:

sqlplus leasing/Password123@'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=database-cluster-scan.localdomain)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=pdb1)))' as sysdba

Run the leasing.ddl script. You can safely ignore error messages as long as the active table is successfully created. For example:
```
@leasing.ddl
```

Verify that the active table has been created on the leasing tablespace. For example:

select table_name from user_tables;
 
TABLE_NAME
-------------------------------------------------------------------
ACTIVE

Create the Leasing Multi Data Source

The leasing multi data source is the WebLogic Server connections to the Oracle RAC database instances that contain the leasing table. WebLogic Server uses this data source when migrating a managed server from one machine to another.

To create the leasing multi data source, do the following:

Log on to the WebLogic Administration Console.
From the Domain Structure tree, expand Services and Data Sources.

The Summary of JDBC Data Source screen is displayed.
Click Lock & Edit.
Click New, then Multi Data Source.

The Create a New JDBC Multi Data Source screen is displayed.
In the Name field, enter leasing.
In the JNDI Name field, enter jdbc/leasing as the JNDI name.
From the Algorithm Type list, select Failover.
Click Next.

The Select Targets screen is displayed.
In the Clusters table, select All servers in the cluster.
Click Next.

The Select Data Source Type screen is displayed.
Select Non-XA Driver.
Click Next.

The Add Data Sources is displayed.
Click Create a New Data Source.

The JDBC Data Source Properties screen is displayed.
In the Name field, enter leasing-rac1.
In the JNDI Name field, enter jdbc/leasing-rac1.
In the Database Type list, select Oracle.
Click Next.
In the Database Driver list, select Oracle's Driver (Thin) for RAC Service-Instance connections; Versions:Any.
Click Next.

The Transaction Options screen is displayed.
Deselect Supports Global Transaction. Keep One-Phase Commit selected.
Click Next.

Connection Properties is displayed.
In the Service Name field, enter the service name of the database.
In the Database Name field, enter the database SID of the first Oracle RAC database instance. For example, rac1.
In the Host Name field, enter the database SCAN host name.
In the Port field, enter the port number of the SCAN host name.
In the Database User Name, enter leasing.
In the Password and Confirm Password fields, enter the leasing schema password.
Click Next.

The Test Database Connection screen is displayed.
Click Test Configuration and do one of the following:
- If the test failed, click Back to review the previous screens.
- If the test succeeded, click Next.
  
  The Select Targets screen is displayed.
In the Clusters table, select All servers in the cluster.
Click Finish.

The Add Data Sources screen is displayed.
Select leasing-rac1 from the Available table and move it to the Chosen table.
Repeat steps 13 to 32 with the following changes:
- In the Name field, enter leasing-rac2.
- In the JNDI Name field, enter jdbc/leasing-rac2.
- In the Database Name field, enter the database SID of the second Oracle RAC database instance. For example, rac2.
- Select leasing-rac2 from the Available table and move it to the Chosen table.
Click Finish.
Click Activate Changes.

Configure the Cluster for Whole Server Migration

To configure the cluster for whole server migration, do the following:

Log on to the WebLogic Administration Console.
From the Domain Structure tree, expand Environment and select Clusters.

The Summary of Clusters screen is displayed.
Select the OSM cluster.

The Configuration tab is displayed.
Select the Migration sub tab.
Click Lock & Edit.
In the Candidate Machines For Migration Servers tables, select the machines from the Available table that you want to use in whole server migration scenarios. Move the selected machines to the Chosen table.
From the Migration Basis list, select Database.
From the Data Source For Automatic Migration list, select leasing.
From the Auto Migration Table Name, enter ACTIVE.
Click Save.
Click Activate Changes.

Configure Managed Servers for Whole Server Migration

To configure managed servers for whole server migration, do the following:

Log on to the WebLogic Administration Console.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Click Lock & Edit.
Select a managed server from the list.

The Configuration tab is displayed.
Select the Migration sub tab.
In the Migration Configuration area, select Automatic Server Migration Enabled.

Note:

This option enables node manager to automatically create and remove the floating IP addresses associated to the managed servers that it runs. It also enables node manager to migrate a managed server to another machine.
From the Candidate Machines area in the Available table, select the machine name that you want the managed server to primarily run on and move the machine to the Chosen table.
Select and move the second machine to the chosen table.

Note:

If node manager cannot start the machine on the first machine in the chosen list, it will migrate the managed server to the second machine.
Select the Services sub tab.
In the Default Store area, in the Directory field, enter a path to a shared directory that is also accessible on the machine where the managed server can migrate to.

For example:
```
/mnt/shares/oracle/cluster/defaultdatastore/
```
Click Save.
Click on the Advanced link below the Default Store area.
Deselect Enable File Locking.
Click Save.
Click Activate Changes.
Repeat this procedure for all other managed servers that can be migrated.

Testing Whole Server Migration

To test whole server migration, do the following:

Open a terminal on each machine running node manager.
Go to domain_home/bin for your base domain.
Run the following command which starts nodemanager:
```
startNodeManager.sh
```
Open a second terminal on the machine running the administration server.
Go to domain_home/bin for your base domain.
Run the following command which starts the administration server:
```
startWebLogic.sh
```
Log on to the WebLogic Administration Console.
From the Domain Structure tree, expand Environment and select Machines.

The Summary of Machines screen is displayed.
Select a machine.

The Configuration tab is displayed.
Select the Monitoring tab.
In the Node Manager Status tab, verify that the Status field shows the Reachable state.
Repeat steps 8 to 11 for all other machines.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Click the Control tab.
Select all managed servers that you want to start.
Click Start.
Click Yes.
Open a terminal on one of the machines.
Run the following command:
```
ps -ef | grep managed_server
```
where managed_server is the name of the managed server you want to test for whole server migration. For example:
```
ps -ef | grep osmms1
oracle   12785 12721  9 21:13 ?    00:01:36 /usr/java/jdk1.8.0_121/bin/java ...
```
The first set of numbers in the response is the process ID (PID) of the managed server. Run the following command to terminate the managed server process:
```
kill -9 pid
```
where pid is the process ID of the managed sever process you want to terminate.
Node manager will attempt to restart the managed server on the current machine. Wait for a few minutes, then repeat steps 19 to 20.

After you terminate the managed server PID a second time, go to the second machine and view the output from the terminal that you used to start node manager. Node manager should begin creating new directories for the migrating managed server, then starting up the managed server and allocating the floating IP address to the second machine. For example:

<INFO> <wls1213_domain> <osmms1> <Creating directory "/u01/app/oracle/config/domains/wls1213_domain/servers/osmms1/logs">
<INFO> <wls1213_domain> <osmms1> <Creating directory "/u01/app/oracle/config/domains/wls1213_domain/servers/osmms1/security">
<INFO> <wls1213_domain> <osmms1> <Creating directory "/u01/app/oracle/config/domains/wls1213_domain/servers/osmms1/data/nodemanager">
<INFO> <wls1213_domain> <osmms1> <Creating directory "/u01/app/oracle/config/domains/wls1213_domain/servers/osmms1/tmp">
...
<INFO> <Generated command - sudo /sbin/ifconfig eth0:2 192.168.56.121 netmask 255.255.255.0>
<INFO> <Successfully brought 192.168.56.121 netmask 255.255.255.0 online on eth0:2>
<INFO> <wls1213_domain> <osmms1> <The server 'osmms1' is running now.>

Return to the WebLogic Administration Console.
From the Domain Structure tree, select the domain name, then the Monitoring tab, then the Migration sub tab.
Verify the status of your migrated server. For example, in the Start Time column, you can verify when the migration occurred. From the Machines Migrated From and the Machines Migrated To column, you can verify which machines node manager migrated the machine from and to. For more information about these columns, click the Help button in the WebLogic Server administration console.

Migrating a Managed Server Back

To migrate a managed server back to its original location, do the following:

Log on to the WebLogic Administration Console.
From the Domain Structure tree, expand Environment and select Servers.

The Summary of Servers screen is displayed.
Click the Control tab.
Select the migrated managed server.
Click Shutdown the Force Shutdown Now.
Click Yes.
When the managed server has shutdown, select the managed server again.
Click Start.
Click Yes.
Open a terminal on the first machine where the managed server had originally been located.
Run the following command and confirm that the managed server has begun running on the original machine:
```
ps -ef | grep managed_server
```
where managed_server is the name of the managed server you want to test for whole server migration. For example:
```
ps -ef | grep osmms1
oracle   12785 12721  9 21:13 ?    00:01:36 /usr/java/jdk1.8.0_121/bin/java ...
```
From the Domain Structure tree, select the domain name, then the Monitoring tab, then the Migration sub tab.
Verify the status of your migrated server. For example, in the Start Time column, you can verify when the migration occurred. From the Machines Migrated From and the Machines Migrated To column, you can verify which machines node manager migrated the machine from and to. For more information about these columns, click the Help button in the WebLogic Server administration console.

Installing OSM in a Clustered Environment

To install OSM in a clustered environment:

Start the administration server, at least one managed server in the cluster, and the proxy server (if used).

Note:

The OSM installer requires that at least one managed server is running in the cluster during the installation process. The OSM Administration server configures any remaining managed servers when they are started.
Do one of the following:
- Perform an active OSM installation using the procedure described in "Installing OSM" with the following modifications:
  - On the WebLogic Server Connection Information window, enter the IP address and port number of the administration server.
  - On the BEA WebLogic Server/Cluster Selection window, select the cluster you created from the WebLogic Server/Cluster list, and click Next.
- Perform a silent OSM installation as described in "Performing a Silent Installation of OSM."
  
  If you are using an install_cfg.xml file generated by a non-clustered installation, you must set the Handler-Factory parameter to the following value:
```
com.mslv.oms.handler.cluster.ClusteredHandlerFactory
```
After OSM is installed, restart all the servers included in the cluster.