Using Service Migration in an Enterprise Deployment

19 Using Service Migration in an Enterprise Deployment

The Oracle WebLogic Server migration framework supports Whole Server Migration and Service Migration. Whole Server Migration requires more resources and a full start of a managed server, so it involves a higher failover latency than Service Migration. The products included in this EDG support Service Migration. Hence, Service Migration is recommended and this guide explains how to use Service Migration in an Oracle Fusion Middleware enterprise topology. Whole Server migration is out of the scope of this guide.

About Automatic Service Migration in an Enterprise Deployment

Oracle WebLogic Server provides a migration framework that is an integral part of any highly available environment. The following sections provide more information about how this framework can be used effectively in an enterprise deployment.

Understanding the Difference between Whole Server and Service Migration

The Oracle WebLogic Server migration framework supports two distinct types of automatic migration:

Whole Server Migration, where the Managed Server instance is migrated to a different physical system upon failure.

Whole server migration provides for the automatic restart of a server instance, with all its services, on a different physical machine. When a failure occurs in a server that is part of a cluster which is configured with server migration, the server is restarted on any of the other machines that host members of the cluster.

For this to happen, the servers must use a floating IP as listen address and the required resources (transactions logs and JMS persistent stores) must be available on the candidate machines.

See Whole Server Migration in Administering Clusters for Oracle WebLogic Server.
Service Migration, where specific services are moved to a different Managed Server within the cluster.

To understand service migration, it's important to understand pinned services.

In a WebLogic Server cluster, most subsystem services are hosted homogeneously on all server instances in the cluster, enabling transparent failover from one server to another. In contrast, pinned services, such as JMS-related services, the JTA Transaction Recovery Service, and user-defined singleton services, are hosted on individual server instances within a cluster—for these services, the WebLogic Server migration framework supports failure recovery with service migration, as opposed to failover.

See Understanding the Service Migration Framework in Administering Clusters for Oracle WebLogic Server.

Implications of Using Whole Server Migration or Service Migration in an Enterprise Deployment

When a server or service is started in another system, the required resources (such as services data and logs) must be available to both the original system and to the failover system; otherwise, the service cannot resume the same operations successfully on the failover system.

For this reason, both whole server and service migration require that all members of the cluster have access to the same transaction and JMS persistent stores (whether the persistent store is file-based or database-based).

This is another reason why shared storage is important in an enterprise deployment. When you properly configure shared storage, you ensure that in the event of a manual failover (Administration Server failover) or an automatic failover (whole server migration or service migration), both the original machine and the failover machine can access the same file store with no change in service.

In the case of an automatic service migration, when a pinned service needs to be resumed, the JMS and JTA logs that it was using before failover need to be accessible.

In addition to shared storage, Whole Server Migration requires the procurement and assignment of a virtual IP address (VIP). When a Managed Server fails over to another machine, the VIP is automatically reassigned to the new machine.

Note that service migration does not require a VIP.

Understanding Which Products and Components Require Whole Server Migration and Service Migration

Note that the table lists the recommended best practice. It does not preclude you from using Whole Server or Automatic Server Migration for those components that support it.

Component	Whole Server Migration (WSM)	Automatic Service Migration (ASM)
Oracle WebCenter Content	YES	YES (Recommended)
Oracle SOA Suite	YES	YES (Recommended)
Oracle Enterprise Capture	YES	YES (Recommended)

Creating a GridLink Data Source for Leasing

Automatic Service Migration requires a data source for the leasing table, which is a table that resides in a tablespace, and schema created automatically, as part of the Oracle WebLogic Server schemas by the Repository Creation Utility (RCU).

Note:

To accomplish data source consolidation and connection usage reduction, you can reuse the WLSSchemaDatasource as is for database leasing. This data source is already configured with the FMW1412_WLS_RUNTIME schema, where the leasing table is stored.

For an enterprise deployment, you should create a GridLink data source:

Log in to the WebLogic Remote Console.
Navigate to the Edit Tree.
In the structure tree, expand Services and select Data Sources.

On the Summary of Data Sources page, click New and select GridLink Data Source. Enter the following:

Table 19-1 GridLink Data Source Properties

Properties	Description
Name	Enter a logical name for the data source in the Name field. For example, Leasing.
JNDI Names	Enter a name for JNDI. For example, jdbc/leasing.
Targets	Select the cluster that you are configuring for Automatic Service Migration and move to "Chosen".
Data Source Type	Select GridLink Data Source.
Database Driver	Select Oracle's Driver (Thin) for GridLink Connections Versions: Any.
Global Transaction Protocol	Select None.
Listeners	Enter the SCAN address and port for the RAC database, separated by a colon. For example, `db-scan.example.com:1521`.
Service Name	Enter the service name of the database with lowercase characters. For a GridLink data source, you must enter the Oracle RAC service name. For example, `soaedg.example.com`.
Database username	Enter the user name of the WLS Runtime schema. For example, `FMW1412_WLS_RUNTIME`. In this example, `FMW1412` is the prefix you used when you created the schemas as you prepared to configure the domain. Note: The leasing table is created automatically when you create the WLS schemas with the Repository Creation Utility (RCU).
Password	Enter the password you used when you created the WLS schema in RCU.
Protocol	Leave the default value (TCP).
Fan Enabled	You must check this option.
ONS Nodes	Leave it empty. ONS node list is automatically retrieved when the database is 12.2 or higher version.
ONS Wallet and password	You can leave this field empty.
Test Configuration	You must enable this option.

Click Create.
Commit changes in the shopping cart.

Configuring Automatic Service Migration in an Enterprise Deployment

The services used by the different SOA components in this Enterprise Deployment Guide are already configured with Automatic Service Migration when following the Configuration Wizard steps provided in this guide. For any other custom services, you can use the following steps to configure service migration.

Setting the Leasing Mechanism and Data Source for an Enterprise Deployment Cluster

Before you can configure automatic service migration, you must verify the leasing mechanism and data source that is used by the automatic service migration feature.

Note:

To accomplish data source consolidation and connection usage reduction, you can reuse the WLSRuntimeSchemaDataSource datasource as is for database leasing. This datasource is already configured with the FMW1412_WLS_RUNTIME schema, where the leasing table is stored.

The following procedure assumes that you have configured the Leasing data source either by reusing the WLSRuntimeSchemaDataSource or a custom datasource that you created as described in Creating a GridLink Data Source for Leasing.

Log into the WebLogic Remote Console.
Navigate to the Edit Tree.
In the structure tree, expand Environment > Clusters.
The Summary of Clusters page appears. Click the cluster for which you want to configure migration.
Click the Migration tab.
Verify that Database is selected in the Migration Basis drop-down menu.
In the Data Source for Automatic Migration drop-down menu, select the Leasing data source that you created in Creating a GridLink Data Source for Leasing. Select the WLSRuntimeSchemaDataSource for data source consolidation.
Click Save.
Commit changes in the shopping cart
Restart the managed servers for the changes to be effective. If you are configuring other aspects of ASM in the same configuration change session, you can use a final unique restart to reduce downtime.

Changing the JTA Migration Settings for the Managed Servers in the Cluster

After you set the leasing mechanism and data source for the cluster, you can enable automatic JTA migration for the Managed Servers that you want to configure for service migration. Note that this topic applies only if you are deploying JTA services as part of your enterprise deployment.

For example, this task is not required for Oracle WebCenter Content enterprise deployments.

To change the migration settings for the Managed Servers in each cluster:

Log into the WebLogic Remote Console.
Navigate to the Edit Tree.
In the structure tree, expand the Environment node and click Servers.

The Summary of Servers page appears.
Expand the name of the server you want to modify.
Navigate to JTA Migratable Target.
In the the JTA Migration Policy drop-down menu, select Failure Recovery.

In the JTA Candidate Servers section of the page, leave the Chosen list box empty. If you do not select any servers from the Available list box, all the available servers in the cluster become candidates for service migration.
Click Save.
Commit the changes in the shopping cart.
Restart the Managed Servers and the Administration Server for the changes to be effective.

If you are configuring other aspects of ASM in the same configuration change session, you can use a final unique restart to reduce downtime.

About Selecting a Service Migration Policy

When you configure Automatic Service Migration, you select a Service Migration Policy for each cluster. This topic provides guidelines and considerations when selecting the Service Migration Policy.

For example, products or components running singletons or using Path services can benefit from the exactly-once policy. With this policy, if at least one Managed Server in the candidate server list is running, the services hosted by this migratable target are active somewhere in the cluster if servers fail or are administratively shut down (either gracefully or forcibly). This can cause multiple homogenous services to end up in one server on startup.

When you use this policy, you should monitor the cluster startup to identify what servers are running on each server. You can then perform a manual failback, if necessary, to place the system in a balanced configuration.

Other Fusion Middleware components are better suited for the failure-recovery policy.

See Policies for Manual and Automatic Service Migration in Administering Clusters for Oracle WebLogic Server.

Setting the Service Migration Policy for Each Managed Server in the Cluster

After you modify the JTA migration settings for each server in the cluster, you can then identify the services and set the migration policy for each Managed Server in the cluster, using the WebLogic Remote Console:

Log into the WebLogic Remote Console.
Navigate to the Edit Tree.
In the structure tree, expand Environment > Migratable Targets.
Click the name of the first Managed Server in the cluster.
In the Service Migration Policy drop-down menu, select the appropriate policy for the cluster. See About Selecting a Service Migration Policy.
In the Candidate tab, leave the Chosen list box empty. If you do not select any servers from the Available list box, all the available servers in the cluster become candidates for service migration.
Click Save.
Repeat Step 2 to Step 6 for each of the additional Managed Servers in the cluster.
Commit changes in the shopping cart.
Restart the managed servers for the changes to be effective.

If you are configuring other aspects of ASM in the same configuration change session, you can use a final unique restart to reduce downtime.

Validating Automatic Service Migration

After you configure automatic service migration for your cluster and Managed Servers, validate the configuration, as follows:

If you have not already done so, log into the WebLogic Remote Console.
Navigate to the Monitoring Tree.
In the structure tree, expand Environment > Migration.
Click Service Migration Data Runtimes.

The console displays a list of migratable targets and their current hosting server.
In the Migratable Targets table, select a row for one of the migratable targets.
Note the value in the Migrated To property.
Use the operating system command line to stop the first Managed Server.
Use the following command to end the Managed Server Process and simulate a crash scenario:
```
kill -9 pid
```
In this example, replace pid with the process ID (PID) of the Managed Server. You can identify the PID by running the following UNIX command:
```
ps -ef | grep managed_server_name
```
Note:

After you kill the process, the Managed Server might be configured to start automatically. In this case, you must kill the second process using the kill –9 command again.

Watch the terminal window (or console) where the Node Manager is running.

You should see a message indicating that the selected Managed Server has failed. The message is similar to the following:

<INFO> <domain_name> <server_name> 
<The server 'server_name' with process id 4668 is no longer alive; waiting for the process to die.>
<INFO> <domain_name> <server_name> 
<Server failed during startup. It may be retried according to the auto restart configuration.>
<INFO> <domain_name> <server_name>
<Server failed but will not be restarted because the maximum number of restart attempts has been exceeded.>

Return to the WebLogic Remote Console and refresh the table of Service Migration Data Runtimes; verify that the migratable targets are transferred to the remaining, running Managed Server in the cluster:
- Verify that the Migrated to value for the process you killed is now updated to show that it has been migrated to a different host.
- Verify that the value in the Status of Last Migration column for the process is Succeeded.
Open and review the log files for the Managed Servers that are now hosting the services; look for any JTA or JMS errors.
Note:

For JMS tests, it is a good practice to get message counts from destinations and make sure that there are no stuck messages in any of the migratable targets:

For example, for uniform distributed destinations (UDDs):
1. Log into the WebLogic Remote Console and navigate to the Monitoring Tree.
2. Navigate to Dashboards and click JMS Destinations.
3. Order by Destination Name and look for the destination.
  
  Tip:
  You can copy this dashboard and create a custom one to filter by specific destination name.
4. Review the Messages Current Count and Messages Pending Count values.

Failing Back Services After Automatic Service Migration

When Automatic Service Migration occurs, Oracle WebLogic Server does not support failing back services to their original server when a server is back online and rejoins the cluster.

As a result, after the Automatic Service Migration migrates specific JMS services to a backup server during a fail-over, it does not migrate the services back to the original server after the original server is back online. Instead, you must migrate the services back to the original server manually.

To fail back a service to its original server, use WLST migrate command. For more information, see WLST Command Reference for Oracle WebLogic Server.